Federal Contact Centers
Mechanism for Sharing Metrics and Oversight Practices along with Improved Data Needed Gao ID: GAO-06-270 February 8, 2006Federal agencies have increasingly relied on contact centers--centers handling inquiries via multiple channels such as telephone, Web page, e-mail, and postal mail--as a key means of communicating with the public. Many of these centers are contractor-operated. Concerns exist about the accuracy of responses provided through contractor-operated centers. This report examines (1) the extent to which the contract terms and oversight practices for contact centers at selected agencies emphasize the importance of providing accurate information to the public, and (2) whether guidance for the operation of contact centers and basic information needed to provide general oversight exist. GAO reviewed one contractor-operated contact center at each of six agencies: the Centers for Disease Control and Prevention (CDC), General Services Administration (GSA), U.S. Postal Service (USPS), and the Departments of Defense, Labor, and Education (DOD, DOL, and Education).
The contracts and oversight practices for the contact centers of the six agencies reviewed, which handle millions of inquiries annually, varied significantly regarding the emphasis they placed on providing accurate information to the public. Although federal policy for disseminating information to the public specifically emphasizes accuracy, only four of the six agencies include accuracy as a performance metric in their contracts. With respect to oversight, only two of the six agencies used all four of the accuracy-related oversight practices we identified--regular knowledge database reviews, regular contact monitoring, postcontact customer satisfaction surveys, and validation of contractor reports. Although each agency used some form of oversight to assess the accuracy of the information provided by its contact center, each agency differed regarding how it implemented these practices. There is no governmentwide guidance or standards for operating contact centers--including guidance on specifying accuracy as a contract performance metric or as a key focus for oversight. Some agencies indicated that had federal guidance been available, it would have helped them establish performance indicators and develop oversight policies and practices. Recognizing the need for operational standards for contact centers, an interagency working group recently proposed draft guidelines to OMB and other federal agencies, but OMB has no plans to issue these guidelines or any standards for use by agencies. Additionally, until recently the federal government had not collected data on the universe of federal contact centers. OMB and GSA attempted to collect data on the number, types, and costs of federal contact centers in 2004, but the data collected were incomplete. In addition, no governmentwide procurement information was reported to the Federal Procurement Data System (FPDS) in fiscal years 2000 through 2004 using the reporting code for telephone call centers, which OMB said is the appropriate code for contact centers. The five agencies we reviewed that report data to FPDS used a variety of different codes, some because they believe that the telephone call center code is too narrow to cover the services of their multichannel contact centers.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-270, Federal Contact Centers: Mechanism for Sharing Metrics and Oversight Practices along with Improved Data Needed
This is the accessible text file for GAO report number GAO-06-270
entitled 'Federal Contact Centers: Mechanism for Sharing Metrics and
Oversight Practices along with Improved Data Needed' which was released
on March 10, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
February 2006:
Federal Contact Centers:
Mechanism for Sharing Metrics and Oversight Practices along with
Improved Data Needed:
GAO-06-270:
GAO Highlights:
Highlights of GAO-06-270, a report to congressional requesters:
Why GAO Did This Study:
Federal agencies have increasingly relied on contact centers”centers
handling inquiries via multiple channels such as telephone, Web page, e-
mail, and postal mail”as a key means of communicating with the public.
Many of these centers are contractor-operated. Concerns exist about the
accuracy of responses provided through contractor-operated centers.
This report examines (1) the extent to which the contract terms and
oversight practices for contact centers at selected agencies emphasize
the importance of providing accurate information to the public, and (2)
whether guidance for the operation of contact centers and basic
information needed to provide general oversight exist. GAO reviewed one
contractor-operated contact center at each of six agencies: the Centers
for Disease Control and Prevention (CDC), General Services
Administration (GSA), U.S. Postal Service (USPS), and the Departments
of Defense, Labor, and Education (DOD, DOL, and Education).
What GAO Found:
The contracts and oversight practices for the contact centers of the
six agencies reviewed, which handle millions of inquiries annually,
varied significantly regarding the emphasis they placed on providing
accurate information to the public. Although federal policy for
disseminating information to the public specifically emphasizes
accuracy, only four of the six agencies include accuracy as a
performance metric in their contracts. With respect to oversight, only
two of the six agencies used all four of the accuracy-related oversight
practices we identified”regular knowledge database reviews, regular
contact monitoring, postcontact customer satisfaction surveys, and
validation of contractor reports. Although each agency used some form
of oversight to assess the accuracy of the information provided by its
contact center, each agency differed regarding how it implemented these
practices.
There is no governmentwide guidance or standards for operating contact
centers”including guidance on specifying accuracy as a contract
performance metric or as a key focus for oversight. Some agencies
indicated that had federal guidance been available, it would have
helped them establish performance indicators and develop oversight
policies and practices. Recognizing the need for operational standards
for contact centers, an interagency working group recently proposed
draft guidelines to OMB and other federal agencies, but OMB has no
plans to issue these guidelines or any standards for use by agencies.
Additionally, until recently the federal government had not collected
data on the universe of federal contact centers. OMB and GSA attempted
to collect data on the number, types, and costs of federal contact
centers in 2004, but the data collected were incomplete. In addition,
no governmentwide procurement information was reported to the Federal
Procurement Data System (FPDS) in fiscal years 2000 through 2004 using
the reporting code for telephone call centers, which OMB said is the
appropriate code for contact centers. The five agencies we reviewed
that report data to FPDS used a variety of different codes, some
because they believe that the telephone call center code is too narrow
to cover the services of their multichannel contact centers.
Agency Oversight Practices Used for Ensuring Accuracy of Information:
[See Table 4]
What GAO Recommends:
GAO makes recommendations to the Office of Management and Budget (OMB)
and GSA to improve the sharing of metrics and oversight practices for
contact centers, and to improve data reporting on contracts for these
services. OMB and GSA concurred with our recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-06-270.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact William T. Woods at (202)
512-8214 or woodsw@gao.gov.
Contents:
Letter:
Results in Brief:
Background:
Emphasis on Providing Accurate Information to the Public Varies:
Governmentwide Guidance and Information on Contact Centers Do Not
Exist:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Contact Center Profiles for Six Agencies Reviewed:
Appendix III: Comments from the Department of Defense:
Appendix IV: Comments from the General Services Administration:
Appendix V: Comments from the Department of Health & Human Services:
Appendix VI: GAO Contact and Staff Acknowledgments:
Appendix VII: Related GAO Products:
Tables:
Table 1: Agency Contact Centers Reviewed:
Table 2: Summary of Key Performance Metrics in Agency Contracts:
Table 3: Description of Oversight Practices for Ensuring Accuracy of
Information:
Table 4: Agency Oversight Practices Used for Ensuring Accuracy of
Information:
Table 5: NAICS Codes Used for Contact Center Contracts Reviewed:
Table 6: Contact Center Profiles for Agencies Reviewed:
Abbreviations:
CDC: Centers for Disease Control and Prevention:
DOD: Department of Defense:
DOL: Department of Labor:
FPDS: Federal Procurement Data System:
GSA: General Services Administration:
NAICS: North American Industry Classification System:
OFPP: Office of Federal Procurement Policy:
OMB: Office of Management and Budget:
TMA: TriCare Management Activity:
USPS: U.S. Postal Service:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
February 8, 2006:
The Honorable Henry A. Waxman:
Ranking Member:
Committee on Government Reform:
House of Representatives:
The Honorable Edolphus Towns:
Ranking Member:
Subcommittee on Government Management, Finance, and Accountability:
Committee on Government Reform:
House of Representatives:
Each year millions of citizens contact the government for information
critical to their health, finances, or general well-being. In recent
years, federal agencies have increasingly relied on contact centers--
centers handling inquiries via multiple channels such as telephone, Web
page, e-mail, and postal mail--as an important means of communicating
with the general public. This was demonstrated most recently during the
response to Hurricane Katrina. One of the principal ways the federal
government delivers results, according to the Office of Management and
Budget (OMB), is by providing timely and accurate information to
citizens.[Footnote 1] Improving access to high-quality government
information and services is an important focus for OMB's current
electronic government initiative, which is an element of the
President's Management Agenda. Information provided to the public can
be evaluated by various measures, including accuracy, timeliness,
completeness, courtesy, and overall customer service satisfaction. OMB
guidance specifies accuracy as one of the basic elements of quality for
information provided by agencies to the public.[Footnote 2] Incorrect
information provided in a fast and courteous manner may be satisfying
initially, but in the long run, it may be of little use or could be
detrimental to the individual.
Many agencies have contracted with private sector firms to operate
contact centers. Recent GAO reports identified high error rates in the
accuracy of policy-oriented responses given to the public through one
contractor-operated contact center.[Footnote 3] In this context, you
asked us to review how agencies are overseeing their contractor-
operated contact centers to ensure that they provide accurate
information to the public. Specifically, we (1) identified the extent
to which the contract terms and oversight practices for selected
agencies' contractor-operated contact centers emphasize the importance
of providing accurate information to the public, and (2) determined
whether guidance for the operation of contact centers and basic
information needed to provide general oversight exist.
To identify contract terms and oversight practices for contact centers,
we selected one contact center at each of six agencies. We selected
centers that handle over 1 million inquiries annually and provide
information to citizens that could significantly affect their finances,
health, or safety.[Footnote 4] Table 1 lists the contact centers we
selected for our review and the main public inquiry issues they handle.
Table 1: Agency Contact Centers Reviewed:
Agency: Department of Defense (DOD) TriCare Management Activity (TMA)
North region[A];
Contact center: Healthnet's contact center;
Public inquiry issues handled: Medical benefit coverage, enrollments,
and claims processing.
Agency: Department of Education (Education);
Contact center: Federal Student Aid Information Center;
Public inquiry issues handled: Aid application status and loan issues.
Agency: Department of Health and Human Services' Centers for Disease
Control and Prevention (CDC);
Contact center: CDC INFO contact center;
Public inquiry issues handled: Disease prevention, detection, and
outbreak control information.
Agency: Department of Labor (DOL);
Contact center: National Contact Center;
Public inquiry issues handled: Job issues, workplace safety, pension
and health benefits.
Agency: General Services Administration (GSA);
Contact center: National Contact Center;
Public inquiry issues handled: General information and referrals for
any government program; orders for government publications.
Agency: U.S. Postal Service (USPS);
Contact center: National Contact Center;
Public inquiry issues handled: Mail delivery and shipping issues.
Source: GAO.
[A] DOD TMA is divided into three separate contract service regions--
North, South, and West.
[End of table]
For each of these centers, we interviewed agency officials and reviewed
the contract terms and performance measures, as well as the policies,
procedures, reports, and tools used to oversee and evaluate the
contractors' operation of the contact centers. In addition, we visited
four contractor-operated centers to observe their operations and
quality control procedures. We did not independently assess the
operations of the contact centers or evaluate the effectiveness of the
contractors' quality control processes. We identified industry
practices for ensuring the accuracy of information provided by contact
centers and interviewed representatives from two major contact center
industry groups.
To determine the guidance provided to agencies and the information
gathered by the federal government about contact centers, we
interviewed OMB and GSA officials and reviewed OMB guidance, the
results of OMB's request for information from federal agencies about
their centers, GSA's baseline survey of a sample of agencies' centers,
and data from the Federal Procurement Data System (FPDS). We also
reviewed the report of the GSA-sponsored interagency working group--the
Citizen Service Levels Interagency Committee--that developed potential
standards for federal contact centers. Appendix I includes a more
detailed discussion of our scope and methodology, and appendix II
contains additional details on the six contact centers we reviewed. Our
work was conducted from February through November 2005 in accordance
with generally accepted government auditing standards.
Results in Brief:
The contracts and oversight practices six agencies used for their
contractor-operated contact centers varied significantly regarding the
emphasis they placed on providing accurate information to the public.
Although federal policy for disseminating information to the public
specifically emphasizes accuracy, only four of the six agencies'
contracts explicitly include accuracy as a performance metric. Each
agency we reviewed used some form of oversight to assess the accuracy
of the information provided by its contact center, but only two of the
six agencies used all four of the accuracy-related oversight practices
we identified, such as contact monitoring and independent review of
contractor performance reports. Each agency differed regarding how it
implemented the various practices. For example, some agencies monitor
calls only on an ad hoc basis, whereas others use a more structured
system of monitoring to ensure accuracy. This variance was due to a
number of factors, such as differences among the agencies in staffing
levels, funding, and the use of guidance specific to the agency.
There is no governmentwide guidance or standards for operating contact
centers--including guidance on including accuracy as a contract
performance metric or as a focus for oversight. Some agencies indicated
that had federal guidance been available, it would have helped them
establish performance indicators for service quality, identify the most
effective contract types and provisions, and develop oversight policies
and practices. Recognizing the need for operational standards, an
interagency working group recently proposed draft guidelines to federal
agencies and OMB. Officials at OMB have reviewed the proposed
guidelines but do not plan to issue any governmentwide guidance at this
time; they believe agencies can refer to the proposed guidelines for
guidance. Additionally, until recently the federal government has not
collected data on the universe of federal contact centers that could be
used in oversight of their operations. In 2004, OMB and GSA attempted
to collect data on the number, types, and costs of federal contact
centers. The data collected are incomplete, however, because OMB did
not follow up with nonresponding agencies and the survey GSA used to
collect the data had methodological flaws. In addition, each of the
agencies we reviewed that reported information to FPDS chose a
different reporting code for its contact center. None of the agencies
used the code that OMB says covers contact center services because at
least some of the agencies believe the code is limited to telephone
services. OMB is considering issuing a clarification to the description
of the code to explain that the code includes not just telephones, but
also Web sites, e-mails, facsimiles, and so forth, in its next update
to the North American Industry Classification System (NAICS) manual.
This report contains recommendations to OMB and GSA that are intended
to promote the sharing of metrics and oversight practices for contact
center operations and to ensure that governmentwide data gathered on
contact centers provide reliable information. OMB and three of the six
agencies whose centers we reviewed provided comments on a draft of this
report. OMB, CDC, and GSA concurred with our recommendations. DOD did
not concur with our draft report because it believed that we did not
describe all the efforts taken to ensure the accuracy of its contact
center information. We recognize that DOD takes steps, through its
contractor and otherwise, to ensure the accuracy of its contact center
information. Nevertheless, we found no specific performance metric in
the contract itself regarding accuracy, and we believe our report
correctly depicts DOD's use of the oversight practices we identified.
Background:
In 2001, the President announced his management agenda for making the
government more focused on citizens and results, which included
expanding Electronic Government (E-Government). The President's E-
Government Strategy identified several governmentwide initiatives with
a goal of eliminating redundant systems and significantly improving the
government's quality of customer service for citizens and businesses.
The expected results of the E-Government initiative include providing
high-quality customer service regardless of whether a citizen contacts
an agency by phone, in person, or on the worldwide Web. The E-
Government Act of 2002 codified the President's E-Government
initiatives and expanded OMB's leadership role by establishing the
Office of E-Government and Information Technology within OMB.[Footnote
5] The act also requires that agencies comply with OMB E-Guidance. One
of the 24 presidential E-Government initiatives is developing and
deploying governmentwide citizen customer service using industry best
practices that will provide citizens with timely, consistent responses
about government information and services via e-mail, telephone,
Internet, and publications.
By congressional direction, OMB also is responsible for establishing
and issuing governmentwide guidelines to federal agencies for ensuring
the quality of the information disseminated to the public.[Footnote 6]
In response to this direction, OMB issued guidance to agencies in
February 2002 that defined the quality of information to include
accuracy as one of its fundamental elements and directed agencies to
develop procedures for reviewing and substantiating the quality of
their information before dissemination. [Footnote 7]
Contact centers are one method agencies use to disseminate information
to the public. In the past, public inquiries to the government were
often made by telephone and thus federal agencies began establishing
call centers. With evolving technology, citizen inquiries to the
government now come through various channels such as e-mails, Web-based
forms, facsimiles, Web chat rooms, and traditional postal mail. As a
result, agencies have established multichannel contact centers to
handle these inquiries.[Footnote 8] Contact centers rely on automated
and live telephone response systems, Web site technologies, and trained
customer service representatives to provide information to the public.
For contractor-operated contact centers, the agency typically provides
either scripted responses or the content from which the contractor
creates its own scripted responses. The scripts are used for the
prerecorded telephone response systems, Web pages, and preformatted
responses given by the customer service representatives. Contact
centers are staffed in tiers by generalist or specialist
representatives or a combination of both. Usually, Tier 1 staff handle
general information inquiries and direct more complex or personal
issues to specialized Tier 2 or Tier 3 staff or to the agency's subject
matter experts.
One method for obtaining information on the contact centers that are
operated by contractors on behalf of the government is to review data
from FPDS. FPDS is used to report individual procurement transactions,
which include the industrial classification of the goods and services
procured by the federal government. FPDS was implemented by OMB's
Office of Federal Procurement Policy (OFPP) in 1978 in response to the
Office of Federal Procurement Policy Act of 1974 requirement of
establishing a system for collecting and developing information about
federal procurement contracts.[Footnote 9] Since 1982, GSA has
administered FPDS on OFPP's behalf. In 2003, the system was revised and
is now called FPDS-Next Generation.[Footnote 10] A wide range of users,
including those with the executive and legislative branches, rely on
FPDS data for information on agency contracting actions, governmentwide
procurement trends, and achievement of goals related to small business.
Emphasis on Providing Accurate Information to the Public Varies:
The six agencies we reviewed emphasized accuracy of contact center
information to varying degrees through the quality assurance mechanisms
of their contracts and various oversight practices. Four of the six
included a specific metric to measure contractor performance related to
providing accurate information to the public, but only one of the six
used all four of the oversight practices we identified--such as
actively monitoring contacts--to ensure that accurate information is
provided to the public.
Most Agencies' Contract Metrics Include Accuracy, but Some Did Not:
Each of the six agencies we reviewed specified key performance metrics
that its contractor is required to meet. These performance metrics
define the minimum level of quality acceptable to the agency and
provide the basis against which the contractor is to be evaluated. We
found that four of the six agencies' contracts included accuracy of
information in one or more of the key performance metrics. The
remaining two agencies did not have specific metrics that addressed the
need to provide accurate information to the public. Table 2 summarizes
the key performance metrics specified in the contracts we reviewed and
indicates through shading those that specifically address providing
accurate information.
Table 2: Summary of Key Performance Metrics in Agency Contracts:
Agency: CDC;
Key performance metrics:
* Quality of service--accuracy and completeness of response and
customer service skills;
* Customer satisfaction--point value based on monthly average of survey
respondents providing favorable rating;
* Cost management--efficiency of operational costs;
* Business performance--ability to collaborate with other related
contractors.
Agency: DOD-TMA;
Key performance metrics:
* Busy signals--required percentage of calls received without busy
signal;
* Hold time--limit to the number of seconds of hold time during the
call.
Agency: DOL;
Key performance metrics:
* Quality of service--accuracy of response and customer service skills;
* Staff occupancy--Minimum percentage of time that customer service
representatives are actively handling inquiries;
* First contact resolution--required percentage of contacts resolved
during first contact;
* Service level--percentage of calls answered within specified number
of seconds;
* Average speed of answer--required percentage of calls answered within
specific number of seconds;
* Abandonment rate--minimum percentage of calls that disconnect before
being handled by the automated system or a customer service
representative;
* All trunks busy--minimum percentage of time all telephone trunk lines
are busy;
* Percentage of calls completed in Interactive Voice Response system--
minimum percentage of calls handled by the automated response system;
* Customer satisfaction--percentage of survey respondents providing
favorable rating;
* Hold time--limit to the number of seconds of hold time while waiting
in call queue.
Agency: Education;
Key performance metrics:
* Accuracy of agent answer on calls/e-mails/online live help/facsimile--
required percentage without errors;
* Customer satisfaction--percentage of survey respondents providing
favorable rating;
* Average speed of answer-- required percentage of calls answered
within specific number of seconds while maintaining minimal abandonment
rate.
Agency: GSA;
Key performance metrics:
* Accuracy of agent answer on calls/e-mails/postal/facsimile--required
percentage without errors;
* Knowledge base audits--number of knowledge pieces reviewed that meet
accuracy checks;
* Customer satisfaction--percentage of survey respondents providing
favorable rating on automated surveys at end of call, outbound call
surveys, and e-mail surveys; required percentage increase in the number
of survey respondents providing favorable ratings each quarter;
* Average speed of answer--required percentage of calls answered within
specific number of seconds and e-mails/postal/faxes/orders responded to
within specific number of business days;
* Service availability--required percentage of time that services are
available to the public.
Agency: USPS;
Key performance metrics:
* First contact resolution-- required percentage of inquiries resolved
during first contact (call/e- mail/postal/fax);
* Service level--required percentage of calls answered within specific
number of seconds;
* Customer satisfaction-- percentage of survey respondents providing
favorable rating on automated surveys at end of call, outbound call
surveys, and e-mail surveys.
Source: GAO analysis of agencies' contracts.
Note: Shaded metrics are those that specifically address accuracy of
information.
[End of table]
Agencies' Contractor Oversight Practices Vary:
The Federal Acquisition Regulation requires agencies to perform and
document oversight of their contractors' performance to ensure the
government receives high-quality services as specified in the
contract.[Footnote 11] Agency oversight provides quality assurance
independent of the contractors' own quality control processes. Although
each agency employed some oversight practices, only two of the six
agencies we reviewed used all four of the oversight practices we
identified for ensuring that accurate information is provided to the
public. Each agency emphasized accuracy of information to varying
degrees within its practices.
On the basis of our review of industry contact center practices and the
practices employed by the agencies considered leaders in government
contact centers, we identified four agency oversight practices related
to ensuring that accurate information is provided to the public via a
contractor-operated contact center.[Footnote 12] Table 3 describes the
four accuracy-related oversight practices.
Table 3: Description of Oversight Practices for Ensuring Accuracy of
Information:
Oversight practice: Regular knowledge database management;
Description: Regular review of the information used by customer service
representatives to respond to inquiries, known in the industry as the
"knowledge database;" "regular" is defined as occurring at least
annually.
Oversight practice: Regular contact monitoring;
Description: Regular reviews of the information provided in calls or e-
mails to evaluate how well the customer service representatives handled
the inquiry; the evaluation typically uses a score sheet that allows
the reviewer to rate the customer service representatives in multiple
areas, such as courtesy, accuracy of information provided, timeliness,
completeness, and so forth; "regular" is defined as occurring at least
on a weekly basis.
Oversight practice: Postcontact customer satisfaction surveys;
Description: Surveys asking if the individual was satisfied with the
service received from the contact center; survey questions ask about
the individual's opinions, including, to a limited extent, whether the
information received was accurate; "postcontact" surveys refer to those
initiated by the agency subsequent to the initial contact and do not
include self-selected surveys available at the end of a telephone
conversation.
Oversight practice: Validation of contractor reports;
Description: Validating the data provided in contractor-prepared
reports to ensure accuracy; these reports, which could be provided
daily, weekly, monthly, or quarterly, include operational information
such as the center's workload volumes, transaction handling times, and
results of the contractor's contact monitoring.
Source: GAO analysis based on review of industry practices and
discussions with officials from the agencies we reviewed.
[End of table]
The first two practices, knowledge database management and agency
contact monitoring, provide direct oversight regarding accuracy of
information, because they focus on detecting inaccuracies in the source
information used to provide responses to the public and in the actual
responses provided by the customer service representatives. The
remaining two practices, customer satisfaction surveys and validation
of contractor-prepared reports, are more indirect methods of ensuring
accuracy in that they review customers' reactions to the information
provided and independent agency corroboration of the contractor's
reporting on its own quality procedures. The agencies we reviewed
varied with respect to how they implemented these practices. This
variance was due to a number of factors, such as differences among the
agencies in staffing levels, funding, and the use of guidance specific
to the agency. Table 4 shows the extent to which the six agencies we
reviewed employ each of the accuracy-related oversight practices.
Table 4: Agency Oversight Practices Used for Ensuring Accuracy of
Information:
Agency: CDC;
Agency oversight practices: Regular knowledge database review: Planned;
Agency oversight practices: Regular contact monitoring: Daily by third-
party contractor;
Agency oversight practices: Postcontact customer satisfaction surveys:
Planned;
Agency oversight practices: Validation of contractor reports: By third-
party contractor.
Agency: DOD-TMA North;
Agency oversight practices: Postcontact customer satisfaction surveys:
Yes;
Agency: DOL;
Agency oversight practices: Regular knowledge database review:
Continuous basis;
Agency oversight practices: Regular contact monitoring: Weekly.
Agency: Education;
Agency oversight practices: Regular knowledge database review:
Annually;
Agency oversight practices: Regular contact monitoring: Weekly.
Agency: GSA;
Agency oversight practices: Regular knowledge database review:
Continuous basis;
Agency oversight practices: Regular contact monitoring: Weekly;
Agency oversight practices: Postcontact customer satisfaction surveys:
On publication orders.
Agency: USPS;
Agency oversight practices: Regular knowledge database review:
Continuous basis;
Agency oversight practices: Regular contact monitoring: Weekly;
Agency oversight practices: Postcontact customer satisfaction surveys:
Yes;
Agency oversight practices: Validation of contractor reports: With
third-party contractor assistance.
Source: GAO analysis.
[End of table]
Oversight Practice 1: Regular Knowledge Database Reviews:
Most of the agencies we reviewed had a structured process for ensuring
accurate information is maintained in the knowledge database. DOL,
Education, GSA, and USPS approve contractor-developed information that
is created based on government-provided materials. These agencies then
perform periodic reviews of the information in the knowledge database.
CDC currently prepares all scripted responses and Web site information,
which the contractor is required to use, and plans to implement annual
reviews of the knowledge database, starting at the first anniversary of
operation in February 2006. TMA allows the contractor to develop
information based on material TMA provides, but does not review the
information used by the contractor to respond to public inquires. DOD
said that TMA relies on the expertise and skills of its contractor to
provide the required services.
Oversight Practice 2: Regular Agency Contact Monitoring:
Almost all of the agencies we reviewed perform regular monitoring of
the contractor's responses to the public to help assess whether
accurate information is provided. CDC, DOL, Education, GSA, and USPS
each monitor a number of contacts on a regular basis, although accuracy
of information is addressed to varying degrees in the score sheets. For
example, accuracy is clearly weighted as an important aspect of the
call in CDC's score sheet. Therefore if an inaccurate answer is
provided, the contractor "fails" for that call and the customer service
representative is counseled. On the other hand, Education's score sheet
does not clearly weight accuracy of information. Education and its
contractor staff could not explain how providing inaccurate information
on a call would be indicated on the monitoring score sheet. In addition
to giving different weights to accuracy, the five agencies also vary in
terms of the frequency with which they monitor their contacts.
Education and USPS each employ one full-time staff to monitor a
selection of the contact centers' contacts. CDC has a third-party
contractor monitor the contact center on a daily basis and uses this
assessment in the determination of the contractor's award fee. GSA
staff monitor a sample of calls on a weekly basis and started
performing quarterly audits of the contractor's monitoring efforts in
November 2005. The sixth center, TMA, only monitors calls on an ad hoc
basis when officials visit the contact center.
Oversight Practice 3: Postcontact Customer Satisfaction Surveys:
Three of the agencies we reviewed conduct customer satisfaction surveys
subsequent to the initial contact from an individual. GSA, TMA, and
USPS conduct customer satisfaction surveys, which ask, to limited
degrees, questions that address the accuracy of information provided.
While providing some level of insight regarding accuracy, customer
surveys may not always provide a valid basis for oversight of the
accuracy of information, since they usually ask the individual's
opinion on the service provided. If the survey is conducted too closely
to the time of the inquiry, the individual may not have had time to act
upon the information to know whether it is accurate or not. CDC plans
to implement three types of postcontact customer satisfaction surveys
through a third-party contractor beginning in June 2006. DOL does not
conduct postcontact surveys because it does not maintain personal
information on the individuals that contact the agency.
Oversight Practice 4: Validation of Contractor Reports:
Three of the six agencies we reviewed take steps to validate the
information in the contractor-prepared reports related to contact
center performance. These reports generally include some aspects
related to accuracy of information provided to the public, such as the
contractor's results of its monitoring of contacts. CDC and USPS
validate to some degree the reports provided by the contractor. GSA
conducts quarterly audits of its contractor's supporting data. Although
DOL, Education, and TMA review their contractor reports, they rely upon
the reports without validation. According to GAO's standards for
internal control in the federal government, good internal control
practices require that agencies validate the performance reports
provided by the contractor to ensure the information is valid.[Footnote
13]
Governmentwide Guidance and Information on Contact Centers Do Not
Exist:
The federal government does not have comprehensive, centralized
guidance for operating a contact center or for overseeing a contractor-
operated center. Although operation and oversight of contact centers
are the responsibility of individual agencies, GSA, in consultation
with OMB, determined that governmentwide standards would be useful. GSA
sponsored an interagency committee that recently provided draft
guidelines for operating federal contact centers to OMB and other
federal agencies. However, OMB told us it does not plan to issue any
governmentwide guidance based on the committee's recommended guidelines
at this time, because OMB has not identified the operation of contact
centers as an area of concern. Furthermore, until recently, no
governmentwide information specific to contact centers has been
collected. Initial attempts to gather governmentwide information about
the number and type of activities that agencies use to provide public
information proved to be inadequate for providing a comprehensive
governmentwide view of contact centers. In addition, officials from the
agencies we reviewed told us that no industry classification code in
FPDS currently covers the full range of services provided by a contact
center.
Limited Federal Guidance for Operating and Overseeing Contractor-
Operated Contact Centers:
In its 2004 report on the electronic government initiative, OMB
highlighted the importance of delivering timely and accurate
information to the public and stated that there are opportunities to
apply existing and emerging best practices to achieve increases in
productivity and delivery of services and information.[Footnote 14] To
date, however, OMB has established only limited guidance on preferred
practices at contact centers. The only OMB guidance[Footnote 15] we
found that specifically related to contact centers is focused on the
use of performance-based contracting for such services.[Footnote 16]
This guidance is dated and limited in its coverage and does not provide
guidance on performance metrics for contact centers or oversight
practices.
Because of the need for governmentwide standards for operating contact
centers, GSA in consultation with OMB, took the initiative to form an
interagency working group to propose guidelines to OMB and other
federal agencies. Formed in March 2005, the Citizen Service Levels
Interagency Committee is composed of 58 contact service representatives
from 33 executive branch agencies. In addition to relying on their
knowledge in running contact centers, the committee also had a
contractor perform two studies to provide insight on citizens'
expectations when contacting government agencies for information and
current industry metrics, benchmarks, and best practices for operating
contact centers. The committee submitted a report with 37 proposed
standards for operating contact centers to OMB in September 2005,
including four standards specifically related to ensuring accuracy. The
committee plans to continue to work on additional contact center issues
and to help agencies implement any contact center standards that OMB
might endorse. In October 2005, OMB officials stated that they had
reviewed the committee's report but did not plan to issue any
governmentwide guidance based on the committee's recommended guidelines
at this time, because OMB has not identified the operation of contact
centers as an area of concern. OMB stated further that if agencies need
additional guidance in developing their standards, they can refer to
the committee's report.
The agencies we reviewed each performed independent research to develop
their contracts and formulate a management strategy for operating their
contact centers. Performing independent research resulted in
duplication of efforts across agencies, using limited resources and
taking valuable time. For example, to develop guidance, the Department
of Education performed market research, worked with a contractor on
customer services and related standards, and studied industry best
practices on a limited basis. Similarly, CDC sent out a request for
information to industry to gain insight on the technology available for
operating contact centers before it developed its contract. CDC then
performed market research, reviewed industry practices, and visited
other government contact centers, such as that of the Social Security
Administration and the Centers for Medicare and Medicaid Services, to
learn about the practices of government-and contractor-operated
centers.
In 2004, GSA created a multiple-award contract called FirstContact to
assist agencies in contracting for contact center services. Under this
multiple-award contract, agencies can issue a task order to any of five
preapproved contractors to operate a contact center. Using FirstContact
will minimize the time and effort required of agencies to locate a
contractor to manage their centers. To date, the Department of Homeland
Security's Federal Emergency Management Agency, GSA, and the Department
of Health and Human Services have placed six task orders against this
contract, and three other agencies are looking to place orders as well.
For example, the Federal Emergency Management Agency recently used this
contract vehicle to quickly provide contact center services for the
influx of calls and applications for government assistance in the
aftermath of Hurricane Katrina. Although agencies now have this
multiple-award contract as a mechanism to assist them in contracting
for contact center service, they still must develop specific
performance metrics and oversight practices specific to their center.
Governmentwide Data on Contact Centers Do Not Exist:
Given the lack of governmentwide information on the activities that
provide information to the public, OMB made an initial attempt to
collect such data in 2004. For this effort, OMB issued a data request
to all executive branch agencies to obtain basic data, such as the
contact center name, volume of contacts, and whether performance and
cost metrics are collected, for every activity that provides
information directly to the public. OMB normally uses data requests as
a census tool to acquire a snapshot of the current budget environment
of the government. Agencies responded by self-identifying over 1,800
activities that currently provide information to citizens using various
communication channels such as telephone, e-mail, and Internet Web
sites. The individual activities identified ranged in size from a
couple of employees who answer telephone calls as part of their duties
to contact centers with staffs of several hundred employees who handle
millions of inquiries through several channels. Of the 1,800 activities
identified, over 500 categorized themselves as contact centers. Since
it was making a nonstandard data request, OMB performed little follow-
up on nonresponding agencies and did not verify reported results. We
noted that some large agencies, such as DOD, did not report any
activities that provide information to the public.
In an effort to expand on the information collected through the OMB
data request, GSA surveyed 360 activities--approximately a quarter of
those who responded to the data request--to develop a baseline snapshot
of governmentwide activities providing information to the public.
However, GSA's survey methodology was flawed because the agency
selected its sample from an incomplete universe, had a low survey
response rate, and did not perform a nonresponder analysis. Thus, the
survey results did not provide a representative view of activities
across the government. GSA plans to conduct a follow-up survey of
government activities in 2007.
While OMB and GSA information regarding the universe of federal contact
centers is incomplete, another potential source of information on those
contact centers that are contracted out by the government is the FPDS.
Since its inception in 1978, the FPDS has served as the governmentwide
system for collecting federal procurement data. Five of the six
agencies we reviewed, however, each used a different code to report
their contact center procurement actions to FPDS.[Footnote 17]
Officials from agencies we reviewed told us that no current North
American Industry Classification System (NAICS) code covers the full
range of services provided by a contact center.[Footnote 18] Table 5
lists the NAICS codes used by the five agencies reporting data to FPDS.
Table 5: NAICS Codes Used for Contact Center Contracts Reviewed:
Agency: CDC;
NAICS code: 518210;
NAICS definition: Data processing, hosting, and related services.
Agency: DOD-TMA;
NAICS code: 524114;
NAICS definition: Direct health and medical insurance carriers.
Agency: DOL;
NAICS code: 541519;
NAICS definition: Other computer- related services.
Agency: Education;
NAICS code: 541512;
NAICS definition: Computer systems design services.
Agency: GSA;
NAICS code: 519190;
NAICS definition: All other information services.
Source: GAO analysis.
[End of table]
The five agencies that reported to FPDS chose different NAICS codes for
different reasons. Officials from DOL and GSA stated that they chose
alternative NAICS codes because the definition provided for telephone
call center does not cover all of the activities handled in a contact
center. Education and DOD-TMA officials explained that they chose NAICS
codes that encompassed the main work of the contract, since the contact
center is only a portion of the work in a contract for a larger
program. CDC chose its NAICS code based upon the information technology
services required for creating its contact center.
No governmentwide procurement information was reported to FPDS using
the NAICS codes for telephone call centers in fiscal years 2000 through
2004. This category of NAICS codes--56142--is defined as establishments
primarily engaged in answering telephone calls and relaying messages or
in telemarketing activities. Although officials from three of the
agencies we reviewed expressed the opinion that the definition for
telephone call centers is too narrow to encompass all the work
performed by a contact center, OMB told us that the telephone call
center code is the correct code to use. Specifically, OMB stated that
the subcode of 561422--telemarketing bureaus--was written with the
intent to cover all the functions of a contact center. OMB is
considering issuing a clarification to the description of the 56142
codes to explain that these codes include more than telephones--such as
Web sites, e-mails, facsimiles, and so forth--in its next update to the
NAICS manual in 2012.
Conclusions:
Providing timely and accurate information is a key result area for the
federal government. Federal agencies that use contractor-operated
contact centers to meet the public's demand for information assume the
burden of assuring that the information provided by the contractors is
accurate. While the agencies we reviewed have taken a variety of steps
to ensure that their contractor-operated contact centers address
accuracy, at some agencies accuracy clearly does not have the same
priority as other objectives, such as timeliness. Although agencies
need flexibility in meeting the needs of the individuals that contact
them, they also can benefit from the experience gained by their peers
operating other government contact centers. Short of mandating specific
guidance, increased sharing among agencies of successful practices for
managing contact centers may help improve their abilities to write and
oversee contracts for these centers and may avoid needless duplication
of effort. The guidelines proposed by the Citizen Service Levels
Interagency Committee are a step in this direction. However, whether
this effort will result in information sharing across agencies is
uncertain. Leveraging knowledge gained by other agencies through the
sharing of effective practices could be enhanced by governmentwide
leadership. OMB's leadership of the electronic government initiative,
its role in guiding agency dissemination of public information, as well
as its procurement policy role, put it in an ideal position to
facilitate the exchange of information among agencies to ensure
effective oversight of contractors in meeting the public's need for
timely and accurate information.
While OMB and GSA have taken initial steps to enhance the oversight of
federal contact centers by gathering some information on the universe
of these centers, it is not clear whether the data collected provide
enough information for governmentwide oversight of contact center
operations or whether GSA's planned data collection efforts will do so
either. With additional reliable information, OMB may be able to more
quickly identify and act on emerging problems and opportunities. In
addition, FPDS can also be more effective in identifying the number of
contracts and dollars obligated for contact centers across the
government, but only if the agencies consistently use the appropriate
NAICS code for these services.
Recommendations for Executive Action:
To facilitate the sharing of sound oversight practices for the
operation of contact centers, to help ensure that providing accurate
information to the public by contact centers is a priority outcome, and
to improve the quality of information gathered about these centers, we
recommend that the Director of the Office of Management and Budget take
the following actions:
* Building on efforts begun by the GSA-sponsored interagency committee,
work with agencies to develop a mechanism for sharing performance
metrics and oversight practices for contact centers. Continued efforts
should stress that providing accurate information to the public needs
to be a key factor in the oversight of federal contact centers.
* Take steps to ensure consistent reporting on contact centers by
developing an industry category or specific code definition in NAICS
that encompasses all the services provided by contact centers or by
providing further instruction to agencies regarding the appropriate
NAICS code to use for contact centers.
To improve the quality of information about federal contact centers, we
recommend that the Administrator of General Services take the following
action:
* Ensure that further efforts to develop governmentwide data on contact
center operations--such as the survey planned for next year--employ
sound methodologies to ensure that the resulting information is
representative of the activities across the government.
Agency Comments and Our Evaluation:
We requested comments on a draft of this report from the Office of
Management and Budget and each of the six agencies we reviewed--
Department of Defense, Department of Education, Department of Health
and Human Services, Department of Labor, General Services
Administration, and U.S. Postal Service. The Office of Management and
Budget provided oral comments in which they concurred with our findings
and recommendations. The Department of Defense, Department of Health
and Human Services, and the General Services Administration provided
written comments that are reproduced in appendices III, IV, and V,
respectively. OMB and most of the agencies also provided technical
comments, which we incorporated as appropriate.
The Department of Health and Human Services and the General Services
Administration also concurred with our findings and recommendations.
The Department of Defense did not concur with our draft report because
it believes the report does not fully reflect all the metrics and
practices DOD and its contractor use to ensure the accuracy of
information provided to TRICARE beneficiaries. In its comments, DOD
emphasizes that its approach to contracting for contact center
operations relies on the contractor to use industry standards for
ensuring information accuracy. DOD states that standards exist in its
contract related to the accuracy of information provided by telephone.
DOD also cites additional metrics it uses for monitoring contractor
performance. In addition, DOD requires the contractor to have a quality
management program which must be validated by a nationally recognized
third-party organization. DOD points out that it receives monthly
briefings on the operation of the contractor's quality management
program and observes call center operations during site visits.
Finally, DOD explained that it monitors the expertise and skills of the
contractor staff that perform the knowledge management function.
We recognize that DOD has decided to use what it calls the "audit the
auditor" approach to quality assurance. It was not our objective,
however, to assess the merits of any particular approach to ensuring
quality, but rather to determine the extent to which contract terms and
agency oversight practices emphasize the importance of providing
accurate information to the public. In this regard, while the
contractor may use specific standards for accuracy in its quality
management program, we found no specific metric related to accuracy in
the TRICARE contact center contract itself or in the additional metrics
cited in DOD's comments. For the most part, the additional quality
control activities listed by the Department are those of its
contractor, not oversight activities performed by the agency, which was
the focus of our review. While independent validation of the
contractor's quality control program helps to ensure the contractor has
a quality process in place for monitoring its responses to the public,
this does not substitute for DOD oversight activities such as
validating the contractor's reports of its monitoring efforts. In
addition, while DOD performs site visits to oversee the contractor's
operations, it does so only on an ad hoc basis. Based on DOD's
comments, we added additional language to the report regarding DOD's
approach to knowledge management.
As arranged with your offices, unless you publicly announce its
contents earlier, we plan no further distribution of this report until
30 days after the date of this report. We will then send copies of this
report to the Director of the Office of Management and Budget, the
Administrator of General Services, the Postmaster General, and the
Secretaries of the Department of Defense, Department of Education,
Department of Health and Human Services, and Department of Labor. We
will also make copies available to others upon request. Contact points
for our Office of Congressional Relations and Public Affairs may be
found on the last page of this report. This report is available at no
charge on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please call
me at (202) 512-4841. An additional GAO contact and staff who made
contributions to this report are listed in appendix VI.
Signed by:
William T. Woods:
Director, Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
To assess the guidance provided to federal agencies and the information
gathered by the federal government about contact centers, we conducted
interviews with Office of Management and Budget (OMB) and General
Services Administration (GSA) officials, and reviewed related guidance
and results of their initial data collection efforts. We researched and
discussed with OMB the absence of guidance related to the operation and
oversight of contact centers. We also discussed the results of OMB's
2004 request for information to federal agencies that asked for self-
identification of any activities that provide information to the
public. In addition, we reviewed and discussed the results of GSA's
survey of a sample of agency activities that responded to OMB's
request. We did not assess the validity of the data gathered by OMB and
GSA. However, a GAO methodologist reviewed the GSA survey methodology
and identified its weaknesses. In addition, we monitored the progress
of the GSA-sponsored working group--the Citizen Service Levels
Interagency Committee--as it developed and recommended standards to OMB
for federal contact centers. We did not assess the committee's
recommendations as a whole, but rather reviewed how accuracy of
information was addressed within its proposed standards. We reviewed
data from the Federal Procurement Data System for the past 5 fiscal
years to determine if any contract actions were reported using the code
for telephone call center services.[Footnote 19]
To describe federal agencies' efforts to ensure accurate information is
provided to the public by contractor-operated centers, we reviewed the
contract terms and oversight activities for one center at each of six
agencies. We selected centers that handle over 1 million inquiries
annually and provide information to citizens that could significantly
affect their finances, health, or safety.[Footnote 20] The contact
centers selected for our review are:
* Department of Defense TriCare Management Activity (TMA) North region-
-Healthnet's contact center: provides general and personalized medical
benefit and coverage information and processes enrollments and claims
for military families in the North region;[Footnote 21]
* Department of Education (Education)--Federal Student Aid Information
Center: provides general information about applications and loan issues
and personalized information on the status of applications and loans to
the public and academic community;
* Department of Health and Human Services' Centers for Disease Control
and Prevention (CDC)--CDC INFO contact center: provides information
about health and safety issues--including prevention, detection, and
outbreak control--to the public and medical professionals;
* Department of Labor (DOL)--National Contact Center: provides general
information and referrals regarding job issues, workplace safety, and
pension and health benefits to the public and employers;
* General Services Administration--National Contact Center: provides
general information and referrals related to any agency or government
program; and:
* U.S. Postal Service (USPS)--National Contact Center: provides general
and individualized information on mail delivery and shipping issues to
the public and businesses.
To complete our review, we interviewed management and staff responsible
for oversight of the contractor-operated contact center at each agency.
We reviewed the performance metrics specified in the agency's contract
as well as the related reports used to oversee and evaluate the
contractors' operation of the contact centers. In addition to
conducting discussions with the agencies, we visited four contractor-
operated centers to observe their operations and quality control
procedures. Specifically, we visited locations for the GSA center
operated by ICT Group, CDC and Education centers operated by Pearson
Government Solutions, and the DOL center operated by Datatrac
Information Services.[Footnote 22] At each center we interviewed
management and customer service representatives regarding the oversight
practices used to monitor the accuracy of information. We did not test
the contractors' internal control procedures or validate any data from
their sample reports.
We identified industry practices for ensuring the accuracy of
information provided by contact centers, interviewed representatives
from two major contact center industry groups--the Society of Consumer
Affairs Professionals and the Incoming Calls Management Institute--and
attended the 2005 Government Customer Support Conference. In addition,
we reviewed prior GAO reports concerning contact centers. We also
discussed contact center issues with other GAO teams that were
currently reviewing or had recently reviewed other federal contact
centers. Our work was conducted from February through November 2005 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix II: Contact Center Profiles for Six Agencies Reviewed:
Table 6: Contact Center Profiles for Agencies Reviewed:
Agency: Contact center;
Centers for Disease Control and Prevention: CDC INFO;
Department of Defense TriCare Management Activity--North: Healthnet's
contact center;
Department of Labor: National Contact Center;
Department of Education: Federal Student Aid Information Center;
General Services Administration: National Contact Center;
U.S. Postal Service: National Contact Center.
Agency: Purpose of public inquiries;
Centers for Disease Control and Prevention: Medical disease issues--
including prevention, detection, and outbreak control;
Department of Defense TriCare Management Activity--North: Medical
benefits and coverage issues, enrollment, and claims processing;
Department of Labor: Job loss support, wage information, workplace
safety, pension and health benefits; fulfillment of orders for posters,
fact sheets, and handbooks;
Department of Education: Federal Student Aid application and loan
issues;
General Services Administration: Any citizen issues for any agency in
the government; fulfillment orders for government publications;
U.S. Postal Service: Mail delivery and shipping information.
Agency: Methods of contact handled;
Centers for Disease Control and Prevention: Telephone, e-mail;
Department of Defense TriCare Management Activity--North: Telephone, e-
mail;
Department of Labor: Telephone, e-mail;
Department of Education: Telephone, e-mail, facsimile, Web site, online
live help;
General Services Administration: Telephone, e-mail, Web site;
U.S. Postal Service: Telephone, e-mail, facsimile, Web sites.
Agency: Main hours of operation;
Centers for Disease Control and Prevention: Daily 24 hours/day;
Department of Defense TriCare Management Activity--North: Daily 7:00
a.m. to 7:00 p.m. (eastern time);
Department of Labor: Monday-Friday 8:00 a.m. to 8:00 p.m. (eastern
time)[A];
Department of Education: Monday-Friday 8:00 a.m. to midnight; Saturday
9:00 a.m. to 6:00 p.m. (eastern)[A];
General Services Administration: Monday-Friday 8:00 a.m. to 8:00 p.m.
(eastern)[A];
U.S. Postal Service: Monday-Friday 8:00 a.m. to 8:30 p.m; Saturday 8:00
a.m. to 6:00 p.m. (eastern time).
Agency: Languages supported (beyond English);
Centers for Disease Control and Prevention: Spanish;
Department of Defense TriCare Management Activity--North: Spanish;
Department of Labor: Spanish and 140 additional languages via
translation service;
Department of Education: Spanish;
General Services Administration: Spanish[B];
U.S. Postal Service: Spanish.
Agency: Fiscal year 2005 call volume;
Centers for Disease Control and Prevention: 314,749; from February to
September 2005 (estimated at 2.7 million calls when fully
operational)[C];
Department of Defense TriCare Management Activity--North: 5,236,793;
Department of Labor: 1,347,476;
Department of Education: 5,780,158;
General Services Administration: 1,700,000;
U.S. Postal Service: 64,001,962.
Agency: Fiscal year 2005 e-mail volume;
Centers for Disease Control and Prevention: 834; from February to
September 2005 (estimated at 18,000 e-mails when fully operational)[C];
Department of Defense TriCare Management Activity--North: 56,160;
Department of Labor: 71,765;
Department of Education: 488,158;
General Services Administration: 107,000;
U.S. Postal Service: 1,041,210.
Agency: Contractor;
Centers for Disease Control and Prevention: Pearson Government
Solutions;
Department of Defense TriCare Management Activity--North: Healthnet
Federal Services;
Department of Labor: Datatrac Information Services;
Department of Education: Pearson Government Solutions;
General Services Administration: ICT Group;
U.S. Postal Service: Convergys.
Agency: Date of award;
Centers for Disease Control and Prevention: September 1, 2004;
Department of Defense TriCare Management Activity--North: September 1,
2003;
Department of Labor: September 30, 2001;
Department of Education: February 1, 2005;
General Services Administration: November 28, 2004;
U.S. Postal Service: January 27, 2003.
Agency: Period of performance;
Centers for Disease Control and Prevention: 2-year base plus 5 1-year
options;
Department of Defense TriCare Management Activity--North: 7-month base
plus 5 1-year options;
Department of Labor: 1-year base plus 4 1-year options;
Department of Education: 1-year base plus 9 1-year options;
General Services Administration: 1-year base plus 4 1-year options;
U.S. Postal Service: 4-year base plus 6 1-year options.
Agency: Total value of contract at award (in millions of dollars);
Centers for Disease Control and Prevention: $73.7;
Department of Defense TriCare Management Activity--North: $2,298; (the
contact center is only a portion of this contract)[D];
Department of Labor: $23;
Department of Education: $80.1; (the contact center is only a portion
of this contract)[D];
General Services Administration: $29;
U.S. Postal Service: $254.6 (for 4-year base only).
Agency: Payment structure;
Centers for Disease Control and Prevention: Cost plus award fee;
Department of Defense TriCare Management Activity-
-North: Firm fixed price plus award fee;
Department of Labor: Time and materials;
Department of Education: Firm fixed price plus award fee;
General Services Administration: Firm fixed price;
U.S. Postal Service: Firm fixed price plus award fee.
Agency: Number of customer service representatives;
Centers for Disease Control and Prevention: 50-100;
Department of Defense TriCare Management Activity--North: 600-650;
Department of Labor: 50-100;
Department of Education: 400-450;
General Services Administration: 50-100;
U.S. Postal Service: 1,200-1,300.
Agency: Number of locations;
Centers for Disease Control and Prevention: 3;
Department of Defense TriCare Management Activity--North: 6;
Department of Labor: 2;
Department of Education: 3;
General Services Administration: 1;
U.S. Postal Service: 3.
Source: GAO analysis.
[A] DOL provides service 24 hours a day for the Occupational Safety and
Health Administration toll-free number and provided service 24 hours a
day during hurricane relief efforts; Education extends its hours during
student aid application season; GSA provides service 24 hours a day
under emergency situations.
[B] Although only Spanish is provided at this time, GSA's contract
allows for the provision of additional languages.
[C] The CDC contact center is in its second year of operation and is
consolidating the work for 40 different toll-free numbers over a total
period of 4 years.
[D] The contact center is a portion of a larger service contract. The
value shown here is for the entire contract, as the agency could not
provide a breakdown of the cost for the contact center alone.
[End of table]
[End of section]
Appendix III: Comments from the Department of Defense:
THE ASSISTANT SECRETARY OF DEFENSE:
HEALTH AFFAIRS:
1200 DEFENSE PENTAGON:
WASHINGTON, DC 20301-1200:
JAN 24 2005:
William T. Woods:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G. Street, N.W.:
Washington, DC 20548:
Dear Mr. Woods:
This is the Department of Defense (DoD) response to the Government
Accountability Office (GAO) Draft Report, "FEDERAL CONTACT CENTERS:
Mechanism for Sharing Metrics and Oversight Practices Along with
Improved Data Needed," dated December 21, 2005, GAO Code 120407/GAO-06-
270).
Thank you for the opportunity to review and comment on the draft
report. Access to healthcare is one of my highest priorities. Ensuring
our beneficiaries are able to contact our call centers to address
questions and receive accurate and timely information is vitally
important to the success of our TRICARE program. This report provides
invaluable information on bow other agencies support their contact
centers and ensure the accuracy of information that might be useful in
how we operate our centers.
I want to take this opportunity to clarify the contracting approach TMA
uses, as well as the Department's approach to performance management
from both a contractual and operational perspective.
Overall, I do not concur with the report's findings and
recommendations. The data contained in the report portrays that TMA is
not concerned with nor takes efforts to ensure the accuracy of
information provided to our beneficiaries. In developing our new
TRICARE contracts (T-NEX) we utilized state-of-the-art contracting
techniques to obtain best practices of our contractors to provide
services and healthcare. Further, your report suggests that TMA only
utilizes two performance metrics. This is incorrect. We use many
additional measures (also noted in your assessment of other agencies)
such as first contact resolution, average speed of answer, abandonment
rate, hold time, staffing ratios, service level, and customer
satisfaction. Therefore, your comparison between the six agencies in
the report and TMA is inaccurate. Our contact centers are highly
effective according to ratings we receive from our beneficiaries, and
according to our measures. Additional information addressing this is
enclosed.
My points of contact are Ms. Christine VanCleave at (303) 676-3639 or
Mr. Gunther Zimmerman at (703) 681-3492.
Sincerely,
Signed by:
William Winkenwerder, Jr.,
Enclosure: As stated:
GAO DRAFT REPORT -DATED December 21, 2005 GAO CODE 120407/GAO-06-270:
"FEDERAL CONTACT CENTERS: Mechanism for Sharing Metrics and Oversight
Practices Along with Improved Data Needed"
DEPARTMENT OF DEFENSE COMMENTS:
GENERAL:
Following acquisition reforms principles, in 2003 the TRICARE
Management Activity (TMA) awarded managed care support contracts that
reflected best commercial practices and performance outcomes. This
represented a divergence from our historical approach where the
Government mandated processes, procedures and standards and incurred
the much higher costs of these Government specified provisions.
Acquisition reform principles required the Government to assess risk
and only specify Government processes, procedures and standards where
industry standards are not acceptable. This does not mean that
standards do not exist. It simply means that the Government is
accepting of industry-proposed standards when these standards are
expected to achieve the Government's specified outcome of the contract.
TMA, the Surgeons General, and the Assistant Secretary of Defense for
Health Affairs determined that one of the five objectives of our
healthcare delivery contracts was to achieve "beneficiary satisfaction
at the highest level possible throughout the period of performance.."
With this contract objective defined, a risk assessment of every
activity occurring through the contracts was conducted to determine
where there was a high risk of failure associated with commercial best
practices. In relation to our call center requirements, the
Government's experts determine that commercial "hold times," "blockage
rates," and the commercial use of "automated response units" and call
routing equipment represent a high risk of failure related to
maintaining beneficiary satisfaction at the highest possible level.
Because of the risk associated with commercial best practices, the
Government mandated standards for these functions. The same risk
assessment was conducted related to an information accuracy standard.
The Government team determined that industry could not afford errors in
their commercial sector from either a customer satisfaction or
liability standpoint and, as such, assessed the risk associated with
accepting commercial standards for accuracy as low. In the single
contract considered by the GAO, the quality control program mandated by
the TRICARE Operations Manual, Chapter l, Section 4, paragraph 4.2.
[NOTE 1] specifically requires standards and monitoring of the accuracy
of telephone based responses.
NOTE:
[1] The contract, Section C, paragraph C-3.1. incorporates the TRICARE
Operations Manual as "an integral part of this contract."
Health Net Federal Services (HNFS) complied with this requirement and
proposed a 95% standard. Contractually, this standard requires HNFS to
"establish and operate an internal quality management/quality
improvement program covering every aspect of the contractor's
operation, both clinically and administratively." (See contract
provision C-7.25.) [NOTE 2] This approach allows the Government to
focus its surveillance efforts on the operationally high risk areas,
monitor areas of reasonable risk, and be aware of areas with minimal
risk. At present telephone accuracy is an area assessed as reasonable
risk. IMA assesses contract performance by mandating internal sampling
and requiring monthly monitoring. During on-site visits, TMA samples
the work of HNFS quality management department to validate the accuracy
of reported information and the implementation of corrective actions.
This approach effectively utilizes the skills of Government personnel
who must focus on the outcome of the call, including accuracy,
timeliness, tone, and follow-up. Each of these individual components is
effectively monitored through a contractor operated Quality
Management/Quality Improvement (QM/QI) process that is validated by the
Government and a nationally recognized body such as International
Standards Organization (ISO).
TMA also uses independent sources for performance monitoring. HNFS is
required, by contract, to obtain ISO certification of their quality
management program. This independent certification provides the
Government with additional assurance that the quality management
processes proposed are utilized, that these processes measure intended
objectives, and that results are accurately reported. HNFS has obtained
this certification and continues to manage based on this certification
criteria. A second ISO 9001:2000 surveillance audit of the HNFS
Business Management System is tentatively scheduled for January 2006.
The assessment will include site visits to HNFS corporate headquarters
in Rancho Cordova CA, Hampton VA and San Diego CA Medical Management
offices, the Mid Atlantic Sub-regional Headquarters in Norfolk VA, and
three of the sub- region's TRICAR , Service Centers (Kenner Army Health
Clinic, Seymour Johnson Air Force Base Clinic, and Cherry Point Naval
Hospital.
The TRICARE Regional Office (TRO) Director and staff personally observe
call center operations when visiting HNFS corporate offices, and
receive monthly performance briefings on HNFS quality management
programs. The HNFS call center has an extensive training program for
new associates, and an ongoing monitoring program for all associates
that includes evaluation of recorded calls and "chair-side" sessions
evaluating the accuracy of information in real time. All calls are
recorded and reviewed for accuracy within a few days. If erroneous
information is observed, the beneficiary is contacted and correct
information communicated. HNFS has processes in place to ensure that
only the latest "approved" information is accessible to customer
agents.
NOTE:
[2] The Internal Quality Management/Quality Improvement Program was a
distinguishing feature in the evaluation of offerors.
In summary, standards do exist in TRICARE contracts related to the
accuracy of information provided by telephone. These standards are
assessed on a routine basis (one percent of all calls) and reported
monthly. Telephone operations are observed and monitored during field
visits, and HNFS provides monthly information briefings discussing
quality management performance. TMA uses the audit the auditor approach
to validate the accuracy of the information provided monthly and
require independent ISO certification of the processes employed. This
approach allows for the implementation of cost effective industry
standards and avoids placing the Government in the position of managing
and monitoring individual contractor employees in favor of managing the
outcome of the contracts.
* Page 6. Most Agencies Contract Metrics Include Accuracy, but Some Did
Not. The GAO states that four of the six agencies' contracts included
accuracy of information in one more of the key performance metrics. The
remaining two agencies (to include TMA) did not have specific metrics
that addressed the need to provide accurate information to the public.
TMA Comment: The perception created by this narrative is misleading.
TMA utilizes a "best practice" methodology in negotiating contracts in
which we rely upon the contractor to meet numerous standards in
supporting our requirements. No mention of this approach is cited in
the paragraph.
Page 6. Table 2: Summary of Key Performance Metrics in Agency
Contracts. The table identifies two metrics -busy signals and hold
times. TMA Comment: The table does not identify the "best practice"
approach utilized in our contracts, nor the many additional metrics
that we routinely monitor.
Page 9. Table 4: Agency Oversight Practices Used for Ensuring Accuracy
of Information. The table identifies oversight practices utilized by
the six agencies reviewed. TMA Comment: For TMA, only one oversight
practice is displayed - post contact customer satisfaction survey.
Again, the display of facts in this manner creates an inaccurate
portrayal that TMA is not concerned about the accuracy of information
provided to beneficiaries.
Page 9. Oversight Practice 1: Regular Knowledge Database Reviews. GAO
indicates that the other agencies maintain a structure process for
ensuring accurate information is maintained and provide scripted
responses and web site information which the contractors use in
responding to beneficiaries. GAO states that TMA allows the contractor
to develop information based on material TMA provides, but does not
review the information used by the contractor to respond to public
inquiries. TMA Comment: As identified earlier, by our use of "best
practices" we monitor the expertise and skills of the contactor to
provide the services we require.
[End of section]
Appendix IV: Comments from the General Services Administration:
GSA:
GSA Administrator:
January 26, 2006:
The Honorable David M. Walker:
Comptroller General of the United States:
Government Accountability Office:
Washington DC 20548:
Dear Mr. Walker:
The General Services Administration (GSA) appreciates the opportunity
to review your draft report, "Federal Contact Centers, Mechanism for
Sharing Metrics and Oversight Practices Along with Improved Data
Needed" (GAO-06-270). GSA has nearly 40 years of experience managing
programs that respond to public inquiries and is a recognized source of
assistance and expertise in this field. In fiscal year 2005 alone, we
facilitated the response to 8.5 million calls and emails from citizens.
Through the Presidential E-Gov initiative, USA Services, we have
provided leadership governmentwide to help agencies improve their
response to citizens.
The following are our comments to your draft report:
GAO draft report: The chart on page 9 indicates that GSA's National
Contact Center does not include "accuracy of information" in two of the
four key performance metrics used to evaluate contractor performance.
GSA Response: By December of 2005, GSA had put into effect activities
in each of the four areas. When your audit team visited GSA's National
Contact Center in June 2005, we had just begun to implement the terms
and conditions of a new contract with a new contractor. The chart below
shows the difference between what your team found when they reviewed
activities in June and the current status.
Table 4: Agency Oversight Practices Used for Ensuring Accuracy of
Information:
[See Table 4]
Specific actions that we have taken include the following.
Regular knowledge database review: GSA staff now reviews the National
Contact Center's database on a daily basis and has begun to conduct
quarterly audits, completing its first audit in December 2005. Although
the Government provided the version of the knowledge database used by
the previous contractor, the current contractor is responsible for its
further development and maintenance. They accomplish this by reviewing
the nature of telephone inquiries and the on-line use of the knowledge
base.
Regular contact monitoring: In addition to our regular contact
monitoring, GSA is conducting a quarterly audit of the contractor's
quality control efforts. The first audit was completed in November
2005.
Post contact customer satisfaction surveys: Although GSA's ability to
conduct post-contact customer satisfaction surveys of callers on its
information lines is limited by our not collecting information on
callers (in compliance with the letter and spirit of the Privacy Act),
we do conduct such surveys of our customers who order publications.
These surveys, performed under the auspices of the American Customer
Satisfaction Index, do include a review of the accuracy of the
transactions and therefore assist us in our evaluation of the
contractor's efforts.
Validation of contractor reports: We have started to perform validation
of contractor reports in two ways. One is a review of the quantity
measurements by our ability to review reports from MCI and track the
daily quantity of incoming e-mail. We validate their quality
assessments by regularly scheduled (at least monthly) calibration
sessions. These sessions assure that the contractor and the Government
independently apply the same standards to public inquiries. As
mentioned above, we also perform quarterly audits.
GAO report: Page 12, "Although agencies now have this multiple-award
contract as a mechanism to assist them in contracting for contact
center service, they still must develop specific performance metrics
and oversight practices specific to their center."
GSA response: We are proud that USA Services, a Presidential E-Gov
initiative managed by GSA, has provided governmentwide leadership by
establishing the Citizen Service Levels Interagency Committee (CSLIC).
USA Services established this group in February 2005 and it has now
completed recommendations to improve agency responses to citizens'
email and telephone inquiries. These recommendations are enclosed and
also available online at http://usaservices.gov/cslic.htm. The work of
CSLIC is ongoing. For the remainder of fiscal year 2006, four
subcommittees will be working toward certain deliverables. The items
below are just one example from each subcommittee:
1) Implementing CSLIC's Recommendations:
A template to help agencies determine their compliance with CSLIC
recommendations;
2) Online Resources:
A website for the Federal contact center community;
3) Measuring Customer Satisfaction:
A library of customer satisfaction measurement methods, and:
4) Drafting Additional Recommendations:
CSLIC has already improved its original report, to stress the
importance of accurate information being delivered to citizens. CSLIC
plans to draft additional recommendations related to, for example,
serving people with disabilities.
GAO draft report: Page 15, "..we recommend that the Administrator of
General Services.. ensure that the further efforts to develop
governmentwide data on contact center operations - such as the survey
planned for next year - employ sound methodologies so that the
resulting information is representative of the activities across the
government."
GSA Response: We concur with your recommendation for GSA executive
action and have already begun to develop a governmentwide assessment of
citizen service activities. We are working with the Office of
Management and Budget (OMB) to ensure a comprehensive gathering of
data. If the resources are available, our survey methodology will pay
particular attention to the number of agencies sampled and to the
analysis of non-responding agencies.
Information about USA Services can be found online at
http://usaservices.gov. If you have any questions about the information
that we have provided in this response, please feel free to contact Mr.
Stuart Willoughby, Director, USA Services, E-Gov Program Office, at
stuart.willoughby@gsa.gov or at (202) 501-9121.
We appreciate the opportunity to respond to GAO draft report GAO-06-270
and look forward to having our comments reflected in the final report.
Sincerely,
Signed by:
David L. Bibb:
Acting Administrator:
Enclosure:
[End of section]
Appendix V: Comments from the Department of Health & Human Services:
DEPARTMENT OF HEALTH & HUMAN SERVICES:
Office of Inspector General:
Washington, D.C. 20201:
JAN 24 2006:
Mr. William T. Woods:
Director:
Acquisition & Sourcing Management:
U.S. Government Accountability Office:
Washington, DC 20548:
Dear Mr. Woods:
Enclosed are the Department's comments on the U.S. Government
Accountability Office's (GAO's) draft report entitled, "FEDERAL CONTACT
CENTERS: Mechanism for Sharing Metrics and Oversight Practices Along
with Improved Data Needed" (GAO-06-270). These comments represent the
tentative position of the Department and are subject to reevaluation
when the final version of this report is received.
The Department appreciates the opportunity to comment on this draft
report before its publication.
Sincerely,
Signed by:
Daniel R. Levinson:
Inspector General:
Enclosure:
The Office of Inspector General (OIG) is transmitting the Department's
response to this draft report in our capacity as the Department's
designated focal point and coordinator for U.S. Government
Accountability Office reports. OIG has not conducted an independent
assessment of these comments and therefore expresses no opinion on
them.
COMMENTS OF THE DEPARTMENT OF HEALTH AND HUMAN SERVICES ON THE U.S.
GOVERNMENT ACCOUNTABILITY OFFICE'S (GAO) DRAFT REPORT ENTITLED,
"FEDERAL CONTACT CENTERS: MECHANISM FOR SHARING METRICS AND OVERSIGHT
PRACTICES ALONG WITH IMPROVED DATA NEEDED" (GAO-06-270):
The Department of Health and Human Services (HHS) appreciates the
opportunity to comment on the Government Accountability Office's (GAO)
draft report.
HHS's Centers for Disease Control and Prevention's (CDC) consolidated
contact center, CDC-INFO, began operation in February 2005 with a 4-
year phased implementation. Contact volumes to date include
approximately 700,000 telephone calls and 10,000 e-mail inquiries. CDC
staff constantly reevaluate not only content and content accuracy as
new topics are added but also revisit standard operating procedures to
ensure that all contacts are handled appropriately. In addition,
sampling strategies are adjusted to ensure that valid samples are
reviewed and evaluated. HHS and CDC consider not only accuracy but also
quality and completeness of service to be equally important indicators,
and CDC is diligent in its efforts to achieve these goals.
[End of section]
Appendix VI: GAO Contact and Staff Acknowledgments:
GAO Contact:
John K. Needham (202) 512-5274:
Acknowledgments:
In addition to the contact named above, Ruth Eli DeVan, William
McPhail, Jean Lee, David Schilling, Nyankor Matthews, Robert Swierczek,
John Krump, Monica Wolford, and Karen O'Conor made key contributions to
this report.
[End of section]
Appendix VII: Related GAO Products:
Improvements Needed to the Federal Procurement Data System-Next
Generation. GAO-05-960R. Washington, D.C.: September 27, 2005.
Social Security Administration: Additional Actions Needed in Ongoing
Efforts to Improve 800-Number Service. GAO-05-735. Washington, D.C.:
August 8, 2005.
Immigration Services: Better Contracting Practices Needed at Call
Centers. GAO-05-526. Washington, D.C.: June 30, 2005.
Federal Thrift Savings Plan: Customer Service Practices Adopted by
Private Sector Plan Managers Should be Considered. GAO-05-38.
Washington, D.C.: January 18, 2005.
Medicare: Accuracy of Responses from the 1-800-MEDICARE Help Line
Should Be Improved. GAO-05-130. Washington, D.C.: December 8, 2004.
Medicare: Call Centers Need to Improve Responses to Policy-Oriented
Questions from Providers. GAO-04-669. Washington, D.C.: July 16, 2004.
Reliability of Federal Procurement Data, GAO-04-295R. Washington, D.C.:
December 30, 2003.
Medicare: Communications with Physicians Can Be Improved. GAO-02-249.
Washington, D.C.: February 27, 2002.
IRS Telephone Assistance: Limited Progress and Missed Opportunities to
Analyze Performance in the 2001 Filing Season. GAO-02-212. Washington,
D.C.: December 7, 2001.
IRS Telephone Assistance: Quality of Service Mixed in the 2000 Filing
Season and below IRS' Long-Term Goal. GAO-01-189. Washington, D.C.:
April 6, 2001.
IRS Telephone Assistance: Opportunities to Improve Human Capital
Management. GAO-01-144. Washington, D.C.: January 30, 2001.
Customer Service: Human Capital Management at Selected Public and
Private Call Centers. GAO/GGD-00-161. Washington, D.C.: August 22,
2000.
Social Security Administration: Information on Monitoring 800 Number
Telephone Calls. GAO/HEHS-98-56R. Washington, D.C.: December 8, 1997.
Social Security Administration: More Cost-Effective Approaches Exist to
Further Improve 800-Number Service. GAO/HEHS-97-79. Washington, D.C.:
June 11, 1997.
FOOTNOTES
[1] Office of Management and Budget, Expanding E-Government: Partnering
for a Results-Oriented Government, (Washington, D.C.: Dec. 2004).
[2] Office of Management and Budget, Guidelines for Ensuring and
Maximizing the Quality, Objectivity, Utility, and Integrity of
Information Disseminated by Federal Agencies, Federal Register, Vol.
67, No. 36 (Feb. 22, 2002).
[3] GAO, Medicare: Accuracy of Responses from the 1-800-MEDICARE Help
Line Should Be Improved, GAO-05-130 (Washington, D.C.: Dec. 8, 2004);
GAO, Medicare: Call Centers Need to Improve Responses to Policy-
Oriented Questions from Providers, GAO-04-669 (Washington, D.C.: Jul.
16, 2004).
[4] One agency we selected for our review--the Centers for Disease
Control and Prevention--is in its first year of a 4-year consolidation
of workloads. The contact center is projected to receive an estimated
2.7 million inquiries annually when it is fully operational.
[5] E-Government Act of 2002, Public Law 107-347, Section 101 (December
17, 2002). The U.S. Postal Service considers itself exempt from this
act.
[6] Consolidated Appropriations Act 2001, Public Law 106-554, Section
515 (December 21, 2000).
[7] Federal Register, volume 67, number 36, pages 8452-8460, February
22, 2002.
[8] See appendix IV for a list of GAO reports concerning call and
contact centers.
[9] Office of Federal Procurement Policy Act of 1974, Public Law 93-400
(August 30,1974).
[10] GAO, Improvements Needed to the Federal Procurement Data System-
Next Generation, GAO-05-960R, (Washington, D.C.: Sept. 27, 2005).
[11] Federal Acquisition Regulation, at Sections 37.602-2 and 46.104
(July 2005).
[12] Some of the agencies we reviewed indicated that training is also a
key part of ensuring accuracy. However, since training was provided as
part of the contractors' quality control processes, we did not consider
it an oversight practice.
[13] GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999); Internal Control
Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.: August
2001).
[14] Office of Management and Budget, Expanding E-Government:
Partnering for a Results-Oriented Government, (Washington, D.C.:
December 2004).
[15] Office of Federal Procurement Policy, Performance-Based Concepts
for Telephone Call Center Contracting, (Washington, D.C.: 1997).
[16] Performance-based contracts specify the desired outcomes and allow
the contractors to determine how best to achieve those outcomes, rather
than prescribe the methods contractors should use. Performance-based
contracts can encourage contractors to be innovative and to find cost-
effective ways of delivering services.
[17] USPS does not report to FPDS.
[18] The NAICS code is used to classify the industry type for the
product or service being purchased. It replaced the Standard Industrial
Code.
[19] Despite the shortcomings of FPDS noted in previous GAO reports,
using FPDS did not adversely affect our work. GAO, Improvements Needed
to the Federal Procurement Data System-Next Generation, GAO-05-960R
(Washington, D.C.: Sept. 27, 2005), and Reliability of Federal
Procurement Data, GAO-04-295R (Washington, D.C.: Dec. 30, 2003).
[20] One agency we selected for our review--Centers for Disease Control
and Prevention--is in its first year of a 5-year consolidation of
workloads. The contact center is projected to receive an estimated 2.7
million inquiries annually when it is fully operational.
[21] DOD TMA is divided into three separate contract service areas--
North, South, and West.
[22] The contractors consider the locations of their call centers to be
proprietary information.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
