Department of Homeland Security
Better Planning and Oversight Needed to Improve Complex Service Acquisition Outcomes Gao ID: GAO-08-765T May 8, 2008The Department of Homeland Security (DHS) has relied on service acquisitions to meet its expansive mission. In fiscal year 2006, DHS spent $12.7 billion to procure services. To improve service acquisition outcomes, federal procurement policy establishes a preference for a performance-based approach, which focuses on developing measurable outcomes rather than prescribing how contractors should perform services. This testimony focuses on how contract outcomes are influenced by how well DHS components have defined and developed contract requirements and performance standards, as well as the need for improved assessment and oversight to ensure better acquisition outcomes. GAO's statement is based on its report being released today, which reviewed judgmentally selected contracts for eight major investments at three DHS components--the Coast Guard, Customs and Border Protection (CBP), and the Transportation Security Administration (TSA)-- totaling $1.53 billion in fiscal years 2005 and 2006; prior GAO and DHS Inspector General reviews; management documents and plans; and related data, including 138 additional contracts, primarily for basic services from the Coast Guard, CBP, TSA, and Immigration and Customs Enforcement.
Over the past several years, GAO has found that appropriate planning, structuring, and monitoring of agency service acquisitions, including those that are performance-based, can help minimize the risk of cost overruns, delayed delivery, and unacceptably quality. Several prior GAO and DHS Inspector General reviews of major DHS investments using a performance-based approach point to such shortcomings. While all of the contracts GAO reviewed at the Coast Guard, CBP, and TSA had outcome-oriented requirements, contracts for four of the eight investments did not have well-defined requirements, or a complete set of measurable performance standards, or both at the time of contract award or start of work. These service contracts experienced cost overruns, schedule delays, or did not otherwise meet performance expectations. In contrast, contracts for the other four investments had well-defined requirements linked to measurable performance standards and met the standards for contracts that had begun work. In managing its service acquisitions, including those that are performance-based, DHS has faced oversight challenges that have limited its visibility over service acquisitions and its ability to make informed acquisition management decisions. Notably, the department lacks reliable data on performance-based service acquisitions. About half of the 138 contracts identified by DHS as performance-based had none of the elements DHS requires for such contracts: a performance work statement, measurable performance standards, or a quality assurance surveillance plan. Such inaccurate data limit DHS's ability to perform management assessments of these acquisitions. In addition, the Chief Procurement Officer, who is responsible for departmentwide procurement oversight, has not conducted management assessments of performance-based service acquisitions. To help DHS improve outcomes for its service acquisitions, including those that are performance-based, GAO recommended that DHS routinely assess requirements for complex investments to ensure that they are well-defined, and develop consistently measurable performance standards linked to those requirements. GAO also recommended that DHS systematically evaluate the outcomes of major investments and relevant contracting methods and improve the quality of data to facilitate identifying and assessing the use of various contracting methods. DHS generally concurred with GAO's recommendations, noting some departmental initiatives to improve acquisition management.
GAO-08-765T, Department of Homeland Security: Better Planning and Oversight Needed to Improve Complex Service Acquisition Outcomes
This is the accessible text file for GAO report number GAO-08-765T
entitled 'Department Of Homeland Security: Better Planning and
Oversight Needed to Improve Complex Service Acquisition Outcomes' which
was released on May 8, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Committee on Homeland Security, House of Representatives:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 10:00 a.m. EDT:
Thursday, May 8, 2008:
Department Of Homeland Security:
Better Planning and Oversight Needed to Improve Complex Service
Acquisition Outcomes:
Statement of John P. Hutton, Director: Acquisition and Sourcing
Management:
GAO-08-765T:
GAO Highlights:
Highlights of GAO-08-765T, a testimony before the Committee on Homeland
Security, House of Representatives.
Why GAO Did This Study:
The Department of Homeland Security (DHS) has relied on service
acquisitions to meet its expansive mission. In fiscal year 2006, DHS
spent $12.7 billion to procure services. To improve service acquisition
outcomes, federal procurement policy establishes a preference for a
performance-based approach, which focuses on developing measurable
outcomes rather than prescribing how contractors should perform
services.
This testimony focuses on how contract outcomes are influenced by how
well DHS components have defined and developed contract requirements
and performance standards, as well as the need for improved assessment
and oversight to ensure better acquisition outcomes.
GAO‘s statement is based on its report being released today, which
reviewed judgmentally selected contracts for eight major investments at
three DHS components”the Coast Guard, Customs and Border Protection
(CBP), and the Transportation Security Administration (TSA)”totaling
$1.53 billion in fiscal years 2005 and 2006; prior GAO and DHS
Inspector General reviews; management documents and plans; and related
data, including 138 additional contracts, primarily for basic services
from the Coast Guard, CBP, TSA, and Immigration and Customs
Enforcement.
What GAO Found:
Over the past several years, GAO has found that appropriate planning,
structuring, and monitoring of agency service acquisitions, including
those that are performance-based, can help minimize the risk of cost
overruns, delayed delivery, and unacceptably quality. Several prior GAO
and DHS Inspector General reviews of major DHS investments using a
performance-based approach point to such shortcomings. While all of the
contracts GAO reviewed at the Coast Guard, CBP, and TSA had outcome-
oriented requirements, contracts for four of the eight investments did
not have well-defined requirements, or a complete set of measurable
performance standards, or both at the time of contract award or start
of work. These service contracts experienced cost overruns, schedule
delays, or did not otherwise meet performance expectations. In
contrast, contracts for the other four investments had well-defined
requirements linked to measurable performance standards and met the
standards for contracts that had begun work.
In managing its service acquisitions, including those that are
performance-based, DHS has faced oversight challenges that have limited
its visibility over service acquisitions and its ability to make
informed acquisition management decisions. Notably, the department
lacks reliable data on performance-based service acquisitions. About
half of the 138 contracts identified by DHS as performance-based had
none of the elements DHS requires for such contracts: a performance
work statement, measurable performance standards, or a quality
assurance surveillance plan. Such inaccurate data limit DHS‘s ability
to perform management assessments of these acquisitions. In addition,
the Chief Procurement Officer, who is responsible for departmentwide
procurement oversight, has not conducted management assessments of
performance-based service acquisitions.
To help DHS improve outcomes for its service acquisitions, including
those that are performance-based, GAO recommended that DHS routinely
assess requirements for complex investments to ensure that they are
well-defined, and develop consistently measurable performance standards
linked to those requirements. GAO also recommended that DHS
systematically evaluate the outcomes of major investments and relevant
contracting methods and improve the quality of data to facilitate
identifying and assessing the use of various contracting methods. DHS
generally concurred with GAO‘s recommendations, noting some
departmental initiatives to improve acquisition management.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-765T]. For more
information, contact John Hutton at (202) 512-4841 or huttonj@gao.gov.
[End of section]
Mr. Chairman and Members of the Committee:
Thank you for inviting me here today to discuss the Department of
Homeland Security's (DHS) planning and assessment of its complex
service acquisitions. To meet its expansive homeland security mission,
DHS spends billions of dollars on service acquisitions for critical
trade, transportation, and border security investments. In fact, more
than 80 percent of DHS's total procurement dollars are spent on
services. Prior GAO work has found that appropriate planning,
structuring, and monitoring of acquisitions is critical to ensuring
that the services provided meet the government's needs.[Footnote 1] To
help improve service acquisition outcomes, federal procurement policy
calls for agencies to use a performance-based approach to the maximum
extent practicable. This approach includes a performance work statement
that describes outcome-oriented requirements, measurable performance
standards, and quality assurance surveillance. If properly implemented,
these characteristics can help ensure that contracted services meet
cost, schedule, and performance requirements. Other factors, such as
pressure to get programs up and running, additional external
requirements, and technological challenges also impact the ability to
achieve good acquisition outcomes.
While a performance-based approach has been widely accepted, we have
found that agencies face certain challenges in implementing the
approach--especially for complex or major investments. My testimony
today will focus on the particular challenges DHS has confronted.
Specifically, I will discuss how contract outcomes are influenced by
how well DHS components have defined and developed contract
requirements and performance standards. I will also discuss the need
for improved assessment and oversight to ensure better outcomes.
My statement is based on our report that is being released today.
[Footnote 2] This report focused on contracts for major investments
with complex service acquisitions at the Coast Guard, Customs and
Border Protection (CBP), and the Transportation Security Administration
(TSA)--three of the DHS components reporting among the highest
obligations for performance-based service acquisitions in fiscal years
2005 and 2006. We conducted this performance audit in accordance with
generally accepted government auditing standards. Those standards
require that we plan and perform the audit to obtain sufficient,
appropriate evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Summary:
Over the past several years, we have found that if agency service
acquisitions, including those that are performance-based, are not
appropriately planned, structured, and monitored, there is an increased
risk that the government may receive products or services that are over
budget, delivered late, and of unacceptable quality. Several prior GAO
and DHS Inspector General reviews of major DHS investments using a
performance-based approach point to such shortcomings. For the report
we are releasing today, we reviewed contracts for eight major
investments at the Coast Guard, CBP, and TSA and found that all had
outcome-oriented requirements--as required in acquisition regulations
and policy. However, contracts for four of these investments did not
have well-defined requirements, or a complete set of measurable
performance standards, or both at the time of contract award or start
of work. These service contracts experienced cost overruns, schedule
delays, or did not otherwise meet performance expectations. In
contrast, service contracts for the other four investments had well-
defined requirements linked to measurable performance standards and
performed within budget meeting the standards in all cases where
contractors had begun work. DHS components conducted quality assurance
surveillance, and for the contracts that had negative outcomes,
surveillance helped to identify contractor performance weaknesses and
corrective action was taken.
In managing its service acquisitions, including those that are
performance-based, DHS has faced oversight challenges, including a lack
of reliable data and systematic management reviews. Although
contracting and program staff at DHS components told us that they used
a performance-based approach to the maximum extent practicable, the
department does not have reliable data to facilitate required reporting
or perform management assessments of these acquisitions. Our review of
an additional 138 contracts, which were primarily for basic services,
found that about half of the contracts coded by DHS as performance-
based had none of the three elements DHS requires: a performance work
statement, measurable performance standards, or a quality assurance
surveillance plan. Inaccurate data limit DHS's visibility over service
acquisitions and the department's ability to make informed acquisition
management decisions. The Chief Procurement Officer (CPO), who has
responsibility for departmentwide procurement oversight, has begun some
initial review of performance-based service acquisitions, but has not
conducted management assessments of this acquisition method.
Background:
Over the last decade, the use of federal service contracting has
increased and now accounts for over 60 percent of federal procurement
dollars spent annually. A performance-based approach to federal service
contracting was introduced during the 1990s, representing a shift from
specifying the way in which contractors should perform work to
specifying acquisition outcomes. Regardless of the contracting method,
focusing on outcomes and collaboration among multiple stakeholders in
the contracting process has been acknowledged as sound contract
management. In 2000, federal procurement law established a performance-
based approach as the preferred acquisition method for services.
[Footnote 3] The Federal Acquisition Regulation requires all
performance-based service acquisitions to include:
* a performance work statement that describes outcome-oriented
requirements in terms of results required rather than the methods of
performance of the work;
* measurable performance standards describing how to measure contractor
performance in terms of quality, timeliness, and quantity; and:
* the method of assessing contract performance against performance
standards, commonly accomplished through the use of a quality assurance
surveillance plan.[Footnote 4]
A 1998 Office of Federal Procurement Policy (OFPP) study on performance-
based contracts--based largely on contracts for basic services, such as
janitorial or maintenance services--showed that a number of anticipated
benefits had been achieved, including reduced acquisition costs,
increased competition for contracts, and improved contractor
performance.[Footnote 5] However, implementing a performance-based
approach is often more difficult for complex acquisitions, such as
information technology, than it is for basic services, because agencies
begin with requirements that are less stable, making it difficult to
establish measurable outcomes. Such complex acquisitions may need to
have requirements and performance standards continually refined
throughout the life-cycle of the acquisition for a contractor to
deliver a valuable service over an extended period of time. OFPP also
has noted in policy that certain types of services, such as research
and development, may not lend themselves to outcome-oriented
requirements.
To encourage agencies to apply a performance-based approach to service
acquisitions, the Office of Management and Budget (OMB) established
governmentwide performance targets, which increased to 50 percent of
eligible service contract dollars for the current fiscal year. In
January 2007, the congressionally mandated Acquisition Advisory Panel
reported that performance-based acquisition has not been fully
implemented in the federal government, despite OMB encouragement, and
recommended that OMB adjust the governmentwide target to reflect
individual agency assessments and plans.[Footnote 6] In May 2007, OMB's
OFPP issued a memo providing that agencies, at a minimum, were expected
to meet targets established and report on them in their management
plans. In response, DHS's CPO established a performance-based target of
25 percent for fiscal year 2007, increasing to 40 percent by fiscal
year 2010, that was included in DHS's Performance-Based Management
Plan. The Acquisition Advisory Panel also recommended that OFPP issue
more explicit implementation guidance and create an "Opportunity
Assessment" tool to help agencies identify when they should consider
using this acquisition method.
Requirements and Standards Definition Influence Program Outcomes:
Our work has found that performance-based acquisitions must be
appropriately planned and structured to minimize the risk of the
government receiving services that are over cost estimates, delivered
late, and of unacceptable quality.[Footnote 7] Specifically, we have
emphasized the importance of clearly defined requirements to achieving
desired results and measurable performance standards to ensuring
control and accountability. Prior GAO and DHS Inspector General reviews
of complex DHS investments using a performance-based approach point to
a number of shortcomings. For example, in June 2007, we reported that a
performance-based contract for a DHS financial management system,
eMerge2, lacked clear and complete requirements, which led to schedule
delays and unacceptable contractor performance.[Footnote 8] Ultimately,
the program was terminated after a $52 million investment. In March
2007, we similarly reported that the Coast Guard's performance-based
contract for replacing or modernizing its fleet of vessels and
aircraft, Deepwater, had requirements that were set at unrealistic
levels and were frequently changed.[Footnote 9] This resulted in cost
escalation, schedule delays, and reduced contractor accountability. The
DHS Inspector General has also indicated numerous opportunities for DHS
to make better use of sound practices, such as well-defined
requirements.[Footnote 10]
Consistent with our prior work, definition of requirements and
performance standards influenced outcomes for the eight complex
investments we reviewed. In using a performance-based approach, sound
contracting practices dictate that required contract outcomes or
requirements be well-defined, providing clear descriptions of results
to be achieved. While all eight contracts for these investments had
outcome-oriented requirements, the requirements were not always well-
defined.[Footnote 11] Further, contracts for half of the investments
did not have a complete set of measurable performance standards.
Appendix I provides a summary of our analysis of the requirements,
performance standards, and outcomes for the eight performance-based
contracts for major investments we reviewed.
Complex investments with contracts that did not have well-defined
requirements or complete measurable performance standards at the time
of contract award or start of work experienced either cost overruns,
schedule delays, or did not otherwise meet performance expectations.
For example, contracts for systems development for two CBP major
investments lacked both well-defined requirements and measurable
performance standards prior to the start of work and both experienced
poor outcomes. The first, for DHS's Automated Commercial Environment
(ACE) Task Order 23 project--a trade software modernization effort--was
originally estimated to cost $52.7 million over a period of
approximately 17 months.[Footnote 12] However, the program lacked
stable requirements at contract award and, therefore, could not
establish measurable performance standards and valid cost or schedule
baselines for assessing contractor performance. Software requirements
were added after contract award, contributing to a project cost
increase of approximately $21.1 million, or 40 percent, over the
original estimate. Because some portions of the work were delayed to
better define requirements, the project is not expected to be completed
until June 2009--about 26 months later than planned.
The second, Project 28 for systems development for CBP's Secure Border
Initiative (SBInet)--a project to help secure a section of the United
States-Mexico border using a surveillance system--did not meet expected
outcomes due to a lack of both well-defined requirements and measurable
performance standards. CBP awarded the Project 28 contract planned as
SBInet's proof of concept and the first increment of the fielded SBInet
system before the overall SBInet operational requirements and system
specifications were finalized. More than 3 months after Project 28 was
awarded, DHS's Inspector General reported that CBP had not properly
defined SBInet's operational requirements and needed to do so quickly
to avoid rework of the contractor's systems engineering. We found that
several performance standards were not clearly defined to isolate the
contractor's performance from that of CBP employees, making it
difficult to determine whether any problems were due to the
contractor's system design, CBP employees, or both. As a result, it was
not clear how CBP intended to measure compliance with the Project 28
standard for probability of detecting persons attempting to illegally
cross the border. Although it did not fully meet user needs and its
design will not be used as a basis for future SBInet development, DHS
fully accepted the project after an 8-month delay.[Footnote 13] In
addition, DHS officials have stated that much of the Project 28 system
will be replaced by new equipment and software.
Conversely, we found that contracts with well-defined requirements
linked to measurable performance standards delivered results within
budget and provided quality service. For example, contracted security
services at the San Francisco International Airport for TSA's Screening
Partnership Program had well-defined requirements, and all measurable
performance standards corresponded to contract requirements--an
improvement from our prior reviews of the program.[Footnote 14] The
requirements for gate, checkpoint, and baggage screening services
clearly stated that the contractor should use technology and staff to
prevent prohibited items from entering sterile areas of the airport and
should work to minimize customer complaints while addressing in a
timely manner any complaints received. The performance standards
assessed how often screeners could successfully detect test images of
prohibited items in checked baggage; the percentage of audited records
and inspected equipment, property, and materials that were well-kept,
operational, and recorded on maintenance logs; and whether all new
hires received the required training before assuming their screening
responsibilities. In terms of expected outcomes, the contractor
achieved a 2.2 percent cost underrun during the first 5 months of the
contract and exceeded most requirements.
Unreliable Data and Lack of Management Review Constrain Oversight:
In managing its service acquisitions, including those that are
performance-based, DHS has faced oversight challenges, including a lack
of reliable data and systematic management reviews. DHS contracting and
program representatives told us that they use a performance-based
approach to the maximum extent practicable. However, DHS does not have
reliable data--either from the Federal Procurement Data System-Next
Generation (FPDS-NG), the governmentwide database for procurement
spending, or at a departmentwide level--to systematically monitor or
evaluate or report on service acquisitions, including those that are
performance-based. Reliable data are essential to overseeing and
assessing the implementation of contracting approaches, acquisition
outcomes, and making informed management decisions. Moreover, the Chief
Procurement Officer (CPO), who has responsibility for departmentwide
procurement oversight, has begun some initial review of performance-
based service acquisitions, but has not conducted systematic management
assessments of this acquisition method.
Our analysis of information provided by contracting representatives at
the Coast Guard, CBP, Immigration and Customs Enforcement (ICE), and
TSA showed that about 51 percent of the 138 contracts we identified in
FPDS-NG as performance-based had none of the required performance-based
elements: a performance work statement, measurable performance
standards, and a method of assessing contractor performance against
performance standards. Only 42 of the 138 contracts, or 30 percent, had
all of the elements, and about 18 percent had some but not all of the
required performance-based acquisition elements (see table 1).
Table 1: Review of Performance-Based Elements on Selected Contracts:
Performance-based elements: All elements;
Coast Guard: 18; Customs and Border Protection: 3;
Immigration and Customs Enforcement: 0;
Transportation Security Administration: 21;
Total contracts: 42;
Percentage of total contracts: 30.4.
Performance-based elements: Some elements;
Coast Guard: 16; Customs and Border Protection: 0;
Immigration and Customs Enforcement: 5;
Transportation Security Administration: 4;
Total contracts: 25;
Percentage of total contracts: 18.1.
Performance-based elements: No elements;
Coast Guard: 20;
Customs and Border Protection: 5;
Immigration and Customs Enforcement: 34;
Transportation Security Administration: 12;
Total contracts: 71;
Percentage of total contracts: 51.5.
Performance-based elements: Total;
Coast Guard: 54;
Customs and Border Protection: 8;
Immigration and Customs Enforcement: 39;
Transportation Security Administration: 37;
Total contracts: 138;
Percentage of total contracts: 100.0.
Source: GAO analysis of DHS review of 138 contracts coded as
performance-based in FPDS-NG.
[End of table]
Lacking reliable FPDS-NG data, reports on the use of performance-based
contracts for eligible service obligations are likely inaccurate. Data
reported on the use of performance-based contracts by service types--
ranging from basic, such as janitorial and landscaping, to complex,
such as information technology or systems development--requested by
OFPP in July 2006--are also likely misleading. The Acquisition Advisory
Panel and DHS's CPO also have raised concerns regarding the accuracy of
the performance-based designation in FPDS-NG. The Acquisition Advisory
Panel's 2007 report noted from its review at 10 federal agencies that
42 percent of the performance-based contracts the panel reviewed had
been incorrectly coded.
Inaccurate federal procurement data is a long-standing governmentwide
concern. Our prior work and the work of the General Services
Administration's Inspector General have noted issues with the accuracy
and completeness of FPDS and FPDS-NG data.[Footnote 15] OMB has
stressed the importance of submitting timely and accurate procurement
data to FPDS-NG and issued memos on this topic in August 2004 and March
2007. Accurate FPDS-NG data could facilitate the CPO's departmentwide
oversight of service acquisitions, including those that are performance-
based.
At a departmentwide level, CPO representatives responsible for
procurement oversight indicated that they have not conducted systematic
assessments including costs, benefits, and other outcomes of a
performance-based approach. To improve the implementation of
performance-based acquisitions, CPO representatives established a work
group in May 2006 to leverage knowledge among DHS components. They also
noted that they are working with OFPP to develop a best practices guide
on measurable performance standards and to gather good examples of
performance-based contracts. In addition, the CPO has implemented a
departmentwide acquisition oversight program, which was designed with
the flexibility to address specific procurement issues, such as
performance-based service acquisitions, and is based on a series of
component-level reviews.[Footnote 16] Some initial review of
performance-based acquisitions has begun under this program, but
management assessment or evaluation of the outcomes of this acquisition
method has not been conducted.
Conclusion and Recommendations:
Consistent with federal procurement policy, DHS has emphasized a
performance-based approach to improve service acquisition outcomes.
However, in keeping with our prior findings, DHS's designation of a
service acquisition as performance-based was not as relevant as the
underlying contract conditions. Sound acquisition practices, such as
clearly defining requirements and establishing complementary measurable
performance standards, are hallmarks of successful service
acquisitions. In the cases we reviewed as well as in prior findings
where these key elements were lacking, DHS did not always achieve
successful acquisition outcomes. The report we are releasing today
recommends that the Secretary of Homeland Security take several actions
to increase DHS's ability to achieve improved outcomes for its service
acquisitions, including those that are performance-based. These actions
include routinely assessing requirements for complex investments to
ensure that they are well-defined and developing consistently
measurable standards linked to those requirements; systematically
evaluating outcomes of major investments and relevant contracting
methods; and improving the quality of FPDS-NG data to facilitate
identifying and assessing the use of various contracting methods. DHS
generally concurred with our recommendations, noting some departmental
initiatives under way to improve acquisition management. However, the
department's response did not address how the CPO's process and
organizational changes at the departmental level will impact component-
level management and assessment of complex acquisitions to improve
outcomes. Improving acquisition management has been an ongoing
challenge since the department was established and requires sustained
management attention.
Mr. Chairman, this concludes my prepared statement. I would be pleased
to respond to any questions that you or other members of the committee
may have at this time.
Contacts and Acknowledgments:
For further information about this statement, please contact John P.
Hutton at (202) 512-4841 or huttonj@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this statement. Staff making key contributions to this
statement were Amelia Shachoy, Assistant Director; Jeffrey Hartnett;
Sean Seales; Karen Sloan; and Don Springman.
[End of section]
Appendix I: Key Characteristics of Eight Performance-Based Service
Contracts:
Table: Key Characteristics of Eight Performance-Based Service
Contracts:
Major investment by component: Coast Guard: Response Boat Medium;
Service: Research, analysis, and financial and information management;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor submitted all required documentation on time; met
project management quality standards; and maintained electronic
archiving and restoration standards.
Major investment by component: Customs and Border Protection: Automated
Commercial Environment;
Service: Trade systems software development (task order 23);
Well-defined requirements[A]: contract did not meet the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Costs increased by 40 percent ($21.1 million). More than a
year behind schedule; unplanned software redesign.
Major investment by component: Customs and Border Protection: National
Prime Integration;
Service: Maintenance of equipment used at border crossings, airports,
and seaports;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Costs increased by 53 percent ($24 million). Maintenance wait
times were longer than planned.
Major investment by component: Customs and Border Protection: SBInet;
Service: Project 28 border surveillance systems development and
fielding;
Well-defined requirements[A]: contract did not meet the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: DHS rejected initial acceptance of Project 28. The project
was delayed 8 months with final acceptance in February 2008. DHS noted
that the contractor met the requirements, but the project did not fully
meet DHS's needs and the technology will not be replicated in future
SBInet development.
Major investment by component: Transportation Security Administration:
Electronic Baggage Screening Program;
Service: Maintenance for explosive trace detection machines;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor exceeded the performance standard for machine
downtime with a score 1 hour less than required and operated at cost
through the second quarter of fiscal year 2007.
Major investment by component: Transportation Security Administration:
Screening Partnership Program;
Service: Passenger screening services at one airport;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor exceeded most performance standards; for example:
threat detection performance and false alarm rates exceeded the quality
standards. Contractor had cost underrun of 2.2 percent ($677,000).
Major investment by component: Transportation Security Administration:
Secure Flight;
Service: Maintaining database used to screen airline passenger data;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Initial contractor planning reports were inadequate; system
experienced operational downtime; surveillance reports identified poor
contractor performance. Contractor generally met time frames and
delivered within budget.
Major investment by component: Transportation Security Administration:
Transportation Worker Identification Credential;
Service: Issuing identification credentials to maritime workers;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Outcomes not available at the time of our review.
Source: GAO analysis.
[A] Well-defined requirements should provide clear descriptions of
results to be achieved at the time of the award or start of work and
primary requirements should not change substantially following contract
award.
[B] The set of measurable performance standards for a contract enables
the government to assess all aspects of the contractor's work in terms
of quality, timeliness, and quantity. The contract's performance
standards are also linked to the requirements.
[End of table]
[End of section]
Footnotes:
[1] For example, GAO, Defense Acquisitions: Tailored Approach Needed to
Improve Service Acquisition Outcomes, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-20] (Washington, D.C.: Nov.
9, 2006).
[2] GAO, Department of Homeland Security: Better Planning and
Assessment Needed to Improve Outcomes for Complex Service Acquisitions,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-263]
(Washington, D.C.: Apr. 22, 2008).
[3] Floyd D. Spence National Defense Authorization Act for Fiscal Year
2001, Pub. L. No. 106-398 § 821(a) (2000) required that the Federal
Acquisition Regulation (FAR) be revised to establish a preference for
the use of a performance-based approach in the acquisition of services,
which was done in FAR 37.102(a), providing that performance-based
acquisition is the preferred method for acquiring services and
generally is to be used to the maximum extent practicable. In addition,
this act established a preference for using firm fixed-price contracts
or task orders--where a specified price is paid regardless of the
contractor's incurred costs--when using a performance-based approach
for service acquisitions.
[4] FAR 37.601; FAR 37.602(b); FAR 37.604. A fourth element,
performance incentives, is required where appropriate.
[5] Office of Management and Budget, Office of Federal Procurement
Policy, A Report on the Performance-Based Service Contracting Pilot
Project, May 1998.
[6] Report of the Acquisition Advisory Panel to the Office of Federal
Procurement Policy and the United States Congress, January 2007.
[7] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20].
[8] GAO, Homeland Security: Departmentwide Integrated Financial
Management Systems Remain a Challenge, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-536] (Washington, D.C.:
June 21, 2007).
[9] GAO, Coast Guard: Status of Efforts to Improve Deepwater Program
Management and Address Operational Challenges, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-575T] (Washington
D.C.: Mar. 8, 2007).
[10] See for example, Department of Homeland Security Inspector
General, Major Management Challenges Facing the Department of Homeland
Security, OIG-08-11 (Jan. 2008), and Department of Homeland Security
Inspector General, Transportation Security Administration's Information
Technology Managed Services Contract, OIG-06-23 (Feb. 2006).
[11] FAR 2.101 specifically provides that a performance work statement
for performance-based acquisitions describe the required results in
clear, specific, and objective terms with measurable outcomes.
[12] Begun in 2001, ACE is intended to replace and supplement existing
cargo processing technology and will be developed and deployed in a
series of increments. The goals of ACE include (1) supporting border
security by enhancing analysis and information sharing with other
government agencies and providing CBP with the means to decide before a
shipment reaches the border if it should be targeted or expedited and
(2) streamlining time-consuming and labor-intensive tasks for CBP
personnel and the trade community through a national trade account and
single Web-based interface. Task Order 23 was the sole focus of our
review.
[13] GAO, Secure Border Initiative: Observations on the Importance of
Applying Lessons Learned to Future Projects, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T] (Washington, D.C.:
Feb. 27, 2008).
[14] GAO, Aviation Security: Preliminary Observations on TSA's Progress
to Allow Airports to Use Private Passenger and Baggage Screening
Services, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-126]
(Washington, D.C.: Nov. 19, 2004), and Aviation Security: Progress
Made to Set Up Program Using Private-Sector Airport Screeners, but
More Work Remains, GAO-06-166 (Washington, D.C.: Mar. 31, 2006).
[15] For example, GAO, Reliability of Federal Procurement Data,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-295R]
(Washington, D.C.: Dec. 30, 2003); GAO, Improvements Needed to the
Federal Procurement Data System-Next Generation, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-05-960R] (Washington, D.C.:
Sept. 27, 2005); and General Services Administration Inspector
General, Review of the Federal Procurement Data System-Next
Generation (FPDS-NG), Report Number A040127/O/T/F06016 (March 2006).
[16] GAO, Department of Homeland Security: Progress and Challenges in
Implementing the Department's Acquisition Oversight Plan, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-900[ (Washington, D.C.: June
2007).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
