Department of Homeland Security
Better Planning and Assessment Needed to Improve Outcomes for Complex Service Acquisitions Gao ID: GAO-08-263 April 22, 2008The Department of Homeland Security (DHS) has relied on service acquisitions to meet its expansive mission. In fiscal year 2006, DHS spent $12.7 billion to procure services. To improve service acquisition outcomes, federal procurement law establishes a preference for a performance-based approach, which focuses on developing measurable outcomes rather than prescribing how contractors should perform services. GAO was asked to (1) evaluate the implementation of a performance-based approach in the context of service acquisitions for major, complex investments, and (2) identify management challenges that may affect DHS's successful acquisitions for major investments, including those using a performance-based approach. GAO reviewed judgmentally selected contracts for eight major investments at three DHS components totaling $1.53 billion in fiscal years 2005 and 2006; prior GAO and DHS Inspector General reviews; management documents and plans; and related data, including 138 additional contracts for basic services.
All service contracts for the eight major, complex investments GAO reviewed had outcome-oriented requirements; however, four of these contracts did not have well-defined requirements, a complete set of measurable performance standards, or both. These service contracts experienced cost overruns, schedule delays, or did not otherwise meet performance expectations. In contrast, service contracts for the other four investments GAO reviewed had well-defined requirements linked to measurable performance standards. Contractors had begun work on three of these four contracts and performed within budget meeting the standards. This finding is consistent with prior GAO work on service acquisitions, which has highlighted the criticality of sound acquisition planning to develop well-defined requirements and measurable performance standards to achieving desired outcomes. In the four cases that had negative outcomes, program officials identified the contractor performance weaknesses through quality assurance surveillance and took corrective actions. Prior GAO work has found that if acquisitions, including those that are performance-based, are not appropriately planned, structured, and monitored, there is an increased risk that the government may receive products or services that are over budget, delivered late, and of unacceptable quality. In managing its service acquisitions, including those that are performance based, DHS has faced workforce and oversight challenges. Prior GAO work has highlighted the importance of having the right people with the right skills to achieve successful acquisition outcomes. Contracts for two major investments with negative cost and schedule outcomes did not have the staff needed to adequately plan and execute the contracts. Further, while representatives for several of the contracts GAO reviewed indicated that contracting and program staff worked well together, some senior acquisition representatives at the component level indicated that a lack of collaboration between these key stakeholders has been a challenge when developing and managing complex service acquisitions. In terms of oversight, component contracting and program officials said they used a performance-based approach to the maximum extent practicable; however, DHS does not have reliable data to facilitate required reporting, informed decisions, and analyzing acquisition outcomes. GAO's review also found that about half of an additional 138 contracts for basic services identified as performance-based did not have any of the elements intended to foster good outcomes: a performance work statement, measurable performance standards, and a quality assurance surveillance plan. DHS's Chief Procurement Officer (CPO)--who is responsible for departmentwide oversight of acquisitions--has several efforts under way to address some of these workforce and oversight issues. One initiative is an acquisition oversight program that is intended to assess (1) compliance with federal acquisition guidance, (2) contract administration, and (3) business judgment. However, this oversight program has not yet included an evaluation of the outcomes of contracting methods such as performance-based service acquisition.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-263, Department of Homeland Security: Better Planning and Assessment Needed to Improve Outcomes for Complex Service Acquisitions
This is the accessible text file for GAO report number GAO-08-263
entitled 'Department of Homeland Security: Better Planning and
Assessment Needed to Improve Outcomes for Complex Service Acquisitions'
which was released on May 8, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
April 2008:
Department of homeland security:
Better Planning and Assessment Needed to Improve Outcomes for Complex
Service Acquisitions:
GAO-08-263:
GAO Highlights:
Highlights of GAO-08-263, a report to congressional requesters.
Why GAO Did This Study:
The Department of Homeland Security (DHS) has relied on service
acquisitions to meet its expansive mission. In fiscal year 2006, DHS
spent $12.7 billion to procure services. To improve service acquisition
outcomes, federal procurement law establishes a preference for a
performance-based approach, which focuses on developing measurable
outcomes rather than prescribing how contractors should perform
services.
GAO was asked to (1) evaluate the implementation of a performance-based
approach in the context of service acquisitions for major, complex
investments, and (2) identify management challenges that may affect
DHS‘s successful acquisitions for major investments, including those
using a performance-based approach.
GAO reviewed judgmentally selected contracts for eight major
investments at three DHS components totaling $1.53 billion in fiscal
years 2005 and 2006; prior GAO and DHS Inspector General reviews;
management documents and plans; and related data, including 138
additional contracts for basic services.
What GAO Found:
All service contracts for the eight major, complex investments GAO
reviewed had outcome-oriented requirements; however, four of these
contracts did not have well-defined requirements, a complete set of
measurable performance standards, or both. These service contracts
experienced cost overruns, schedule delays, or did not otherwise meet
performance expectations. In contrast, service contracts for the other
four investments GAO reviewed had well-defined requirements linked to
measurable performance standards. Contractors had begun work on three
of these four contracts and performed within budget meeting the
standards. This finding is consistent with prior GAO work on service
acquisitions, which has highlighted the criticality of sound
acquisition planning to develop well-defined requirements and
measurable performance standards to achieving desired outcomes. In the
four cases that had negative outcomes, program officials identified the
contractor performance weaknesses through quality assurance
surveillance and took corrective actions. Prior GAO work has found that
if acquisitions, including those that are performance-based, are not
appropriately planned, structured, and monitored, there is an increased
risk that the government may receive products or services that are over
budget, delivered late, and of unacceptable quality.
In managing its service acquisitions, including those that are
performance based, DHS has faced workforce and oversight challenges.
Prior GAO work has highlighted the importance of having the right
people with the right skills to achieve successful acquisition
outcomes. Contracts for two major investments with negative cost and
schedule outcomes did not have the staff needed to adequately plan and
execute the contracts. Further, while representatives for several of
the contracts GAO reviewed indicated that contracting and program staff
worked well together, some senior acquisition representatives at the
component level indicated that a lack of collaboration between these
key stakeholders has been a challenge when developing and managing
complex service acquisitions. In terms of oversight, component
contracting and program officials said they used a performance-based
approach to the maximum extent practicable; however, DHS does not have
reliable data to facilitate required reporting, informed decisions, and
analyzing acquisition outcomes. GAO‘s review also found that about half
of an additional 138 contracts for basic services identified as
performance-based did not have any of the elements intended to foster
good outcomes: a performance work statement, measurable performance
standards, and a quality assurance surveillance plan. DHS‘s Chief
Procurement Officer (CPO)”who is responsible for departmentwide
oversight of acquisitions”has several efforts under way to address some
of these workforce and oversight issues. One initiative is an
acquisition oversight program that is intended to assess (1) compliance
with federal acquisition guidance, (2) contract administration, and (3)
business judgment. However, this oversight program has not yet included
an evaluation of the outcomes of contracting methods such as
performance-based service acquisition.
What GAO Recommends:
DHS generally concurred with GAO‘s recommendations that DHS develop
measurable standards consistently linked to well-defined requirements,
evaluate acquisition outcomes for major investments, and improve data
quality to help identify and assess contracting methods and outcomes.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-263]. For more
information, contact John Hutton (202) 512-4841 or huttonj@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Definition of Requirements and Performance Standards Influenced
Contract Outcomes:
DHS Has Faced Workforce and Oversight Challenges in Managing Its
Service Acquisitions:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Homeland Security:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Key Characteristics of Eight Performance-Based Service
Contracts:
Table 2: Less Than Expected Performance Identified through Surveillance
and Action Taken:
Table 3: Review of Performance-Based Elements on Selected Contracts:
Figure:
Figure 1: Percent of Contracts and Obligations by Contract Type for 42
DHS Performance-Based Service Acquisitions (Fiscal Years 2005 and
2006):
Abbreviations:
ACE: Automated Commercial Environment:
CBP: Customs and Border Protection:
CPO: Chief Procurement Officer:
DHS: Department of Homeland Security:
FAR: Federal Acquisition Regulation:
FEMA: Federal Emergency Management Agency:
FPDS-NG: Federal Procurement Data System-Next Generation:
ICE: Immigration and Customs Enforcement:
NPI: National Prime Integration:
OFPP: Office of Federal Procurement Policy:
SBInet: Secure Border Initiative Network:
TSA: Transportation Security Administration:
TWIC: Transportation Worker Identification Credential:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 22, 2008:
The Honorable Joseph I. Lieberman:
Chairman:
The Honorable Susan M. Collins:
Ranking Member:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Daniel K. Akaka:
Chairman:
The Honorable George V. Voinovich:
Ranking Member:
Subcommittee on Oversight of Government Management, the Federal
Workforce and the District of Columbia:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Bennie G. Thompson:
Chairman:
Committee on Homeland Security:
House of Representatives:
In fiscal year 2006, the Department of Homeland Security (DHS) spent
$12.7 billion on services, representing more than 80 percent of its
total procurement spending for that year. DHS has relied on service
acquisitions to meet its expansive homeland security mission. However,
prior GAO work has found that if acquisitions are not appropriately
planned, structured, and monitored, there is an increased risk that the
services provided will not fulfill intended acquisition outcomes or,
ultimately, meet government needs.[Footnote 1]
To help improve service acquisition outcomes, federal procurement
policy calls for agencies to use a performance-based approach to the
maximum extent practicable. This approach includes: a performance work
statement that describes outcome-oriented requirements, measurable
performance standards, and quality assurance surveillance. These
characteristics, if properly implemented, can help ensure that the
services meet cost, schedule, and performance requirements. Since this
concept was introduced in the 1990s, it has become widely accepted as a
sound acquisition approach.
Despite its governmentwide acceptance, concerns have been raised about
how well agencies are using a performance-based approach, particularly
for complex or major investments. Given the significant amount of DHS
spending on services and its critical mission, you asked us to (1)
evaluate the implementation of a performance-based approach in the
context of service acquisitions for major, complex investments; and (2)
identify management challenges that may affect DHS's successful
implementation of service acquisitions for major investments, including
those using a performance-based approach.
To conduct our work, we focused on the Coast Guard, Customs and Border
Protection (CBP), Immigrations and Custom Enforcement (ICE), and the
Transportation Security Administration (TSA)--four of the five DHS
components reporting the highest obligations for performance-based
service acquisitions in fiscal years 2005 and 2006.[Footnote 2] To
evaluate DHS's implementation of a performance-based approach in the
context of service acquisitions for major investments,[Footnote 3] we
judgmentally selected eight major investments at the Coast Guard, CBP,
and TSA[Footnote 4] based on size and complexity, and reviewed actions
for one performance-based service contract associated with each
investment.[Footnote 5] These contracts had a combined estimated value
of $1.53 billion and included services provided in the acquisition and
sustainment phases of the investments.[Footnote 6] We assessed these
contracts against acquisition regulations and policy and interviewed
contracting and program management staff. Performance-based service
acquisitions were difficult to identify and were not clearly distinct
from other service acquisitions because they did not always include all
required elements. Therefore, we reviewed prior GAO reports and DHS
Inspector General reports to provide a broader context on DHS's major
service acquisitions.
To identify management challenges that may affect DHS's successful
implementation of service acquisitions for major investments, we
analyzed management documents and plans; staffing data; and oversight
mechanisms, including the Federal Procurement Data System-Next
Generation (FPDS-NG), the governmentwide database for procurement
spending. From FPDS-NG, we judgmentally selected 138 contracts for
primarily basic services coded as performance-based and awarded in
fiscal years 2005 or 2006 at Coast Guard, CBP, ICE, and TSA with
obligations greater than $1 million dollars and asked the components to
verify whether these contracts included the required performance-based
elements. Finally, we interviewed contracting and program management
staff and DHS Chief Procurement Office (CPO) representatives and met
with representatives from the Office of Management and Budget's Office
of Federal Procurement Policy (OFPP). For more information on our scope
and methodology, see appendix I.
We conducted this performance audit from January 2007 to April 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Results in Brief:
All service contracts for the eight major, complex investments we
reviewed had outcome-oriented requirements, but the requirements were
not always well-defined. Four of the eight contracts we reviewed did
not have well-defined requirements, or a complete set of measurable
performance standards, or both at the time of contract award or start
of work. These service contracts experienced cost overruns, schedule
delays, or did not otherwise meet performance expectations. In
contrast, service contracts for the other four investments had well-
defined requirements linked to measurable performance standards.
Contractors had begun work on three of these four contracts and
performed within budget meeting the standards. This finding is
consistent with our prior work on service contracting, which has
highlighted the criticality of sound acquisition planning to develop
stable, well-defined requirements and measurable performance standards
to achieving desired outcomes. Components conducted quality assurance
surveillance for the contracts we reviewed. For the four contracts that
had negative outcomes, surveillance helped DHS components to identify
contractor performance weaknesses and corrective actions were taken.
Prior GAO work has found that if acquisitions, including those that are
performance-based, are not appropriately planned, structured, and
monitored, there is an increased risk that the government may receive
products or services that are over budget, delivered late, and of
unacceptable quality.
In managing its service acquisitions, including those that are
performance-based, DHS has faced workforce and oversight challenges.
Our prior work has highlighted the importance of having the right
people with the right skills to achieve successful acquisition
outcomes. Contracts for two of the eight major investments we reviewed
that had negative cost or schedule outcomes did not have the staff
needed to adequately plan and execute the contracts. Further, while
representatives for several of the contracts we studied indicated
contracting and program staff worked together effectively, at the
component level, senior acquisition representatives indicated that a
lack of collaboration between the program and contracting offices in
general has been challenging. In terms of oversight, DHS component
contracting and program staff told us they used a performance-based
approach to the maximum extent practicable; however, the department
does not have reliable data to facilitate reporting or perform
management assessments of these acquisitions. Moreover, while the CPO
has implemented a departmentwide acquisition oversight program, the CPO
has not performed management assessments of performance-based service
acquisitions. Our review of an additional 138 contracts for primarily
basic services found that about half of the contracts identified as
performance-based in FPDS-NG did not have any of the three elements: a
performance work statement, measurable performance standards, and a
quality assurance surveillance plan, which are intended to foster good
outcomes. Inaccurate data limit DHS's visibility over service
acquisitions and the department's ability to make informed acquisition
management decisions. DHS's CPO has several efforts under way to
address departmentwide workforce and oversight challenges. One
initiative is an acquisition oversight program that is intended to
assess (1) compliance with federal acquisition guidance, (2) contract
administration, and (3) business judgment. However, this oversight
program has not yet included an evaluation of the outcomes of
contracting methods such as performance-based service acquisition.
To increase DHS's ability to improve outcomes for its service
acquisitions, including those that are performance-based, we are
recommending that the Secretary of Homeland Security take several
actions. These actions include improving acquisition planning for
requirements for major, complex investments to ensure they are well-
defined, and developing consistently measurable performance standards
linked to those requirements; systematically evaluating the outcomes of
major investments and relevant contracting methods at a departmentwide
level; and improving the quality of FPDS-NG data so that DHS can more
accurately identify and assess the use and outcomes of various
contracting methods. In written comments on a draft of this report, DHS
generally concurred with our recommendations and provided some
information on efforts under way to improve acquisition management. The
department's comments are reprinted in appendix II. OMB did not comment
on the findings or conclusions of this report. DHS and OMB each
provided technical comments, which we incorporated as appropriate and
where supporting documentation was provided.
Background:
Over the last decade, the use of federal service contracting has
increased and now accounts for over 60 percent of federal procurement
dollars spent annually. Prior GAO work has found that if acquisitions
are not appropriately planned, structured, and monitored, there is an
increased risk that the services provided will not fulfill intended
acquisition outcomes or, ultimately, meet agency needs.
A performance-based approach, if successfully implemented, can help
improve service acquisition outcomes. This concept--introduced during
the 1990s--represented a shift in emphasis from specifying methods in
which contracted work should be performed to specifying acquisition
outcomes. In 2000, federal procurement law established a performance-
based approach as the preferred acquisition method for services.
[Footnote 7] The FAR requires all performance-based service
acquisitions to include:
* a performance work statement that describes outcome-oriented
requirements in terms of results required rather than the methods of
performance of the work;
* measurable performance standards describing how to measure contractor
performance in terms of quality, timeliness, and quantity; and:
* the method of assessing contract performance against performance
standards, commonly accomplished through the use of a quality assurance
surveillance plan.[Footnote 8]
In addition, federal procurement law established a preference for using
firm fixed-price contracts or task orders--where a specified price is
paid regardless of the contractor's incurred costs--when using a
performance-based approach for service acquisitions.[Footnote 9] A
performance-based acquisition approach also calls for greater
collaboration among procurement and program representatives throughout
the contract period and often requires more time for acquisition
planning and management than other contracting methods. Agencies may
also involve the contractor in the multi-functional team responsible
for developing performance-based acquisition planning documents to
leverage the contractor's technical expertise. Including stakeholders
with varied knowledge and skills enables acquisition teams to determine
at the earliest point possible that all aspects of the acquisition are
necessary, executable, and tailored to the transaction's risk level.
Regardless of the contracting method, focusing on outcomes and
collaboration among multiple stakeholders in the contracting process
has been acknowledged as sound contract management. Other factors, such
as pressure to get programs up and running, additional external
requirements, and technological challenges also impact the ability to
achieve good acquisition outcomes.
A 1998 OFPP study on performance-based contracts--based largely on
contracts for basic services, such as janitorial or maintenance
services--showed that a number of anticipated benefits had been
achieved: reduced acquisition costs, increased competition for
contracts, and improved contractor performance.[Footnote 10] However,
our prior work and the work of others has found that implementing a
performance-based approach is often more difficult for complex
acquisitions because agencies begin with requirements that are less
stable, making it difficult to establish measurable outcomes. OFPP has
noted in policy that certain types of services, such as research and
development, may not lend themselves to outcome-oriented
requirements.[Footnote 11] CPO representatives also have noted that
defining outcome-oriented requirements and measurable performance
standards may be more challenging for certain types of services, such
as research and development or professional and management support
services. Further, complex service contracts, such as those for
information technology, may need to have requirements and performance
standards continually refined throughout the life-cycle of the
acquisition for a contractor to deliver a valuable service over an
extended period of time.
To encourage agencies to apply a performance-based approach to service
acquisitions, the Office of Management and Budget established
governmentwide performance targets: 20 percent of eligible service
contract dollars for fiscal year 2002, 40 percent for fiscal year 2005
and fiscal year 2006, 45 percent for fiscal year 2007, and 50 percent
for fiscal year 2008.
In 2007, the congressionally mandated Acquisition Advisory Panel
[Footnote 12] reported on the implementation of a performance- based
approach and concluded that agencies were not clearly defining
requirements, not identifying meaningful quality measures, not
effectively managing the contract, and had limited data to make
informed decisions. The panel made recommendations to improve
application of the performance-based approach in three areas: goal
setting and management, guidance for acquisition staff, and data
quality.[Footnote 13] For example, the panel recommended that the
Office of Management and Budget adjust the governmentwide target to
reflect individual agency assessments and plans.
In May 2007, OFPP issued a memo directing agencies, at a minimum, to
meet targets established and report on them in their management plans.
In response, DHS's CPO, who is responsible for creating departmentwide
policies and processes for managing and overseeing the acquisition
function, established a performance-based target of 25 percent for
fiscal year 2007, increasing to 40 percent by fiscal year 2010, that
was included in DHS's Performance-Based Management Plan. Meeting these
targets is a responsibility shared between the CPO and the component
heads, in accordance with DHS' system of dual accountability for
managing the acquisition function.[Footnote 14]
Definition of Requirements and Performance Standards Influenced
Contract Outcomes:
Requirements definition and performance standards influenced outcomes
for performance-based contracts associated with eight major, complex
investments. While all eight contracts for the investments we reviewed
had outcome-oriented requirements, the requirements were not always
well-defined.[Footnote 15] Further, contracts for half of the
investments did not have a complete set of measurable performance
standards. Major investments with contracts that did not have well-
defined requirements or complete measurable performance standards at
the time of contract award or start of work experienced either cost
overruns, schedule delays, or did not otherwise meet performance
expectations. Conversely, contracts with well-defined requirements
linked to measurable performance standards delivered results within
budget and provided quality service. This finding is consistent with
our body of work on service contracting, which has emphasized the
importance of sound acquisition planning to develop well-defined
requirements and measurable performance standards to achieve desired
results. Table 1 summarizes our analysis of the requirements,
performance standards, and outcomes for the eight performance-based
contracts we reviewed. Component program staff conducted surveillance
for these contracts, and in the four cases that had negative outcomes,
identified less than expected performance and took action.
Table 1: Key Characteristics of Eight Performance-Based Service
Contracts:
Major investment by component: Coast Guard: Response Boat Medium;
Service: Research, analysis, and financial and information management;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor submitted all required documentation on time; met
project management quality standards; and maintained electronic
archiving and restoration standards.
Major investment by component: Customs and Border Protection: Automated
Commercial Environment;
Service: Trade systems software development (task order 23);
Well-defined requirements[A]: contract did not meet the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Costs increased by 40 percent ($21.1 million). More than a
year behind schedule; unplanned software redesign.
Major investment by component: Customs and Border Protection: National
Prime Integration;
Service: Maintenance of equipment used at border crossings, airports,
and seaports;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Costs increased by 53 percent ($24 million). Maintenance wait
times were longer than planned.
Major investment by component: Customs and Border Protection: SBInet;
Service: Project 28 border surveillance systems development and
fielding;
Well-defined requirements[A]: contract did not meet the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: DHS rejected initial acceptance of Project 28. The project
was delayed 8 months with final acceptance in February 2008. DHS noted
that the contractor met the requirements, but the project did not fully
meet DHS's needs and the technology will not be replicated in future
SBInet development.
Major investment by component: Transportation Security Administration:
Electronic Baggage Screening Program;
Service: Maintenance for explosive trace detection machines;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor exceeded the performance standard for machine
downtime with a score 1 hour less than required and operated at cost
through the second quarter of fiscal year 2007.
Major investment by component: Transportation Security Administration:
Screening Partnership Program;
Service: Passenger screening services at one airport;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Contractor exceeded most performance standards; for example:
threat detection performance and false alarm rates exceeded the quality
standards. Contractor had cost underrun of 2.2 percent ($677,000).
Major investment by component: Transportation Security Administration:
Secure Flight;
Service: Maintaining database used to screen airline passenger data;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract partially met the
criteria;
Outcomes: Initial contractor planning reports were inadequate; system
experienced operational downtime; surveillance reports identified poor
contractor performance. Contractor generally met time frames and
delivered within budget.
Major investment by component: Transportation Security Administration:
Transportation Worker Identification Credential;
Service: Issuing identification credentials to maritime workers;
Well-defined requirements[A]: contract met or mostly met the criteria;
Measurable performance standards[B]: contract met or mostly met the
criteria;
Outcomes: Outcomes not available at the time of our review.
Source: GAO analysis.
[A] Well-defined requirements should provide clear descriptions of
results to be achieved at the time of the award or start of work and
primary requirements should not change substantially following contract
award.
[B] The set of measurable performance standards for a contract enables
the government to assess all aspects of the contractor's work in terms
of quality, timeliness, and quantity. The contract's performance
standards are also linked to the requirements.
[End of table]
Contracts That Lacked Stable, Well-Defined Requirements, Complete
Measurable Performance Standards, or Both Experienced Negative
Outcomes:
Although all contracts we reviewed for the eight major, complex
investments had outcome-oriented requirements, contracts for four
investments--two for systems development and two for operations and
maintenance--did not have stable and well-defined requirements, a full
complement of measurable performance standards, or both. These four
contracts experienced schedule delays, cost increases, or other
negative outcomes. Prior GAO and DHS Inspector General work on other
DHS major investments have similarly found that when requirements were
not well-defined, performance did not meet expectations. Our work has
found that performance-based acquisitions must be appropriately planned
and structured to minimize the risk of the government receiving
services that are over cost estimates, delivered late, and of
unacceptable quality.[Footnote 16] Specifically, we have emphasized the
importance of clearly defined requirements to achieving desired results
and measurable performance standards to ensuring control and
accountability.[Footnote 17]
Systems Development Efforts:
Two CBP systems development contracts lacked both well-defined
requirements and measurable performance standards prior to the start of
work. The first--ACE Task Order 23 project, a trade software
modernization effort under a letter contract[Footnote 18]--was
originally estimated to cost $52.7 million over a period of
approximately 17 months. DHS contracting representatives told us that
some technical requirements were not fully defined within the original
scope of work, and some new software requirements were added after
contract award. Because many requirements were not fully defined in
advance of the contract's award, the program was unable to establish
clear, measurable performance standards and valid cost or schedule
baselines for assessing contractor performance. The need to redefine
contract requirements for the ACE program contributed to significant
schedule delays and cost increases.[Footnote 19] ACE Task Order 23 was
completed at the end of October 2007 with a project cost increase of
approximately $21.1 million, or 40 percent, over the original estimate.
As of April 2008, DHS reported that some portions of the work were
delayed to better define requirements, and the final portion of the
software was projected to be completed in June 2009, about 26 months
later than planned.
The second, CBP's Secure Border Initiative Network (SBInet) Project 28
contract, was intended to help secure a section of the United States-
Mexico border using a surveillance system. Project 28 was planned to be
SBInet's proof of concept, as well as the first increment of the
fielded SBInet system. CBP awarded the Project 28 contract before the
overall SBInet operational requirements and system specifications were
finalized. Additional design requirements for equipment installation
were added after contract award. DHS's Inspector General reported in
February 2007--more than 3 months after the Project 28 contract was
awarded--that CBP had not properly defined SBInet's operational
requirements and needed to do so quickly to avoid rework of the
contractor's systems engineering. Several performance standards were
not clearly defined to isolate the contractor's performance from that
of CBP employees, making it difficult to determine whether any problems
were due to the contractor's system design, CBP employees, or both.
Other performance standards were difficult to measure because CBP did
not plan a controlled operational test, which is a test conducted in a
separate, instrumented test area that resembles the border environment.
CBP planned to conduct operational testing, which is less rigorous,
after the system was fully accepted. As a result, it was not clear how
CBP intended to measure compliance with the Project 28 standard for
probability of detecting persons attempting to illegally cross the
border. Another standard stated that the acceptable level of contractor
performance would be determined in the future. Project 28, planned as
an 8-month, $20 million firm fixed-price effort,[Footnote 20] did not
meet the June 2007 delivery date, and in August 2007 CBP notified the
contractor that it did not approve delivery because the system failed
the initial acceptance test. DHS conditionally accepted the system in
December 2007 and fully accepted it in February 2008. DHS noted that
the contractor met the requirements; however, Project 28 resulted in a
product that did not fully meet user needs and its design will not be
used as a basis for future SBInet development.[Footnote 21] In
addition, DHS officials have stated that much of the Project 28 system
will be replaced by new equipment and software.
Operations and Maintenance Efforts:
Performance-based contracts for two other major investments--CBP's
National Prime Integration (NPI) contract and a TSA Secure Flight
contract--included well-defined requirements but did not have a full
complement of measurable performance standards. These operations and
maintenance contracts had less than expected contractor performance or
significant cost increases.
CBP's NPI contract for maintenance of inspection, detection, and
surveillance devices at border crossings, airports, and seaports
included well-defined requirements for conducting equipment deployment,
maintenance, and replacement as well as engineering and logistics
support. For example, the requirements for equipment corrective
maintenance specify the contractor shall conduct maintenance, including
parts and labor, as a result of equipment or system failure. However,
the contract did not incorporate performance standards for post-
maintenance inspections and other standards needed to fully assess the
quality of equipment maintenance performed by the contractor. CBP
representatives acknowledged the need for additional measurable
performance standards addressing quality of equipment maintenance to
help ensure good contractor performance. While CBP does not plan to
incorporate additional performance measures until September 2008, CBP
revised the contract documentation in December 2007 to emphasize
program management. Regarding contract outcomes, during the last year
of the contract, costs were about $24 million, or 53 percent, higher
than original estimates, in part because of increases in the quantity
of equipment being supported. Customer wait times have been much longer
than expected for equipment maintenance, and wait time standards have
been revised to between 24 and 96 hours depending on the type of
equipment. Average wait times from September 2006 to September 2007
were between 118 and 106 hours, not meeting the revised standard.
A TSA Secure Flight contract for operations and maintenance of the
federal terrorist watchlist database, which is used to screen airline
passenger data, also experienced less than expected outcomes. The
contract we reviewed contained well-defined requirements and measurable
performance standards for operation, maintenance, repair, and upgrade
of the system. For example, the requirements specify that the
contractor shall provide on-site technical and management support.
Although the contract had some measurable performance standards, the
contract did not have the full complement of standards needed to
monitor performance, according to program representatives. For example,
one standard required no operational system downtime and the contractor
was not always able to meet this standard. After we reviewed the
performance standards, program representatives subsequently
incorporated some additional, improved standards to provide increased
visibility over how well the contractor responds to system outages.
These standards are designed to measure the time required to switch
from the primary to the secondary system for planned and unplanned
system outages. Additionally, according to program and contracting
representatives, the contractor initially delivered inadequate plans
for staffing and operations, and did not always meet the operational
system downtime requirement, but was within budget and generally met
established time frames.
Prior Reviews Have Similar Findings:
The inability to manage requirements and meet cost, schedule, and
performance objectives is not limited to the components and systems we
reviewed. Several prior GAO and DHS Inspector General reviews of major
DHS investments using a performance-based approach point to such
shortcomings. For example:
* In June 2007, we reported that DHS's performance-based contract for a
departmentwide financial management system, eMerge2, did not have clear
or complete requirements.[Footnote 22] Consequently, the program
experienced schedule delays and less than acceptable contractor
performance and was ultimately terminated after a $52 million
investment.
* In March 2007, we reported that the Coast Guard's performance-based
contract for replacing or modernizing its fleet of vessels and
aircraft, Deepwater, had requirements that were set at unrealistic
levels and then were frequently changed.[Footnote 23] This resulted in
cost escalation, schedule delays, and reduced contractor
accountability. The DHS Inspector General has also indicated that
numerous opportunities exist for DHS to make better use of sound
practices, such as well-defined requirements. The DHS Inspector General
recently concluded that Deepwater had poorly defined requirements and
inadequate oversight that contributed to ineffective or inefficient
results and increased costs.[Footnote 24]
* In February 2006, DHS's Inspector General reported that TSA's
Information Technology Managed Services contract had unclear
requirements that contributed to negative outcomes, including an 85
percent increase to nearly $834 million in DHS's 3-year cost estimate
and a failure to deliver expected information technology capabilities.
[Footnote 25]
Contracts with Well-Defined Requirements and Measurable Performance
Standards Experienced Positive Outcomes:
Contracts for four other major investments we reviewed--one at the
Coast Guard and three at TSA--had well-defined requirements and
measurable performance standards. These requirements did not
significantly change in scope after contract award. Contracts for
program management support for the Response Boat-Medium program,
maintenance of the Electronic Baggage Screening Program, and passenger
and baggage screening for the Screening Partnership Program met the
performance standards and were within budget. We were unable to
evaluate outcomes for the fourth contract associated with the
Transportation Worker Identification Credential (TWIC) program because
key work under the contract was delayed.
Program management support and related services we reviewed for the
Coast Guard's Response Boat-Medium investment had well-defined
requirements, and two of three performance standards were measurable.
The contracted services included research and project support,
financial and information management, and Web site support. The
requirements for research and project support clearly stated the
contractor was to provide project planning and execution support;
documentation, database, and internet support; and administrative
support. The performance standards were linked directly to the
requirements. The two measurable standards were for timely submission
of documents and electronic records keeping. The Coast Guard did not
indicate how it would measure contractor performance for the third
standard addressing research analysis and advisory services. Coast
Guard representatives said the contractor had fulfilled the
requirements within budget and on time, but did not document contractor
performance.
A maintenance contract for TSA's Electronic Baggage Screening Program
also had well-defined program management and technical requirements.
For example, the requirements for maintenance clearly stated the
contractor should perform preventative and corrective maintenance on
detection machines, providing specific definitions for each of these
tasks. The contract also had measurable performance standards linked
directly to the technical requirements. Specifically, program
representatives used the 'mean downtime' performance standard to
calculate the average number of hours a machine was unable to perform
its mission during a 16-hour day as a means of evaluating how quickly
the contractor restored the detection equipment to operation. For
fiscal years 2006 through 2007, the contractor met or exceeded the mean
downtime performance standard, which has become more rigorous each
year. Additionally, the contractor has stayed within the budget
established in the firm fixed-price contract. The current maintenance
contract improved TSA's ability to manage costs. Previously, DHS's
Inspector General found TSA's initial electronic baggage screening
maintenance contracts did not follow sound contracting practices. The
Inspector General recommended TSA take action to control costs by
modifying the type of contract used.[Footnote 26] TSA subsequently
withdrew the initial maintenance contract and issued four new
contracts, one of which we reviewed, for maintenance of detection
machines at over 400 airports in the United States.
Contracted security services at the San Francisco International Airport
for the Screening Partnership Program also had well-defined
requirements, and all measurable performance standards corresponded to
contract requirements. This was an improvement from our prior reviews
of TSA's Screening Partnership Program, which found that TSA had not
finalized performance-based planning documentation, including
developing measurable performance standards.[Footnote 27] The contract
we reviewed contained well-defined requirements for gate, checkpoint,
and baggage screening; basic preventative and corrective maintenance;
initial, recurrent, and remedial training; and recruiting and
supervising screeners. For example, the requirements for gate,
checkpoint, and baggage screening services clearly stated the
contractor should use technology and staff to prevent prohibited items
from entering sterile areas of the airport and should work to minimize
customer complaints while addressing in a timely manner any complaints
received. The performance standards assessed how often screeners could
successfully detect test images of prohibited items in checked baggage;
the percentage of audited records and inspected equipment, property,
and materials that were well-kept, operational, and recorded on
maintenance logs; and whether all new hires received the required
training before assuming their screening responsibilities. In terms of
expected outcomes, the contractor achieved a 2.2 percent cost underrun
during the first 5 months of the contract and exceeded most
requirements.
While the worker enrollment services contract for TSA's TWIC program
for a tamper-resistant biometric credential for maritime workers
included well-defined requirements and measurable performance
standards, we were unable to review the contract's outcomes as key work
under the contract was delayed.[Footnote 28] Prior GAO work found that
during the TWIC program's prototype phase, the program contract costs
doubled due to expanded requirements after contract award.[Footnote 29]
The contract we reviewed contained well-defined requirements for
developing enrollment centers; providing information technology
maintenance services; and setting up help desk services for TSA
employees, maritime workers, and other TWIC stakeholders. For example,
the requirements for enrollment centers clearly stated the contractor
was to provide maintenance services; infrastructure support such as
work stations; and operations, management, and administrative support.
Moreover, the contract's performance standards were measurable and
linked to the requirements. For example, the program developed
performance standards assessing the amount of time the contractor took
to respond to and resolve each caller's issue and to enroll maritime
workers in the program. By using measurable performance standards
linked to the contract's requirements, TSA has improved its ability to
identify and address performance issues and avoid negative outcomes.
Quality Assurance Surveillance Identified Less Than Expected
Performance:
Agencies are required to ensure that contractors are providing timely
and quality services and mitigate contractor performance problems. The
FAR indicates that surveillance plans provide for contract quality
assurance by specifying contractor work requiring surveillance and the
method of surveillance.[Footnote 30] While not all of the contracts we
reviewed had a quality assurance surveillance plan in place at contract
award, in all cases DHS components monitored and assessed contractor
performance against the contract performance standards. In the four
cases where we found poor contract outcomes, program staff identified
and documented less than expected performance and took action as shown
in table 2.
Table 2: Less Than Expected Performance Identified through Surveillance
and Action Taken:
Major investment by component: Customs and Border Protection: Automated
Commercial Environment Task Order 23;
Service: Developing trade systems software;
Performance: Costs increased by 40 percent ($21.1 million) and
contractor did not achieve project milestones according to schedule;
Action taken: Contractor did not earn full incentive fees.
Major investment by component: Customs and Border Protection: National
Prime Integration;
Service: Maintaining equipment used at border crossings, airports, and
seaports;
Performance: Longer than expected maintenance wait times;
Action taken: Contractor did not earn full award fees.
Major investment by component: Customs and Border Protection: SBInet
Project 28;
Service: Developing and fielding border surveillance systems;
Performance: System failed the initial performance test and project was
delayed 8 months with final acceptance in February 2008;
Action taken: Contractor did not receive 20 percent of contract payment
and DHS did not initially accept the system.
Major investment by component: Transportation Security Administration:
Secure Flight;
Service: Maintaining database used to screen airline passenger data;
Performance: Inadequacies identified in initial contractor planning
reports provided to TSA; operational downtime; and other forms of less
than expected performance;
Action taken: Contractor did not earn full award fees and DHS worked
with contractor to increase staffing levels.
Source: GAO analysis.
[End of table]
In two of these cases--ACE and Secure Flight--program representatives
identified less than expected performance through the use of a quality
assurance surveillance plan. In contrast, the NPI program office told
us they did not have a quality assurance surveillance plan for the
first year of the contract, but instead relied on award fee plan
criteria to assess contractor performance. The SBInet program office
monitored Project 28's execution through other means, such as program
reviews, technical interchange meetings, status and issue reporting,
and observation of contractor testing. Our prior work has found that if
performance-based acquisitions are not appropriately planned and
structured, including surveillance planning, there is an increased risk
that the government may receive products or services that are over
budget, delivered late, and of unacceptable quality.[Footnote 31]
DHS Has Faced Workforce and Oversight Challenges in Managing Its
Service Acquisitions:
Insufficient workforce and limited oversight of acquisition outcomes
has presented significant challenges for DHS in implementing its
service acquisitions, including those that use a performance-based
approach. Having adequate staff who collaborate across functions, and
reliable data are among the key success factors that we have identified
for achieving intended outcomes for all types of service acquisitions.
[Footnote 32] However, DHS continues to lack contracting and program
staff with the expertise needed to adequately plan or monitor
contractor execution of requirements, and some component acquisition
representatives indicated a lack of collaboration between the program
and contracting offices. In terms of oversight, inaccurate data has
limited departmentwide visibility into DHS's service acquisitions,
including those that are performance-based.
Sufficient and Experienced Staff Are Needed to Ensure Successful
Service Acquisitions:
Our prior work has highlighted the importance of having the right
people with the right skills to achieve successful acquisition
outcomes.[Footnote 33] However, DHS has not fully defined the types of
positions or numbers of staff for each position for its acquisition
workforce. CPO representatives identified acquisition staff shortages
as one of the primary obstacles to successful acquisitions, including
those that are performance-based. CPO representatives have established
departmentwide interim staffing goals addressing minimum and optimal
staffing levels for contract specialists for each component. As of
February 2008, CPO representatives reported that approximately 61
percent of the minimum required staff, and 38 percent of the optimal
level of contract specialists, were in place.
Insufficient contracting and program office staff was a challenge for
two of the investments we reviewed--SBInet and ACE. In these two cases,
contracts were not on schedule or contractor performance was otherwise
less than expected. In November 2006--the month following SBInet
Project 28's contract award--the DHS Inspector General reported that
DHS did not have the capacity needed to effectively plan, oversee, and
execute the SBInet investment; administer its contracts; and control
costs and schedule.[Footnote 34] For example, SBInet had a shortage of
engineers, logistical personnel, contracting officers, and cost
analysts. In January 2007, DHS's Investment Review Board confirmed
there were critical staffing shortages in the SBInet investment,
including Project 28. In February 2007, we reported that SBInet
representatives expressed concern about finding an adequate number of
program office staff with the right expertise. We subsequently reported
in February 2008 that the SBInet program office staffing level had
reached 305 positions out of a staffing goal of 470 by the end of
fiscal year 2008.[Footnote 35] CBP representatives expressed concerns
that staffing shortfalls could affect the agency's capacity to provide
adequate contractor oversight. ACE program officials also indicated
they were significantly understaffed, which prevented them from fully
defining the requirements for ACE Task Order 23. Officials told us the
investment had less than 10 percent of the staff they needed at the
time the requirements were being defined.
In its fiscal year 2007 departmentwide performance-based service
acquisition management plan, DHS also emphasized the importance of
filling program and contracting vacancies with staff experienced in
developing and implementing the required elements of performance-based
acquisitions. Contracting and program management staff for TSA's
Screening Partnership and Electronic Baggage Screening programs stated
that their previous experience helped them to develop well-defined
requirements and measurable performance standards. In these cases,
contracts were on budget and contract performance met or exceeded the
government's requirements. Representatives from Coast Guard, CBP, and
TSA noted success in supplementing expertise through acquisition
support offices, also known as centers of excellence, to improve
acquisition planning. Component representatives told us this support
could help acquisition staff improve the quality of contract documents,
including performance work statements. These support offices can serve
as a force multiplier, allowing their high-demand skill-sets to be used
across contracts and helping to improve contracting outcomes across the
component.
According to CPO representatives, hiring knowledgeable acquisition
staff has been difficult due to a limited pool of qualified acquisition
professionals and intense competition between the government and the
private sector for those professionals.[Footnote 36] The CPO has the
following ongoing departmentwide initiatives to recruit, train, and
define its acquisition workforce that have the potential to better
position DHS to address acquisition workforce challenges. These
include:
* centralize recruiting activities across the department;
* institute an acquisition intern program;
* employ direct hire authority[Footnote 37] and rehire
annuitants;[Footnote 38]
* reemphasize training opportunities and establishing a centralized
acquisition workforce training fund; and:
* conduct workforce planning.
The need for improved collaboration among the acquisition workforce is
also a recurrent theme in our work on acquisition management. We have
noted that acquisition groups should have procedures in place that
empower staff to collaborate when procuring goods and services and have
controls and incentives to ensure collaboration occurs.[Footnote 39] In
particular, leading organizations generally employ a multifunctional
approach including contracting, finance, legal, and other participants
as needed to help develop cost-effective acquisition approaches and
help ensure financial accountability. When requirements are dynamic, as
was the case for some of the investments we studied, there is a greater
need for more sophisticated collaboration--a point echoed by CBP and
TSA representatives who emphasize the importance of early collaboration
for complex acquisitions. OFPP guidance also emphasizes that
performance-based service acquisition is a collective responsibility
involving representatives from budget, technical, contracting,
logistics, legal, and program offices.
For the eight major investments we reviewed, program and contracting
offices jointly approved the acquisition plans, and representatives for
several of the contracts we studied indicated they worked together
effectively. However, at the component level, senior acquisition staff
at TSA and CBP indicated that collaboration between contracting and
program offices in general has been challenging. For example, they
cited poor timing when bringing together key stakeholders in the
contracting process, and in some cases, the program offices' preference
for prescriptive requirements rather than a more outcome-oriented
approach. Coast Guard and TSA acquisition representatives we spoke with
indicated they have ongoing efforts to promote greater collaboration.
Limited Data Inhibits DHS's Ability to Oversee Its Service
Acquisitions:
While DHS contracting and program representatives told us they use a
performance-based approach to the maximum extent practicable, DHS does
not have reliable data, either from FPDS-NG or at a departmentwide
level, to systematically monitor or evaluate service acquisitions,
including those that are performance-based acquisitions. Reliable data
are essential to overseeing and assessing the implementation of
contracting approaches, acquisition outcomes, and making informed
management decisions.
Our review of 138 contracts at the Coast Guard, CBP, ICE, and TSA coded
in FPDS-NG as performance-based illustrates the difficulty in
identifying such contracts.[Footnote 40] According to contracting
representatives at the Coast Guard, CBP, ICE, and TSA, only 42, about
30 percent, of these contracts could be confirmed as containing the
required performance-based elements--a performance work statement,
measurable performance standards, and a method of assessing contractor
performance against performance standards. About 18 percent had some
but not all of the required performance-based acquisition elements, and
about 51 percent--totaling about $347.3 million--had none of the
required elements (see table 3). This analysis indicates that the FPDS-
NG data are not reliable for reporting on the performance target for
eligible service obligations. For example, in fiscal year 2006, DHS
reported awarding performance-based contracts for 21 percent of
eligible service contract dollars. However, due to unreliable data, DHS
is likely farther away from meeting the governmentwide target than
previously reported. In addition, in July 2006 OFPP requested all
agencies to report in their 5-year management plan on the use of
performance-based contracts by service types--ranging from basic, such
as janitorial and landscaping, to complex, such as information
technology or systems development. The Acquisition Advisory Panel and
DHS's CPO also have raised concerns regarding the accuracy of the
performance-based designation in FPDS-NG. The Acquisition Advisory
Panel reported from its review at 10 federal agencies that 42 percent
of the performance-based contracts it reviewed had been incorrectly
coded.[Footnote 41]
Table 3: Review of Performance-Based Elements on Selected Contracts:
Performance-based elements: All elements;
Coast Guard: 18;
Customs and Border Protection: 3;
Immigration and Customs Enforcement: 0;
Transportation Security Administration: 21;
Total contracts: 42;
Percent of total contracts: 30.4.
Performance-based elements: Some elements;
Coast Guard: 16;
Customs and Border Protection: 0;
Immigration and Customs Enforcement: 5;
Transportation Security Administration: 4;
Total contracts: 25;
Percent of total contracts: 18.1.
Performance-based elements: No elements;
Coast Guard: 20;
Customs and Border Protection: 5;
Immigration and Customs Enforcement: 34;
Transportation Security Administration: 12;
Total contracts: 71;
Percent of total contracts: 51.5.
Performance-based elements: Total;
Coast Guard: 54;
Customs and Border Protection: 8;
Immigration and Customs Enforcement: 39;
Transportation Security Administration: 37;
Total contracts: 138;
Percent of total contracts: 100.
Source: GAO analysis of DHS review of 138 contracts coded as
performance-based in FPDS-NG.
[End of table]
According to DHS contracting representatives, contracts were miscoded
as performance-based for several reasons. Contracting staff may have
coded contracts as performance-based without the presence of the
required elements. Contracting staff also may not have followed data
validation procedures. Finally, there are technical FPDS-NG issues. For
example, DHS contracting and program staff said FPDS-NG automatically
applies the same designation to all orders issued under the master
contract although some individual orders may or may not be performance-
based.[Footnote 42] While DHS officials acknowledged this can occur,
they stated that the data field in FPDS-NG could still be edited for
orders issued under the master contract and that it was incumbent on
DHS contracting representatives to review the designation for every
contract action to ensure they are coded correctly.
Inaccurate federal procurement data is a long-standing governmentwide
concern. Our prior work and the work of the General Services
Administration's Inspector General have noted issues with the accuracy
and completeness of FPDS and FPDS-NG data.[Footnote 43] The Office of
Management and Budget has stressed the importance of submitting timely
and accurate procurement data to FPDS-NG and issued memos on this topic
in August 2004 and March 2007.
Accurate FPDS-NG data could facilitate the CPO's departmentwide
oversight of service acquisitions, including those that are performance-
based. For example, the selection of a contract type is the principal
means agencies have for allocating financial risk between the
government and the contractor. For performance-based services, federal
law establishes a preference for using firm fixed-price as opposed to
cost reimbursable or time and materials contracts.[Footnote 44] In
addition, the CPO has instructed components to avoid using time-and-
material contracts for services. We reviewed the contract type for the
42 contracts DHS confirmed were performance-based. Our analysis showed
that DHS awarded about 70 percent of these 42 contracts as fixed-price;
these actions represented about 42 percent of total obligations for
these awards. About 37 percent of total obligations were awarded as
cost-reimbursable, and 7 percent were awarded as time and materials
contracts (see fig. 1).
Figure 1: Percent of Contracts and Obligations by Contract Type for 42
DHS Performance-Based Service Acquisitions (Fiscal Years 2005 and
2006):
[See PDF for image]
This figure is a vertical bar graph depicting the following data:
Contract type: Fixed price;
Contracts: 69.0%;
Obligations: 41.9%.
Contract type: Cost reimbursable;
Contracts: 9.5%;
Obligations: 37.0%.
Contract type: Time and materials/labor hours;
Contracts: 11.9%;
Obligations: 6.7%.
Contract type: Other[A];
Contracts: 9.5%;
Obligations: 14.4%.
Source: GAO analysis of DHS confirmed performance-based service
acquisitions.
[A] "Other" includes contracts for which the contract type field in
FDPS-NG was blank, designated as being none of the contract types
listed within FPDS-NG, or coded with more than one contract type.
[End of figure]
At a departmentwide level, CPO representatives responsible for
acquisition oversight indicated they have not conducted systematic
assessments including costs, benefits, and other outcomes of a
performance-based approach. To improve the implementation of
performance-based acquisitions, CPO representatives established a work
group in May 2006 to leverage knowledge among DHS components. They also
noted that they are working with OFPP to develop a best practices guide
on measurable performance standards and to gather good examples of
performance-based contracts. In addition, the CPO has implemented a
departmentwide acquisition oversight program that is intended to assess
(1) compliance with federal acquisition guidance, (2) contract
administration, and (3) business judgment. This program was designed
with the flexibility to address specific procurement issues, such as
performance-based service acquisitions, and is based on a series of
component-level reviews.[Footnote 45] However, this oversight program
has not yet included an evaluation of the outcomes of this acquisition
method.
Conclusions:
As part of its expansive homeland security mission, DHS spends billions
of dollars on service acquisitions for critical trade, transportation,
and border security investments. Consistent with federal procurement
policy, DHS has emphasized a performance-based approach to improve
service acquisition outcomes. However, in keeping with our prior
findings, DHS's designation of a service acquisition as performance-
based was not as relevant as the underlying contract conditions. Sound
acquisition practices, such as clearly defining requirements;
establishing complementary measurable performance standards; and
planning and conducting surveillance in order to take corrective
actions are all hallmarks of successful service acquisitions. In the
cases we reviewed, as well as in prior findings lacking these key
elements, DHS did not always achieve successful acquisition outcomes.
To improve DHS's acquisition management, the CPO is taking steps to
address departmentwide workforce and oversight needs. However, improved
data accuracy and systematic evaluation of contracting methods and
outcomes are also key to successful oversight efforts. Continued
emphasis on these areas is required to prevent less than expected
acquisition outcomes for the department's critical mission.
Recommendations for Executive Action:
To increase DHS's ability to achieve improved outcomes for its service
acquisitions, including those that are performance-based, we recommend
that the Secretary of Homeland Security implement the following three
actions:
* routinely assess requirements for major, complex investments to
ensure they are well-defined and develop consistently measurable
standards linked to those requirements;
* at a departmentwide level, systematically evaluate the outcomes of
major investments and relevant contracting methods; and:
* continuously improve the quality of FPDS-NG data to facilitate the
ability to accurately identify and assess the use and outcomes of
various contracting methods.
Agency Comments and Our Evaluation:
We provided a draft of this report to DHS and OMB for review and
comment. In written comments, DHS generally concurred with our findings
and recommendations and provided some information on efforts under way
to improve acquisition management. The department's comments are
reprinted in appendix II. OMB did not comment on the findings or
conclusions of this report. DHS and OMB each provided technical
comments, which we incorporated as appropriate and where supporting
documentation was provided.
In response to our recommendations that DHS (1) routinely assess
requirements for major, complex investments to ensure they are well-
defined, and develop consistently measurable standards linked to those
requirements, and (2) systematically evaluate the outcomes of major
investments and relevant contracting methods, DHS stated that the CPO
is working to strengthen acquisition and procurement. For example, DHS
noted that the CPO is revising the investment review process; reviewing
major programs and investments; and building the capability to manage
complex efforts by ensuring program offices are properly structured and
staffed with the right people and skills. To assist in accomplishing
these goals, DHS noted that the CPO has established a new division to
provide oversight and support for acquisition programs. We have ongoing
work at DHS reviewing the results of DHS's oversight of major
acquisitions and plan to report on these initiatives in the final
product for that engagement. Improving acquisition management has been
an ongoing challenge since the department was established and requires
sustained management attention. However, DHS's response did not address
how the CPO's process and organizational changes will impact component-
level management and assessment of complex acquisitions to improve
outcomes.
Concerning the third recommendation, that DHS continuously improve the
quality of FPDS-NG data to facilitate the ability to accurately
identify and assess the use and outcomes of various contracting
methods, DHS stated that as part of the CPO's oversight reviews, the
accuracy of the FPDS-NG data is validated for the review sample,
including whether contracts have been properly coded as performance-
based. DHS also added that the CPO is an active member of the OFPP
group working to improve FPDS-NG and has established a governance board
whereby the CPO reaches out to DHS components to improve data
collection. While these initiatives may be steps in the right
direction, DHS's response did not present the results of the oversight
reviews, or state how these coordination efforts address the causes of
miscoding or how they will improve the quality of FPDS-NG data in the
future.
DHS also noted other initiatives to improve the acquisition workforce,
including working to obtain qualified acquisition professionals and
conducting staffing studies to better define the department's needs. We
also have ongoing work at DHS on the acquisition workforce and plan to
report on the results of DHS's efforts in the final product for that
engagement.
As agreed with your offices, unless you publicly announce the contents
of this report, we plan no further distribution for 30 days from the
report date. At that time, we will send copies of this report to the
Secretary of Homeland Security, the Director of the Office of
Management and Budget, and other interested congressional committees.
We will also make copies available to others upon request. In addition,
this report will also be available at no charge on GAO's Web site at
[hyperlink, http://www.gao.gov].
If you have questions about this report or need additional information,
please contact me at (202) 512-4841 or huttonj@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this report.
Signed by:
John P. Hutton, Director:
Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
[End of section]
To evaluate selected Department of Homeland Security (DHS) components'
implementation of a performance-based approach in the context of
service acquisitions for major investments, we conducted a detailed
file review of eight judgmentally selected performance-based contracts
for major, complex investments with a total value greater than $100
million.[Footnote 46] To identify the components with the highest
obligations using a performance-based approach in fiscal years 2005 and
2006, we used the Federal Procurement Data System-Next Generation (FPDS-
NG) and selected four components--Coast Guard, Customs and Border
Protection (CBP), Immigration and Customs Enforcement (ICE), and the
Transportation Security Administration (TSA).[Footnote 47] We excluded
the Federal Emergency Management Agency (FEMA) from our review because
of atypical fiscal year 2006 spending on Gulf Coast hurricane relief
efforts.
To identify contracts for major investments, we utilized several
different approaches.[Footnote 48] Three issues required us to modify
our plans--the lack of information in FPDS-NG, inaccurate DHS
contracting data on major investments, and inaccurate information
provided by one DHS component. Because the FPDS-NG major program field
was typically blank, we were unable to use this field to identify
contracts associated with DHS major investments. As a result, we asked
DHS to provide us with a list of performance-based contracts associated
with the four components' major investments. DHS provided a list
including 76 contracts for 16 major investments. Due to data
inaccuracies related to the contract numbers on DHS's list, we were
only able to identify performance-based contracts associated with two
major investments. We selected one contract from CBP's Automated
Commercial Environment and one from TSA's Secure Flight that had the
highest obligations. To obtain additional contracts to review, we met
with CBP and TSA contracting representatives and they identified five
other major investments with performance-based contracts, and we
included one from each investment in our review. Coast Guard
contracting representatives told us none of their major investments
used performance-based contracts. However, we randomly selected and
reviewed four contracts and seven orders at the Coast Guard from DHS's
list of major investments, and found that five of them were performance-
based. Of these, we selected three performance-based orders from one
contract for one major Coast Guard investment--the Response Boat-
Medium--to include in our review. Lastly, ICE contracting
representatives also reported that none of their major investments used
a performance-based approach and provided us with their best examples
of a performance-based contract. We reviewed these examples and found
that none of them were associated with a major investment. As a result,
we excluded ICE from our contract review.
For the selected contracts, with a combined estimated value of $1.53
billion, we reviewed documentation, including acquisition plans,
performance work statements and statements of objectives, quality
assurance surveillance plans, and government or contractor performance
reports where available. Performance-based service acquisitions were
difficult to identify and were not clearly distinct from other service
acquisitions because they did not always include all required elements.
Therefore, we reviewed prior GAO and DHS Inspector General reports to
provide a broader context for service acquisition and other DHS major
investments. We reviewed public laws, federal and agency acquisition
regulations, and the Office of Management and Budget's Office of
Federal Procurement Policy (OFPP) memoranda and best practices. We also
interviewed procurement and program representatives to determine the
extent to which the performance-based planning process influenced
expected outcomes.
To identify management challenges that may affect DHS's successful
implementation of service acquisitions for major investments, including
those that use a performance-based approach, we analyzed management
documents and plans; staffing data; and oversight mechanisms, including
the Federal Procurement Data System-Next Generation (FPDS-NG), the
governmentwide database for procurement spending. From FPDS-NG, we
judgmentally selected 138 contracts for primarily basic services coded
as performance-based and awarded in fiscal years 2005 or 2006 at Coast
Guard, CBP, ICE, and TSA with obligations greater than $1 million
dollars and asked the components to verify whether these contracts
included the required performance-based elements outlined in FAR
subpart 37.6. Finally, we interviewed contracting and program
management staff and DHS Chief Procurement Office (CPO) representatives
and met with representatives from the Office of Management and Budget's
Office of Federal Procurement Policy (OFPP). To evaluate the extent to
which DHS used different contract types for performance-based
acquisitions, we reviewed the contract type on all contracts DHS
verified as performance-based.
We conducted this performance audit from January 2007 to April 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Homeland Security:
U.S.Department of Homeland Security:
Washington, DC 20528:
April 11,2008:
Mr. John P. Hutton:
Director:
Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Hutton:
Thank you for the opportunity to review and comment on the Government
Accountability Office's (GAO's) draft report GAO-08-263 entitled
Department Of Homeland Security: Better Planning and Assessment Needed
to Improve Outcomes for Complex Service Acquisitions. Technical
comments have been provided under separate cover.
The Department of Homeland Security (DHS) generally concurs with the
findings and recommendations in the draft report and offers the
following with respect to the report's three recommendations:
GAO Recommendation #1: Routinely assess requirements for major, complex
investments to ensure they are well-defined and develop consistently
measurable standards linked to those requirements.
GAO Recommendation #2: At a department-wide level, systematically
evaluate the outcomes of major investments and relevant contracting
methods.
DHS Response to Recommendations #1 and #2: DHS is in the midst of many
crucial acquisitions that are vital to its success. The Chief
Procurement Officer (CPO) is working to strengthen acquisition and
procurement by institutionalizing solid processes, including the
following actions:
A. Strengthening the requirements and investment review processes by
improving the joint requirements council and Investment Review Board
(IRB) process. We are preparing to initiate a new Department-wide
requirements process and have reinvigorated our investment review
process;
B. Reviewing the major programs and investments to ensure that the
requirements are clear, cost estimates are valid, technology risks are
properly assessed, schedules are realistic, contract vehicles are
proper, and the efforts are well managed. We have held one formal
Deputy Secretary IRB and are projecting one per month. We are also
beginning the processes to conduct paper IRBs and Deputy UnderSecretary
for Management IRBs, as well as establishing Acquisition Program
Baselines and authorizing execution to the APB for all Level 1 and 2
programs;
C. Building the capability to manage complex efforts by ensuring that
program offices are properly structured and staffed with the right
people and skills to ensure efficient and effective program management
and oversight; and to aggressively hire where we have known shortages;
and;
D. Examining best practice metrics in use by other departments with the
intent to start implementation this year.
The Acquisition Program Management Division (APMD) of the Office of the
Chief Procurement Officer (OCPO) began operations in August 2007. The
division was established to provide oversight and support for
acquisition programs. To date, APMD has performed Quick Look
assessments of 37 Level 1 programs and has overseen Deep Dive reviews
of the SBInet and Advance Spectroscopic Portal (ASP) programs. APMD has
provided advice and guidance to a number of programs, particularly in
the area of cost benefit analysis. Currently the APMD team is focused
on an aggressive Investment & Acquisition process re-engineering
effort. The effort includes replacing DHS Management Directive 1400
Investment Review Process, establishing revised investment and
acquisition decision procedures, as well as processes for acquisition
program baselining, periodic reporting, acquisition of services, and
other initiatives as they are identified.
CPO is also working to assure that DHS obtains qualified acquisition
professionals at the right time with the right skill-set. Competition
for these professionals is intense within the Washington, D.C. area. To
resolve our personnel shortages, we are intensifying our human capital
planning efforts to minimize skill and competency gaps as well as
minimize our critical vacancies and reliance on contractors. For
example, in response to the OMB 1102 Contracting Workforce Competency
Gap Survey, we developed a training plan that spans the next three
years. This training plan targets the contracting functional area
within the DHS Acquisition Workforce, but it will also benefit other
acquisition career fields including program management and Contracting
Officer's Technical Representatives.
CPO is also currently conducting staffing studies to better define our
acquisition workforce needs. Currently our workforce includes program
managers and contract specialists. As part of our human capital
planning efforts, we will be identifying other required acquisition
career fields such as test and evaluation, systems engineering,
logistics, and cost estimating. We are aggressively working to ensure
that each acquisition position, upon definition, is encumbered by an
acquisition professional trained and certified at the appropriate
level. To this end, we are continuously reviewing and updating our
Acquisition Training Program, the underpinning of a good certification
program. We are utilizing the Defense Acquisition Workforce Improvement
Act framework to develop DHS certification standards. We have also
centralized a number of recruiting activities including issuing
Department-wide vacancy announcements. Our centralized recruitment
efforts to date have focused primarily on contracting professionals.
Expansion to other acquisition career fields will occur as each series
is defined and Department-wide needs are identified. This initiative
supplements our Components' on-going recruitment efforts with a goal of
recruiting the best candidates available. In 2005, the Department
commenced the Acquisition Fellows Program. The goal of the Fellows
Program was to attract new talent at the entry level into our
acquisition positions, and retain and train them through a professional
career development program. Building on the success of the Acquisition
Fellows Program, CPO formalized and modeled it to further resemble the
highly successful Department of Defense program. This year CPO received
funding for the Acquisition Professional Career Program and 33 of the
requested 66 positions were funded. In FY 2009, the plan provides for a
total of 100 positions to be funded. Our inaugural class in the
Acquisition Professional Career Program began in January 2008 and a
second class will begin in June 2008. Our goal is to grow this program
to 300 positions by FY 2011 to fill our critical acquisition positions.
GAO Recommendation #3: Continuously improve the quality of Federal
Procurement Data System-Next Generation (FPDS-NG) data to facilitate
the ability to accurately identify and assess the use and outcomes of
various contracting methods.
DHS Response to Recommendation #3: As part of CPO's oversight reviews,
the accuracy of the FPDS data is validated for the review sample,
including whether the contract has been properly coded as a performance
based contract. In addition, OCPO is an active member of the Office of
Federal Procurement Policy group that is working to improve FPDS. OCPO
has also established a Governance Board whereby CPO reaches out to the
DHS Components to improve data collection.
Thank you again for the opportunity to comment on this draft report and
we look forward to working with you on future homeland security issues.
Sincerely,
Signed by:
Penelope G. McCormack:
Acting Director:
Departmental Audit Liaison:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
John Hutton, (202) 512-4841, or huttonj@gao.gov:
Staff Acknowledgments:
In addition to the individual named above key contributors to this
report were Amelia Shachoy, Assistant Director; Don Springman; Jeffrey
Hartnett; Sean Seales; Alex Winograd; Karen Sloan; Julia Kennon; Lynn
Milan; Art James; Marie Ahearn; and Kenneth Patton.
[End of section]
Footnotes:
[1] GAO, Defense Acquisitions: Tailored Approach Needed to Improve
Service Acquisition Outcomes, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-20] (Washington, D.C.: Nov. 9, 2006).
[2] We excluded the Federal Emergency Management Agency (FEMA) because
of atypical fiscal year 2006 spending on Gulf Coast hurricane relief
efforts.
[3] DHS has categorized major investments as Levels 1 and 2 and Level 3
(information technology). For the purposes of our review, we focused on
Level 1 investments. A Level 1 major investment is defined in DHS
Management Directive 1400 as an investment with greater than $100
million in total acquisition costs including planning, or for
information technology investments, a life-cycle cost greater than $200
million.
[4] ICE officials told us that none of their major investments used a
performance-based approach.
[5] We reviewed either the contract or selected orders on the contract.
[6] DHS's investment review process includes three phases: pre-
acquisition, acquisition, and sustainment. The acquisition phase
includes systems development, and the sustainment phase includes
operations and maintenance.
[7] Floyd D. Spence National Defense Authorization Act for Fiscal Year
2001, Pub. L. No. 106-398 § 821(a) (2000) required that the Federal
Acquisition Regulation (FAR) be revised to establish a preference for
the use of a performance-based approach in the acquisition of services,
which was done in FAR 37.102(a), providing that performance-based
acquisition is the preferred method for acquiring services and
generally is to be used to the maximum extent practicable.
[8] FAR 37.601; FAR 37.602(b); FAR 37.604. A fourth element,
performance incentives, is required where appropriate.
[9] Floyd D. Spence National Defense Authorization Act for Fiscal Year
2001, Pub. L. No. 106-398, § 821(a) (2000); FAR 37.102(a).
[10] Office of Management and Budget, Office of Federal Procurement
Policy, A Report on the Performance-Based Service Contracting Pilot
Project, May 1998.
[11] Office of Federal Procurement Policy Memorandum, "Increasing the
Use of Performance-based Service Acquisitions," September 7, 2004.
[12] Services Acquisition Reform Act of 2003, Title XIV, National
Defense Authorization Act for Fiscal Year 2004, Pub. L. No. 108-136, §
1423 (2003); National Defense Authorization Act for Fiscal Year 2006,
Pub. L. No. 109-163, § 843 (2006) (extended the deadline for the
Panel's report by 6 months).
[13] Report of the Acquisition Advisory Panel to the Office of Federal
Procurement Policy and the United States Congress, January 2007.
[14] Responsibility for the acquisition function at DHS is shared
between the CPO and each DHS component head. Eight DHS components have
internal procurement offices with a Head of Contracting Activity (HCA)
who reports directly to the component head and is accountable to the
CPO. The eight components are: Coast Guard, CBP, FEMA, Federal Law
Enforcement Training Center, ICE, Office of Procurement Operations,
Secret Service, and TSA. The HCA for each component has overall
responsibility for the day-to-day management of the component's
acquisition function. See GAO, Department of Homeland Security:
Progress and Challenges in Implementing the Department's Acquisition
Oversight Plan, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-
900] (Washington, D.C.: June 2007).
[15] In using the performance-based approach, sound contracting
practices dictate that required contract outcomes or requirements be
well-defined, providing clear descriptions of results to be achieved.
FAR 2.101 specifically provides that a performance work statement for
performance-based acquisitions describe the required results in clear,
specific, and objective terms with measurable outcomes.
[16] GAO, Coast Guard: Observations on the Fiscal Year 2008 Budget,
Performance, Reorganization, and Related Challenges, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-489T] (Washington D.C.: Apr.
18, 2007).
[17] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20]; and GAO,
Coast Guard: Status of Efforts to Improve Deepwater Program Management
and Address Operational Challenges, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-575T] (Washington D.C.: Mar. 8, 2007).
[18] A letter contract authorizes a contractor to begin work and incur
costs before reaching a final agreement on contract terms and
conditions, including price.
[19] GAO, Information Technology: Improvements for Acquisition of
Customs Trade Processing System Continue, but Further Efforts Needed to
Avoid More Cost and Schedule Shortfalls, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-46] (Washington, D.C.: Oct.
25, 2007).
[20] The contractor testified in October 2007 before subcommittees of
the House Homeland Security Committee, that it spent approximately
double the original contract amount.Because this is a firm fixed price
contract, absent changes or delays created by the government, CBP is
not responsible for these additional costs.
[21] GAO, Secure Border Initiative: Observations on the Importance of
Applying Lessons Learned to Future Projects, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T] (Washington D.C.: Feb.
27, 2008).
[22] GAO, Homeland Security: Departmentwide Integrated Financial
Management Systems Remain a Challenge, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-536] (Washington, D.C.: June
21, 2007).
[23] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-575T].
[24] Department of Homeland Security Inspector General, Major
Management Challenges Facing the Department of Homeland Security, OIG-
08-11 (January 2008).
[25] Department of Homeland Security Inspector General, Transportation
Security Administration's Information Technology Managed Services
Contract, OIG-06-23 (February 2006).
[26] Department of Homeland Security Inspector General, Evaluation of
TSA's Contract for the Installation and Maintenance of Explosive
Detection Equipment at United States Airports, OIG-04-44 (September
2004).
[27] GAO, Aviation Security: Preliminary Observations on TSA's Progress
to Allow Airports to Use Private Passenger and Baggage Screening
Services, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-126]
(Washington, D.C.: Nov. 19, 2004); and GAO, Aviation Security: Progress
Made to Set Up Program Using Private-Sector Airport Screeners, but More
Work Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-166]
(Washington, D.C.: Mar. 31, 2006).
[28] TSA awarded a contract in January 2007 to issue biometric
credentials to maritime workers. TSA officials told us it took more
time than originally planned to bring the TWIC prototype enrollment
system into alignment with new government security standards. During
the course of our review, the schedule to initiate work was delayed
from March to October 2007.
[29] GAO, Transportation Security: DHS Should Address Key Challenges
before Implementing the Transportation Worker Identification Credential
Program, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-982]
(Washington, D.C.: Sept. 29, 2006).
[30] FAR 37.604 and 46.401.
[31] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-489T].
[32] GAO, Framework for Assessing the Acquisition Function at Federal
Agencies, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-218G]
(Washington, D.C.: Sept. 2005), and GAO-07-20.
[33] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-20].
[34] Department of Homeland Security Inspector General, Risk Management
Advisory for the SBInet Program Initiation, OIG-07-07 (November 2006).
[35] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-508T].
[36] We have ongoing work on DHS's acquisition workforce and plan to
report on these issues in the final product for that engagement.
[37] Services Acquisition Reform Act of 2003, Div. A, Title XIV,
National Defense Authorization Act for Fiscal Year 2004, Pub. L. No.
108-136, § 1413 (2003) allowed federal agencies to exercise direct hire
authority in accordance with regulations issued by the Office of
Personnel Management. This authority was extended to September 30,
2012, by the National Defense Authorization Act for Fiscal Year 2008,
Pub. L. No. 110-181, Div. A, Title VIII, § 853 (2008).
[38] General Services Administration Modernization Act, Pub. L. No. 109-
313, § 4 (2006) granted agency heads authority, in consultation with
the OPM and OFPP, to approve reemployment of annuitants in fields
related to acquisition management.
[39] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-218G].
[40] The 138 contracts were awarded in fiscal years 2005 or 2006 with
obligations greater than $1 million, totaling about $1.3 billion.
[41] Report of the Acquisition Advisory Panel to the Office of Federal
Procurement Policy and the United States Congress, January 2007.
[42] Known as Indefinite Delivery Indefinite Quantity (IDIQ) contracts,
these contracts provide for an indefinite quantity of supplies or
services within stated limits during a fixed period of time. The
government places orders for individual requirements.
[43] GAO, Reliability of Federal Procurement Data, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-04-295R] (Washington, D.C.: Dec.
30, 2003); GAO, Improvements Needed to the Federal Procurement Data
System-Next Generation, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-05-960R] (Washington, D.C.: Sept. 27, 2005); and General
Services Administration Inspector General, Review of the Federal
Procurement Data System-Next Generation (FPDS-NG), Report Number
A040127/O/T/F06016 (March 2006).
[44] For fixed price contracts, a specified price is paid regardless of
the contractor's costs, minimizing the financial risk to the
government. For cost reimbursable and time and materials contacts, the
government generally assumes the risk of cost overruns. For example,
cost reimbursable contracts provide for the government to pay
reasonable, allowable and allocable costs incurred by the contractor up
to the contract's price ceiling. Time and materials contracts provide
for acquiring services on the basis of fixed costs for labor hours and
materials at the stated contract ceiling price. In time and materials
contracts, if services delivered do not meet contract requirements and
the government exercises its right to have the contractor correct the
deficiencies, the government pays the additional labor and material
costs to do so, excluding profit.
[45] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-900].
[46] DHS has categorized major investments as Levels 1 and 2 and Level
3 (information technology). For the purposes of our review, we focused
on Level 1 investments. A Level 1 major investment is defined in DHS
Management Directive 1400 as an investment with greater than $100
million in total acquisition costs including planning, or for
information technology investments, a life-cycle cost greater than $200
million.
[47] During our review TSA was exempt from the FAR and followed the
Acquisition Management System, developed by the Federal Aviation
Administration, which included acquisition policy providing that
service contracts should incorporate performance-based contracting
methods.
[48] We selected eight major investments and reviewed either a
performance-based contract or selected orders on a performance-based
contract associated with each investment.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
