Defense Acquisitions
Better Weapon Program Outcomes Require Discipline, Accountability, and Fundamental Changes in the Acquisition Environment
Gao ID: GAO-08-782T June 3, 2008
Since 1990, GAO has designated the Department of Defense's (DOD) management of major weapon system acquisitions a high risk area. DOD has taken some action to improve acquisition outcomes, but its weapon programs continue to take longer, cost more, and deliver fewer capabilities than originally planned. These persistent problems--coupled with current operational demands--have impelled DOD to work outside of its traditional acquisition process to acquire equipment that meet urgent warfighter needs. Poor outcomes in DOD's weapon system programs reverberate across the entire federal government. Over the next 5 years, DOD plans to invest about $900 billion to develop and procure weapon systems--the highest level of investment in two decades. Every dollar wasted on acquiring weapon systems is less money available for other priorities. This testimony describes DOD's current weapon system investment portfolio, the problems that contribute to cost and schedule increases, and the potential impacts of recent legislative initiatives and DOD actions aimed at improving outcomes. It also provides some observations about what is needed for DOD to achieve lasting reform. The testimony is drawn from GAO's body of work on DOD's acquisition, requirements, and funding processes, as well as its most recent annual assessment of selected DOD weapon programs.
DOD's portfolio of weapon system programs has grown at a pace that far exceeds available resources. From 1992 to 2007, the estimated acquisition costs remaining for major weapons programs increased almost 120 percent, while the annual funding provided for these programs only increased 57 percent. Current programs are experiencing, on average, a 21-month delay in delivering initial capabilities to the warfighter--often forcing DOD to spend additional funds on maintaining legacy systems. Systemic problems both at the strategic and at the program level underlie cost growth and schedule delays. At the strategic level, DOD's processes for identifying warfighter needs, allocating resources, and developing and procuring weapon systems--which together define DOD's overall weapon system investment strategy--are fragmented and broken. At the program level, weapon system programs are initiated without sufficient knowledge about system requirements, technology, and design maturity. Lacking such knowledge, managers rely on assumptions that are consistently too optimistic, exposing programs to significant and unnecessary risks and ultimately cost growth and schedule delays. At the same time, frequent turnover of program managers and an increased reliance on contractors increases the government's risk of losing accountability. Recognizing the need for more discipline and accountability in the acquisition process, Congress recently enacted legislation part of which requires decision-makers to certify that programs meet specific criteria at key decision points early in the acquisition process. Likewise, DOD has recently begun to develop several initiatives that are based in part on congressional direction and GAO recommendations. If adopted and implemented properly, these measures could provide a foundation for establishing a well balanced investment strategy, sound business cases for major weapon system acquisition programs, and a better chance to spend resources wisely. While legislation and policy revisions can help guide change, DOD must begin making better choices that reflect joint capability needs and match requirements with resources or the department will continue to experience poor acquisition outcomes. DOD investment decisions continue to be dictated by the services who propose programs that overpromise capabilities and underestimate costs to capture the funding needed to start and sustain development programs. The transitory nature of leadership further undermines successful reform. To better ensure warfighter capabilities are delivered when needed and as promised, incentives must encourage a disciplined, knowledge-based approach, and a true partnership with shared goals must be developed among the department, the military services, the Congress, and the defense industry.
GAO-08-782T, Defense Acquisitions: Better Weapon Program Outcomes Require Discipline, Accountability, and Fundamental Changes in the Acquisition Environment
This is the accessible text file for GAO report number GAO-08-782T
entitled 'Defense Acquisitions: Better Weapon Program Outcomes Require
Discipline, Accountability, and Fundamental Changes in the Acquisition
Environment' which was released on June 3, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Committee on Armed Services, U.S. Senate:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 9:30 a.m. EDT:
Tuesday, June 3, 2008:
Defense Acquisitions:
Better Weapon Program Outcomes Require Discipline, Accountability, and
Fundamental Changes in the Acquisition Environment:
Statement of Katherine V. Schinasi, Managing Director:
Acquisition and Sourcing Management:
GAO-08-782T:
GAO Highlights:
Highlights of GAO-08-782T, a testimony before the Committee on Armed
Services, U.S. Senate.
Why GAO Did This Study:
Since 1990, GAO has designated the Department of Defense‘s (DOD)
management of major weapon system acquisitions a high risk area. DOD
has taken some action to improve acquisition outcomes, but its weapon
programs continue to take longer, cost more, and deliver fewer
capabilities than originally planned. These persistent problems”coupled
with current operational demands”have impelled DOD to work outside of
its traditional acquisition process to acquire equipment that meet
urgent warfighter needs.
Poor outcomes in DOD‘s weapon system programs reverberate across the
entire federal government. Over the next 5 years, DOD plans to invest
about $900 billion to develop and procure weapon systems”the highest
level of investment in two decades. Every dollar wasted on acquiring
weapon systems is less money available for other priorities.
This testimony describes DOD‘s current weapon system investment
portfolio, the problems that contribute to cost and schedule increases,
and the potential impacts of recent legislative initiatives and DOD
actions aimed at improving outcomes. It also provides some observations
about what is needed for DOD to achieve lasting reform. The testimony
is drawn from GAO‘s body of work on DOD‘s acquisition, requirements,
and funding processes, as well as its most recent annual assessment of
selected DOD weapon programs.
What GAO Found:
DOD‘s portfolio of weapon system programs has grown at a pace that far
exceeds available resources. From 1992 to 2007, the estimated
acquisition costs remaining for major weapons programs increased almost
120 percent, while the annual funding provided for these programs only
increased 57 percent. Current programs are experiencing, on average, a
21-month delay in delivering initial capabilities to the
warfighter”often forcing DOD to spend additional funds on maintaining
legacy systems.
Systemic problems both at the strategic and at the program level
underlie cost growth and schedule delays. At the strategic level, DOD‘s
processes for identifying warfighter needs, allocating resources, and
developing and procuring weapon systems”which together define DOD‘s
overall weapon system investment strategy”are fragmented and broken. At
the program level, weapon system programs are initiated without
sufficient knowledge about system requirements, technology, and design
maturity. Lacking such knowledge, managers rely on assumptions that are
consistently too optimistic, exposing programs to significant and
unnecessary risks and ultimately cost growth and schedule delays. At
the same time, frequent turnover of program managers and an increased
reliance on contractors increases the government‘s risk of losing
accountability.
Recognizing the need for more discipline and accountability in the
acquisition process, Congress recently enacted legislation part of
which requires decision-makers to certify that programs meet specific
criteria at key decision points early in the acquisition process.
Likewise, DOD has recently begun to develop several initiatives that
are based in part on congressional direction and GAO recommendations.
If adopted and implemented properly, these measures could provide a
foundation for establishing a well balanced investment strategy, sound
business cases for major weapon system acquisition programs, and a
better chance to spend resources wisely.
While legislation and policy revisions can help guide change, DOD must
begin making better choices that reflect joint capability needs and
match requirements with resources or the department will continue to
experience poor acquisition outcomes. DOD investment decisions continue
to be dictated by the services who propose programs that overpromise
capabilities and underestimate costs to capture the funding needed to
start and sustain development programs. The transitory nature of
leadership further undermines successful reform. To better ensure
warfighter capabilities are delivered when needed and as promised,
incentives must encourage a disciplined, knowledge-based approach, and
a true partnership with shared goals must be developed among the
department, the military services, the Congress, and the defense
industry.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-782T]. For more
information, contact Katherine V. Schinasi at (202) 512-4841 or
schinasik@gao.gov.
[End of section]
Mr. Chairman and Members of the Committee:
I am pleased to be here today to discuss the Department of Defense's
(DOD) management of its major weapon system acquisitions--an area that
has been on GAO's high risk list since 1990. Prior to and since that
time, Congress and DOD have continually explored ways to improve
acquisition outcomes without much to show for their efforts. DOD's
major weapon system programs continue to take longer, cost more, and
deliver fewer quantities and capabilities than originally planned.
Current operational demands have highlighted the impact of these
persistent problems as DOD has been forced to work outside of its
traditional acquisition process to acquire equipment that meet
warfighter needs.
Investment in weapons acquisition programs is now at its highest level
in two decades. The department expects to invest about $900 billion
(fiscal year 2008 dollars) over the next 5 years on development and
procurement with more than $335 billion invested specifically in major
defense acquisition programs. Given the size of this investment, poor
outcomes in DOD's weapon system programs reverberate across the entire
federal government. Every dollar wasted during the development and
acquisition of weapon systems is money not available for other internal
and external budget priorities--such as the war on terror and mandatory
payments to growing entitlement programs.
My statement today is drawn from our body of work on DOD's acquisition,
requirements, and funding processes, as well as our annual assessment
of selected DOD weapon programs. As you requested, I will focus on (1)
the performance of DOD's major defense acquisition program portfolio;
(2) the underlying systemic problems that contribute to poor cost and
schedule outcomes; (3) recent legislative initiatives and DOD actions
aimed at addressing these problems; and (4) the extent to which those
initiatives and actions can be expected to improve the future
performance of DOD's major defense acquisition programs. Our work was
conducted in May 2008 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
Summary:
Since fiscal year 2000, DOD significantly increased the number of major
defense acquisition programs and its overall investment in them. During
this same time period, acquisition outcomes have not improved. Based on
our analysis, total acquisition costs for the fiscal year 2007
portfolio of major defense acquisition programs increased 26 percent
and development costs increased by 40 percent from first estimates--
both of which are higher than the corresponding increases in DOD's
fiscal year 2000 portfolio. In most cases, the programs we assessed
failed to deliver capabilities when promised--often forcing warfighters
to spend additional funds on maintaining legacy systems. Our analysis
shows that current programs are experiencing, on average, a 21-month
delay in delivering initial capabilities to the warfighter, a 5-month
increase over fiscal year 2000 programs.
Several underlying systemic problems at the strategic level and at the
program level continue to contribute to poor weapon system program
outcomes. At the strategic level, DOD does not prioritize weapon system
investments and the department's processes for matching warfighter
needs with resources are fragmented and broken. Furthermore, the
requirements and acquisition processes are not agile enough to support
programs that can meet current operational requirements. At the program
level, programs are started without knowing what resources will truly
be needed and are managed with lower levels of product knowledge at
critical junctures than expected under best practices standards. In the
absence of such knowledge, managers rely heavily on assumptions about
system requirements, technology, and design maturity, which are
consistently too optimistic. This exposes programs to significant and
unnecessary technology, design, and production risks, and ultimately
damaging cost growth and schedule delays. DOD officials are rarely held
accountable for these poor outcomes and the acquisition environment
does not provide the appropriate incentives for contractors to stay
within cost and schedule targets, making them a strong enabler of the
status quo.
Recent congressionally mandated changes to the DOD acquisition system,
as well as initiatives being pursued by the department, include
elements that could improve DOD's overall investment strategy and the
soundness of the programs it allows to move forward. However, it is
still too early to determine the impact those changes have had on
programs. Recognizing the need for more discipline and accountability
in the acquisition process, Congress enacted legislation that requires
decision-makers to certify that programs meet specific criteria at key
decision points early in the acquisition process, and are measured
against their original baseline estimates for the purpose of assessing
and reporting unit cost growth. Recent legislation also requires DOD to
report on its strategies for balancing the allocation of funds and
other resources among major defense acquisition programs and to
identify strategies for enhancing the role of program managers in
carrying out acquisition programs. DOD has begun several policy
initiatives including a new concept decision review initiative,
acquisition approaches with shorter and more certain delivery time
frames, a requirement for more prototyping early in programs, and the
establishment of review boards to monitor weapon system configuration
changes, which are designed to enable key department leaders to make
informed decisions before a program starts and maintain discipline once
it begins.
While legislation and policy revisions can help guide change, DOD must
begin making better choices that reflect joint capability needs and
match requirements with resources or the department will continue to
experience poor acquisition outcomes. DOD and the military services
cannot continue to view success through the prism of securing the
funding needed to start and sustain new programs. Sound programs should
be the natural outgrowth of a disciplined knowledge-based process.
DOD's policy emphasizes the importance of a knowledge-based approach,
but practice does not always follow policy. The transitory nature of
leadership and the stovepiped process further undermines successful
reform. Meaningful and lasting reform will not be achieved until the
right incentives are established and accountability is bolstered at all
levels of the acquisition process--both within the department and in
the defense industry. Finally, unless all of the players involved with
acquisitions--the Congress, DOD, and perhaps most importantly, the
military services--have unified goals, outcomes are not likely to
improve.
DOD Has Too Many Acquisition Programs Competing for Limited Resources,
while Program Costs and Schedules Continue To Increase:
DOD's portfolio of major acquisition programs has grown at a pace that
far exceeds available resources. From 1992 to 2007, the estimated
acquisition costs needed to complete the major acquisition programs in
DOD's portfolio increased almost 120 percent, while the funding
provided for these programs only increased 57 percent, creating a
fiscal bow wave that may be unsustainable (see fig. 1).
Figure 1: Costs Remaining Versus Annual Appropriations for Major
Defense Acquisitions (Dollars in billions):
[See PDF for image]
Fiscal year: 1992;
Annual RDTE and Procurement Appropriations: $100;
Costs Remaining for Major Defense Acquisitions: $324.
Fiscal year: 1993;
Annual RDTE and Procurement Appropriations: $91;
Costs Remaining for Major Defense Acquisitions: $271.
Fiscal year: 1994;
Annual RDTE and Procurement Appropriations: $79;
Costs Remaining for Major Defense Acquisitions: $253.
Fiscal year: 1995;
Annual RDTE and Procurement Appropriations: $78;
Costs Remaining for Major Defense Acquisitions: $341.
Fiscal year: 1996;
Annual RDTE and Procurement Appropriations: $78;
Costs Remaining for Major Defense Acquisitions: $345.
Fiscal year: 1997;
Annual RDTE and Procurement Appropriations: $79;
Costs Remaining for Major Defense Acquisitions: $331.
Fiscal year: 1998;
Annual RDTE and Procurement Appropriations: $82;
Costs Remaining for Major Defense Acquisitions: $280.
Fiscal year: 1999;
Annual RDTE and Procurement Appropriations: $89;
Costs Remaining for Major Defense Acquisitions: $233.
Fiscal year: 2000;
Annual RDTE and Procurement Appropriations: $94;
Costs Remaining for Major Defense Acquisitions: $309.
Fiscal year: 2001;
Annual RDTE and Procurement Appropriations: $104;
Costs Remaining for Major Defense Acquisitions: $234.
Fiscal year: 2002;
Annual RDTE and Procurement Appropriations: $111;
Costs Remaining for Major Defense Acquisitions: $586.
Fiscal year: 2003;
Annual RDTE and Procurement Appropriations: $137;
Costs Remaining for Major Defense Acquisitions: $579.
Fiscal year: 2004;
Annual RDTE and Procurement Appropriations: $148;
Costs Remaining for Major Defense Acquisitions: $722.
Fiscal year: 2005;
Annual RDTE and Procurement Appropriations: $165;
Costs Remaining for Major Defense Acquisitions: $745.
Fiscal year: 2006;
Annual RDTE and Procurement Appropriations: $157;
Costs Remaining for Major Defense Acquisitions: $841.
Fiscal year: 2007;
Annual RDTE and Procurement Appropriations: $160;
Costs Remaining for Major Defense Acquisitions: $767.
Source: DOD (data); GAO (analysis and presentation).
[End of figure]
The total acquisition cost of DOD's 2007 portfolio of major programs
under development or in production has grown by nearly $300 billion
over initial estimates. While DOD is committing substantially more
investment dollars to develop and procure new weapon systems, our
analysis shows that the 2007 portfolio is experiencing greater cost
growth and schedule delays than the fiscal years 2000 and 2005
portfolios (see table 1).[Footnote 1] For example, total acquisition
costs for programs in DOD's fiscal year 2007 portfolio have increased
26 percent from first estimates--compared to a 6-percent increase for
programs in its fiscal year 2000 portfolio. We found a similar trend
for total RDT&E costs and unit costs.
Table 2: Analysis of DOD Major Defense Acquisition Program Portfolios
(Fiscal year 2008 dollars):
Portfolio size: Number of programs;
Fiscal year 2000 portfolio: 75;
Fiscal year 2005 portfolio: 91;
Fiscal year 2007 portfolio: 95.
Portfolio size: Total planned commitments;
Fiscal year 2000 portfolio: $790 Billion;
Fiscal year 2005 portfolio: $1.5 Trillion;
Fiscal year 2007 portfolio: $1.6 Trillion.
Portfolio size: Commitments outstanding;
Fiscal year 2000 portfolio: $380 Billion;
Fiscal year 2005 portfolio: $887 Billion;
Fiscal year 2007 portfolio: $858 Billion.
Portfolio performance: Change to total RDT&E costs from first estimate;
Fiscal year 2000 portfolio: 27 percent;
Fiscal year 2005 portfolio: 33 percent;
Fiscal year 2007 portfolio: 40 percent.
Portfolio performance: Change in total acquisition cost from first
estimate;
Fiscal year 2000 portfolio: 6 percent;
Fiscal year 2005 portfolio: 18 percent;
Fiscal year 2007 portfolio: 26 percent.
Portfolio performance: Estimated total acquisition cost growth;
Fiscal year 2000 portfolio: $42 Billion;
Fiscal year 2005 portfolio: $202 Billion;
Fiscal year 2007 portfolio: $295 Billion.
Portfolio performance: Share of programs with 25 percent or more
increase in program acquisition unit cost;
Fiscal year 2000 portfolio: 37 percent;
Fiscal year 2005 portfolio: 44 percent;
Fiscal year 2007 portfolio: 44 percent.
Portfolio performance: Average schedule delay in delivering initial
capabilities;
Fiscal year 2000 portfolio: 16 months;
Fiscal year 2005 portfolio: 17 months;
Fiscal year 2007 portfolio: 21 months.
Source: GAO analysis of DOD data.
Note: Data were obtained from DOD's Selected Acquisition Reports (dated
December 1999, 2004, and 2006) or, in a few cases, data were obtained
directly from program offices. Number of programs reflects the programs
with Selected Acquisition Reports. In our analysis we have broken a few
Selected Acquisition Report programs (such as Missile Defense Agency
systems) into smaller elements or programs. Not all programs had
comparative cost and schedule data, and these programs were excluded
from the analysis where appropriate. Also, data do not include full
costs of developing Missile Defense Agency systems.
[End of table]
Continued cost growth results in less funding being available for other
DOD priorities and programs, while continued failure to deliver weapon
systems on time delays providing critical capabilities to the
warfighter. Put simply, cost growth reduces DOD's buying power. As
program costs increase, DOD must request more funding to cover the
overruns, make trade-offs with existing programs, delay the start of
new programs, or take funds from other accounts. Delays in providing
capabilities to the warfighter result in the need to operate costly
legacy systems longer than expected, find alternatives to fill
capability gaps, or go without the capability. The warfighter's urgent
need for the new weapon system is often cited when the case is first
made for developing and producing the system. However, DOD has already
missed fielding dates for many programs and many others are behind
schedule. On average, the current portfolio of programs has experienced
a 21-month delay in delivering initial operational capability to the
warfighter, and 14 percent are more than 4 years late.
Fragmented Processes, Unexecutable Business Cases, and Limited
Accountability Underlie Poor Acquisition Outcomes:
Poor program execution contributes to and flows from shortfalls in
DOD's requirements and resource allocation processes. Over the past
several years our work has highlighted a number of underlying systemic
causes for cost growth and schedule delays both at the strategic and at
the program level. At the strategic level, DOD's processes for
identifying warfighter needs, allocating resources, and developing and
procuring weapon systems--which together define DOD's overall weapon
system investment strategy--are fragmented and broken. At the program
level, the military services propose and DOD approves programs without
adequate knowledge about requirements and the resources needed to
successfully execute the program within cost, schedule, and performance
targets. In addition, DOD officials are rarely held accountable for
poor decisions or poor program outcomes.
Key Acquisition Support Processes Are Fragmented and Result in Unsound
Programs:
DOD largely continues to define war fighting needs and make investment
decisions on a service-by-service basis, and assess these requirements
and their funding implications under separate decision-making
processes. While DOD's requirements process provides a framework for
reviewing and validating needs, it does not adequately prioritize those
needs and is not agile enough to meet changing warfighter demands. A
senior Army acquisition official recently testified before Congress
that because the process can take more than a year, it is not suitable
for meeting urgent needs related to ongoing operations; and a recent
study by the Center for Strategic and International Studies indicates
that the process is unwieldy and officials are now trying to find ways
to work around it. Ultimately, the process produces more demand for new
programs than available resources can support. This imbalance promotes
an unhealthy competition for funds that encourages programs to pursue
overly ambitious capabilities, develop unrealistically low cost
estimates and optimistic schedules, and to suppress bad news.
Similarly, DOD's funding process does not produce an accurate picture
of the department's future resource needs for individual programs--in
large part because it allows programs to go forward with unreliable
cost estimates and lengthy development cycles--not a sound basis for
allocating resources and ensuring program stability. Invariably, DOD
and the Congress end up continually shifting funds to and from
programs--undermining well-performing programs to pay for poorly
performing ones.
Initiating Programs with Unexecutable Business Cases Sets Them Up to
Fail:
At the program level, the key cause of poor outcomes is the consistent
lack of disciplined analysis that would provide an understanding of
what it would take to field a weapon system before system development.
Our body of work in best practices has found that an executable
business case is one that provides demonstrated evidence that (1) the
identified needs are real and necessary and that they can best be met
with the chosen concept and (2) the chosen concept can be developed and
produced within existing resources--including technologies, funding,
time, and management capacity. Although DOD has taken steps to revise
its acquisition policies and guidance to reflect the benefits of a
knowledge-based approach, we have found no evidence of widespread
adoption of such an approach in the department. Our most recent
assessment of major weapon systems found that the vast majority of
programs began development with unexecutable business cases, and did
not attain, or plan to achieve, adequate levels of knowledge before
reaching design review and production start--the two key junctures in
the process following development start (see figure 2).
Figure 1: Knowledge Achievement for Weapon System Programs in 2008
Assessment at Key Junctures:
[See PDF for image]
This figure contains three pie-charts, as well as a table, with the
following information depicted:
Key junctures: Best practices;
Development start: Knowledge point 1; Mature all critical technologies;
Design review: Knowledge point 2; Achieve knowledge point 1 on time and
complete 90 percent of engineering drawings;
Production start: Knowledge point 3; Achieve knowledge points 1 and 2,
and have all critical processes under statistical control.
Key junctures: DOD outcomes[A];
Development start: 12 percent of programs;
Design review: 4 percent of programs;
Production start: 0 percent of programs[B].
Source: GAO analysis of DOD data.
[A] Not all programs provided information for each knowledge point or
had passed through all three key junctures.
[B] In our assessment of two programs, the Light Utility Helicopter and
the Joint Cargo Aircraft, are depicted as meeting all three knowledge
points when they began at production start. We excluded these two
programs from our analysis because they were based on commercially
available products and we did not assess their knowledge attainment
with our best practices metrics.
[End of figure]
Knowledge gaps are largely the result of a lack of disciplined systems
engineering analysis prior to beginning system development. Systems
engineering translates customer needs into specific product
requirements for which requisite technological, software, engineering,
and production capabilities can be identified through requirements
analysis, design, and testing. Early systems engineering provides
knowledge that enables a developer to identify and resolve gaps before
product development begins. Because the government often does not
perform the proper up-front analysis to determine whether its needs can
be met, significant contract cost increases can occur as the scope of
the requirements change or become better understood by the government
and contractor. Not only does DOD not typically conduct disciplined
systems engineering prior to beginning system development, it has
allowed new requirements to be added well into the acquisition cycle.
The acquisition environment encourages launching ambitious product
developments that embody more technical unknowns and less knowledge
about the performance and production risks they entail. A new weapon
system is not likely to be approved unless it promises the best
capability and appears affordable within forecasted available funding
levels. We have recently reported on the negative impact that poor
systems engineering practices have had on several programs such as the
Global Hawk Unmanned Aircraft System, F-22A, Expeditionary Fighting
Vehicle, Joint Air-to-Surface Standoff Missile and others.[Footnote 2]
With high levels of uncertainty about technologies, design, and
requirements, program cost estimates and related funding needs are
often understated, effectively setting programs up for failure. We
recently assessed the service and independent cost estimates for 20
major weapon system programs and found that the independent estimate
was higher in nearly every case, but the difference between the
estimates was typically not significant. We also found that both
estimates were too low in most cases, and the knowledge needed to
develop realistic cost estimates was often lacking. For example,
program Cost Analysis Requirements Description documents--used to build
the program cost estimate--are not typically based on demonstrated
knowledge and therefore provide a shaky foundation for estimating
costs. Cost estimates have proven to be off by billions of dollars in
some of the programs we reviewed. For example, the initial Cost
Analysis Improvement Group estimate for the Expeditionary Fighting
Vehicle program was about $1.4 billion compared to a service estimate
of about $1.1 billion, but development costs for the system are now
expected to be close to $3.6 billion. Estimates this far off the mark
do not provide the necessary foundation for sufficient funding
commitments and realistic long-term planning.
Constraining development cycles would make it easier to more accurately
estimate costs, and as a result, predict the future funding needs and
effectively allocate resources. We have consistently emphasized the
need for DOD's weapon programs to establish shorter development cycles.
DOD's conventional acquisition process often requires as many as 10 or
15 years to get from program start to production. Such lengthy cycle
times promote program funding instability--especially when considering
DOD's tendency to change requirements and funding as well as frequent
changes in leadership. Constraining cycle times to 5 or 6 years would
force programs to conduct more detailed systems engineering analyses,
lend itself to fully funding programs to completion, and thereby
increase the likelihood that their requirements can be met within
established time frames and available resources. An assessment of DOD's
acquisition system commissioned by the Deputy Secretary of Defense in
2006 similarly found that programs should be time-constrained to reduce
pressure on investment accounts and increase funding stability for all
programs.
Accountability Suffers When Program Managers Lack the Authority to
Shape Programs:
When DOD consistently allows unsound, unexecutable programs to pass
through the requirements, funding, and acquisition processes,
accountability suffers. Program managers cannot be held accountable
when the programs they are handed already have a low probability of
success. In addition, program managers are not empowered to make go or
no-go decisions, have little control over funding, cannot veto new
requirements, and have little authority over staffing. At the same
time, program managers frequently change during a program's
development. Our analysis indicates that the average tenure for
managers on 39 major acquisition programs started since March 2001 was
about 17 months--less than half the length of the average system
development cycle time of 37 months. Such frequent turnover makes it
difficult to hold program managers accountable for the business cases
that they are entrusted to manage and deliver.
The government's control over and accountability for decisions is
complicated by DOD's growing reliance on technical, business, and
procurement expertise supplied by contractors. This reliance can reach
a point where the foundation on which decisions are based may be
largely crafted by individuals who are not employed by the government,
who are not bound by the same rules governing their conduct, and who
are not required to disclose whether they have financial or other
personal interests that conflict with the responsibilities they have
performing contract tasks for DOD. Further, in systems development, DOD
typically uses cost-reimbursement contracts, in which DOD generally
pays the allowable costs incurred for the contractor's best efforts, to
the extent provided by the contract. This may contribute to an
acquisition environment that is not conducive for incentivizing
contractors to follow best practices and keep cost and schedule in
check.
Recent Congressional Initiatives and DOD Actions Aim to Promote a More
Disciplined, Knowledge-Based Acquisition Approach:
Recognizing the need for more discipline and accountability in the
acquisition process, Congress recently enacted legislation that, if
followed, could result in a better chance to spend resources wisely.
Likewise, DOD has recently begun to develop several initiatives, based
in part on congressional direction and GAO recommendations that, if
implemented properly, could also provide a foundation for establishing
a well balanced investment strategy and sound, knowledge-based business
cases for individual acquisition programs.
Legislation Could Have a Positive Impact on Acquisition Outcomes:
Over the past 3 years, Congress has enacted legislation that requires
DOD to take certain actions which, if followed, could instill more
discipline into the front-end of the acquisition process when key
knowledge is gained and ultimately improve acquisition outcomes. For
example, 2006 and 2008 legislation require decision-makers to certify
that specific levels of knowledge have been demonstrated at key
decision points early in the acquisition process before programs can
enter the technology development phase or the system development phase.
The 2006 legislation also requires programs to use their original
baseline estimates--and not only their most recent estimates--when
reporting unit cost threshold breaches. It also requires an additional
assessment of the program if certain thresholds are reached. Other key
legislation requires DOD to report on the department's strategies for
balancing the allocation of funds and other resources among major
defense acquisition programs, and to identify strategies for enhancing
the role of program managers in carrying out acquisition programs. (For
more detailed description of recent legislation, see appendix I).
Recent DOD Actions Provide Opportunities for Improvement:
DOD has initiated actions aimed at improving investment decisions and
weapon system acquisition outcomes, based in part on congressional
direction and GAO recommendations. Each of the initiatives is designed
to enable more informed decisions by key department leaders well ahead
of a program's start, decisions that provide a closer match between
each program's requirements and the department's resources. For
example:
* DOD is experimenting with a new concept decision review, different
acquisition approaches according to expected fielding times, and panels
to review weapon system configuration changes that could adversely
affect program cost and schedule.
* DOD is also testing portfolio management approaches in selected
capability areas to facilitate more strategic choices about how to
allocate resources across programs and also testing the use of capital
budgeting as a potential means to stabilize program funding.
* In September 2007, the Office of the Under Secretary of Defense for
Acquisition, Technology and Logistics issued a policy memorandum to
ensure weapons acquisition programs were able to demonstrate key
knowledge elements that could inform future development and budget
decisions. This policy directed pending and future programs to include
acquisition strategies and funding that provide for contractors to
develop technically mature prototypes prior to initiating system
development, with the hope of reducing technical risk, validating
designs and cost estimates, evaluating manufacturing processes, and
refining requirements.
* DOD also plans to implement new practices that reflect past GAO
recommendations intended to provide program managers more incentives,
support, and stability. The department acknowledges that any actions
taken to improve accountability must be based on a foundation whereby
program managers can launch and manage programs toward greater
performance, rather than focusing on maintaining support and funding
for individual programs. DOD acquisition leaders have told us that any
improvements to program managers' performance hinge on the success of
these departmental initiatives.
* In addition, DOD has taken actions to strengthen the link between
award and incentive fees with desired program outcomes, which has the
potential to increase the accountability of DOD programs for fees paid
and of contractors for results achieved.
If adopted and implemented properly these actions could provide a
foundation for establishing sound, knowledge-based business cases for
individual acquisition programs, and the means for executing those
programs within established cost, schedule, and performance goals.
Concluding Observations on Achieving Successful and Lasting Reform:
DOD understands what it needs to do at the strategic and at the program
level to improve acquisition outcomes. The strategic vision of the
current Under Secretary of Defense for Acquisition, Technology and
Logistics acknowledges the need to create a high-performing, boundary-
less organization--one that seeks out new ideas and new ways of doing
business and is prepared to question requirements and traditional
processes. Past efforts have had similar goals, yet we continue to find
all too often that DOD's investment decisions are service-and program-
centric and that the military services overpromise capabilities and
underestimate costs to capture the funding needed to start and sustain
development programs. This acquisition environment has been
characterized in many different ways. For example, some have described
it as a "conspiracy of hope," in which industry is encouraged to
propose unrealistic cost estimates, optimistic performance, and
understated technical risks during the proposal process and DOD is
encouraged to accept these proposals as the foundation for new
programs. Either way, it is clear that DOD's implied definition of
success is to attract funds for new programs and to keep funds for
ongoing programs, no matter what the impact. DOD and the military
services cannot continue to view success through this prism. Adding
pressure to this environment are changes that have occurred within the
defense supplier base. In 2006, a DOD-commissioned study found that the
number of fully competent prime contractors competing for programs had
been reduced from more than 20 in 1985 to only 6. This limits DOD's
ability to maximize competition to reduce costs and encourage
innovation.
More legislation can be enacted and policies can be written, but until
DOD begins making better choices that reflect joint capability needs
and matches requirements with resources, the acquisition environment
will continue to produce poor outcomes. It should not be necessary to
take extraordinary steps to ensure needed capabilities are delivered to
the warfighter on time and within costs. Executable programs should be
the natural outgrowth of a disciplined, knowledge-based process. While
DOD's current policy supports a knowledge-based, evolutionary approach
to acquiring new weapons, in practice decisions made on individual
programs often sacrifice knowledge and realism in favor of
revolutionary solutions. Meaningful and lasting reform will not be
achieved until DOD changes the acquisition environment and the
incentives that drive the behavior of DOD decision-makers, the military
services, program managers, and the defense industry. Finally, no real
reform can be achieved without a true partnership among all these
players and the Congress.
Mr. Chairman, this concludes my prepared statement. I would be happy to
answer any questions you may have at this time.
Contacts and Staff Acknowledgments:
For further information about this statement, please contact Katherine
V. Schinasi at (202) 512-4841 or schinasik@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this testimony. Individuals who made key
contributions to this statement include Michael J. Sullivan, Director;
Ronald E. Schwenn, Assistant Director; Megan Hill; Travis J. Masters;
Karen Sloan; and Alyssa B. Weir.
[End of section]
Appendix I: Recent Legislative Initiatives:
Legislation: Section 801; National Defense Authorization Act for Fiscal
Year 2006; Pub. L. No. 109-163;
Major Components: 10 U.S.C. § 2366a (as amended) - Milestone B
Certification; Before a major defense program can receive approval to
start system development, the Milestone Decision Authority (MDA) must
certify that, for example:
* the program is affordable when considering DOD's ability to
accomplish the program's mission using alternative systems and the per
unit and total acquisition costs in the context of the Future Year
Defense Plan;
* reasonable cost and schedule estimates have been developed for system
development and production;
* appropriate market research has been conducted prior to technology
development to reduce duplication of existing technology and products;
and;
* the technology in the program has been demonstrated in a relevant
environment. MDA may waive one or more requirements if the MDA
determines that without a waiver, DOD would be unable to meet critical
national security objectives.
Legislation: Section 802; National Defense Authorization Act for Fiscal
Year 2006; Pub. L. No. 109-163;
Major Components: 10 U.S.C. § 2433 (as amended) - Unit Cost Reports;
Amended reporting and certification requirements for major defense
programs that exceed baseline costs, by:
* creating two types of growth thresholds--"significant cost growth"
and "critical cost growth";
* basing new thresholds on the percentage increases in both the
original and current baseline estimate for the program;
* incorporating these thresholds into existing unit cost reporting
requirements; and;
* requiring that in the event of a breach of the critical cost growth
threshold, the Secretary of Defense, in coordination with the Joint
Requirements Oversight Council, to (1) assess the reasons for the cost
growth, the projected cost to either complete the program with current
or reasonably modified requirements, and the rough order of magnitude
costs for a reasonable alternative system or capability and (2) certify
that the program is essential to national security; no less costly,
equally capable alternatives exist; new cost estimates are reasonable;
and an adequate management structure is in place to control costs.
Legislation: Section 853; John Warner National Defense Authorization
Act for Fiscal Year 2007; Pub. L. No. 109-364;
Major Components: Program Manager Empowerment and Accountability;
Required the Secretary of Defense to develop a strategy for enhancing
the role of DOD program managers in developing and carrying out defense
acquisition programs that addressed matters such as:
* enhanced training;
* improved career paths and opportunities;
* incentives for recruitment and retention of highly qualified
individuals;
* improved resources and support;
* increased accountability;
* enhanced monetary and non-monetary awards for successful
accomplishment of program objectives;
Required that DOD guidance for major defense programs be revised to
address program manager qualifications, resources, responsibilities,
tenure and accountability. Guidance for taking programs from
development to production was to address matters such as:
* the need for performance agreements between program managers and MDAs
that set forth expected parameters for cost, schedule and performance
and include commitments by both parties to ensure parameters are met
and;
* the extent to which a program manager should continue in the position
without interruption until the delivery of the first production units.
Legislation: Section 817; National Defense Authorization Act for Fiscal
Year 2008; Pub. L. No. 110-181;
Major Components: Investment Strategy for Major Defense Acquisition
Programs; Required the Secretary of Defense to submit to the
congressional defense committees a report on DOD's strategies for
balancing the allocation of funds and other resources among major
defense acquisition programs. The report was to address topics such as
DOD's ability to:
* establish priorities among needed capabilities and assess resources
needed to achieve such capabilities and;
* balance costs, schedule and requirements of major defense programs to
ensure the most efficient use of resources.
The report also was to address the role of a Tri-Chair Committee
comprised of the Under Secretary of Defense for Acquisition,
Technology, and Logistics; the Vice Chairman of the Joint Chiefs of
Staff; and the director of Program Analysis and Evaluation, among
others; in the resource allocation process.
Legislation: Section 943; National Defense Authorization Act for Fiscal
Year 2008; Pub. L. No. 110-181;
Major Components: 10 U.S.C. § 2366b Milestone A Certification; Before a
major defense program can receive approval to begin technology
development, the MDA must, after consulting with the Joint Requirements
Oversight Council (JROC) on matters related to program requirements and
military needs, certify that, for example:
* the system fulfills an approved initial capabilities document;
* the system is necessary and appropriate if it duplicates a capability
already provided by an existing system; and;
* the cost estimate for the system has been submitted and the level of
resources required to develop and procure the system is consistent with
the priority level assigned by the JROC.
If a milestone A certified major defense program exceeds the cost
estimate for the system submitted at the time of certification by at
least 25 percent prior to milestone B approval, the MDA and JROC shall
determine whether the level of resources required to develop and
procure the system remains consistent with the priority level assigned.
The Secretary of Defense was also asked to review guidance and take
steps to ensure that DOD does not initiate a technology development
program for a major weapon system without milestone A approval.
Source: GAO.
[End of table]
[End of section]
Related GAO Products:
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-467SP].
Washington, D.C.: March 31, 2008.
Best Practices: Increased Focus on Requirements and Oversight Needed to
Improve DOD's Acquisition Environment and Weapon System Quality.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-294]. Washington,
D.C.: Feb. 1, 2008.
Cost Assessment Guide: Best Practices for Estimating and Managing
Program Costs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-
1134SP], Washington, D.C.: July 2007.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-406SP].
Washington, D.C.: March 30, 2007.
Best Practices: An Integrated Portfolio Management Approach to Weapon
System Investments Could Improve DOD's Acquisition Outcomes.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-388], Washington,
D.C.: March 30, 2007.
Best Practices: Stronger Practices Needed to Improve DOD Technology
Transition Processes. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
06-883]. Washington, D.C.: September 14, 2006.
Best Practices: Better Support of Weapon System Program Managers Needed
to Improve Outcomes. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-
06-110]. Washington, D.C.: November 1, 2005.
Defense Acquisitions: Major Weapon Systems Continue to Experience Cost
and Schedule Problems under DOD's Revised Policy. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-06-368]. Washington, D.C.: April
13, 2006.
DOD Acquisition Outcomes: A Case for Change. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-06-257T]. Washington, D.C.:
November 15, 2005.
Defense Acquisitions: Stronger Management Practices Are Needed to
Improve DOD's Software-Intensive Weapon Acquisitions. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-04-393]. Washington, D.C.: March
1, 2004.
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems' Total Ownership Costs. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-57]. Washington, D.C.: February 11, 2003.
Defense Acquisitions: Factors Affecting Outcomes of Advanced Concept
Technology Demonstration. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-52]. Washington, D.C.: December 2, 2002.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-02-701]. Washington, D.C.: July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-469T].
Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to
Better Weapon System Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-01-288]. Washington, D.C.: March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better
Weapon System Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/NSIAD-00-199]. Washington, D.C.: July 31, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-
NSIAD-00-137]. Washington, D.C.: April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System Programs
Implement Best Practices. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/NSIAD-99-206]. Washington, D.C.: August 16, 1999.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/NSIAD-99-162]. Washington, D.C.: July 30, 1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-99-
116]. Washington, D.C.: March 17, 1999.
Defense Acquisitions: Improved Program Outcomes Are Possible.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-98-123].
Washington, D.C.: March 17, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/NSIAD-98-56]. Washington, D.C.: February 24, 1998.
Best Practices: Commercial Quality Assurance Practices Offer
Improvements for DOD. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/NSIAD-96-162]. Washington, D.C.: August 26, 1996.
[End of section]
Footnotes:
[1] Our analysis in this area reflects comparisons of performance for
programs meeting DOD's criteria for being a major defense acquisition
program in fiscal year 2007 and programs meeting the same criteria in
fiscal years 2005 and 2000. The analysis does not include all the same
systems in all 3 years.
[2] GAO, Best Practices: Increased Focus on Requirements and Oversight
Needed to Improve DOD's Acquisition Environment and Weapon System
Quality, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-294]
(Washington, D.C.: Feb. 1, 2008).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: