Defense Acquisitions
DOD's Practices and Processes for Multiyear Procurement Should Be Improved Gao ID: GAO-08-298 February 7, 2008DOD spends $10 billion annually on multiyear procurement (MYP) contracts for weapons systems. MYPs may save money through more efficient relationships with suppliers and producers, but may also suffer losses if cancelled and can limit future budget flexibility. Recently, Congress has been concerned about DOD's management of the process and savings realized by MYPs. GAO was asked to evaluate DOD's review process for MYP candidates; examine MYP program outcomes; identify the impact of changes to MYP savings threshold guidance, and determine how much DOD validates MYP performance. To do this, GAO reviewed statutes and other guidance, held discussions with relevant officials, examined DOD budget justifications and contracts, and conducted limited case studies.
DOD's process for justifying multiyear programs leaves questions about the appropriateness of some approved MYPs and the cost effectiveness of investments made for the risks assumed, as indicated by recent submissions for the F-22A and V-22. Although the law has clear requirements for stable, low risk programs with realistic cost and savings estimates, lack of guidance and a rigorous process is not achieving this. It is difficult to precisely determine the impact of multiyear contracting on procurement costs. GAO studies of three recent MYPs identified unit cost growth ranging from 10 to 30 percent compared to original estimates, due to changes in labor and material costs, requirements and funding, and other factors. In some cases, actual MYP costs were higher than estimates for annual contracts. Although annual contracts also have unit cost growth, it is arguably more problematic for MYP's because of the up-front investments and the government's exposure to risk over multiple years. MYP savings were on average higher before changes in law called for "substantial savings" rather than a specific quantitative standard. Other factors--lower quantities of modern systems procured, stricter cancellation liability allowances, and contraction in the defense industrial base--may have also impacted savings by lessening opportunities for more efficient purchases, a key attribute of MYPs. DOD does not track multiyear results against original expectations and makes little effort to validate if actual savings were achieved. GAO's case studies indicate that evaluating actual MYP results provides valuable information on the veracity of original estimates in the justification packages, the impacts on costs and risks from internal and external events, and lessons learned that can be used to improve future multiyear candidates and savings opportunities.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-298, Defense Acquisitions: DOD's Practices and Processes for Multiyear Procurement Should Be Improved
This is the accessible text file for GAO report number GAO-08-298
entitled 'Defense Acquisitions: DOD's Practices and Processes for
Multiyear Procurement Should Be Improved' which was released on
February 8, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office: GAO:
Report to the Committee on Armed Services, U.S. Senate:
February 2008:
Defense Acquisitions:
DOD's Practices and Processes for Multiyear Procurement Should Be
Improved:
GAO-08-298:
GAO Highlights:
Highlights of GAO-08-298, a report to the Committee on Armed Services,
U.S. Senate.
Why GAO Did This Study:
DOD spends $10 billion annually on multiyear procurement (MYP)
contracts for weapons systems. MYPs may save money through more
efficient relationships with suppliers and producers, but may also
suffer losses if canceled and can limit future budget flexibility.
Recently, Congress has been concerned about DOD‘s management of the
process and savings realized by MYPs. GAO was asked to evaluate DOD‘s
review process for MYP candidates; examine MYP program outcomes;
identify the impact of changes to MYP savings threshold guidance, and
determine how much DOD validates MYP performance. To do this, GAO
reviewed statutes and other guidance, held discussions with relevant
officials, examined DOD budget justifications and contracts, and
conducted limited case studies.
What GAO Found:
DOD‘s process for justifying multiyear programs leaves questions about
the appropriateness of some approved MYPs and the cost effectiveness of
investments made for the risks assumed, as indicated by recent
submissions for the F-22A and V-22. Although the law has clear
requirements for stable, low risk programs with realistic cost and
savings estimates, lack of guidance and a rigorous process is not
achieving this.
It is difficult to precisely determine the impact of multiyear
contracting on procurement costs. GAO studies of three recent MYPs
identified unit cost growth ranging from 10 to 30 percent compared to
original estimates, due to changes in labor and material costs,
requirements and funding, and other factors. In some cases, actual MYP
costs were higher than estimates for annual contracts. Although annual
contracts also have unit cost growth, it is arguably more problematic
for MYP‘s because of the up-front investments and the government‘s
exposure to risk over multiple years.
MYP savings were on average higher before changes in law called for
’substantial savings“ rather than a specific quantitative standard.
Other factors”lower quantities of modern systems procured, stricter
cancellation liability allowances, and contraction in the defense
industrial base”may have also impacted savings by lessening
opportunities for more efficient purchases, a key attribute of MYPs.
DOD does not track multiyear results against original expectations and
makes little effort to validate if actual savings were achieved. GAO‘s
case studies indicate that evaluating actual MYP results provides
valuable information on the veracity of original estimates in the
justification packages, the impacts on costs and risks from internal
and external events, and lessons learned that can be used to improve
future multiyear candidates and savings opportunities.
Multiyear Contract Dollars and Percentage of Total Defense Procurement
Obligation Authority:
[See PDF for image]
This figure is a combination line and vertical bar graph. The following
approximated data is depicted:
Fiscal year: 2000;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $50 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 18%.
Fiscal year: 2001;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $56 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 16%.
Fiscal year: 2002;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately$58 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 15%.
Fiscal year: 2003;
Total service obligation authority for mutiyear contracts:
approximately $11 billion;
Total DOD procurement budget: approximately $65 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
14%.
Fiscal year: 2004;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $80 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 12%.
Fiscal year: 2005;
Total service obligation authority for mutiyear contracts:
approximately $12 billion;
Total DOD procurement budget: approximately $78 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 14%.
Fiscal year: 2006;
Total service obligation authority for mutiyear contracts:
approximately $8 billion;
Total DOD procurement budget: approximately $74 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 12%.
Fiscal year: 2007;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $100 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 11%.
Source: GAO analysis of DOD annual reports to the Congress.
[End of figure]
What GAO Recommends:
GAO is making recommendations that DOD improve the outcomes of
multiyear justification reviews by improving guidance, providing third
party validation of MYP estimates, implementing a centralized database
for MYP information, and conducting after-action assessments of
completed MYP contracts. DOD concurred with two of the recommendations
and partially concurred with the other two, stating that it will
consider whether the delays and expense of third party validation are
warranted by the benefits and that several factors limit the usefulness
of after-action assessments. GAO believes its recommendations remain
valid and can improve MYP outcomes.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.GAO-08-298]. For more information, contact
Michael Sullivan at (202) 512-4841 or sullivanm@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD's Practices and Processes for Justifying and Approving Multiyear
Programs Could Be Improved:
Actual Results on Some Completed Contracts Were Much Different Than
Predicted in Budget Justifications:
Estimated Savings for Candidate Programs Are Trending Downward:
Multiyear Contracting Results Are Uncertain:
Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contacts and Staff Acknowledgments:
Tables:
Table 1: Statutory Requirements for Multiyear Procurement Candidates:
Table 2: Unit Cost Growth for Three Multiyear Contracts:
Table 3: Characteristics of Aircraft MYP Candidate Programs:
Figures:
Figure 1: Multiyear Contract Dollars and Percentage Share of Total
Defense Procurement Obligation Authority:
Figure 2: Contract Provisions That Impacted Case Study Multiyear
Procurements:
Figure 3: Legislative History Timeline for Multiyear Savings
Requirement:
Figure 4: Range of Estimated Savings for Multiyear Programs:
Abbreviations:
DFARS: Defense Federal Acquisition Regulation Supplement:
DOD: Department of Defense:
EOQ: economical order quantities:
FAR: Federal Acquisition Regulation:
FMR: Financial Management Regulation:
FPDS-NG: Federal Procurement Data System-Next Generation:
ICAR: Individual Contract Action Report:
IDA: Institute for Defense Analyses:
MYP: multiyear procurement:
OSD: Office of the Secretary of Defense:
USD: Under Secretary of Defense:
[End of section]
United States Government Accountability Office: Washington, DC 20548:
February 7, 2008:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
For the past 25 years, the Department of Defense (DOD) has used
multiyear contracts to procure thousands of major weapon systems,
investing about $10 billion annually in recent years. Multiyear
procurement is a special authority to contract for up to 5 program
years of requirements in one year. When used appropriately, multiyear
contracting can save money compared to a series of annual contracts by
allowing more economic procurement from suppliers and more efficient
production. Multiyear contracting can also entail some risks of
substantial losses if a program is reduced or a contract is canceled
early and can limit DOD's and the Congress' future budget flexibility.
To identify multiyear candidates that demonstrate sufficient benefits
and manageable risks, the law requires certain criteria be met before
an agency can enter into a multiyear contract.
During discussions on the President's Budget for fiscal year 2007,
committee members were concerned about the amount of projected savings
realized on multiyear contracts and had questions about the DOD's
management and oversight of programs with multiyear contracts.
Accordingly, you asked us to (1) evaluate DOD's review process for
multiyear procurement candidates submitted to Congress for approval;
(2) examine cost and program outcomes on selected multiyear programs;
(3) identify impacts from changes in guidance and interpretation of the
savings requirement; and (4) determine the extent to which DOD tracks
and validates multiyear performance.
In conducting our work, we identified statutory criteria and
implementing policies and procedures used by DOD to prepare and
evaluate multiyear justification proposal packages. We discussed
management oversight, practices, and results of the justification
process with cognizant officials from each of the military departments
and the Office of the Secretary of Defense (OSD). We examined DOD
historical budgetary and contractual records to compile and summarize
data on prior and current multiyear programs. We looked in more detail
at nine major programs representing each military department: three had
completed multiyear contracts, two were beginning multiyear contracts,
and four programs were just recently authorized by the Congress.
Several programs have been approved for more than one multiyear
contract and, collectively, these programs comprised almost one-third
of approvals granted since 1996. We researched the legislative history
on the required savings criterion and DOD's efforts to interpret and
apply it. We reviewed and discussed two recent multiyear studies
conducted by the defense research organizations, RAND National Defense
Research Institute and the Institute for Defense Analyses (IDA).
Appendix I further explains the report's scope and methodology. We
conducted this performance audit from June 2007 to February 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Results in Brief:
Multiyear contracting is big business and promises savings at some risk
to the government, yet DOD's management direction and process for
justifying multiyear programs to the Congress is limited, raising
questions about the appropriateness of some approved multiyear programs
and the cost effectiveness of investments made for the risks assumed.
We identified concerns about the relative stability and savings
potential of two recently approved programs, the F-22A Raptor and V-22
Osprey. We found differences in how officials interpreted and applied
the statutory criteria and in the methods and data used to compute
contract cost and savings. Further, few records are kept to document
decisions and supporting evidence. The statutory criteria establish
requirements for stable, low risk programs with realistic cost and
savings estimates, but DOD has not provided sufficient guidance and a
rigorous, disciplined process to ensure high quality, consistent
decisions supported by strong empirical evidence.
Although it is difficult to precisely determine the impact of multiyear
contracting on actual procurement costs, our case studies of completed
multiyear contracts for the C-17A Globemaster, F/A-18E/F Super Hornet,
and Apache Longbow Helicopter identified significant unit cost growth,
ranging from 10 to 30 percent compared to the original estimates
provided to Congress. All three programs--presumably approved based on
their stability--were significantly impacted during contract execution
by labor and material cost increases, changes in requirements and
funding, and other factors that helped drive up total contract costs
much beyond original projections. Savings also do not appear to have
materialized as expected in the budget justifications submitted to the
Congress and ultimately more funding was needed to buy the systems. In
two of the three cases, actual costs for multiyear procurement exceeded
original estimates for annual contracts. While both annual and
multiyear contracting are prone to the underestimation of costs and
overstatement of benefits as we have noted in our prior body of work on
defense acquisitions, the stakes are arguably higher for multiyear
programs because of the increased up-front investment required,
considerable cost increases if a program is significantly restructured,
and the greater liabilities incurred if multiyear programs are
canceled.
The meaning and application of the savings requirement for multiyear
contracts have evolved over time. Expectations of 10 percent savings or
more were emphasized during the 1980s, but replaced with a "substantial
savings" requirement in fiscal year 1991. While a direct cause and
effect relationship cannot be demonstrated, our analysis of multiyear
programs approved by Congress shows that estimated savings were on
average higher in the years before the "substantial savings"
requirement was established than after, although there were wide ranges
below and above 10 percent during both periods. Other factors--lower
quantities of modern systems being procured, stricter cancellation
liability allowances, and contraction in the defense industrial base--
may also decrease estimated savings for current and future systems by
lessening the opportunities for achieving benefits from economic
quantity buys, a key attribute of multiyear contracts.
DOD does not have a formal mechanism for tracking multiyear results
against original expectations and makes few efforts to validate whether
actual savings were achieved by multiyear procurement. It does not
maintain comprehensive central records and historical information that
could be used to enhance oversight and knowledge about multiyear
performance to inform and improve future multiyear procurement (MYP)
candidates. DOD and defense research centers officials said it is
difficult to assess results because of the lack of historical
information on multiyear contracts, comparable annual costs, and the
dynamic acquisition environment. Despite these limitations, our case
studies indicate that evaluating the actual results from multiyear
contracting provide valuable information regarding the veracity of
original estimates in the justification packages, the impacts on costs
and risks from internal and external events, and lessons learned that
can be used to improve future MYP candidates and savings opportunities.
GAO is making four recommendations to enhance the multiyear procurement
approval process for major DOD weapon systems that include improving
guidance related to the multiyear procurement decision criteria,
establishing a third party validation process for multiyear candidate
programs, maintaining a central database for monitoring major DOD
weapon system multiyear procurements, and conducting after-action
assessments of completed multiyear contracts used to procure major DOD
weapon systems. In written comments on our draft report, DOD concurred
with our two recommendations on improving guidance and maintaining a
central database. DOD partially concurred with our two recommendations
for third party validations and after-action assessments, stating that
they may be of value for selected, but not all, programs. We believe
that third party validations can improve the consistency and quality of
cost and savings estimates that are integral to congressional and DOD
decisions on multiyear proposals. Our review identified inconsistent
practices and varying degrees of quality and completeness in the
preparation and internal review of initial cost and savings estimates
made by the weapon system program offices. After-action assessments can
provide a continuing database of lessons learned that can benefit
future programs.
Background:
Multiyear contracting is a special authority for acquiring more than
one year's requirements --including weapon systems--under a single
contract award without having to exercise an option for each program
year after the first. It is an exception to the full-funding policy
that requires the entire procurement cost of a weapon or piece of
equipment to be funded in the year in which the item is procured. Under
a multiyear procurement, DOD can contract for up to 5 years of
quantities, although funding is still appropriated on an annual basis.
Multiyear procurement can potentially save money and improve the
defense industrial base by permitting the more efficient use of a
contractor's resources. Multiyear contracts are expected to achieve
lower unit costs compared to annual contracts through one or more of
the following sources: (1) purchase of parts and materials in economic
order quantities[Footnote 1] (EOQ), (2) improved production processes
and efficiencies, (3) better utilized industrial facilities, (4)
limited engineering changes due to design stability during the
multiyear period, and (5) cost avoidance by reducing the burden of
placing and administering annual contracts. Multiyear procurement also
offers opportunities to enhance the industrial base by providing
defense contractors a longer and more stable time horizon for planning
and investing in production and by attracting subcontractors, vendors,
and suppliers. However, multiyear procurement also entails certain
risks that must be balanced against potential benefits, such as the
increased costs to the government should the multiyear contract be
changed or canceled and decreased annual budget flexibility for the
program and across DOD's portfolio of weapon systems. Additionally,
multiyear contracts often require greater budgetary authority in the
earlier years of the procurement to economically buy parts and
materials for multiple years of production than under a series of
annual buys.
Although DOD had been entering into multiyear contracts on a limited
basis prior to the 1980s, the Department of Defense Authorization Act,
1982,[Footnote 2] codified the authority for DOD to procure on a
multiyear basis major weapon systems that meet certain criteria. Since
that time, DOD has annually submitted various weapon systems as
multiyear procurement candidates for congressional authorization. Over
the past 25 years, Congress has authorized the use of multiyear
procurement for approximately 140 acquisition programs, including some
systems approved more than once. Section 2306b of title 10, United
States Code, governs the use of multiyear contracting authority for the
procurement of property by DOD. It specifies six statutory
requirements, or criteria, that an acquisition program is expected to
meet in order to be considered for multiyear contracting. These
criteria are listed in table 1 below.
Table 1: Statutory Requirements for Multiyear Procurement Candidates:
Criteria: Substantial savings;
Description: Candidate programs should demonstrate that use of a
multiyear contract will result in substantial savings in the total
estimated costs when compared to the use of a series of annual
contracts for the same procurement.
Criteria: Stability of the requirement; Description: Candidate programs
should demonstrate that the minimum need to be purchased in terms of
total quantity, production rate, and procurement rate is expected to be
substantially unchanged during the multiyear contract period.
Criteria: Stability of funding;
Description: Candidate programs should have a reasonable expectation
that sufficient funding will be requested by DOD to carry out the
contract and avoid cancellation over the proposed multiyear contract
period.
Criteria: Stable design;
Description: Candidate programs should be able to demonstrate that they
have technical risks that are not excessive over the multiyear period
and that the items procured should be substantially unchanged during
the multiyear period.
Criteria: Realistic cost estimates;
Description: Candidate programs should be able to demonstrate realistic
estimates of contract cost and projected multiyear savings/cost
avoidance through the use of a multiyear contract strategy.
Criteria: National security;
Description: Candidate programs should be able to show that the use of
a multiyear contracting strategy will promote the national security
interests of the United States government.
Source: GAO analysis and 10 U.S.C. 2306b.
[End of table]
Expected costs to be avoided should be sufficient to offset the added
risk the government assumes with a multiyear contract in the form of a
cancellation liability, decreased flexibility in future funding
decisions and any erroneous assumptions in the estimates. Immature,
volatile programs and those at risk of future changes should not be
proposed as MYP candidates because such instability puts the savings
attributed to efficiencies of production and EOQ buying at risk. The
multiyear approach should be reserved for established production
operations and low risk technology.
In submitting candidates for multiyear authorization by the Congress,
the heads of the respective military departments vouch that each
program complies with the criteria in table 1. Additionally, the
Secretary of Defense is required to certify to Congress that the
current future years defense program fully funds the support costs
associated with the multiyear contract and that planned production will
not be less than the minimum economic rates given the existing tooling
and facilities. Multiyear contracts historically account for a
substantial share of the defense procurement dollar. Figure 1 shows
that DOD has budgeted about $10 million annually for multiyear
contracts since fiscal year 2000, accounting for more than 13 percent
of DOD's total budget for procurement over this time frame. Over the
period, the general trend shows an increase in total defense
procurement, but multiyear obligations holding fairly steady, resulting
in a downward trend of the percentage obligated on multiyear contracts.
For 2007, the large increase in total defense procurement caused a drop
below 12 percent obligated on multiyear.
Figure 1: Multiyear Contract Dollars and Percentage Share of Total
Defense Procurement Obligation Authority:
[See PDF for image]
This figure is a combination line and vertical bar graph. The following
approximated data is depicted:
Fiscal year: 2000;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $50 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 18%.
Fiscal year: 2001;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $56 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 16%.
Fiscal year: 2002;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately$58 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 15%.
Fiscal year: 2003;
Total service obligation authority for mutiyear contracts:
approximately $11 billion;
Total DOD procurement budget: approximately $65 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
14%.
Fiscal year: 2004;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $80 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 12%.
Fiscal year: 2005;
Total service obligation authority for mutiyear contracts:
approximately $12 billion;
Total DOD procurement budget: approximately $78 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 14%.
Fiscal year: 2006;
Total service obligation authority for mutiyear contracts:
approximately $8 billion;
Total DOD procurement budget: approximately $74 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 12%.
Fiscal year: 2007;
Total service obligation authority for mutiyear contracts:
approximately $10 billion;
Total DOD procurement budget: approximately $100 billion;
Multiyear contracts as a percentage of total DOD procurement budget:
approximately 11%.
Source: GAO analysis of DOD annual reports to the Congress.
[End of figure]
DOD's Practices and Processes for Justifying and Approving Multiyear
Programs Could Be Improved:
The statutory criteria for a multiyear procurement require that a
candidate program make realistic cost estimates, expect to achieve
substantial savings, and provide adequate evidence that the program is
stable in terms of funding, requirements, and design. Some recent
programs of questionable stability and savings submitted to the
Congress for multiyear authorization raise concerns about DOD's
management and controls for justifying multiyear candidates. We found
that DOD does not provide sufficient guidance and direction to ensure a
rigorous, disciplined process supported by adequate empirical data for
preparing and reviewing multiyear candidates. This increases the risk
of poor outcomes and inappropriate, unstable programs approved for
multiyear procurement.
Questions about the Appropriateness of Recently Approved Programs:
We reviewed DOD's multiyear justification data submitted in recent
defense budgets and, in particular, examined two newly approved
programs--the Air Force's F-22A fighter and the joint V-22 tilt rotor
aircraft. The F-22A acquisition has had a turbulent history with a
lengthy development period, major cost increases and quantity
decreases, changes in mission, and disagreements within DOD about the
total number required. The Air Force's submission of the F-22A for
multiyear procurement generated considerable debate over its merits and
whether it met the legal and business conditions conducive to success.
We also examined the V-22 tilt rotor aircraft multiyear proposal,
another acquisition program with a turbulent history and continuing
challenges.
Funding and Requirements Issues on F-22A Raptor Program:
The latest restructure of the F-22A acquisition occurred in December
2005. DOD extended production 2 years, added four aircraft and $1
billion in procurement funds, and proposed to buy the final 60 aircraft
under two separate 3-year multiyear contracts for the airframes and
engines. Multiyear costs and savings estimates were not completed in
time for submission with the fiscal year 2007 defense budget. The Air
Force later submitted the completed MYP justification package with
estimated total multiyear costs of $8.7 billion and projected savings
of $235 million, or 2.6 percent, compared to estimated annual
contracts. Multiyear proponents cited the projected total dollar
savings as substantial and believed there was little risk that the
remaining 60 aircraft would not be procured. On the other hand,
multiyear critics argued that the low percent of savings predicted, the
short time frame for accruing savings, and the program's relatively
unstable past made it an inappropriate multiyear candidate.
In prior work, we determined that the restructured F-22A program was
underfunded and questioned whether the proposed multiyear strategy met
statutory criteria.[Footnote 3] We identified concerns about savings,
funding, requirements, and design stability that we believed needed to
be addressed before the multiyear plan could be justified. For example,
the Air Force did not fully fund the multiyear proposal and asked for
incremental funding. Also, a major development program to add new
capabilities and improve reliability of the F-22A has begun; these
efforts could result in future design modifications which may require
retrofit onto aircraft purchased under the multiyear contract. We also
noted that having only a 3-year period of performance at the end of
production limited the ability to achieve savings normally expected
under multiyear authority such as EOQ buys and cost reduction
initiatives to improve manufacturing efficiency.
To provide for EOQ buys, Congress subsequently added $210 million to
the F-22A advance procurement budget. In authorizing a multiyear
contract for the F-22A, Congress specified certain conditions to be met
and prohibited the use of incremental funding.[Footnote 4] The
Secretary of Defense was required to certify that all statutory
requirements have been met, including the determination that the
contract will result in substantial savings of the total anticipated
costs of carrying out the program through annual contracts. DOD
submitted its certifications to the Congress in June 2007 and
subsequently awarded the F-22A multiyear contract in August 2007. For
the fiscal year 2009 budget cycle, the Air Force continues to propose
buying more aircraft than the 183 in the current defense plan.
Stability and Production Concerns about the V-22 Osprey Program:
DOD submitted the V-22 Osprey for multiyear procurement authorization
in the fiscal year 2007 President's Budget. Officials proposed a 5-year
multiyear contract to acquire 185 aircraft for about $10.1 billion.
Multiyear savings were projected at $435 million, or 4.2 percent,
compared to the estimated costs for annual contracts.
Ongoing changes in quantity, funding, design, and concerns about
production raise questions about the stability of this program and its
appropriateness for multiyear contracting. Subsequent to congressional
authorization for multiyear contracting, DOD reduced its planned
procurements quantity from 185 to 167 due to service funding
limitations with DOD cutting the proposed procurement request for
fiscal year 2009 by $234 million.
Development and test efforts continue with a number of design changes
under review to address serious safety, reliability, and performance
problems. The program office has aggressively prioritized these issues
and is making improvements to the V-22 platform by funding engineering
design changes for the correction of deficiencies. One such deficiency
is leaking hydraulic fluid causing engine compartment fires. Design
changes to fix this deficiency are being studied and implemented. In
comments on a draft of this report the program office stated it is
confident that these engineering design changes will address the
hydraulic leak problems.
To date, DOD has procured 111 aircraft in 11 years. Production aircraft
continue to be conditionally accepted with deviations and waivers.
Engineering investigations to fix these issues are not complete as the
program continues to work to minimize these deviations and waivers.
Even so, the planned production rate for the multiyear period is twice
the current fiscal year 2007 production rate of 17 V-22s. This
increase, coupled with design and production problems, raises concerns
over the contractor's ability to meet such a demand. DOD reviews and
assessments of the V-22 production ramp up have endorsed the increase
with known risks that require continued management. Officials told us
that the supplier base should be able to meet the elevated production
rate, but expressed concerns about the availability of spare parts and
the challenges in managing manufacturing and installation at three
different and dispersed facilities.
Insufficient Guidance and Management Direction by DOD to Ensure High
Quality Decisions:
The statutory criteria in section 2306b of title 10, United States
Code, establish a framework for limiting multiyear contracts to very
stable programs that appropriately balance risks with anticipated
savings. DOD's process and practices for justifying multiyear
candidates, however, provide little specific guidance on the meaning
and application of the criteria, a process that allows for subjective
interpretations about how well a particular program meets the criteria.
Cost and savings estimating techniques and data also vary considerably
and inadequate records are kept to document decisions and supporting
reasons. As a result, costs, savings, and evidence supporting stability
is not consistently prepared, reviewed, and documented.
Concerns about DOD's Processes and Practices for Justifying Multiyear
Candidates:
The statutory criteria are general in tone and qualitative by nature to
provide application over a wide range of programs. The regulations DOD
uses to implement the criteria provide contract policies and establish
a process for developing and justifying multiyear candidates.[Footnote
5] Each candidate program prepares a budget justification package,
normally for inclusion with the annual defense budget submission that
provides funding requirements and estimated cost savings expected under
a multiyear contract compared to the estimated costs for a series of
annual contracts. The justification package also includes short
statements about how programs are believed to meet each of the six
statutory criteria. Weapon system acquisition program officials prepare
the package for subsequent reviews by military service acquisition
commands, service headquarters offices, and OSD offices responsible for
program policy and budget oversight. Approved candidates are submitted
to the Congress for authorization.
We discussed DOD's justification process, multiyear contracting
policies, and management practices with OSD and military service
officials at each organizational level. We collected historical and
budget data on approved multiyear programs since fiscal year 1982 and
tracked more recent multiyear candidates through the budget process. We
specifically examined nine major programs representing each military
department: three had completed multiyear contracts, two were beginning
multiyear contracts, and four programs were just recently authorized by
the Congress. Several programs have been approved for more than one
multiyear contract and, collectively, these programs accounted for 17
of the 59 approvals granted since 1996.
We found that DOD provides little in the way of supplemental guidance
to operationalize the statutory criteria by amplifying terms such as
"reasonable," "substantial," and "stable" and quantifying where
possible to provide more objectivity and rigor to the multiyear review
process. Guidance for the most part restates the statutory criteria and
establishes formats for submitting budget justification materials, but
does not provide much elucidation for interpreting and applying the
criteria and establishing internal evidence standards for demonstrating
criteria are met.[Footnote 6] From our review of justification packages
and our discussions with DOD officials responsible for generating and
reviewing multiyear justification packages, we determined that
reviewers interpret and apply criteria differently and that the methods
and data used to compute contract costs and savings and providing
evidence to document program stability vary in quality and
sophistication.
For example, officials we talked to at all levels of the review process
had different ideas and perspectives on what constituted substantial
savings when applying the criterion. An official in the Navy, for
example, expected programs to project at least 10 percent savings, but
would consider candidates under that level. Some Army officials wanted
to see 10 percent savings or hundreds of millions of dollars. An OSD
official applied a "rule of thumb" of 4 to 5 percent. He said that
programs under that amount would typically be more closely evaluated to
ensure they were viable, but that programs over that amount would
generally receive a less detailed check of reasonableness and to ensure
paperwork requirements were met. An Air Force official told us that a 5
percent savings level should be considered the floor for a genuinely
viable candidate for multiyear and a 10 percent savings level
achievable, although he cautioned that recent statutory changes to
eligible cancellation ceiling costs will likely have a negative impact
on future multiyear savings.
Further, review of the justification packages for the F-22A and the V-
22, both submitted in the same fiscal year for approval, indicated
differences in how the design stability criterion was applied. Initial
operational capability, an important milestone for stability, had been
declared 2 years prior to requesting multiyear authority on the F-22A,
while initial operational capabilities for both variants of the V-22
are not scheduled to be achieved until after multiyear approval was
granted, at least 2 years later for the Navy's variant.
Comparing multiyear savings and judging reasonableness of the estimates
is complicated because the techniques and data used to estimate cost
savings can vary substantially in form, sophistication, and detail. For
example, for its first multiyear contract proposal on the C-17A, the
Air Force simply used the prime contractor's offer to save 5.5 percent,
a figure that was considered a "management challenge" the contractor
believed it could meet. We found other instances where it was unclear
what data was used in formulating the savings estimates. For example,
the Army's budget justifications for the Bradley Fighting Vehicle and
Abrams Tank estimated savings at 5 percent and 10 percent,
respectively, and provided very sketchy details to document how the
savings estimates were derived and compliance with the stability
criteria.
On the other hand, some analyses can be very involved, provide a range
of estimated savings, and use independent third parties to validate
data. For example, the Navy's F-18E/F cost estimates and methodologies
were independently verified by OSD's Cost Analysis Improvement Group.
For the F-22A multiyear proposal, two defense research centers
developed contract costs and savings estimates. Several very different
methodologies were employed, including cost improvement curves based on
historical actual production costs, production cost estimate models for
single year contracts less reductions for expected savings, and
summation of savings initiatives from the prime and subcontractors.
These studies produced a wide range of potential savings, from $235
million to $643 million.
Also, cost and savings estimates in general may be subject to biases
and other factors that impact their fidelity and reliability. Our
extensive past body of work on DOD's major acquisitions suggests that
to gain approval and continued funding for a weapon system, the
acquisition environment encourages programs to submit overoptimistic
estimates about a weapon system's readiness for production and to
underestimate its costs. Systems therefore appear more affordable from
an investment perspective and can fit within forecasts of available
funds. These circumstances invariably lead to acquisition programs
costing substantially more than originally estimated.[Footnote 7]
Furthermore, prior reports have discussed the importance of using
present value analysis to account for the time value of money when
evaluating and comparing costs of alternative annual and multiyear
contracts. The timing of government expenditures is expected to differ
with a multiyear contract expected to have relatively more up-front
costs (to fund EOQ for example) and lower costs in the outyears
compared to a series of annual contracts.[Footnote 8] Although the
justification packages are required to have a present value analysis of
the savings estimates, according to an OSD official and a defense
research study, the cost savings estimates in then-year dollars are the
primary estimates used in making cost decisions. Our review of the
justification packages appear to confirm this because the section of
the multiyear exhibit highlighting the benefit to the government
contained only the then-year dollar estimate rather than the present
value estimate.
Documentation and Record-Keeping Deficiencies:
Through our discussions with officials and inspection of records, we
determined that DOD's review process for the multiyear justification
budget packages does not adequately capture important information and
events to document decisions and help ensure that consistent and
reliable determinations are made regarding multiyear criteria. Once
approved, OSD officials stated few records are kept on multiyear
programs regarding how they determined whether multiyear candidates met
the six statutory criteria. According to OSD and service officials,
much of the discussion on a program proposed for multiyear should have
already taken place during regular executive-level reviews of major
weapon systems and been agreed upon before the multiyear justification
package is reviewed for submission in the budget request. Review of the
justification package then essentially becomes a paperwork formality
rarely involving any surprises. Also, we found programs can be proposed
"out of cycle" with the President's Budget submission--as in the case
of the F-22A, and may not be included in the budget details that could
affect the review path the multiyear candidate takes to obtain
approval. Without maintaining records that document decisions and the
data supporting them, it is difficult to ensure the quality and
comprehensiveness of stakeholder reviews based on the criteria,
fidelity of the data used, and supporting rationales for decisions.
Finally, officials at every level of the multiyear justification
process--from program offices, through higher headquarters, and on to
primary OSD action offices--indicated that they recently were appointed
to their current positions or the person responsible during the
multiyear justification process was no longer in that position. We
believe this contributed to "knowledge gaps," historical record-keeping
deficiencies, and differences in interpretation and application of
multiyear decision criteria. Turnover implies loss of experience and
corporate knowledge; in this environment, improved and more definitive
guidance and retention of comprehensive historical records is even more
important.
Actual Results on Some Completed Contracts Were Much Different Than
Predicted in Budget Justifications:
Implementing the statutory criteria requires realistic estimates of
multiyear and annual contract costs. This requirement provides fidelity
to savings projections and allows for accurate estimates of funding
requirements over the life of the multiyear contract. We reviewed cost
performance and results on completed multiyear contracts for the Air
Force's C-17A Globemaster transport, the Navy's F/A-18E/F Super Hornet
fighter, and the Army's Apache Longbow helicopter.[Footnote 9] Although
the precise impact of multiyear contracting is difficult to determine,
our analysis shows that these programs--presumably approved based on
their demonstrated stability--experienced substantial changes during
contract execution. These changes significantly increased unit costs
and drove up total funding requirements much beyond the estimates
submitted to Congress in the budget justification materials. Each was
also impacted by contract provisions and changes in business
conditions.
Cost Increases Lead to Questions about Realism of Budget Estimates Used
to Justify Multiyear:
We found that unit cost growth on these programs ranged from 10 to 30
percent more than projected by the budget justification data. Table 2
shows the growth in unit and total contract costs. We also found that,
for two of the three programs, actual multiyear contract costs exceeded
the original estimates for annual contract costs. The third program,
the F/A-18E/F, came in below annual estimates, but also bought fewer
systems than planned. This reduction in quantity would have also likely
decreased annual costs had that alternative been selected.
Table 2: Unit Cost Growth for Three Multiyear Contracts:
Weapon system: C-17A;
Original savings projection: 5.5%;
Original quantity: 80;
Final quantity: 80;
Original MYP contract estimated: $14,354.0;
Original unit cost: $179.43;
Final MYP contract value: $16,614.0;
Final unit cost: $207.68;
Unit cost increase (percent): 15.7%.
Weapon system: F/A-18E/F;
Original savings projection: 7.4%;
Original quantity: 222;
Final quantity: 210;
Original MYP contract estimated: 8,840.8;
Original unit cost: 39.82;
Final MYP contract value: 9,221.8;
Final unit cost: 43.91;
Unit cost increase (percent): 10.3%.
Weapon system: Apache;
Original savings projection: 7.6%;
Original quantity: 232;
Final quantity: 232;
Original MYP contract estimated: 1,596.4;
Original unit cost: 6.88;
Final MYP contract value: 2,078.8;
Final unit cost: 8.96;
Unit cost increase (percent): 30.2%.
Source: GAO analysis of MYP justification packages and selected
acquisition reports.
[End of table]
Substantial cost increases for completed multiyear contracts on the
three programs meant that Congress had to eventually provide
considerably more funding than originally budgeted. We do not know how
cost growth affected the level of savings achieved, if any, because we
do not know how an alternative series of annual contracts would have
fared. In comments on a draft of this report, DOD officials stressed
that cost growth due to labor and material price escalation under a
multiyear contract would likely have also occurred under an alternative
series of annual contracts. The final MYP contract values in table 3
also include price increases resulting from engineering design changes
made to the baseline weapon system. Although these factors may limit
the ability to make inferences about the level of savings achieved, a
case could be made that multiyear savings and costs did not materialize
as presented in the multiyear justification materials.
As discussed earlier, our past body work suggests that defense
acquisition programs are prone to underestimating costs and overstating
readiness. While this tendency would apply to annual as well as
multiyear contracts, it is arguably more problematic for multiyear
contracts because of the government's increased exposure to risk over
multiple years. DOD officials agreed with us that multiyear contracting
should be held to a high standard because of its special requirements,
funding commitments, and risks.
Impacts on Cost and Performance from Contract Provisions and Other
Factors:
We also collected information on multiyear contract provisions for the
three programs with completed multiyear contracts and for the recently
awarded F-22A contract. Each of these programs awarded a fixed-price
contract for the multiyear procurement, but they were not always firm-
fixed-price contracts, which typically entail the least risk to the
government. The multiyear contracts contained standard provisions that
provided flexibility to increase or decrease costs based on inflation,
labor rate changes, and/or material cost fluctuations.[Footnote 10] The
multiyear contracts also included provisions for early cancellation,
quantity variations, and/or design changes. In some cases, the
government waived provisions for cost and pricing data, which according
to officials decreased the government's insight. Figure 2 below shows
the frequency of the various contract provisions in our case studies.
Figure 2: Contract Provisions That Impacted Case Study Multiyear
Procurements:
[See PDF for image]
This figure is a table depicting the following data:
Economic price adjustments:
C-17A MYP I: [Check];
C-17A MYP II: [Check];
F/A-18E/F MYP I: [Check];
F/A-18E/F MYP II: [Check];
Apache MYP I: [Check];
Apache MYP II: [Check];
F-22A MYP: [Check].
Variation in quantity:
C-17A MYP I: [Empty];
C-17A MYP II: [Empty];
F/A-18E/F MYP I: [Check];
F/A-18E/F MYP II: [Check];
Apache MYP I: [Empty];
Apache MYP II: [Empty];
F-22A MYP: [Empty].
Engineering changes:
C-17A MYP I: [Check];
C-17A MYP II: [Check];
F/A-18E/F MYP I: [Check];
F/A-18E/F MYP II: [Check];
Apache MYP I: [Check];
Apache MYP II: [Check];
F-22A MYP: [Check].
Cost/pricing data waivers:
C-17A MYP I: [Check];
C-17A MYP II: [Check];
F/A-18E/F MYP I: [Empty];
F/A-18E/F MYP II: [Check];
Apache MYP I: [Check];
Apache MYP II: [Check];
F-22A MYP: [Empty].
Cancellation ceiling (unfunded):
C-17A MYP I: [Check];
C-17A MYP II: [Check];
F/A-18E/F MYP I: [Check];
F/A-18E/F MYP II: [Empty];
Apache MYP I: [Check];
Apache MYP II: [Check];
F-22A MYP: [Check].
Source: DOD contract file information and officials.
[End of figure]
Case Studies Provide Details on Meeting Cost and Program Expectations:
The three case studies provide insight into multiyear contracting
expectations and realities and the internal and external factors that
affected actual execution. Each case raises some questions about the
accuracy of cost and savings estimates used to justify multiyear
procurement and the degree to which requirements, design, and funding
were stable.
C-17A Globemaster: Risky Strategy Affected Multiyear Funding Stability:
The estimated savings for the first C-17A Globemaster multiyear
contract was simply a percentage amount submitted by the contractor.
For the second multiyear contract, Air Force officials assumed the same
percent of savings and added additional savings based on the use of a
controversial funding strategy. This strategy relied on incremental
funding, advanced buys of parts, and large potential cancellation
liability to maintain a production schedule of 15 aircraft per year
even though all of these aircraft had not been fully funded. The
unfunded liability to the Air Force had the contract been canceled
eventually grew to $1.5 billion. Concerned that this incremental
funding strategy violated DOD's full-funding policy and could
potentially violate the Anti-Deficiency Act, Congress increased C-17A
procurement funding a total of $745 million in fiscal years 2003 and
2005 to fully fund all aircraft. In annual DOD appropriations acts,
Congress also has prohibited incremental funding of multiyear
contracts.[Footnote 11]
In addition, the costs for both C-17A multiyear contracts were affected
by economic price adjustments. On the first multiyear, overhead costs
were significantly increased as a result of a merger between two major
defense contractors. The Air Force subsequently paid $150 million to
cover cost increases resulting from the merger and another $50 million
to remove the clause from the contract. By the end of the multiyear
contract, unit costs had increased 15 percent. The multiyear
justification materials submitted to Congress supported a plan to buy
80 aircraft at an average cost of $179 million; instead the Air Force
eventually paid about $207 million per aircraft. During the second
multiyear contract, the contractor made large contributions to its
pension fund, which triggered the price adjustment clause and resulted
in a potential cost increase of over $530 million. The Air Force is in
the process of restructuring the contract to reduce this amount.
The Air Force also awarded two multiyear contracts for the C-17A
engine, the F117. These procurements appear to have been successful
with demonstrated stability during the multiyear period and price
breaks based on the multiyear contract. The F117 engine is a
commercially available engine with a stable design and manufacturing
process. There were no engineering or design changes; no advanced
procurement or EOQ requirements; and no cancellation ceilings
associated with either contract. The only potential cost risk was the
economic price adjustment clause, but officials stated that the actual
adjustments were not exclusively in the contractor's favor. According
to the program office, the first multiyear contract resulted in a
savings of 10 percent, more than it had originally expected. Savings
from the second multiyear contract are consistent with the original
estimate.
F/A-18E/F Super Hornet: Design and Requirement Changes Impact Costs:
For its first F/A-18E/F multiyear contract, the Navy did not award a
firm-fixed-price contract because the program was early in the
production phase and there were still ongoing design development
efforts on the airframe. During the contract period, the economic price
adjustment clause resulted in the Navy paying an additional $378
million because of labor rate and material cost increases. The first
multiyear contract also included a variation-in-quantity clause that
permitted an upward or downward adjustment of six aircraft per year.
Annual quantities and the specific mix of buys between the two models
changed more than once during the multiyear period. By the end of the
multiyear contract, the number of aircraft procured had dropped from
222 to 210 aircraft and the average unit costs had increased by 10
percent, compared to the budget estimates.
In a May 2000 report, we had questioned whether the Navy was ready to
enter into its first multiyear contract for full-rate production.
Deficiencies identified during operational testing had not been
corrected, and to avoid costly retrofitting and redesign, we believed
that corrections should be made and tested before entering into the
contract.[Footnote 12]
The Navy proposed buying another 210 Super Hornets on a second
multiyear contract, but later changed the requirement to procure 154
Super Hornets and 56 of the new E/A-18G Growler, an electronic attack
variant still in product development. The Navy's total requirement for
the Super Hornet had been reduced, and the new Growler was needed to
replace aging EA-6B aircraft in the electronic attack mission. The
follow-on multiyear also included a variation-in-quantity clause, but
this time it only covered upward adjustments. Multiyear costs and
funding were further impacted by the economic price adjustment clauses.
As in the C-17A's case, the F/A-18E/F multiyear contract was affected
by the contractor's large pension fund contribution. The Navy estimated
that it could have been obligated to pay over $1 billion, which is
nearly the same as the amount of cost savings originally estimated to
justify the multiyear contract award. However, the Navy renegotiated
the terms of the clause and restructured the contract to bring the
price adjustment down to about $152 million.
Longbow Apache: Modifications and Unplanned Work Increased Unit Costs:
The Army's Longbow Apache helicopter experienced significant cost
increases during both its multiyear procurements. Army officials stated
that increases were largely because of aircraft modifications and
unplanned work. These modifications included a voice data recorder and
an improved rotor blade assembly that would enhance operational safety.
Contract costs were also increased by additional unplanned work.
Program officials explained that it is very difficult to predict the
condition of fielded aircraft when they return to be upgraded or
remanufactured. Along with normal wear and tear, many operational
aircraft were returned with extensive corrosion and battle damage;
others had been cannibalized for parts. Remanufacturing these aircraft
required significantly more effort and funding than originally planned.
By the end of the first multiyear contract, the Apache's average unit
cost had increased by 30 percent; at the end of the second multiyear,
these costs had increased by 25 percent.
While it may be difficult to predict unusual wear and tear on a system
and it is common to incorporate new modifications over time, it is
especially problematic to roll these costs into the multiyear contract
that had been assumed stable and that had been justified based on
initial cost estimates without these new add-ons. OSD cost analysts are
studying this issue to determine the proper accounting for
modifications under multiyear contracting.
Estimated Savings for Candidate Programs Are Trending Downward:
The amount and percentage of savings expected from a multiyear contract
compared to a series of estimated annual contracts is the most visible
and perhaps the most critical criteria in the eyes of many
stakeholders. The savings requirement in definition and application has
evolved over the years. A threshold level of 10 percent savings
emphasized during the 1980s was eliminated and replaced with a
nonquantifiable requirement for "substantial savings" since fiscal year
1991, allowing wide flexibility in its interpretation.[Footnote 13]
Although a direct causal link is not demonstrated, our analysis of
multiyear programs approved by Congress shows that estimated savings
were on average higher before the substantial savings requirement than
after. Other factors--lower quantities of modern systems being
procured, stricter termination liability allowances, and contraction in
the defense industrial base--may also contribute to decreased estimated
savings for current and future systems by lessening the benefits from
large quantity buys and efficient production rates, key attributes of
multiyear contracts.
Multiyear Savings Percentages Were Generally Lower after Changes in the
Statutory Savings Criteria:
When Congress codified the authority for multiyear procurement
contracting in December 1981,[Footnote 14] there was no specific
savings criterion in the law that candidate programs had to meet.
However, the impetus behind multiyear was provided by DOD studies at
the time predicting savings averaging 10 to 20 percent. The 10 percent
figure became a savings benchmark for decision-makers in the early
1980s when judging the merits of candidate programs. In the late 1980s,
this benchmark became a threshold requirement for many candidate
programs as Congress began stipulating a 10 to 12 percent savings
amount in annual defense authorization acts for selected programs.
In November 1989, Congress decided to codify the 10 percent savings
requirement and other conditions and limitations previously imposed on
an annual basis.[Footnote 15] However, a year later, that threshold was
struck from the U.S. Code, and the requirement for "substantial
savings" was substituted after DOD had argued that a rigid threshold
limited the potential for savings on stable, low risk programs
projecting lesser savings amounts.[Footnote 16] This substantial
savings requirement has remained unchanged since November 1990, with
specific savings requirements stipulated in annual legislation for two
candidate programs approved by Congress during this time. Figure 3
illustrates the evolution of the savings requirement.
Figure 3: Legislative History Timeline for Multiyear Savings
Requirement:
[See PDF for image]
This figure is an illustration of the Legislative History Timeline for
Multiyear Savings Requirement. The following data is depicted:
April 1981:
The ’Carlucci Memo“ encourages the use of multiyear procurement because
it could result in average dollar savings of 10 to 20 percent, becoming
an unofficial MYP savings benchmark.
December 1981:
Statutory authority for multiyear procurement codified by Congress. But
the legislation did not provide any specific savings criteria requiring
instead that the savings estimates be realistic.
November 1985:
Authorization act including a requirement for a specific savings
percentage is passed by Congress. It stipulated for certain Army
programs that multiyear cost should be no more than 90 percent of
annual estimated costs.
1985 – 1988:
Defense authorization acts passed during this time each require some
sort of a savings percentage for multiyear contracts on certain major
weapon system programs.
November 1989:
Congress passes legislation that amends provisions governing the use of
multiyear contracts by adding a 10 percent savings requirement for all
multiyear procurement contracts proposed for major weapon systems.
November 1990:
Congress passes legislation that strikes out the 10 percent savings
requirement put in place the year before and replaces it with a
’substantial savings“ requirement.
1991 - Present:
The substantial savings requirement remains unchanged since it was
first codified in November 1990.
Source: GAO analysis.
[End of figure]
Our analysis of estimated savings for approved multiyear programs
determined that the average savings level trended lower after the
substantial savings criterion was adopted. Although programs have been
approved during both eras over a wide range of savings below and above
10 percent, the change in the law provided decision makers the
flexibility to propose and approve candidate programs since fiscal year
1991 with lower savings estimates on average compared to the 1980s. As
previously discussed, an unofficial rule of thumb for savings normally
expected is now down to 4 to 5 percent.
Figure 4 illustrates the range of savings and general trend as the
savings requirement evolved. It suggests that a shift did occur since
the change to substantial savings in 1991. A larger proportion of
multiyear programs with estimated savings of less than 10 percent were
approved after 1991. On the other hand, candidate programs approved
prior to 1991 show a larger proportion of savings of 10 percent or
more. This finding is supported by a defense consultant study that
calculated estimated savings averaged 13 percent for candidate programs
from 1982 to 1989.[Footnote 17]
Figure 4: Range of Estimated Savings for Multiyear Programs:
[See PDF for image]
This figure is a stacked bar graph depicting the following data:
Estimated savings percentage: less than 5;
1991-present (after the "substantial savings" requirement): 7 MYP
programs;
1982-1990 before the change in the savings requirement): 3 MYP
programs;
Total: 10 MYP programs.
Estimated savings percentage: 5-10;
1991-present (after the "substantial savings" requirement): 19 MYP
programs;
1982-1990 before the change in the savings requirement): 16 MYP
programs;
Total: 35 MYP programs.
Estimated savings percentage: 10-15[A];
1991-present (after the "substantial savings" requirement): 8 MYP
programs;
1982-1990 before the change in the savings requirement): 27 MYP
programs;
Total: 35 MYP programs.
Estimated savings percentage: greater than 15;
1991-present (after the "substantial savings" requirement): 4 MYP
programs;
1982-1990 before the change in the savings requirement): 10 MYP
programs;
Total: 14 MYP programs.
Source: GAO analysis of DOD budget and prior GAO work.
[A] The '10-15' category includes those MYP candidates with estimated
savings of exactly 10 percent.
[End of figure]
Other Factors Affecting Estimated Savings:
In addition to the revised savings requirement, other factors may also
be impacting the level of savings for current and future multiyear
programs:
* Smaller quantities procured. The higher cost for modern weapon
systems and changes in required force structure has resulted in
generally smaller procurement quantities for new systems compared to
predecessor systems. Smaller quantities of systems bought under
multiyear contracts may not provide the same opportunity to achieve
savings through such mechanisms as EOQ buys of parts and materials.
Past multiyears, on the F-16 Falcon aircraft and Black Hawk
helicopters, for example, procured hundreds of systems. In contrast,
the more recent multiyear procurement of the F/A-18E/F Super Hornet
procured less than half the F-16 quantity. Analysis of the data on MYP
candidate programs that we collected and summarized in table 3 below
showed that the median multiyear procurement quantity for aircraft MYP
candidate programs declined over 40 percent from the 1980s to the
present era with a concurrent decline in average savings. With fewer
aircraft procured during the multiyear period, savings from economic
buys and optimized production is typically smaller.
Table 3: Characteristics of Aircraft MYP Candidate Programs:
Median MYP quantity;
1980s: 252 units;
1990s-2000s: 140 units.
Median MYP savings percentage;
1980s: 10.7%;
1990s-2000s: 7.2%.
Source: GAO analysis of DOD budget justification materials and prior
GAO work.
[End of table]
* Cancellation liability changes. If a multiyear contract is canceled,
the government is liable for reimbursing the contractor for its
incurred costs up to a negotiated cancellation ceiling typically in the
contract. Until recently, DOD has been able to include both recurring
and nonrecurring costs.[Footnote 18] Multiyear contracts awarded during
the past 25 years have included a cancellation liability, but the
cancellation ceiling for the C-17A multiyear contract awarded in 2002
was considered very large with a potential liability of more than $1.5
billion. Some members of Congress were concerned by its size and
concluded that this large liability inappropriately committed the
government to a production schedule for which funding had not been
appropriated. As a result, for the past several years Congress has
limited the cancellation liability for multiyear contracts to
nonrecurring costs only.[Footnote 19] Some DOD officials expect this
change to limit savings on current and future multiyear contracts as
contractors choose not to bear financial risks previously borne by the
government. A major source of multiyear savings is the EOQ buys, and
without including these kinds of recurring costs in the cancellation
ceiling, fewer contractors may buy supplies and materials up front in
bulk to limit their risks should the multiyear contract be canceled
early.
* Declining competition. Some DOD and defense research center officials
believe that the consolidation and contraction of the defense industry
and resulting decline in competition and contraction among vendors and
suppliers, make it harder to wring savings from EOQ buys. For example,
an OSD official stated that because the F-22A was originally designed
well over a decade ago, it is now experiencing diminishing
manufacturing sources on many components as well as parts and equipment
obsolescence. Similarly, a defense research center official believes
that diminishing manufacturing sources negatively impacted the
multiyear savings potential for the F-22A.
Multiyear Contracting Results Are Uncertain:
DOD does not perform post contract assessments of completed multiyear
contracts to validate actual results and determine cost effectiveness
of its investments. Some prior studies, including GAO work, provide
some limited, but inconclusive insights into multiyear results and
benefits. OSD and the military services do not maintain adequate,
comprehensive records on historical and current multiyear contracts
that could facilitate better tracking and provide perspective for
future multiyear efforts.
DOD Does Not Have a Formal Process for Assessing and Tracking Multiyear
Contract Results:
DOD does not have a formal process for assessing the cost and
performance of completed multiyear contracts. According to DOD
officials, once a program is approved for multiyear, they do not track
subsequent performance nor validate actual results against expectations
established in the budget justification submissions. Also, they do not
have an official method to capture and share lessons learned with other
programs considering multiyear contracts that could improve prospects
for successful outcomes.
However, we did find that individual program offices may make efforts
to ascertain benefits and learn lessons that can be applied to future
multiyear submissions, but it appears that these efforts are isolated.
Based on prior multiyear experiences, F/A-18E/F Super Hornet program
officials decided to require cost and pricing data to better inform
cost estimates for their next multiyear application. Also, they based
expected savings on cost reduction initiatives rather than EOQ buys
because they believed these initiatives had a better return on
investment. Similarly, the results of our case studies discussed
earlier demonstrate that assessing actual results can glean valuable
information about contract costs and performance that can be used to
improve future multiyear outcomes.
Prior Studies of Multiyear Cost Effectiveness Show Limited and
Inconclusive Results:
Some attempts to assess historical multiyear performance have been
made, but validating actual savings is elusive. According to DOD and
defense research center officials as well as the studies they
conducted, calculating the actual cost savings from the use of a
multiyear contract and comparing results to original expectations is
very problematic for several reasons: (1) multiyear cost and other
program data is unavailable; (2) lack of comparable data on costs of
annual contracts; and (3) original assumptions change from the
justification package, such as design modifications and variations in
buy quantities, labor, and material rates.
Recent studies by two defense research centers attempted to gain a
historical perspective on actual multiyear savings achieved for past
contracts in order to provide context as to the relative level of
expected savings for the F-22A multiyear proposal. In reports, both
centers noted that the government does not collect or save data needed
to do a detailed analysis and that, once programs are approved and
implemented, important assumptions on which original savings estimates
were based often changed.
In particular, since a series of annual contracts are not executed for
the same procurement once a multiyear strategy has been adopted,
comparisons of actual multiyear costs can only be compared to
hypothetical estimates of annual contracts. Further, changes that occur
in a multiyear contract environment can also occur in an annual
contract environment, and the exact effect on the actual costs for the
annual contracts is unknown. In reviewing the literature, one center
noted that it found "very few examples of serious and methodologically
credible attempts to validate claimed savings and savings percentages
after the fact once programs had been completed," and none that
produced, in the center's opinion, definitive findings.[Footnote 20]
In the case of the F/A-18E/F program, researchers from one center felt
they had more data available on this multiyear contract than for
others, and they attempted to validate actual savings. They concluded
that while the available data supported savings in the neighborhood of
original estimates, a definitive answer would require a more detailed
analysis of EOQ data and cost reduction initiatives. Similarly, in our
1988 report on the first F-16 multiyear procurement, we concluded that
savings were likely achieved from EOQ buys in the order of magnitude
expected, but we were unable to make definitive conclusions about total
savings achieved.[Footnote 21]
Inadequate Records Maintained on Current and Historical Multiyear
Programs:
Undergirding attempts to track and assess multiyear performance is the
quality and sufficiency of data. As discussed earlier in this report,
we found that DOD at all organizational levels does not keep adequate
records on multiyear programs to document stakeholder reviews and the
empirical evidence used to justify multiyear contracts. OSD Comptroller
officials told us that, while they are the final authority on approving
multiyear candidates and the ultimate owner of the review process, they
do not track multiyear packages through the approval process and after
the final decision has been made to submit it to the Congress. DOD has
no comprehensive, central information system that records the status of
multiyear candidates in-process, candidates that have been approved or
disapproved, detailed information on how multiyear criteria were
applied, or information on specific criteria contained in the final
justification package.
DOD is not required to, nor does it maintain a central database of
historical or ongoing multiyear contracts. In response to our
inquiries, the OSD office ultimately responsible for the multiyear
procurement review process was unable to provide us the justification
packages for over half of the programs approved by Congress since 1992,
including very recent submissions. To obtain more complete and accurate
data that could be used to track performance and conduct trend
analyses, information must be compiled from many different sources,
including budgets, program office files, contractor studies, and
contracting databases. For example, our efforts to identify and track
multiyear contract information contained in two major federal
databases[Footnote 22] were only partially successful. Some valuable
multiyear contract information was readily available through these
sources. However, the type of data stored and storage format--as well
as issues pertaining to reliability, consistency, and comparability--
limited their current usefulness in tracking and evaluating multiyear
contracts. However, with some improvements particularly with
reliability, these databases could support future studies.
Conclusions:
The statutory criteria for approving the multiyear procurement of major
DOD weapon systems clearly establish requirements to limit multiyear
authorization to stable, low risk programs with realistic cost
estimates and reasonable expectation for savings. To move forward
otherwise is to accept significant risks with little chance of reward.
However, DOD does not have an adequate process with controls in place
to ensure multiyear candidates meet all the criteria and are supported
by sufficient empirical evidence. Inconsistent application of criteria,
questionable cost and savings estimates, and inadequate documentation
increase potential for approving inappropriate, unstable multiyear
programs and incurring costly, poor outcomes when plans go awry and
conditions change. Improving guidance, ensuring decisions are informed
by knowledge, and maintaining better records are critical needs, as
well as important tools for retaining corporate memory given frequent
turnover of personnel at all levels of the justification review
process. It is not possible to calculate accurately the cost to
taxpayers that has resulted from these conditions, but the lack of a
disciplined and rigorous process that demands knowledge about stability
and costs provides potential for significant waste of taxpayer dollars.
Furthermore, DOD does not track and evaluate actual performance on
multiyear contracts for major DOD weapon systems. Once a contract is
awarded for a multiyear program, little effort is made to collect data
and assess actual results to compare performance against original
expectations and to validate savings and other benefits achieved.
Assessing results could provide valuable insights and lessons learned
on prior experience and identify opportunities to improve future
multiyear procurements. Not having a clear picture of actual
performance further emphasizes the criticality of getting it right up
front by ensuring only appropriate programs go to Congress for
approval.
Therefore, despite a long history and substantial funding for major DOD
weapons system multiyear contracts, DOD does not know whether it has
gotten a reasonable return on its investments for the extra risks
incurred. Some concerns noted in this report, such as the practice of
understating costs and overselling benefits, apply also to annual
contracts, but the standards should be higher for multiyear contracts
because of the larger up-front investments required and the
government's exposure to risk should the program fail or be
substantially changed. Strengthening both the front end of the process-
-identifying and justifying good candidates--and the back end--
assessing results and gleaning lessons learned from completed
contracts--can help ensure costs and risks are adequately balanced for
new multiyear programs and improve future outcomes. Underpinning both
ends, it is important to capture and make available essential data on
multiyear decisions and subsequent performance that can be readily
accessed by stakeholders and prospective users of multiyear procurement
authority.
Recommendations:
To improve the outcomes of the multiyear justification reviews of major
DOD weapon systems, the Secretary of Defense should direct appropriate
offices within the Under Secretary of Defense (USD) (Comptroller) and
USD (Acquisition, Technology & Logistics) to: (1) improve and expand
guidance provided to military services to better define multiyear
decision criteria for major DOD weapon systems and to facilitate more
consistent, objective, and knowledge-based evaluations of these
multiyear candidates within DOD; (2) establish a process for third
party validation of the costs and savings data submitted for such
candidate programs; and (3) implement a central database for
maintaining historical records and for effectively monitoring and
tracking major DOD weapon system multiyear procurements, to include
documenting the specific decisions made by stakeholders and their
rationales for decisions.
To provide lessons learned for informing and improving future major DOD
weapon system multiyear candidate programs and to ensure DOD is earning
a sufficient return on its investments in multiyear contracts for major
DOD weapon systems, the Secretary of Defense should direct that the
responsible military service, in conjunction with appropriate elements
within OSD, conduct after-action assessments at the conclusion of all
multiyear contracts used to procure major DOD weapon systems to
determine their effectiveness in achieving predicted benefits while
managing associated risks. These assessments should identify major
deviations, if any, between the unit costs predicted in the multiyear
justification package and the unit costs actually incurred. The
assessments should also substantiate--to the extent practicable--
savings achieved and identify reasons and causes contributing to
overall performance results and attempt to isolate those issues
peculiar to the multiyear contract from those that would likely have
also affected annual contracts if a multiyear strategy had not been
employed. Internally, DOD should use the results of these assessments
to provide lessons learned to both industry and the government that can
help inform and lead to better and more supportable decisions on future
multiyear candidate programs.
Agency Comments and Our Evaluation:
DOD provided us with written comments on a draft of this report. The
comments appear in appendix II. DOD also separately provided technical
comments which we reviewed and incorporated as appropriate. In written
comments, DOD concurred with our two recommendations to improve
guidance and to implement a central database for maintaining records
and tracking multiyear programs. DOD partially concurred with the other
two recommendations.
DOD partially concurred with our recommendation to establish a process
for third party validation of the costs and savings data submitted for
candidate programs. In its comments, DOD stated that independent third
party validations of cost and savings are done on selected programs
and, in developing new guidance in response to our first
recommendation, would consider whether the benefits of requiring
validation on all programs warrant the delays and costs of validation.
Our review of five new proposals and six approved multiyear contracts
found only one such instance, and our discussions with service and OSD
officials show that third party validations are rare. We believe that
independent third party validations would result in more accurate and
comprehensive cost and savings information critical to congressional
and DOD decision making on multiyear candidates. Our review identified
inconsistent practices in preparing and reviewing multiyear proposals
and varying degrees of quality and completeness in the initial cost and
savings estimates made by the weapon system program offices. An
independent third party check would help ensure that appropriate
multiyear candidates are submitted to the Congress for approval.
DOD also partially concurred with our recommendation that after-action
assessments be conducted to provide lessons learned for informing and
improving future multiyear candidate programs. In its comments, DOD
agreed that after-action reports may be of value for certain multiyear
programs, but questioned the value for all programs because (1) the
extensive time before the assessment results are known and can be
applied; and (2) the difficulty in determining actual savings. The
intent of the recommendation is to learn lessons that can be applied to
strengthen future multiyear proposals and improve their prospects for
success. This in formulation is not time-bounded as DOD has contracted
for studies that drew useful lessons from programs many years earlier.
Collecting and distributing data on lessons learned would provide a
continuing database of knowledge for future programs. Furthermore,
while we recognize difficulties and constraints in calculating actual
savings, this does not preclude the department from making good faith
efforts that can provide valuable, albeit imperfect information. Also,
after-action assessments include more than savings calculations. As
pointed out in the report, one program assessed itself and identified
important contractual features and other factors that it used to
improve subsequent multiyear proposals. We note that the practice of
doing after-action reports is widespread in the department and used for
many different kinds of activities, including military contingency
operations and logistics functions, and that these efforts provide
planners and decision makers with critical lessons learned for applying
to and improving future actions.
We are sending copies of this report to the Secretary of Defense; and
the Secretaries of the Air Force, Army, and Navy; and the Director of
the Office of Management and Budget. We will provide copies to others
on request. This report will also be available at no charge on GAO's
Web site at [hyperlink, http://www.gao.gov]. Key contributors to this
report were Bruce Fairbairn, Assistant Director; Noah Bleicher; Matthew
Drerup; Mary Jo Lewnard; Rae Ann Sapp; and Bob Swierczek.
If you have any questions about this report or need additional
information, please call me at (202) 512-4841 (sullivanm@gao.gov).
Contact points for the offices of Congressional Relations and Public
Affairs are located on the last page of this report.
Signed by:
Michael J. Sullivan:
Director, Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
Our review was limited to major DOD weapon systems that have received
congressional approval in annual defense authorization and/or
appropriations acts to award a multiyear contract and meet the
statutory requirements identified in 10 U.S.C. § 2306b. Work was
performed at the Offices of the Secretary of Defense and the three
military service headquarters (Navy, Army, and Air Force), in
Washington, D.C.; Naval Air Systems Command, in Patuxent River,
Maryland; Aviation Missile Command, in Huntsville, Alabama;
Aeronautical Systems Center, at Wright-Patterson Air Force Base, Ohio;
Institute of Defense Analysis (IDA), Alexandria, Virginia; and RAND
National Defense Research Institute, Arlington, Virginia.
To evaluate DOD's multiyear review process, we compiled a list of the
all the candidate programs approved by Congress to use a multiyear
contract by examining DOD authorization and appropriations acts going
back to fiscal year 1982 when the statutory language in 10 U.S.C §
2306b was first enacted. We then reviewed DOD multiyear justification
packages submitted to Congress in recent defense budgets, identified
statutory criteria authorizing the use of multiyear procurement, and
considered regulatory policies and procedures used within the Services
and at OSD to prepare and evaluate multiyear justification packages. We
discussed with officials at acquisition program offices and at higher
command review levels how they interpreted and applied the statutory
criteria and guidance to evaluate the appropriateness and feasibility
of multiyear candidates. We reviewed DOD and congressional actions on
recent multiyear candidates and examined specifically two major
programs recently approved for multiyear contracting--the F-22 Raptor
and V-22 Osprey--to illustrate how this process works and address some
questions raised about the appropriateness of these candidates and data
used in the justification packages. For these systems, we extensively
drew upon GAO's work in prior and ongoing engagements.
We conducted limited case studies for selected multiyear aircraft
contracts to assess outcomes and the internal and external events
affecting performance. Because our sample of DOD aircraft multiyear
contracts was not randomly selected, we cannot project our findings to
other programs. These case studies included three major DOD weapon
systems--C-17A Globemaster, F/A-18E/F Super Hornet, and the Apache
Longbow Helicopter. These aircraft programs have fully executed at
least one multiyear contract in the recent past and also have ongoing
follow-on multiyear contracts. We reviewed the multiyear proposal
packages submitted to Congress, annual budget information, Selected
Acquisition Reports, contract file documentation, and information in
DOD contract databases for multiyear contracts awarded by these
programs. We calculated unit cost changes and identified key
programmatic and environmental changes impacting the execution of the
multiyear programs and compared these to the original projections in
their multiyear justification packages. To identify the characteristics
of multiyear contracts and how they affect the costs, risks, and
savings for selected multiyear aircraft contracts, we reviewed contract
file documentation and information on the DOD DD 350 Individual
Contract Action Report (ICAR) database and the Federal Procurement Data
System-Next Generation (FPDS-NG). As part of the case study approach,
we reviewed the types of contracts, contract clauses, and other
contract modifications to determine how they affected the unit costs
under a multiyear contract. Further, we discussed the multiyear
contracts included in our limited case study approach and the F-22A
program's August 2007 multiyear contract award with program officials
to help us assess the effects that specific contract provisions have on
unit costs during contract execution.
To research the legislative history underlying the adoption and
subsequent repeal of the 10-percent savings requirement and the current
requirement that defense multiyear procurement contracts achieve
"substantial savings," we reviewed the evolution of multiyear savings
criteria in early defense initiatives, the fiscal year 1982
codification, subsequent amendments, and the savings criteria contained
in annual authorization and appropriations acts subsequent to the
granting of multiyear authority. To make comparisons in average and
median levels of estimated savings on multiyear candidate programs as
the criteria for awarding a multiyear contract changed, we reviewed
savings estimates in the budget justification packages for multiyear
candidates submitted to Congress since 1982 and past GAO reviews of
multiyear candidates conducted in the 1980s. We were able to obtain
savings estimates for approximately 94 of the 141 approvals granted by
Congress to award a multiyear contract.
To determine the extent to which DOD tracks performance and validates
savings and other benefits actually achieved by multiyear contracts, we
evaluated the kind and extent of cost data and program information
maintained at the services and OSD, and how they use these data to
determine whether predicted savings and other benefits were actually
achieved by multiyear contracting. We also reviewed DOD policies and
guidance for estimating and validating multiyear savings, and discussed
with DOD officials practices used to estimate and monitor multiyear
savings. We discussed with DOD officials record-keeping and management
oversight requirements and reasons why they do not have a formal
process for assessing multiyear results. We reviewed two recent major
studies done by defense research firms that summarized estimated
savings on historical programs and performed case studies on selected
programs to identify key events and practices that affected ultimate
performance. We discussed with IDA and RAND officials their cost
estimating techniques and assumptions used in their F-22A multiyear
studies that supported the planned multiyear contract. We reviewed and
summarized their attempts to validate savings and other benefits from
prior multiyear programs. We also reviewed prior GAO work on selected
weapon systems and the results of our work during the 1980s when
Congress regularly asked us to review DOD's multiyear candidates.
We conducted this performance audit from June 2007 to February 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition Technology And Logistics:
3000 Defense Pentagon:
Washington DC 20301-3000:
February 1, 2008:
Mr. Michael Sullivan:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Sullivan:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-08-298, "Defense Acquisitions: DoD's Practices and
Processes for Multiyear Procurement Should Be Improved," dated December
21, 2007, (GAO Code 120649).
The Department acknowledges receipt of the draft report. DoD's written
comments are enclosed.
Signed by:
Shay D. Assad:
Director, Defense Procurement and Acquisition Policy:
Enclosure: As stated:
GAO Draft Report Dated December 21, 2007:
GAO-08-298 (GAO Code 120649):
"Defense Acquisitions: DOD's Practices And Processes For
Multiyear Procurement Should Be Improved"
Department Of Defense Comments To The GAO Recommendation:
DOD General Comments. The department is strongly committed to multiyear
(MY) procurements which (1) offer substantial savings through improved
economies and efficiencies in production processes, and better
utilization of industrial facilities; (2) enhance the attractiveness of
and competition for Government requirements at the subcontracting
level; and (3) reduce the administrative burden in the placement and
administration of contracts. A key factor in the successful use of MY
procurements is the intelligent selection of the programs.
In Appendix I: Scope & Methodology, GAO indicated that their review was
limited to major DOD weapon systems. Accordingly, the department
suggests that the recommendations only be applied to this category of
MY procurements, and not to other categories, e.g., procurements of
energy commodities and energy-related services.
The description of the "stability of funding" criteria in Table 1 of
the report should be revised to more accurately reflect the intent of
statute (10 USC 2306b(a)(3)). The phrase "the head of the agency will
request funding" rather than the phrase "sufficient funding will be
provided by DOD" better reflects the statute. Likewise, the description
of the "stable design" criteria in the same table should be revised by
substituting the phrase "technical risks that are not excessive" for
the phrase "low technical risk" (10 USC 2306b(a)(4)).
The department will also comply with the recent amendments to the
multiyear statute, 10 U.S.C. 2306b, made by section 811 of the National
Defense Authorization Act for Fiscal Year, 2008.
Recommendation 1: The GAO recommended that the Secretary of Defense
should direct appropriate offices within the Under Secretary of Defense
(USD)(Comptroller) and USD (Acquisition, Technology and Logistics) to
improve and expand guidance provided to military services to better
define MY decision criteria and to facilitate more consistent,
objective, and knowledge-based evaluations of MY candidates within the
department. (p.28/GAO Draft Report)
DOD Response: Concur. To facilitate evaluations of MY candidates at the
time they are proposed and prior to a request for MY contract
authority, guidance is needed to better define the Office of the
Secretary of Defense's review process. The guidance will result in an
enhanced deliberative process for the evaluation of multiyear
candidates, and improved historical records that document the reviews
and the supporting rationales for the decisions. The department plans
to issue this guidance within one year after the final GAO report is
published.
Recommendation 2: The GAO recommended that the Secretary of Defense
should direct appropriate offices within the USD (Comptroller) and USD
(Acquisition, Technology and Logistics) to establish a process for
third party validation of the costs and savings data submitted for
candidate programs. (p. 28/GAO Draft Report)
DOD Response. Partially concur. Costs and savings are reviewed twice;
once when MY authority is proposed and again at the conclusion of the
contract negotiations when it is determined that the costs and savings
meet the requirements of the statute.
OSD's Cost Analysis Improvement Group or one of several Federally
Funded Research and Development Centers (FFRDC) perform an independent
cost assessment on selected programs. Also, the procedures for
negotiating a major contract include consideration of independent cost
audits and assessments from the Defense Contract Management Agency and
the Defense Contract Audit Agency.
When developing OSD's guidance described in response to recommendation
1, the Department will consider whether the benefits of requiring third
party validation of preliminary estimates of costs and savings data for
all programs warrant the delays and the expense of validation, since a
final determination of statutory compliance must be made as part of the
contract negotiation and award process.
Recommendation 3: The GAO recommended that the Secretary of Defense
should direct appropriate offices within the USD (Comptroller) and USD
(Acquisition, Technology and Logistics) to implement a central database
for maintaining historical records and for effective monitoring and
tracking multiyear programs, to include documenting the specific
decisions made by stakeholders and their rationales for decisions. (p.
28/GAO Draft Report)
DOD Response. Concur. DOD will implement this action concurrent with
implementation of the guidance described in DOD response to
recommendation 1.
Recommendation 4: The GAO recommended that the Secretary of Defense
direct that the responsible military service, in conjunction with
appropriate elements within OSD, conduct an after action assessment at
the conclusion of each multiyear contract to determine its
effectiveness in achieving predicted benefits while managing associated
risks. (p. 28/GAO Draft Report)
DOD Response. Partially concur. The department concurs that an after
action assessment may be of value for certain programs. However, in
general, it is not known whether the value in performing an assessment
(e.g., lessons learned) on all MY programs outweighs (1) the extensive
time before the assessment results are known and can be applied to
similar programs; and (2) the difficulty in determining actual savings.
First, the assessment can not begin until after performance is complete
and data on actual costs is available, e.g., 7-8 years after contract
award. It is not apparent how valuable the results would be to the
department after such a long period has transpired. Second, as
indicated in the report, savings are difficult to determine due to too
many factors that may affect costs during the life of a MY contract,
e.g., changes in quantity, economic conditions, or configuration (e.g.,
to address safety, reliability, or performance problems); the impact of
cost reduction initiatives, subsequent changes in statutory and
regulatory rules that may impact costs, etc. As recognized in the
report, calculating the cost of a MY contract if these unanticipated
changes had not occurred would be especially challenging and difficult
to validate. Likewise, it is difficult to determine how an alternative
series of annual contracts would have fared under these conditions.
[End of section]
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contact:
Bruce Fairbairn, (937) 258-7952, fairbairnb@gao.gov:
Staff Acknowledgments:
Noah Bleicher; Matthew Drerup; Mary Jo Lewnard; Rae Ann Sapp; and Bob
Swierczek:
[End of section]
Footnotes:
[1] The purchase of certain materials or parts by the prime contractor
from vendors in quantities greater than those needed for any single
year of production under the multiyear contract. The goal is to
minimize the costs of these items by buying in larger more economically
efficient quantities and avoid the expenses related to additional
production line set-ups and terminations that an annual buy approach
would necessitate.
[2] Pub. L. No. 97-86, § 909 (1981).
[3] GAO, Tactical Aircraft: Questions Concerning the F-22A's Business
Case, GAO-06-991T (Washington, D.C.: July 25, 2006) and GAO, Tactical
Aircraft: DOD Should Present a New F-22A Business Case before Making
Further Investments, GAO-06-455R (Washington, D.C.: June 20, 2006).
[4] John Warner National Defense Authorization Act for Fiscal Year
2007, Pub. L. No. 109-365 §134 (2006).
[5] Principally, the Federal Acquisition Regulation (FAR), and the
Defense Federal Acquisition Regulation Supplement (DFARS), and DOD
Financial Management Regulation (FMR) 7000.14-R.
[6] At the Air Force, we identified a guidebook developed in the 1980s
that provides some details and examples that could aid officials in
deciding whether a multiyear contract would be appropriate and in
developing justification materials. We found, however, that some
officials were either unaware of this or mainly used the FAR and DFAR
guidance. Officials knew of no comparable document in the Army, Navy,
and OSD.
[7] GAO, DOD Acquisition Outcomes: A Case for Change, GAO-06-257T
(Washington, D.C.: Nov. 15, 2005) and GAO, Weapons Acquisition: A Rare
Opportunity for Lasting Change, GAO/NSIAD-93-15 (Washington, D.C.: Dec.
1, 1992).
[8] GAO, An Assessment of the Army's Multiple Launch Rocket System
Multiyear Contract, (GAO/NSIAD-86-5 Oct. 28, 1985); GAO Procurement: An
Assessment of the Air Force's F-16 Aircraft Multiyear Contract (GAO/
NSAID-86-38, February 1986); and GAO Procurement: Assessment of DOD'
Multiyear Contract Candidates (GAO/NSAID-87-202-BR August 1987).
[9] For comparability purposes and availability of data, we selected
these three aircraft programs, each with recently completed multiyear
contracts and ongoing follow-on multiyear contracts. Our findings
cannot be extended to multiyear contracting in total, but do provide
illustrations of what can happen during execution of a multiyear
contract.
[10] The Federal Acquisition Regulation (FAR) contains examples of
standard economic price adjustment clauses at FAR § 52.216-2 through 4
and value engineering at 52.248-1 through 3.
[11] See Department of Defense Appropriations Act, 2005, Pub. L. No.
108-287, § 8008 (2004), and similar provisions in subsequent DOD
appropriations acts.
[12] GAO, Defense Acquisitions: F/A-18E/F Aircraft Does Not Meet All
Criteria for Multiyear Procurement, GAO/NSIAD-00-158 (Washington, D.C.:
May 26, 2000).
[13] The Conference Report for the National Defense Authorization Act
for Fiscal Year 2008 indicates a preference for a 10 percent estimated
savings level for future multiyear candidates. See H.R. Conf. Rep. No.
110-447, at 951 (2007).
[14] Department of Defense Authorization Act, 1982, Pub. L. No. 97-86,
§ 909 (1981).
[15] National Defense Authorization Act for Fiscal Years 1990 and 1991,
Pub. L. No. 101-189, § 805 (1989).
[16] National Defense Authorization Act for Fiscal Year 1991, Pub. L.
No. 101-510, § 808 (1990).
[17] Institute for Defense Analyses, F/A-22 Independent Cost Estimate
(August 2005).
[18] Recurring costs are production costs, such as labor and materials
that vary with the quantity being produced. Nonrecurring costs are
fixed expenses that do not vary with the quantity, such as capital
investments in facilities, equipment, and tooling.
[19] See Department of Defense Appropriations Act, 2005, Pub. L. No.
108-287, § 8008 (2004) and similar provisions in subsequent DOD
appropriations acts.
[20] RAND Corporation, National Defense Research Institute, F-22A
Multiyear Procurement Program: An Assessment of Cost Savings, 2007.
[21] GAO, Procurement: An Assessment of the Air Force's F-16 Aircraft
Multiyear Contract, GAO/NSIAD-86-39 (Washington, D.C.: Feb. 20, 1986).
[22] The two databases were the DOD DD350 Individual Contract Action
Report, and the Federal Procurement Data System-Next Generation (FPDS-
NG).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
