Defense Acquisitions
Issues Concerning Airlift and Tanker Programs Gao ID: GAO-07-566T March 7, 2007The Department of Defense (DOD) has continuing efforts to modernize its airlift and tanker fleets by investing billions of dollars to modify legacy airlift systems, such as the C-5 and C-130, and procure new aircraft, such as a tanker replacement. Acquisition has been on GAO's list as a high risk area since 1990. GAO has reported that elements contributing to a sound business case for an acquisition are missing or incomplete as DOD and the services attempt to acquire new capabilities. Those elements include firm requirements, mature technologies, a knowledge-based acquisition strategy, a realistic cost estimate, and sufficient funding. Acquisition problems that include failure to limit cost growth, schedule delays, and quantity reductions persist, but fiscal realities will not allow budgets to accommodate these problems any longer. Today's testimony addresses (1) the analyses supporting the Department of Defense's (DOD) mobility capabilities and requirements and (2) actions that are needed to improve the outcomes of weapon system acquisitions. For this testimony, GAO drew from issued reports, containing statements of the scope and methodology used, as well as recently completed work not yet reported. GAO's work was performed in accordance with generally accepted government auditing standards.
Past GAO reports, including two recently issued, raise concerns about the quality of analyses underpinning the programmatic decision-making surrounding DOD's airlift requirements. In September 2006, GAO issued our report (GAO-06-938) on DOD's Mobility Capabilities Study (MCS). The MCS determined that the projected mobility capabilities are adequate to achieve U.S. objectives with an acceptable level of risk during the period from fiscal years 2007 through 2013; that is, the current U.S. inventory of aircraft, ships, prepositioned assets, and other capabilities are sufficient, in conjunction with host nation support. GAO's report stated that conclusions of the MCS were based on incomplete data and inadequate modeling and metrics that did not fully measure stress on the transportation system. GAO further observed that the MCS results were incomplete, unclear, or contingent on further study, making it difficult to identify findings and evaluate evidence. It was not clear how the analyses done for the study support DOD's conclusions and GAO suggested that decision makers exercise caution in using the results of this study to make programmatic decisions. In March 2007, GAO reported (GAO-07-367R) on the lack of mandatory analyses to support a passenger and cargo capability for the new replacement refueling aircraft, the KC-X tanker. Contrary to mandatory Air Force implementing guidance, the Air Force proposed a capability without analyses identifying an associated gap, shortfall, or redundancy. GAO believes that without sound analyses, the Air Force may be at risk of spending several billion dollars unnecessarily for a capability that may not be needed to meet a gap or shortfall and made recommendations to the Secretary of Defense that included conducting the requiring analyses necessary to establish capabilities. Successful acquisition programs make sound decisions based on critical product knowledge to ensure that program investments are getting promised returns--on time delivery, within estimated costs, and with expected capabilities. However, GAO has shown in its work that DOD practices diverge from best development practices intended to produce good outcomes and, as a result, have experienced significant cost growth and schedule delays. DOD expects to invest over $12 billion in new and improved capabilities in four airlift programs discussed in this testimony between now and 2013--C-5 Avionics Modernization Program, C-5 Reliability Enhancement and Reengining Program, C-130 Avionics Modernization Program, and the C-130J acquisition program. GAO found that all four programs failed at basic systems engineering practices to 1) fully analyze the resources needed to integrate proven commercial technologies, 2) achieve a stable design before beginning system demonstration, and 3) demonstrate the aircraft would work as required before making large production investments.
GAO-07-566T, Defense Acquisitions: Issues Concerning Airlift and Tanker Programs
This is the accessible text file for GAO report number GAO-07-566T
entitled 'Defense Acquisitions: Issues Concerning Airlift and Tanker
Programs' which was released on March 8, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the House Committee on Armed Services, Air and Land Forces
Subcommittee:
United States Government Accountability Office:
GAO:
For Release on Delivery Expected at 3:00 p.m. EST:
Wednesday, March 7, 2007:
Defense Acquisitions:
Issues Concerning Airlift and Tanker Programs:
Statement of William M. Solis, Director:
Defense Capabilities and Management Issues and:
Michael J. Sullivan, Director:
Acquisition and Sourcing Management Issues:
GAO-07-566T:
GAO Highlights:
Highlights of GAO-07-566T, a testimony to House Committee on Armed
Services, Air and Land Forces Subcommittee
Why GAO Did This Study:
The Department of Defense (DOD) has continuing efforts to modernize its
airlift and tanker fleets by investing billions of dollars to modify
legacy airlift systems, such as the C-5 and C-130, and procure new
aircraft, such as a tanker replacement. Acquisition has been on GAO‘s
list as a high risk area since 1990. GAO has reported that elements
contributing to a sound business case for an acquisition are missing or
incomplete as DOD and the services attempt to acquire new capabilities.
Those elements include firm requirements, mature technologies, a
knowledge-based acquisition strategy, a realistic cost estimate, and
sufficient funding. Acquisition problems that include failure to limit
cost growth, schedule delays, and quantity reductions persist, but
fiscal realities will not allow budgets to accommodate these problems
any longer.
Today‘s testimony addresses
(1) the analyses supporting the Department of Defense‘s (DOD) mobility
capabilities and requirements and (2) actions that are needed to
improve the outcomes of weapon system acquisitions.
For this testimony, GAO drew from issued reports, containing statements
of the scope and methodology used, as well as recently completed work
not yet reported. GAO‘s work was performed in accordance with generally
accepted government auditing standards.
What GAO Found:
Past GAO reports, including two recently issued, raise concerns about
the quality of analyses underpinning the programmatic decision-making
surrounding DOD‘s airlift requirements. In September 2006, GAO issued
our report (GAO-06-938) on DOD‘s Mobility Capabilities Study (MCS). The
MCS determined that the projected mobility capabilities are adequate to
achieve U.S. objectives with an acceptable level of risk during the
period from fiscal years 2007 through 2013; that is, the current U.S.
inventory of aircraft, ships, prepositioned assets, and other
capabilities are sufficient, in conjunction with host nation support.
GAO‘s report stated that conclusions of the MCS were based on
incomplete data and inadequate modeling and metrics that did not fully
measure stress on the transportation system. GAO further observed that
the MCS results were incomplete, unclear, or contingent on further
study, making it difficult to identify findings and evaluate evidence.
It was not clear how the analyses done for the study support DOD‘s
conclusions and GAO suggested that decision makers exercise caution in
using the results of this study to make programmatic decisions. In
March 2007, GAO reported (GAO-07-367R) on the lack of mandatory
analyses to support a passenger and cargo capability for the new
replacement refueling aircraft, the KC-X tanker. Contrary to mandatory
Air Force implementing guidance, the Air Force proposed a capability
without analyses identifying an associated gap, shortfall, or
redundancy. GAO believes that without sound analyses, the Air Force may
be at risk of spending several billion dollars unnecessarily for a
capability that may not be needed to meet a gap or shortfall and made
recommendations to the Secretary of Defense that included conducting
the requiring analyses necessary to establish capabilities.
Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting
promised returns--on time delivery, within estimated costs, and with
expected capabilities. However, GAO has shown in its work that DOD
practices diverge from best development practices intended to produce
good outcomes and, as a result, have experienced significant cost
growth and schedule delays. DOD expects to invest over $12 billion in
new and improved capabilities in four airlift programs discussed in
this testimony between now and 2013”C-5 Avionics Modernization Program,
C-5 Reliability Enhancement and Reengining Program, C-130 Avionics
Modernization Program, and the C-130J acquisition program. GAO found
that all four programs failed at basic systems engineering practices to
1) fully analyze the resources needed to integrate proven commercial
technologies, 2) achieve a stable design before beginning system
demonstration, and 3) demonstrate the aircraft would work as required
before making large production investments.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-566T].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact William M. Solis at (202)
512-8365 or solisw@gao.gov and Michael J. Sullivan at (202)-512-4841 or
sullivanm@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
We are pleased to be here to discuss the development of aircraft
program requirements and issues related to the acquisition process. We
recently issued our report on high risk areas in the federal
government, which lists DOD's acquisition process as one longstanding
area of concern.[Footnote 1] Acquisition has been on this list since
1990. As we have reported, DOD knows what to do to achieve more
successful outcomes, but finds it difficult to apply the necessary
discipline and controls or assign much needed accountability. DOD has
not been employing a knowledge-based development approach and business
cases have not measured up. DOD has just begun piloting some corrective
actions, so the proof of actual implementation may be years away. These
initiatives also may not necessarily be applied to programs already
under way.
DOD has continuing efforts to modernize its airlift and tanker fleets
by investing billions of dollars to modify legacy airlift systems, such
as the C-5 and C-130, and procure new aircraft, such as the KC-X
replacement tanker. We have reported in the past that a sound business
case for an acquisition contains firm requirements, mature
technologies, a knowledge-based acquisition strategy, a realistic cost
estimate, and sufficient funding. However, we have found many of these
elements are missing or incomplete as DOD and the services attempt to
acquire new capabilities. Persistent acquisition problems include
failure to identify needs versus wants and to limit cost growth,
schedule delays, and quantity reductions, but fiscal realities will not
allow budgets to accommodate these problems any longer.
Today I will highlight for you some issues related to the analyses
supporting the Department of Defense's (DOD) mobility capabilities and
requirements and Mike Sullivan will discuss actions that are needed to
improve the outcomes of weapon system acquisitions. For this testimony,
we drew from issued reports, which contain statements of the scope and
methodology used, as well as recently completed work not yet reported.
Our work was performed in accordance with generally accepted government
auditing standards.
Summary:
DOD has an mandate to deliver high-quality products to warfighters,
when they need them and at a price the country can afford. However, our
work shows that acquisition problems will likely persist until DOD
provides a better foundation for buying the right things, the right
way. This foundation begins with setting requirements that are based on
adequate and complete analyses, using current operational data and
updated, effective models. For the past several years, we have reported
our concerns with the analyses done to support requirements and have
recently issued two reports that raise concerns about the quality of
analyses underpinning the programmatic decision-making surrounding
DOD's airlift requirements. In September 2006, we issued our report on
DOD's Mobility Capabilities Study (MCS). The MCS determined that the
projected mobility capabilities are adequate to achieve U.S. objectives
with an acceptable level of risk during the period from fiscal years
2007 through 2013; that is, the current U.S. inventory of aircraft,
ships, prepositioned assets, and other capabilities are sufficient, in
conjunction with host nation support. In our report, we stated that
conclusions of the MCS were based on incomplete data and inadequate
modeling and metrics that did not fully measure stress on the
transportation system.[Footnote 2] We further observed that, in some
cases, the MCS results were incomplete, unclear, or contingent on
further study, making it difficult to identify findings and evaluate
evidence. It is not clear how the analyses done for the study support
DOD's conclusions and we suggested that decision makers exercise
caution in using the results of this study to make programmatic
decisions. This week, we issued a report on the lack of mandatory
analyses to support a passenger and cargo capability for the new
replacement refueling aircraft, the KC-X tanker.[Footnote 3] Contrary
to mandatory Air Force implementing guidance, the Air Force proposed a
capability without an analyses identifying an associated gap,
shortfall, or redundancy. Air Force officials could not provide
supporting information sufficient to explain this discrepancy between
the required analyses and their proposal.
Successful acquisition programs make sound decisions based on critical
product knowledge to ensure that program investments are getting
promised returns--on time delivery, within estimated costs, and with
expected capabilities. This is important because DOD expects to invest
over $12 billion in new and improved capabilities in four airlift
programs discussed in this testimony between now and 2013--the C-5
Avionics Modernization Program, C-5 Reliability Enhancement and
Reengining Program, C-130 Avionics Modernization Program, and the C-
130J acquisition program. These four programs have diverged from the
best development practices intended to produce good outcomes and as a
result have experienced significant cost growth and schedule delays. We
found that all four programs failed at basic systems engineering
practices to 1) fully understand the resources needed to integrate
proven commercial technologies, 2) achieve a stable design before
beginning system demonstration, and 3) demonstrate the aircraft would
work as required before making large production investments. As a
result, each has encountered significant delays in delivering
capability to the field and as a group have spent $962.3 million
(fiscal year 2007 dollars) more than planned for development.
Background:
DOD must be capable of rapidly deploying armed forces to respond to
contingency and humanitarian operations around the world. Airlift and
tanker aircraft play a vital role in providing this capability. Over
the past 25 years, DOD has invested almost $141 billion to develop,
procure, and modify its airlift and tanker forces with an additional
investment planned for fiscal years 2007 through 2011 of $32 billion.
Recent annual funding levels are at the highest levels in two decades.
(See figure 1.)
Figure 1: Planned and Future DOD Investments in Airlift and Tanker
Fleets (1982 to 2011):
[See PDF for image]
Source: GAO analysis of DOD data.
Note: Based on DOD's Fiscal Years Defense Program (2007).
[End of figure]
In December 2005, DOD issued a report on the study of its mobility
capabilities. The goal of this Mobility Capabilities Study was to
identify and quantify the mobility capabilities needed to support U.S.
strategic objectives into the next decade. The MCS determined that the
projected mobility capabilities are adequate to achieve U.S. objectives
with an acceptable level of risk during the period from fiscal years
2007 through 2013; that is, the current U.S. inventory of aircraft,
ships, prepositioned assets, and other capabilities are sufficient, in
conjunction with host nation support. The MCS emphasized that continued
investment in the mobility system, in line with current departmental
priorities and planned spending, is required to maintain these
capabilities in the future. This includes, for example, fully funding
Army prepositioned assets as planned and completing a planned
reengineering of the C-5 aircraft.
In our previous reports concerning acquisition outcomes and best
practices, we have noted the importance of matching warfighter
requirements with available resources, a responsibility shared by the
requirements and acquisition communities in DOD. As described in Air
Force implementing guidance, there is within DOD a distinct separation
between the requirements authority and acquisition authority.[Footnote
4] Under this guidance, this separation requires early and continued
collaboration between both communities.
Analyses Used to Determine Mobility and Tanker Capabilities Were
Inadequate:
Analyses done for the MCS contained methodological limitations that
create concerns about the adequacy and completeness of the study and
decision makers approving the KC-X tanker proposal lacked required
analyses identifying need and associated risk for a passenger and cargo
capability.
Mobility Capabilities Study Limitations Raise Questions about Adequacy
and Completeness:
While DOD used an innovative approach in conducting the study and
acknowledged some methodological limitations in its report, it did not
fully disclose how these limitations could affect the MCS conclusions
and recommendations. In September 2006, we reported that DOD's
conclusions were based, in some instances, on incomplete data and
inadequate modeling and metrics that did not fully measure stress on
the transportation system, and that, in some cases, MCS results were
incomplete, unclear, or contingent on further study, making it
difficult to identify findings and evaluate evidence. It is not clear
how the analyses done for the study supported DOD's conclusions, and we
suggested that decision makers exercise caution in using the results of
this study to make programmatic decisions.
As measured against relevant generally accepted research standards, we
identified limitations in the MCS study and report that raise
questions. Among our findings:
* Aspects of modeling and data were inadequate in some areas because
data were lacking and some of the models used could not simulate all
relevant aspects of the missions. The report did not explain how these
limitations could affect the study results or what the effect on the
projected mobility capabilities might be. Relevant research standards
require that models used are adequate for the intended purpose and
represent a complete range of conditions, and also that data used are
properly generated and complete. For example, the MCS modeled
hypothetical homeland defense missions rather than missions for
homeland defense demands from a well-defined and approved concept of
operations for homeland defense because the specific details of the
missions were still being determined, and DOD acknowledged that the
data used may be incomplete. The MCS also was unable to model the
flexible deterrent options/deployment order process to move units and
equipment into theater due to lack of data, but the study assumed a
robust use of this process, which in one scenario accounted for
approximately 60 percent of the airlift prior to beginning combat
operations.[Footnote 5] In addition, the MCS report contains more than
80 references to the need for improved modeling, and 12 of these
references call for additional data or other refinements. Additionally,
the MCS modeled the year 2012 to determine the transportation
capabilities needed for the years 2007 through 2013. The year 2012 did
not place as much demand for mobility assets in support of smaller
military operations, such as peacekeeping, as other years. However, DOD
officials considered 2012--the year modeled--as "most likely" to occur
and stated that statistically it was not different from other years in
the 2007 to 2013 period even though the number of smaller military
operations is the least of any of the years reviewed.
As I mentioned, we have reported before on the lack of data available
for analysis that could benefit decision makers. In September 2005, we
reported that the Air Force captured data on short tons transported but
did not systematically collect and analyze information on operational
factors, such as weather and runway length, that impact how much can be
loaded on individual missions.[Footnote 6] Therefore, Air Force
officials could not know how often it met its secondary goal to use
aircraft capacity as efficiently as possible. Without this information,
Air Mobility Command officials do not know the extent to which
opportunities exist to use aircraft more efficiently and whether
operational tempo, cost, and wear and tear on aircraft could be
reduced. In addition, DOD officials do not have the benefit of such
analysis to determine future airlift requirements for planning
purposes.
* While the MCS concluded that combined U.S. and host nation
transportation assets were adequate to meet U.S. objectives with
acceptable risk, the report, in describing the use of warfighting
metrics in its analyses, does not provide a clear understanding of the
direct relationship of warfighting objectives to transportation
capabilities. Acknowledging this point, the report stated that further
analysis is required to understand the operational impact of increased
or decreased strategic lift on achieving warfighting objectives.
Relevant generally accepted research standards require that conclusions
be supported by analyses. The use of warfighting metrics is a measure
to determine whether combat tasks, such as achieving air superiority,
are achieved. However, they do not measure whether appropriate
personnel, supplies, and equipment arrived in accordance with
timelines. As a result, we could not determine how the study concluded
that planned transportation assets were adequate because the study did
not contain a transparent analysis to support its conclusion or a clear
roadmap in the report to help decision makers understand what that
conclusion meant in terms of type and number of mobility assets needed.
Previous DOD mobility studies primarily used mobility metrics, which
measured success in terms of tons of equipment and personnel moved per
day to accomplish military objectives. The use of both warfighting and
mobility metrics to measure success would allow decision makers to know
whether combat tasks were achieved and how much strategic
transportation is needed to accomplish those tasks.
* In some cases, the MCS results were incomplete, unclear, or
contingent on further study, making it difficult to identify findings
and evaluate evidence. Relevant research standards require results to
be presented in a complete, accurate, and relevant manner. For example,
the report contains several recommendations for further studies and
assessments, five of which are under way. However, at the time of our
report, DOD had no plans to report the effect of these studies on the
MCS results after the studies are complete. In addition, the report
contains qualified information that is not presented clearly, such as
varying assessments of intratheater assets in three different places in
the report. The lack of clarity and conciseness of the reported results
can limit the study's usefulness to decision makers and stakeholders.
The MCS report also made recommendations to conduct further studies,
develop plans and strategies, and improve data collection and mobility
models. In fact, DOD officials told us at the time that a Mobility
Capabilities Study-2006 was underway, as well as studies on
intratheater lift, aerial refueling, and other mobility issues.
However, unless DOD addresses the concerns I just outlined for you,
decision makers may be unable to clearly understand the operational
implications of the study results and make fully informed programmatic
investment decisions concerning mobility capabilities. Also, some of
the underlying assumptions used in the MCS have now changed
significantly, such as the assumption that Army prepositioned equipment
is in place and fully funded, which will no longer be the case.
Therefore, the MCS analyses and results, which would be the starting
point for any new studies, may no longer be relevant.
Mandatory Analyses Done to Support Passenger and Cargo Capabilities in
the KC-X Tanker Were Incomplete:
Mandatory Air Force policy requires Air Force organizations to use a
formal capabilities-based approach to identify, evaluate, develop,
field, and sustain capabilities that compete for limited resources.
Contrary to mandatory Air Force implementing guidance, however, the Air
Force proposal for a replacement refueling aircraft, the KC-X tanker,
included a passenger and cargo capability without analyses identifying
an associated gap, shortfall, or redundant capability. According to
mandatory Air Force implementing guidance, analyses supporting the
decision-making process should assess a capability based on the effects
it seeks to generate and the associated operational risk of not having
it. In this case, the supporting analyses determined neither need nor
risk with regard to a passenger and cargo capability. Air Force
officials could not provide supporting information sufficient to
explain this discrepancy between the analyses and their proposal.
Without sound analyses, the Air Force may be at risk of spending
several billion dollars unnecessarily for a capability that may not be
needed to meet a gap or shortfall.
Military decision makers approved the passenger and cargo capability as
a requirement although supporting analyses identified no need or
associated risk. Mandatory Air Force implementing guidance states that
senior leaders must use the documented results of analyses to confirm
the identified capability requirement. The Air Force Requirements for
Operational Capabilities Council validated, and the Chairman of the
Joint Chiefs of Staff's Joint Requirements Oversight Council validated
and approved, KCX tanker proposal with a passenger and cargo
capability. Following the approvals of the oversight councils, DOD
plans to solicit proposals and award a contract for the KC-X tanker
late in fiscal year 2007. At this time, the Under Secretary of Defense
for Acquisition, Technology and Logistics, who supervises DOD
acquisition,[Footnote 7] must certify, as Milestone Decision Authority
for the proposed tanker acquisition, that, among other things, the
Joint Requirements Oversight Council has accomplished its statutory
duties and that the proposed program is in compliance with DOD policies
and regulations.[Footnote 8] However, the absence of analyses
identifying a capability gap, shortfall, or redundancy, and the Joint
Requirements Oversight Council approval of the program without these
analyses is contrary to policy and implementing guidance and could
preclude certification of the program by the Under Secretary. Absent
this certification, the acquisition program for the KC-X tanker cannot
begin.[Footnote 9]
In this report, we recommended that the Secretary of Defense direct the
Secretary of the Air Force to accomplish the required analyses to
evaluate the proposed passenger and cargo capability so as to determine
if there is a gap, shortfall, or redundancy, assess the associated
risk, and then submit such documentation to the Joint Requirements
Oversight Council for validation. We also recommended that, once these
analyses are completed, the Secretary of Defense direct the Chairman,
Joint Chiefs of Staff, to formally notify the Under Secretary of
Defense for Acquisition, Technology and Logistics that such analyses
have been completed as required prior to certification of the program
to Congress.
DOD disagreed with our first recommendation to accomplish the required
analyses. In its comments, DOD stated that through the Joint
Capabilities Integration and Development System process, the Air Force
presented analysis and rationale for the passenger and cargo
capability. DOD further stated that its Joint Requirements Oversight
Council and the Air Force concluded that the analysis was sufficient
justification for the capability and the Joint Requirements Oversight
Council validated the requirement. However, as our report points out,
DOD did not perform the required analyses and failed to identify a gap,
shortfall, or redundancy for the passenger and cargo capability.
Considering the requirement for analyses that separate needs from wants
and the risk of unnecessary expenditures in this multi-year multi-
billion dollar acquisition program, we continue to believe that our
recommendation has merit and that the analyses required by mandatory
guidance are necessary to inform the decision that begins the
acquisition.
DOD agreed with our recommendation to formally notify the Under
Secretary of Defense for Acquisition, Technology and Logistics once the
required analyses have been completed. However, DOD did not offer
assurance that the Air Force would accomplish the required analyses
that evaluate the proposed passenger and cargo capability as we
recommended, and then submit such documentation to the Joint
Requirements Oversight Council for validation. We believe that the time
it could take to accomplish the required analyses and submit the
analyses for revalidation by the Joint Requirements Oversight Council,
could delay the Under Secretary's certification until just prior to the
Milestone B decision, and may frustrate the congressional oversight
that would otherwise be permitted under section 2366a.[Footnote 10] We
believe that in a program committing $120 billion over several decades,
the review confirming that needs are justified should occur as far in
advance of program initiation as possible.
In light of the DOD comments on our report, we have put forward a
matter for congressional consideration. Specifically, we are suggesting
that Congress consider requiring:
* in addition to the certification described by section 2366a of title
10, United States Code, the Under Secretary of Defense for
Acquisitions, Technology and Logistics make a specific certification
that the Air Force employed a sound, traceable, and repeatable process
producing analyses that determined if there is a gap, shortfall, or
redundancy and assessed the associated risk with regard to passenger
and cargo capability for the KC-135 Recapitalization, and:
* consistent with service policy, these analyses are made available to
the Joint Requirements Oversight Council prior to the Under Secretary's
certification of the program pursuant to section 2366a of title 10,
United States Code.
The Air Force intends to replace the fleet of more than 500 tankers and
the Mobility Capabilities Study of 2005 set the requirement for tankers
at a range of between 520 to 640 aircraft. Replacement of this fleet is
estimated to cost a minimum of $72 billion. Compared to a refueling
aircraft without a passenger and cargo capability, the inclusion of the
capability is estimated, according to the Analysis of Alternatives done
for the KC-X tanker, to increase costs by 6 percent. The Joint
Requirements Oversight Council approval of the proposal of a
replacement tanker aircraft with the passenger and cargo capability,
without an established need supported by analyses and without an
analysis of risk, could result in an unnecessary expenditure of at
least $4.3 billion by our estimates.
In our August 1996 report, U.S. Combat Air Power: Aging Refueling
Aircraft Are Costly to Maintain and Operate, we recommended
consideration of a dual-use aircraft that could conduct both aerial
refueling and airlift operations as a replacement for the KC-
135.[Footnote 11] We recommended that the Secretary of Defense require
that future studies and analyses of replacement airlift and tanker
aircraft consider accomplishing the missions with a dual-use aircraft.
DOD only partially concurred with this recommendation, expressing
concern at that time about how a dual-use aircraft would be used and
whether one mission area might be degraded to accomplish the second
mission. The lack of analyses done to support the current proposal
still does not give DOD officials information about how a dual-use
aircraft would be used or whether the primary mission of aerial
refueling would be degraded.
Employing Best Practices Can Shape Better Program Outcomes for DOD's
Airlift Acquisitions:
Over the past 25 years, DOD has invested more than $140 billion on its
airlift and tanker forces. Success for acquisitions requires sound
decisions to ensure that program investments are getting promised
returns--on time deliveries to the field, predictable costs, and
sufficient capability. We have reviewed four major airlift programs and
found they did not meet delivery schedules and were over cost. These
programs did not involve huge technological leaps but presented
significant design challenges to integrate new systems into the older
aircraft. A consistent problem plaguing the programs was an
insufficient job of analyzing the requirements and resources at the
programs' outset, a key systems engineering activity. The divergence
between these programs' experience and best product development
practices are contributing factors to their outcomes.
Outcomes of Certain Airlift Programs:
We assessed four airlift programs as part of our annual assessment of
DOD's major acquisition programs and each has experienced cost growth
and schedule delays. Despite being based largely on low technological
risks involving mature systems, these programs have failed to deliver
on the business cases that justified their initial investment. DOD
estimates it will need over $12 billion between 2007 and 2013 to
develop, modify, or procure these aircraft. The specific airlift
programs include:
* The Air Force's C-5 Avionics Modernization Program (AMP) is intended
to improve the mission capability rate and transport capabilities, as
well as reduce ownership costs by incorporating global air traffic
management, navigation and safety equipment, modern digital equipment,
and an all-weather flight control system.
* The Air Force's C-5 Reliability Enhancement and Reengining Program
(RERP) is intended to enhance the reliability, maintainability, and
availability of the C-5 through engine replacements and modifications
to subsystems such as the electrical and fuel subsystems. The C-5
aircraft will require installation of the AMP capabilities before the
aircraft engines can be replaced.
* The Air Force's C-130 Avionics Modernization Program (AMP) is
intended to standardize the cockpit configurations and avionics of
different models of C-130 aircraft by providing such things as
communication and navigational system upgrades, terrain avoidance and
warning system, dual flight management systems, and new data links.
* The C-130J, the latest model of the C-130 aircraft series, is
designed primarily for the transport of cargo and personnel within a
theater of operation. Variants of the C-130J are being acquired by the
Air Force (e.g., Air Mobility Command and Special Operations Command),
Marine Corps, and Coast Guard.
Each of these programs has experienced problems that have impacted cost
and schedule (see table 1). The net effect of the outcomes to date is
that DOD is now paying more to modify or acquire these systems and the
warfighter has had to wait longer than initially planned before new
capability is delivered. For example, the Air Force now expects by 2011
to have completed the modification of about 135 fewer C-130 airlift
aircraft when compared to its plan 2 years ago.
Table 1: Outcomes of Selected Airlift Programs (as of fiscal year 2006)
(FY 2007 dollars in millions):
Weapon Systems: C-5 AMP;
Latest Development Cost: $432.1;
Development Cost Change Since Start: 17 percent;
Program Acquisition Unit Cost Increases: 86 percent;
Program schedule delays: Initial operational capability delayed about 1
year.
Weapon Systems: C-5 RERP;
Latest Development Cost: $1,342.9;
Development Cost Change Since Start: (16 percent);
Program Acquisition Unit Cost Increases: 10 percent;
Program schedule delays: Initial operational capability delayed over 2
years.
Weapon Systems: C-130 AMP;
Latest Development Cost: $1,627.8;
Development Cost Change Since Start: 128 percent;
Program Acquisition Unit Cost Increases: 43 percent;
Program schedule delays: First production delivery delayed over 2
years.
Weapon Systems: C-130J;
Latest Development Cost: $262.9;
Development Cost Change Since Start: Not applicable;
Program Acquisition Unit Cost Increases: 26 percent;
Program schedule delays: First production delivery delayed about 1 ½
years.
Source: GAO analysis of DOD data:
Note: C-130J development costs have increased by 2,347 percent but this
includes costs to correct deficiencies and add new capabilities.
[End of table]
We anticipate there could be additional cost increases and schedule
delays reported in the future. For example, the C-130 AMP fiscal year
2008 budget indicates that the total program costs have increased
almost $700 million and planned quantities have been reduced from 434
units to 268 units--nearly doubling the program acquisition unit costs
since December 2005. The program recently notified Congress of a
critical Nunn-McCurdy breach concerning its unit cost
increases.[Footnote 12] The budget also shows the Air Force plans to
fund the modification of 110 C-5 aircraft with AMP improvements instead
of 59 aircraft as stated in last year's budget. According to C-5 RERP
program officials, total program costs are expected to increase due to
costs with the engine, pylons, and labor.
Airlift Programs Have Not Captured Critical Product Knowledge at Key
Decision Points:
Over the last several years, we have undertaken a body of work that
examines weapon acquisition issues from the perspective that draws upon
lessons learned from best commercial practices for product development.
We have found that a key to successful product development is the
formulation of a business case that provides demonstrated evidence that
(1) the warfighter need exists and that it can best be met with the
chosen concept and (2) the concept can be developed and produced within
existing resources--including proven technologies, design knowledge,
adequate funding, and adequate time to deliver the product when needed.
The business case is then executed through an acquisition process that
is anchored in knowledge. Leading firms ensure a high level of
knowledge is achieved at key junctures in development, characterized as
knowledge points described below:
* Knowledge point 1: A match must be made between the customer's needs
and the developer's available resources--technology, engineering
knowledge, time, and funding--before a program starts.
* Knowledge point 2: The product's design must be stable and must meet
performance requirements before beginning system demonstration. This is
primarily evidenced by the release of 90 percent of the design drawings
by the critical design review and successful system integration.
* Knowledge point 3: The product must be producible within cost,
schedule, and quality targets and demonstrated to work as intended
before production begins.
There is a synergy in this process, as the attainment of each
successive knowledge point builds on the preceding one. We have found
that if the knowledge based acquisition concept is not applied, a
cascade of negative effects becomes magnified in the product
development and production phases of an acquisition program leading to
cost increases and schedule delays, poor product quality and
reliability, and delays in getting new capability to the warfighter
(see figure 2).
Figure 2: Illustration of Cascading Negative Effects of Failing to
Follow a Knowledge-Based Acquisition Approach:
[See PDF for image]
Source: GAO.
[End of figure]
DOD programs often do not capture sufficient knowledge by critical
junctures but decide to move forward regardless. The airlift systems we
reviewed were not immune to this condition and have experienced
unnecessary cost growth and schedule delays as a result. While we do
not have in-depth knowledge on the specific details for these programs,
we do have a broad understanding of the basic underpinnings that led to
the problems. All of the programs were considered low technological
risks by DOD because they planned to rely extensively on proven
commercial and modified off the shelf technology for its new
capabilities. However, these acquisitions have turned out to be more
difficult than expected. The programs did not follow sound systems
engineering practices for analyzing requirements and for ensuring a
well integrated design at the right time. As a result, each program has
encountered some difficulty in achieving design and production maturity
as the program moved forward. Some of the causes to problems
encountered include:
* Failing to fully analyze the resources needed to integrate proven
commercial technologies and subsystems into a military system before
initiating development.
* Not achieving a stable design before beginning system demonstration
phase resulting in costly design changes and rework.
* Failing to demonstrate the aircraft would work as required before
making large production investments.
In all these instances where appropriate knowledge was not captured
before moving forward, the impact has resulted in a predictable need
for additional resources as shown below in specific airlift programs.
C-5 AMP:
The C-5 AMP entered production without demonstrating that the system
worked as intended and was reliable. The program entered production
just 2 months after flight testing started and ran into significant
design problems while trying to complete development. Problems
uncovered after flight test began required modifications to the
aircraft design which increased by 50 percent the number of engineering
drawings needed for the system. Addressing these problems delayed the
initial operational capability by a year and contributed to the
significant growth in the program's unit costs. Even today, 4 years
after production was initiated, performance concerns remain with the C-
5 AMP. The Director of Operational Test and Evaluation recently
reported that the C-5 AMP is not operationally suitable because of high
component failure rates, inadequate diagnostics systems, and low
reliability rates.
C-5 RERP:
The C-5 RERP did not demonstrate design stability before entering the
system demonstration phase which resulted in rework and schedule
delays. At the time the program entered system demonstration, program
officials believed that they had released 90 percent of the design
drawings but had not successfully demonstrated that the subsystems
could be integrated onto the C-5 aircraft. During system integration
activities the program found that the "pylon/thrust reverser" had to be
redesigned to address overweight conditions and safety concerns. The
program's design efforts have also been hampered by the fact that its
success is dependent upon the success of the C-5 AMP program.
Presently, according to test officials, the C-5 AMP design is not
mature enough to provide a baseline design for the RERP efforts. These
design issues have contributed to an increase in costs and a 2-year
delay in delivering an initial operational capability.
C-130 AMP:
The C-130 AMP began development in 2001 without a clear understanding
of the resources needed to integrate proven commercial technologies
into a military system. According to the program office, they clearly
underestimated the complexity of the engineering efforts that were
needed to modify the different models of the C-130. At the critical
design review held in 2005--the point that the design is expected to be
stable and ready to begin the system demonstration phase--the program
had not proven that the subsystems and components could be successfully
integrated into the product.Upon integrating the new avionics into the
test aircraft, program officials realized that it had significantly
underestimated (by 400 percent) the amount of wiring and the number of
harnesses and brackets needed for the installation. As a result, the
design had to be reworked, delaying the delivery of the test aircraft
and increasing costs.
C-130J:
The Air Force procured the C-130J without assurances that the aircraft
would work as intended. Program officials believed the design was
mature when procurement began in 1996, largely because the C-130J
evolved from earlier models and was offered as a commercial
item[Footnote 13]. However, the C-130J has encountered numerous
deficiencies that had to be corrected in order to meet the minimum
warfighter requirements delaying the initial aircraft delivery to the
warfighter by about 1.5 years. DOD testing officials still report
performance issues with the aircraft resulting in it being rated as
partially mission capable. The performance issues involve the
aircraft's ability to meet its airdrop operations requirements, its
effectiveness in non-permissive threat environments, and
maintainability issues. Program officials plan to address the
deficiencies as part of a C-130J modernization effort.
Conclusion:
As we said at the beginning, our work shows that acquisition problems
will likely persist until DOD provides a better foundation for buying
the right things the right way. This involves making tough tradeoff
decisions as to which programs should be pursued, and, more
importantly, not pursued, making sure programs are executable,
establishing and locking in needed requirements before programs are
ever started, and making it clear who is responsible for what and
holding people accountable when these responsibilities are not
fulfilled. Recognizing this, DOD has tried to embrace best practices in
its policies, as well as taking many other actions. However, DOD still
has trouble distinguishing between wants and needs. Because of our
concerns about the analyses done for both the MCS, which has broad
implications for DOD's mobility needs, and the KC-X tanker
requirements, we would urge Congress and other decision makers to
exercise caution when making airlift and tanker investment decisions.
DOD will continue to face challenges in modernizing its forces with new
demands on the federal dollar created by changing world conditions.
Consequently, it is incumbent upon DOD to find and adopt best product
development practices that can allow it to manage its weapon system
program in the most efficient and effective way. Success over the long
term will depend on following knowledge-based acquisition practices as
well as DOD leadership's commitment to improving outcomes.
The four acquisition cases we cite in this testimony are not atypical
for all programs. Even with no major technological invention necessary
to meet the warfighters needs in these cases, acquisition outcomes are
not good. There are consequences to these outcomes. The warfighter does
not receive needed capability on time and the Department and Congress
must spend additional unplanned money to correct mistakes--an expense
they can ill afford. A knowledge-based product development process
steeped in best practices from systems engineering can solve many of
these problems before they start. DOD knows how to do this and, in
fact, informs its acquisition policy with systems engineering rules. It
should redouble its efforts to drive these policies into practice.
Mr. Chairman and members of the Subcommittee, this concludes our
prepared statement. We would be pleased to answer any questions you may
have.
Contact and Staff Acknowledgements:
For further information about this statement, please contact William M.
Solis at 202-512-8365 or Solisw@gao.gov or Michael J. Sullivan at 202-
512-4841 or Sullivanm@gao.gov. Contact points for Offices of
Congressional Relations and Public Affairs may be found on the last
page of this statement.
GAO staff who made major contributions to this testimony include Marie
Ahearn, Ann Borseth, Cheryl Andrew, Claudia Dickey, Mike Hazard,
Matthew Lea, Oscar Mardis, Sean Merrill, Karen Thornton, and Steve
Woods.
[End of section]
Related GAO Products:
Defense Acquisitions: Major Weapon Systems Continue to Experience Cost
and Schedule Problems under DOD's Revised Policy. GAO-06-368.
Washington, D.C.: April 13, 2006:
Defense Acquisitions: Assessments of Selected Major Weapon Programs.
GAO-06-391. Washington, D.C.:March 31, 2006.
DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. Washington,
D.C.: November 15, 2005.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15,
2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to
Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8,
2001.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30,
1999.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. GAO/NSIAD-98-56. Washington, D.C.:
February 24, 1998.
FOOTNOTES
[1] GAO, High Risk Series: An Update, GAO-07-310 (Washington, D.C.:
Jan. 2007).
[2] GAO, Defense Transportation: Study Limitations Raise Questions
about the Adequacy and Completeness of the Mobility Capabilities Study
and Report, GAO-06-938 (Washington, D.C.: Sept. 20, 2006).
[3] GAO, Defense Acquisitions: Air Force Decision to Include a
Passenger and Cargo Capability in Its Replacement Refueling Aircraft
Was Made without Required Analyses, GAO-07-367R (Washington, D.C.: Mar.
6, 2007).
[4] Air Force Instruction 10-601, Capabilities-Based Requirements
Development, ¶ 1.2 (July 31, 2006).
[5] Deployment orders are issued to deploy specific capabilities as
commitment decisions are made, rather than a deploying unit's full set
of equipment or capabilities. Flexible Deterrent Options (FDOs) provide
escalation options during the initial stages of a conflict. FDOs are
employed under certain conditions to deter adversarial actions contrary
to U.S. interests.
[6] GAO, Defense Transportation: Air Mobility Command Needs to Collect
and Analyze Better Data to Assess Aircraft Utilization. GAO-05-819
(Washington, D.C.: Sept. 29, 2005).
[7] 10 U.S.C. §133 (b)(1) (2006).
[8] 10 U.S.C. § 2366a § (a)(7) and (a)(10) (2006).
[9] 10 U.S.C. § 2366a (a) (2006) and Department of Defense Instruction
5000.2, Operation of the Defense Acquisition System, ¶ 3.7.1.2 (May 12,
2003).
[10] In respect to acquisition programs, milestones are established in
DODI 5000.2 and are the points where a recommendation is made and
approval is sought regarding starting or continuing a program into the
next phase. In this instance, the decision at Milestone B is to enter
into the system development and demonstration phase pursuant to
guidance prescribed by the Secretary of Defense and to begin the
acquisition program.
[11] GAO, U.S. Combat Air Power: Aging Refueling Aircraft Are Costly to
Maintain and Operate. GAO / NSIAD-96-160, Washington D.C.: (August 8,
1996).
[12] 10 U.S.C. § 2433 establishes the requirement for unit cost reports
if certain thresholds for program costs are exceeded (known as unit
cost or Nunn-McCurdy breaches). DOD is required to report to Congress
and, if applicable, certify the program to Congress.
[13] In October 2006, the program completed the transition to a non-
commercial negotiated contract.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
