Uncertainties Remain Concerning the Airborne Laser's Cost and Military Utility
Gao ID: GAO-04-643R May 17, 2004In 1996, the Air Force launched an acquisition program to develop and produce a revolutionary laser weapon system, the Airborne Laser (ABL), capable of defeating an enemy ballistic missile during the boost phase of its flight. Over the last 8 years, the program's efforts to develop this technology have resulted in significant cost growth and schedule delays. These events led Senate Members to request that we answer the following questions: (1) how much and why has the ABL's cost increased since the program's inception; (2) what is the expected military utility of the initial ABL aircraft; (3) what support systems will be required when the ABL is fielded and what is the likely cost of those systems; and (4) have recent program changes resulted in a more cost effective strategy for developing the weapon?
The prime contractor's costs for developing ABL have nearly doubled from the Air Force's original estimate and additional cost growth is occurring. The cost growth occurred primarily because the program did not adequately plan for and could not fully anticipate the complexities involved in developing the system. MDA continues to face significant challenges in developing the ABL's revolutionary technologies and in achieving cost and schedule stability. From 1996 through 2003, the value of the prime contract, which accounts for the bulk of the program's cost, increased from about $1 billion to $2 billion. According to our analysis, costs could increase between $431 million to $943 million more through first full demonstration of the ABL system. Cost growth has been spurred by rework that was necessary because rapid prototyping forced the program to integrate components before all subcomponents were fully tested. In addition, fabricating ABL's unique components and developing its complex software proved more costly and time-consuming than anticipated. Although ABL's prime contractor has added additional personnel to the contract, the program is faced with a bow wave of uncompleted work from prior years. Recognizing that the technology development activities directed by the contract could not be completed within the contract's cost ceiling, the ABL program office began development of a new cost estimate for completing these activities. Predictions of the military utility of the initial ABL aircraft are still highly uncertain because these forecasts are not based on any demonstrated capability of the system, but rather on modeling, simulations, and analysis. These assessment tools predict that the initial Airborne Laser will be militarily useful against most theater and intercontinental ballistic missiles; but flight-test data are not yet available to anchor these tools. According to test officials, the models and simulations are adequate for establishing system parameters, but may not be sufficient for estimating the effectiveness of a fielded system. Additionally, other factors will influence ABL's military utility, including the availability of support infrastructure and the number of aircraft available. When it is fielded, ABL is expected to require unique support for its laser and beam control and fire control components in addition to the support burdens attached to all high-value air assets. ABL will also require unique maintenance, such as re-calibration and re-coating of beam control and fire control subcomponents. In addition, it will require the typical support systems needed by other high-value air assets, such as escort aircraft for protection. MDA has not yet determined the cost of ABL's unique support systems, but operating costs for other high-value air assets range from about $24,000 per hour to $92,000 per hour. MDA refocused the ABL program in February 2004 to pursue a more cost-effective development strategy. The program now plans to follow a knowledge-based rather than a schedule-driven approach to the element's development. In addition, the program has delayed indefinitely the purchase of a second aircraft and a test facility.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Robert E. Levin Team: Government Accountability Office: Acquisition and Sourcing Management Phone: (202) 512-3519GAO-04-643R, Uncertainties Remain Concerning the Airborne Laser's Cost and Military Utility
This is the accessible text file for GAO report number GAO-04-643R
entitled 'Uncertainties Remain Concerning the Airborne Laser's Cost and
Military Utility' which was released on May 19, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
May 17, 2004:
The Honorable Bill Nelson:
Ranking Minority Member:
Subcommittee on Strategic Forces:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Akaka:
United States Senate:
Subject: Uncertainties Remain Concerning the Airborne Laser's Cost and
Military Utility:
In 1996, the Air Force launched an acquisition program to develop and
produce a revolutionary laser weapon system, the Airborne Laser (ABL),
capable of defeating an enemy ballistic missile during the boost phase
of its flight. Over the last 8 years, the program's efforts to develop
this technology have resulted in significant cost growth and schedule
delays. These events led you to request that we answer the following
questions: (1) How much and why has the ABL's cost increased since the
program's inception? (2) What is the expected military utility of the
initial ABL aircraft? (3) What support systems will be required when
the ABL is fielded and what is the likely cost of those systems? (4)
Have recent program changes resulted in a more cost effective strategy
for developing the weapon?
After we began our review, the Missile Defense Agency (MDA) refocused
the ABL program to pursue a more knowledge-based approach, where
program knowledge is demonstrated at critical points in the development
cycle.[Footnote 1] We examined the planned changes to determine whether
they will result in a more cost-effective strategy for developing the
ABL element. This report summarizes that information and transmits the
briefing charts that have been revised to address program changes made
by MDA after our presentation to your staff on March 4, 2004.
In conducting our review, we analyzed Department of Defense (DOD), MDA,
Air Force, and ABL program documents and interviewed key program
officials. To determine how much ABL's cost increased since the program
was initiated and why this growth occurred, we reviewed both the ABL
prime contract awarded in 1996 and significant modifications made to
the contract through July 2003. We also assessed the contractor's cost
and schedule performance by analyzing contractor cost performance
reports using Earned Value Management principles. To determine the
expected military utility of the initial ABL aircraft, we interviewed
key DOD, MDA, and intelligence officials and analyzed relevant
documents, including studies and reports. To determine the necessary
support systems and their costs, we examined relevant documents and
held discussions with the Air Force Air Combat Command. We also
reviewed the Air Force Total Ownership Cost database to identify
support systems and costs for other high value air assets. To determine
whether recent program changes resulted in a more cost-effective
strategy for developing the ABL, we compared the program office's
previous plans, goals and activities with its recent efforts to refocus
the program. We conducted our work from October 2003 through May 2004
in accordance with generally accepted government auditing standards.
Results in Brief:
The prime contractor's costs for developing ABL have nearly doubled
from the Air Force's original estimate and additional cost growth is
occurring. The cost growth occurred primarily because the program did
not adequately plan for and could not fully anticipate the complexities
involved in developing the system. MDA continues to face significant
challenges in developing the ABL's revolutionary technologies and in
achieving cost and schedule stability. From 1996 through 2003, the
value of the prime contract, which accounts for the bulk of the
program's cost, increased from about $1 billion to $2 billion.
According to our analysis, costs could increase between $431 million to
$943 million more through first full demonstration of the ABL system.
Cost growth has been spurred by rework that was necessary because rapid
prototyping forced the program to integrate components before all
subcomponents were fully tested. In addition, fabricating ABL's unique
components and developing its complex software proved more costly and
time-consuming than anticipated. Although ABL's prime contractor has
added additional personnel to the contract, the program is faced with a
bow wave of uncompleted work from prior years. Recognizing that the
technology development activities directed by the contract could not be
completed within the contract's cost ceiling, the ABL program office
began development of a new cost estimate for completing these
activities.
Predictions of the military utility of the initial ABL aircraft are
still highly uncertain because these forecasts are not based on any
demonstrated capability of the system, but rather on modeling,
simulations, and analysis. These assessment tools predict that the
initial Airborne Laser will be militarily useful against most theater
and intercontinental ballistic missiles; but flight-test data are not
yet available to anchor these tools. According to test officials, the
models and simulations are adequate for establishing system parameters,
but may not be sufficient for estimating the effectiveness of a fielded
system. Additionally, other factors will influence ABL's military
utility, including the availability of support infrastructure and the
number of aircraft available.
When it is fielded, ABL is expected to require unique support for its
laser and beam control and fire control components in addition to the
support burdens attached to all high-value air assets.[Footnote 2] For
example, to carry out a wider range of missions, ABL will need laser
fuel production facilities close to the theater of operations. ABL will
also require unique maintenance, such as re-calibration and re-coating
of beam control and fire control subcomponents. In addition, it will
require the typical support systems needed by other high-value air
assets, such as escort aircraft for protection. MDA has not yet
determined the cost of ABL's unique support systems, but operating
costs[Footnote 3] for other high-value air assets range from about
$24,000 per hour to $92,000 per hour.
MDA refocused the ABL program in February 2004 to pursue a more cost-
effective development strategy. The program now plans to follow a
knowledge-based rather than a schedule-driven approach to the element's
development. For example, the program is no longer working to deliver a
contingency sensor capability in Block 2004. Instead it will
concentrate on maturing and demonstrating ABL's critical technologies.
In addition, the program has delayed indefinitely the purchase of a
second aircraft and a test facility.
To provide better information to decision makers as they consider
whether to continue investing in the ABL program, we are recommending
that DOD provide an analysis that quantifies the confidence that
decision makers should have in the new cost estimate. DOD concurred.
Background:
From 1996 to 2001, the ABL program was an Air Force major defense
acquisition program. However, in October 2001, the Department of
Defense (DOD) transferred responsibility for the program to the
Ballistic Missile Defense Organization (now MDA) where the ABL became
one element of the Ballistic Missile Defense System (BMDS). The ABL's
mission is to destroy enemy ballistic missiles in the boost phase as
part of the layered defense strategy. Similar to other BMDS elements,
the ABL is being developed incrementally in a series of 2-year blocks.
During the first block, known as Block 2004, MDA's goal is to mature
the ABL's critical technologies.
The ABL includes four major components: a modified 747 aircraft, which
serves as the platform for other components; a high-energy chemical
laser; a beam control and fire control system; and a battle management
and command and control system. The high-energy laser is a chemical
oxygen-iodine laser that generates energy through chemical reactions.
The prototype, or demonstrator aircraft, that is under development will
include six laser modules that will be linked together to produce a
laser beam with megawatts of power. The beam control and fire control
component is designed to track and stabilize the beam so that its
energy remains focused on a small area of an enemy missile. ABL uses
its high-energy laser to defeat enemy missiles by rupturing a missile's
motor casing, causing the missile to lose power. MDA expects that the
battle management and command and control component will plan and
execute each ABL engagement.
Program Complexity Drives Cost Growth:
Because it did not adequately plan for and could not fully anticipate
the complexity of developing and demonstrating ABL's critical
technologies, the program has experienced continual cost growth and
schedule delays. The prime contract, which consumes the bulk of the
program's funding, will reach its cost ceiling in May 2004. The program
office is in the process of developing a new cost estimate as the basis
for raising the contract's ceiling price. Additionally, in recognition
of the contract's rising cost, MDA has increased its cost goal for
completing ABL's technology development phase during Block 2004.
Contract's Current Value Has Doubled Since 1996 and Continues to
Increase:
From the contract's award in 1996 through 2003, the cost of ABL's
primary research and development contract increased from about $1
billion to about $2 billion. In fiscal year 2003 alone, work completed
by the contractor cost about $242 million more than expected. The
contractor also experienced schedule delays and was unable to complete
$28 million of work planned for the fiscal year. Based on the
contractor's 2003 cost and schedule performance, we estimate that the
prime contract will exceed the contractor's July 2003 cost estimate of
about $2.1 billion by between $431 million and $943 million through
first full demonstration of the ABL system.
Since 1996, the ABL program has experienced several major
restructurings and contract re-baselines, due primarily to the
unforeseen complexity in manufacturing and integrating critical
technology. According to program officials, rapid prototyping
(integrating components into a prototype system prior to demonstrating
the maturity of all critical technologies) limited subcomponent
testing, causing rework and changing requirements. Today the program
faces a bow wave of incomplete work from previous years even though the
prime contractor has increased the number of people devoted to the
program and has added additional shifts to bring the work back on
schedule. In addition, unanticipated difficulties in software coding
and integration issues, as well as difficulty in manufacturing advanced
optics and laser components, have caused cost growth.
Increasing Cost of Prime Contract Causes Revisions in Technology
Development Cost Goal:
According to program officials, the prime contract will reach its cost
ceiling in May 2004. The ABL program office is in the process of
developing a new cost estimate for the remaining technology development
activities, including component ground tests, beam control and fire
control flight-tests, integration of components into a working ABL
prototype, and a demonstration of the prototype's lethality against a
boosting ballistic missile. The program will use this estimate as the
basis for raising the contract's ceiling price.
MDA also recognized the impact that the prime contract's rising cost is
having on the agency's cost goal for completing ABL's technology
development. In February 2003, MDA reported in its fiscal year 2004
Budget Estimate Submission[Footnote 4] that its cost goal for maturing
and demonstrating ABL's critical technologies during Block 2004 was
about $494 million. However, in its fiscal year 2005 submission, MDA
increased its Block 2004 cost goal by $583 million to $1.077 billion by
moving ABL funds from Blocks 2006 and 2008 to Block 2004. The program
manager told us that MDA plans to continue updating its cost goal for
completing technology development as part of each year's budgeting
process. To better assess the program's annual funding needs, the
contractor will plan work in about 12-month increments.
Successful developers have found that it is difficult to make highly
reliable cost and schedule estimates until the maturity of a product's
critical technologies has been demonstrated. The ABL program has not
yet reached this point. One well-established tool for providing
decision makers with an increased understanding of the reliability of
developed cost estimates is an "uncertainty analysis." An uncertainty
analysis simulates a model by randomly and repeatedly generating values
for certain variables. After hundreds or thousands of trials, one can
view the statistical results and determine the confidence level in any
outcome. For example, the model may show that there is a 10-percent
chance that the project can be completed for $50 million, a 50-percent
chance that it can be completed for $70 million, or a 90-percent chance
that the project will cost $100 million or less. Although there remains
a gap in the knowledge MDA needs to make highly reliable estimates of
the cost to complete ABL's technology development, other programs have
found that uncertainty analyses help to understand the size of such a
gap.
Military Utility of Initial ABL Aircraft Is Highly Uncertain:
Predictions of the military utility of the initial ABL aircraft are
highly uncertain because these forecasts are based on modeling,
simulations, and analysis, rather than the demonstrated capability of
the system. The Airborne Laser program office predicts that the ABL
prototype being developed during Block 2004 will have a capability to
defeat most classes of theater and intercontinental ballistic missiles.
However, program officials base these predictions on analyses, models,
and simulations that are not yet anchored by flight test data. These
assessment tools take into consideration various factors, such as the
projected power output of the 6-module laser weapon, the stand-off
range that ABL maintains to protect itself from threat weapons, and the
characteristics of the threat missiles. Should any of the assumptions
made in modeling these factors prove incorrect, the output of the
assessment tools might prove similarly incorrect. According to test
officials, models and simulations are adequate for establishing system
parameters, but may not be sufficient for understanding the
effectiveness of a fielded system.[Footnote 5] In addition, the
Department of Defense has not yet assessed the Airborne Laser's
military utility and has no fixed time frame for doing so.
Even if the program office's assessment proves to be correct, other
factors, such as reduced time on station and limited forward operating
capability, will limit ABL's initial military utility. For example,
with only one aircraft, 24-hour operations are not possible. In
addition, in order to operate from a forward location the aircraft will
require a forward-based chemical facility for replenishment of the
chemicals that fuel the high-energy laser. Without these facilities,
the Airborne Laser is capable of performing three constrained missions.
Support Costs Will Likely Be Higher Than Those of Other High Value Air
Assets:
The ABL will require unique support in addition to the already
substantial support required for all high-value air assets. For
example, to remain on station for extended periods of time, ABL will
need a production facility close to the theater of operations that can
store and mix chemicals for the high-energy laser. Likewise, ABL will
require unique maintenance, such as re-calibration and re-coating of
beam control and fire control subcomponents; chemical facilities must
be secured; and a ground support squadron and chemicals must be
transported to a forward location.
The ABL will also require support systems, such as a protective escort,
that are typical of all high-value air assets. Our review of the U.S.
Air Force Total Ownership Costs Database[Footnote 6] shows that
operating costs for other high-value air assets, such as the Airborne
Warning and Control System or Joint Surveillance Target Attack Radar
System, range from about $24,000 per hour to $92,000 per hour. While
the ABL program office has identified some of the support systems ABL
will require once it is fielded, the program does not plan to fully
address this issue, including support costs, before Block 2006.
Program Changes Likely to Result in a More Cost-Effective Strategy:
MDA made program changes in February 2004 that should make its strategy
for developing ABL more cost-effective. The primary change is that the
program will no longer try to develop ABL according to a set schedule,
but will follow a more knowledge-based approach. For example, MDA no
longer plans to deliver a contingency sensor capability in the Block
2004 time frame. Instead it plans to focus on ground-and flight-testing
components and demonstrating technologies before proceeding to the next
development phase. However, it does not intend to lose sight of its
overall objective of testing the prototype ABL aircraft against a
short-range ballistic missile in what has become known as a lethal
demonstration. Program officials also told us that they plan to defer
indefinitely the purchase of a second aircraft and other hardware,
which MDA initially planned to initiate during Block 2004.
Conclusion:
We believe the ABL program made a sound and cost-effective decision to
defer the purchase of the second aircraft and additional hardware and
to pursue a more knowledge-based approach to the element's development.
It has also taken an important first step by setting a cost goal for
ABL's technology development phase. However, good investment decisions
depend upon an understanding of the total funds that will be needed to
obtain an expected benefit, and, at this time, MDA has not been able to
provide decision makers assurances that the agency's cost projections
to complete technology development can be relied upon. Decision makers
could make more informed decisions about further investments in the ABL
program if they understood the likelihood and confidence associated
with MDA's cost projections.
Recommendation:
To provide a better framework for making investment decisions during
the program's research and development phase, we recommend that the
Secretary of Defense direct the Director, MDA, to complete an
uncertainty analysis on the contractor's new cost estimate that
quantifies the confidence that may be placed in the estimate.
Agency Comments and Our Evaluation:
In written comments to a draft of this report (see encl. II), the DOD
concurred with our recommendation. DOD also provided separate technical
comments, which we have incorporated as appropriate.
As requested by your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution of it until 30
days from the date of this letter.
At that point, copies of this report will be sent to the Secretary of
Defense; the Director, Missile Defense Agency; and other interested
congressional committees. We will also make copies available to others
upon request. In addition, the report will be available at no charge on
the GAO Web site at http://www.gao.gov.
Should you or your staff have any questions on matters discussed in
this report, please contact me on (202) 512-4841 or Barbara Haynes at
(256) 922-7535. Principal contributors to this report were Beverly
Breen, Alan Frazier, LaTonya Miller, and Karen Richey.
Signed by:
Robert E. Levin
Director, Acquisition and Sourcing Management:
Enclosure:
Comments From the Department of Defense:
ACQUISITION, TECHNOLOGY AND LOGISTICS:
OFFICE OF THE UNDER SECRETARY OF DEFENSE:
3000 DEFENSE PENTAGON
WASHINGTON, DC 20301-3000:
Mr. Robert E. Levin:
Director, Acquisition and Sourcing Management
U. S. General Accounting Office:
441 G. Street, N.W.
Washington, DC 20548:
Dear Mr. Levin:
This is the Department of Defense's (DoD's) response to the GAO Draft
Report, "MISSILE DEFENSE: Uncertainties Remain Concerning the Airborne
Laser's Cost and Military Utility" dated April 9, 2004 (GAO Report 04-
643R/Audit 120290).
The DoD has reviewed the draft report. We concur with the
recommendation, with the specific comments enclosed. We also recommend
some factual corrections. My action officer for this effort is Lt Col
Mark Arbogast, (703) 695-7328, mark.arbogast@osd.mil.
We appreciate the opportunity to comment on the draft report.
Sincerely,
Signed for:
Glenn F. Lamartin
Director:
Defense Systems:
Enclosure: As stated:
MAY 3 2004:
GAO DRAFT REPORT - DATED April 9, 2004 GAO Report 04-643R/Audit 120290:
"MISSILE DEFENSE: Uncertainties Remain Concerning the Airborne Laser's
Cost and Military Utility" (GAO Report 04-643R/Audit 120290)":
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATION:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
direct the Director, Missile Defense Agency (MDA) to complete an
uncertainty analysis on the contractor's new cost estimate that
quantifies the confidence that may be placed in the estimate (page 10,
GAO Draft Report).
DOD RESPONSE: Concur. However, it should be noted that the program
office is producing the new cost estimate, as mentioned on page 3 of
the report's discussion, not the contractor. The program office has
already been using uncertainty analysis in its early formulation of the
new estimate.
(120290):
FOOTNOTES
[1] A knowledge-based approach contains three phases that are each
distinguished by the knowledge attained. During technology development,
scientists apply scientific knowledge to a practical engineering
problem and demonstrate that components with the desired form, fit, and
function can be developed. During product development, the second
phase, engineers integrate components into a stable design and
demonstrate that the design will result in a product that meets the
customer's needs and can be produced with the time and money available.
The final phase, production, is the manufacturing of the product.
[2] Examples of other high-value air assets are the Airborne Warning
and Control System and the Joint Surveillance Target Attack Radar
System.
[3] According to the U.S. Air Force Total Ownership Costs Database,
operating costs include costs for mission personnel, unit-level
consumption, intermediate maintenance, depot maintenance, contractor
support, sustaining support, and indirect support.
[4] For the last 2 fiscal years, MDA has issued a document known as a
Budget Estimate Submission that lays out planned expenditures for each
near-term block by element and the activities planned during those
blocks.
[5] We did not examine the models and simulations that MDA uses to
forecast ABL's capability or attempt to replicate MDA's predictions of
ABL's military utility.
[6] The Air Force Total Ownership Costs database provides detailed cost
information on all Air Force major weapon systems.
