Defense Acquisitions
Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach Gao ID: GAO-10-311 February 25, 2010By law, GAO is directed to assess the annual progress the Missile Defense Agency (MDA) made in developing and fielding the Ballistic Missile Defense System (BMDS). GAO also assessed MDA's progress toward improving accountability and transparency in agency operations, management processes, and its acquisition strategy. To accomplish this, GAO reviewed asset fielding schedules, test plans and reports, as well as pertinent sections of Department of Defense (DOD) policy to compare MDA's current level of accountability with that of other DOD programs. GAO's fiscal year 2009 assessment of MDA's cost, schedule, and performance progress is more limited than previous assessments because MDA removed key components of schedule and performance goals from its annual report of goals. In addition, though it had committed to, MDA did not report total cost estimates in 2009.
Fiscal year 2009 was an unprecedented year of transition for MDA as it experienced its first change of administration, its third MDA Director, shifts in plans for missile defense in Europe as well as a shift in focus for technology development from intercepting missiles during the boost phase to the early intercept phase. Such changes present new challenges for MDA but also opportunities to strengthen acquisition approaches. (1) Progress: MDA achieved several accomplishments. For example, MDA revised its testing approach to better align tests with modeling and simulation needs and undertook a new targets development effort to resolve longstanding problems supplying sufficient and reliable targets. The agency also demonstrated increased levels of performance for some elements through flight and ground testing. Fiscal year 2009 testing indicates an increased level of interoperability among multiple elements, improving both system-level performance and advancing the BMDS models and simulations needed to predict performance. In addition, the agency delivered 83 percent of the assets it planned to deliver by the end of fiscal year 2009. (2) Challenges: While there was progress, all BMDS elements had delays in conducting tests, were unable to accomplish all planned objectives, and experienced performance challenges. Poor target performance continued to be a problem, causing several test delays and leaving several test objectives unfulfilled. The test problems also precluded MDA from gathering key knowledge and affected development of advanced algorithms and homeland defense. These test problems continued to affect the models and simulations used to assess the overall performance of the BMDS. Consequently, comprehensive assessments of its capabilities and limitations are still not possible. MDA also redefined its schedule baseline, eliminating goals for delivering integrated capabilities so we were not able to assess progress in this area. Despite these problems, MDA proceeded with production and fielding of assets. (3) Transparency, Accountability, and Oversight: In 2009, the significant adjustments MDA made to its acquisition approach--terminating the block structure; reducing, eliminating, or not reporting key baselines; and terminating its capability declaration process--and adjustments to the material reported to Congress reduced the transparency and accountability MDA had begun to build. However, MDA is beginning to implement several initiatives--including the adoption of key principles of DOD acquisition regulations--that could improve transparency and accountability and lay the foundation needed for oversight. If these initiatives are implemented in accordance with knowledge-based acquisition principles, an opportunity exits to improve the BMDS acquisition by ensuring MDA programs begin with realistic, transparent plans and baselines. While these initial steps hold promise, they will take time to fully implement and once implemented they will need to be sustained over time and consistently applied.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-10-311, Defense Acquisitions: Missile Defense Transition Provides Opportunity to Strengthen Acquisition Approach
This is the accessible text file for GAO report number GAO-10-311
entitled 'Defense Acquisitions: Missile Defense Transition Provides
Opportunity to Strengthen Acquisition Approach' which was released on
February 25, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
February 2010:
Defense Acquisitions:
Missile Defense Transition Provides Opportunity to Strengthen
Acquisition Approach:
GAO-10-311:
GAO Highlights:
Highlights of GAO-10-311, a report to congressional committees.
Why GAO Did This Study:
By law, GAO is directed to assess the annual progress the Missile
Defense Agency (MDA) made in developing and fielding the Ballistic
Missile Defense System (BMDS). GAO also assessed MDA‘s progress toward
improving accountability and transparency in agency operations,
management processes, and its acquisition strategy. To accomplish
this, GAO reviewed asset fielding schedules, test plans and reports,
as well as pertinent sections of Department of Defense (DOD) policy to
compare MDA‘s current level of accountability with that of other DOD
programs. GAO‘s fiscal year 2009 assessment of MDA‘s cost, schedule,
and performance progress is more limited than previous assessments
because MDA removed key components of schedule and performance goals
from its annual report of goals. In addition, though it had committed
to, MDA did not report total cost estimates in 2009.
What GAO Found:
Fiscal year 2009 was an unprecedented year of transition for MDA as it
experienced its first change of administration, its third MDA
Director, shifts in plans for missile defense in Europe as well as a
shift in focus for technology development from intercepting missiles
during the boost phase to the early intercept phase. Such changes
present new challenges for MDA but also opportunities to strengthen
acquisition approaches.
Progress:
MDA achieved several accomplishments. For example, MDA revised its
testing approach to better align tests with modeling and simulation
needs and undertook a new targets development effort to resolve
longstanding problems supplying sufficient and reliable targets. The
agency also demonstrated increased levels of performance for some
elements through flight and ground testing. Fiscal year 2009 testing
indicates an increased level of interoperability among multiple
elements, improving both system-level performance and advancing the
BMDS models and simulations needed to predict performance. In
addition, the agency delivered 83 percent of the assets it planned to
deliver by the end of fiscal year 2009.
Challenges:
While there was progress, all BMDS elements had delays in conducting
tests, were unable to accomplish all planned objectives, and
experienced performance challenges. Poor target performance continued
to be a problem, causing several test delays and leaving several test
objectives unfulfilled. The test problems also precluded MDA from
gathering key knowledge and affected development of advanced
algorithms and homeland defense. These test problems continued to
affect the models and simulations used to assess the overall
performance of the BMDS. Consequently, comprehensive assessments of
its capabilities and limitations are still not possible. MDA also
redefined its schedule baseline, eliminating goals for delivering
integrated capabilities so we were not able to assess progress in this
area. Despite these problems, MDA proceeded with production and
fielding of assets.
Transparency, Accountability, and Oversight:
In 2009, the significant adjustments MDA made to its acquisition
approach”terminating the block structure; reducing, eliminating, or
not reporting key baselines; and terminating its capability
declaration process”and adjustments to the material reported to
Congress reduced the transparency and accountability MDA had begun to
build. However, MDA is beginning to implement several initiatives”
including the adoption of key principles of DOD acquisition
regulations”that could improve transparency and accountability and lay
the foundation needed for oversight. If these initiatives are
implemented in accordance with knowledge-based acquisition principles,
an opportunity exits to improve the BMDS acquisition by ensuring MDA
programs begin with realistic, transparent plans and baselines. While
these initial steps hold promise, they will take time to fully
implement and once implemented they will need to be sustained over
time and consistently applied.
What GAO Recommends:
GAO makes a number of recommendations for MDA to increase transparency
and accountability through more thorough and consistent reporting of
baselines and variances to those baselines. In addition, GAO
recommended that MDA apply more knowledge-based approaches to its new
acquisition initiatives and report these initiatives and their
associated commitments to Congress. In response, DOD fully concurred
with 9 of the 10 recommendations and partially concurred with 1.
View [hyperlink, http://www.gao.gov/products/GAO-10-311] or key
components. For more information, contact Cristina Chaplain at (202)
512-4841 or chaplainc@gao.gov.
[End of section]
Contents:
Letter:
Background:
Significant Adjustments Made to Missile Defense in Fiscal Year 2009:
Progress Made in Fiscal Year 2009:
Significant Challenges Remain in Developing the BMDS:
MDA Lacks Controls and Mechanisms Needed to Establish Transparency,
Accountability, and Oversight:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Department of Defense:
Appendix II: Scope and Methodology:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: MDA's BMDS Elements:
Table 2: Fiscal Year 2009 BMDS Test Achievements:
Table 3: BMDS Deliveries and Total Fielded Assets as of September 30,
2009:
Table 4: BMDS Test and Target Issues:
Table 5: Status of Fiscal Year 2008 Director's Knowledge Points to Be
Achieved through Tests:
Table 6: MDA Commitments to Improve Transparency, Accountability, and
Oversight:
Abbreviations:
ABL: Airborne Laser:
Aegis BMD: Aegis Ballistic Missile Defense:
AN/TPY-2: Army Navy/Transportable Radar Surveillance - Model 2:
BMDS: Ballistic Missile Defense System:
C2BMC: Command, Control, Battle Management, and Communications:
DOD: Department of Defense:
GBI: Ground-based Interceptor:
GMD: Ground-based Midcourse Defense:
MDA: Missile Defense Agency:
MDEB: Missile Defense Executive Board:
SM-3: Standard Missile-3:
STSS: Space Tracking and Surveillance System:
THAAD: Terminal High Altitude Area Defense:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
February 25, 2010:
Congressional Committees:
Fiscal year 2009 was an unprecedented year of transition for the
Missile Defense Agency (MDA) as it experienced its first change of
administration, its third MDA Director, its third Ballistic Missile
Defense System (BMDS) acquisition strategy, and a reduction of
approximately $1 billion from its annual budget request. Established
in 2002 and directed by the President to "deploy a set of initial
missile defense capabilities beginning in 2004," MDA has developed and
deployed missile defense capabilities while struggling to provide the
transparency and accountability necessary for the Department of
Defense's (DOD) largest single acquisition program--spending between
approximately $7 billion and $9.5 billion per year.
Since 2002, Congress has directed GAO to assess MDA's annual fiscal
year cost, schedule, testing, and performance progress in developing
the BMDS.[Footnote 1] We have delivered assessments of MDA's progress
covering fiscal years 2003 through 2008 and are currently mandated to
continue delivering assessments through fiscal year 2013.[Footnote 2]
According to this mandate we are required to assess MDA's fiscal year
progress against the annual goals it is required to report to Congress
each February. However, as agreed with your staff, this year we are
providing a more limited assessment of MDA's progress than we have in
the past because MDA removed key components of the schedule and
performance goals it had previously reported from its annual report of
goals. MDA also did not report the total acquisition cost goals that
it had committed to include in this year's report, nor did it report
top-level test goals. In addition, the agency did not report its
annual goals until August 2009--only a month and a half before the end
of fiscal year 2009.
This report provides an assessment of MDA's cost, schedule, testing,
and performance progress in fiscal year 2009 as well as problems and
challenges in these areas. We also report on the progress MDA made in
improving accountability and transparency over the past year. Although
prior reports have included an assessment of MDA's cost progress on
its prime contracts, we will report on this analysis in more detail in
a separate report. In addition, given the number, scope and breadth of
changes made to missile defense this year, we discuss those changes as
well as their implications on acquisitions.
In order to provide some measure of MDA's progress, given the late and
limited goals established by MDA, we had to determine alternative
comparison points and we discussed these with MDA and our
congressional clients. We assessed MDA's progress in testing by
comparing the tests conducted against the goals as presented in its
October 2008 Integrated Master Test Plan and the annually submitted
budget justification documents. In terms of performance, we assessed
the capabilities demonstrated through the tests. For schedule, we
compared progress against fiscal year 2008 goals, budget justification
material, and the September 2008 execution master fielding schedule.
To assess progress during fiscal year 2009, we examined the
accomplishments of eight BMDS elements that MDA is currently
developing and fielding: the Aegis Ballistic Missile Defense (Aegis
BMD); Airborne Laser (ABL); BMDS Sensors; Command, Control, Battle
Management, and Communications (C2BMC); Ground-based Midcourse Defense
(GMD); Space Tracking and Surveillance System (STSS); Targets and
Countermeasures; and Terminal High Altitude Area Defense (THAAD).
[Footnote 3] These elements collectively account for about 76 percent
of MDA's research and development budget.[Footnote 4] We also examined
MDA's Fiscal Year 2009 BMDS Accountability Report, Program Execution
Reviews, test plans and reports, and production plans. We interviewed
officials within program offices and within MDA functional
directorates, such as the Directorate for Advanced Technology. In
addition, we discussed the elements' test programs and results with
the BMDS Operational Test Agency and DOD's Office of the Director,
Operational Test and Evaluation.
To follow up on the progress MDA made to improve transparency and
accountability, we held discussions with officials in MDA's
Directorate of Business Operations to discuss MDA's block structure
termination. In addition, we reviewed pertinent DOD policies to
compare MDA's current level of accountability with that of other DOD
programs. We interviewed officials from the Office of the Under
Secretary of Defense for Acquisition, Technology and Logistics to
discuss its role in conducting quarterly execution reviews as well as
the oversight role of the Missile Defense Executive Board. Lastly, we
met with officials involved in MDA's Integration Synchronization Group
to discuss how the agency is managing and reporting against its
internal baselines. Our scope and methodology is discussed in more
detail in appendix II.
We conducted this performance audit from April 2009 to February 2010
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
MDA's BMDS is being designed to counter ballistic missiles of all
ranges--short, medium, intermediate, and long. Since ballistic
missiles have different ranges, speeds, sizes, and performance
characteristics, MDA is employing an integrated and layered
architecture to provide multiple opportunities to destroy ballistic
missiles before they can reach their targets. The system's
architecture includes networked space-based sensors as well as ground-
and sea-based radars, ground-and sea-based interceptor missiles, and a
command and control, battle management, and communications network
providing the warfighter with the necessary communication links to the
sensors and interceptor missiles.
A possible engagement scenario to defend against an intercontinental
ballistic missile would occur as follows:
* Infrared sensors aboard early-warning satellites detect the hot
plume of a missile launch and alert the command authority of a
possible attack.
* Upon receiving the alert, land-or sea-based radars are directed to
track the various objects released from the missile and, if so
designed, to identify the warhead from among spent rocket motors,
countermeasures, and debris.
* When the trajectory of the missile's warhead has been adequately
established, an interceptor--consisting of a kill vehicle mounted atop
a booster--is launched to engage the threat. The interceptor boosts
itself toward a predicted intercept point and releases the kill
vehicle.
* The kill vehicle uses its onboard sensors and divert thrusters to
detect, identify, and steer itself into the warhead. With a combined
closing speed of approximately 10 kilometers per second (22,000 miles
per hour), the warhead is destroyed above the atmosphere through a
"hit to kill" collision with the kill vehicle.
Some interceptors use sensors to steer themselves into the inbound
ballistic missile. Inside the atmosphere, weapon systems kill the
ballistic missile using a range of mechanisms, such as direct
collision between the interceptor missile and the inbound ballistic
missile, or using the combined effects of a blast fragmentation
warhead (heat, pressure, and shrapnel) in cases where a direct hit
does not occur.
In the August 2009 BMDS Accountability Report, MDA presents the BMDS
performance from the perspectives of homeland defense and regional/
theater capabilities. Homeland defense uses the capabilities of Ground-
based Interceptors (GBI), Aegis BMD assets, and BMDS radars against
the threat from intercontinental and intermediate-range ballistic
missiles, while regional and theater defense use Aegis BMD Standard
Missile-3 (SM-3) and THAAD interceptors with mobile radars against
threats from medium-range missiles and short-range ballistic missiles.
Table 1 provides a brief description of eight BMDS elements that are
currently under development by MDA.
Table 1: MDA's BMDS Elements:
BMDS element: Aegis Ballistic Missile Defense;
Missile defense role: Aegis BMD is a ship-based missile defense system
designed to destroy short-to intermediate-range ballistic missiles
during the midcourse phase of their flight; its capability has been
expanded to include the terminal phase of flight using the Standard
Missile-2 Block IV missile. Aegis BMD's mission is twofold: to provide
an engagement capability against regional ballistic missile threats
that is mobile, global, and deployable and can destroy ballistic
missiles both above and within the atmosphere, as well as a forward
deployed combatant to search, detect and track ballistic missiles of
all ranges and transmit track data to the BMDS, performing a strategic
role in homeland defense. To date, 19 ships have been upgraded for the
Aegis BMD mission. MDA is planning to procure 329 Aegis BMD SM-3
missiles from fiscal years 2004 through 2018.
BMDS element: Airborne Laser;
Missile defense role: ABL is an air-based missile defense system
designed to destroy all classes of ballistic missiles during the boost
phase of their flight. ABL employs a high-energy chemical laser to
rupture a missile's motor casing, causing the missile to lose thrust
or flight control. MDA plans to demonstrate proof of concept in a
series of system demonstrations in 2010 where the ABL will attempt to
shoot down a ballistic missile. The current program is not expected to
result in an operational system.
BMDS element: BMDS Sensors;
Missile defense role: MDA is developing various stand-alone radars for
fielding. These include forward-based sensors; mobile, sea-based
sensors; and upgrades to existing early-warning radars. The BMDS uses
these sensors to identify and continuously track ballistic missiles in
all phases of flight.
BMDS element: Command, Control, Battle Management and Communications;
Missile defense role: C2BMC is the integrating element of the BMDS.
Its role is to provide deliberate planning, situational awareness,
sensor management, and battle management for the integrated BMDS.
BMDS element: Ground-based Midcourse Defense;
Missile defense role: GMD is a ground-based missile defense system
designed to destroy intercontinental ballistic missiles during the
midcourse phase of their flight. Its mission is to protect the U.S.
homeland against ballistic missile attacks from North Korea and the
Middle East. MDA is planning on emplacing 30 operational interceptors
at Fort Greely, Alaska, and Vandenberg Air Force Base, California, by
the end of fiscal year 2010.
BMDS element: Space Tracking and Surveillance System;
Missile defense role: In fiscal year 2009, MDA successfully launched
two low-orbit demonstration satellites. The program will pursue a
testing schedule to demonstrate STSS's capabilities, including missile
detection and tracking throughout all phases of flight and intercept
assessment in the context of the BMDS. Lessons learned from the STSS
satellites will inform the design of a Precision Tracking Space Sensor
experimental prototype and associated command and control battle
management, communication and fire control networks.
BMDS element: Targets and Countermeasures;
Missile defense role: MDA maintains a series of targets used in BMDS
flight tests to present authentic threat scenarios. The targets are
designed to encompass the full spectrum of threat missile ranges and
capabilities. Under its Flexible Target Family, MDA is currently
developing one long-range 72-inch target, the LV-2, which can be
modified in various ways to represent evolving threats. The first
launch of the LV-2 target is scheduled for 2010.
BMDS element: Terminal High Altitude Area Defense;
Missile defense role: THAAD is a ground-based missile defense system
designed to destroy short-and medium-range ballistic missiles during
the late-midcourse and terminal phases of flight. Its mission is to
defend deployed U.S. forces and population centers. MDA plans to field
a THAAD Battery, which includes 24 missiles, in 2010 and a second
battery in 2011.
Source: MDA data.
[End of table]
Significant Adjustments Made to Missile Defense in Fiscal Year 2009:
The new administration proposed significant changes to the BMDS
program in 2009 including program terminations and changes to some of
the BMDS elements we reported on in the past, as well as changes to
plans for missile defense in Europe. Administration proposals
culminated in reductions of approximately $1 billion from MDA's budget
request for fiscal year 2010. In the spring of 2009, the Secretary of
Defense recommended termination of the Multiple Kill Vehicle element.
Originally designed as an optional warhead for all midcourse
interceptors, MDA terminated the Multiple Kill Vehicle element because
of feasibility issues raised about this technology, which was still in
its early stages of development, as well as a decision to refocus
MDA's resources on new technologies aimed at early intercept of
ballistic missiles.[Footnote 5] MDA also terminated its Kinetic Energy
Interceptor element because of technical issues, its incompatibility
with operational infrastructures, and delays during development. It
was originally designed as a mobile land-based missile defense system
to destroy medium, intermediate, and intercontinental ballistic
missiles during the boost and midcourse phases of their flight. The
ABL program was also significantly affected by the Secretary of
Defense's proposal to designate it as a technology program and cancel
the plans for the purchase of a second aircraft that would have
provided an operational capability. In addition, MDA requested
increased funding for the Aegis BMD and THAAD programs for fiscal year
2010 following administration recommendations. MDA plans to use these
funds to move both elements toward meeting full funding policies, to
increase production for Aegis BMD and THAAD interceptors, to increase
the interceptor production rate and number of THAAD batteries, and to
increase the number of Aegis BMD ships.
MDA is also responding to the new administration's shift in its
approach to European missile defense. In September 2009, DOD altered
its approach to European defense, which originally focused on GBIs
from the GMD element and a large fixed radar as well as transportable
X-Band radars, and is now focusing on providing defenses against long-
range threats to the United States and short-, medium-, and
intermediate-range Iranian threats to Europe. This new "Phased,
Adaptive Approach" consists primarily of Aegis BMD sea-based and land-
based systems and interceptors, as well as various sensors to be
deployed over time as the various capabilities are matured. According
to DOD, this new approach offers a number of improvements over the
previous architecture, such as providing missile defenses sooner with
greater flexibility to meet evolving threats, providing more
opportunities to involve close allies, and delivering greater
capability to defend against a large number of threat missiles.
In addition, during fiscal year 2009, MDA transitioned to a new
Director and the agency's development effort was rebalanced to focus
more on regional/theater missile defense. This rebalancing included
shifting technology development efforts from boost-phase intercept
technologies to early intercept technologies (or ascent phase). MDA
officials state that because early intercept technology initiates
intercept as early as possible to execute a shoot-look-shoot tactic
and defeat a threat before countermeasures are deployed, it will
ultimately reduce the number of interceptors required to defeat a raid
of threat missiles and save on the costs of maintaining a significant
number of expensive interceptors to destroy advanced countermeasures
in a later phase of a threat missile's flight. According to the MDA
Director, this technology will force the deployment of countermeasures
early in flight where they are less effective.
In June 2009, MDA also began to change its acquisition management
strategy. From its inception in 2002 to December 2007, MDA managed the
acquisition of missile defense capabilities by organizing the
development effort into 2-year increments known as blocks. Each block
was intended to provide the BMDS with capabilities that enhanced the
development and overall performance of the system. The first 2-year
block--Block 2004--fielded a limited initial capability that included
early versions of the GMD, Aegis BMD, Patriot Advanced Capability-3,
and C2BMC elements as well as various sensors. The agency's second 2-
year block--Block 2006--culminated on December 31, 2007, and fielded
additional BMDS assets. On December 7, 2007, according to MDA in
response to recommendations from GAO, MDA's Director announced a new
acquisition management strategy to better communicate its plans and
goals to Congress. The agency's new approach was based on fielding
capabilities that address particular threats as opposed to a biennial
time period. This approach divided fielding capabilities into five
blocks.[Footnote 6]
The capabilities-based five-block approach included several positive
changes, including the commitment by DOD to establish total
acquisition costs and unit cost for selected block assets, including
in a block only those elements or components that will be fielded
during the block, and abandoning the practice of deferring work from
one block to another. MDA was still transitioning to this new
capabilities-based block approach when the MDA Director terminated it
in June 2009--a year and a half after it was created. According to
MDA, the agency terminated the capability-based block structure to
address the explanatory statement accompanying the Department of
Defense Appropriations Act, 2009, which stated that MDA's
"justification materials should no longer be presented in the Block
format, but rather by fiscal year for each activity within the program
element." The agency has decided that it will manage the BMDS as a
single integrated program and is in the process of determining how it
will implement changes to its acquisition management strategy.
Progress Made in Fiscal Year 2009:
In fiscal year 2009, MDA achieved several noteworthy accomplishments.
For example, MDA revised its testing approach to better align tests
with modeling and simulation needs and is undertaking a new targets
development effort to resolve long-standing problems supplying
sufficient and reliable targets. The agency also demonstrated
increased levels of performance for some of its BMDS elements through
flight and ground testing. MDA testing achievements during the year
indicate an increased level of interoperability among multiple
elements, improving both system-level performance and advancing the
validation of BMDS models and simulations needed to predict
performance. In addition, the agency delivered most of the assets as
planned by the end of fiscal year 2009.
MDA Revised Its Testing Approach:
In fiscal year 2009, MDA revised its testing approach in response to
GAO and DOD concerns. In March 2009 we reported that MDA's Integrated
Master Test Plan--its test baseline--was not effective for management
and oversight because it was revised frequently, only extended through
the following fiscal year and was not well integrated with other key
aspects of testing such as target acquisitions.[Footnote 7] Most of
the annual revisions to the test baseline were occurring either
because MDA changed the substance of tests, changed the timing of
tests, or added tests to the baseline. In other instances, MDA
canceled planned tests which also affected the test baseline. In
addition, the BMDS Operational Test Agency identified several
limitations in the previous BMDS test program, including unaccredited
models and simulations, flight test artificialities, and inadequate
modeling of some environmental conditions.[Footnote 8]
Members of Congress also expressed concern with MDA's test approach.
For example, in the fiscal year 2008 National Defense Authorization
Act conference report, conferees noted that MDA failed to ensure an
adequate testing program and that its test and targets program needed
to be managed in a way that fully supported high-priority near-term
programs.
MDA extensively revised its test plan in fiscal year 2009 to address
many of these concerns. For example, the new Integrated Master Test
Plan bases test scenarios on modeling and simulation needs and extends
the test baseline through 2015, which allows for better estimation of
target needs, range requirements, and test assets. As part of the
revised test plan, MDA scheduled dedicated periods of developmental
and operational testing, during which the system configuration will
remain fixed to allow the warfighter to carry out training, tactics,
techniques, and procedures for developmental and operational
evaluation. Additionally, the new test plan will provide sufficient
time after test events to conduct a full post-test analysis. These
improvements are important because BMDS performance cannot be fully
assessed until models and simulations are accredited and validated and
the test program cannot be executed without meeting its targets needs.
MDA Began New Target Development Effort:
In July 2009, MDA also initiated a new target acquisition strategy to
address recurring target performance issues and increases in target
costs. According to the Director of MDA, this new target approach is
based on streamlining a set of classes of targets to increase quality
control of an inventory of identical targets that represent general
threat characteristics to account for intelligence uncertainties. He
further stated that a goal of the new target acquisition strategy is
to minimize the number of targets needed to emulate specific threats
and establish backup targets, which will be available in 2012.
Targets have been a recurring cause of flight test delays,
cancellations, and failures since 2006. In the past, we reported that
the THAAD program was unable to achieve its first intercept attempt in
2006 because the target malfunctioned.[Footnote 9] The program also
experienced target anomalies in 2007 that precluded the completion of
two radar characterization tests. During the same year, the GMD
program experienced long-term effects on its flight test schedule when
it was unable to achieve all primary test objectives because of a
target failure. We also reported in March 2009 that the Aegis BMD
program was unable to conduct an intercept because the target was not
available.[Footnote 10] In addition, in its January 2009 report to the
defense committees, MDA acknowledged target availability and
reliability problems and reported its plan for a new target
acquisition strategy to address these issues and improve costs,
quality, and reliability.
In revising its target acquisition strategy, MDA solicited input from
industry in an effort to better understand possible new target
solutions that might be available to improve cost, quality, and
performance. To leverage industry capability and promote a more
competitive contract environment, MDA decided to use multiple
contractors with multiple contracts instead of a single prime
contract, increasing its flexibility to respond to changing program
requirements. The agency plans to award a new contract for each class
of target needed to execute the BMDS test plan. MDA will begin making
decisions on contract awards and new target designs over the next
year. According to program officials, MDA originally planned to issue
five requests for proposals for new contracts in fiscal year 2010 and
one additional request in fiscal year 2011. However, to reflect
changes in the test plan and subsequent changes to the acquisition
strategy, the program now plans to issue two requests for proposals in
fiscal year 2010 and one in fiscal year 2011. The Targets and
Countermeasures program anticipates that the first targets will be
delivered under the new strategy in fiscal year 2012, and the first
intercontinental ballistic missile target is expected to be delivered
in fiscal year 2013.
MDA also made progress in several ongoing target development efforts
that could enhance the ability to test the BMDS. During fiscal year
2009, the Targets and Countermeasures program made progress in
developing four new targets--the LV-2 target, Aegis Readiness
Assessment Vehicle-C target, a new medium range target, and the
Extended-Long Range Air Launched Target. Each target adds a new
capability to MDA's target portfolio. For example, the LV-2 target
provides the potential for significantly expanding the intermediate
range payload and range performance over current inventory
capabilities. The Aegis Readiness Assessment Vehicle-C target provides
a new, low-cost capability as it is designed to contribute additional
separating and maneuvering capabilities in short-and medium-range
targets. MDA's new Medium Range Target provides improved kill
assessment capability at this range.[Footnote 11] In addition, the new
Extended-Long Range Air Launched Target is a medium-range target that
provides a greater range capability than previous air-launched targets
and adds the ability to deploy associated objects--a capability not
currently available in other similar target types. MDA expects each of
these targets to be ready for use in flight tests in fiscal year 2010.
[Footnote 12]
MDA Demonstrated Some Improved Performance through Testing:
In fiscal year 2009, MDA conducted several ground tests and flight
tests demonstrating improved performance in several areas of the BMDS
including element-level functionality, theater and regional
performance, and interoperability. Table 2 identifies key test events
achieved in fiscal year 2009 for each element.
Table 2: Fiscal Year 2009 BMDS Test Achievements:
Element: ABL;
Significant test achievements:
* Achieved first live tracking tests against boosting targets;
* Achieved first firing of high energy laser in flight.
Element: Aegis BMD;
Significant test achievements:
* Verified capability to destroy multiple targets during an engagement.
Element: C2BMC;
Significant test achievements:
* Demonstrated capability to display information from both strategic
and theater scenarios on one common operational picture;
* Displayed situational awareness data for Patriot units.
Element: GMD;
Significant test achievements:
* Utilized correlated sensor data from four sensors to intercept a
target.
Element: Sensors;
Significant test achievements:
* Demonstrated that the upgraded early warning radar could conduct
missile warning, space surveillance, and missile defense during a live
intercept of a target missile.
Element: STSS;
Significant test achievements:
* Completed ground testing, integration of components, and launch of
two demonstration satellites.
Element: THAAD;
Significant test achievements:
* Conducted a salvo launch of two interceptors against single target.
Sources: GAO (presentation); MDA (data).
[End of table]
In June 2009, the ABL program successfully completed its first two
tracking tests against boosting missile targets. These tests marked
the first time ABL demonstrated a complete low-power engagement
sequence against a boosting target. In addition, the ABL was able to
demonstrate its ability to fire its high energy laser in an airborne
environment during a flight test in August 2009. During this test, the
laser was fired into a calorimeter on board the aircraft to capture
the laser's energy and measure performance characteristics of the
laser's beam.
The Aegis BMD program also demonstrated increased levels of element
performance through Navy fleet exercises and developmental tests. For
example, Aegis BMD demonstrated, for the first time, its capability to
destroy a ballistic missile in the terminal phase of flight using
Standard Missile-2 Block IV missiles while simultaneously conducting a
mission using the Standard Missile-2 Block IIIA missile against a
cruise missile target. The program also conducted successful
developmental component tests for the next generation of the Aegis BMD
interceptor--the SM-3 Block IB. Developmental testing will continue
into 2010. In addition, the program successfully demonstrated that the
latest software release of the Aegis BMD system had the capability to
support the program's next generation interceptor during simulated SM-
3 Block IB engagements.
The C2BMC program also satisfied multiple test objectives and
increased its capability in fiscal year 2009. The program participated
in many system-level tests during the year that enabled it to
demonstrate multiple capabilities, including improved situational
awareness and sensor management.[Footnote 13] During testing, C2BMC
used multi-sensor correlation and provided integrated situational
awareness for weapons release decisions.
GMD, for the first time, used information from multiple sensors to
develop and successfully conduct an intercept of a live target during
a flight test. In December 2008, target information from four
different sensors and satellite data were input into the GMD fire
control system to develop an intercept plan. The involvement of
multiple sensors provides better information to develop an engagement.
In addition, GMD made progress in addressing BMDS Operational Test
Agency concerns regarding the formatting, tracking, and accounting of
messages from GMD sensors. For example, MDA added test instrumentation
to collect data for regional/theater tests communications. However,
the agency still faces ongoing challenges assessing timeliness with
the exchange of messages at the strategic level. According to BMDS
Operational Test Agency officials, they continue to work with MDA to
resolve this issue. Key to the integration and functionality of the
BMDS is communications and message traffic. The timely reception of
messages from sensors to weapon systems is key to support decisions
and achieve effective intercepts. In March 2009, we reported that
these data management problems prevented the analysis of the
timeliness of message data, according to BMDS Operational Test Agency
officials.[Footnote 14]
The STSS program successfully completed the ground testing and
integration of components to support the launch of its two
demonstration satellites in September 2009. These satellites will use
onboard infrared sensors to detect, track, and discriminate ballistic
missiles throughout their trajectories.
THAAD also demonstrated improved element-level functionality when it
successfully launched a salvo of two THAAD interceptors to intercept a
separating target inside the earth's atmosphere. The primary
interceptor hit the target and the second interceptor hit the largest
remaining piece of target debris seconds later.
BMDS Demonstrated Improved Theater and Regional Performance and an
Increased Level of Interoperability:
Regional and theater BMDS assets--Aegis BMD and THAAD--succeeded in
demonstrating improved interoperability in fiscal year 2009. For
example, during a THAAD intercept test, Aegis BMD tracked a target and
provided the information to THAAD's fire control. As a result, the
missile was successfully engaged by THAAD. Additionally, during this
test, the forward-based radar supporting THAAD was also able to
discriminate the threat reentry vehicle from other objects and provide
the information to support the engagement. According to program
officials, the THAAD element reported that C2BMC provided accurate and
timely status information for the BMDS as well as situational
awareness of the test to the warfighter.
MDA also demonstrated interoperability for BMDS elements during
several ground tests in fiscal year 2009.[Footnote 15] For example,
during one ground test--GTD-03--MDA successfully demonstrated
simultaneous theater and regional capabilities using operational BMDS
hardware and actual communications between them. In addition, MDA
demonstrated simultaneous BMDS capabilities to conduct training while
the BMDS network remained operational during this test. This
capability allows MDA to conduct development activities while
maintaining readiness to engage in missile defense operations. This
ground test also allowed several BMDS elements to demonstrate that
they could successfully exchange data with other elements.
Additionally, in December 2008 numerous elements worked together to
support system-level post-flight reconstruction needed to validate
BMDS models and simulations.[Footnote 16] This system-level post-
flight reconstruction for flight test FTX-03 was the first ever and
was highly successful because different MDA groups achieved the same
results, according to MDA officials.
MDA Made Progress Developing and Integrating Models and Simulations:
MDA took significant steps forward in fiscal year 2009 in developing
the modeling and simulation tools necessary to understand BMDS
performance against strategic and theater/regional threats. Because
the potential combinations of BMDS configurations, intercept
scenarios, and missile threats are too numerous for ground and flight
testing, assessing overall BMDS performance depends upon the use of
models and simulations to understand the capabilities and limitations
of the system. Such an end-to-end system-level simulation brings
together the capabilities of various element models in order to
analyze how the BMDS integrated and fielded radars, communication
networks, and interceptors perform during scenarios.[Footnote 17]
However, to work effectively these models and simulations need to be
anchored to data from ground and flight tests and validated by
independent evaluators--the BMDS Operational Test Agency--in order to
have confidence in their results.[Footnote 18] Moreover, the system-
level simulation itself is expected to change over time as additional
models become available to represent the evolving BMDS configuration.
In March 2009 we reported that MDA experienced several problems in its
overall modeling and simulation program, which negatively affected the
2007 performance assessment and led to the cancellation of the 2008
performance assessment.[Footnote 19] Performance Assessment 2007 was
unsuccessful primarily because of inadequate flight and ground test
data for verification and validation to support accreditation and a
lack of common threat and environment input data among element models.
MDA officials canceled their 2008 performance assessment efforts in
April 2008 because of developmental risks associated with modeling and
simulations, focusing instead on testing and models for Performance
Assessment 2009.
In fiscal year 2009, MDA made some progress integrating the individual
element models and simulations for Performance Assessment 2009. A
leading accomplishment was the development of a system-level
simulation for regional and theater scenarios in addition to existing
strategic scenarios for a more complete analysis of BMDS performance.
Performance Assessment 2007 only included homeland defense scenarios
against strategic threats. One of MDA's goals for the performance
assessment is the integration of models that communicate like the
networked BMDS. As of October 2009, Performance Assessment 2009
achieved interactive communications among the element models and
simulations. In addition, MDA achieved consistency in representing the
threat missile and post-intercept data among all models and scenarios,
which was also a weakness of Performance Assessment 2007. Finally, the
BMDS Operational Test Agency observed that conducting Performance
Assessment 2009 is helping to build confidence in BMDS-level
simulation capability for the subsequent Performance Assessment 2010.
MDA Delivered Many Assets as Planned in Fiscal Year 2009:
In fiscal year 2009, MDA met many of its delivery goals. Four MDA
elements--Aegis BMD, GMD, Sensors, and C2BMC--were scheduled to
deliver a total of 41 assets and capabilities in fiscal year 2009. MDA
delivered 34 of these assets or 83 percent. Table 3 outlines BMDS
asset deliveries in fiscal year 2009.
Table 3: BMDS Deliveries and Total Fielded Assets as of September 30,
2009:
BMDS element: Aegis BMD;
Fiscal year 2009 delivery goals: 10 SM-3 missiles;
Assets delivered in fiscal year 2009: 19 SM-3 missiles;
Total assets available (cumulative total of assets since 2005):
47 SM-3 missiles.
BMDS element: Aegis BMD;
Fiscal year 2009 delivery goals: 20 Aegis BMD Weapon System 3.6.1
installations;
Assets delivered in fiscal year 2009: 18 Aegis BMD Weapon System 3.6.1
installations;
Total assets available (cumulative total of assets since 2005):
18 Aegis BMD Weapon System 3.6.1 installations;
1 Aegis BMD Weapon System 4.0.1 installation.
BMDS element: GMD;
Fiscal year 2009 delivery goals: GBIs number 28-30;
Assets delivered in fiscal year 2009: GBI 28;
Total assets available (cumulative total of assets since 2005):
28 GBIs[A].
BMDS element: GMD;
Fiscal year 2009 delivery goals: 1 additional silo;
Assets delivered in fiscal year 2009: 1 silo;
Total assets available (cumulative total of assets since 2005):
31 silos[B].
BMDS element: C2BMC;
Fiscal year 2009 delivery goals: 1 fielding and activation site;
Assets delivered in fiscal year 2009: None[C];
Total assets available (cumulative total of assets since 2005):
7 combatant command suites;
64 Web browsers;
54 enterprise workstations;
4 Global Engagement Manager workstations;
1 Global Engagement Manager suite.
BMDS element: Sensors;
Fiscal year 2009 delivery goals: 1 additional AN/TPY-2 radar;
Site construction, deployment, activation, and fielding for 1AN/TPY-2
radar;
Near-term discrimination software;
Thule radar upgrades;
Thule radar site construction;
Thule radar communication upgrade;
Assets delivered in fiscal year 2009: 1 AN/TPY-2 radar;
Site construction, deployment, activation, and fielding for 1AN/TPY-2
radar;
Near-term discrimination software;
Thule radar site construction;
Total assets available (cumulative total of assets since 2005):
6 AN/TPY-2 radars;
2 operational sites;
1 Near-term discrimination software;
1 Thule radar site construction.
Sources: GAO (presentation); MDA (data).
Note: Goals for GMD, Sensors and C2BMC asset deliveries came from the
Fiscal Year 2008 BMDS Block Baselines and Goals, while Aegis BMD
delivery goals were informed by Aegis BMD budget justification
documents.
[A] The total includes GBIs numbered 25, 26, and 27 which were
delivered in fiscal year 2009, 1 year after their initially scheduled
delivery date in fiscal year 2008.
[B] The total includes 26 silos at Fort Greely and 5 silos (one for
test use only) at Vandenberg Air Force Base.
[C] Program officials told us that C2BMC's delays were not driven by
technical issues but by programmatic changes needed by other elements,
the warfighter, and the new administration.
[End of table]
Aegis BMD planned to install the Aegis Weapons System 3.6.1 software
on 20 ships and deliver 10 SM-3 missiles in fiscal year 2009. The
program met its goal to deliver the 10 missiles and began to deliver
additional rounds, initially designated for 2010, ahead of schedule.
However, the program fell behind on its goal of installing the 3.6.1
software on 20 ships, delivering 18 by the end of the fiscal year
2009. Aegis BMD officials pointed out that all ship sets were
available but because of real-world national security situations,
these ships were not available for installations in fiscal year 2009.
Nonetheless, one of the remaining ships was completed in December 2009
and another will be completed by March 2010. In fiscal year 2009,
Aegis BMD also delivered an additional ship set with the next
generation Aegis BMD Weapon System, 4.0.1, for a total of 19 ship
deliveries.
The GMD program also partially met its delivery goals in fiscal year
2009. The program delivered an additional silo at Vandenberg Air Force
Base as planned, but lagged in its GBI deliveries. For example, in
fiscal year 2009, GMD emplaced three interceptors that were initially
planned for fiscal year 2008 and only one of the three interceptors
planned for fiscal year 2009.
The Sensors program met most of its delivery goals, successfully
fielding a new near-term discrimination algorithm, activating an
additional AN/TPY-2 radar site, and delivering an additional AN/TPY-2
radar. However, it fell short of meeting all of its delivery goals for
the fiscal year. Although the program completed the construction for
the Thule radar site ahead of schedule in fiscal year 2008, it was
unable to deliver Thule radar communications and upgrades as planned
in fiscal year 2009. These activities have been delayed until fiscal
year 2010.
Finally, C2BMC delivered four additional C2BMC Web browsers, five work
stations, and an additional combatant command suite. Additionally, the
program office rolled out the Global Engagement Manager suite and
added four work stations that support it. However, it was unable to
meet its schedule baseline goal of an additional fielding and site
activation to declare its next spiral operational. This was due to
major program restructures needed to accelerate C2BMC capabilities for
other BMDS elements as well as programmatic changes to fulfill
warfighter requests and meet new administration direction.
Significant Challenges Remain in Developing the BMDS:
While there was progress in addressing concerns about test planning
and target development as well as in delivering assets, all BMDS
elements experienced delays in conducting tests, were unable to
accomplish all planned objectives, and experienced performance
challenges. Poor target performance continued to be a problem causing
several test delays and leaving several test objectives unfulfilled.
The test problems also precluded the agency from gathering key
knowledge through tests specified by the MDA Director that were
originally planned to be completed in fiscal year 2008.[Footnote 20]
MDA's efforts to develop advanced algorithms and its efforts to
demonstrate homeland defense were also affected by target issues.
These shortfalls in testing continued to delay validation of the
models and simulations used to assess the overall performance of the
BMDS. Consequently, comprehensive assessments of the capabilities and
limitations of the BMDS are still not possible. MDA also redefined its
schedule baseline, eliminating goals for delivering integrated
capabilities so we were not able to assess MDA's progress in this key
area.
MDA Experienced Testing Delays and Shortfalls Across the Board in
Fiscal Year 2009:
During fiscal year 2009, although several tests showed progress in
individual elements and some system-level capabilities, all BMDS
elements experienced test delays and shortfalls in part because of
problems with the availability and performance of target missiles.
None of the elements conducted all planned tests as scheduled and none
achieved all planned objectives. Table 4 outlines BMDS test and target
issues in fiscal year 2009.
Table 4: BMDS Test and Target Issues:
Element: ABL;
Issues: Optics failure experienced during test. Delayed first full
demonstration test from fiscal year 2009 until fiscal year 2010.
Element: Aegis BMD;
Issues: Target availability, test range requirements, and programmatic
changes delayed key test from fiscal year 2008 until at least 2013.[A]
Element: C2BMC;
Issues: Changes in the scope of C2BMC's ongoing development effort as
well as issues with BMDS-level models and simulation tools have
delayed C2BMC's spiral capability development testing.[B]
Element: GMD;
Issues: Target failed to release countermeasures during December 2008
flight test--FTG-05;
target modeling delayed subsequent test--FTG-06--until January 2010.[C]
Element: Sensors;
Issues: Target failures prevented Sensors element from developing
discrimination capability as planned.
Element: STSS;
Issues: Integration difficulties delayed launch of demonstration
satellites.
Element: Targets and Countermeasures;
Issues: Flexible Target Family delivery delayed and experienced cost
growth.
Element: THAAD;
Issues: Target availability reduced planned fiscal year 2009
objectives.
Source: GAO (presentation); MDA (data).
[A] According to Aegis BMD officials, this test has been canceled,
however several objectives will be included in FTM-23 scheduled for
2013.
[B] Program officials told us that C2BMC's scope changes were not
driven by technical issues but by programmatic changes needed by other
elements, the warfighter, and the new administration.
[C] FTG-06 was conducted on January 31, 2010, but did not achieve all
of its objectives. The GBI failed to intercept the target as planned.
According to an MDA official, the agency has convened a Failure Review
Board and expects results from its investigation to take months.
[End of table]
Two BMDS elements--ABL and C2BMC--experienced delays in achieving
fiscal year 2009 test events. For example, ABL experienced delays in
development and ground testing that resulted in the delay of its first
full flight test demonstration until fiscal year 2010. Additionally,
C2BMC was unable to conduct testing needed to further develop its next
spiral capability because of BMDS-level delays in developing the
models and simulations needed to conduct this testing. Major program
restructures needed to accelerate C2BMC capabilities for other BMDS
elements and programmatic changes to fulfill warfighter requests and
meet new administration direction also contributed to C2BMC's
inability to conduct planned fiscal year 2009 testing.
As noted in table 4, targets affected the BMDS test program for four
elements in fiscal year 2009. The Aegis BMD, GMD, Sensors, and THAAD
test program were affected by either target availability or target
reliability and performance issues. In fiscal year 2009, targets
contributed to a test cancellation and test delays and prevented
elements from completing tests or achieving all test objectives. One
test for Aegis BMD--FTM-15--was originally projected to use the new
Flexible Target Family's LV-2 target in fiscal year 2008, but because
of qualification difficulties, the target was unavailable and the test
was not conducted. This test was planned as the first Aegis BMD SM-3
engagement against an intermediate-range target. It was also expected
to verify interoperability of Aegis BMD, a Sensors radar, and C2BMC.
As of December 2009, MDA had canceled the test and planned to combine
several of the FTM-15 objectives with those in a future flight test in
2013--FTM-23. However, as of February 2010, the Director of MDA stated
that the test is being rescheduled for 2011. Test documentation was
not provided for our review so it remains unclear whether the test
will include the original test objectives, target, and BMDS hardware
and software configurations.
The GMD and Sensors programs were also unable to complete all planned
objectives because of a target failure during an intercept test.
During a December 2008 flight test--FTG-05--the target failed to
release planned countermeasures. A similar target failure was
experienced in a prior 2008 test--FTX-03--and MDA's risk assessments
leading up to the FTG-05 test could not determine the root cause of
the failure. These risk assessments determined that a similar failure
would be "likely" and the consequences "severe" if MDA proceeded with
the test in December 2008, even after taking mitigation steps.
[Footnote 21] According to the Defense Contract Management Agency, the
cost to execute FTG-05 exceeded $210 million. This was the last
planned flight test using this type of target. As a result of the
target failure, GMD was unable to assess the Capability Enhancement-I
kill vehicle against countermeasures. According to the July 2009
Integrated Master Test Plan, this test is now planned to be conducted
in the third quarter of fiscal year 2011--nearly 4 years after this
configuration completed fielding.
The GMD program had to delay its second planned fiscal year 2009
intercept test--FTG-06--to fiscal year 2010 because pretest analysis
raised concerns that the target may not perform as required. This test
was important because it was planned as the first test of GMD's
enhanced version of the kill vehicle called the Capability Enhancement
II exoatmospheric kill vehicle. This test was also designed to
demonstrate a long-flight time for the GBI and GMD's capability
against countermeasures. In early 2009, MDA altered the target to
present a more representative threat. Since MDA did not have modeling
data to represent the new characteristics of the target, MDA officials
were concerned about the target's expected performance and decided to
delay the test. In January 2010, MDA conducted FTG-06. However, all
test objectives were not met as the GBI failed to intercept the target
as planned. According to an MDA official, a Failure Review Board was
convened to investigate the test results, but its investigation is
expected to take months to complete.
As we reported in March 2009, THAAD program officials had to
reschedule the planned fiscal year 2008 BMDS-level event, FTT-10, into
fiscal year 2009 because of a target malfunction.[Footnote 22] THAAD
successfully completed this test event in fiscal year 2009. In
addition, a Short Range Air Launch Target planned for use in a third
quarter fiscal year 2009 THAAD flight test, FTT-11, had a component
failure and subsequently needed to be requalified. This failure caused
the THAAD program to modify its planned flight test objectives and
move the test into fiscal year 2010, also resulting in delays to a
subsequent test--FTT-12. FTT-11 was conducted in December 2009 but
could not be completed due to failure of the target missile. The air-
launched target was successfully deployed from a transport aircraft,
but the target's rocket motor did not ignite. The THAAD interceptor
was not launched and test objectives were not achieved. According to
the Director of MDA, the Failure Review Board was concluding its
investigation of the root cause of this failure. The board's report
was not available during our audit.
Target reliability and failures in fiscal year 2009 also prevented
several elements from achieving all planned objectives. In March 2009,
Aegis BMD experienced target difficulties when two refurbished lower-
cost Army targets for a short-range mission fell short of their
expected trajectory. One target was outside the intercept control area
and Aegis BMD was not able to fire the interceptor because of safety
limitations. In the second test, the target, while short of its
expected trajectory, fell in the intercept control area and was
successfully intercepted.
It Will Be Several Years Before New Target Acquisition Approach
Addresses Problems:
It will be several years before MDA's new approach to target
development and acquisitions will be fully implemented because most
targets needed through fiscal year 2011 are already under contract and
will not be affected by the new strategy. The activities under
existing contracts will not be complete until 2013. Moreover, MDA's
implementation of a new acquisition management strategy does not
necessarily mean that any particular target currently being used, such
as the LV-2, will be phased out of the test program. MDA could decide
to continue to use an existing target under the new strategy, and as a
result, some existing target missiles could continue to be procured
under new contracts.
MDA has not presented a complete business case for proceeding with a
new target acquisition management strategy. A complete business case
includes establishing top-level cost, schedule, and performance
baselines available internally and externally for oversight.[Footnote
23] It is the essential first step in any acquisition program because
it sets the stage for acquisition and execution. Program officials
told us that they would have cost, schedule, and performance baselines
finalized and documented as part of the decision to proceed with new
contract awards. These baselines, however, will be very detailed and
spread across multiple documents and therefore are unsuitable for
internal and external oversight. The officials further stated that
they do not intend to establish top-level cost, schedule, and
performance baseline measures similar to approved program baselines
that are established for DOD's major defense acquisitions to provide
accountability.[Footnote 24] In September 2008, we reported that MDA
had difficulty in developing and supplying new targets in part because
a sound business case was not developed before significant decisions
were made. In that report we recommended that MDA develop cost,
schedule and performance baselines as part of an effort to establish a
sound business case for each new class of target under development.
[Footnote 25]
As part of the new target development efforts, MDA also developed a
new cost model. However, because the cost model and test baseline are
continually updated, the Targets and Countermeasure program continues
to lack solid cost baselines against which progress can be measured.
According to the Director of MDA, the agency will continue to update
its cost model as the Integrated Master Test Plan changes, noting that
where the technical content of the test plan remains constant, cost,
schedule, and performance baselines can be measured from year to year.
However, as we reported in March 2009, the Integrated Master Test Plan
changes frequently. In fact, the latest approved version is dated July
2009, and according to MDA's Director, a revised version of the
Integrated Master Test Plan is expected in March 2010, which limits
the baseline's stability to approximately 8 months and limits our
ability to measure MDA's progress against a cost baseline.
MDA's ability to develop an accurate cost baseline is also affected by
the lack of historical data available for targets or for other similar
missiles. Program officials said that they are now collecting more
useful cost data for new contracts by requiring more detailed cost
reporting from their contractors. This approach will allow program
officials to gather more complete and accurate data over time to make
the new cost model a more powerful cost estimating tool.
Testing and Targets Problems Affected Planned Progress of Overall BMDS
Development:
The inability of MDA to successfully conduct its test plan precluded
the agency from collecting critical information needed for key
decisions and significantly affected development of advanced
algorithms and homeland defense capabilities.
Key Director's Knowledge Points Delayed Again:
In fiscal year 2009, MDA was unable to accomplish any of the
Director's knowledge points that were to be achieved through tests.
Several of these tests were originally planned for fiscal year 2008,
but were delayed into 2009 and then again delayed into fiscal years
2010 and 2011. Table 5 shows the original test date and MDA's current
estimate for obtaining the necessary knowledge.
Table 5: Status of Fiscal Year 2008 Director's Knowledge Points to Be
Achieved through Tests:
Knowledge point: Assess capability to deliver real time engagement
tracks;
Knowledge gained: Verification of initial Global Engagement Manager
capability to support ballistic missile defense level sensor/weapon
system pairing;
Flight and ground test: GTD-03;
Original test date: Fourth quarter fiscal year 2008;
Current projection: To be determined[A].
Knowledge point: Verify 72 inch Flexible Target Family;
Knowledge gained: Confirmation of 72 inch performance. Viability of
Flexible Target Family concept to efficiently configure and transport
target to launch facility. Confidence to discontinue use of the
Strategic Target System;
Flight and ground test: FTM-15[B];
Original test date: Fourth quarter fiscal year 2008;
Current projection: Second Quarter Fiscal Year 2010.
Knowledge point: Demonstrate high acceleration booster;
Knowledge gained: Confirmation of Boost Phase Capability alternative
to ABL and High Acceleration Booster for Midcourse Defense (mobile and
fixed sites);
Flight and ground test: FTK-01;
Original test date: Fourth quarter fiscal year 2008;
Current projection: Deleted[C].
Knowledge point: Confirm Constellation affordability;
Knowledge gained: Space sensor performance against operationally
realistic targets confirmed with existing Block 2006 technology
(anchors performance-cost baseline for future STSS);
Flight and ground test: FTS-01;
Original test date: Fourth quarter fiscal year 2008;
Current projection: Deleted.
Knowledge point: Verify capability to conduct launch on the tactical
digital information link ballistic missile engagement;
Knowledge gained: Assessment of Aegis BMD 3.6 and SM-3 Block IA
performance and ability to successfully engage and intercept a long
range ballistic missile target and to use an off-board sensor's track
data via Link-16 to initiate that engagement;
Flight and ground test: FTM-15;
Original test date: Fourth quarter fiscal year 2008;
Current projection: To be determined[D].
Knowledge point: Confirm Constellation performance;
Knowledge gained: Space sensor performance against operationally
realistic targets confirmed with existing Block 2006 technology
(anchors performance-cost baseline for future STSS);
Flight and ground test: FTS-03[E];
Original test date: Fourth quarter fiscal year 2008;
Current projection: Third quarter fiscal year 2011.
Sources: GAO (presentation); MDA (data).
[A] After the knowledge point had been established in 2007, MDA split
it into two parts. Although GTD-03 was completed in the second quarter
of fiscal year 2009 and was the test that was originally set to meet
this knowledge point, MDA altered its plan. The first part of the
knowledge point, Verification of initial Global Engagement Manager
capability to support ballistic missile defense level sensor/weapon
system pairing, will meet its objectives during the GTX-04 series,
which will not be completed until the second quarter of fiscal year
2011 while the second part, Verification of Aegis Launch on C2BMC, was
originally set to meet its objectives during FTM-15. However, FTM-15
has been canceled, and program officials told us that several
objectives would be rolled into future tests, such as FTM-23.
[B] MDA changed the test to verify the 72 inch Flexible Target Family
target from FTM-15 to FTG-06. FTG-06 was conducted on January 31,
2010, but did not achieve all of its objectives. Therefore, it is
unclear whether the knowledge point was fulfilled.
[C] MDA's cancellation of the Kinetic Energy Interceptor program
during fiscal year 2009 caused its subsequent test events, including
FTK-01, to be canceled.
[D] As of December 2009, MDA had canceled the test and planned to
combine several of the FTM-15 objectives with those in a future flight
test in 2013--FTM-23. However, as of February 2010, the Director, MDA
states that the test is being rescheduled for 2011. Test documentation
was not provided for our review so it remains unclear whether the test
will include the original test objectives, target, and BMDS hardware
and software configurations.
[E] FTS-03 is now designated FTS-02.
[End of table]
Advanced Algorithm and Homeland Defense Affected:
Target issues continued to affect MDA's ability to fully develop
algorithms needed for discrimination capability. In March 2009, we
reported that multiple elements experienced test failures which caused
delays in collecting data needed to develop discrimination capability.
[Footnote 26] For example, in 2007, two THAAD radar characterization
tests were unsuccessful because of target anomalies. These tests were
designed with characteristics needed for radar observation in support
of advanced discrimination algorithm development. However, target
problems prevented an opportunity for the radar to exercise all of the
planned algorithms, causing a loss of expected data. Similarly, in a
2008 sensor characterization test, the target failed to release its
countermeasures, which prevented the sensors from collecting expected
data. Consequently, MDA was unable to fully develop discrimination
algorithms as planned.
In fiscal year 2009, MDA continued to be unable to develop its
advanced algorithms as planned as key tests that were designed to
reduce the maturation risk were affected by targets. For example, the
Sensors and GMD elements were unable to collect data to develop their
advanced algorithms when the target failed to release countermeasures
and present the expected scene complexity during FTG-05. The
subsequent delay to the next intercept test--FTG-06--until January
2010 has also reduced the data MDA had expected in fiscal year 2009
for the development of discrimination capability. Additionally, target
unavailability caused MDA to delay a THAAD test--FTT-11--from fiscal
year 2009. This test was designed to provide data for the development
of advanced algorithms for the THAAD radars. The test was conducted in
fiscal year 2010 but could not be completed because the target
malfunctioned during deployment. According to the Director of MDA, the
Failure Review Board was concluding its investigation of the root
cause of this failure. The board's report was not available during our
audit.
Likewise, GMD continues to experience delays demonstrating increased
interceptor performance for homeland defense as the two aforementioned
tests--FTG-05 and FTG-06--were not conducted as planned. As we
testified in February 2009, MDA had expected to conduct seven GMD
interceptor flight tests from the start of fiscal year 2007 through
the first quarter of fiscal year 2009. However, MDA was able to
conduct only two, which, according to the Director of Operational Test
and Evaluation, has limited the complete sets of information necessary
for validating ground-based interceptor models. MDA also delayed the
other planned flight test, FTG-06, beyond fiscal year 2009 because of
target issues and an anomaly with a component of the Sea-Based X-band
radar. As of June 2009, MDA estimated this test to cost over $236
million while the Defense Contract Management Agency estimated the
cost to exceed $310 million. These costs are likely understated
because they do not include all of the cost increases of delaying the
test first to September 2009, nor do they include any cost increases
of further delaying the test until the second quarter of fiscal year
2010.
Although the Aegis BMD missile--SM-3 Block IA--capability against an
intermediate range ballistic missile is not a requirement, MDA has
planned for years and invested millions of dollars in a plan to test
the Aegis BMD system and SM-3 Block IA interceptor against this type
of threat. At the start of fiscal year 2009, Aegis BMD officials
intended to conduct this test in the third quarter of fiscal year
2009. However, as of December 2009, MDA had canceled the test and
planned to combine several objectives with those in a future flight
test in 2013. As of February 2010, the Director of MDA stated that the
test is being rescheduled for 2011. Test documentation was not
provided for our review, so it remains unclear whether the test will
include the original test objectives, target, and BMDS hardware and
software configurations.
New Test Plan Not Yet Complete, Not Fully Tied to Resources, and Not
Synchronized with Other Management Baselines:
MDA's new July 2009 test plan was intended to provide stability;
however, program officials already anticipate major revisions and
alterations. According to MDA officials, budget decisions and the
presidential decision to implement a European phased, adaptive
approach, drove changes to the test and targets program. For example,
the new strategy for European missile defense will primarily utilize
Aegis BMD interceptors as opposed to GMD interceptors. Tests in
support of developing this capability have not yet been added to the
test plan. The Director of MDA stated that his agency is coordinating
with the Office of the Director, Operational Test and Evaluation and
with the BMDS Operational Test Agency to address these changes.
According to the Director of MDA, flight and ground testing to support
phases one through four of the Phased Adaptive Approach will be
baselined in the March 2010 Integrated Master Test Plan, but the test
plan was not available for our review during our audit.
One way MDA's new testing approach was intended to provide stability
is that it was structured to slow the spiral development fielding
process, allowing the warfighter to gain confidence in the BMDS before
fielding decisions are made. However, BMDS Operational Test Agency
officials told us that changes to hardware and software configurations
need to follow the process jointly agreed to with MDA, noting that
changes to the operational baseline should not occur until the
appropriate developmental tests and operational tests have been
completed. After the adoption of the new test plan through October
2009, MDA continued to incorporate software changes as updates to the
operational baseline. According to Operational Test Agency officials,
most of the proposed and approved software changes had not been
through system-level testing and immediately made future test
configurations in the Integrated Master Test Plan invalid. Changes
made without full system-level testing, could result in possible
adverse effects to the BMDS and the warfighter's ability to use the
system effectively. The BMDS Operational test Agency continues to work
with MDA on these issues. BMDS Operational Test Officials told us that
they have seen improvements since October 2009, noting that there has
been an increase in early coordination and presentation of data to
support interim releases of software and hardware. According to these
officials, these improvements coupled with the new warfighter and MDA-
accepted approach for testing--allowing developmental testing to occur
before operational testing and before new capabilities are delivered
to the Warfighter--will likely resolve issues encountered with
frequent changes to software and hardware.
We testified in February 2009 that the success of MDA's new approach
to testing hinges on providing sufficient resources, among other
factors.[Footnote 27] However, these resource challenges continue to
affect the test plan because MDA's new test plan was not fully
resourced when it was approved in July 2009. In addition, BMDS
Operational Test Agency officials also raised concerns that the
Integrated Master Test Plan is not currently resourced to support the
necessary personnel to analyze the tests or the performance assessment.
Until the new development efforts are fully reflected in the test
plan, MDA will also not be able to fully integrate that plan with
other key aspects of testing and development, such as the acquisition
of targets. The test plan is one of six management baselines MDA uses
to track program progress. However, MDA determined that these
baselines consist of a disparate set of non-integrated business
processes.[Footnote 28] More importantly, MDA acknowledged that there
is inconsistent management, configuration control, integration, and
synchronization of existing manual processes. MDA is developing new
business tools to automate the integration of these baselines and
projects. While it will take several years for the agency to integrate
these baselines using those tools and synchronize them with other key
testing and development efforts, the initial capability to
automatically integrate cost, schedule, and performance baselines will
be available in early fiscal year 2011.
Overall BMDS Performance Cannot Be Assessed Because MDA Models Are Not
Sufficiently Mature:
MDA models and simulations have not matured sufficiently to assess
overall BMDS performance and may not fully mature until 2016, instead
of 2011as we reported last year. According to the BMDS Operational
Test Agency, it could not project which models and simulations could
be accredited for Performance Assessment 2009.[Footnote 29] It expects
to make its determination in July 2010 at the earliest. Further,
functionality shortfalls diminished the usable scope and integration
issues have delayed the execution of Performance Assessment 2009 by at
least 6 months. As a result, the BMDS Operational Test Agency did not
use the Performance Assessment 2009 data in its 2009 annual
operational assessment as it had once intended. According to these
officials, because of the known limitations and the changes to the
BMDS operational configuration that will occur in 2010, the BMDS
Operational Test Agency also will not be able to use the results as
part of its 2010 annual operational assessment.
MDA officials acknowledged that their primary challenge for the next
several years will be obtaining enough flight test data to anchor and
accredit the models. Moreover, the BMDS Operational Test Agency is
still concerned about the effect on the validation of models due to
artificialities in flight tests, particularly for GMD.[Footnote 30]
The BMDS Operational Test Agency believes that the validation of
models will improve as artificialities in flight tests are reduced.
Another unresolved modeling and simulation weakness in the testing
program has been addressing different weather conditions.[Footnote 31]
MDA, in concert with the BMDS Operational Test Agency, is addressing
modeling deficiencies with respect to weather conditions, but specific
plans to resolve this weakness were not available during our audit.
Finally, the BMDS Operational Test Agency anticipates that
deficiencies in modeling the BMDS communications system at the
regional and theater levels that exist in Performance Assessment 2009
will improve in the subsequent Performance Assessment 2010.
Progress in Delivering Integrated Capabilities Could Not Be Assessed:
In 2008, we assessed MDA's capability delivery progress against its
integrated capability schedule goals and found that many slipped to
2009.[Footnote 32] We are no longer able to assess MDA's progress in
delivering integrated capabilities because, in fiscal year 2009, the
agency eliminated integrated capability delivery goals from its
schedule baseline. In its most recent BMDS Accountability Report, MDA
redefined its schedule baseline to consist solely of hardware and
software deliveries spread across fiscal years. MDA assigned schedule
metrics to asset deliveries on an element level only and removed key
schedule measures--engagement sequence groups--from its August 2009
BMDS Accountability Report that tracked integrated block capability
deliveries and provided a means for assessing the readiness of BMDS
capabilities, integration, and functionality. Thus, MDA provided no
information about its progress and plans to deliver integrated BMDS
capabilities.
MDA previously identified its capability delivery schedule goals and
baselines within the block structure, in terms of assets and
engagement sequence groups made available for fielding in a particular
timeframe. Under this capabilities-based five-block acquisition
management strategy, some blocks contained schedule baselines for
deliveries of significant increments of capabilities against
particular threats, culminating in the full capability declaration at
a projected date. According to MDA, engagement sequence groups created
manageable combinations of system configurations and provided a
structure to assess BMDS performance. Because MDA presented early,
partial and full capability delivery dates for individual engagement
sequence groups, engagement sequence groups served as baseline to
measure the schedule of integrated capability deliveries.[Footnote 33]
MDA officials told us that the agency eliminated engagement sequence
groups as measures of integrated capability deliveries to address
warfighter concerns. According to MDA officials, the warfighter did
not assess engagement sequence groups since they were organized in a
way that did not align with warfighter operations, tactics, and
procedures. During our audit, MDA had not replaced these previously
reported integrated capability delivery baselines with new metrics.
However, according to the Director of MDA, the agency is working to
develop new baselines and schedules from which progress can be
measured. In addition, agency officials told us that MDA is
transitioning to an incremental BMDS capability delivery concept.
However, MDA did not provide a definition of incremental BMDS
capability deliveries or define them as schedule goals in the August
2009 BMDS Accountability Report. MDA also did not identify anticipated
delivery dates for its performance metrics; however, the Director of
MDA stated that developmental baselines are anticipated to be
developed, reviewed and approved by the third quarter of fiscal year
2010. Furthermore, major MDA documents designed to communicate MDA's
BMDS schedule are not synchronized. Although MDA officials told us
that they have recently synchronized the Integrated Master Schedule
with the Integrated Master Test Plan, the two documents' schedule
still does not correspond to the BMDS Master Plan. The Integrated
Master Test Plan will be revised in February 2010, rendering all three
documents again unsynchronized with MDA's acquisition strategy and
programmatic decisions.
While it has eliminated its externally reported integrated capability
declaration goals, MDA continues to internally track capability
declarations for at least two of its assets--the Sea-based X-band
radar and the Shariki AN/TPY-2 radar--whose capability declarations
slipped again in fiscal year 2009. The Sea-based X-band radar partial
capability declaration appears to have slipped from fiscal year 2009
to fiscal year 2010, while full capability will be declared with less
knowledge than initially planned. According to MDA officials, the
agency was planning for a partial capability declaration in June 2009,
following successful execution of four test events--GTI-03, FTX-03,
FTG-05, and GTD-03--and analysis. However, these events slipped over
the course of the year, and according to MDA, the partial capability
declaration was delayed to fiscal year 2010. According to the Director
of MDA, the capability declaration is currently planned to occur after
analysis can include both FTG-06 and a test--CD-03--planned for
September 2010. It remains unclear what effect the problems
encountered in FTG-06 will have on the declaration decision.
The Shariki radar was designated by MDA to reach a full capability
declaration by December 2008, but that was subsequently delayed to
July 2009. The radar was to undergo the military mission capability
assessment, in which the warfighter verifies the radar's readiness for
full operational use by the services in the context of the present
BMDS architecture. To date, the full capability declaration has not
been made. Consequently, the date for the full mission capability has
not been determined. Furthermore, as with the Sea-based X-band radar,
the decision has not been made as to whether the Shariki radar
capability declaration process will continue under the original plan
or migrate to the new approach.
BMDS Assets Still Being Delivered Despite Developmental Problems and
Test Delays:
Despite testing delays, developmental problems, and the continued
inability to complete the Director's test-related knowledge points,
MDA proceeded with manufacturing, production, and fielding of BMDS
assets prior to operational testing and evaluation.[Footnote 34]
The Aegis BMD program intends to execute a contract modification in
the second quarter of 2010 to acquire 18 operationally configured SM-3
Block IB missiles, used for testing and fielding. These 18 SM-3 Block
IB missiles were originally justified in the fiscal year 2010 budget
request as needed for flight testing and for delivery to the fleet as
operational assets. According to MDA's September 2009 SM-3 Block IB
utilization plan, 2 missiles are to be used for flight tests, 10 are
to be used for fleet deployment, and 6 are to be used for either fleet
proficiency training or fleet deployment. However, MDA is proceeding
with the contract modification even though flight testing of a fully
integrated prototype for this missile type in an operational
environment will not have occurred. The first flight test--FTM-16--
that could demonstrate some performance of the missile is currently
scheduled for the third quarter of fiscal year 2011. In addition, the
program is still maturing several critical technologies, such as the
throttleable divert and attitude control system, and developmental
testing of these technologies will not be complete until after the
manufacturing decision for these 18 missiles. The manufacturing
decision is also scheduled to occur almost a year before the
manufacturing readiness review--currently scheduled for the second
quarter of fiscal year 2011. Consequently, approval for production of
this missile is scheduled before the results of developmental testing
to demonstrate that the technologies and design are fully mature,
before the first flight test demonstrates the system functions as
intended, and before the readiness to begin manufacturing has been
assessed--all of which increases the risk of costly design changes
while production is underway. The Director of MDA and the Assistant
Secretary of the Navy for Research, Development and Acquisition
approved a developmental baseline in January 2010 that set production
criteria and projected an initial production decision for 74 SM-3
Block IB missiles in the third quarter of fiscal year 2011.
GMD continues to manufacture and field the Capability Enhancement II
exoatmospheric kill vehicle prior to having it verified through
operationally realistic flight testing. In March 2009, we reported
that MDA had planned to conduct an intercept test to assess Capability
Enhancement II exoatmospheric kill vehicle in the first quarter of
fiscal year 2008--months before emplacing any interceptors with this
configuration.[Footnote 35] However, developmental problems with the
new configuration's inertial measurement unit and problems with the
target delayed the first flight test with the Capability Enhancement
II configuration--FTG-06--until the fourth quarter of fiscal year
2009. This test was again delayed because of modeling uncertainties
with the target and failures experienced with the Sea-Based X-Band
radar during testing. GMD officials stated that they do not plan to
adjust deliveries of the Capability Enhancement II exoatmospheric kill
vehicle because of the test delay. However, MDA officials told us that
they will not add Capability Enhancement II to the operational
baseline until after FTG-06 has been conducted. As previously noted,
FTG-06 was conducted in January 2010 but was unsuccessful. According
to the July 2009 revised Integrated Master Test Plan, the next planned
intercept test with a similar configuration as FTG-06--a three-stage
booster and a Capability Enhancement II exoatmospheric kill vehicle--
is not scheduled to take place until at least fourth quarter fiscal
year 2012. If MDA delivers Capability Enhancement II exoatmospheric
kill vehicle units as currently scheduled, it will have delivered all
of the Capability Enhancement II exoatmospheric kill vehicles that are
currently under contract before the test is conducted.
MDA's concurrent approach to developing and fielding assets has led to
concerns about the performance of some fielded assets. In March 2009,
we reported that MDA had initiated a refurbishment program in 2007 to
replace questionable parts and that some improvements had already been
introduced into the manufacturing flow.[Footnote 36] However,
according to program officials, they discovered additional problems
during early refurbishments causing the program to expand its effort.
Additionally, as MDA continues to manufacture ground based
interceptors, it is discovering additional process and design issues,
and the corrective actions are being incorporated into the
refurbishment program. The program has three categories for
refurbishments--minimal, moderate, and extensive--with the cost of
each vehicle going through refurbishment varying from vehicle to
vehicle. MDA originally estimated that the cost for extensive
refurbishment of an individual interceptor could reach as high as $24
million.
MDA Lacks Controls and Mechanisms Needed to Establish Transparency,
Accountability, and Oversight:
MDA continues to face challenges with transparency, accountability,
and oversight controls and mechanisms. In establishing MDA in 2002,
the Secretary of Defense directed the agency to develop the BMDS as a
single program using a capabilities-based, spiral upgrade approach to
quickly deliver a set of integrated defensive capabilities. To
accomplish this mission, MDA was granted exceptional flexibility in
setting requirements and managing the acquisition. This flexibility
allowed MDA to begin delivering an initial defensive capability in
2004, but at the expense of transparency and accountability.
Since our first MDA report in 2004, we have repeatedly found that
MDA's approach for building its cost, schedule, and performance goals
hindered transparency and limited accountability of the BMDS
development effort. Specifically in April 2004, we reported that MDA's
goals did not provide a reliable and complete baseline for
accountability purposes and decision making because these goals varied
year to year, did not include all associated costs, and were based on
assumptions about performance that were not explicitly stated. These
conclusions still hold true for several aspects of the BMDS
acquisition strategy. For example, MDA's goals change continuously,
cost baselines have yet to be established, and some details regarding
performance goals are still not explicitly stated. Since 2004, we have
also made recommendations to develop baselines and report variances to
those baselines to promote a higher level of transparency and
accountability for the agency; to adjust its block strategy to ensure
that it was knowledge-based and aligned with agency goals; and to
strengthen oversight by, for example, having the Missile Defense
Executive Board (MDEB) consider the extent to which MDA could adapt
and adopt aspects of DOD's standard acquisition policies to enhance
oversight.
Members of Congress have also expressed concerns regarding the block
strategy, acquisition management strategy, accountability, and
oversight of MDA. For example, in 2007, the House Appropriations
Committee directed MDA to "develop a system-wide plan to report
according to the spirit of existing acquisition laws to improve
accountability and transparency of its program."[Footnote 37] More
recently, in the National Defense Authorization Act for Fiscal Year
2008, Congress required MDA to establish acquisition cost, schedule,
and performance baselines for each system element that has entered the
equivalent of the systems development and demonstration phase of
acquisition or is being produced or acquired for operational fielding.
[Footnote 38] MDA is not yet fully compliant with this requirement.
However, officials indicated that they are working toward fulfilling
this requirement, but the expected date for full compliance was
unknown at the time of our audit.
While MDA has committed to take actions to address concerns about
accountability and transparency, it has made limited progress in
implementation, as shown in table 6.
Table 6: MDA Commitments to Improve Transparency, Accountability, and
Oversight:
MDA commitment: Implement a capabilities-based block structure [A];
Status: Partially fulfilled.
MDA commitment: Establish schedule, budget, and performance baselines
for certain blocks [B] where a firm commitment could be made to
Congress;
Status: Not fulfilled.
MDA commitment: Report these baselines in its annual publication of
goals;
Status: Not fulfilled.
MDA commitment: Identify significant variances from expected outcomes.
Schedule delays, budget increases, and performance shortfalls will be
explained as variances;
Status: Partially fulfilled.
MDA commitment: Obtain independent cost estimates before establishing
unit costs or cost baselines for blocks;
Status: Not fulfilled.
MDA commitment: Present MDA's plans, progress, and problems in
bimonthly quarterly execution reviews beginning in 2008 [C];
Status: Not fulfilled.
MDA commitment: Establish cost baselines at block level [A];
Status: Not fulfilled.
MDA commitment: Establish unit cost estimates for BMDS assets being
acquired and delivered to the warfighter;
Status: Partially fulfilled.
Source: GAO analysis and presentation of MDA data.
[A] According to MDA, on December 7, 2007, in response to
recommendations from GAO, MDA's Director instituted a new capabilities-
based block approach to better communicate its plans and goals to
Congress. However, in June 2009, while MDA was still transitioning to
the capabilities-based block approach, it was terminated by the
Director.
[B] MDA provided some budget, schedule, and performance baselines for
some blocks in its January 2008 Statement of Goals. However, in 2009,
it eliminated the block structure and its ability to maintain these
baselines and measure progress against them. In its 2009 goals, the
agency only provided portions of the previous schedule baselines and
did not include any schedule metrics for its performance baseline as
it had in its 2008 goals. Additionally, the agency did not report any
budget baselines or data in its 2009 goals.
[C] Quarterly execution reviews were terminated in June 2008, with
only one meeting held after MDA's commitment.
[End of table]
Block Structure:
MDA's termination of its capabilities-based block approach in June
2009 marked the third acquisition management strategy for the BMDS in
the last 3 years and effectively reduced transparency and
accountability for the agency. As previously noted, MDA has organized
the development of the BMDS using two different block approaches in
the past--(1) sequential 2-year blocks of BMDS-wide integrated
capabilities and (2) five capabilities-based blocks of different MDA
elements against particular threats. Changing the block structure is
problematic because each time the block structure is changed, the
connection is obscured between the old block structure's scope and
resources and the new block structure's rearranged scope and
resources. This makes it difficult for decision makers to hold MDA
accountable for expected outcomes and clouds transparency of the
agency's efforts.
In March 2008, we reported that the agency's capabilities-based block
approach had begun to provide improvements to transparency and
accountability, but as we recommended, transparency and accountability
could have been further improved with MDA's development and reporting
of full acquisition cost estimates as well as independent verification
of those costs.[Footnote 39]
Cost:
For the seventh year, we are unable to assess MDA's actual costs
against a baseline for total acquisition costs because the agency did
not fulfill its commitment to baseline such costs. In its response to
recommendations we made in March 2008, the agency committed to develop
cost estimates, to obtain independent verification of those estimates,
and to develop total acquisition cost baselines based on those
estimates by blocks when a firm commitment could be made to Congress.
Cost estimates were to be independently reviewed by DOD's Cost
Analysis Improvement Group.[Footnote 40] However, the Cost Analysis
Improvement Group did not complete its review of any cost estimates
because MDA was unable to provide all of the information and data it
needed. MDA officials stated that they plan to develop capability
increments at the program level, from which MDA will establish cost
estimates at some point in the future. MDA officials also stated that
they do not yet know when those estimates would be established or
reported, although the Director of MDA informed us that his agency has
recently reached agreement with DOD's Cost Analysis Improvement Group
to independently estimate THAAD's cost and that other independent cost
estimates of other BMDS elements will follow. Therefore, the timeframe
for developing and reporting total acquisition cost baselines was also
undetermined at the time of our audit.
For the first time, MDA--in its August 2009 BMDS Accountability
Report--provided unit costs for key Aegis BMD, THAAD, Sensors, and GMD
assets. Normally, unit costs are reported in two ways: (1) program
acquisition unit cost, which is the total cost for the development and
procurement of the acquisition program divided by the total quantity,
or (2) average procurement unit cost, which is the total procurement
funds divided by the fielded quantity. MDA reported the latter for its
BMDS assets noting that the elimination of the capabilities-based
block structure made calculation of program acquisition unit cost more
difficult and time consuming. Reporting the average procurement unit
cost, while providing valuable insight into unit costs, excludes MDA's
historical and ongoing large investment in development of the assets.
Considering this, MDA's current approach to reporting unit costs
provides a limited and incomplete view of those costs. According to
the August 2009 BMDS Accountability Report, MDA plans to present both
average procurement unit cost and program acquisition unit cost in the
2010 BMDS Accountability Report. However, according to MDA officials,
they are in the process of baselining the different elements and they
may not complete their estimates in time for this year's BMDS
Accountability Report.
Reporting Variances:
MDA met its commitment to identify and report significant performance
variances in 2009. The agency describes a performance variance as any
predicted or confirmed shortfall in BMDS performance metrics. In
August 2009, MDA reported that there were no variances in its
performance metrics. However, MDA did not fulfill commitments to
identify and report significant budget variances against established
budget baselines. When MDA reported its annual goals in August 2009,
budget variances were not reported. In addition, MDA did not include
any budget information in its 2009 annual goals. In its prior annual
submission of goals, budget data were reported for each block,
element, and capability associated with the BMDS.
DOD Internal Oversight:
The extent of MDA's efforts to improve DOD internal oversight was also
less than planned in 2009. When it was established in 2002, MDA was
given unprecedented flexibility to defer application of DOD
acquisition policies and therefore given autonomy from the standard
internal DOD management framework. According to MDA's February 2008
report to Congress, the agency planned to enhance senior-level DOD
oversight through both the MDEB which was established in 2007 and
through quarterly reviews with the Under Secretary of Defense for
Acquisition, Technology, and Logistics. MDA noted that these senior-
level reviews were to substitute for the traditional accountability
and oversight mechanisms conducted within DOD's standard requirements,
acquisition, and budget processes. However, after reviewing one MDA
element early in fiscal year 2009, MDEB reviews of individual MDA
programs were put on hold during the remainder of the fiscal year. An
MDEB official told us that the board will resume program reviews once
MDA makes a determination of each BMDS element's acquisition status
and determines their next major acquisition decision points.
The MDEB was also established to provide oversight of significant
issues facing the BMDS. However, the MDEB appeared to be less involved
with certain decisions for the BMDS in 2009. For example, according to
an MDEB official, the MDEB was heavily involved in the agency's plan
to implement a new block approach in 2007. However, the MDEB was not
consulted prior to the Director of MDA's decision to terminate the
capabilities-based block approach. According to MDA, the MDEB was
notified of the termination during a briefing on MDA's proposed fiscal
year 2011 budget--2 months after the decision was made. The MDEB
accepted the MDA budget proposals without reference to blocks. In
addition, MDA did not brief the MDEB on the decision itself either,
but rather notification was made via the restructuring of the budget
proposal around programs rather than blocks.
Beginning in 2008, MDA also committed to present its plans, progress,
and problems to the Under Secretary of Defense for Acquisition,
Technology and Logistics in a quarterly execution review every other
month, alternating with the MDEB meetings. According to MDA's February
2008 Plan to Enhance the Accountability and Transparency of the
Ballistic Missile Defense Program, quarterly execution reviews were
expected to focus on actual results against schedule, budget, and
performance goals and baselines as well as on earned value cost
variances. However, these reviews were terminated in June 2008, with
only one meeting held after the February report to congressional
defense committees. According to MDA officials, the MDA Director and
officials in the Office of the Under Secretary of Defense for
Acquisition, Technology and Logistics decided that MDEB meetings were
sufficient to fulfill the intent of quarterly execution reviews. The
Director of MDA informed us that as the MDA Acquisition Executive, he
is executing performance evaluation reviews and the agency is also
reviewed by the Office of the Under Secretary of Defense for
Acquisition, Technology and Logistics. According to the Director of
MDA, the first performance evaluation review will be of the Aegis BMD
system in the second quarter fiscal year 2010.
New MDA Initiatives Provide Opportunity to Increase Transparency and
Accountability:
Although key controls and mechanisms needed to establish a sound
acquisition process for MDA are still lacking, MDA has initiatives
underway that could improve the transparency, accountability, and
oversight of the acquisition of the BMDS. In June 2009, the MDA
Director testified before the Senate Armed Services Committee that MDA
is responding to the Weapon System Acquisition Reform Act of
2009[Footnote 41] through the establishment of acquisition milestone
decisions.[Footnote 42] These decisions are designed to ensure
appropriate competitive acquisition strategies. He further noted that
as the Acquisition Executive for the initial phases of missile
defense, he is implementing milestone review and baseline reporting
processes that are closely aligned with the principles of DOD's
acquisition policies, commonly referred to as the DOD 5000 series. He
further noted that he recognized the need to incorporate the tenets of
the DOD 5000 series to ensure that programs are affordable, are
justified by the warfighter, and demonstrate acceptable risk through a
milestone review process overseen by the MDEB. He also stated that MDA
intends to separate the management of its technology and development
programs. The Director testified that under his authority, potential
programs that may provide technological or material solutions for MDA
will undergo a Milestone "A" decision to determine if they should
become programs.[Footnote 43] These technology-based programs will be
managed by knowledge points and incubated until maturity, at which
time MDA along with the service acquisition executive will be able to
make a Milestone "B" decision as to whether the program should be
converted to a development program. He explained that the Under
Secretary of Defense for Acquisition, Technology and Logistics will
make Milestone "C" production decisions regarding the programs.
We were only able to obtain limited insight into these initiatives
because the agency only determined how they will be implemented at the
end of our audit and was just beginning to implement them. In regards
to the milestone decisions, the Director of MDA indicated that the
agency is undertaking a baseline phase review process. The agency is
transitioning to managing the six developmental baselines at the
project element level. These baselines will be approved in
developmental baseline reviews and managed through quarterly
performance element reviews. MDA has identified three phases of
development where baselines are approved--technology development,
product development, and initial production phases--which may ensure
that the appropriate level of knowledge is obtained before
acquisitions move from one phase to the next. Approval of the product
development and initial production baselines will be jointly reviewed
by the Director of MDA and the respective service acquisition
executive. In addition, while our draft was being reviewed by MDA, the
Director of MDA provided us with initial information regarding the
definition of these new phases and the process for establishing cost,
schedule, or performance baselines. Based on our initial briefing on
MDA's new process, it may include many of the necessary elements of a
sound business case--such as establishing top-level cost, schedule,
and performance measures that are available internally and externally
for oversight.
Although we were unable to fully evaluate MDA's new initiatives, these
initiatives do offer an opportunity for the agency to increase
transparency and accountability if they are implemented in accordance
with knowledge-based acquisition principles, leading to the
establishment of sound business cases and realistic baselines.
Over the past 10 years, we have conducted extensive research on
successful programs and have found that successful defense programs
ensure that their acquisitions begin with realistic plans and
baselines prior to the start of development. We have previously
reported that the key cause of poor weapon system outcomes, at the
program level, is the consistent lack of disciplined analysis that
would provide an understanding of what it would take to field a weapon
system before system development begins. We have reported that there
is a clear set of prerequisites that must be met by each program's
acquisition strategy to realize successful outcomes.[Footnote 44]
These prerequisites include the following:
* Establishing a clear, knowledge-based, executable business case for
the product. An executable business case is one that provides
demonstrated evidence that (1) the identified needs are real and
necessary and can best be met with the chosen concept and (2) the
chosen concept can be developed and produced within existing
resources--including technologies, funding, time, and management
capacity. Knowledge-based acquisition principles and business cases
combined are necessary to establish realistic cost, schedule and
performance baselines. Without documented realistic baselines there is
no foundation to accurately measure program progress.
* Separating technology development activities from product
development activities. As noted earlier, the Director of MDA plans to
separate technology development and product development for the BMDS.
The process of developing technology culminates in discovery--the
gathering of knowledge--and must, by its nature, allow room for
unexpected results and delays. When immature technologies are brought
onto the critical path of product development programs too early, they
often cause long delays in an environment where large workforces must
be employed; complex tools, plants, and facilities must be operated;
long and expensive supplier networks must be paid; and the product
itself must sometimes be redesigned once the final form of the
technologies is known. Ensuring that only mature technologies are
brought into product development is a key step for successful programs.
* Employing early systems engineering discipline in order to develop
realistic cost and schedule estimates prior to development start.
Early systems engineering provides the knowledge a product developer
needs to identify and resolve performance and resource gaps before
product development begins, either by reducing requirements, deferring
them to the future, or increasing the estimated cost for the weapon
system's development. Requirements that are too risky given the state
of technology and design should not be allowed into this expensive
environment. MDA's Director noted that he has taken steps to enhance
systems engineering by designating a senior executive position to
establish engineering policy, ensure the disciplined practice of
systems engineering fundamentals, and develop the systems engineering
competencies of the missile defense workforce; creating knowledge
centers; and increasing the number of recent engineering school
graduates. While these initiatives hold promise for the future, they
could provide further enhancements if they are used as the foundation
to develop realistic cost and schedule estimates for the BMDS.
These practices could address MDA's past problems of initiating
programs and beginning system development based on limited systems
engineering knowledge. These programs depended on critical
technologies that were immature and not ready for product development
or production. The Director of MDA acknowledged the importance of
changing MDA's acquisition approach to adopt knowledge-based
acquisition processes.
Conclusions:
In order to respond to a presidential directive to deliver a missile
defense capability in a rapid manner, MDA has been given unprecedented
funding and decision-making flexibility. This flexibility has allowed
concurrent development, testing, manufacturing and fielding and
enabled MDA to quickly develop and field the first increment of
capability in 2005. However, while this approach has expedited the
fielding of assets, it also resulted in less transparency and
accountability than is normally present in a major weapon program.
Since the program's inception, MDA's lack of baselines and its
management of the BMDS with high levels of uncertainty about
requirements and program cost estimates effectively set the missile
defense program on a path to an undefined destination at an unknown
cost. Across the agency, these practices left programs with limited
knowledge and few opportunities for crucial management oversight and
decision making concerning the agency's investment and the
warfighter's continuing needs. At the program level, these practices
contributed to quality problems affecting targets acquisitions, which
in turn, hampered MDA's ability to conduct tests as planned.
As MDA transitions to new leadership, a new acquisition strategy, a
new test strategy, and a shift in emphasis toward early intercept
capabilities, the agency has an opportunity to chart a course that
enables transparency and accountability as well as flexibility, and it
appears committed to doing so. Importantly, the Director of MDA has
begun new initiatives in accordance with guiding principles of DOD's
acquisition policies, which already embrace knowledge-based practices
and sound management controls. The Director of MDA intends to apply
these new policies to each element or appropriate portions of the
elements, as is currently done across DOD, in order to provide a
foundation for the Congress and others to assess progress and hold
senior leadership accountable for outcomes.
These initial steps are promising, but it will take time to fully
implement them and once implemented they will need to be sustained and
the tools consistently used in order to establish accountability. If
this is done effectively, with baselines set at a program level, MDA
can respond to strategic changes affecting the overall configuration
of the system without losing basic knowledge about cost, schedule, and
performance. Such actions do not have to result in a slower or more
burdensome acquisition process. In the past, weapon programs often
rushed into systems development before they were ready, in part
because DOD's acquisition process did not require early formal
milestone reviews and programs would rarely be terminated once
underway. Over time, in fact, these changes could help programs
replace risk with knowledge, thereby increasing the chances of
developing weapon systems within cost and schedule targets while
meeting user needs.
Recommendations for Executive Action:
As MDA implements its initiatives to improve transparency,
accountability, and oversight, and begins efforts to manage and
oversee MDA at the element level, we recommend that the Secretary of
Defense direct MDA to take the following eight actions:
* Establish cost, schedule, and performance baselines for the
acquisition of each new class of target when it is approved by the
Director prior to proceeding with acquisition and report those
baselines to Congress.
* Obtain independent Cost Assessment and Program Evaluation cost
estimates in support of these cost baselines.
* Ensure that program acquisition unit costs for BMDS assets are
reported in the BMDS Accountability Report, to provide Congress with
more complete and comprehensive information by including development
costs.
* Update DOD's Plan to Enhance the Accountability and Transparency of
the Ballistic Missile Defense Program to reflect MDA's current
initiatives and include dates for fulfilling each commitment.
* Report top-level test goals for each element, or appropriate
portions thereof, to Congress in the next BMDS Accountability Report.
* Develop and report to Congress in the annual BMDS Accountability
Report a measure for schedule baseline goals that incorporates
delivering integrated capabilities to the warfighter.
* Develop and report to Congress in the annual BMDS Accountability
Report the dates at which performance baselines will be achieved.
* Report to Congress variances against all established baselines.
Several of these actions, such as establishing cost, schedule, and
performance baselines, have been recommended in prior GAO reports or
addressed in legislation. This report, however, restates these
recommendations in the context of changes made to the missile defense
program, for example, the deletion of the block structure and
increased focus on elements.
We further recommend that the Secretary of Defense direct MDA to take
the following two actions:
* Delay the manufacturing decision for SM-3 Block IB missiles intended
for delivery to the fleet as operational assets until after (1) the
critical technologies have completed developmental testing, (2) a
successful first flight test demonstrates that the system functions as
intended, and (3) the successful conclusion of the manufacturing
readiness review.
* Ensure that developmental hardware and software changes are not made
to the operational baseline that disrupt the assessments needed to
understand the capabilities and limitations of new BMDS developments.
Agency Comments and Our Evaluation:
DOD provided written comments on a draft of this report. These
comments are reprinted in appendix I. DOD also provided technical
comments, which were incorporated as appropriate.
DOD fully concurred with 9 of our 10 recommendations, including our
recommendation to establish cost, schedule, and performance baselines
for the acquisition of each new class of target when it is approved by
the MDA Director prior to proceeding with acquisition and report those
baselines to Congress. In response to our recommendation, DOD
commented that MDA has already established and the Director has
approved cost, schedule, and performance baselines for the acquisition
of each new class of target. The department noted that these baselines
are contained in multiple documents and will be brought together in a
Target Program Baseline prior to contract award. However, MDA should
ensure that the Target Program Baseline establishes top-level cost,
schedule, and performance baseline measures similar to approved
program baselines that are established for DOD's major defense
acquisitions and available for internal and external oversight. It is
unclear whether MDA will make its Target Program Baseline available
internally for oversight and report it to Congress as we recommended.
DOD partially concurred with our recommendation that the Secretary of
Defense direct MDA to delay the manufacturing decision for SM-3 Block
IB missiles intended for delivery to the fleet as operational assets
until after (1) the critical technologies have completed developmental
testing, (2) a successful first flight test demonstrates that the
system functions as intended, and (3) the successful conclusion of the
manufacturing readiness review. In response to this recommendation,
DOD stated that manufacturing of SM-3 Block IB missiles to support
testing is under way, but the production decision for SM-3 Block IB
missiles used for fleet operation is planned to occur after criteria
listed in our recommendation have been met. However, during our
review, we found that the 18 SM-3 Block IB missiles in question were
originally justified in the fiscal year 2010 budget request as needed
for "flight testing and for delivery to the fleet as operational
assets." In addition, Aegis BMD Program Office responses related to
this matter indicate that these missiles will be used operationally if
a security situation requires it. Furthermore, according to MDA's
September 2009 SM-3 Block IB utilization plan briefed and approved by
the MDA Acquisition Strategy Board, only 2 of these missiles are
specifically designated for flight tests, while 10 are to be used for
fleet deployment and 6 are to be used for either fleet proficiency
training or fleet deployment. Based on this information, the contract
modification to acquire these 18 SM-3 Block IB missiles will take
place before the critical technologies are fully matured at the
conclusion of FTM-16--the first SM-3 Block IB end-to-end flight test
of a fully integrated, production-representative prototype. Thus, we
maintain that approval for manufacturing of these 18 SM-3 Block IB
missiles--the majority of which will be deployed to the fleet--is
scheduled to occur before the results of developmental testing to
demonstrate that the technologies and design are fully mature, before
the first flight test demonstrates the system functions as intended,
and before the readiness to begin manufacturing has been assessed--all
of which increase the risk of costly design changes and retrofit.
We are sending copies of this report to the Secretary of Defense and
to the Director of MDA. The report also is available at no charge on
the GAO Web site at [hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix III.
Signed by:
Cristina Chaplain:
Director Acquisition and Sourcing Management:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition, Technology And Logistics:
3000 Defense Pentagon:
Washington, DC 20301-3000:
February 16, 2010:
Ms. Christina Chaplain:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Ms. Chaplain:
This is the Department of Defense (DOD) response to the GAO draft
report, GA0-10-311, 'Defense Acquisitions: Missile Defense Transition
Provides Opportunity to Strengthen Acquisition Approach," dated
December 22, 2009 (GAO Code 120826).
The DOD concurs with nine of the draft report's recommendations and
partially concurs with one. The rationale for our position is included
in the enclosure. I submitted separately a list of technical comments
for your consideration. I would appreciate an opportunity to discuss
disposition of those comments.
We appreciate the opportunity to comment on the draft report. My point
of contact for this effort is Mr. David Crim, (703) 697-5385,
david.crim@osd.mil.
Sincerely,
Signed by:
David G. Ahern:
Director:
Portfolio Systems Acquisition:
Enclosure: As stated:
[End of letter]
GAO Draft Report ” Dated December 22, 2009:
GAO Code 120826/GA0-10-311:
"Defense Acquisitions: Missile Defense Transition Provides Opportunity
To Strengthen Acquisition Approach"
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Missile Defense Agency (MDA) to establish cost, schedule,
and performance baselines for the acquisition of each new class of
target when it is approved by the Director prior to proceeding with
acquisition and report those baselines to Congress.
DOD Response: Concur. MDA has already established and the Director has
approved cost, schedule, and performance baselines for the acquisition
of each new class of target. These baselines are contained in multiple
documents and will be brought together in a Target Program Baseline
(TPB) prior to contract award.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the MDA to obtain independent Cost Assessment and Program
Evaluation cost estimates in support of these cost baselines.
DOD Response: Concur. Independent cost estimates will be developed to
support the Under Secretary of Defense (Acquisition, Technology and
Logistics) (USD (AT&L)) initial production decisions.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the MDA ensure that program acquisition unit costs for BMDS
assets are in the Ballistic Missile Defense System (BMDS)
Accountability Report, to provide Congress with more complete and
comprehensive information by including development costs.
DOD Response: Concur. MDA will report Program Acquisition Unit Cost
(PAUC) for appropriate Element and Components in the BMDS
Accountability Report.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the MDA to update DoD's Plan to Enhance the Accountability and
Transparency of the Ballistic Missile Defense Program to reflect MDA's
current initiatives and include dates for fulfilling each commitment.
DOD Response: Concur. MDA will update the Plan to Enhance the
Accountability and Transparency of the Ballistic Missile Defense
Program dated February 4, 2008 (Report to Congress) during FY 2010.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the MDA to report top-level test goals for each element, or
appropriate portions thereof, to Congress in the next BMDS
Accountability Report.
DOD Response: Concur. MDA will describe test events planned for the
fiscal year and the associated description of each event from the
Integrated Master Test Plan in the BMDS Accountability Report.
Recommendation 6: The GAO recommends that the Secretary of Defense
direct the MDA to develop and report to Congress in the annual BMDS
Accountability Report a measure for schedule baseline goals that
incorporates delivering integrated capabilities to the Warfighter.
DOD Response: Concur. MDA will develop and report to Congress in the
annual BMDS Accountability Report a measure for schedule baseline
goals that incorporates delivering integrated capabilities to the
Warfighter. MDA has established integrated capability increment goals
for the Ballistic Missile Defense System. MDA will base the delivery
of integrated capability increments on ground tests, flight tests,
performance assessments, and validated Warfighter needs. The Agency's
plan and schedule for tests and performance assessments are in its
Integrated Master Test Plan and Integrated Master Schedule.
Recommendation 7: The GAO recommends that the Secretary of Defense
direct the MDA to develop and report to Congress in the annual BMDS
Accountability Report the dates at which performance baselines will be
achieved.
DOD Response: Concur. MDA will report the dates at which the
performance stated in the annual BMDS Accountability Report are
scheduled to be achieved, beginning with the Fiscal Year 2010 BMDS
Accountability Report.
Recommendation 8: The GAO recommends that the Secretary of Defense
direct the MDA to report to Congress variances against all established
baselines.
DOD Response: Concur. Variances will be reported, as appropriate,
against all established baselines, in the BMDS Accountability Report.
Recommendation 9: The GAO recommends that the Secretary of Defense
direct the MDA delay manufacturing decision for SM-3 block IB missiles
intended for delivery to the fleet as operational assets until after
(1) the critical technologies have completed developmental testing,
(2) a successful first flight test demonstrates that the system
functions as intended, and (3) the successful conclusion of the
manufacturing readiness review.
DOD Response: Partially concur. The production decision for SM-3 Block
IB missiles used for Fleet operation is planned to occur after the
GAO's criteria listed in the recommendation has been met. Note
however, that manufacturing of SM-3 Block IB missiles to support
testing is underway.
Recommendation 10: The GAO recommends that the Secretary of Defense
direct the MDA ensure that the developmental hardware and software
changes are not made to the operational baseline that disrupt the
assessments needed to understand the capabilities and limitations of
new BMDS developments.
DOD Response: Concur. MDA is implementing a more rigorous fielding
strategy that involves maintaining a BMDS configuration for an
extended period of time to allow for the appropriate level of testing,
performance assessment and training before fielding new capability. At
the same time, DOD remains committed to responding to Warfighter
requirements for contingency fielding when it is deemed necessary. A
lower level of confidence may be associated with such fielding, but
the more rigorous testing, training and assessment activity will
continue in parallel.
[End of section]
Appendix II: Scope and Methodology:
To examine the progress that eight Missile Defense Agency (MDA)
elements have made during fiscal year 2009 toward schedule, testing,
and performance baselines, we developed data collection instruments
that were completed by each element's program office. These
instruments collected detailed information on prime contracts, design
reviews, test schedules and results, element performance, noteworthy
progress, lessons learned, and challenges facing the elements during
the fiscal year. In addition, we reviewed individual element Program
Execution Reviews, test plans and reports, production plans, staffer
day briefings, and other requirements documents. We held interviews
with officials in each element's program office and followed up on the
information we received with MDA's Agency Operations Office; the
Department of Defense's (DOD) Office of the Director, Operational Test
and Evaluation; and MDA's Ballistic Missile Defense System (BMDS)
Operational Test Agency.
To further review individual element and BMDS-level performance
progress during the fiscal year, we met with officials in MDA's
Modeling and Simulation Directorate at the Missile Defense Integration
and Operations Center, individual element program offices, and MDA's
BMDS Operational Test Agency to discuss modeling and simulations plans
and procedures as well as other performance metrics. We also reviewed
DOD and MDA policies, memos, and flight test plans related to modeling
and simulations. In addition, we reviewed various elements'
verification, validation, and accreditation plans, MDA performance
briefings, and validation, verification, and accreditation plans for
MDA's BMDS Performance Assessment 2009.
We assessed MDA's testing and target development progress by reviewing
MDA's Integrated Master Test Plans, Integrated Master Schedule, target
acquisition plan, and target business case analysis. In addition, we
met with officials in the Targets and Countermeasures Program Office
to obtain information on MDA's acquisition management strategy
including plans for cost, schedule, and testing. We also met with
MDA's testing directorate, MDA's BMDS Operational Test Agency, and
DOD's Office of the Director of Test and Evaluation to discuss the
progress, challenges, and lessons learned during fiscal year 2009
testing.
To analyze MDA's changing acquisition approach and the agency's
progress in addressing issues related to transparency, accountability,
and oversight, we interviewed officials from the Office of the Under
Secretary of Defense for Acquisition, Technology and Logistics; MDA's
Agency Operations Directorate; MDA's Advanced Technology Directorate;
and MDA's Office of Quality, Safety, and Mission Assurance
Directorate. We also reviewed various MDA statements and documents
related to MDA's block structure. We reviewed DOD acquisition system
policy and various DOD directives to gain insight into other DOD
systems' accountability and oversight mechanisms. We also analyzed
MDA's acquisition directives and Missile Defense Executive Board
briefings to examine MDA's current level of oversight. In addition, we
reviewed MDA budget estimate submission justifications, Integrated
Master Test Plans, the Ballistic Missile Defense Master Plan, the BMDS
Accountability Report, and prior reports that outlined the agency's
baselines and goals.
Our work was performed both at MDA headquarters in Arlington, Virginia
and at various program offices located in Huntsville, Alabama. In
Arlington we met with officials from the Aegis Ballistic Missile
Defense Program Office; Airborne Laser Program Office; Command,
Control, Battle Management, and Communications (C2BMC) Program Office;
MDA's Agency Operations Office; DOD's Office of the Director,
Operational Test and Evaluation; and the Office of the Under Secretary
of Defense for Acquisition, Technology and Logistics. In Huntsville,
Alabama we interviewed officials from the Ground-based Midcourse
Defense (GMD) Program Office, the Sensors Program Office, the Terminal
High Altitude Area Defense Project Office, the Targets and
Countermeasures Program Office, the Advanced Technology Directorate,
and the Office of the Director for BMDS Tests. We met with officials
from the Missile Defense Integration and Operations Center at
Schriever Air Force Base in Colorado Springs, Colorado, to discuss the
C2BMC and Space Tracking and Surveillance System elements as well as
to receive further information on MDA's models and simulations.
Additionally, we interviewed Raytheon officials in Tucson, Arizona, to
discuss the Kinetic Energy Interceptor, GMD, and Aegis BMD elements'
status.
In December 2007, the conference report accompanying the National
Defense Authorization Act for Fiscal Year 2008 noted the importance of
DOD and MDA providing information to GAO in a timely and responsive
manner to facilitate the review of ballistic missile defense programs.
During the course this audit, we experienced significant delays in
obtaining information from MDA. During the audit, MDA did not always
provide GAO staff with expeditious access to requested documents and
articles of information, which delayed some audit analysis and
contributed to extra staff hours. Of the documents and information we
requested, we received approximately 24 percent within the 10 to15
business day protocols that were agreed upon with MDA.[Footnote 45]
Pre-existing documentation took MDA on average about 28 business days
to provide and many pre-existing documents took 40 business days or
more to be provided to GAO. Notwithstanding these delays, we were able
to obtain the information needed to satisfy our objectives in
accordance with generally accepted government auditing standards.
We conducted this performance audit from April 2009 to February 2010
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Cristina Chaplain (202) 512-4841 or chaplainc@gao.gov:
Acknowledgments:
In addition to the contact named above, David Best, Assistant
Director; LaTonya Miller; Ivy Hübler; Tom Mahalek; Steven Stern;
Meredith Allen Kimmett; Wiktor Niewiadomski; Kenneth E. Patton; Karen
Richey; Robert Swierczek; and Alyssa Weir made key contributions to
this report.
[End of section]
Footnotes:
[1] National Defense Authorization Act for Fiscal Year 2002, Pub. L.
No. 107-107, § 232(g) (2001); Ronald W. Reagan National Defense
Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375, § 233
(2004); National Defense Authorization Act for Fiscal Year 2006, Pub.
L. No. 109-163, § 232 (2006); John Warner National Defense
Authorization Act for Fiscal Year 2007, Pub. L. No. 109-364, § 224
(2006); and National Defense Authorization Act for Fiscal Year 2008,
Pub. L. No. 110-181, § 225 (2008).
[2] We did not assess MDA's progress in fiscal year 2002 as the agency
did not establish goals for that fiscal year. We delivered the
following reports for fiscal years 2003 through 2007: GAO, Missile
Defense: Actions Are Needed to Enhance Testing and Accountability,
[hyperlink, http://www.gao.gov/products/GAO-04-409] (Washington, D.C.:
Apr. 23, 2004); Defense Acquisitions: Status of Ballistic Missile
Defense Program in 2004, [hyperlink,
http://www.gao.gov/products/GAO-05-243] (Washington, D.C.: Mar. 31,
2005); Defense Acquisitions: Missile Defense Agency Fields Initial
Capability but Falls Short of Original Goals, [hyperlink,
http://www.gao.gov/products/GAO-06-327] (Washington, D.C.: Mar. 15,
2006); Defense Acquisitions: Missile Defense Acquisition Strategy
Generates Results but Delivers Less at a Higher Cost, [hyperlink,
http://www.gao.gov/products/GAO-07-387] (Washington, D.C.: Mar. 15,
2007); Defense Acquisitions: Progress Made in Fielding Missile
Defense, but Program Is Short of Meeting Goals, [hyperlink,
http://www.gao.gov/products/GAO-08-448] (Washington, D.C.: Mar. 14,
2008); and Defense Acquisitions: Production and Fielding of Missile
Defense Components Continue with Less Testing and Validation Than
Planned,[hyperlink, http://www.gao.gov/products/GAO-09-338]
(Washington, D.C.: Mar. 13, 2009).
[3] In [hyperlink, http://www.gao.gov/products/GAO-09-338], we
reported on two more elements, the Kinetic Energy Interceptor and the
Multiple Kill Vehicle, which were canceled by the agency during fiscal
year 2009. This report does not contain an assessment of these two
elements. The BMDS also includes a ninth element and tenth element,
the Patriot Advanced Capability-3 and the European Component. The
Patriot Advanced Capability-3 has been transferred to the Army for
production, operation, and sustainment. The European Component is
being developed to defend the homeland, allies, and deployed forces in
Europe. This report does not evaluate the Patriot Advanced Capability-
3 because its initial development is complete and is now being managed
by the Army.
[4] The total research, development, test and evaluation amount used
in the computation does not include the Multiple Kill Vehicle or
Kinetic Energy Interceptor elements' budgeted funds. These elements
requested funding in fiscal year 2009, but were also canceled during
the fiscal year and are not elements we are reviewing in this report.
[5] The term early intercept refers to an engagement early in the
battlespace that could optimize the ability to execute a shoot-look-
shoot tactic, may force less effective deployment of countermeasures,
could minimize the potential impact of debris, and could potentially
reduce the number of interceptors required to defeat a raid of threat
missiles.
[6] The five blocks were as follows: Block 1.0: Defend U.S. from
Limited North Korean Long-Range Threats; Block 2.0: Defend Allies and
Deployed Forces from Short-to Medium-Range Threats in One Region/
Theater; Block 3.0: Expand Defense of the U.S. to Include Limited
Iranian Long-Range Threats; Block 4.0: Defend Allies and Deployed
Forces in Europe from Limited Iranian Long-Range Threats; and Block
5.0: Expand Defense of Allies and Deployed Forces from Short-to
Intermediate-Range Threats in Two Regions/Theaters.
[7] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[8] The BMDS Operational Test Agency conducts independent operational
assessments of BMDS capability to defend the United States, its
deployed forces, friends, and allies against ballistic missiles of all
ranges and in all phases of flight. MDA funds all BMDS Operational
Test Agency activities.
[9] GAO, Defense Acquisitions: Sound Business Case Needed to Implement
Missile Defense Agency's Targets Program, [hyperlink,
http://www.gao.gov/products/GAO-08-1113] (Washington, D.C.: Sept. 26,
2008).
[10] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[11] The new Medium Range Target was successfully launched in October
2009.
[12] The Aegis Readiness Assessment Vehicle-C target was successfully
flown in November 2009.
[13] Situational awareness is defined as the degree to which the
perception of the current environment mirrors reality.
[14] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[15] Ground tests play a vital role in the development of technologies
for missile defense by providing officials detailed information about
hardware and software system functionality.
[16] A key step in the process of anchoring models is post-flight
reconstruction, when data from a past flight test scenario are run in
a simulated environment to check and adjust the accuracy of the models.
[17] An end-to-end simulation represents a complete BMDS engagement--
from enemy missile launch to attempted intercept by BMDS kill vehicle.
[18] The BMDS Operational Test Agency provides an independent
accreditation of MDA models and simulations.
[19] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[20] In May 2007, the Director of MDA established test knowledge
points to provide critical information for making key decisions
regarding the BMDS. According to MDA, these knowledge points were
unique management approaches chosen to manage MDA's critical program
risks and to demonstrate the performance of the BMDS.
[21] MDA assesses risk into five categories of likelihood--(1) remote,
(2) unlikely, (3) possible, (4) likely and (5) highly likely; and five
categories of consequence--(1) minimal, (2) moderate, (3) significant,
(4) extensive, and (5) severe.
[22] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[23] We previously reported in GAO-07-387 that to provide
accountability, major defense acquisition programs are required by
statute to document program goals in an approved baseline description
(10 U.S.C. § 2435) that, as implemented by DOD, has been approved by a
higher-level DOD official prior to the program's initiation. The
acquisition program baseline, derived from the users' best estimates
of cost, schedule, and performance requirements, provides decision
makers with the program's total cost for an increment of work, average
unit costs for assets to be delivered, the date that an initial
operational capability will be fielded, and the weapon's intended
performance parameters. The baseline is considered the program's
initial business case--evidence that the concept of the program can be
developed and produced within existing resources. Once approved, major
acquisition programs are required to measure their program against the
baseline or to obtain approval from a higher-level acquisition
executive before making significant changes. When MDA was established
in 2002, it was given the flexibility to delay entry of BMDS into the
DOD acquisition cycle. Because the baseline requirement is typically
triggered by a program's entry into the engineering and manufacturing
development (formerly the system development and demonstration) phase
of the DOD acquisition cycle during which the weapon system is
designed and then demonstrated in tests, and the BMDS has not formally
entered into the DOD acquisition cycle, this requirement does not yet
apply.
[24] Though MDA is not yet required to establish an acquisition
program baseline because of the acquisition flexibilities it has been
granted, Congress has enacted legislation requiring MDA to establish
some baselines. The Fiscal Year 2005 National Defense Authorization
Act, Pub. L. No. 108-375, § 234(e), required the Director, MDA, to
establish and report annually to Congress a cost, schedule, and
performance baseline for each block configuration being fielded. MDA
has since terminated its block approach. In addition, the National
Defense Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, §
223(g) required that no later than the submittal of the budget for
fiscal year 2009, MDA shall "establish acquisition cost, schedule and
performance baselines" for BMDS elements that have entered the
equivalent of system development and demonstration or are being
produced and acquired for operational fielding.
[25] [hyperlink, http://www.gao.gov/products/GAO-08-1113].
[26] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[27] GAO, Defense Acquisitions: Charting a Course for Improved Missile
Defense Testing, [hyperlink, http://www.gao.gov/products/GAO-09-403T]
(Washington, D.C.: Feb. 25, 2009).
[28] MDA's six baselines are technical, schedule, resource, contract,
test, and operational. These baselines give the MDA Director
management information for the BMDS that he uses to make decisions
that affect multiple weapon system programs over multiple fiscal years.
[29] The BMDS Operational Test Agency provides an independent
accreditation of MDA models and simulations.
[30] The BMDS Operational Test Agency defines artificialities as BMDS
architecture, targets, procedures, and conditions that exist in flight
tests but would not exist in the real world. Flight test
artificialities are introduced for a number of reasons, such as
increased chances of success, range safety, data collection, and asset
availability. According to BMDS Operational Test Agency officials, the
complexity associated with the strategic mission of the GMD system
makes challenges associated with artificialities greater.
[31] Weather conditions include rain, clouds, and snow. Severe sea
states, ice loads, or winds could render tests unsafe to execute.
[32] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[33] MDA used the incremental declaration process to designate BMDS
capability for its blocks. Three capability designations--early,
partial, and full capability declarations--were applied to all BMDS
elements, their hardware and software components, and engagement
sequence groups. Each capability designation in the delivery schedule
represented upgraded capacity to support the overall function of BMDS
in its mission as well as the level of MDA confidence in the system's
performance.
[34] We also recently reported in GAO, Missile Defense: DOD Needs to
More Fully Assess Requirements and Establish Operational Units before
Fielding New Capabilities, GAO-09-856 (Washington, D.C.: Sept. 16,
2009), that DOD fielded missile defense equipment to units before
those units were fully organized, manned, and trained to execute all
of their ballistic missile defense responsibilities.
[35] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[36] [hyperlink, http://www.gao.gov/products/GAO-09-338].
[37] H.R. Rep. No. 110-279, at 382 (2007).
[38] Pub. L. No. 110-181 § 223(g).
[39] [hyperlink, http://www.gao.gov/products/GAO-08-448].
[40] The Weapon Systems Acquisition Reform Act of 2009, Pub. L. No.
111-23, transferred the functions of the Cost Analysis Improvement
Group to the Office of the Director of Cost Assessment and Program
Evaluation.
[41] The Weapon Systems Acquisition Reform Act of 2009, Pub. L. No.
111-23, included provisions aimed at improving the organization and
procedures of the Department of Defense for the acquisition of major
weapon systems and addressed areas such as cost assessment and program
evaluation, developmental test and evaluation and systems engineering,
and performance assessments and root cause analyses for major defense
acquisition programs.
[42] The milestone decision authority, is the designated individual
with overall responsibility for a program. The milestone decision
authority has the authority to approve entry of an acquisition program
into the next phase of the acquisition process and is accountable for
cost, schedule, and performance reporting to higher authority,
including congressional reporting. The milestone decision authority
for MDA is the Director of MDA.
[43] The defense acquisition management framework defines the stages
through which typical programs proceed. As each stage concludes, a
decision must be made by the milestone decision authority to initiate,
continue, advance, adjust, or terminate a project or program work
effort or phase. The review associated with each of these decision
points typically addresses program progress and risk, affordability,
program trade-offs, acquisition strategy updates, and the development
of exit criteria for the next phase or effort. Milestone decision
points are Milestone A, for entry into the technology development
phase; Milestone B, for entry into the engineering and manufacturing
development stage (formerly known as system development and
demonstration phase); and Milestone C, for entry into the production
and deployment phase.
[44] GAO, Defense Acquisitions: Measuring the Value of DOD's Weapon
Programs Requires Starting with Realistic Baselines, [hyperlink,
http://www.gao.gov/products/GAO-09-543T] (Washington, D.C.: Apr. 1,
2009).
[45] These data are current as of December 18, 2009.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
