Missile Defense
Alternate Approaches to Space Tracking and Surveillance System Need to Be Considered Gao ID: GAO-03-597 May 23, 2003The Department of Defense's Missile Defense Agency (MDA) is developing a ballistic missile defense system designed to counter a wide spectrum of ballistic missile threats. A future element of this system is the Space Tracking and Surveillance System (STSS). STSS will eventually be composed of a constellation of satellites that will work together to detect and track missiles throughout all phases of their flight. GAO was asked to analyze MDA's approach to demonstrate capabilities for STSS.
MDA purposely adopted a strategy that would evolve STSS over time rather than trying to make a big leap in its capability, deferring some requirements, and calling for competition in the development of the sensors aboard the satellite. Recent decisions, however, will limit MDA's ability to achieve its original goals as well as the knowledge that could be gained from its satellite demonstrations. Specifically, MDA recently reduced its efforts to sustain competition by eliminating funds set aside to procure an alternative satellite sensor from a competing contractor. It now plans to fund only efforts to design an alternative sensor. If it chooses to pursue STSS as part of the missile defense system, STSS may end up being more expensive in the future because MDA could be locked into a single contractor for the design and production of the large constellation of satellites. MDA decided to delay development and launch of new demonstrators in order to focus on completing development of two legacy satellites. MDA already knows that it would like to pursue different designs and different technologies for its target system given that the legacy satellites do not support a producible design. As a result, delaying work on the next generation of satellites will delay work that could offer a better basis from which MDA could build an operational capability. MDA's decision to launch in 2007 lacks important knowledge. MDA has established a launch date before it has completed its assessment of the working condition of the equipment it needs to assemble in order to finish building the two satellites it would like to launch. As a result, it does not know the extent of work that must be done or how much it will cost because the number components found to be in working or non-working order have not yet been identified. MDA has considered pursuing alternate approaches, but all are constrained by the need to participate in 2006-2007 missile defense tests. These approaches include (1) launching the legacy satellites in 2008 instead of 2007 and (2) stopping work on the legacy satellites and focusing instead on developing new demonstrators. Both of these approaches would enable MDA to inject more competition into the STSS program, reduce scheduling risks, and demonstrate more capabilities. However, they also have drawbacks; primarily, they would delay MDA's ability to make informed trade-offs on missile defense sensors.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-597, Missile Defense: Alternate Approaches to Space Tracking and Surveillance System Need to Be Considered
This is the accessible text file for GAO report number GAO-03-597
entitled 'Missile Defense: Alternate Approaches to Space Tracking and
Surveillance System Need to Be Considered' which was released on May
23, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on Strategic Forces, Committee on Armed
Services, U.S. Senate:
United States General Accounting Office:
GAO:
May 2003:
Missile Defense:
Alternate Approaches to Space Tracking and Surveillance System Need to
Be Considered:
GAO-03-597:
GAO Highlights:
Highlights of GAO-03-597, a report to Subcommittee on Strategic
Forces, Senate Committee on Armed Services
Why GAO Did This Study:
The Department of Defense‘s Missile Defense Agency (MDA) is developing
a ballistic missile defense system designed to counter a wide spectrum
of ballistic missile threats. A future element of this system is the
Space Tracking and Surveillance System (STSS). STSS will eventually
be composed of a constellation of satellites that will work together
to detect and track missiles throughout all phases of their flight.
GAO was asked to analyze MDA‘s approach to demonstrate capabilities
for STSS.
What GAO Found:
MDA purposely adopted a strategy that would evolve STSS over time
rather than trying to make a big leap in its capability, deferring
some requirements, and calling for competition in the development of
the sensors aboard the satellite. Recent decisions, however, will
limit MDA‘s ability to achieve its original goals as well as the
knowledge that could be gained from its satellite demonstrations.
Specifically:
* MDA recently reduced its efforts to sustain competition by
eliminating funds set aside to procure an alternative satellite sensor
from a competing contractor. It now plans to fund only efforts to
design an alternative sensor. If it chooses to pursue STSS as part of
the missile defense system, STSS may end up being more expensive in
the future because MDA could be locked into a single contractor for
the design and production of the large constellation of satellites.
* MDA decided to delay development and launch of new demonstrators in
order to focus on completing development of two legacy satellites.
MDA already knows that it would like to pursue different designs and
different technologies for its target system given that the legacy
satellites do not support a producible design. As a result, delaying
work on the next generation of satellites will delay work that could
offer a better basis from which MDA could build an operational
capability.
* MDA‘s decision to launch in 2007 lacks important knowledge. MDA has
established a launch date before it has completed its assessment of
the working condition of the equipment it needs to assemble in order
to finish building the two satellites it would like to launch. As a
result, it does not know the extent of work that must be done or how
much it will cost because the number components found to be in working
or non-working order have not yet been identified.
MDA has considered pursuing alternate approaches, but all are
constrained by the need to participate in 2006-2007 missile defense
tests. These approaches include (1) launching the legacy satellites
in 2008 instead of 2007 and (2) stopping work on the legacy satellites
and focusing instead on developing new demonstrators. Both of these
approaches would enable MDA to inject more competition into the STSS
program, reduce scheduling risks, and demonstrate more capabilities.
However, they also have drawbacks; primarily, they would delay MDA‘s
ability to make informed trade-offs on missile defense sensors.
What GAO Recommends:
To optimize MDA‘s approach to demonstrating space-based missile
tracking capabilities, GAO recommends that MDA focus spending to
assessing what needs to be done to complete work on existing satellite
components so that it has a reasonable basis for its cost and
scheduling estimates. GAO also recommends that MDA assess
alternatives to its current strategy that may offer opportunities to
reduce risks and gain more knowledge. In commenting on a draft of
this report, DOD partially concurred with two of our recommendations
and concurred with two others. In its comments, DOD stated that it
would not be prudent to delay launching satellites given the need to
make overall ballistic missile defense system sensor assessments.
www.gao.gov/cgi-bin/getrpt?GAO-03-597.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact Katherine Schinasi at
(202) 512-4841 or SchinasiK@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Capabilities Remain to Be Proven through Testing in Space:
MDA's Approach to Demonstrate STSS Capabilities:
MDA May Not Be Able to Achieve Original Goals With Its Revised
Strategy:
Alternate Approaches May Garner More Knowledge:
Conclusions:
Recommendations for Executive Actions:
Agency Comments and Our Evaluation:
Scope and Methodology:
Appendix: Comments from the Department of Defense:
Tables:
Table 1: DOD's Attempts to Demonstrate or Field Space-Based Missile
Tracking Capabilities and Their Outcomes:
Table 2: TRL Assessment for Critical Technologies for Legacy
Satellites:
Table 3: Work That Must Be Done on Legacy Satellites:
Table 4: Potential Risks for the Current STSS Strategy:
Table 5: Comparison of MDA's Strategy and Alternatives Not
Being Considered:
Figure:
Figure 1: Notional Configuration of STSS and the Ballistic Missile
Defense System:
Abbreviations:
AEHF: Advanced Extremely High Frequency:
DOD: Department of Defense:
MDA: Missile Defense Agency:
MSX: Midcourse Space Experiment:
SBIRS: Space-Based Infrared System:
STSS: Space Tracking and Surveillance System:
TRL: Technology readiness level:
United States General Accounting Office:
Washington, DC 20548:
May 23, 2003:
The Honorable Wayne Allard
Chairman
The Honorable Bill Nelson
Ranking Minority Member
Strategic Forces Subcommittee
Committee on Armed Services
United States Senate:
The Department of Defense's (DOD) Missile Defense Agency (MDA) is
developing a ballistic missile defense system designed to counter a
wide spectrum of ballistic missile threats. A future element of this
system is the Space Tracking and Surveillance System (STSS), formerly
known as Space-Based Infrared System-low (SBIRS-low). STSS will
eventually be composed of a constellation of an as yet undefined number
of satellites that will work together to detect and track missiles
throughout all phases of their flight--from launch through midcourse
and finally into reentry phase--and pass that information to other
missile defense elements. The satellites will orbit earth at a low
altitude, and they will carry infrared sensors and supporting
subsystems based on sophisticated technologies. DOD currently expects
to spend about $3.1 billion on STSS through 2009.
DOD has initiated several programs and spent several billion dollars
over the past 2 decades trying to develop a system to track missiles
from space, but has not yet demonstrated certain critical capabilities.
While some capabilities have been demonstrated through computer
modeling and simulations, DOD believes all of the capabilities need to
be proven in space before a large number of satellites can be acquired.
Given the challenges associated with the program, you requested that we
determine which capabilities still need to be demonstrated for STSS,
analyze MDA's approach for doing so, and identify alternative
approaches for demonstrating the capabilities that may offer better
outcomes.
Results in Brief:
To be able to track missiles from space, MDA still needs to demonstrate
that:
* tracking information can be passed between sensors within a
satellite;
* tracking information can be passed between satellites;
* missiles can be tracked in the midcourse phase of their flight;
* data from two satellites at different locations and angles can be
successfully integrated, processed, and analyzed;
* data from the satellites can be successfully passed to other space-,
air-, land-, and sea-based platforms;
* satellites can operate and make some decisions autonomously; and:
* satellites can discriminate warheads from decoys.
Achieving these capabilities is technically challenging given the
difficulties associated with tracking cool objects against the cold
background of space as well as the harsh space environment and the
short time frames required to successfully identify, track, and
intercept an incoming warhead. Yet MDA believes most of these
capabilities are needed to have a system that can play a useful role in
the overall missile defense system. Two capabilities--autonomous
operation and discrimination--do not need to be demonstrated as
quickly, but they would significantly enhance a space-based missile
tracking system.
MDA purposely adopted a strategy that would evolve STSS over time
rather than trying to make a big leap in capability, as had been the
strategy in the past. It deferred requirements that were too
technically challenging or beyond its immediate missile defense
mission. MDA also called for competition in the development and
production of the sensors aboard the satellite that would detect a
missile launch (acquisition sensor) and track a missile flight
(tracking sensor) so that costs could be contained in the future and
the best technical solution could be pursued. In addition, MDA opted to
launch "demonstration" satellites before developing and producing them
in large numbers. This strategy helps to reduce risks because it
ensures technology is sufficiently mature and capabilities have been
demonstrated before a greater investment is made.
Recent decisions, however, will limit MDA's ability to achieve its
original goals as well as the knowledge that could be gained from its
satellite demonstrations.
* In order to take part in broader missile defense tests scheduled for
2006 and 2007, MDA decided to retrieve satellites and ground components
that were partially built in a previous effort and put into storage 4
years ago, complete the assembly of this equipment, and launch two
satellites in 2007. Using these satellites in the 2007 test will help
MDA to make trade-off decisions among missile defense systems. To be
able to launch both satellites in 2007, however, MDA eliminated its
plans to have two contractors compete in the production of satellite
acquisition sensors. Instead, the program office now plans to fund the
separate development of an alternative sensor design, but if the funds
available do not allow for a meaningful design effort, it will be
canceled. By choosing this approach, overall program costs could be
higher because MDA could be locked in to using a single contractor for
the production of a larger constellation of satellites.
* In order to complete the development of the legacy satellites for
launch in 2007, MDA also decided to delay the development and launching
of new demonstrators. While MDA could learn a great deal about missile
tracking capabilities from the legacy satellites, MDA already knows
that it would like to pursue different designs and different
technologies for its target system given that the legacy satellites do
not support a producible design. As a result, delaying work on the next
generation of satellites will delay work that could offer a better
basis from which MDA could build operational capability.
* MDA's decision to launch in 2007 was based on limited knowledge. MDA
established a launch date before it had completed its assessment of the
working condition of the equipment it needs to assemble in order to
finish building the two satellites it would like to launch. As a
result, it does not know the extent of work that must be done or how
much it will cost. More specifically, while MDA may know the cost to
test the satellite component hardware, it does not know how many
components will be found in nonworking order, nor the costs to fix
these components. Moreover, MDA has identified a number of activities
that will pose scheduling risks, such as (1) completing development of
software for the ground segment and the infrared sensor software and
(2) integrating the payload hardware and software. Though MDA has set
aside funds to cover the risks, it will not have the knowledge it needs
to really know if it can meet its target date until early 2004--when
its assessment of the working condition of the existing equipment will
be complete.
MDA has considered alternative approaches, but has not pursued any that
would not allow STSS to participate in 2006-2007 testing. Alternative
approaches not considered include (1) launching the legacy satellites
in 2008 instead of 2007, which would allow another year to complete
development of the legacy satellites and procure a sensor of different
design, and (2) stopping work on the legacy satellites and focusing
instead on developing new technology, which would enable MDA to
demonstrate and eventually field an operational capability sooner than
its current approach. Both of these approaches would enable MDA to
inject more competition into the STSS program, reduce scheduling risks,
and demonstrate more capabilities. However, they also have drawbacks;
primarily, they would delay MDA's ability to make informed trade-offs
among missile defense systems.
We are making recommendations in this report that are intended to guide
MDA in selecting the best approach for demonstrating missile tracking
capabilities from space. DOD partially concurred with two of our
recommendations and concurred with two others. In its comments, DOD
stated that it would not be prudent to delay launching satellites given
the need to make overall ballistic missile defense system sensor
assessments.
Background:
DOD is developing a ballistic missile defense system designed, over
time, to counter a wide spectrum of ballistic missile threats. It will
rely on space and ground-based systems to detect and track missiles;
ground-, sea-, and air-based systems to intercept missiles in all
stages of flight (which includes boost, midcourse, and reentry); and an
overarching command and control system to plan and execute actions to
counter enemy attacks.
STSS will serve as the satellite network that will detect and track
missiles throughout their flight and relay necessary cuing data to
other elements in the missile defense system. The satellites will orbit
the earth at low altitudes in order to allow for better missile viewing
angles and high resolution.[Footnote 1] Each satellite will contain two
infrared sensors--one to watch for bright missile plumes during the
boost phase (acquisition sensor) and one to follow the missile through
midcourse and reentry (tracking sensor). To provide for worldwide
coverage, STSS would consist of a large constellation of satellites
(between 21 and 28) as well as a supporting ground infrastructure. MDA
has decided that significantly fewer satellites could be used to
provide a meaningful capability based on the contributions and
configurations of the other elements in MDA's ballistic missile defense
system. However, at this time MDA has not decided on the number of
satellites that it plans to acquire.
Figure 1: Notional Configuration of STSS and the Ballistic Missile
Defense System:
[See PDF for image]
[End of figure]
History of Problems in Developing a Missile Tracking System:
DOD has had considerable difficulty for almost 20 years in developing a
space-based missile tracking capability. Though it has spent several
billion dollars through a series of development and acquisition
programs since 1984, it has not launched a single satellite or
demonstrated any space-based missile tracking capabilities from space
using technologies similar to those to be used by STSS. This is partly
due to the technical challenges associated with building a system like
STSS. For example, the satellites' sensors need to be able to track
missiles in the midcourse phase of their flight, when missiles can no
longer be easily detected by their bright plume. To do this, detection
sensors must be cooled to very low temperatures for very long periods
of time to detect and track a cool warhead against the cold background
of space. In addition, systems aboard the satellite and on the ground
must send that data to other missile defense systems quickly enough to
allow them to target and destroy incoming missiles and they must work
under harsh environmental conditions of space. This requires fast data
processing and communication links as well as materials that can
withstand radiation and cold temperatures.
Within this environment of significant technical challenges, DOD has
not yet established a program that it could execute. As we have
reported[Footnote 2] over the years, DOD and the Air Force did not
relax rigid requirements to more closely match technical capabilities
that were achievable. Program baselines were set based on artificial
time and/or money constraints. Over time, it became apparent that the
lack of knowledge of program challenges had led to overly optimistic
schedules and budgets that were funded at less than what was needed.
Attempts to stay on schedule by approving critical milestones without
meeting program criteria resulted in higher costs and more slips in
technology development efforts. For example, our 1997 and 2001 reviews
of DOD's $1.7 billion SBIRS-low program, STSS' immediate predecessor,
showed that the program would enter the product development phase with
critical technologies that were immature and with optimistic deployment
schedules. In order to reduce costs, schedule, performance, and
technical risks, we recommended that DOD restructure the program and
analyze alternatives to satisfy critical ballistic missile defense
requirements in case SBIRS-low could not be deployed according to the
original acquisition strategy. DOD eventually restructured the SBIRS-
low program because of the cost and scheduling problems, and it put the
equipment it had partially built into storage. Table 1 further
highlights problems affecting space-based missile tracking programs
since 1990.
Table 1: DOD's Attempts to Demonstrate or Field Space-Based Missile
Tracking Capabilities and Their Outcomes:
Program start: 1990; Program title: Brilliant Eyes; Purpose/mission:
Development program. Acquire and track missiles during late boost and
midcourse phases and discriminate warheads from decoys; Problems:
Program funding was sharply reduced for Brilliant Eyes and other space-
based systems. The lack of funding hindered the program from meeting
its objectives; Outcome: No demonstration satellites launched. The
Congress transferred the program from the Ballistic Missile Defense
Organization (now MDA) to the Air Force in 1993.
Program start: 1993; Program title: Space and Missile Tracking System;
Purpose/mission: Development program. Acquire and track missiles and
discriminate warheads from decoys during post-boost phases; Problems:
Technical, funding, and management problems delayed the scheduled
launch of two demonstration satellites; Outcome: No demonstration
satellites were launched. In 1994, DOD consolidated its infrared space
requirements and selected the Space-Based Infrared System as a "system
of systems" approach. Program was terminated.
Program start: 1996; Program title: Space-Based Infrared System-low;
Purpose/mission: Acquisition program. Support national and theater
missile defense by tracking missiles over their entire flights and
discriminating warheads from decoys in supporting the missile defense
mission; Problems: Negative trends in cost, schedule, and performance
estimates for the SBIRS-low program resulted in DOD taking it off an
acquisition track, and returning it to a sustained and deliberate
technology development track; Outcome: Satellite demonstration effort
canceled and the program development risk reduction phase restructured.
Program subsequently transferred from the Air Force to MDA.
Program start: 2002; Program title: Space Tracking and Surveillance
System; Purpose/mission: Development program. Acquire, detect, and
track ballistic missiles through a series of increasingly capable and
interoperable satellites and ground infrastructure; Problems: Not
applicable. Program has just begun; Outcome: Not applicable. Program
has just begun.
Source: GAO.
[End of table]
Congressional Actions:
In October 2000, the Congress directed the Air Force to transfer the
program to the Ballistic Missile Defense Organization (now MDA). The
Senate Committee on Armed Services directed a study of alternatives to
SBIRS-low as part of the fiscal year 2002 budget authorization process.
These alternatives were to include ground-, sea-, and air-based sensors
such as radar systems. MDA is currently expected to complete this study
in 2003. The Committee directed that the report contain (1) an analysis
of essential national missile defense requirements that SBIRS-low would
fulfill and what alternative systems could also fulfill such
requirements; (2) a quantitative assessment of national missile defense
system performance without SBIRS-low or any alternative system; (3) a
quantitative assessment of the national missile defense system
performance with SBIRS-low and with each alternative system; (4) yearly
cost estimates for SBIRS-low and of each alternative system beginning
with fiscal year 2002, including all previous fiscal years and all
fiscal years through deployment of a fully operational system; (5) a
risk assessment of SBIRS-low and of each alternative system; and (6) a
qualitative assessment of the strengths and weaknesses of SBIRS-low and
each alternative system.
In addition, the Congress denied the $385 million DOD requested for the
program for fiscal year 2002, but it provided $250 million for a
satellite sensor technology effort, of which STSS would be a part. MDA
was also directed by the House Appropriations Committee to take STSS
out of the acquisition process and manage it as a sustained and
deliberate technology development effort.
Capabilities Remain to Be Proven through Testing in Space:
DOD believes the following capabilities are needed to have a space-
based missile tracking system that can play a useful role in the
overall missile defense system. These capabilities have not yet been
demonstrated in space, although DOD has had successes in demonstrating
some related on-orbit capabilities through experimental satellites.
* Acquisition-to-track hand over:
The ability of one satellite to detect or "acquire" a missile launch
and to transmit this data to its internal tracking sensor. The tracking
sensor would then continue tracking the missile after the acquisition
sensor has completed its detection function.
* Satellite-to-satellite hand over:
The ability of two or more low-earth orbiting satellites to pass along
missile tracking data through two-way cross-links. This is a
challenging capability to demonstrate given the low orbits and flight
path geometry of the satellites. DOD has no military flight experience
linking two or more low-earth orbiting satellites through two-way
cross-links. The Iridium System, a private network of low orbiting
satellites, can establish cross-links, but it does not have the
timeliness and low bit error rate requirement of STSS. Further, only
voice data (versus analytical data) is transmitted from one fixed user
to another (in comparison with a moving satellite's speed), there are
ground stations to assist in the process, and dropped links are not
mission-critical as they would be for STSS. DOD's Milstar communication
satellites use cross-links, but they operate in a much higher orbit in
fixed positions relative to one another, so this experience also does
not translate directly to STSS.[Footnote 3]
* Midcourse tracking:
The ability to (1) accurately track cool objects from thousands of
kilometers away, which depends on sensitive sensors and accurate
pointing capabilities, and (2) stereo tracking, which requires the
capability to transfer and fuse data from multiple sensors in space
while viewing the target missile from differing ranges and angles.
Midcourse stereo tracking (two satellites reporting tracking
information on one missile's flight) is more desirable because it
results in more precise information on the missile's location. Some
missile tracking capabilities were demonstrated during DOD's 1996
Midcourse Space Experiment (MSX), which launched a satellite that
collected data on a missile launch using optical sensors. However, this
satellite did not conduct the same kind of functions that STSS would be
required to perform, nor did it demonstrate all of the same
technologies.[Footnote 4]
* Dual mission data processing:
The ability to process and analyze data from two satellites that view
one event from two different angles and locations.
* Missile defense system integration:
The ability to transmit and fuse STSS data with data provided by other
space-, air, land-, and sea-based sensors--including legacy and
emerging systems belonging to DOD and U.S. allies--and to use the
results effectively in missile defense operations.
There are two capabilities that DOD believes do not need to be
demonstrated as quickly, but they would significantly enhance a space-
based missile tracking system. They are:
* Autonomous operation:
The ability of each satellite to operate as a self-contained unit and
to perform some decision-making functions before downlinking the
results. Because satellites will be moving at speeds of more than
15,000 mph relative to one another and across different orbital planes,
as well as moving in and out of the target missile's range,
calculations and decisions must be made and data passed between sensors
and satellites within seconds. This is a desired future capability.
* Discrimination:
Countering more advanced and sophisticated threats will require DOD to
be able to detect and track multiple objects and differentiate the
threatening warhead from decoys. Given technical challenges, DOD
deferred plans to achieve this capability for STSS. However, it plans
to achieve this capability for the missile defense system as a whole
before 2015.
MDA's Approach to Demonstrate STSS Capabilities:
MDA could demonstrate space-based missile tracking capabilities by
either continuing earlier efforts or developing new satellites. At the
beginning of the STSS program, MDA chose to combine both, focusing
first on assembling and launching existing satellites and second on
developing new satellites. MDA also sought to avoid the mistakes made
in previous space-based missile tracking efforts by adopting a more
flexible, knowledge-based development strategy and calling for
competition in aspects of satellite development. Recently, MDA decided
to launch the first two demonstration satellites in 2007 and launch the
first next generation satellite in 2011.
Potential Approaches Available to MDA:
MDA could demonstrate space-based missile tracking capabilities by
either relying on legacy satellites or developing new satellites or a
combination of both.[Footnote 5] Specifically, MDA could complete work
on satellite and ground components that were partially built during the
Air Force's effort (SBIRS-low) and put into storage 4 years ago. The
satellites were intended to serve as precursors to a constellation of
operational satellites. The capabilities that were built into the
legacy components include acquisition to track hand over, satellite-to-
satellite hand over, stereo midcourse tracking, and a limited
capability to discriminate the types of missiles launched.
MDA could also develop more capable and more robust satellites based on
newer technology. The satellites could be equipped with more accurate
sensors, faster data processing capacity, and longer lasting
components. The new satellites could also be designed to include
features not available to the existing satellites, such as adding an
autonomous operations capability. As with any approach, a ground
segment capable of supporting future demonstration satellites would be
needed.
At the beginning of the STSS program, MDA decided it would pursue a
combination of both approaches. Specifically, it would complete the
assembly of satellite and ground components already in storage and
launch them to coincide with broader missile defense tests that would
take place in 2006-2007. This would allow MDA to establish a basis for
making trade-off decisions between space-, sea-, and air-based missile
defense sensors (for example, radar systems). MDA also decided to
develop a newer design, including more robust technologies envisioned
for the target system.
MDA Sought to Avoid Past Mistakes with a More Flexible Strategy:
At the onset of STSS, MDA adopted a more flexible product development
approach that would maximize competition. For example:
* As with all missile defense elements, MDA called for a strategy that
would evolve STSS over time, rather than trying to make a big leap in
its capability. This means that new technology would be incorporated
into subsequent increments so that the product's capability would
evolve over time. Our work has shown this approach reduces risk because
it introduces less new content and technology into a program's design
and development effort. An evolutionary strategy also enables
developers to deliver a series of interim capabilities to the customer
more quickly.
* Under its evolutionary approach, MDA deferred requirements that were
too technically challenging or beyond its immediate missile defense
mission. For example, MDA deferred the requirement for a discrimination
capability and has not decided whether the next STSS development block
will perform discrimination. It also deferred requirements for STSS
missions beyond missile defense, including technical intelligence and
battlespace characterization. Instead, these missions would be
addressed only to the extent that inherent or residual capabilities
could satisfy them. Our work has also shown that programs are more
successful when customers are willing to defer requirements that demand
more time or unproven technologies to succeeding versions of the
product. In essence, this flexibility helps to ensure the product can
be developed within available resources.[Footnote 6]
* MDA called for competition in the development and production of the
sensors onboard the satellite that would detect a missile launch
(acquisition sensor) and track a missile flight (tracking sensor) so
that costs could be contained in the future and the best technical
solution could be pursued. Specifically, one satellite would host
sensors from one subcontractor and another satellite would host sensors
from a competing subcontractor. Since contractors may use different
materials to build the infrared sensors, different detector
technologies, and different production methods, performance could vary
considerably. In describing the STSS approach, the Director of MDA
stated that injecting competition into sensor development was necessary
to reduce risks, particularly since MDA planned to award a single
contract to a prime contractor.
* MDA decided to fly "demonstration" satellites before developing and
producing them in larger numbers. This practice enables MDA to see how
components and subsystems work together as a system in a realistic
environment before a greater investment of procurement funds is made.
Our work has also shown this to be a practice used by successful
programs.[Footnote 7]
:
Recent Decisions on STSS Strategy Provide More Time to Learn from First
Two Demonstration Satellites:
After MDA laid the foundation for approaching STSS, it decided to
complete development and testing of two satellites and ground station
equipment it acquired under SBIRS-low; launch the first satellite in
2006 and the second in 2007. Then, beginning in 2003, MDA would pursue
development of new demonstration satellites with more robust technology
and launch them beginning in 2010. It would launch and demonstrate a
series of satellites until it arrived at a design that could be used to
support a bigger constellation of satellites for the missile defense
system. The Air Force signed a contract with Northrop Grumman in August
2002 valued at $868.7 million to (1) design, manufacture, and deliver
the satellites and test and check out the satellites on orbit; (2)
develop a ground system; and (3) conduct preliminary engineering
analyses on the new demonstration satellites.
In late 2002, MDA made significant changes to its strategy after it
decided to allocate less funding to the STSS program in order to fund
other missile defense elements. Specifically, it decided to continue
the STSS program by integrating and testing the existing satellites,
but launch them in tandem in 2007 instead of sequentially in 2006 and
2007. Work on a single new satellite would begin in 2003, instead of a
pair of satellites as had been originally planned. The program office
plans to define the capabilities for the follow-on satellite in mid-
2003 and until then, the design, technologies, and specifications for
the new satellite will not be known. Work on the new satellite will be
stretched out, but MDA hopes to launch the new satellite in 2011, only
1 year later than planned. The STSS program office has programmed about
$1 billion to complete work, launch, and operate the legacy satellites
and $1.3 billion for fiscal years 2004 through 2009 for work on the
new, follow-on satellite effort.
This change has some benefits in that certain capabilities could be
demonstrated over a longer on-orbit period of time. Under the original
strategy, satellite-to-satellite hand over, midcourse tracking, and
dual mission data processing would only have been demonstrated for
about 8 months since this would be the amount of time that the
satellites would be fully operational together. By contrast, under the
new strategy, these same capabilities could be demonstrated for as long
as 2 years since the satellites will be launched in tandem. Moreover,
under the previous strategy, only partial integration with the missile
defense test bed could be demonstrated because data from the 2006
satellite would be processed off line. There could also be delays in
processing data because the ground segment may not be fully integrated
with the missile defense test bed until 2008. (The completion of ground
connectivity between the STSS ground station and the missile defense
system does not yet have a definitive schedule.) Since satellites are
expected to be fully operable for 2 years, integration could be
demonstrated during the latter part of the second satellite's life. But
this would limit the extent to which MDA can assess STSS functions in
the context of the overall system.
MDA is using tools to measure the maturity of critical technologies on
the legacy satellites. Specifically, as the table below shows, MDA has
assessed critical technologies for the legacy satellites using
technology readiness levels (TRL). TRLs measure maturity along a scale
of one to nine. TRL 1 characterizes the least mature technologies
representing the point where scientific research begins to be
translated into technologies basic properties. A TRL 9 represents the
most mature, an actual application of the technologies in its final
form under mission conditions. DOD guidance states that a TRL 7, which
means the system has been demonstrated in an operational environment,
is desired but that a TRL 6 represents acceptable risk for a space-
related technology to enter product development. At a TRL 6, the
subsystem or system has been demonstrated in a relevant environment.
MDA expects critical technologies on the legacy satellites to be at a
TRL 6 by June 2006. Moving from a TRL 5 to a TRL 6 to a TRL 7 represents
a significant investment.
Table 2: TRL Assessment for Critical Technologies for Legacy
Satellites:
Technology area: Acquisition sensor; TRL at fall 2002: 5;
Projected TRL at June 2006: 6.
Technology area: Tracking sensor; TRL at fall 2002: 5;
Projected TRL at June 2006: 6.
Technology area: Single-stage cryocooler; TRL at
fall 2002: 5; Projected TRL at June 2006: 6.
Technology area: Two-stage cryocooler; TRL at
fall 2002: 5; Projected TRL at June 2006: 6.
Technology area: Satellite communication cross-links; TRL at
fall 2002: 6; Projected TRL at June 2006: 6.
Technology area: On-board processor; TRL at
fall 2002: 6; Projected TRL at June 2006: 6.
Source: Air Force.
[End of table]
Table 3 highlights the main activities that must be done to complete
work on the legacy satellites. MDA developed a schedule to support the
original plan to launch in 2006 and 2007. It is in the process of
establishing the dates that these activities would need to be done by
in order to support the new tandem launch date of 2007.
Table 3: Work That Must Be Done on Legacy Satellites:
Area: Systems engineering and assessment; Work: This includes
validating specifications and configurations, establishing performance
baselines, and assessing and integrating ground test data analyses.
Work under this component also includes analyzing the performance of
the satellites after launch.
Area: Sensor development; Work: This includes redesign efforts to the
track sensor, along with assembly, and integration and test. The track
sensor is on the critical path, and all work on this sensor needs to be
completed in time to allow for integration onto the spacecraft. Under
the original strategy, this meant the work should be done by July 2004.
While work progresses on the sensor, software in support of the sensor
will also be developed.
Area: Spacecraft development; Work: Efforts include developing the
spacecraft test bed, harness, and software. Activities also include
integration and test of the satellite before and after launch.
Satellite integration and test is on the critical path and was to begin
in February 2005 and be completed in November 2005 under the original
strategy. At that time the satellite will be shipped for launch.
Area: Ground segment development; Work: Activities include designing
and developing the ground systems, installing hardware in the ground
facilities, and integrating and testing the systems. The ground segment
will involve more software development than the other satellite
segments. Also, operational procedures will be developed and training
on ground systems is to take place.
Area: System test and operations planning; Work: This includes
developing the system and flight test plans. Readiness reviews are to
be complete by September 2005, under the original strategy. Other
activities planned include training and rehearsals, operations crew
test training, test operations, and site and satellite operations.
Source: Air Force Space and Missile Systems Center.
[End of table]
MDA May Not Be Able to Achieve Original Goals With Its Revised
Strategy:
MDA's approach to STSS will limit its ability to achieve its original
goals for the program as well as the knowledge that could be gained
from its satellite demonstrations.
* First, the program office decided to forego pursuing production of
on-board sensors from competing contractors, as originally planned. As
a result, MDA will not have the ability to benefit from competition.
* Second, to stay within its budget, the STSS program office made a
trade-off decision to develop only one new satellite rather than two
and to delay work on the new satellite. This decision will delay MDA's
ability to learn about new satellite designs and technologies needed
for an operational capability.
* Third, MDA's decision to launch in 2007 is not knowledge-based. At
this point, it does not know the extent of work that needs be done on
the legacy satellites since it has not completed its assessment of the
condition of the components that have been in storage for 4 years.
Moreover, it is uncertain as to whether some of the activities it does
know it must undertake in order to integrate and test legacy satellite
systems can be completed in time for the 2007 launch. To its credit,
MDA has set aside extra funds for tasks that present particular
scheduling risk. But until it knows more about the working condition of
the satellite hardware and software, it cannot be sure of its ability
to deliver on time.
Limiting Competition Could Increase Long-term Costs and Risks:
MDA's decision to tentatively fund the design, but not the production,
of a sensor from a competing contractor as part of the first effort
will potentially increase long-term costs and risks. Specifically, it
will preclude MDA from gaining knowledge about competing sensors and
selecting the one that offers the best capability. Moreover, it
precludes another contractor from gaining experience in building
infrared sensors, potentially hampering MDA's ability to compete work
in the future and making the system more costly over the long term.
We recently reported, for example, that DOD's effort to develop a new
generation of communication satellites (the Advanced Extremely High
Frequency (AEHF) satellite program) incurred significant cost and
scheduling problems partly because of its decision to consolidate
contractors into one team. In commenting on our findings, DOD admitted
that its major failing with the program was the acceptance of this
team's proposed approach of an overly optimistic performance, schedule,
and profit baseline. Part of the reason DOD chose to limit competition
was to launch a new satellite as soon as possible. In hindsight, it
recognized that this worsened the situation because the contracting
team could not follow through on its original promises, and DOD had
nowhere else to turn to for a better solution.
Also, under the SBIRS-low program, the Air Force awarded a contract in
1995 for the development of two technology demonstration satellites to
the same contractor. It later recognized the need for competition for
the demonstration to lower costs and reduce schedule and technical
risks because otherwise only one contractor would gain the experience
and knowledge needed to build the full constellation of satellites. To
prevent similar problems, in 1996, the Air Force requested offers for
an alternative system concept from one or more contractors to
demonstrate and validate critical design issues. The goal of this
effort was to stimulate competition, resolve key technical and
production risks, and create a plan to ensure SBIRS-low deployment
schedules can be executed.
Delaying Work on New Design Will Delay Work Toward Operational
Capability:
The decision to launch the two existing satellites in tandem in 2007,
instead of 2006 and 2007, will provide MDA with more time to assess the
working condition of the satellites' components and to complete work on
assembling the satellites. It will also offer more time for MDA to
assess capabilities such as satellite-to-satellite hand over since the
two satellites will be operating as a pair for a longer period of time.
However, MDA already knows that it would like to pursue different
designs and different technologies for its target system as the legacy
system is based on technologies that are more than a decade old. Its
recent decision to delay work on a new satellite will merely delay the
opportunity to learn more about a design that could offer a future
operational capability and a better basis for making trade-off
decisions among missile defense sensors. In fact, for the next few
years, resources will largely be devoted to work on the legacy
satellites versus the new satellites. In 2004, about 92 percent of STSS
funds will go to the legacy satellites versus 8 percent to the new
ones.
Decision to Launch in 2007 Is Not Knowledge-Based:
MDA has not yet completed its assessment of the working condition of
satellite hardware and software, so it does not know the full extent of
work that needs to be done on the legacy satellites. Moreover, MDA
officials recognize that through the process of testing, assembling,
and integrating the hardware and software components, unforeseen
problems could arise that may make it more difficult to complete the
satellites in time for the 2007 launch. These problems could include
completing ground segment software and the infrared sensor software as
well as integrating payload hardware and software. Table 4 highlights
these and other activities MDA believes have costs and scheduling
risks.
MDA and the STSS program office have set aside $47.2 million in funding
to address these potential problems and also drafted risk mitigation
plans, which are to be updated in 2003. This money will be used for
independent review teams that can help the program office assess what
work needs to be done as well as other resources (for example, more
personnel) and activities (for example, contingency planning) needed to
prevent scheduling delays. The program office has also identified the
need for $26 million in additional funding to more fully address the
risks involved with hardware and software issues, which has been funded
from the contractor's management reserves. Program officials said that
if the costs prove to be too high after the assessment of the working
condition of the satellites, they will terminate the legacy effort and
move onto the new demonstrators.
:
Table 4: Potential Risks for the Current STSS Strategy:
Risk: Condition of the satellite hardware and software; Description:
Hardware and software must be tested to determine their working
condition after being in storage for 4 years; Potential effect: If
hardware and software do not test as expected, launch delays could be
significant, particularly for the first launch; Risk mitigation
funding: (millions): $ 9.9.
Risk: Completion of the ground segment software; Description: The
schedule to design, develop, and test the ground software is aggressive
and requires these tasks to be done concurrently. Also, the
requirements for the ground segment software were significantly
increased; Potential effect: If the 31-month schedule does not play
out, software costs could increase, and the ground segment might not
meet the scheduled launches for the existing satellites; Risk
mitigation funding: (millions): $ 5.0.
Risk: Uncertainty of the STSS Block 2006 performance; Description:
Integrated flight test scenarios and targets have not been defined or
analyzed; STSS performance analyses are incomplete; and infrared sensor
tests will not be completed for 2 more years; Potential effect: If
problems surface, costs could increase, schedules could be delayed, and
STSS could provide less on-orbit performance and utility for the
ballistic missile defense system test bed; Risk mitigation funding:
(millions): $14.1.
Risk: Completion of the infrared sensor software; Description: Software
requirements are undefined and software interface issues could require
software redesign; Potential effect: If the issues are not resolved,
costs could increase, and payload testing and delivery could be
delayed, which would delay the launch(es); Risk mitigation funding:
(millions): $14.6.
Risk: Integration of the payload hardware and software; Description:
Many integration and test activities conducted in serial must be
successful. The schedule has little slack for test equipment or
component failures; Potential effect: If this work does not proceed as
planned, costs would increase and delivery of the payloads would be
late, which could delay the launch(es); Risk mitigation funding:
(millions): $ 2.9.
Risk: Thermal modifications to the infrared sensor payload;
Description: Thermal performance and its impact on long-wave infrared
performance will not be known until the sensor has been built and
tested; Potential effect: If lower than expected thermal performance
occurs, costs would increase and satellite-tracking capabilities would
be reduced; Risk mitigation funding: (millions): $ 0.8.
Source: Air Force Space and Missile Systems Center.
[End of table]
Risk 1: Working Condition of Satellite Hardware and Software Has Not
Been Assessed:
MDA will not know the extent of work needed on satellite hardware and
software until late 2004. When the Air Force canceled the flight
demonstration system satellites in 1999, the development of hardware
and software was not completed, and the problem areas that had been
identified had not been fully documented, leaving a knowledge gap that
will need to be closed before MDA proceeds with further development.
The legacy components have been retrieved from storage; however, they
still need to be tested to determine their working condition. In
November 2002, testing started on the first satellite's payload
components (including the acquisition and tracking sensors). As part of
the SBIRS-low effort, the tracking sensor was tested last year and
found to be in working order. This testing should be completed in
October 2003. The satellite's spacecraft hardware has been visually
inspected, and it will be tested from May 2003 to September 2003. The
spacecraft hardware for the second satellite has also been visually
inspected. Under MDA's schedule for its original strategy, testing for
the second satellite was planned for September 2003 through November
2003, and payload testing was planned for December 2003 to August 2004.
Risk 2: Time for Completing Ground Segment Software May Be
Insufficient:
MDA expects that many tasks needed to design, develop, and test the
ground software will need to be done concurrently to meet the new
schedule. The effort will also be complicated by the fact that the
requirements for the ground software significantly changed in 2002, at
the time of the program restructure. Specifically, the software will
need to support the future generation and eventual larger constellation
of satellites, whereas the ground software associated with the legacy
satellites was originally supposed to support two demonstration
satellites. Program officials acknowledged that they would not have a
high level of confidence in the software cost estimate or software
schedule until the preliminary design review for the software occurred,
which occurred in March 2003.
Risk 3: Critical Tests for Assessing Whether Performance Is Acceptable
Will Not Be Done Until Shortly Before Launch Preparations:
Critical tests for assessing preflight performance will not be done and
analyzed until MDA is close to the point where it needs to begin launch
preparations. For example, integrated flight test scenarios and targets
have not yet been defined or analyzed and, as originally scheduled,
infrared sensor performance tests will not be completed until mid-
calendar year 2005--a few months before MDA would need to stop work on
the satellites and begin launch preparations. Program officials pointed
to other factors that will make preparing for performance tests
difficult, including the fact that MDA had not yet identified
interfaces with other missile defense elements or integrated test plans
and schedules.
Our reports have shown that pushing such testing to the latter stages
of a development program is very risky. Specifically, it prevents
programs from using test results to improve design. It also raises the
risk that problems will not be discovered until a point where it
becomes very costly and time-consuming to fix them. Moreover, our
reports have also shown that when testing occurs at latter stages, the
amount of testing that is actually conducted is significantly less than
planned.[Footnote 8]
Risks 4, 5, and 6: Considerable Work Remains to Be Done on Infrared
Sensors and Software:
Considerable work needs to be done on the infrared sensors and software
within a short period of time. Three activities are particularly
critical: (1) completing software development for the acquisition
sensor, (2) integrating payload hardware and software, and (3)
modifying the tracking sensor to accommodate requirements for long-wave
infrared performance.
* Software development for the infrared sensors is on the critical path
and must be completed in time to support integration and testing of the
sensors. Software originally developed for the legacy satellites'
sensors was never completed, and the sensors' software requirements are
not completely defined, which may delay software development. According
to the program office, the lack of time scheduled to perform early
software testing on sensors could result in a delay in detecting and
resolving errors. Most of the $14.6 million risk mitigation funding in
this area has been earmarked to address the lack of early software
testing. Also, software interface issues could require a redesign of
the software.
* Integration of the payload hardware and software will be complex
because many serial integration and test activities must be successful,
and the schedule has little slack for test equipment or component
failures, according to the program office.
* The tracking sensor needs additional modifications to accommodate
long-wave infrared performance requirements. However, the impact of the
modifications will not be known until the sensor has been built and
tested. (The sensor was originally designed for mid-wave infrared
performance. In the middle of the SBIRS-low program, the long-wave
infrared requirements were imposed on the program. While modifications
were made to meet this requirement, the sensor still generates more
heat than the satellite coolers were designed to handle. As such, the
sensor still needs thermal modifications to improve its long-wave
infrared performance.):
Software Development Schedules Tend to Be Optimistic:
Since software development is a risk in many areas of the STSS program,
MDA faces an overriding challenge in accurately predicting what work
will need to be done in developing software related to the program.
Reports show that this is a significant problem for many space and
other weapon system programs--commercial and military. For example, in
a series of studies completed in the 1990s, the Standish Group[Footnote
9] found that the average cost overrun was 189 percent, the average
schedule overrun was 222 percent of the original estimate, and, on
average, only 61 percent of the projects were delivered with originally
specified features or functions attributable to software development.
In November 2000, the Defense Science Board reported that the majority
of problems associated with DOD software development programs are a
result of undisciplined execution. The Board found that troubled
programs lacked well thought-out, disciplined program management and/or
software development processes. Meaningful cost, schedule, and
requirement baselines were also lacking, making it virtually impossible
to track progress against them.[Footnote 10]
We have stated in previous reports that software development schedules
were optimistic for DOD's AEHF satellite system and SBIRS-high program.
For example, the Air Force originally estimated that the AEHF payload
and spacecraft bus required approximately 257,000 lines of software
code, but as the requirements and capabilities of the satellite system
were better understood, the estimate grew to approximately 466,000
lines of software code. AEHF's ground segment also increased from about
1.1 million lines of software code to nearly 1.7 million. In early
2002, during the last SBIRS-high program restructure, Air Force
officials estimated that the amount of ground segment software had
grown 48 percent, while the amount of space segment software had grown
28 percent.
MDA and the prime contractor recognize that software presents a risk
across the board for STSS. The prime contractor has decided to manage
the work on the STSS program based on milestones that are 6 months
earlier than the contractual satellite launch date milestones. This
means that delays attributable to software development up to 6 months
will not directly affect the satellite launch schedule.
Alternate Approaches May Garner More Knowledge:
There are other approaches MDA could pursue, but they have not been
considered because they would not allow STSS to participate in 2006-
2007 missile defense testing. These include (1) planning the launch of
the legacy satellites for 2008 and (2) canceling work on completing the
existing satellites and focusing solely on developing new satellites.
These approaches would enable MDA to inject competition into the STSS
program, reduce scheduling risks, and demonstrate more capabilities.
Both approaches also have drawbacks, primarily, they would delay MDA's
ability to make informed trade-offs between STSS and other competing
surveillance and tracking capabilities, such as ground-, sea-, and air-
based radar systems. Table 5 compares these alternatives in terms of
achieving capabilities to MDA's original and current strategies.
Table 5: Comparison of MDA's Strategy and Alternatives Not Being
Considered:
[See PDF for image]
[End of table]
Delaying Launches of Existing Satellites Could Reduce Scheduling Risk
and Offer More Knowledge:
One approach not being considered involves delaying the launch of the
legacy satellites until 2008. This approach offers several advantages
over the current strategy.
* First, it would reduce program risks by allowing more time to
complete the development and testing of satellite hardware and software
that have been in storage for 4 years, and to complete software
development and testing for the ground segment.
* Second, it would allow time to complete integration of the ground
segment with the missile defense test bed (scheduled for 2008) and
ensure that both satellites would have enough on-orbit life remaining
so that the satellites and ground segment could be tested together
while fully integrated into the ballistic missile defense system test
bed.
* Third, it would allow MDA to fund both the design and production of a
competing contractor's acquisition sensor. This would ensure that
competition remains viable for the development of future series of
satellites, and it is key to MDA getting the best prices and technical
solution.
* Fourth, satellites launched in 2008 will likely still have some
residual capability when the new satellite is launched in 2010,
allowing them to interact together to provide increased knowledge.
One drawback is a potential delay in demonstrating capabilities and
technologies, since MDA's current plan would begin to demonstrate some
capability in 2007. But more importantly, this approach will delay the
benefit of incorporating on-orbit lessons learned into the upgraded
design, because this newer design will be well underway by the time the
satellites are launched. Instead, MDA will have to wait for a future
effort before it can incorporate these lessons learned.
Focusing Solely on Developing New Demonstration Satellites Can Allow
MDA to Develop More Robust Satellites Quicker:
MDA is also not considering focusing solely on developing and
demonstrating new satellites that can offer operational capability once
a limited or full constellation is fielded. However, this approach
could demonstrate most capabilities needed for an operational system
with at least two satellites in orbit at the same time for some
duration. Specifically, satellite-to-satellite hand over could be
demonstrated since the new satellites would be designed to be
compatible. Additionally, the new satellites could be fully integrated
with the missile defense test bed. Midcourse stereo tracking and STSS
mission dual data processing would also be demonstrated. Discrimination
capability could be demonstrated, depending on the design selected.
There are other benefits of pursuing the newer technologies beyond
meeting these capabilities. The newer technology satellites would have
increased lifetimes. In addition, the satellites' sensors would likely
be more sensitive and able to detect cooler targets. Software upgrades
would continue to evolve to meet a newer generation of needs with the
new technology satellites. Moreover, this approach would allow more
time to test with the longer on-orbit life expected from these newer
satellites. Finally, MDA could reach a decision to field an operational
capability sooner than with any other approach.
There are also drawbacks to this approach. There could be at least a 2-
year delay in demonstrating capability. In addition, the technology
risk would be greater because the critical technology for the new
satellites is less mature. For example, the TRLs for the new satellites
currently range from 4 to 5, whereas the TRLs for the existing
satellites, according to the Air Force, range from 5 to 6.[Footnote 11]
Conclusions:
Over about the last 20 years, DOD has invested billions of dollars to
develop a missile tracking capability from space. Past efforts show
that a heavy focus on meeting schedules can debilitate an effort to the
point of failure. Yet DOD is at risk of repeating past mistakes because
it has made decisions that are largely focused on meeting its 2007
launch date rather than making sure the satellites and ground station
can work as intended and that it can gain the maximum knowledge at the
lowest cost. Given the research and development nature of the program
at this point, MDA has the ability to study and consider alternative
ways of moving forward with the existing satellite components with
greater emphasis on gaining knowledge from its demonstration
satellites. If research and development is not the primary goal and
operational capability is, MDA should stop its investment in completing
the existing satellites and concentrate on developing new satellites.
Recommendations for Executive Actions:
To better ensure the Missile Defense Agency's approach to validate
space-based sensors and technologies for missile acquisition, tracking,
and discrimination, we recommend that the Secretary of Defense direct
the Director, MDA to take the following actions.
* Focus spending on its STSS contract to assessing the working
condition of the legacy satellites and what additional work is
necessary to develop, test, and launch the existing satellites so that
MDA has more knowledge on which to build cost and schedule estimates.
* Use this assessment to conduct a broader analysis of alternative
approaches, including the possibility of delaying launches to 2008 as
well as dropping the development of the existing satellite components
and focusing instead on developing demonstration satellites based on
later generation technology.
* Further, use this assessment to find ways to ensure that competition
at the sensor level is part of all efforts to develop missile tracking
capabilities.
* If this assessment concludes that MDA should follow a different path
for STSS, renegotiate the STSS contract to account for this change.
:
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from the
Director of Defense Systems within the Office of the Under Secretary of
Defense for Acquisition, Technology, and Logistics. DOD partially
concurred with our first two recommendations and concurred with our
third and fourth recommendations. In response to the first two
recommendations, DOD noted that efforts to develop, test, and launch
the legacy satellite hardware is well understood and on contract, and
that a delay in launching the first STSS satellites is not prudent,
given the overall missile defense sensor assessments that are to be
made. In response to the third and fourth recommendations, DOD agreed
that the sensor payload competition is central for risk mitigation and
that if DOD pursued a different strategy, contract adjustments would be
warranted. DOD also offered additional corrections and suggestions to
clarify our draft report, which we have incorporated as appropriate.
DOD's comments appear in appendix I.
In responding to our first recommendation, DOD agreed with the need to
assess the working condition of the legacy satellite hardware, but did
not explicitly concur with the need to focus spending on this
assessment. DOD further commented that its efforts to develop, test,
and launch the legacy hardware is well understood. This comment,
however, is based on the assumption that all of the hardware will be
found in working condition and performing within acceptable technical
parameters. Our point is that the condition of the legacy hardware will
not be known until after all of the hardware checks have been
conducted. Because its knowledge of the condition of the legacy
satellites is not complete, MDA's decisions to develop older technology
versus pursuing new technology and to launch legacy satellites in 2007
may not have the expected results. If key satellite components are
found to be in unacceptable working condition, MDA may be forced to
spend more time and money than currently estimated to execute its
strategy.
In responding to our second recommendation, DOD asserted that it had
already conducted a broader analysis of alternative approaches to
development of space-based sensor support to the missile defense
system. As noted in our report, however, this analysis did not include
the alternative of launching legacy satellites in 2008 instead of 2007
or focusing solely on development of new technology. We also disagree
with DOD's comment that a delay in launching the first STSS satellites
is not prudent, given the overall ballistic missile defense system
sensor assessments that are to be made. First, MDA is striving to
launch the STSS satellites in 2007 to support the 2007 test bed in
order to allow DOD to make informed decisions about the composition of
the missile defense sensor architecture. However, launching both legacy
satellites in 2007 also has some long-term affordability consequences.
For example, to be able to fund a launch in 2007, MDA has decided not
to fund the procurement of a satellite sensor from a competing
contractor. Instead, MDA plans to fund the development of an
alternative sensor design from a competing contractor, if the funds
available are sufficient for meaningful design work. By reducing
competition, MDA may well face higher long-term costs to develop STSS
because it may have to rely on a single contractor. Moreover,
competition will enable MDA to pursue the best technical solution for
STSS. Second, a primary goal of the initial STSS satellites is to
demonstrate key capabilities that have never before been demonstrated
from space. By adopting a strategy designed to meet the target launch
date, however, MDA will be constrained in its ability to learn about
these capabilities. For example, it will not be able to fully assess
how well STSS will interact with other missile defense systems because
the legacy systems will only be partially integrated with the missile
defense test bed. Third, the history of the STSS program warrants a
broader assessment of alternative investment approaches. The legacy
satellites that MDA is relying on experienced technical and schedule
difficulties as well as significant cost growth when they were
developed under the SBIRS-low program, STSS's precursor. The SBIRS-low
program as a whole was also schedule driven, it faced technical
challenges, and although almost
$2 billion was spent on this program, not a single satellite was
launched. The demonstration portion of the program was eventually
canceled in 1999. To avoid similar problems, we believe that MDA should
examine approaches that offer ways to maximize competition and reduce
cost and scheduling risks even if that means a delay in its assessment
of STSS's participation in the missile defense test bed.
Scope and Methodology:
To determine what capabilities DOD still needs to demonstrate in
support of a missile tracking capability from space, we reviewed
briefings of program goals, acquisition and test plans, management
reports, and internal memoranda relevant to the development of STSS.
Specifically, we reviewed the system element reviews, MDA's Director
Guidance, and the element capability specification, from MDA and the
Air Force's Space and Missile Systems Center. We also held discussions
on STSS capabilities with officials at MDA, the Space and Missile
Systems Center in Los Angeles, California, and Northrop-Grumman Space
Technology in Redondo Beach, California. We also reviewed documentation
from the Director, Operational Test and Evaluation, Office of the
Secretary of Defense, regarding the SBIRS-low program and its risks.
To identify and assess DOD's prior and current approaches for
demonstrating missile-tracking capabilities, we reviewed the STSS
September and December 2002 System Element Reviews, program briefings,
and the STSS contract. We also held discussions with officials at the
Space and Missile Systems Center. In deriving the different approaches,
we relied on program briefings and supplemented this information with
our own institutional knowledge and experience in reviewing space
systems. Through trial and error for estimation purposes, we extended
launch dates, launched satellites in tandem, or both, to see which
could result in increased knowledge to the program. Through briefings
with officials from the Space and Missile Systems Center and our own
assessments, we determined what the advantages and disadvantages would
be to each approach.
To determine the risk areas involved with the different alternatives
for accomplishing the STSS mission, we reviewed our prior work on the
STSS program and the STSS System Element Review and launch schedules
and discussed with officials at the Missile Defense Agency, the Air
Force Space and Missile Systems Center, and Northrop-Grumman Space
Technology, the work needed to finish development and testing of the
existing demonstration satellites. We reviewed schedule and funding
information for developing both hardware and software for the
demonstration satellites, to include whether components can be fully
integrated, tested, and validated before launch. We also reviewed our
prior reports and testimonies on practices characterizing knowledge-
based acquisition processes.
We performed our work from July 2002 through March 2003 in accordance
with generally accepted government auditing standards.
We plan to provide copies of this report to the Chairmen and Ranking
Minority Members of the Senate Committee on Armed Services; the Senate
Committee on Appropriations, Subcommittee on Defense; the House
Committee on Armed Services; and the House Committee on Appropriations,
Subcommittee on Defense; the Secretary of Defense; and the Director,
Missile Defense Agency. We will make copies available to others upon
request. In addition, the report will be available at no charge on the
GAO Web site at http://www.gao.gov/.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Key contributors to this report were
Cristina Chaplain, Art Gallegos, Tony Beckham, Joseph Dewechter, Dave
Hubbell, Sigrid McGinty, Karen Sloan, Jim Solomon, Hai Tran, and Randy
Zounes.
Katherine V. Schinasi
Director, Acquisition and Sourcing Management:
Signed by Katherine V. Schinasi:
[End of section]
Appendix I: Comments from the Department of Defense:
ACQUISITION, TECHNOLOGY AND LOGISTICS:
Ms. Katherine Schinasi:
Director, Acquisition and Sourcing Management U. S. General Accounting
Office:
441 G. Street, N.W. Washington, DC 20548:
OFFICE OF THE UNDER SECRETARY OF DEFENSE:
3000 DEFENSE PENTAGON WASHINGTON, DC 20301-3000:
1 MAY 2003:
Dear Ms. Schinasi:
This is the Department of Defense's (DoD's) response to the GAO Draft
Report, GAO-03-597, "MISSILE DEFENSE: Alternative Approaches to Space
Tracking and Surveillance System Need to be Considered" dated April 3,
2003 (GAO Code 120156).
The DoD concurs with some recommendations in the report and partially
concurs with the others. Specific comments on each recommendation, and
proposed corrections, are enclosed. My action officer for this effort
is Lt Col Mark Arbogast, (703) 695-7328, mark. arbogast@osd.mil.
We appreciate the opportunity to comment on the draft report.
Sincerely,
Glenn F. Lamartin,
Director Defense Systems:
Signed by Glenn F. Lamartin:
Attachments: As Stated:
GAO DRAFT REPORT - DATED APRIL 3, 2003 GAO CODE 120156/GAO-03-597:
"MISSILE DEFENSE: Alternative Approaches to Space Tracking and
Surveillance System Need to be Considered":
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
direct MDA to focus spending on its STSS contract to assessing the
working condition of the legacy satellites and what additional work is
necessary to develop, test, and launch the existing satellites so that
MDA has more knowledge on which to build cost and schedule estimates.
(p. 23/GAO Draft Report):
DOD RESPONSE: Partially concur.
The assessment described in this report is part of the Department's
ongoing Space Tracking and Surveillance System (STSS) activity. The
efforts required to develop, test, and launch the Flight Demonstration
System hardware are well understood and on contract. The assessments of
the status of the flight 1 and flight 2 legacy satellite hardware are
expected to be complete by September 2003, and February 2004,
respectively.
RECOMMENDATION 2: The GAO recommended that the Secretary of Defense
direct MDA to use this assessment to conduct a broader analysis of
alternative approaches, including the possibility of delaying launches
to 2008 as well as dropping the development of the existing satellite
components and focusing instead on developing demonstration satellites
based on later generation technology. (p. 23/GAO Draft Report):
DOD RESPONSE: Partially concur.
The Department already conducted an initial assessment of the Flight
Demonstration System (FDS) satellites and used this assessment to
conduct the analysis of alternative approaches to development of space-
based sensor support to Ballistic Missile Defense. We believe the
current Block 2006 STSS program will yield critical data which will
expand our understanding of the effectiveness of a space-based infrared
system. Data acquired through the launch and operation of these
satellites will allow the Department to make informed decisions about
the composition of the Ballistic Missile Defense System sensor
architecture. A delay in launching the first STSS satellites is not
prudent. The Department is now conducting a more thorough assessment of
the satellites and preparing them for a 2007 launch as part of the
Block 06 Ballistic Missile Defense System.
RECOMMENDATION 3: The GAO recommended that the Secretary of Defense
direct MDA to use this assessment to find ways to ensure that
competition at the sensor level is part of all efforts to develop
missile-tracking capabilities. (p. 23/GAO Draft Report):
DOD RESPONSE: Concur.
Competition in the area of sensor payloads is central to the
Department's risk mitigation strategies on STSS.
RECOMMENDATION 4: If this assessment concludes that MDA should follow a
different path for STSS, the GAO recommended that the Secretary of
Defense direct MDA to renegotiate the STSS contract for this change.
(p. 23/GAO Draft Report):
DOD RESPONSE: Concur.
If the Department decides that a different strategy is in order for
STSS, appropriate adjustments to the contract will be made.
[End of section]
FOOTNOTES
[1] The satellites will operate about 1,350 kilometers above the earth.
By comparison, satellites in geo-synchronous orbit operate at about
36,000 kilometers.
[2] U.S. General Accounting Office, Defense Acquisitions: Space-Based
Infrared System-low at Risk of Missing Initial Deployment Date,
GAO-01-6 (Washington, D.C.: Feb. 28, 2001) and National Missile
Defense: Risk and Funding Implications for the Space-Based Infrared Low
Component GAO/NSIAD-97-16 (Washington, D.C.: Feb. 25, 1997).
[3] The Milstar satellite communications system provides secure, jam
resistant, worldwide communications to meet essential wartime
requirements for high-priority military users.
[4] The MSX spacecraft had 5 primary sensors with a total of 11 optical
sensors, precisely aligned so that activity of various targets can be
viewed simultaneously with multiple sensors. Four months after its
launch, MSX successfully observed and tracked a 20-minute ballistic
missile test flight. MSX collected more than 800 seconds of high-
quality data on this missile test. MSX tracked missiles by relying on a
sensor that was cooled using a passive technique whereas the sensor on-
board STSS is to rely on an active and mechanical approach.
[5] MDA could supplement both approaches by incorporating knowledge
from the results of tests of other satellites with missile tracking
capabilities, though there are limitations to the knowledge that could
be applied to STSS. These include past tests such as the 1996 MSX test
discussed earlier and upcoming tests such as one MDA will be conducting
in the near future with a satellite build by Spectrum Astro to collect
infrared data on intercontinental ballistic missiles during the boost
phase. Data to be collected under the contract will be used to verify
MDA's future selection of a kill vehicle and tracking sensors for
missile engagements during the boost and ascent phases. The data will
also build the foundation for developing guidance and homing algorithms
for MDA's ground-based, boost-, and ascent-phase interceptors. The
satellite will be designed for an on-orbit lifetime of at least 1 year,
with the objective of a 2-year on-orbit life, and is planned for launch
into low earth orbit in June 2004.
[6] U.S. General Accounting Office, Best Practices: Better Matching of
Needs and Resources Will Lead to Better Weapon System Outcomes
GAO-01-288 (Washington, D.C.: March 8, 2001).
[7] U.S. General Accounting Office, Best Practices: A More Constructive
Test Approach Is Key to Better Weapon System Outcomes, GAO/NSIAD-00-199
(Washington, D.C.: July 31, 2000).
[8] GAO/NSIAD-00-199.
[9] The CHAOS Report, the Standish Group International, Inc. (West
Yarmouth, Mass.: 1995).
[10] Report of the Defense Science Board Task Force on Defense
Software, Office of the Under Secretary of Defense for Acquisition and
Technology (Washington, D.C.: Nov. 2000).
[11] This means that for the new satellites, components are testable
but do not exist in their final assembled configuration. For the
existing satellite hardware, prototypes are available that are very
close in their final form, fit, and function, and performance has been
demonstrated in a relevant environment.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to daily E-mail alert for newly
released products" under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
