Defense Acquisitions
Incentives and Pressures That Drive Problems Affecting Satellite and Related Acquisitions Gao ID: GAO-05-570R June 23, 2005In fiscal year 2006, the Department of Defense (DOD) expects to spend more than $23 billion to develop, acquire, and operate satellites and other space-related systems. These systems are becoming increasingly critical to every facet of military operations as well as the U.S. economy and homeland security. Satellite systems collect information on the capabilities and intentions of potential adversaries. They enable U.S. military forces to be warned of missile attacks and to communicate and navigate while avoiding hostile actions. They provide information that allows forces to precisely attack targets in ways that minimize collateral damage and loss of life. DOD's satellites also enable global communications; television broadcasts; weather forecasting; disaster planning; navigation of ships, planes, trucks, and cars; and synchronization of computers, communications, and electric power grids. DOD's introduction of these desirable capabilities over time has not come without difficulties. Space system acquisitions have experienced problems over the past several decades that have driven up costs by hundreds of millions, even billions of dollars, stretched schedules by years, and increased performance risks. In some cases, capabilities have not been delivered to the warfighter after decades of development. As a result of these problems, DOD is now contending with important trade-off decisions, such as the following. Whether to keep striving to build its Space-Based Infrared System (SBIRS) High as intended or cut back on capabilities. This system is intended to replace and upgrade an older generation of missile-warning satellites, but its cost has already more than doubled and continues to increase, and its schedule has stretched for years. Whether and how much to employ lower orbiting satellites equipped with similar capabilities to facilitate missile defense activities. DOD had spent two decades on this effort without launching a single satellite. Cost and schedule problems forced DOD to rebaseline the program several times. Overall affordability of missile defense has driven DOD to assess whether to continue with this particular effort as well as pursue development of a newer generation of missile-tracking satellites. Whether to limit the acquisition of new communication satellites, known as the Advanced Extremely High Frequency (AEHF) satellites, in favor of developing a newer generation of laser-linked satellites, known as the Transformational Satellite Communications System (TSAT). The AEHF program is running over cost and schedule, but it incorporates more mature technologies. TSAT promises dramatically greater bandwidth and processing capabilities and is considered integral to DOD's efforts to network all of its weapon systems, but there is much less certainty as to how much the system will cost or when it can be delivered because critical technologies are not mature. Whether to pursue incremental increases in capability for the Global Positioning System or embark on a more expensive program that would offer more dramatic capability advances. Two years ago, we issued a report to Congress that analyzed reports we had previously issued on satellite and other space-related programs over the past two decades as well as other studies. Our 2003 report identified common problems affecting those acquisitions. Generally, the problems we identified were common to DOD weapons acquisitions and were recognized within DOD and the space community. In February 2005, Congress requested that we identify underlying incentives and pressures that drive the problems we had identified earlier. Congress also asked that we complete our fieldwork by April 2005 to support the subcommittee's decisions on DOD's appropriations.
The officials we spoke with for this review cited a set of incentives and pressures underlying the space acquisition problems that are largely reflective of a lack of an overall investment strategy and a corresponding tendency to set start dates for programs before a sound business case for them has been established. Specifically, they told us that DOD starts more programs than it can afford and rarely prioritizes them for funding purposes. Such an approach has cascading effects--from creating negative behaviors associated with competing for funds, to increasing technology challenges, to creating unanticipated and disruptive funding shifts, to stretching out schedules in order to accommodate the whole portfolio of space programs. Our previous reports have found these pressures are long-standing and common to weapon acquisitions, not just space systems. In addition, officials we spoke with also cited pressures resulting from having a diverse array of officials and organizations involved with the acquisition process, tensions between the S&T and acquisition communities as to who is better suited to translate technology concepts into reality, pressures resulting from short tenures among staff critical to achieving acquisition success, and difficulties in overseeing contractors.
GAO-05-570R, Defense Acquisitions: Incentives and Pressures That Drive Problems Affecting Satellite and Related Acquisitions
This is the accessible text file for GAO report number GAO-05-570R
entitled 'Defense Acquisitions: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions' which was
released on June 24, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Washington, DC 20548:
June 23, 2005:
The Honorable C.W. Bill Young:
Chairman, Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
Subject: Defense Acquisitions: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions:
Dear Mr. Chairman:
In fiscal year 2006, the Department of Defense (DOD) expects to spend
more than $23 billion to develop, acquire, and operate satellites and
other space-related systems. These systems are becoming increasingly
critical to every facet of military operations as well as the U.S.
economy and homeland security. Satellite systems collect information on
the capabilities and intentions of potential adversaries. They enable
U.S. military forces to be warned of missile attacks and to communicate
and navigate while avoiding hostile actions. They provide information
that allows forces to precisely attack targets in ways that minimize
collateral damage and loss of life. DOD's satellites also enable global
communications; television broadcasts; weather forecasting; disaster
planning; navigation of ships, planes, trucks, and cars; and
synchronization of computers, communications, and electric power grids.
DOD's introduction of these desirable capabilities over time has not
come without difficulties. Space system acquisitions have experienced
problems over the past several decades that have driven up costs by
hundreds of millions, even billions of dollars, stretched schedules by
years, and increased performance risks. In some cases, capabilities
have not been delivered to the warfighter after decades of development.
As a result of these problems, DOD is now contending with important
trade-off decisions, such as the following.
* Whether to keep striving to build its Space-Based Infrared System
(SBIRS) High as intended or cut back on capabilities. This system is
intended to replace and upgrade an older generation of missile-warning
satellites, but its cost has already more than doubled and continues to
increase, and its schedule has stretched for years.
* Whether and how much to employ lower orbiting satellites equipped
with similar capabilities to facilitate missile defense activities. DOD
had spent two decades on this effort without launching a single
satellite. Cost and schedule problems forced DOD to rebaseline the
program several times. Overall affordability of missile defense has
driven DOD to assess whether to continue with this particular effort as
well as pursue development of a newer generation of missile-tracking
satellites.
* Whether to limit the acquisition of new communication satellites,
known as the Advanced Extremely High Frequency (AEHF) satellites, in
favor of developing a newer generation of laser-linked satellites,
known as the Transformational Satellite Communications System (TSAT).
The AEHF program is running over cost and schedule, but it incorporates
more mature technologies. TSAT promises dramatically greater bandwidth
and processing capabilities and is considered integral to DOD's efforts
to network all of its weapon systems, but there is much less certainty
as to how much the system will cost or when it can be delivered because
critical technologies are not mature.
* Whether to pursue incremental increases in capability for the Global
Positioning System or embark on a more expensive program that would
offer more dramatic capability advances.
Two years ago, we issued a report to your subcommittee that analyzed
reports we had previously issued on satellite and other space-related
programs over the past two decades as well as other studies. Our 2003
report identified common problems affecting those
acquisitions.[Footnote 1] Generally, the problems we identified were
common to DOD weapons acquisitions and were recognized within DOD and
the space community. In February 2005, you requested that we identify
underlying incentives and pressures that drive the problems we had
identified earlier. You also asked that we complete our fieldwork by
April 2005 to support the subcommittee's decisions on DOD's
appropriations.
To respond to your request, we analyzed a wide body of GAO, DOD, and
industry studies (see enc. III) that discuss acquisition problems and
underlying incentives and pressures, including our work on best
practices in weapon system development that we have conducted over the
past decade, our individual reviews of space system acquisitions and
crosscutting problems, DOD's independent study of problems affecting
SBIRS High, past DOD studies of crosscutting problems with space system
acquisitions, and a more recent DOD joint task force study on the
acquisition of national security space programs. We also conducted
interviews with more than 40 individuals (see enc. IV)--including
experienced space acquisition program managers and program executive
officials within Air Force Space Command and its Space and Missile
Systems Center, officials responsible for science and technology (S&T)
activities that support space, former and current officials within the
Office of the Secretary of Defense who have specific responsibility for
space oversight or more general weapon system acquisition policy and
oversight, and individuals representing various aspects of industry. We
conducted our review from February 2005 to April 2005 in accordance
with generally accepted government auditing standards.
Results in Brief:
The officials we spoke with for this review cited a set of incentives
and pressures underlying the space acquisition problems that are
largely reflective of a lack of an overall investment strategy and a
corresponding tendency to set start dates for programs before a sound
business case for them has been established. Specifically, they told us
that DOD starts more programs than it can afford and rarely prioritizes
them for funding purposes. Such an approach has cascading effects--from
creating negative behaviors associated with competing for funds, to
increasing technology challenges, to creating unanticipated and
disruptive funding shifts, to stretching out schedules in order to
accommodate the whole portfolio of space programs. Our previous reports
have found these pressures are long-standing and common to weapon
acquisitions, not just space systems. In addition, officials we spoke
with also cited pressures resulting from having a diverse array of
officials and organizations involved with the acquisition process,
tensions between the S&T and acquisition communities as to who is
better suited to translate technology concepts into reality, pressures
resulting from short tenures among staff critical to achieving
acquisition success, and difficulties in overseeing contractors.
We are not making recommendations in this report because it was not
within the scope of our work to determine the actions needed to
redirect the complex set of incentives and pressures affecting space
programs. However, as we point out, our previous reports have already
made recommendations--some of which have been implemented--that we
believe would enable DOD to put space acquisition and other weapons
programs on a sounder footing. In commenting on our report, DOD pointed
out that it has recently taken steps such as improving requirements
setting for all weapons systems and ensuring that decisions to start
space acquisition programs are based on adequate knowledge. Where
appropriate in this report, we also identify and present our views on
solutions being discussed and implemented within DOD.
Background:
The majority of satellite acquisition programs that DOD has pursued
over the past several decades cost more than expected and took longer
to develop and launch than planned. In our 2003 report, we tied these
results to four problems.
1. Requirements for what the satellite needed to do and how well it
must perform were not adequately defined at the beginning of a program
or were changed significantly once the program had begun.
2. Investment practices were weak. For example, potentially more cost-
effective approaches were not examined and cost estimates were
optimistic.
3. Acquisition strategies were poorly executed. For example,
competition was reduced in order to get a program started quickly, or
DOD did not adequately oversee contractors.
4. Technologies were not mature enough to be included in product
development.
We also reported that several factors contributed to these problems.
First, DOD often set dates for delivering capabilities on the basis of
optimism rather than the knowledge that critical technologies would
work as intended by those dates. As a result, activities essential to
understanding and containing costs, maximizing competition among
contractors, and testing technologies were compressed or not done.
Second, a diverse array of organizations with competing interests have
been involved in overall satellite development--from the individual
military services to testing organizations, contractors, civilian
agencies, and, in some cases, international partners. This created
challenges in making tough trade-off decisions. Third, space
acquisition programs have historically attempted to satisfy all
requirements in a single step, regardless of the design challenge or
the immaturity of technologies to achieve the full capability. This
approach made it difficult to match requirements to available resources
(in terms of time, money, and technology). We also reported that other
factors created challenges for the satellite acquisition programs we
reviewed. These include a shrinking industrial base, a declining space
workforce, difficulties associated with testing satellites in a
realistic environment, as well as challenges associated with launching
satellites.
DOD's own reviews have identified similar problems as our review and
expanded on factors that helped drive those problems. Most recently,
DOD conducted a Defense Science Board/Air Force Scientific Advisory
Board Joint Task Force study in 2003 (known as the Young Panel report)
to assess the acquisition of national security space programs and
develop a road map for reform. The Young Panel found that over time,
"cost has replaced mission success" as the primary driver in managing
acquisitions, resulting in excessive technical and schedule risk.
Specifically, the Young Panel reported that program managers face far
less scrutiny on program technical performance than they do on
executing against the cost baseline. The Young Panel said there are a
number of reasons this is so detrimental--the primary ones being that
space is unforgiving, thousands of good decisions can be undone by a
single engineering flaw or workmanship error, and these flaws and
errors can result in catastrophe. The best way to avoid such problems
is an unrelenting emphasis on quality. The Young Panel noted that in
the past, space programs had embraced this approach. Our own reports
have shown that space programs have not done a good job at executing
against their cost baselines. For example, costs for one of DOD's most
important programs, SBIRS High, have more than doubled, and they
continue to grow. Our studies have also found that cost increases
within DOD's space programs are often attributable to the fact that
programs were started without sufficient knowledge as to what resources
would be needed to achieve success. The Young Panel similarly
recognized that the best cost performance is achieved when there is an
emphasis on mission success, which means taking steps to reduce
technical and schedule risk and making investments that enhance
quality.
The Young Panel also found that unrealistic cost estimates had led to
unrealistic budgets and unexecutable programs. Specifically, the panel
found that the space acquisition system is strongly biased to produce
unrealistically low cost estimates throughout the process. During
program formulation, advocacy tends to dominate and a strong motivation
exists to minimize program cost estimates. Moreover, proposals from
competing contractors typically reflected the minimum program content
and a "price to win." Our own studies as well as other DOD studies have
found that unrealistic estimates are common among all weapon systems,
not just space systems, and that low estimates help ensure that the
program will win support over competing programs and be funded.
Like our study, the Young Panel also found that undisciplined
definition of and uncontrolled growth in requirements contributed to
cost growth and schedule delays and that flawed acquisition strategies
did so as well. In particular, the Young Panel, as well as a preceding
review of SBIRS High, found that the adoption of a Total System
Performance Responsibility policy in the 1990s--which lessened the
government program management role in favor of a stronger industry
role--essentially eroded the government's ability to effectively manage
and oversee space programs and placed too much responsibility on
industry to define requirements and make tradeoff decisions. Over time,
this shift as well as other well-intended reforms resulted in declines
in critical capabilities within the government space workforce,
particularly for systems engineering.
DOD has recognized that problems with its space acquisitions need to be
addressed, and it has taken a range of actions, including shifting away
from Total System Performance Responsibility to stronger government
management and oversight of space programs, strengthening cost
estimating capabilities, adding independent oversight reviews to the
decision-making process, and adding discipline to requirements setting.
However, our recent reports and testimonies have recommended that DOD
also focus on ensuring that acquisition programs not begin until
adequate knowledge has been accumulated on critical technologies and
suggested that DOD still needs to guide its overall space portfolio
with an investment strategy that makes high-level trade-offs before
beginning programs. Moreover, our reports on all weapon system
acquisitions have continually pointed out a need to recognize and find
ways to address the underlying incentives and pressures that drive
acquisition problems. Without doing so, the impact of changes in
policies or processes will continue to be limited.
Incentives and Pressures that Drive Space System Acquisition Problems:
The officials we spoke with for this review cited a set of incentives
and pressures underlying the space acquisition problems that are
largely reflective of a lack of an overall investment strategy and
priority setting and a corresponding tendency to set start dates for
programs before a sound business case for them has been established. In
addition, officials we spoke with also cited pressures resulting from
having a diverse set of individuals and organizations involved with the
acquisition process, tensions between the S&T and acquisition
communities as to who is better suited to translate technology concepts
into reality, pressures resulting from short tenures among staff key to
achieving acquisition success, and difficulties in overseeing
contractors. Our own reviews have identified similar incentives and
pressures and found them common among weapon system acquisitions.
Too Many Programs Competing for Funding:
Many of the officials we spoke with identified pressures and incentives
that are rooted in the widely held belief that DOD starts more space
programs than it can afford and rarely prioritizes programs for funding
purposes before or after starting them. Such an approach has cascading
effects--from creating negative behaviors associated with competing for
funds, to increasing technology challenges, to creating unanticipated
and disruptive funding shifts, to stretching out schedules in order to
accommodate the whole portfolio of space programs. Figure 1 highlights
the cycle of pressures when DOD initiates too many programs with too
little funding.
Figure 1: Overview of Pressures Resulting from Beginning More Programs
than DOD Can Afford in the Long Run:
[See PDF for image]
[End of figure]
Specifically, officials told us the following.
* DOD starts more programs than it can afford over the long run,
forcing programs to underestimate costs and overpromise capability.
This was attributed to both the Office of the Secretary of Defense and
the Air Force. The September 11, 2001, terror attacks on the United
States spurred DOD to attempt to pursue even more satellite programs,
believing that there was now a greater need for persistent surveillance
and more robust communication and networking capabilities.
* When faced with a lower budget, senior executives within Office of
the Secretary of Defense and the Air Force would rather make across-
the-board cuts to all space programs than hard decisions as to which
ones to keep and which ones to cancel or cut back.
* Because programs are funded annually and priorities have not been
established, competition for funding continues over time, forcing
programs to view success as the ability to secure the next installment
rather than the end goal of delivering capabilities when and as
promised.
* More often than not, DOD seeks substantial leaps in capability versus
incremental leaps. While this approach helps a program to gain support,
it substantially increases the technical challenge and the level of
unknowns about a program at the time it is started.
* Having to continually "sell" a program also creates incentives to
suppress bad news about the program's status and avoid activities that
uncover bad news.
* Launching demonstrators in space is a good way to reduce risks and
learn about technologies before starting a new acquisition program. But
because of the high cost of testing technologies in space and the
overall competition for funding, programs are incentivized not to
pursue this approach. At the same time, resources outside acquisition
programs devoted to testing in an operational environment are
declining.
* DOD faces resource shortages beyond funding because it starts more
programs than it can afford. Principally, it does not have a sufficient
workforce to support space acquisitions or experienced program managers
to guide them.
Our previous reports have found that these pressures are long-standing
and common to weapon acquisitions, not just space acquisitions. The
competition within DOD to win funding and get approval to start a new
program is intense, creating strong incentives to make a weapon system
stand out from existing or alternative systems. If the system does not
stand out or prevail over alternatives, the program could be
terminated. Moreover, overall DOD funding constraints put a high
priority on affordability, making it important for program sponsors to
provide cost estimates that will fit within the funding constraints.
Instead of forcing trade-offs, challenging performance requirements--
when coupled with other constraints, such as cost or the weight of the
satellite--can drive product developers to pursue exotic solutions and
technologies that, in theory, can do it all.
Moreover, in weapon acquisitions, optimistic cost estimates are
encouraged because they help gain program approval and attract
budgetary resources. The consequences of cost growth are not directly
felt by an individual program because they are "accommodated" through
delivery delays and quantity changes and by spreading the cost impact
across many programs.
We have also reported that the practice of breaching cost and schedule
objectives to meet difficult requirements would not persist without a
customer's cooperation. Unlike commercial customers, DOD customers tend
to be tolerant of cost overruns and delays in order to get a high-
performance weapon system. Traditionally, customers have been willing
to wait long periods of time for a capability. They would rather wait
for the most desirable system to be developed than accept a less
capable system, thinking that they may not get the opportunity to
acquire a new or modified system in the future.
Our recent reports on space and other weapon systems have suggested
that having a departmentwide investment strategy for weapon systems or
even space systems would help reduce these pressures. Critical
components of such a strategy would include identifying overall
capabilities and how to achieve them, that is, what role space will
play versus other air-, sea-, and land-based assets; identifying
priorities for funding; and implementing mechanisms that would enforce
the strategy. DOD has made revisions to its requirements-setting and
budgeting processes to strengthen investment planning. However, it is
unclear as to how these changes will be implemented over time and
whether they can serve as a foundation for directing S&T and
acquisition investments.
To help close knowledge gaps at the onset of programs and shorten
development time, DOD has adopted an evolutionary development approach-
-that is, pursuing incremental increases in capability versus
significant leaps. Our examinations of best practices have found that
this approach can decrease time and cost for development because it
closes gaps in unknowns. Many of the officials we spoke with believe
that evolutionary development could be achieved in space by developing
constellations of larger numbers of smaller, more affordable satellites
instead of constellations of a few, very large and heavy, complex
satellites (commonly referred to as "Battlestar Galacticas" in the
space community). In addition to reducing cost and time associated with
longer, more challenging programs, this approach could help keep the
space industrial base more productive. Complementary alternatives
include developing common rather than unique satellite components,
cheaper and more responsive launch systems, as well as systems that
enable DOD to modify and fix satellites in orbit. DOD is pursuing a
range of S&T efforts along these lines. However, DOD's executive agent
for space recently testified that these approaches are not technically
suitable for some of the capabilities DOD is now pursuing, such as
Space Radar (formerly the Space-Based Radar program) and TSAT. We will
be undertaking a review to further assess the potential that these
approaches offer for producing better outcomes as well as potential
barriers to integrating them into the acquisition process.
Another solution that has been advocated by the Young Panel and many of
the officials we spoke with as a way of addressing gaps between
resources and requirements is management reserves. The Young Panel
recommended using reserves only to execute the approved program
baseline and not for new requirements. The officials we spoke with also
said that management reserves may not be needed as much as they
currently are if programs do a better job of matching resources to
requirements before they begin. In addition, several officials noted
that broader investment strategies should be in place so that DOD can
afford management reserves.
Diverse Array of Officials and Organizations Involved with Space
Systems Add Pressure to Requirements Setting:
The officials we spoke with widely agreed that the diverse array of
officials and organizations involved with a space program make it even
more difficult to pare back and control requirements. As officials we
spoke with pointed out, space systems may suffer from more requirements
pressures because there is usually a very broad constituency behind
each satellite program. The Global Positioning System, for example, not
only serves military users; it also serves civilians, supports various
key economic sectors such as transportation and communications, and is
used by allies. The National Polar-Orbiting Operational Environmental
Satellite System currently under development will serve military
weather forecasters as well as civilian forecasters and a broad
community of scientists studying environmental issues. The Space Radar
system is expected to play a major role in transforming military as
well as intelligence-collecting operations and other critical
governmental functions, such as homeland security. As a result, when
starting these new systems, space program managers can expect to be
inundated with competing demands--not just among military users--but
also among civilian and industry users.
Our prior reports have identified related pressures with all weapon
systems. More than 30 organizations within the requirements community
may have a hand in determining a weapon system's performance
requirements before a contractor with systems engineering expertise can
identify the gaps between the requirements and available resources.
This process means the "doability" of the requirements is often not
known with certainty until well into product development or until a
significant percentage of funds planned to develop the system has been
invested. By this point in time, customers' expectations have been set,
making it difficult to change requirements if gaps between requirements
and available resources are found.
In the past, DOD has not implemented effective mechanisms to help
mitigate these pressures. In fact, as DOD's own studies have shown that
these pressures were exacerbated when DOD pursued its Total System
Performance Responsibility approach because it turned over
responsibilities related to requirements definition to contractors who
had less understanding and ability to negotiate requirements, leaving
program managers in the position of having to continually address
requirements growth without additional resources. Moreover, the Young
Panel observed that space program managers have not had the authority
needed to make trade-offs between requirements and control requirements
growth. The panel recommended giving program managers this authority,
accompanied by greater accountability for requirements.
Developing Technologies within the S&T Environment versus the
Acquisition Environment:
An important problem cited in our reports about space system
acquisition programs is the tendency to take on technology development
that should occur within the S&T environment. Our reports have stressed
that the S&T environment is more forgiving and less costly than the
acquisition environment, which is focused on delivery. This is because
events such as test failures, new discoveries, and time spent in
attaining knowledge are considered normal in the S&T environment rather
than negative. Further, when acquisition programs take on technology
development, estimates for cost, schedule, and performance are formally
approved without the benefit of knowing that technologies will work as
intended.
Officials we spoke with for this review and previous reviews cited a
number of reasons that program managers and senior leaders choose to
have acquisition programs take on technology development activities
that should occur within the S&T environment.
* The lengthy development period required for space systems puts
pressure on program managers to continually develop technologies. There
is a fear that if these technologies do not reach maturity during this
time frame, they will be outdated by the time the satellites are ready
to be launched.
* Once a program has formally begun, it is easier to secure current and
future years' funding.
* Satellites tend to last longer than expected, and they cannot be
retrieved for upgrades, putting more pressure on programs to push for
attaining as much technological capability as possible within the
acquisition program.
* The acquisition community does not believe that labs in charge of
developing space technologies adequately understand its needs--in terms
of capabilities and time frames--and would rather pursue its own goals.
* Program managers also believe that they would have more control over
technology development if it was conducted by contractors who answered
to them rather than to DOD labs.
* DOD has not had an effective strategy for steering activities within
the S&T community to ensure that they will eventually fit in with
acquisition needs. (Note: DOD has recently developed a space S&T
strategy. We reported on this effort in January 2005.)
Our previous reports have found that many of these views tend to work
against, rather than for, DOD's ability to achieve timely technology
advances. When acquisition programs seek to translate advanced concepts
into reality, they invariably run into problems that require time and
money to fix. The effects of these problems are often revealed in a
later stage of development, where they have reverberating effects on
other aspects of the acquisition program and often require reworking
design. For example, early technological problems as well as more
recent system integration issues have severely affected the SBIRS High
program, among others.
We also previously found that DOD's new space acquisition policy
increased acquisition risks by allowing programs to begin without
having technologies demonstrated in an operational or simulated
environment or even begun with technologies in even lower stages of
maturity.
Many officials believed this policy was necessary because of the unique
aspects of space acquisition programs, that is, their long length,
their complexity, and the high cost of operational testing associated
with space systems. As we have reported in the past, however, DOD has
found ways to test sensors and other critical technologies on
experimental satellites in the past, and it has built and launched
technology demonstrator satellites before starting acquisition
programs. Moreover, as noted earlier, the length of space and other
weapon system development can be reduced by pursuing evolutionary
development. This approach does not prevent DOD from concurrently
seeking technological advances, but such activity should occur outside
an acquisition program, rather than inside, to minimize disruptions.
Last, officials within the Office of the Secretary of Defense did not
believe that space programs warranted a separate approach than other
weapon systems. They noted that ships, for example, have unique aspects
but still fall under the same acquisition policy as other weapon
systems, which encourages programs to test technologies in an
operational environment before starting. They also noted that having
allowed space programs to follow a separate acquisition process has
effectively reduced direct oversight from the Office of the Secretary
of Defense.
DOD has recently revised its space acquisition policy, in part to
encourage programs to attain more knowledge about technologies before
starting. It has also taken steps to strengthen its commitment to fully
fund space programs. However, the revised policy still allows space
acquisition programs to begin before demonstrating technologies in an
operational or simulated environment.
Short Tenures and Workforce Deficiencies May Disrupt Programs as Well
as DOD's Overall Ability to Implement Reform:
Short tenures for top leadership and program managers within the Air
Force and the Office of the Secretary of Defense have lessened the
sense of accountability for acquisition problems and further encouraged
a short-term view of success, according to officials we interviewed.
Turnover makes it difficult for upper-level managers to establish
effective working relationships with program managers, resulting in
less trust when divulging problems.
These concerns have been echoed in prior GAO and DOD reports. DOD has
taken action over the past decade to lengthen the tenure of program
managers. However, the Young Panel reported that the average tenure of
a space program manager is just 2 years and recommended that tenures be
extended to a minimum of 4 years to minimize disruption to programs.
Even with this extension, it is likely that programs, which typically
last longer than 4 years, will continue to experience turnover in
program management. Many current programs, including SBIRS High, the
Space Tracking and Surveillance System, Global Positioning System II
modernization effort, and AEHF, were started in the mid-to late 1990s.
We have also reported in the past that the short tenures typical of
program managers make it difficult for them to change the system of
incentives because other participants can wait out reforms they oppose.
Moreover, DOD acquisition executives do not necessarily stay in their
positions long enough to develop the needed long-term perspective or to
effectively change traditional incentives.
Officials we spoke with frequently cited other workforce-related
deficiencies that put pressure on program managers and acquisition
executives. For example, there are not enough experienced program
managers to run space programs and or enough experts in software
engineering--a consequence of starting more programs than DOD can
afford and effectively manage. Earlier policies of having industry
assume more responsibility also contributed to this dearth of expertise
within DOD. As a result, DOD has increasingly relied on outside experts
to help manage programs. At the same time, a limited number of these
outside experts are available to provide technical support to DOD's
various space programs.
Industry-Related Pressures:
Officials we spoke with pointed out a number of pressures associated
with contractors who develop space systems for the government--mostly
having to do with the level of oversight and insight program managers
have with their contractors as well as pressures among contractors to
produce low-cost estimates while bidding on contracts. Specific
concerns mentioned include the following.
* Nonincumbent contractors are often able to submit a lower price than
the incumbent because they can be optimistic without being challenged
by DOD. These optimistic estimates enable them to win new contracts. At
the same time, however, nonincumbents are not necessarily the best
organizations to carry out the development program, particularly
because they do not have the technical and management experience
associated with the legacy system being replaced.
* Industry has been consolidated to a point where there may be only one
company that can develop a needed component for a satellite system.
This has enabled contractors to hold some programs hostage.
* Program managers are often not equipped to understand what is behind
a contractor's proposal, particularly because contractors are not
likely to disclose technical risks and highlight other negative
aspects.
* Industry puts pressure on programs to have contractors develop
critical technologies within an acquisition environment versus having
the labs do it. When labs build technologies, the government allows the
contractors that work on the system that would ultimately use the
technologies to scrap them in favor of employing their own methods and
expertise.
* Program managers are not always experienced enough to stand up to
contractors when development is being mismanaged. Program managers also
may not understand the best ways to incentivize contractors and gain
insight into their performance.
* Contractors are facing workforce pressures similar to those
experienced by the government, that is, not enough technical expertise
to develop highly complex space systems. (Our recent report on space
S&T echoed this concern as well, pointing out that several studies have
found that both industry and the U.S. government face substantial
shortages of scientists and engineers and that recruitment of new
personnel is difficult because the space industry is one of many
sectors competing for the limited number of trained scientists and
engineers.)
* Some space programs are facing pressures related to funding and
technology development because of an expectation widely held in the
1990s that the commercial space market would experience a boom. At the
time, DOD decreased funding for some capabilities, principally space
launch, assuming the market could pay for a portion of research and
development and that economies of scale would result. It also relied on
the commercial sector to develop knowledge about production of
satellites that eventually were purchased as part of the Wideband
Gapfiller Satellite program. However, when anticipated commercial
orders using the same technologies did not pan out, the government
experienced unanticipated schedule delays.
Conclusion:
By delving into the underlying incentives and pressures that cause
space system acquisitions to go awry, DOD will be better equipped to
take the steps needed to attain successful outcomes. The comments of
the 40-plus experts interviewed for this report reinforce our past
findings that those steps should include the development of an overall
investment strategy that prioritizes funding and the establishment of a
sound business case before starting an acquisition program.
Agency Comments and Our Evaluation:
In written comments on our draft report (see enc. I), DOD presented its
views in two primary areas. First, in responding to our conclusion that
problems in space acquisitions are largely reflective of a lack of an
investment strategy for space programs, DOD commented that it is
implementing a new requirements process--known as the Joint
Capabilities Integration and Development System (JCIDS)--designed to
ensure that each new military program is aligned with current and
future joint needs. We have acknowledged in our past work that the
implementation of JCIDS is a positive step toward realizing a DOD-wide
investment strategy for weapon systems. In using this new process to
achieve better outcomes, DOD will need to systematically prioritize its
weapon system programs against funding plans and consistently perform
rigorous analysis of alternatives that weigh the costs and benefits of
achieving each desired capability via a space platform versus an air,
land, or sea platform. Until DOD's strategic weapon system plans for
the future are better linked to DOD's budget, these programs will
continue to experience funding shortfalls, the shifting of funding from
program to program, and accompanying schedule delays.
Second, DOD commented on our conclusion that problems in space
acquisitions are also a result of DOD's tendency to begin these
programs before establishing a sound business case. DOD pointed out
that criteria in its new space acquisition policy, which was updated on
December 27, 2004, are designed to ensure a program's readiness to
proceed into the development phase (or "program start"). We recognize
that the new policy should increase knowledge about space programs
before investment decisions are made but remain concerned that DOD will
start acquisition programs and commit to cost, schedule, and
performance baselines before it has established a sound business case-
-which we have found to be a match between requirements and resources
(time, money, and mature technologies).
DOD also provided a set of comments that it termed "corrections to
errors in fact." In enclosure II, we respond to this set of comments;
changes made to the draft report and areas of disagreement between us
and DOD are highlighted.
We are sending copies of this report to the Secretaries of Defense and
the Air Force and interested congressional committees. We will make
copies available to others on request. In addition, the report will be
available on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Other staff making key contributions to
this report include Cristina Chaplain, Maricela Cherveny, Lily Chin,
Art Gallegos, Jean Harker, John Krump, and Nancy Rothlisberger.
Sincerely yours,
Signed by:
Robert E. Levin, Director:
Acquisition and Sourcing Management:
[End of section]
Enclosure I: Comments from the Department of Defense:
OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE:
6000 DEFENSE PENTAGON:
NETWORKS AND INFORMATION INTEGRATION:
WASHINGTON, DC 20301-5000:
JUN 6 2005:
Mr. Robert E. Levin:
Director, Acquisition and Sourcing Management:
U.S. General Accounting Office:
Washington, D.C. 20548:
Dear Mr. Levin:
This is the Department of Defense (DoD) response to the GAO draft
report, "DEFENSE ACQUISITIONS: Incentives and Pressures That Drive
Problems Affecting Satellite and Related Acquisitions," dated May 2,
2005 (GAO Code 120402/GAO-05-570R).
The Department of Defense acknowledges receipt of the draft report and
does not concur with all the GAO's issues. Our formal comments are
attached. We also attempted to correct several factual errors and
provided some routine administrative corrections.
Please note that the Under Secretary of the Air Force updated the
National Security Space Acquisition Policy 03-01 on December 27, 2004.
This revised version contains additional guidance in some of the areas
which the GAO expressed concern. Until such time as programs have the
opportunity to progress under the revised policy, we will not begin to
see evidence of change. We welcome the opportunity to work with the GAO
to ensure the final report reflects a clear understanding of those
aspects of the space acquisition process over which the Department has
control.
I recommend that the GAO reassess the impact of the revised National
Security Space Acquisition Policy 03-01 following a year of
implementation.
Signed for:
Linton Wells II:
Principal Deputy:
Enclosure: As stated:
GAO DRAFT REPORT DATED MAY 2, 2005 GAO-05-570R (GAO CODE 120402):
"DEFENSE ACQUISITIONS: INCENTIVES AND PRESSURES THAT DRIVE PROBLEMS
AFFECTING SATELLITE AND RELATED ACQUISITIONS":
DEPARTMENT OF DEFENSE COMMENTS TO THE GAO CONCERNS:
CONCERN 1: The GAO found that space acquisition problems are largely
reflective of a lack of an overall investment strategy (p. 3/GAO Draft
Report).
DOD COMMENT: The basis behind a sound investment strategy includes
establishing a sound requirements identification and prioritization
process, determining the architectural context of the investment, and
deciding on the amount of risk that will be accepted within resource
constraints. The Department's recently developed Joint Capabilities
Integrations and Development System (JCIDS) represents a major step
toward outlining the necessary discipline in the requirements process.
In addition, the Milestone Decision Authority (MDA) has required
architectural context determinations be made prior to all Defense Space
Acquisition Board Reviews held since inception of the NSS Acquisition
Policy 03-01. These are completed as part of the Information Support
Plan and integrated architectural products development, and they
outline the relevance of the space system being proposed in light of
crucial interfaces with or impacts upon supporting/supported systems,
external organizations, and missions. After the requirements are
validated through the JCIDS process and the relevance of the new system
in the overall architecture is established, the MDA determines if a new
system is feasible and affordable - with an appropriate focus on
mission success. Prior to starting a new space program, the MDA
determines the amount of acceptable program risk given the urgency of
need and potentially available resources. When followed with rigor and
discipline, this end-to-end process provides a sizable improvement in
the refinement of our existing space investment strategy. Finally, the
acquisition process requires constant coordination with the Planning,
Programming, Budgeting and Execution process and consideration of other
competing portfolios, in addition to the space portfolio, on a program-
by-program basis. We continue to look for areas of improvement in all
aspects of our current space investment strategy while allowing the
process to mature.
CONCERN 2: The GAO found that space acquisition problems are largely
reflective of a tendency to set start dates for programs before a sound
business case for them has been established (p. 3/GAO Draft Report).
DOD COMMENT: National Security Space (NSS) Acquisition Policy 03-01
identifies Key Decision Point (KDP)-B as the official "Program
Initiation" point for an NSS program. In terns of establishing a solid
business case, KDP-B is the point by which a funding baseline must be
established. The purpose of KDP-B is to determine the program's
readiness to begin the preliminary design development activities of
Phase B.
It is designed to increase confidence in the selected NSS system
altemative(s) by assessing the estimated risk levels and projected
performance envelope at a detailed engineering level. Where feasible,
critical technology should complete testing in a relevant environment
during Phase B. Technology that has not been tested in a relevant
environment should be deferred to the next increment. Key acquisition
documentation required for review during the KDP-B Independent Program
Assessment include: initial integrated architecture from the system
program office; updated system level CONOPS from concept sponsor;
Milestone Decision Authority (MDA)-approved Acquisition Strategy; JROC-
approved updated Capability Development Document; Director, OT&E
approved Test and Evaluation Master Plan; Integrated Program Summary;
draft Acquisition Decision Memorandum; and draft Acquisition Program
Baseline. Based on a thorough assessment of a program's maturity as
outlined above, the DoD Space MDA decides on readiness to proceed to
the next acquisition phase and consequently, "official" program start.
CORRECTIONS TO ERRORS IN FACT:
1. Page 2, first bullet. Report states, "'This system [SBIRS] is
intended to replace an older generation of missile-warning
satellites...." This phrasing fails to acknowledge that in addition to
serving as a replacement for DSP, SBIRS will also provide critical, new
capabilities for current and future threats. SBIRS will significantly
improve upon DSP . with JROC-validated capabilities that are essential
for countering today's and tomorrow's threats. It is extremely
important to understand this distinction between DSP and SBIRS,
especially given the drive to "cut back on [SBIRS] capabilities" as
mentioned earlier in the document.
2. Page 2, same bullet. Report goes on to state that delays in the
Space-Based Infrared System (SBIRS) High Program are "forcing DoD to
operate the older generation of satellites much longer than expected."
The DoD is operating the Defense Support Program satellites longer than
expected because they are fortuitously living longer than expected.
3. Page 10, second full paragraph. Report implies that unlike the rest
of the DoD, the space community has not adopted the evolutionary
development approach--that is, pursuing incremental increases in
capability versus significant leaps. NSS 03-01 states that Evolutionary
Acquisition (EA) is the preferred strategy for rapid acquisition of
mature technology for the user. The System Program Director/ Program
Manager should describe the program's EA strategy in the program's
Acquisition Strategy. The two main processes to perform EA are:
a) Spiral Development. In this process, a desired capability is
identified, but the end-state requirements are not known at program
initiation. Those requirements are refined through demonstration and
risk management, there is continuous user feedback, and each increment
provides the user the best possible capability. The requirements for
future increments depend on feedback from users and technology
maturation.
b) Incremental Development. In this process, a desired capability is
identified, an end-state requirement is known, and that requirement is
met over time by development of several increments, each dependent on
available mature technology.
4. Page 14, first paragraph, last sentence states: "In fact, problems
related to technology discovery have severely impacted the SBIRS High
program, among others." Technology discovery has not impacted SBIRS
development. The 2002 Independent Review Team assessed the SBIRS space
segment as technologically mature. The issues identified by the IRT
involving immaturity to enter development were not technology issues.
To quote the IRT report:
"In general, faulty and overly optimistic assumptions laid the ground
work for SBIRS High program activation in 1996. These included:
extensive software reuse; high software productivity levels; commercial
practice benefits; economic order efficiencies with satellite lot buys;
availability of technical models; mature understanding of requirements,
Concept of Operations (CONOPS), and interface specifications; and
management stability."
The same IRT report also noted that SBIRS was following a phased
implementation approach to address operational requirements.
5. Page 14, second paragraph states: "We also previously found that
DoD's new space acquisition policy increased acquisition risks by
allowing programs to begin without having technologies proven in an
operational or simulated environment or even begun with technologies in
even lower stages of maturity." KDP-B is the official "Program
Initiation" point for an NSS program as stated in the response to
Finding 2. Design work is best tested when a technical baseline has
been established. We are committed to testing technology in a relevant
environment no later than KDP-C. Because the design baseline is not
finalized until after Phase B, there is no technical baseline to be
"proven in an operational or simulated environment" before "allowing
programs to begin," that is, by KDP-B. Regarding maturity, the program
office is required to conduct Technology Readiness Assessments during
Phases A and B. In addition, for KDP-B and KDP-C, the Component Science
and Technology Executive will conduct an independent review of the
program office generated Technology Readiness Assessment. Finally, the
Integrated Program Summary's detailed Risk Management section addresses
technical maturity. At each KDP and Build Approval, the program office
is expected to identify the key technology components of the system and
provide an assessment of the maturity of each key component using the
Technology Readiness Level (TRL) method identified in the DoD
Acquisition Guidebook. The Independent Program Assessment Team (IPAT)
reviews the program office assessment and determines if, in their view,
all key technology components of the program have been identified. The
]PAT also provides its own independent assessment of the maturity of
the key components using the TRL method. The intent is to make the DoD
Space MDA knowledgeable of the state of key component maturity so
appropriate direction can be given in the Acquisition Decision
Memorandum for additional technology maturation/risk reduction
activities.
6. Page 14, last paragraph cites several unique aspects of space
acquisition programs: long length, complexity, and high cost of
operational testing. A more commonly quoted set of criteria is: low
quantities produced, continual development during the acquisition
cycle, long operational life, infeasibility of conducting operational
testing, and inability to repair once on orbit.
7. Page 14, last paragraph states that officials within the Office of
the Secretary of Defense "noted that allowing space programs to follow
a separate acquisition process has effectively blocked-out oversight
from the Office of the Secretary o^ Defense." NSS 03-01 is a policy of
inclusiveness. In addition to the Vice Chairman of the Joint Chiefs of
Staff co-chairing the Defense Space Acquisition Board with the MDA,
multiple OSD-level advisors and representatives serve as DSAB
principals and, as such, advise the MDA. The OSD Cost Analysis
Improvement Group is also responsible for developing independent cost
analyses of DoD space MDAPs in support of the DoD space MDA's DSAB
process. The role of OSD in oversight of defense space programs with
respect to the Planning, Programming, Budgeting and Execution System
remains unchanged. Each Service regularly submits Selective Acquisition
Reports, Unit Cost Reports, and Defense Acquisition Executive Summary
reports to the OSD staff. OSD is also required to coordinate on key
acquisition documentation, such as the Acquisition Strategy,
Acquisition Decision Memorandum, and Acquisition Program Baseline, Test
and Evaluation Master Plan, and Integrated Program Summary. The March
1, 2003 OSD Report to Congress on Defense Space Acquisition Programs
provides further detail on this topic.
8. Page 22, Enclosure III, Officials Interviewed for This Review, lists
the Secretary of the Air Force's Space Plans and Policy Division as
being interviewed. The correct organization is the Office of the Under
Secretary of the Air Force's Directorate of Space Acquisition.
[End of section]
Enclosure II: Department of Defense Comments and GAO's Responses:
The Department of Defense (DOD) stated that we failed to acknowledge
that the SBIRS High system, as currently planned, would serve to
replace the Defense Support Program and improve on the capabilities it
provides.
GAO's Response: We have changed the text in this report to reflect that
SBIRS High is intended to also upgrade existing capabilities.
5. DOD stated that it is operating its Defense Support Program
satellites longer than expected because they are "fortuitously living
longer than expected."
GAO's Response: Although DOD's original plan was to begin launching
SBIRS High satellites in 2002 regardless of the health or longevity of
the Defense Support Program satellites, we revised this point to
concentrate on the acquisition of SBIRS High.
DOD stated that our draft report implies that unlike the rest of DOD,
the space community has not adopted the evolutionary development
approach--that is, pursuing incremental increases in capability versus
significant leaps.
GAO's Response: While the space acquisition policy has a section on
Evolutionary Acquisition, the Air Force continues to pursue significant
leaps in technology within its acquisition programs. For example, DOD
plans to migrate from the Advance Extremely High Frequency (AEHF)
satellites with radio frequency crosslinks capable of transmitting data
at 60 megabits per second to the Transformational Satellite
Communication System (TSAT) with laser crosslinks capable of supporting
20,000 megabits per second.
DOD commented that technology discovery has not affected the
development of SBIRS High.
GAO's Response: Although we agree that currently the major problems on
SBIRS High are related to system integration, earlier in the program,
there were technical development problems related to the sensors and
satellites. We revised the text to show that system integration issues
have also affected the development of SBIRS High.
DOD stated that it is committed to testing technology in a relevant
environment no later than key decision point C. DOD added that
regarding maturity, the program office is required to conduct
technology readiness assessments. Finally, DOD stated that the
integrated program summary's detailed risk management section addresses
technical maturity.
GAO's Response: We note that the recently revised space acquisition
policy added processes to assess the maturity of critical technologies
and provided that "where feasible, critical technology should complete
testing in a relevant environment during Phase B." The policy also
states that technology that has not been tested in a relevant
environment should be moved to the next increment. However, the policy
still allows programs to continue to mature technology while they are
designing the system and undertaking other product development
activities during Phase B. Our work on best practices shows that
successful acquisition programs do not start product development unless
a match between requirements and the resources (time, technology, and
money) can be made, and technologies should be matured in an
environment that is focused on technology development. Under the DOD
space acquisition policy, programs are allowed to enter Phase B with
technologies that are immature.
Regarding our citing some unique aspects of space acquisition programs,
DOD mentioned low quantities produced, continual development during the
acquisition cycle, long operational life, infeasibility of conducting
operational testing, and inability to repair once on orbit.
GAO's Response: The three unique aspects of space acquisition programs
cited in our report were those given by interviewed officials.
Concerning the DOD-mentioned criteria, our work has found that
technology development is best conducted before product development
rather than continually throughout the acquisition cycle. Also, low
quantities produced and long operational life are aspects shared by
many non-space weapon systems. In addition, we have reported that
although operational testing is expensive, it is feasible.
DOD took issue with a comment that the separate space acquisition
process has effectively blocked out oversight from the Office of the
Secretary of Defense (OSD). DOD clarified that its space policy is one
of inclusiveness and identified the various organizational participants
in the space acquisition process.
GAO's Response: We note that DOD is taking issue with comments we
obtained from officials within OSD. Because this information is opinion
and is qualified as such in this report, there are no errors to
correct. The space acquisition policy does provide for OSD stakeholders
on the Defense Space Acquisition Board. However, it is the Under
Secretary of the Air Force who has milestone decision authority over
space programs, and not the Under Secretary of Defense for Acquisition,
Technology, and Logistics, who has milestone decision authority for
other weapon systems. We revised our report to state that the separate
space acquisition process has effectively reduced rather than blocked
direct OSD oversight.
DOD commented that we had incorrectly identified the Secretary of the
Air Force's Space Plans and Policy Division as being interviewed.
GAO's Response: We changed that item in enclosure IV to read the
"Directorate of Space Acquisition."
[End of section]
Enclosure III: Related Reports:
DOD:
Report of the Defense Science Board/Air Force Scientific Advisory Board
Joint Task Force on Acquisition of National Security Space Programs,
May 2003 (also referred to as the "Young Panel report") and the July
2004 update to this report.
Space-Based Infrared System Independent Review Team, Final Report,
February 2002.
GAO:
Defense Acquisitions: Assessments of Selected Major Weapon Programs,
GAO-05-301 (Washington, D.C.: March 31, 2005).
Technology Development: New DOD Space Science and Technology Strategy
Provides Basis for Optimizing Investments but Future Versions Need to
Be More Robust, GAO-05-155 (Washington, D.C.: Jan. 28, 2005).
Defense Acquisitions: Space-Based Radar Effort Needs Additional
Knowledge before Starting Development, GAO-04-759 (Washington, D.C.:
July 23, 2004).
Defense Acquisitions: Risks Posed by DOD's New Space Systems
Acquisition Policy, GAO-04-379R (Washington, D.C.: Jan. 29, 2004).
Space Acquisitions: Committing Prematurely to the Transformational
Satellite Program Elevates Risks for Poor Cost, Schedule, and
Performance Outcomes, GAO-04-71R (Washington, D.C.: Dec. 4, 2003).
Defense Acquisitions: Improvements Needed in Space Systems Acquisition
Policy to Optimize Growing Investment in Space, GAO-04-253T
(Washington, D.C.: Nov. 18, 2003):
Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains
at Risk of Cost and Schedule Overruns, GAO-04-48 (Washington, D.C.:
Oct. 31, 2003).
Defense Acquisitions: Improvements Needed in Space Systems Acquisition
Management Policy, GAO-03-1073 (Washington, D.C.: Sept. 15, 2003).
Military Space Operations: Common Problems and Their Effects on
Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.: June
2, 2003).
Military Space Operations: Planning, Funding, and Acquisition
Challenges Facing Efforts to Strengthen Space Control, GAO-02-738
(Washington, D.C.: Sept. 23, 2002).
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15,
2002).
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices, GAO-02-469T (Washington, D.C.: Feb. 27, 2002).
Best Practices: Better Matching of Needs and Resources Will Lead to
Better Weapon System Outcomes, GAO-01-288 (Washington, D.C.: March 8,
2001).
Defense Acquisitions: Employing Best Practices Can Shape Better Weapon
System Decisions, GAO/T-NSIAD-00-137 (Washington, D.C.: April 26,
2000).
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30,
1999).
Best Practices: Successful Application to Weapon Acquisitions Requires
Changes in DOD's Environment, GAO/NSIAD-98-56 (Washington, D.C.: Feb.
24, 1998).
Other:
Booz, Allen, Hamilton, Space Systems Development Growth Analysis,
McLean, Va., October 2002.
[End of section]
Enclosure IV: Officials Interviewed for This Review:
Office of the Secretary of Defense:
* Office of the Director, Program Analysis and Evaluation and Cost
Analysis Improvement Group:
* Office of Force Transformation:
* Under Secretary of Defense for Acquisition, Technology, and
Logistics:
* Assistant Secretary of Defense for Networks and Information
Integration:
Secretary of the Air Force:
* National Security Space Office:
* Office of Science, Technology, and Engineering:
* Directorate of Space Acquisition:
Air Force Space Command:
* Directorate of Requirements:
* Systems Engineering and Integration Office:
* Space and Missile Systems Center:
* Program Executive Office:
* Advanced Extremely High Frequency Program:
* Development and Transformation Directorate:
* Evolved Expendable Launch Vehicle Program:
* NAVSTAR GPS Joint Program:
* Space-Based Infrared Systems Program:
* Space Radar Program:
Missile Defense Agency:
* Space Tracking and Surveillance System Program:
Naval Research Laboratory:
Various industry and former high-ranking Department of Defense
officials:
[End of section]
FOOTNOTES
[1] GAO, Military Space Operations: Common Problems and Their Effects
on Satellite and Related Acquisitions, GAO-03-825R (Washington, D.C.:
June 2, 2003).
