Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices
Gao ID: GAO-10-315R January 14, 2010The Department of Defense (DOD) has had long-standing difficulties developing and delivering space systems on time and within budget. Some programs have been delayed by years and cost billions of dollars more than their initial estimates. Attempts to reform DOD space acquisitions in the past have sought to leverage commercial approaches or rely more on the commercial sector to meet DOD needs. These efforts have not been successful and, in some cases, have exacerbated problems, particularly with respect to oversight. In view of past challenges with adopting commercial approaches, Congress requested an examination of the following questions: (1) What are the differences between commercial and national security space system missions, requirements, and technology development? (2) What acquisition practices adopted by commercial companies could be used for national security space system acquisitions? (3) Which acquisition practices adopted by commercial companies may not be readily adaptable for national security space system acquisitions?
We found that although DOD and the commercial sector both use satellites for missions such as communications and imagery, DOD's requirements are often more demanding. Consequently, while the commercial sector prefers to utilize only mature technologies in satellite development, DOD satellite development typically involves the development of new technologies to meet its more stringent needs. Additionally, DOD--in mission areas such as missile warning and space surveillance--has requirements that do not exist in the commercial sector. In these areas, DOD funds technology development and acquires specific capabilities because they are not commercially available. Overall, the commercial satellite sector delivers satellites faster than the DOD space sector and it typically does so within estimated costs. In many cases, there is no commercial market that DOD can turn to for innovations in space systems--it must either assume leadership in technology invention or partner with other space development agencies such as NASA. Moreover, the missions and requirements DOD is pursuing, along with the need to serve a variety of highly specialized users, have significant implications on the size, complexity, and risk of its space programs. While commercial and DOD space system missions, requirements, and technology development differ in key ways, the commercial sector has adopted practices that could be applied to DOD space system acquisitions to improve cost, schedule, and performance outcomes. For instance, commercial firms define their requirements before initiating development programs, which helps to close resource gaps prior to program start and limit requirements growth. They tie contractor award and incentive fees to acquisition outcomes. They follow evolutionary product development approaches that enable them to achieve gradual gains in capability in relatively short periods while limiting the extent of technology risk they take on in any one increment. The commercial approach, overall, emphasizes gaining critical knowledge before making long-term commitments. GAO has already recommended these practices for DOD adoption. DOD, in fact, has recognized a need to adopt several of these practices and initiated efforts to do so. At the same time, some acquisition practices adopted by the commercial sector, including exclusive use of firm, fixed-price contracts and developing highly accurate cost estimates, may not be successfully applied to DOD in its current acquisition environment because of factors such as unique requirements and immature technologies at program start. For instance, the use of firm, fixed-price contracts for procuring satellites would require a change in paradigm for DOD space programs--a much higher level of knowledge, including mature technologies and mature design--prior to the start of a program. Currently, however, DOD accepts greater technology and development risks and typically uses cost-reimbursement contracts for the first two satellites to be developed and produced. Some programs use fixed-price contracts for any additional satellites. Using fixed-price contracts for the development phase of a program has not worked well, partly due to the high level of unknowns accepted at program start. In addition, other factors, such as launch delays, program funding instability, changing needs, and the diverse array of organizations involved in DOD space programs pose additional challenges to the use of firm, fixed-price contracts.
GAO-10-315R, Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices
This is the accessible text file for GAO report number GAO-10-315R
entitled 'Briefing on Commercial and Department of Defense Space
System Requirements and Acquisition Practices' which was released on
January 14, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-10-315R:
United States Government Accountability Office:
Washington, DC 20548:
January 14, 2009:
The Honorable John Murtha:
Chairman:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
Subject: Briefing on Commercial and Department of Defense Space System
Requirements and Acquisition Practices:
Dear Mr. Chairman:
The Department of Defense (DOD) has had long-standing difficulties
developing and delivering space systems on time and within budget.
Some programs have been delayed by years and cost billions of dollars
more than their initial estimates. Attempts to reform DOD space
acquisitions in the past have sought to leverage commercial approaches
or rely more on the commercial sector to meet DOD needs. These efforts
have not been successful and, in some cases, have exacerbated
problems, particularly with respect to oversight.
In view of past challenges with adopting commercial approaches, you
requested we examine the following questions: (1) What are the
differences between commercial and national security space system
missions, requirements, and technology development? (2) What
acquisition practices adopted by commercial companies could be used
for national security space system acquisitions? (3) Which acquisition
practices adopted by commercial companies may not be readily adaptable
for national security space system acquisitions? The attached briefing
provides the results of our review. This letter provides a brief
summary of how we conducted our work and the results of our review.
Scope and Methodology:
To conduct our review, we interviewed officials and reviewed and
analyzed documentation on missions, requirements, and technology
development from all major U.S. commercial satellite manufacturers and
selected service providers, the two major space industry associations,
a major space insurance broker, and from DOD--Office of the Secretary
of Defense, Air Force Headquarters and Space and Missile Systems
Center, and other organizations responsible for acquisition oversight,
cost analysis, and program analysis of national security space
programs. We interviewed officials from commercial and DOD
organizations and reviewed documentation of their space acquisition
practices, and compared and contrasted these practices to best
practices GAO has previously reported on. Based on interviews and GAO
reports on space system acquisitions and best practices, we determined
whether specific commercial practices--such as requirements
definition, technology maturity, contracting, and cost estimating--may
or may not be readily adaptable and beneficial to national security
space acquisition programs. It should be noted that the commercial
companies we interviewed are not formally recognized as "best
practices" companies; however, many of these practices align with best
practices we have previously reported on.
It should also be noted that our assessment of the applicability of
space acquisition practices adopted by U.S. commercial companies is
focused primarily on unclassified DOD acquisitions and may not be
applicable to classified National Reconnaissance Office (NRO)
acquisitions because we have not reviewed NRO systems and
requirements. However, under this review, we met with and obtained
perspectives on acquisition practices from NRO officials, which we
incorporated as appropriate.
We conducted this performance audit from November 2008 to August 2009,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives. We provided a
draft of the enclosed briefing to DOD officials for their review and
comment. In an August 25, 2009 response, DOD generally agreed with the
information presented and provided technical comments.
Summary:
We found that although DOD and the commercial sector both use
satellites for missions such as communications and imagery, DOD's
requirements are often more demanding. Consequently, while the
commercial sector prefers to utilize only mature technologies in
satellite development, DOD satellite development typically involves
the development of new technologies to meet its more stringent needs.
Additionally, DOD--in mission areas such as missile warning and space
surveillance--has requirements that do not exist in the commercial
sector. In these areas, DOD funds technology development and acquires
specific capabilities because they are not commercially available.
Overall, the commercial satellite sector delivers satellites faster
than the DOD space sector and it typically does so within estimated
costs. In many cases, there is no commercial market that DOD can turn
to for innovations in space systems--it must either assume leadership
in technology invention or partner with other space development
agencies such as NASA. Moreover, the missions and requirements DOD is
pursuing, along with the need to serve a variety of highly specialized
users, have significant implications on the size, complexity, and risk
of its space programs.
While commercial and DOD space system missions, requirements, and
technology development differ in key ways, the commercial sector has
adopted practices that could be applied to DOD space system
acquisitions to improve cost, schedule, and performance outcomes. For
instance, commercial firms define their requirements before initiating
development programs, which helps to close resource gaps prior to
program start and limit requirements growth. They tie contractor award
and incentive fees to acquisition outcomes. They follow evolutionary
product development approaches that enable them to achieve gradual
gains in capability in relatively short periods while limiting the
extent of technology risk they take on in any one increment. The
commercial approach, overall, emphasizes gaining critical knowledge
before making long-term commitments. GAO has already recommended these
practices for DOD adoption. DOD, in fact, has recognized a need to
adopt several of these practices and initiated efforts to do so.
At the same time, some acquisition practices adopted by the commercial
sector, including exclusive use of firm, fixed-price contracts and
developing highly accurate cost estimates, may not be successfully
applied to DOD in its current acquisition environment because of
factors such as unique requirements and immature technologies at
program start. For instance, the use of firm, fixed-price contracts
for procuring satellites would require a change in paradigm for DOD
space programs--a much higher level of knowledge, including mature
technologies and mature design--prior to the start of a program.
Currently, however, DOD accepts greater technology and development
risks and typically uses cost-reimbursement contracts for the first
two satellites to be developed and produced. Some programs use fixed-
price contracts for any additional satellites. Using fixed-price
contracts for the development phase of a program has not worked well,
partly due to the high level of unknowns accepted at program start. In
addition, other factors, such as launch delays, program funding
instability, changing needs, and the diverse array of organizations
involved in DOD space programs pose additional challenges to the use
of firm, fixed-price contracts.
In our briefing, we concluded that given the magnitude of
unanticipated cost and schedule growth on DOD space system acquisition
programs over the last decade, there is a clear need to adopt
practices that emphasize attaining knowledge up front, minimize
requirements changes late in programs, and provide the right support
and accountability to both program managers and contractors. The
commercial companies we studied were consistent in their adoption of
such approaches and the belief that knowledge-based development has
enabled them to shorten delivery time frames and limit cost growth.
While DOD programs have more inherent risks, DOD has recognized that
its programs can greatly benefit from adopting similar practices and
has initiated actions to do so.
Previous GAO reports and testimonies have identified potential
obstacles to making these improvements as well as areas that still
need to be addressed. We have also stressed that adopting commercial
approaches should not equate to relaxed oversight and decreased
government technical expertise, as has been the case in the past.
Rather, we have recommended how DOD can make trade-offs to reduce
risks earlier and better manage those risks that it does accept. DOD
has generally concurred with these recommendations and has taken
measures to address them, including changes to acquisition policies
and acquisition practices.
Agency Comments:
We provided draft copies of this letter and briefing to DOD for review
and comment. DOD concurred with the content and message presented and
had no written comments.
We are sending copies of this letter and briefing to Department of
Defense and other interested congressional committees. In addition,
these documents will be available at no charge on GAO's Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions, please contact me at (202)
512-4841 or chaplainc@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this letter. Principal contributors to this project were
Arthur Gallegos, Assistant Director; Martin G. Campbell; Kristine R.
Heuwinkel; Laura T. Holliday; Richard Y. Horiuchi; Sylvia Schatz; and
Peter E. Zwanzig.
Sincerely yours,
Signed by:
Cristina T. Chaplain:
Director, Acquisition and Sourcing Management:
Enclosures - 1:
[End of section]
Enclosure I: Briefing Slides:
Briefing to Staff of the Subcommittee on Defense, House Appropriations
Committee:
August 2009:
Commercial and Department of Defense Space System Requirements and
Acquisition Practices:
Contents:
* Introduction:
* Results in Brief:
* Background:
* Scope and Methodology:
* Findings:
* Concluding Observations:
* Backup Slides:
Introduction:
The Department of Defense (DOD) has had long-standing difficulties
developing and delivering space systems on time and within budget.
Some programs have been delayed by years and cost billions of dollars
more than their initial estimates. Attempts to reform DOD space
acquisitions in the past have sought to leverage commercial approaches
or rely more on the commercial sector to meet DOD needs. These efforts
have not been successful and, in some cases, have exacerbated
problems, particularly with respect to oversight. In view of past
challenges with adopting commercial approaches, this briefing
addresses the following objectives:
1. What are the differences between commercial and national security
space system missions, requirements, and technology development?
2. What acquisition practices adopted by commercial companies could be
used for national security space system acquisitions?
3. Which acquisition practices adopted by commercial companies may not
be readily adaptable for national security space system acquisitions?
Results in Brief:
Although DOD and the commercial sector both use satellites for
missions such as communications and imagery, DOD's requirements are
often more demanding. Consequently, while the commercial sector
prefers to utilize only mature technologies in satellite development,
DOD satellite development typically involves the development of new
technologies to meet its more stringent needs. Additionally, DOD”in
mission areas such as missile warning and space surveillance”has
requirements that do not exist in the commercial sector. In these
areas, DOD funds technology development and acquires specific
capabilities because they are not commercially available.
While commercial and DOD space system missions, requirements, and
technology development differ in key ways, the commercial sector has
adopted practices that could be applied to DOD space system
acquisitions to improve cost, schedule, and performance outcomes. Many
of these are practices GAO has already recommended for DOD adoption.
They emphasize gaining critical knowledge before making long-term
commitments. DOD, in fact, has recognized a need to adopt several of
these practices and initiated efforts to do so.
At the same time, some acquisition practices adopted by the commercial
sector, including exclusive use of firm, fixed-price contracts and
developing highly accurate cost estimates, may not be successfully
applied to DOD in its current acquisition environment because of
factors such as unique requirements and immature technologies at
program start.
Background: Commercial Satellite Industry Overview:
* Global revenues from commercial satellite activity in 2008 totaled
$7.2 billion”$5.2 billion manufacturing; $2.0 billion launch.
* U.S. revenues from commercial satellite activity in 2008 totaled
$2.0 billion ($1.8 billion manufacturing; $0.2 billion launch).
Table:
Revenues from U.S. satellite manufacturing and launch activity:
Total U.S. satellite manufacturing (government & commercial):
2004: $3.9 billion;
2005: $3.2 billion;
2006: $5.0 billion;
2007: $4.8 billion;
2008: $3.1 billion.
U.S. commercial satellite manufacturing (% US total):
2004: $1.4 billion; (35.9%);
2005: $1.3 billion; (40.6%);
2006: $1.9 billion; (38.0%);
2007: $2.2 billion; (45.8%);
2008: $1.8 billion; (58.1%).
U.S. commercial launch:
2004: $0.4 billion;
2005: $0.1 billion;
2006: $0.1 billion;
2007: $0.2 billion;
2008: $0.2 billion.
Total U.S. commercial satellite activity (rows 2+3):
2004: $1.8 billion;
2005: $1.4 billion;
2006: $2.0 billion;
2007: $2.4 billion;
2008: $2.0 billion.
Sources: GAO analysis of Satellite Industry Association, Futron
Corporation, and Federal Aviation Administration FAA) data.
Note: All satellite manufacturing revenues are recognized in the year
of satellite launch, and geographically determined by location of
manufacturers' headquarters.
[End of table]
* Primary U.S. commercial satellite manufacturers: six:
- Ball Aerospace, Boeing, General Dynamics, Lockheed Martin Commercial
Space Systems, Orbital Sciences, Space Systems/Loral.
Table: Commercial U.S.-manufactured satellites launched per year 2004
through 2008:
Year: 2004;
Satellites launched: 12.
Year: 2005
Satellites launched: 10.
Year: 2006
Satellites launched: 12.
Year: 2007
Satellites launched: 19.
Year: 2008
Satellites launched: 10.
Source: GAO analysis of FAA data.
Note: Only satellites intended for operational use are included in
this count, and not those intended solely for test, development, or
scientific research. Commercial satellites are defined as those
serving a commercial function or operated by a commercial entity.
[End of table]
* Typical cost of commercial satellites: $75 million”$300 million.
* Typical commercial program length: about 2-3 years.
Background: DOD Satellite Acquisition Overview:
* DOD investment in major space programs in 2008 totaled $6.1 billion--
$3.2 billion for research, development, test and evaluation (RDT&E);
$1.2 billion for procurement; $1.6 billion for launch (Evolved
Expendable Launch Vehicle).
* Primary U.S. defense satellite manufacturers: three:
- Boeing, Lockheed Martin, Northrop Grumman.
Table: DOD satellites launched per year, 2004 through 2008:
Year: 2004;
Satellites launched: 5.
Year: 2005
Satellites launched: 4.
Year: 2006
Satellites launched: 5.
Year: 2007
Satellites launched: 6.
Year: 2008
Satellites launched: 2.
Source: GAO analysis of FAA data.
Note: Only satellites intended for operational use are included in
this count, and not those intended solely for test, development, or
scientific research. DOD satellites are defined as those manufactured
for DOD or any of its military services, or for service-related
entities such as the National Reconnaissance Office and Defense
Advanced Research Projects Agency.
[End of table]
* Program acquisition unit cost of current major DOD satellite
acquisition programs: $216.3 million”$3.1 billion;
- For satellite communications programs”Advanced Extremely High
Frequency, Mobile User Objective System, and Wideband Global SATCOM”
the range is $414.6 million”$2.6 billion.
* Schedule from program start to first launch of current major DOD
satellite acquisition programs: about 5 to 14 years.
Table: DOD Investment in Major Space Programs (Fiscal year 2009
dollars in millions[A]):
RDT&E:
Year: 2008: $3,204.4;
Year: 2009: $2,996.2;
Year: 2010: $2,751.2;
Year: 2011: $2,452.9;
Year: 2012: $1,933.9;
Year: 2013: $1,836.1.
Procurement:
Year: 2008: $2,859.7;
Year: 2009: $4,185.0;
Year: 2010: $4,369.6;
Year: 2011: $3,115.2;
Year: 2012: $3,276.7;
Year: 2013: $2,829.2.
Other[B]:
Year: 2008: $22.5;
Year: 2009: $17.5;
Year: 2010: $18.1;
Year: 2011: $10.5;
Year: 2012: $12.2;
Year: 2013: $0.0.
Total:
Year: 2008: $6,086.6;
Year: 2009: $7,198.7;
Year: 2010: $7,138.8;
Year: 2011: $5,578.6;
Year: 2012: $5,222.7;
Year: 2013: $4,665.3.
Source: GAO analysis of fiscal year 2009 DOD data. Note: Numbers may
not add due to rounding.
[A] Includes the following programs: Advanced Extremely High
Frequency, Evolved Expendable Launch Vehicle, Global Broadcast
Service, Naystar Global Positioning System, Global Positioning System
IIIA, Mobile User Objective System, National Polar-orbiting
Operational Environmental System, Space Based Infrared System High,
Space Based Space Surveillance Block 10, Space Tracking and
Surveillance System, and Wideband Global SATCOM. Does not include
development efforts that have yet to formally initiate acquisitions,
including Third Generation Infrared Surveillance, Infrared
Augmentation Satellite, and Transformational Satellite Communications
System.
[B] Other includes military construction and acquisition operations
and maintenance costs.
[End of table]
Some of DOD's Past Attempts to Leverage Commercial Practices Were
Unsuccessful:
One of DOD's attempts to leverage commercial practices was Total
System Performance Responsibility (TSPR)[A], which:
* Aimed to streamline DOD's acquisition process and leverage
innovation and management expertise from the private sector;
* Gave contractor total responsibility for the integration of a weapon
system and for meeting DOD's requirements;
* Reduced government oversight and shifted key decision-making
responsibilities onto contractors;
* Magnified problems on a number of satellite acquisition programs
because it was implemented in a manner that enabled requirements creep
and poor contractor performance;
* For some programs, TSPR resulted in relaxed specifications and
inspections of the contractor, loss of quality in the manufacturing
process, and poor-quality parts that caused test failures, unexpected
redesigns, and late delivery of parts.
[A] See GAO-07-96 and GAO-09-325. Full citations are provided on
slides 46 and 47.
DOD Has Made Erroneous Assumptions in the Past About Leveraging the
Commercial Sector:
Wideband Global SATCOM[A]:
* DOD attempted to leverage commercial demand for satellites with
similar technologies, but the commercial demand did not materialize;
* Initial operational capability took twice as long as planned due
largely to manufacturing problems.
Evolved Expendable Launch Vehicle[B]:
* DOD attempted to leverage commercial launch demand but the
commercial demand did not materialize;
* The government had to bear most of the cost burden and total program
costs nearly doubled (increased by about 96 percent) from first to
latest cost baseline.
[A] See GAO-07-96, GAO-05-301, and GAO-06-391.
[B] See GAO-08-1039.
Scope and Methodology:
We interviewed officials and reviewed and analyzed documentation on
missions, requirements, and technology development from all major U.S.
commercial satellite manufacturers and selected service providers, the
two major space industry associations, a major space insurance broker,
and from DOD--Office of the Secretary of Defense, Air Force
Headquarters and Space and Missile Systems Center, and other
organizations responsible for acquisition oversight, cost analysis,
and program analysis of national security space programs.
We interviewed officials from commercial and DOD organizations and
reviewed documentation of their space acquisition practices, and
compared and contrasted these practices to best practices GAO has
previously reported on.
Based on interviews and GAO reports on space system acquisitions and
best practices, we determined whether specific commercial practices ”
such as requirements definition, technology maturity, contracting and
cost estimating ” may or may not be readily adaptable and beneficial
to national security space acquisition programs.
Limitations:
* Our assessment of the applicability of space acquisition practices
adopted by U.S. commercial companies is focused primarily on
unclassified DOD acquisitions and may not be applicable to classified
National Reconnaissance Office (NRO) acquisitions because we have not
reviewed NRO systems and requirements. However, under this review, we
met with and obtained perspectives on acquisition practices from NRO
officials, which we incorporated as appropriate.
* The commercial companies we interviewed are not formally recognized
as "best practices" companies; however, we identified practices that
company officials told us helped their programs succeed, and many of
these practices align with best practices we have previously reported
on.
We conducted this performance audit from November 2008 to August 2009,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Defense Offices Visited:
Air Force:
* Under Secretary of the Air Force, Directorate of Space Acquisitions,
Washington, DC:
* Space and Missile Systems Center, Los Angeles Air Force Base, CA:
* Office of the Assistant Secretary for Financial Management and
Comptroller, Washington, DC:
Other Defense:
* Office of the Director, Program Analysis and Evaluation, Washington,
DC:
* Office of the Under Secretary of Defense for Acquisition,
Technology, and Logistics, Washington, DC:
* Office of the Secretary of Defense, Cost Analysis Improvement Group,
Washington, DC:
* National Reconnaissance Office, Chantilly, VA:
* National Security Space Office, Washington, DC:
Commercial and Other Organizations Visited:
Commercial satellite manufacturers:
* Ball Aerospace & Technologies Corp., Boulder, CO:
* Boeing Satellite Systems International, El Segundo, CA:
* Lockheed Martin Commercial Space Systems, Newtown, PA:
* Orbital Sciences Corp., Dulles, VA:
* Space Systems/Loral, Palo Alto, CA:
Other commercial satellite industry:
* DigitalGlobe, Longmont, CO:
* Iridium Satellite, LLC, Bethesda, MD:
* Americom Government Services, McLean, VA:
* Intelsat, Washington, DC:
* International Space Brokers, Rosslyn, VA:
Other:
* Aerospace Corp., El Segundo, CA:
* Former Under Secretary of the Air Force, Denver, CO:
* Futron Corporation, Bethesda, MD:
* Satellite Industry Association, Washington, DC:
* Space Foundation, Washington, DC:
Key Documents Analyzed:
* DOD acquisition policies.
* DOD space program-specific documentation, such as Selected
Acquisition Reports, acquisition decision memoranda, and acquisition
strategies.
* Report to Congress of the Independent Assessment Panel on the
Organization and Management of National Security Space, July 2008.
* Report of the Defense Science Board/Air Force Scientific Advisory
Board Joint Task Force on Acquisition of National Security Space
Programs, May 2003, and update, July 2004.
* Report of the Commission to Assess United States National Security
Space Management and Organization, January 2001.
* GAO reports on space and nonspace acquisitions, best acquisition
practices, cost estimating, program management, contracting, and
personnel management.
Objective 1 Overview:
Commercial and DOD Missions, Requirements, and Technology Development
Differ in Key Ways:
* Missions and Requirements;
* Technology Development and Maturity.
Missions and Requirements:
DOD Satellite Missions and Requirements Differ from Those of the
Commercial Sector:
DOD space system acquisitions meet warfighter and intelligence
community requirements, while commercial space system acquisitions
meet market demands.
In some areas, DOD and commercial needs are similar:
* Communications;
* Imagery.
In some areas, DOD and commercial needs converge, but the commercial
sector relies on government satellites:
* Position, navigation, and timing;
* Weather, climate, and environmental monitoring.
In other areas, DOD has needs that do not apply to the commercial
sector:
* Missile warning;
* Space surveillance;
* Nuclear detonation detection.
DOD Uses Commercially Available Satellite Services to Fulfill Some
Missions:
When DOD needs have aligned with commercially available satellite
products, DOD has purchased services from commercial providers:
* Communications:
- DOD uses commercial fixed satellite services to augment its own
satellite communications capabilities;
- Nevertheless, DOD in some cases tailors these services to meet its
unique needs. For example, DOD established a government gateway, which
includes ground-based systems that, among other things, enable users
to directly access DOD-specific communications systems.
* Imagery:
- DOD is the largest purchaser of U.S. commercial satellite imagery;
- DOD helped fund the development of commercial imagery satellites and
purchases satellite imagery from DigitalGlobe and GeoEye.
DOD Requirements Are Often More Stringent Than Those of the Commercial
Sector:
For mission areas that apply to both DOD and the commercial sector,
such as communications and imagery, DOD's requirements often call for
the delivery of more robust capabilities.
DOD Communications Satellite Requirements Frequently Differ from Those
of the Commercial Sector:
Coverage and availability: DOD typically has more demanding
requirements for coverage and availability, such as for communications
using radio frequencies resistant to atmospheric scintillation in
polar regions and under challenging conditions such as in dense
foliage and adverse weather.
Survivability: DOD typically has more demanding standards for
radiation hardened parts, such as microelectronics, which are:
* designed and fabricated with the specific goal of enduring the
harshest space radiation environments, including nuclear events;
* time consuming to obtain”companies typically need to create separate
production lines and in some cases special facilities; only small
volumes are typically produced.
Jam resistance: DOD has demanding requirements for jam resistance,
addressed via:
* frequency hopping in which the transmission codes "hop" across the
bandwidth at a very rapid rate;
* nulling antennas that cancel out the impact of jamming emitters
designed to disrupt U.S. warfighter communications.
Secure communications: DOD has requirements for sophisticated
encryption to enable communications at varying security levels.
DOD Uses Higher Resolution Imagery Than the Commercial Sector:
U.S. commercial companies are allowed to sell imagery with a maximum
of 50 centimeters resolution in black-and-white mode and about 2
meters in color to non-U.S. government entities.
GeoEye can provide optical imagery of 41 centimeters black-and-white
resolution to DOD and other agencies in the intelligence community.
U.S. government-developed surveillance satellites yield even higher
resolution.
Many DOD Requirements Do Not Align with Commercial Needs, Call for
Technology Invention:
Many DOD space system acquisitions are designed to meet warfighter and
intelligence community requirements that do not align with commercial
needs and often require leading edge technologies. For example:
* Missile warning and defense:
- highly sensitive infrared sensors to detect missile heat signatures;
- cryocoolers that cool focal plane arrays to cryogenic temperatures,
in some cases down to 10 degrees Kelvin, create minimal vibration, and
have sufficiently long lifespans.
* Position, navigation, and timing:
- several synchronized atomic clocks per satellite to enable accurate
time signal triangulation;
- advanced anti-jam capabilities to improve system security, accuracy,
and reliability.
* Weather, climate, and environmental monitoring:
- advanced technology microwave sounder to produce daily global
atmospheric temperature, humidity, and pressure profiles;
- cross-tracked infrared sounder to collect measurements of the
earth's radiation to determine vertical distribution of temperature,
moisture, and pressure in the atmosphere;
- visible infrared imager radiometer suite to collect images and
radiometric data used to provide data on the earth's clouds,
atmosphere, ocean, and land surfaces.
* Nuclear detonation detection:
- technologies to measure neutron time-of-flight spectrum, prompt
gamma rays from a nuclear detonation, and delayed gamma rays from a
nuclear detonation debris cloud.
DOD Frequently Attempts to Meet the Needs of Multiple Customers:
DOD often attempts to meet the needs of multiple users on one
satellite, resulting in more missions and/or payloads per satellite,
for example:
* The National Polar-orbiting Operational Environmental Satellite
System (NPOESS) is managed by a tri-agency program office,
representing the DOD/United States Air Force, the Department of
Commerce's National Oceanic and Atmospheric Administration, and the
National Aeronautics and Space Administration. Its original plans
called for 13 instruments, including 10 environmental sensors and 3
subsystems to meet the needs of weather forecasters, climatologists,
and the military.
* The Space Based Infrared System (SBIRS) is intended to meet
requirements of various parts of DOD and the intelligence community
for multiple missions, including missile warning, missile defense,
technical intelligence, and battlespace awareness.
Commercial companies frequently attempt to meet the needs of one
customer per satellite, often resulting in single-mission satellites.
Technology Development And Maturity:
Companies That Purchase Commercial Satellites Prefer Mature
Technologies:
Companies prefer mature technologies, ideally at technology readiness
level (TRL) 9 (see slide 48 for definitions of TRLs), so that risks,
costs, and schedules are known at program start.
Commercial companies sometimes conduct technology development. When
they do, development is typically incremental and is conducted prior
to program start. In addition, more time is allotted to manufacture
satellites that incorporate a new technology due to the need for
additional integration work.
Although companies prefer to use mature technologies for commercial
satellites, they sometimes employ fairly sophisticated technologies,
such as:
* Large unfurlable antenna reflectors to reduce satellite launch
volume;
* Lithium-ion batteries for reduced satellite weight;
* Ion propulsion for reduced satellite weight;
* Phased array antennas for agile coverage.
DOD Funds Technology Development When Needed Technologies Are Not
Commercially Available:
To meet requirements that are unique to DOD, the government has
supported development of many leading-edge technologies (such as those
shown on slide 24). For these technologies, the industrial base is
supported in large part by DOD, the intelligence community, and NASA.
However, rather than maturing technologies in a robust science and
technology (S&T) environment prior to program start, DOD has
frequently allowed immature technologies into space programs. For
example, we have identified several DOD satellite programs in which
multiple technologies were below TRL 6 at program start, such as
Advanced Extremely High Frequency, NPOESS, SBIRS High, and SBIRS Low.
Development Risk Increases with the Number of New Technologies in a
Program:
According to a 2007 National Research Council Space Studies Board
report:
* Cost growth is closely related to development risk, which increases
nonlinearly with the number of new technologies. As more immature
technologies are included in a program, the likelihood of cost and
schedule problems increases substantially.
[End of Objective 1]
Objective 2 Overview:
Some Commercially Adopted Practices Could Be Applied to DOD Space
System Acquisitions:
* Mature critical technologies prior to program start;
* Use evolutionary product development;
* Define requirements to close resource gaps prior to program start
and limit requirements growth;
* Tie contractor award/incentive fees to acquisition outcomes;
* Empower program managers and hold them accountable;
* Obtain independent oversight.
Ensure Technologies Are Mature Prior to Beginning an Acquisition
Program:
Commercial practice: Use only mature technologies:
* Companies typically look to the government to push and prove
technologies first.
* Technologies included are typically at TRL 9, in order to foster
program stability, ensure reliability, and to obtain favorable
insurance rates.
* When technology discovery is conducted, it is done so prior to
system development.
Prevailing DOD practice: Historically, has used immature technologies:
* Unique nature of missions require technology invention/discovery,
but it is frequently not finished prior to system development.
* Many technologies below TRL 6 at program start.
* DOD acquisition policies and congressional legislation reflect
preference for maturing technologies prior to program start.
* Recent efforts (e.g., GPS IIIA and Operationally Responsive Space
efforts) have demonstrated a change to the practice.
Potential benefit to DOD and obstacles to implementation: Reduce cost
and schedule inefficiencies:
* Achieving a high level of technology maturity prior to program
initiation helps (1) ensure resources and requirements match, and (2)
avoid concurrently developing technologies, finalizing designs, and
demonstrating manufacturing processes, which can lead to cost and
schedule inefficiencies.
Potential obstacle:
* There is a long-standing disconnect between the research
laboratories and acquisition programs; DOD lacks an S&T strategy for
space; and the funding process favors acquisitions over S&T programs.
Unless Revolutionary Technologies Are Required, Use Evolutionary
Product Development:
Commercial practice: Development is evolutionary:
* To achieve stability, reduce risk, and enable short program
schedules: (1) new design elements and new parts are minimized, and
(2) standardized designs and parts are tailored as needed for
customers. This enables companies to focus attention on critical
design, development, and integration.
* Design elements not achievable in the initial development are
planned for future generations of the product, which allows time for
technologies to mature.
Prevailing DOD practice: Historically, has promised revolutionary
advances in capabilities:
* In some cases, revolutionary advances in capabilities may be sought,
such as for first-time or one-of-a-kind efforts to satisfy a new
urgent requirement. However, most programs have attempted to satisfy
all requirements in a single step, regardless of design or technology
challenges.
* DOD frequently adopts extensive new designs and custom-made
spacecraft buses and payloads to meet the needs of multiple users.
* Recent efforts have adopted a more evolutionary strategy.
Potential benefit to DOD and obstacles to implementation: Offers an
initial product quickly and at lower cost while technologies are
matured for the next increment:
* While the user may not initially receive the ultimate capability
under this approach, the initial product is available sooner and at a
lower, more predictable cost.
* Exceptions would involve efforts, such as the first GPS, that
introduce a new capability or programs focused on countering new
threats.
Potential obstacle:
* Competition for funding incentivizes programs to promise
revolutionary advances based on optimistic assumptions.
Define Requirements to Close Resource Gaps Prior to Program Start and
Limit Requirements Growth:
Commercial practice: Ensure requirements are well defined prior to
program start and remain stable:
* Requirements are well defined prior to program start so that costs
and feasibility are understood and trade-offs can be made if needed.
* Requirements are negotiated during the contract proposal process to
align with the developer‘s capabilities and strengths. The percent ’new“
in the design may range from 5-20 percent”for those new aspects,
robust systems engineering is applied prior to program start to
minimize unknowns. Afterward, systems engineering is focused on
integrating mature technologies onto a platform.
Prevailing DOD practice: Requirements are typically not well defined
by program start but largely remain stable:
* Poorly defined requirements have had significant consequences for
funding, time, and technology development. While in the past, some
programs experienced requirements creep resulting in large cost and
schedule increases, DOD has made improvements in this area.
* Systems engineering is conducted after programs have been funded and
launched”too late to identify resource gaps, shape requirements, and
inform estimates. Early focus is on defining requirements and maturing
new technologies.
Potential benefit to DOD and obstacles to implementation: Helps reduce
program cost, schedule, and performance risks:
* Early systems engineering knowledge helps identify and address gaps,
such as overly optimistic requirements that cannot be met with current
resources.
Potential obstacles:
* DOD lacks a robust systems engineering functionality.
* Agreement on requirements for space systems is difficult because a
diverse array of organizations are involved in setting requirements.
Once agreement is achieved, it is difficult to change requirements.
Tie Contractor Award and Incentive Fees to Acquisition Outcomes:
Commercial practice: Incentives and penalties that emphasize on-time
delivery and on-orbit performance motivate satellite developers:
* Satellite customers typically tie about 10 to 20 percent of the
contract value to successful on-orbit performance of the satellite
over its expected life, which is frequently 15 years.
* This performance-based payment is key to developers‘ profitability
and is reduced or eliminated accordingly if there are on-orbit
problems.
Prevailing DOD practice: DOD typically uses award and incentive fee
provisions in its contracts and has withheld fees for poor performance:
* DOD‘s guidance states that award fees must be linked to desired
outcomes and prohibits payment of award fees to contractors for
unsatisfactory performance.
* Almost all current major space acquisitions use award and incentive
fee provisions in contracts for development of initial satellites. We
reported that DOD does not consistently evaluate contractors based on
award-fee criteria related to key acquisition outcomes.[A]
Potential benefit to DOD and obstacles to implementation: If aligned
with acquisition outcomes, award and incentive fees might motivate
good contractor performance:
* We recently reported that DOD has achieved savings on some programs
by limiting the opportunities for earning unearned fees in subsequent
periods and tying award fee criteria to acquisition outcomes.[A]
Potential obstacle:
* Because DOD has not developed methods to evaluate the effectiveness
of award fees, it is unaware of whether these contracts are being used
effectively, poor practices go unnoticed, and positive practices are
isolated.
[A] See GAO-06-66 and GAO-09-630.
Empower Program Managers to Execute Their Programs and Hold Them
Accountable for Outcomes:
Commercial practice: Give program managers decision-making authority
and hold them accountable for acquisition outcomes:
* Program managers are given direct responsibility for the direction,
planning, assessment, and resource control of their programs, which
are fully funded at outset.
* Program managers are held accountable for their decisions and their
performance evaluation is based on how well they meet cost, schedule,
and performance elements.
* Program managers are not held accountable for matters beyond their
control.
Prevailing DOD practice: Program managers lack strong authority and
are generally not held accountable for executing programs within
targets:
* Program managers: have little control over funding stability of
incrementally funded programs and shifting funds within programs;
cannot veto new program requirements, which may overly stretch their
programs; and have little authority over staffing.
* Because there are so many aspects of programs outside the program
manager‘s control, DOD is unable to hold them accountable.
* In the past, DOD program managers had more authority.
Potential benefit to DOD and obstacles to implementation: Improve
performance, cost, and schedule outcomes:
* Empowers program managers and holds them accountable for delivering
new products when needed within quality, cost, and performance targets.
Potential obstacle:
* Measures to empower program managers and hold them accountable will
not be as effective as they could until DOD ensures that acquisition
programs are executable, i.e., the needs can best be met with the
chosen concept and the concept can be developed and produced within
existing resources.
Obtain Independent Oversight of Satellite Developers:
Commercial practice: Insurers provide independent oversight of
satellite developers:
* The insurance industry plays a significant role in overseeing
satellite acquisitions and ensuring quality.
* Insurance underwriters consider factors such as favorable on-orbit
performance records and use of reliable and flight-proven technology
when determining insurance terms.
* In order for customers to obtain favorable insurance terms,
developers adhere to strict quality standards.
Prevailing DOD practice: Some independent oversight is applied:
* While DOD does not obtain insurance for its satellite programs, some
independent oversight is currently being applied.
* Defense Contract Management Agency (DCMA) and Defense Contract Audit
Agency (DCAA) provide independent oversight of developers‘ activities.
Potential benefit to DOD and obstacles to implementation: Helps to
ensure quality program outcomes:
* Provides independent perspectives on program cost, schedule, and
performance risks.
Potential obstacle:
* DOD‘s quality assurance workforce, and the amount of oversight it
can provide, has decreased.
[End of Objective 2]
Objective 3 Overview:
In the Current Acquisition Environment, Some Commercially Adopted
Practices May Not Be Readily Adaptable to DOD Space System
Acquisitions:
* Exclusive use of firm, fixed-price contracts;
* Highly accurate satellite cost estimates;
* Maintenance of long-term relationships with suppliers.
Contract Type: While Commercial Firms Exclusively Use Firm, Fixed-
Price Contracts, DOD Varies Contract Types:
Commercial practice: Exclusive use of firm, fixed-price contracts:
* Commercial customers are risk averse and use firm, fixed-price
contracts to better ensure cost, schedule, and performance parameters
are well understood.
Prevailing DOD practice: Typically uses cost-reimbursement contracts:
* DOD accepts greater technology and development risks. Costs
associated with technology invention are difficult to estimate.
* Our work has found that fixed-price contracting generally has not
worked for DOD space systems due to the high level of unknowns
accepted at program start.
* Almost all current major satellite programs use cost-reimbursement
contracts for the first two satellites to be developed and produced.
Some programs use fixed-price contracts for any additional satellites.
Reasons why practice may not be readily adaptable to DOD in current
acquisition environment: Higher DOD risk levels may require multiple
contracting options:
* There is ongoing debate regarding whether firm, fixed-price
contracting could be applied to DOD space systems.
* Use of firm, fixed-price contracting for the first two satellites
would require a change in paradigm for DOD space programs”a much
higher level of knowledge, including mature technologies and mature
design, would be required prior to program start.
* Other factors, such as launch delays, funding instability, and
changing needs pose additional challenges to the use of firm, fixed-
price contracts.
Cost Estimating: Commercial Satellite Cost Estimates Are More Accurate
Than DOD Cost Estimates:
Commercial practice: Costs are estimated at an 80 to 90 percent
confidence level and accurately capture program content and risk:
* Firm, fixed-price contracting motivates developers to fully
understand costs, which in turn makes cost estimates at the 80 to 90
percent confidence level more feasible.
* Cost estimates accurately capture program content and risk because
developers minimize design changes, rely on mature technologies, and
use multiple information sources to build and cross-check estimates.
Prevailing DOD practice: Now aims to estimate costs at the 80 percent
confidence level, but significant unknowns remain about program
content and risk:
* Back-to-basics policy calls for space acquisition cost estimates at
the 80 percent confidence level. Recent legislation requires
justification of lower confidence level estimates.
* Costs for DOD space acquisitions in recent decades have been
consistently underestimated, exacerbating acquisition problems.
* Recent legislation elevates the role of DOD‘s independent cost
estimating function.
Reasons why practice may not be readily adaptable to DOD in current
acquisition environment: Significant unknowns at program initiation
make it difficult to develop more accurate cost estimates:
* DOD officials stated that the requirement to estimate at the 80
percent confidence level would render the space portfolio unaffordable
due to the significant unknowns at the time programs are initiated.
* In order to develop substantially more accurate estimates,risks
related to factors such as unique requirements and first time use of a
technology limit DOD‘s ability to develop realistic cost estimates and
would need to be retired prior to program initiation.
Supply Chain Relationships: Commercial Firms Foster Long-Term
Relationships with Suppliers While DOD Fosters Competition:
Commercial practice: Strive to maintain long-term relationships with
suppliers:
* Commercial satellite manufacturers often have long-term
relationships with suppliers.
- Some companies‘ sufficiently large and steady manufacturing volume
of satellites that are somewhat standardized enables them to provide
their subcontractors with steady business and to have more than one
supplier for a given part.
Prevailing DOD practice: DOD is impartial to long-term working
relationships with suppliers:
* Because DOD is focused on obtaining a lower cost for its space
acquisitions, DOD has sacrificed long-term relationships to start new
ones if it appears likely to lower costs.
Reasons why practice may not be readily adaptable to DOD in current
acquisition environment: Federal Acquisition Regulation (FAR)
encourages competition in the acquisition process:
* Promoting competition is one of the guiding principles of the FAR.
* While some DOD officials indicated support for longer-term
relationships with suppliers, DOD‘s efforts to promote competition,
lower satellite acquisition volume, and its custom satellite designs
may limit opportunities for long-term relationships.
[End of Objective 3]
Concluding Observations:
The commercial satellite sector delivers satellites faster than the
DOD space sector and it typically does so within estimated costs.
However, DOD and the commercial sector are seeking to develop very
different capabilities. In many cases, there is no commercial market
which DOD can turn to for innovations in space systems”it must either
assume leadership in technology invention or partner with other space
development agencies such as NASA. Moreover, the missions and
requirements DOD is pursuing, along with the need to serve a variety
of highly specialized communities, have significant implications on
the size, complexity, and risk of its space programs.
Nevertheless, given the magnitude of unanticipated cost and schedule
growth on DOD space system acquisition programs over the last decade,
there is a clear need to adopt practices that emphasize attaining
knowledge up front, minimize requirements changes late in programs,
and provide the right support and accountability for both program
managers and contractors. The commercial companies we studied were
consistent in their adoption of such approaches and their belief that
knowledge-based development has enabled them to shorten delivery
timeframes and limit cost growth. While DOD programs will continue to
have more inherent risks, DOD has recognized that its programs can
greatly benefit from adopting similar practices and has initiated
actions to do so.
Previous GAO reports and testimonies have identified potential
obstacles to making these improvements as well as areas that still
need to be addressed. We have also stressed that adopting commercial
approaches should not equate to relaxed oversight and decreased
government technical expertise as has been the case in the past.
Rather, we have recommended how DOD can make tradeoffs to reduce risks
earlier and better manage those that it does accept. DOD has generally
concurred with these recommendations and has taken measures to address
them, including changes to acquisition policies and acquisition
practices.
A list of some of our prior recommendations is provided on slides 44
and 45.
[End of section]
Backup Slides:
Prior GAO Recommendations: Actions Needed to Address Space and Weapon
Acquisition Problems:
Before undertaking new programs:
* Prioritize investments so that projects can be fully funded and it
is clear where projects stand in relation to the overall portfolio.
* Follow an evolutionary path toward meeting mission needs rather than
attempting to satisfy all needs in a single step.
* Match requirements to resources”that is, time, money, technology,
and people”before undertaking a new development effort.
* Research and define requirements before programs are started and
limit changes after they are started.
* Ensure that cost estimates are complete, accurate, and updated
regularly.
* Commit to fully fund projects before they begin.
* Ensure that critical technologies are proven to work as intended
before programs are started.
* Assign more ambitious technology development efforts to research
departments until they are ready to be added to future generations
(increments) of a product.
* Use systems engineering to close gaps between resources and
requirements before launching the development process.
During program development:
* Use quantifiable data and demonstrable knowledge to make go/no-go
decisions, covering critical facets of the program such as cost,
schedule, technology readiness, design readiness, production
readiness, and relationships with suppliers.
* Do not allow development to proceed until certain thresholds are met”
for example, a high proportion of engineering drawings completed or
production processes under statistical control.
* Empower program managers to make decisions on the direction of the
program and to resolve problems and implement solutions.
* Hold program managers accountable for their choices.
* Require program managers to stay with a project to its end.
* Hold suppliers accountable to deliver high-quality parts for their
products through such activities as regular supplier audits and
performance evaluations of quality and delivery, among other things.
* Encourage program managers to share bad news, and encourage
collaboration and communication.
[End of section]
Related GAO Products:
Space Acquisitions: DOD Faces Substantial Challenges in Developing New
Space Systems, [hyperlink, http://www.gao.gov/products/GAO-09-705T]
(Washington, D.C.: May 20, 2009).
Federal Contracting: Guidance on Award Fees Has Led to Better
Practices But is Not Consistently Applied, [hyperlink,
http://www.gao.gov/products/GAO-09-630] (Washington, D.C.: May 29,
2009).
Defense Acquisitions: Measuring the Value of DOD's Weapons Portfolio
Requires Starting With Realistic Baselines, [hyperlink,
http://www.gao.gov/products/GAO-09-543T] (Washington, D.C.: Apr. 1,
2009).
Global Positioning System: Significant Challenges in Sustaining and
Upgrading Widely Used Capabilities, [hyperlink,
http://www.gao.gov/products/GAO-09-325] (Washington, D.C.: Apr. 30,
2009).
Space Acquisitions: Uncertainties in the Evolved Expendable Launch
Vehicle Program Pose Management and Oversight Challenges, [hyperlink,
http://www.gao.gov/products/GAO-08-1039] (Washington, D.C.: Sep. 26,
2008).
Defense Acquisitions: A Knowledge Based Funding Approach Could Improve
Major Weapon System Program Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-08-619] (Washington, D.C.: Jul. 2,
2008).
Space Acquisitions: Major Space Programs Still at Risk for Cost and
Schedule Increases, [hyperlink,
http://www.gao.gov/products/GAO-08552T] (Washington, D.C.: Mar. 4,
2008).
Best Practices: Increased Focus on Requirements and Oversight Needed
to Improve DOD's Acquisition Environment and Weapon System Quality,
[hyperlink, http://www.gao.gov/products/GAO-08-294] (Washington, D.C.:
Feb. 1, 2008).
Best Practices: An Integrated Portfolio Management Approach to Weapon
System Investments Could Improve DOD's Acquisition Outcomes,
[hyperlink, http://www.gao.gov/products/GAO-07-388] (Washington, D.C.:
Mar. 30, 2007).
Space Acquisitions: DOD Needs to Take More Action to Address
Unrealistic Cost Estimates of Space Systems, [hyperlink,
http://www.gao.gov/products/GAO-07-96] (Washington, D.C.: Nov. 16,
2006).
Defense Acquisitions: Assessments of Selected Major Weapon Programs,
[hyperlink, http://www.gao.gov/products/GAO-06-391] (Washington, D.C.:
Mar. 31, 2006).
Best Practices: Better Support of Weapon System Program Managers
Needed to Improve Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-06-110] (Washington, D.C.: Nov. 30,
2005).
Defense Acquisitions: DOD Has Paid Billions in Award and Incentive
Fees Regardless of Acquisition Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-06-66] (Washington, D.C.: Dec.19,
2005).
Briefing on DOD's Report on Commercial Communications Satellite
Services Procurement Processes, [hyperlink,
http://www.gao.gov/products/GAO-05-1019R] (Washington, D.C.: Sep. 27,
2005).
Defense Acquisitions: Incentives and Pressures that Drive Problems
Affecting Satellite and Related Acquisitions, [hyperlink,
http://www.gao.gov/products/GAO-05-570R] (Washington, D.C.: Jun. 23,
2005).
Defense Acquisitions: Assessments of Selected Major Weapon Programs,
[hyperlink, http://www.gao.gov/products/GA0-05-301](Washington, D.C.:
Mar. 31, 2005).
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: Jul. 15,
2002).
Best Practices: Better Matching of Needs and Resources Will Lead to
Better Weapon System Outcomes, [hyperlink,
http://www.gao.gov/products/GA0-01-288] (Washington, D.C.: Mar. 8,
2001).
[End of section:
Table: Technology Readiness Level Descriptions:
Technology readiness level: 1. Basic principles observed and reported.
Description: Lowest level of technology readiness. Scientific research
begins to be translated into applied research and development.
Examples might include paper studies of a technology's basic
properties.
Technology readiness level: 2. Technology concept and/or application
formulated.
Description: Invention begins. Once basic principles are observed,
practical applications can be invented. The application is speculative
and there is no proof or detailed analysis to support the assumption.
Examples are still limited to paper studies.
Technology readiness level: 3. Analytical and experimental critical
function and/or characteristic proof of concept.
Description: Active research and development is initiated. This
includes analytical studies and laboratory studies to physically
validate analytical predictions of separate elements of technology.
Examples include components that are not yet integrated or
representative.
Technology readiness level: 4. Component and/or breadboard validation
in laboratory environment.
Description: Basic technological components are integrated to
establish that the pieces will work together. This is relatively "low
fidelity" compared to the eventual system. Examples include
integration of "ad hoc" hardware in a laboratory.
Technology readiness level: 5. Component and/or breadboard validation
in relevant environment.
Description: Fidelity of breadboard technology increases
significantly. The basic technological components are integrated with
reasonably realistic supporting elements so that the technology can be
tested in a simulated environment. Examples include "high fidelity"
laboratory integration of components.
Technology readiness level: 6. System/subsystem model or prototype
demonstration in a relevant environment.
Description: Representative model or prototype system, which is well
beyond the breadboard tested for TRL 5, is tested in a relevant
environment. Represents a major step up in a technology's demonstrated
readiness. Examples include testing a prototype in a high fidelity
laboratory environment or in simulated realistic environment.
Technology readiness level: 7. System prototype demonstration in a
realistic environment.
Description: Prototype near or at planned operational system.
Represents a major step up from TRL 6, requiring the demonstration of
an actual system prototype in a realistic environment, such as in an
aircraft, vehicle, or space. Examples include testing the prototype in
a test bed aircraft.
Technology readiness level: 8. Actual system completed and "flight
qualified" through test and demonstration.
Description: Technology has been proven to work in its final form and
under expected conditions. In almost all cases, this TRL represents
the end of true system development. Examples include developmental
test and evaluation of the system in its intended weapon system to
determine if it meets design specifications.
Technology readiness level: 9. Actual system "flight proven" through
successful mission operations.
Description: Actual application of the technology in its final form
and under mission conditions, such as those encountered in operational
test and evaluation. In almost all cases, this is the end of the last
"bug fixing" aspects of true system development. Examples include
using the system under operational mission conditions.
Source: GAO and GAO analysis of National Aeronautics and Space
Administration data.
[End of table]
Figure: TRLs Reveal the Knowledge Gap Regarding a Technology's Ability
to Satisfy Requirements:
[Refer to PDF for image: illustration]
The illustration indicates risk levels depicted as downward
stairsteps, number from 1 through 9 plotted against
technology/readiness level.
Knowledge gap regarding a technology's ability to satisfy requirements
within cost and schedule constraints exists with higher risk.
Steps 1 through 5 indicate higher risk if included in product
development.
Steps 6 through 9 indicate lower risk if included in product
development.
Source: GAO.
[End of figure]
Table: Commercial Satellite Industry Overview: (Dollars in billions):
Revenues from satellite manufacturing and launch activity:
World manufacturing (government & commercial customers):
2003: $9.8;
2004: $10.2;
2005: $7.8;
2006: $12.0;
2007: $11.6;
2008: $10.5.
World commercial manufacturing (% world total):
2003: $1.7; (17.3%);
2004: $1.8; (17.6%);
2005: $2.3; (29.5%);
2006: $3.0; (25.0%);
2007: $3.8; (32.8%);
2008: $5.2; (49.5%).
World commercial launch:
2003: $1.2;
2004: $1.0;
2005: $1.2;
2006: $1.4;
2007: $1.5;
2008: $2.0.
Total world commercial satellite activity:
2003: $2.9;
2004: $2.8;
2005: $3.5;
2006: $4.4;
2007: $5.3;
2008: $7.2.
U.S. manufacturing (government & commercial customers):
2003: $4.6;
2004: $3.9;
2005: $3.2;
2006: $5.0;
2007: $4.8;
2008: $3.1.
U.S. commercial manufacturing (% U.S. total):
2003: $1.2; (26.1%);
2004: $1.4; (35.9%);
2005: $1.3; (40.6%);
2006: $1.9; (38.0%);
2007: $2.2; (45.8%);
2008: $1.8; (58.1%).
U.S. commercial launch:
2003: $0.3;
2004: $0.4;
2005: $0.1;
2006: $0.1;
2007: $0.2;
2008: $0.2.
Total U.S. commercial satellite activity (% world commercial satellite
activity):
2003: $1.5; (51.7%);
2004: $1.8; (64.3%);
2005: $1.4; (40.0%);
2006: $2.0; (45.5%);
2007: $2.4; (45.3%);
2008: $2.0; (27.8%).
Sources: GAO analysis of Satellite Industry Association, Futon
Corporation, and Federal Aviation Administration data.
Note: All satellite manufacturing revenues are recognized in the year
of satellite launch, and geographically determined by location of
manufacturers' headquarters.
[End of table]
[End of briefing slides]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
