Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices

Gao ID: GAO-10-315R January 14, 2010

The Department of Defense (DOD) has had long-standing difficulties developing and delivering space systems on time and within budget. Some programs have been delayed by years and cost billions of dollars more than their initial estimates. Attempts to reform DOD space acquisitions in the past have sought to leverage commercial approaches or rely more on the commercial sector to meet DOD needs. These efforts have not been successful and, in some cases, have exacerbated problems, particularly with respect to oversight. In view of past challenges with adopting commercial approaches, Congress requested an examination of the following questions: (1) What are the differences between commercial and national security space system missions, requirements, and technology development? (2) What acquisition practices adopted by commercial companies could be used for national security space system acquisitions? (3) Which acquisition practices adopted by commercial companies may not be readily adaptable for national security space system acquisitions?

We found that although DOD and the commercial sector both use satellites for missions such as communications and imagery, DOD's requirements are often more demanding. Consequently, while the commercial sector prefers to utilize only mature technologies in satellite development, DOD satellite development typically involves the development of new technologies to meet its more stringent needs. Additionally, DOD--in mission areas such as missile warning and space surveillance--has requirements that do not exist in the commercial sector. In these areas, DOD funds technology development and acquires specific capabilities because they are not commercially available. Overall, the commercial satellite sector delivers satellites faster than the DOD space sector and it typically does so within estimated costs. In many cases, there is no commercial market that DOD can turn to for innovations in space systems--it must either assume leadership in technology invention or partner with other space development agencies such as NASA. Moreover, the missions and requirements DOD is pursuing, along with the need to serve a variety of highly specialized users, have significant implications on the size, complexity, and risk of its space programs. While commercial and DOD space system missions, requirements, and technology development differ in key ways, the commercial sector has adopted practices that could be applied to DOD space system acquisitions to improve cost, schedule, and performance outcomes. For instance, commercial firms define their requirements before initiating development programs, which helps to close resource gaps prior to program start and limit requirements growth. They tie contractor award and incentive fees to acquisition outcomes. They follow evolutionary product development approaches that enable them to achieve gradual gains in capability in relatively short periods while limiting the extent of technology risk they take on in any one increment. The commercial approach, overall, emphasizes gaining critical knowledge before making long-term commitments. GAO has already recommended these practices for DOD adoption. DOD, in fact, has recognized a need to adopt several of these practices and initiated efforts to do so. At the same time, some acquisition practices adopted by the commercial sector, including exclusive use of firm, fixed-price contracts and developing highly accurate cost estimates, may not be successfully applied to DOD in its current acquisition environment because of factors such as unique requirements and immature technologies at program start. For instance, the use of firm, fixed-price contracts for procuring satellites would require a change in paradigm for DOD space programs--a much higher level of knowledge, including mature technologies and mature design--prior to the start of a program. Currently, however, DOD accepts greater technology and development risks and typically uses cost-reimbursement contracts for the first two satellites to be developed and produced. Some programs use fixed-price contracts for any additional satellites. Using fixed-price contracts for the development phase of a program has not worked well, partly due to the high level of unknowns accepted at program start. In addition, other factors, such as launch delays, program funding instability, changing needs, and the diverse array of organizations involved in DOD space programs pose additional challenges to the use of firm, fixed-price contracts.



GAO-10-315R, Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices This is the accessible text file for GAO report number GAO-10-315R entitled 'Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices' which was released on January 14, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-10-315R: United States Government Accountability Office: Washington, DC 20548: January 14, 2009: The Honorable John Murtha: Chairman: Subcommittee on Defense: Committee on Appropriations: House of Representatives: Subject: Briefing on Commercial and Department of Defense Space System Requirements and Acquisition Practices: Dear Mr. Chairman: The Department of Defense (DOD) has had long-standing difficulties developing and delivering space systems on time and within budget. Some programs have been delayed by years and cost billions of dollars more than their initial estimates. Attempts to reform DOD space acquisitions in the past have sought to leverage commercial approaches or rely more on the commercial sector to meet DOD needs. These efforts have not been successful and, in some cases, have exacerbated problems, particularly with respect to oversight. In view of past challenges with adopting commercial approaches, you requested we examine the following questions: (1) What are the differences between commercial and national security space system missions, requirements, and technology development? (2) What acquisition practices adopted by commercial companies could be used for national security space system acquisitions? (3) Which acquisition practices adopted by commercial companies may not be readily adaptable for national security space system acquisitions? The attached briefing provides the results of our review. This letter provides a brief summary of how we conducted our work and the results of our review. Scope and Methodology: To conduct our review, we interviewed officials and reviewed and analyzed documentation on missions, requirements, and technology development from all major U.S. commercial satellite manufacturers and selected service providers, the two major space industry associations, a major space insurance broker, and from DOD--Office of the Secretary of Defense, Air Force Headquarters and Space and Missile Systems Center, and other organizations responsible for acquisition oversight, cost analysis, and program analysis of national security space programs. We interviewed officials from commercial and DOD organizations and reviewed documentation of their space acquisition practices, and compared and contrasted these practices to best practices GAO has previously reported on. Based on interviews and GAO reports on space system acquisitions and best practices, we determined whether specific commercial practices--such as requirements definition, technology maturity, contracting, and cost estimating--may or may not be readily adaptable and beneficial to national security space acquisition programs. It should be noted that the commercial companies we interviewed are not formally recognized as "best practices" companies; however, many of these practices align with best practices we have previously reported on. It should also be noted that our assessment of the applicability of space acquisition practices adopted by U.S. commercial companies is focused primarily on unclassified DOD acquisitions and may not be applicable to classified National Reconnaissance Office (NRO) acquisitions because we have not reviewed NRO systems and requirements. However, under this review, we met with and obtained perspectives on acquisition practices from NRO officials, which we incorporated as appropriate. We conducted this performance audit from November 2008 to August 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We provided a draft of the enclosed briefing to DOD officials for their review and comment. In an August 25, 2009 response, DOD generally agreed with the information presented and provided technical comments. Summary: We found that although DOD and the commercial sector both use satellites for missions such as communications and imagery, DOD's requirements are often more demanding. Consequently, while the commercial sector prefers to utilize only mature technologies in satellite development, DOD satellite development typically involves the development of new technologies to meet its more stringent needs. Additionally, DOD--in mission areas such as missile warning and space surveillance--has requirements that do not exist in the commercial sector. In these areas, DOD funds technology development and acquires specific capabilities because they are not commercially available. Overall, the commercial satellite sector delivers satellites faster than the DOD space sector and it typically does so within estimated costs. In many cases, there is no commercial market that DOD can turn to for innovations in space systems--it must either assume leadership in technology invention or partner with other space development agencies such as NASA. Moreover, the missions and requirements DOD is pursuing, along with the need to serve a variety of highly specialized users, have significant implications on the size, complexity, and risk of its space programs. While commercial and DOD space system missions, requirements, and technology development differ in key ways, the commercial sector has adopted practices that could be applied to DOD space system acquisitions to improve cost, schedule, and performance outcomes. For instance, commercial firms define their requirements before initiating development programs, which helps to close resource gaps prior to program start and limit requirements growth. They tie contractor award and incentive fees to acquisition outcomes. They follow evolutionary product development approaches that enable them to achieve gradual gains in capability in relatively short periods while limiting the extent of technology risk they take on in any one increment. The commercial approach, overall, emphasizes gaining critical knowledge before making long-term commitments. GAO has already recommended these practices for DOD adoption. DOD, in fact, has recognized a need to adopt several of these practices and initiated efforts to do so. At the same time, some acquisition practices adopted by the commercial sector, including exclusive use of firm, fixed-price contracts and developing highly accurate cost estimates, may not be successfully applied to DOD in its current acquisition environment because of factors such as unique requirements and immature technologies at program start. For instance, the use of firm, fixed-price contracts for procuring satellites would require a change in paradigm for DOD space programs--a much higher level of knowledge, including mature technologies and mature design--prior to the start of a program. Currently, however, DOD accepts greater technology and development risks and typically uses cost-reimbursement contracts for the first two satellites to be developed and produced. Some programs use fixed- price contracts for any additional satellites. Using fixed-price contracts for the development phase of a program has not worked well, partly due to the high level of unknowns accepted at program start. In addition, other factors, such as launch delays, program funding instability, changing needs, and the diverse array of organizations involved in DOD space programs pose additional challenges to the use of firm, fixed-price contracts. In our briefing, we concluded that given the magnitude of unanticipated cost and schedule growth on DOD space system acquisition programs over the last decade, there is a clear need to adopt practices that emphasize attaining knowledge up front, minimize requirements changes late in programs, and provide the right support and accountability to both program managers and contractors. The commercial companies we studied were consistent in their adoption of such approaches and the belief that knowledge-based development has enabled them to shorten delivery time frames and limit cost growth. While DOD programs have more inherent risks, DOD has recognized that its programs can greatly benefit from adopting similar practices and has initiated actions to do so. Previous GAO reports and testimonies have identified potential obstacles to making these improvements as well as areas that still need to be addressed. We have also stressed that adopting commercial approaches should not equate to relaxed oversight and decreased government technical expertise, as has been the case in the past. Rather, we have recommended how DOD can make trade-offs to reduce risks earlier and better manage those risks that it does accept. DOD has generally concurred with these recommendations and has taken measures to address them, including changes to acquisition policies and acquisition practices. Agency Comments: We provided draft copies of this letter and briefing to DOD for review and comment. DOD concurred with the content and message presented and had no written comments. We are sending copies of this letter and briefing to Department of Defense and other interested congressional committees. In addition, these documents will be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this letter. Principal contributors to this project were Arthur Gallegos, Assistant Director; Martin G. Campbell; Kristine R. Heuwinkel; Laura T. Holliday; Richard Y. Horiuchi; Sylvia Schatz; and Peter E. Zwanzig. Sincerely yours, Signed by: Cristina T. Chaplain: Director, Acquisition and Sourcing Management: Enclosures - 1: [End of section] Enclosure I: Briefing Slides: Briefing to Staff of the Subcommittee on Defense, House Appropriations Committee: August 2009: Commercial and Department of Defense Space System Requirements and Acquisition Practices: Contents: * Introduction: * Results in Brief: * Background: * Scope and Methodology: * Findings: * Concluding Observations: * Backup Slides: Introduction: The Department of Defense (DOD) has had long-standing difficulties developing and delivering space systems on time and within budget. Some programs have been delayed by years and cost billions of dollars more than their initial estimates. Attempts to reform DOD space acquisitions in the past have sought to leverage commercial approaches or rely more on the commercial sector to meet DOD needs. These efforts have not been successful and, in some cases, have exacerbated problems, particularly with respect to oversight. In view of past challenges with adopting commercial approaches, this briefing addresses the following objectives: 1. What are the differences between commercial and national security space system missions, requirements, and technology development? 2. What acquisition practices adopted by commercial companies could be used for national security space system acquisitions? 3. Which acquisition practices adopted by commercial companies may not be readily adaptable for national security space system acquisitions? Results in Brief: Although DOD and the commercial sector both use satellites for missions such as communications and imagery, DOD's requirements are often more demanding. Consequently, while the commercial sector prefers to utilize only mature technologies in satellite development, DOD satellite development typically involves the development of new technologies to meet its more stringent needs. Additionally, DOD”in mission areas such as missile warning and space surveillance”has requirements that do not exist in the commercial sector. In these areas, DOD funds technology development and acquires specific capabilities because they are not commercially available. While commercial and DOD space system missions, requirements, and technology development differ in key ways, the commercial sector has adopted practices that could be applied to DOD space system acquisitions to improve cost, schedule, and performance outcomes. Many of these are practices GAO has already recommended for DOD adoption. They emphasize gaining critical knowledge before making long-term commitments. DOD, in fact, has recognized a need to adopt several of these practices and initiated efforts to do so. At the same time, some acquisition practices adopted by the commercial sector, including exclusive use of firm, fixed-price contracts and developing highly accurate cost estimates, may not be successfully applied to DOD in its current acquisition environment because of factors such as unique requirements and immature technologies at program start. Background: Commercial Satellite Industry Overview: * Global revenues from commercial satellite activity in 2008 totaled $7.2 billion”$5.2 billion manufacturing; $2.0 billion launch. * U.S. revenues from commercial satellite activity in 2008 totaled $2.0 billion ($1.8 billion manufacturing; $0.2 billion launch). Table: Revenues from U.S. satellite manufacturing and launch activity: Total U.S. satellite manufacturing (government & commercial): 2004: $3.9 billion; 2005: $3.2 billion; 2006: $5.0 billion; 2007: $4.8 billion; 2008: $3.1 billion. U.S. commercial satellite manufacturing (% US total): 2004: $1.4 billion; (35.9%); 2005: $1.3 billion; (40.6%); 2006: $1.9 billion; (38.0%); 2007: $2.2 billion; (45.8%); 2008: $1.8 billion; (58.1%). U.S. commercial launch: 2004: $0.4 billion; 2005: $0.1 billion; 2006: $0.1 billion; 2007: $0.2 billion; 2008: $0.2 billion. Total U.S. commercial satellite activity (rows 2+3): 2004: $1.8 billion; 2005: $1.4 billion; 2006: $2.0 billion; 2007: $2.4 billion; 2008: $2.0 billion. Sources: GAO analysis of Satellite Industry Association, Futron Corporation, and Federal Aviation Administration FAA) data. Note: All satellite manufacturing revenues are recognized in the year of satellite launch, and geographically determined by location of manufacturers' headquarters. [End of table] * Primary U.S. commercial satellite manufacturers: six: - Ball Aerospace, Boeing, General Dynamics, Lockheed Martin Commercial Space Systems, Orbital Sciences, Space Systems/Loral. Table: Commercial U.S.-manufactured satellites launched per year 2004 through 2008: Year: 2004; Satellites launched: 12. Year: 2005 Satellites launched: 10. Year: 2006 Satellites launched: 12. Year: 2007 Satellites launched: 19. Year: 2008 Satellites launched: 10. Source: GAO analysis of FAA data. Note: Only satellites intended for operational use are included in this count, and not those intended solely for test, development, or scientific research. Commercial satellites are defined as those serving a commercial function or operated by a commercial entity. [End of table] * Typical cost of commercial satellites: $75 million”$300 million. * Typical commercial program length: about 2-3 years. Background: DOD Satellite Acquisition Overview: * DOD investment in major space programs in 2008 totaled $6.1 billion-- $3.2 billion for research, development, test and evaluation (RDT&E); $1.2 billion for procurement; $1.6 billion for launch (Evolved Expendable Launch Vehicle). * Primary U.S. defense satellite manufacturers: three: - Boeing, Lockheed Martin, Northrop Grumman. Table: DOD satellites launched per year, 2004 through 2008: Year: 2004; Satellites launched: 5. Year: 2005 Satellites launched: 4. Year: 2006 Satellites launched: 5. Year: 2007 Satellites launched: 6. Year: 2008 Satellites launched: 2. Source: GAO analysis of FAA data. Note: Only satellites intended for operational use are included in this count, and not those intended solely for test, development, or scientific research. DOD satellites are defined as those manufactured for DOD or any of its military services, or for service-related entities such as the National Reconnaissance Office and Defense Advanced Research Projects Agency. [End of table] * Program acquisition unit cost of current major DOD satellite acquisition programs: $216.3 million”$3.1 billion; - For satellite communications programs”Advanced Extremely High Frequency, Mobile User Objective System, and Wideband Global SATCOM” the range is $414.6 million”$2.6 billion. * Schedule from program start to first launch of current major DOD satellite acquisition programs: about 5 to 14 years. Table: DOD Investment in Major Space Programs (Fiscal year 2009 dollars in millions[A]): RDT&E: Year: 2008: $3,204.4; Year: 2009: $2,996.2; Year: 2010: $2,751.2; Year: 2011: $2,452.9; Year: 2012: $1,933.9; Year: 2013: $1,836.1. Procurement: Year: 2008: $2,859.7; Year: 2009: $4,185.0; Year: 2010: $4,369.6; Year: 2011: $3,115.2; Year: 2012: $3,276.7; Year: 2013: $2,829.2. Other[B]: Year: 2008: $22.5; Year: 2009: $17.5; Year: 2010: $18.1; Year: 2011: $10.5; Year: 2012: $12.2; Year: 2013: $0.0. Total: Year: 2008: $6,086.6; Year: 2009: $7,198.7; Year: 2010: $7,138.8; Year: 2011: $5,578.6; Year: 2012: $5,222.7; Year: 2013: $4,665.3. Source: GAO analysis of fiscal year 2009 DOD data. Note: Numbers may not add due to rounding. [A] Includes the following programs: Advanced Extremely High Frequency, Evolved Expendable Launch Vehicle, Global Broadcast Service, Naystar Global Positioning System, Global Positioning System IIIA, Mobile User Objective System, National Polar-orbiting Operational Environmental System, Space Based Infrared System High, Space Based Space Surveillance Block 10, Space Tracking and Surveillance System, and Wideband Global SATCOM. Does not include development efforts that have yet to formally initiate acquisitions, including Third Generation Infrared Surveillance, Infrared Augmentation Satellite, and Transformational Satellite Communications System. [B] Other includes military construction and acquisition operations and maintenance costs. [End of table] Some of DOD's Past Attempts to Leverage Commercial Practices Were Unsuccessful: One of DOD's attempts to leverage commercial practices was Total System Performance Responsibility (TSPR)[A], which: * Aimed to streamline DOD's acquisition process and leverage innovation and management expertise from the private sector; * Gave contractor total responsibility for the integration of a weapon system and for meeting DOD's requirements; * Reduced government oversight and shifted key decision-making responsibilities onto contractors; * Magnified problems on a number of satellite acquisition programs because it was implemented in a manner that enabled requirements creep and poor contractor performance; * For some programs, TSPR resulted in relaxed specifications and inspections of the contractor, loss of quality in the manufacturing process, and poor-quality parts that caused test failures, unexpected redesigns, and late delivery of parts. [A] See GAO-07-96 and GAO-09-325. Full citations are provided on slides 46 and 47. DOD Has Made Erroneous Assumptions in the Past About Leveraging the Commercial Sector: Wideband Global SATCOM[A]: * DOD attempted to leverage commercial demand for satellites with similar technologies, but the commercial demand did not materialize; * Initial operational capability took twice as long as planned due largely to manufacturing problems. Evolved Expendable Launch Vehicle[B]: * DOD attempted to leverage commercial launch demand but the commercial demand did not materialize; * The government had to bear most of the cost burden and total program costs nearly doubled (increased by about 96 percent) from first to latest cost baseline. [A] See GAO-07-96, GAO-05-301, and GAO-06-391. [B] See GAO-08-1039. Scope and Methodology: We interviewed officials and reviewed and analyzed documentation on missions, requirements, and technology development from all major U.S. commercial satellite manufacturers and selected service providers, the two major space industry associations, a major space insurance broker, and from DOD--Office of the Secretary of Defense, Air Force Headquarters and Space and Missile Systems Center, and other organizations responsible for acquisition oversight, cost analysis, and program analysis of national security space programs. We interviewed officials from commercial and DOD organizations and reviewed documentation of their space acquisition practices, and compared and contrasted these practices to best practices GAO has previously reported on. Based on interviews and GAO reports on space system acquisitions and best practices, we determined whether specific commercial practices ” such as requirements definition, technology maturity, contracting and cost estimating ” may or may not be readily adaptable and beneficial to national security space acquisition programs. Limitations: * Our assessment of the applicability of space acquisition practices adopted by U.S. commercial companies is focused primarily on unclassified DOD acquisitions and may not be applicable to classified National Reconnaissance Office (NRO) acquisitions because we have not reviewed NRO systems and requirements. However, under this review, we met with and obtained perspectives on acquisition practices from NRO officials, which we incorporated as appropriate. * The commercial companies we interviewed are not formally recognized as "best practices" companies; however, we identified practices that company officials told us helped their programs succeed, and many of these practices align with best practices we have previously reported on. We conducted this performance audit from November 2008 to August 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Defense Offices Visited: Air Force: * Under Secretary of the Air Force, Directorate of Space Acquisitions, Washington, DC: * Space and Missile Systems Center, Los Angeles Air Force Base, CA: * Office of the Assistant Secretary for Financial Management and Comptroller, Washington, DC: Other Defense: * Office of the Director, Program Analysis and Evaluation, Washington, DC: * Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, Washington, DC: * Office of the Secretary of Defense, Cost Analysis Improvement Group, Washington, DC: * National Reconnaissance Office, Chantilly, VA: * National Security Space Office, Washington, DC: Commercial and Other Organizations Visited: Commercial satellite manufacturers: * Ball Aerospace & Technologies Corp., Boulder, CO: * Boeing Satellite Systems International, El Segundo, CA: * Lockheed Martin Commercial Space Systems, Newtown, PA: * Orbital Sciences Corp., Dulles, VA: * Space Systems/Loral, Palo Alto, CA: Other commercial satellite industry: * DigitalGlobe, Longmont, CO: * Iridium Satellite, LLC, Bethesda, MD: * Americom Government Services, McLean, VA: * Intelsat, Washington, DC: * International Space Brokers, Rosslyn, VA: Other: * Aerospace Corp., El Segundo, CA: * Former Under Secretary of the Air Force, Denver, CO: * Futron Corporation, Bethesda, MD: * Satellite Industry Association, Washington, DC: * Space Foundation, Washington, DC: Key Documents Analyzed: * DOD acquisition policies. * DOD space program-specific documentation, such as Selected Acquisition Reports, acquisition decision memoranda, and acquisition strategies. * Report to Congress of the Independent Assessment Panel on the Organization and Management of National Security Space, July 2008. * Report of the Defense Science Board/Air Force Scientific Advisory Board Joint Task Force on Acquisition of National Security Space Programs, May 2003, and update, July 2004. * Report of the Commission to Assess United States National Security Space Management and Organization, January 2001. * GAO reports on space and nonspace acquisitions, best acquisition practices, cost estimating, program management, contracting, and personnel management. Objective 1 Overview: Commercial and DOD Missions, Requirements, and Technology Development Differ in Key Ways: * Missions and Requirements; * Technology Development and Maturity. Missions and Requirements: DOD Satellite Missions and Requirements Differ from Those of the Commercial Sector: DOD space system acquisitions meet warfighter and intelligence community requirements, while commercial space system acquisitions meet market demands. In some areas, DOD and commercial needs are similar: * Communications; * Imagery. In some areas, DOD and commercial needs converge, but the commercial sector relies on government satellites: * Position, navigation, and timing; * Weather, climate, and environmental monitoring. In other areas, DOD has needs that do not apply to the commercial sector: * Missile warning; * Space surveillance; * Nuclear detonation detection. DOD Uses Commercially Available Satellite Services to Fulfill Some Missions: When DOD needs have aligned with commercially available satellite products, DOD has purchased services from commercial providers: * Communications: - DOD uses commercial fixed satellite services to augment its own satellite communications capabilities; - Nevertheless, DOD in some cases tailors these services to meet its unique needs. For example, DOD established a government gateway, which includes ground-based systems that, among other things, enable users to directly access DOD-specific communications systems. * Imagery: - DOD is the largest purchaser of U.S. commercial satellite imagery; - DOD helped fund the development of commercial imagery satellites and purchases satellite imagery from DigitalGlobe and GeoEye. DOD Requirements Are Often More Stringent Than Those of the Commercial Sector: For mission areas that apply to both DOD and the commercial sector, such as communications and imagery, DOD's requirements often call for the delivery of more robust capabilities. DOD Communications Satellite Requirements Frequently Differ from Those of the Commercial Sector: Coverage and availability: DOD typically has more demanding requirements for coverage and availability, such as for communications using radio frequencies resistant to atmospheric scintillation in polar regions and under challenging conditions such as in dense foliage and adverse weather. Survivability: DOD typically has more demanding standards for radiation hardened parts, such as microelectronics, which are: * designed and fabricated with the specific goal of enduring the harshest space radiation environments, including nuclear events; * time consuming to obtain”companies typically need to create separate production lines and in some cases special facilities; only small volumes are typically produced. Jam resistance: DOD has demanding requirements for jam resistance, addressed via: * frequency hopping in which the transmission codes "hop" across the bandwidth at a very rapid rate; * nulling antennas that cancel out the impact of jamming emitters designed to disrupt U.S. warfighter communications. Secure communications: DOD has requirements for sophisticated encryption to enable communications at varying security levels. DOD Uses Higher Resolution Imagery Than the Commercial Sector: U.S. commercial companies are allowed to sell imagery with a maximum of 50 centimeters resolution in black-and-white mode and about 2 meters in color to non-U.S. government entities. GeoEye can provide optical imagery of 41 centimeters black-and-white resolution to DOD and other agencies in the intelligence community. U.S. government-developed surveillance satellites yield even higher resolution. Many DOD Requirements Do Not Align with Commercial Needs, Call for Technology Invention: Many DOD space system acquisitions are designed to meet warfighter and intelligence community requirements that do not align with commercial needs and often require leading edge technologies. For example: * Missile warning and defense: - highly sensitive infrared sensors to detect missile heat signatures; - cryocoolers that cool focal plane arrays to cryogenic temperatures, in some cases down to 10 degrees Kelvin, create minimal vibration, and have sufficiently long lifespans. * Position, navigation, and timing: - several synchronized atomic clocks per satellite to enable accurate time signal triangulation; - advanced anti-jam capabilities to improve system security, accuracy, and reliability. * Weather, climate, and environmental monitoring: - advanced technology microwave sounder to produce daily global atmospheric temperature, humidity, and pressure profiles; - cross-tracked infrared sounder to collect measurements of the earth's radiation to determine vertical distribution of temperature, moisture, and pressure in the atmosphere; - visible infrared imager radiometer suite to collect images and radiometric data used to provide data on the earth's clouds, atmosphere, ocean, and land surfaces. * Nuclear detonation detection: - technologies to measure neutron time-of-flight spectrum, prompt gamma rays from a nuclear detonation, and delayed gamma rays from a nuclear detonation debris cloud. DOD Frequently Attempts to Meet the Needs of Multiple Customers: DOD often attempts to meet the needs of multiple users on one satellite, resulting in more missions and/or payloads per satellite, for example: * The National Polar-orbiting Operational Environmental Satellite System (NPOESS) is managed by a tri-agency program office, representing the DOD/United States Air Force, the Department of Commerce's National Oceanic and Atmospheric Administration, and the National Aeronautics and Space Administration. Its original plans called for 13 instruments, including 10 environmental sensors and 3 subsystems to meet the needs of weather forecasters, climatologists, and the military. * The Space Based Infrared System (SBIRS) is intended to meet requirements of various parts of DOD and the intelligence community for multiple missions, including missile warning, missile defense, technical intelligence, and battlespace awareness. Commercial companies frequently attempt to meet the needs of one customer per satellite, often resulting in single-mission satellites. Technology Development And Maturity: Companies That Purchase Commercial Satellites Prefer Mature Technologies: Companies prefer mature technologies, ideally at technology readiness level (TRL) 9 (see slide 48 for definitions of TRLs), so that risks, costs, and schedules are known at program start. Commercial companies sometimes conduct technology development. When they do, development is typically incremental and is conducted prior to program start. In addition, more time is allotted to manufacture satellites that incorporate a new technology due to the need for additional integration work. Although companies prefer to use mature technologies for commercial satellites, they sometimes employ fairly sophisticated technologies, such as: * Large unfurlable antenna reflectors to reduce satellite launch volume; * Lithium-ion batteries for reduced satellite weight; * Ion propulsion for reduced satellite weight; * Phased array antennas for agile coverage. DOD Funds Technology Development When Needed Technologies Are Not Commercially Available: To meet requirements that are unique to DOD, the government has supported development of many leading-edge technologies (such as those shown on slide 24). For these technologies, the industrial base is supported in large part by DOD, the intelligence community, and NASA. However, rather than maturing technologies in a robust science and technology (S&T) environment prior to program start, DOD has frequently allowed immature technologies into space programs. For example, we have identified several DOD satellite programs in which multiple technologies were below TRL 6 at program start, such as Advanced Extremely High Frequency, NPOESS, SBIRS High, and SBIRS Low. Development Risk Increases with the Number of New Technologies in a Program: According to a 2007 National Research Council Space Studies Board report: * Cost growth is closely related to development risk, which increases nonlinearly with the number of new technologies. As more immature technologies are included in a program, the likelihood of cost and schedule problems increases substantially. [End of Objective 1] Objective 2 Overview: Some Commercially Adopted Practices Could Be Applied to DOD Space System Acquisitions: * Mature critical technologies prior to program start; * Use evolutionary product development; * Define requirements to close resource gaps prior to program start and limit requirements growth; * Tie contractor award/incentive fees to acquisition outcomes; * Empower program managers and hold them accountable; * Obtain independent oversight. Ensure Technologies Are Mature Prior to Beginning an Acquisition Program: Commercial practice: Use only mature technologies: * Companies typically look to the government to push and prove technologies first. * Technologies included are typically at TRL 9, in order to foster program stability, ensure reliability, and to obtain favorable insurance rates. * When technology discovery is conducted, it is done so prior to system development. Prevailing DOD practice: Historically, has used immature technologies: * Unique nature of missions require technology invention/discovery, but it is frequently not finished prior to system development. * Many technologies below TRL 6 at program start. * DOD acquisition policies and congressional legislation reflect preference for maturing technologies prior to program start. * Recent efforts (e.g., GPS IIIA and Operationally Responsive Space efforts) have demonstrated a change to the practice. Potential benefit to DOD and obstacles to implementation: Reduce cost and schedule inefficiencies: * Achieving a high level of technology maturity prior to program initiation helps (1) ensure resources and requirements match, and (2) avoid concurrently developing technologies, finalizing designs, and demonstrating manufacturing processes, which can lead to cost and schedule inefficiencies. Potential obstacle: * There is a long-standing disconnect between the research laboratories and acquisition programs; DOD lacks an S&T strategy for space; and the funding process favors acquisitions over S&T programs. Unless Revolutionary Technologies Are Required, Use Evolutionary Product Development: Commercial practice: Development is evolutionary: * To achieve stability, reduce risk, and enable short program schedules: (1) new design elements and new parts are minimized, and (2) standardized designs and parts are tailored as needed for customers. This enables companies to focus attention on critical design, development, and integration. * Design elements not achievable in the initial development are planned for future generations of the product, which allows time for technologies to mature. Prevailing DOD practice: Historically, has promised revolutionary advances in capabilities: * In some cases, revolutionary advances in capabilities may be sought, such as for first-time or one-of-a-kind efforts to satisfy a new urgent requirement. However, most programs have attempted to satisfy all requirements in a single step, regardless of design or technology challenges. * DOD frequently adopts extensive new designs and custom-made spacecraft buses and payloads to meet the needs of multiple users. * Recent efforts have adopted a more evolutionary strategy. Potential benefit to DOD and obstacles to implementation: Offers an initial product quickly and at lower cost while technologies are matured for the next increment: * While the user may not initially receive the ultimate capability under this approach, the initial product is available sooner and at a lower, more predictable cost. * Exceptions would involve efforts, such as the first GPS, that introduce a new capability or programs focused on countering new threats. Potential obstacle: * Competition for funding incentivizes programs to promise revolutionary advances based on optimistic assumptions. Define Requirements to Close Resource Gaps Prior to Program Start and Limit Requirements Growth: Commercial practice: Ensure requirements are well defined prior to program start and remain stable: * Requirements are well defined prior to program start so that costs and feasibility are understood and trade-offs can be made if needed. * Requirements are negotiated during the contract proposal process to align with the developer‘s capabilities and strengths. The percent ’new“ in the design may range from 5-20 percent”for those new aspects, robust systems engineering is applied prior to program start to minimize unknowns. Afterward, systems engineering is focused on integrating mature technologies onto a platform. Prevailing DOD practice: Requirements are typically not well defined by program start but largely remain stable: * Poorly defined requirements have had significant consequences for funding, time, and technology development. While in the past, some programs experienced requirements creep resulting in large cost and schedule increases, DOD has made improvements in this area. * Systems engineering is conducted after programs have been funded and launched”too late to identify resource gaps, shape requirements, and inform estimates. Early focus is on defining requirements and maturing new technologies. Potential benefit to DOD and obstacles to implementation: Helps reduce program cost, schedule, and performance risks: * Early systems engineering knowledge helps identify and address gaps, such as overly optimistic requirements that cannot be met with current resources. Potential obstacles: * DOD lacks a robust systems engineering functionality. * Agreement on requirements for space systems is difficult because a diverse array of organizations are involved in setting requirements. Once agreement is achieved, it is difficult to change requirements. Tie Contractor Award and Incentive Fees to Acquisition Outcomes: Commercial practice: Incentives and penalties that emphasize on-time delivery and on-orbit performance motivate satellite developers: * Satellite customers typically tie about 10 to 20 percent of the contract value to successful on-orbit performance of the satellite over its expected life, which is frequently 15 years. * This performance-based payment is key to developers‘ profitability and is reduced or eliminated accordingly if there are on-orbit problems. Prevailing DOD practice: DOD typically uses award and incentive fee provisions in its contracts and has withheld fees for poor performance: * DOD‘s guidance states that award fees must be linked to desired outcomes and prohibits payment of award fees to contractors for unsatisfactory performance. * Almost all current major space acquisitions use award and incentive fee provisions in contracts for development of initial satellites. We reported that DOD does not consistently evaluate contractors based on award-fee criteria related to key acquisition outcomes.[A] Potential benefit to DOD and obstacles to implementation: If aligned with acquisition outcomes, award and incentive fees might motivate good contractor performance: * We recently reported that DOD has achieved savings on some programs by limiting the opportunities for earning unearned fees in subsequent periods and tying award fee criteria to acquisition outcomes.[A] Potential obstacle: * Because DOD has not developed methods to evaluate the effectiveness of award fees, it is unaware of whether these contracts are being used effectively, poor practices go unnoticed, and positive practices are isolated. [A] See GAO-06-66 and GAO-09-630. Empower Program Managers to Execute Their Programs and Hold Them Accountable for Outcomes: Commercial practice: Give program managers decision-making authority and hold them accountable for acquisition outcomes: * Program managers are given direct responsibility for the direction, planning, assessment, and resource control of their programs, which are fully funded at outset. * Program managers are held accountable for their decisions and their performance evaluation is based on how well they meet cost, schedule, and performance elements. * Program managers are not held accountable for matters beyond their control. Prevailing DOD practice: Program managers lack strong authority and are generally not held accountable for executing programs within targets: * Program managers: have little control over funding stability of incrementally funded programs and shifting funds within programs; cannot veto new program requirements, which may overly stretch their programs; and have little authority over staffing. * Because there are so many aspects of programs outside the program manager‘s control, DOD is unable to hold them accountable. * In the past, DOD program managers had more authority. Potential benefit to DOD and obstacles to implementation: Improve performance, cost, and schedule outcomes: * Empowers program managers and holds them accountable for delivering new products when needed within quality, cost, and performance targets. Potential obstacle: * Measures to empower program managers and hold them accountable will not be as effective as they could until DOD ensures that acquisition programs are executable, i.e., the needs can best be met with the chosen concept and the concept can be developed and produced within existing resources. Obtain Independent Oversight of Satellite Developers: Commercial practice: Insurers provide independent oversight of satellite developers: * The insurance industry plays a significant role in overseeing satellite acquisitions and ensuring quality. * Insurance underwriters consider factors such as favorable on-orbit performance records and use of reliable and flight-proven technology when determining insurance terms. * In order for customers to obtain favorable insurance terms, developers adhere to strict quality standards. Prevailing DOD practice: Some independent oversight is applied: * While DOD does not obtain insurance for its satellite programs, some independent oversight is currently being applied. * Defense Contract Management Agency (DCMA) and Defense Contract Audit Agency (DCAA) provide independent oversight of developers‘ activities. Potential benefit to DOD and obstacles to implementation: Helps to ensure quality program outcomes: * Provides independent perspectives on program cost, schedule, and performance risks. Potential obstacle: * DOD‘s quality assurance workforce, and the amount of oversight it can provide, has decreased. [End of Objective 2] Objective 3 Overview: In the Current Acquisition Environment, Some Commercially Adopted Practices May Not Be Readily Adaptable to DOD Space System Acquisitions: * Exclusive use of firm, fixed-price contracts; * Highly accurate satellite cost estimates; * Maintenance of long-term relationships with suppliers. Contract Type: While Commercial Firms Exclusively Use Firm, Fixed- Price Contracts, DOD Varies Contract Types: Commercial practice: Exclusive use of firm, fixed-price contracts: * Commercial customers are risk averse and use firm, fixed-price contracts to better ensure cost, schedule, and performance parameters are well understood. Prevailing DOD practice: Typically uses cost-reimbursement contracts: * DOD accepts greater technology and development risks. Costs associated with technology invention are difficult to estimate. * Our work has found that fixed-price contracting generally has not worked for DOD space systems due to the high level of unknowns accepted at program start. * Almost all current major satellite programs use cost-reimbursement contracts for the first two satellites to be developed and produced. Some programs use fixed-price contracts for any additional satellites. Reasons why practice may not be readily adaptable to DOD in current acquisition environment: Higher DOD risk levels may require multiple contracting options: * There is ongoing debate regarding whether firm, fixed-price contracting could be applied to DOD space systems. * Use of firm, fixed-price contracting for the first two satellites would require a change in paradigm for DOD space programs”a much higher level of knowledge, including mature technologies and mature design, would be required prior to program start. * Other factors, such as launch delays, funding instability, and changing needs pose additional challenges to the use of firm, fixed- price contracts. Cost Estimating: Commercial Satellite Cost Estimates Are More Accurate Than DOD Cost Estimates: Commercial practice: Costs are estimated at an 80 to 90 percent confidence level and accurately capture program content and risk: * Firm, fixed-price contracting motivates developers to fully understand costs, which in turn makes cost estimates at the 80 to 90 percent confidence level more feasible. * Cost estimates accurately capture program content and risk because developers minimize design changes, rely on mature technologies, and use multiple information sources to build and cross-check estimates. Prevailing DOD practice: Now aims to estimate costs at the 80 percent confidence level, but significant unknowns remain about program content and risk: * Back-to-basics policy calls for space acquisition cost estimates at the 80 percent confidence level. Recent legislation requires justification of lower confidence level estimates. * Costs for DOD space acquisitions in recent decades have been consistently underestimated, exacerbating acquisition problems. * Recent legislation elevates the role of DOD‘s independent cost estimating function. Reasons why practice may not be readily adaptable to DOD in current acquisition environment: Significant unknowns at program initiation make it difficult to develop more accurate cost estimates: * DOD officials stated that the requirement to estimate at the 80 percent confidence level would render the space portfolio unaffordable due to the significant unknowns at the time programs are initiated. * In order to develop substantially more accurate estimates,risks related to factors such as unique requirements and first time use of a technology limit DOD‘s ability to develop realistic cost estimates and would need to be retired prior to program initiation. Supply Chain Relationships: Commercial Firms Foster Long-Term Relationships with Suppliers While DOD Fosters Competition: Commercial practice: Strive to maintain long-term relationships with suppliers: * Commercial satellite manufacturers often have long-term relationships with suppliers. - Some companies‘ sufficiently large and steady manufacturing volume of satellites that are somewhat standardized enables them to provide their subcontractors with steady business and to have more than one supplier for a given part. Prevailing DOD practice: DOD is impartial to long-term working relationships with suppliers: * Because DOD is focused on obtaining a lower cost for its space acquisitions, DOD has sacrificed long-term relationships to start new ones if it appears likely to lower costs. Reasons why practice may not be readily adaptable to DOD in current acquisition environment: Federal Acquisition Regulation (FAR) encourages competition in the acquisition process: * Promoting competition is one of the guiding principles of the FAR. * While some DOD officials indicated support for longer-term relationships with suppliers, DOD‘s efforts to promote competition, lower satellite acquisition volume, and its custom satellite designs may limit opportunities for long-term relationships. [End of Objective 3] Concluding Observations: The commercial satellite sector delivers satellites faster than the DOD space sector and it typically does so within estimated costs. However, DOD and the commercial sector are seeking to develop very different capabilities. In many cases, there is no commercial market which DOD can turn to for innovations in space systems”it must either assume leadership in technology invention or partner with other space development agencies such as NASA. Moreover, the missions and requirements DOD is pursuing, along with the need to serve a variety of highly specialized communities, have significant implications on the size, complexity, and risk of its space programs. Nevertheless, given the magnitude of unanticipated cost and schedule growth on DOD space system acquisition programs over the last decade, there is a clear need to adopt practices that emphasize attaining knowledge up front, minimize requirements changes late in programs, and provide the right support and accountability for both program managers and contractors. The commercial companies we studied were consistent in their adoption of such approaches and their belief that knowledge-based development has enabled them to shorten delivery timeframes and limit cost growth. While DOD programs will continue to have more inherent risks, DOD has recognized that its programs can greatly benefit from adopting similar practices and has initiated actions to do so. Previous GAO reports and testimonies have identified potential obstacles to making these improvements as well as areas that still need to be addressed. We have also stressed that adopting commercial approaches should not equate to relaxed oversight and decreased government technical expertise as has been the case in the past. Rather, we have recommended how DOD can make tradeoffs to reduce risks earlier and better manage those that it does accept. DOD has generally concurred with these recommendations and has taken measures to address them, including changes to acquisition policies and acquisition practices. A list of some of our prior recommendations is provided on slides 44 and 45. [End of section] Backup Slides: Prior GAO Recommendations: Actions Needed to Address Space and Weapon Acquisition Problems: Before undertaking new programs: * Prioritize investments so that projects can be fully funded and it is clear where projects stand in relation to the overall portfolio. * Follow an evolutionary path toward meeting mission needs rather than attempting to satisfy all needs in a single step. * Match requirements to resources”that is, time, money, technology, and people”before undertaking a new development effort. * Research and define requirements before programs are started and limit changes after they are started. * Ensure that cost estimates are complete, accurate, and updated regularly. * Commit to fully fund projects before they begin. * Ensure that critical technologies are proven to work as intended before programs are started. * Assign more ambitious technology development efforts to research departments until they are ready to be added to future generations (increments) of a product. * Use systems engineering to close gaps between resources and requirements before launching the development process. During program development: * Use quantifiable data and demonstrable knowledge to make go/no-go decisions, covering critical facets of the program such as cost, schedule, technology readiness, design readiness, production readiness, and relationships with suppliers. * Do not allow development to proceed until certain thresholds are met” for example, a high proportion of engineering drawings completed or production processes under statistical control. * Empower program managers to make decisions on the direction of the program and to resolve problems and implement solutions. * Hold program managers accountable for their choices. * Require program managers to stay with a project to its end. * Hold suppliers accountable to deliver high-quality parts for their products through such activities as regular supplier audits and performance evaluations of quality and delivery, among other things. * Encourage program managers to share bad news, and encourage collaboration and communication. [End of section] Related GAO Products: Space Acquisitions: DOD Faces Substantial Challenges in Developing New Space Systems, [hyperlink, http://www.gao.gov/products/GAO-09-705T] (Washington, D.C.: May 20, 2009). Federal Contracting: Guidance on Award Fees Has Led to Better Practices But is Not Consistently Applied, [hyperlink, http://www.gao.gov/products/GAO-09-630] (Washington, D.C.: May 29, 2009). Defense Acquisitions: Measuring the Value of DOD's Weapons Portfolio Requires Starting With Realistic Baselines, [hyperlink, http://www.gao.gov/products/GAO-09-543T] (Washington, D.C.: Apr. 1, 2009). Global Positioning System: Significant Challenges in Sustaining and Upgrading Widely Used Capabilities, [hyperlink, http://www.gao.gov/products/GAO-09-325] (Washington, D.C.: Apr. 30, 2009). Space Acquisitions: Uncertainties in the Evolved Expendable Launch Vehicle Program Pose Management and Oversight Challenges, [hyperlink, http://www.gao.gov/products/GAO-08-1039] (Washington, D.C.: Sep. 26, 2008). Defense Acquisitions: A Knowledge Based Funding Approach Could Improve Major Weapon System Program Outcomes, [hyperlink, http://www.gao.gov/products/GAO-08-619] (Washington, D.C.: Jul. 2, 2008). Space Acquisitions: Major Space Programs Still at Risk for Cost and Schedule Increases, [hyperlink, http://www.gao.gov/products/GAO-08552T] (Washington, D.C.: Mar. 4, 2008). Best Practices: Increased Focus on Requirements and Oversight Needed to Improve DOD's Acquisition Environment and Weapon System Quality, [hyperlink, http://www.gao.gov/products/GAO-08-294] (Washington, D.C.: Feb. 1, 2008). Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD's Acquisition Outcomes, [hyperlink, http://www.gao.gov/products/GAO-07-388] (Washington, D.C.: Mar. 30, 2007). Space Acquisitions: DOD Needs to Take More Action to Address Unrealistic Cost Estimates of Space Systems, [hyperlink, http://www.gao.gov/products/GAO-07-96] (Washington, D.C.: Nov. 16, 2006). Defense Acquisitions: Assessments of Selected Major Weapon Programs, [hyperlink, http://www.gao.gov/products/GAO-06-391] (Washington, D.C.: Mar. 31, 2006). Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes, [hyperlink, http://www.gao.gov/products/GAO-06-110] (Washington, D.C.: Nov. 30, 2005). Defense Acquisitions: DOD Has Paid Billions in Award and Incentive Fees Regardless of Acquisition Outcomes, [hyperlink, http://www.gao.gov/products/GAO-06-66] (Washington, D.C.: Dec.19, 2005). Briefing on DOD's Report on Commercial Communications Satellite Services Procurement Processes, [hyperlink, http://www.gao.gov/products/GAO-05-1019R] (Washington, D.C.: Sep. 27, 2005). Defense Acquisitions: Incentives and Pressures that Drive Problems Affecting Satellite and Related Acquisitions, [hyperlink, http://www.gao.gov/products/GAO-05-570R] (Washington, D.C.: Jun. 23, 2005). Defense Acquisitions: Assessments of Selected Major Weapon Programs, [hyperlink, http://www.gao.gov/products/GA0-05-301](Washington, D.C.: Mar. 31, 2005). Best Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition Outcomes, [hyperlink, http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: Jul. 15, 2002). Best Practices: Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, [hyperlink, http://www.gao.gov/products/GA0-01-288] (Washington, D.C.: Mar. 8, 2001). [End of section: Table: Technology Readiness Level Descriptions: Technology readiness level: 1. Basic principles observed and reported. Description: Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology's basic properties. Technology readiness level: 2. Technology concept and/or application formulated. Description: Invention begins. Once basic principles are observed, practical applications can be invented. The application is speculative and there is no proof or detailed analysis to support the assumption. Examples are still limited to paper studies. Technology readiness level: 3. Analytical and experimental critical function and/or characteristic proof of concept. Description: Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of technology. Examples include components that are not yet integrated or representative. Technology readiness level: 4. Component and/or breadboard validation in laboratory environment. Description: Basic technological components are integrated to establish that the pieces will work together. This is relatively "low fidelity" compared to the eventual system. Examples include integration of "ad hoc" hardware in a laboratory. Technology readiness level: 5. Component and/or breadboard validation in relevant environment. Description: Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so that the technology can be tested in a simulated environment. Examples include "high fidelity" laboratory integration of components. Technology readiness level: 6. System/subsystem model or prototype demonstration in a relevant environment. Description: Representative model or prototype system, which is well beyond the breadboard tested for TRL 5, is tested in a relevant environment. Represents a major step up in a technology's demonstrated readiness. Examples include testing a prototype in a high fidelity laboratory environment or in simulated realistic environment. Technology readiness level: 7. System prototype demonstration in a realistic environment. Description: Prototype near or at planned operational system. Represents a major step up from TRL 6, requiring the demonstration of an actual system prototype in a realistic environment, such as in an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft. Technology readiness level: 8. Actual system completed and "flight qualified" through test and demonstration. Description: Technology has been proven to work in its final form and under expected conditions. In almost all cases, this TRL represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications. Technology readiness level: 9. Actual system "flight proven" through successful mission operations. Description: Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. In almost all cases, this is the end of the last "bug fixing" aspects of true system development. Examples include using the system under operational mission conditions. Source: GAO and GAO analysis of National Aeronautics and Space Administration data. [End of table] Figure: TRLs Reveal the Knowledge Gap Regarding a Technology's Ability to Satisfy Requirements: [Refer to PDF for image: illustration] The illustration indicates risk levels depicted as downward stairsteps, number from 1 through 9 plotted against technology/readiness level. Knowledge gap regarding a technology's ability to satisfy requirements within cost and schedule constraints exists with higher risk. Steps 1 through 5 indicate higher risk if included in product development. Steps 6 through 9 indicate lower risk if included in product development. Source: GAO. [End of figure] Table: Commercial Satellite Industry Overview: (Dollars in billions): Revenues from satellite manufacturing and launch activity: World manufacturing (government & commercial customers): 2003: $9.8; 2004: $10.2; 2005: $7.8; 2006: $12.0; 2007: $11.6; 2008: $10.5. World commercial manufacturing (% world total): 2003: $1.7; (17.3%); 2004: $1.8; (17.6%); 2005: $2.3; (29.5%); 2006: $3.0; (25.0%); 2007: $3.8; (32.8%); 2008: $5.2; (49.5%). World commercial launch: 2003: $1.2; 2004: $1.0; 2005: $1.2; 2006: $1.4; 2007: $1.5; 2008: $2.0. Total world commercial satellite activity: 2003: $2.9; 2004: $2.8; 2005: $3.5; 2006: $4.4; 2007: $5.3; 2008: $7.2. U.S. manufacturing (government & commercial customers): 2003: $4.6; 2004: $3.9; 2005: $3.2; 2006: $5.0; 2007: $4.8; 2008: $3.1. U.S. commercial manufacturing (% U.S. total): 2003: $1.2; (26.1%); 2004: $1.4; (35.9%); 2005: $1.3; (40.6%); 2006: $1.9; (38.0%); 2007: $2.2; (45.8%); 2008: $1.8; (58.1%). U.S. commercial launch: 2003: $0.3; 2004: $0.4; 2005: $0.1; 2006: $0.1; 2007: $0.2; 2008: $0.2. Total U.S. commercial satellite activity (% world commercial satellite activity): 2003: $1.5; (51.7%); 2004: $1.8; (64.3%); 2005: $1.4; (40.0%); 2006: $2.0; (45.5%); 2007: $2.4; (45.3%); 2008: $2.0; (27.8%). Sources: GAO analysis of Satellite Industry Association, Futon Corporation, and Federal Aviation Administration data. Note: All satellite manufacturing revenues are recognized in the year of satellite launch, and geographically determined by location of manufacturers' headquarters. [End of table] [End of briefing slides] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.