Geostationary Operational Environmental Satellites

Acquisition Is Under Way, but Improvements Needed in Management and Oversight Gao ID: GAO-09-323 April 2, 2009

The Department of Commerce's National Oceanic and Atmospheric Administration (NOAA), with the aid of the National Aeronautics and Space Administration (NASA), plans to procure the next generation of geostationary operational environmental satellites, called the Geostationary Operational Environmental Satellite-R series (GOES-R). GOES-R is to replace the current series of satellites, which will likely begin to reach the end of their useful lives in approximately 2014. This series is considered critical to the United States' ability to maintain the continuity of data required for weather forecasting through the year 2028. GAO was asked to (1) determine the status of the GOES-R program, (2) evaluate whether plans for the acquisition address problems experienced on similar programs, and (3) determine whether NOAA's plan will be adequate to support current data requirements. To do so, GAO analyzed contractor and program data and interviewed officials from NOAA and NASA.

NOAA has made progress on the GOES-R acquisition, but the program's cost, schedule, and scope have changed. The GOES-R program has moved into the development phase of its acquisition life cycle. It has awarded development contracts for the instruments and plans to award contracts for the spacecraft and ground segments by mid-2009. However, after reconciling program and independent cost estimates, the program established a new cost estimate of $7.67 billion--a $670 million increase from the prior $7 billion estimate. The program also reduced the number of products the satellites will produce from 81 to 34 and slowed the delivery of these products in order to reduce costs. More recently, the program also delayed key milestones, including the launch of the first satellite, which was delayed from December 2014 to April 2015. Such delays could lead to gaps in satellite coverage if NOAA experiences problems with its current operational satellites before a backup satellite is in orbit. GOES-R has taken steps to address lessons from other satellite programs, but important actions remain to be completed. NOAA has made progress in its efforts to address prior lessons by taking steps to ensure technical readiness on key components, using an acceptable cost estimating approach, implementing techniques to enhance contractor oversight, and regularly briefing agency executives. However, technical challenges remain on both the ground segment and the instruments. In addition, the program did not perform a comprehensive review after rebaselining a critical instrument, and it has not documented all of the reasons for cost overruns. Until these issues are addressed, NOAA faces an increased risk that the GOES-Rprogram will repeat the same mistakes that have plagued other satellite programs. NOAA has a plan to meet some, but not all, data requirements. An instrument that was originally planned as part of the GOES-R satellite was to meet requirements for 15 products that are currently produced, as well as 11 new, technically advanced, products. When NOAA removed this instrument from the GOES-R satellite program, it arranged to obtain the current products from another instrument. However, the agency has not developed plans or a timeline to address the requirements for the new products. Doing so would include justifying the funding for any new initiatives within the agency's investment decision process. Until a decision is made on whether and how to proceed in providing the advanced products, key system users, such as weather forecasters, will not be able to meet their goals for improving the accuracy of severe weather warnings. Further, climate research organizations will not obtain the data they need to enhance the science of climate, environmental, and oceanic observations.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-09-323, Geostationary Operational Environmental Satellites: Acquisition Is Under Way, but Improvements Needed in Management and Oversight This is the accessible text file for GAO report number GAO-09-323 entitled 'Geostationary Operational Environmental Satellites: Acquisition Is Under Way, but Improvements Needed in Management and Oversight' which was released on April 23, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: April 2009: Geostationary Operational Environmental Satellites: Acquisition Is Under Way, but Improvements Needed in Management and Oversight: GAO-09-323: GAO Highlights: Highlights of GAO-09-323, a report to congressional requesters. Why GAO Did This Study: The Department of Commerce‘s National Oceanic and Atmospheric Administration (NOAA), with the aid of the National Aeronautics and Space Administration (NASA), plans to procure the next generation of geostationary operational environmental satellites, called the Geostationary Operational Environmental Satellite-R series (GOES-R). GOES-R is to replace the current series of satellites, which will likely begin to reach the end of their useful lives in approximately 2014. This series is considered critical to the United States‘ ability to maintain the continuity of data required for weather forecasting through the year 2028. GAO was asked to (1) determine the status of the GOES-R program, (2) evaluate whether plans for the acquisition address problems experienced on similar programs, and (3) determine whether NOAA‘s plan will be adequate to support current data requirements. To do so, GAO analyzed contractor and program data and interviewed officials from NOAA and NASA. What GAO Found: NOAA has made progress on the GOES-R acquisition, but the program‘s cost, schedule, and scope have changed. The GOES-R program has moved into the development phase of its acquisition life cycle. It has awarded development contracts for the instruments and plans to award contracts for the spacecraft and ground segments by mid-2009. However, after reconciling program and independent cost estimates, the program established a new cost estimate of $7.67 billion”a $670 million increase from the prior $7 billion estimate. The program also reduced the number of products the satellites will produce from 81 to 34 and slowed the delivery of these products in order to reduce costs. More recently, the program also delayed key milestones, including the launch of the first satellite, which was delayed from December 2014 to April 2015. Such delays could lead to gaps in satellite coverage if NOAA experiences problems with its current operational satellites before a backup satellite is in orbit. GOES-R has taken steps to address lessons from other satellite programs, but important actions remain to be completed. NOAA has made progress in its efforts to address prior lessons by taking steps to ensure technical readiness on key components, using an acceptable cost estimating approach, implementing techniques to enhance contractor oversight, and regularly briefing agency executives. However, technical challenges remain on both the ground segment and the instruments. In addition, the program did not perform a comprehensive review after rebaselining a critical instrument, and it has not documented all of the reasons for cost overruns. Until these issues are addressed, NOAA faces an increased risk that the GOES-R program will repeat the same mistakes that have plagued other satellite programs. NOAA has a plan to meet some, but not all, data requirements. An instrument that was originally planned as part of the GOES-R satellite was to meet requirements for 15 products that are currently produced, as well as 11 new, technically advanced, products. When NOAA removed this instrument from the GOES-R satellite program, it arranged to obtain the current products from another instrument. However, the agency has not developed plans or a timeline to address the requirements for the new products. Doing so would include justifying the funding for any new initiatives within the agency‘s investment decision process. Until a decision is made on whether and how to proceed in providing the advanced products, key system users, such as weather forecasters, will not be able to meet their goals for improving the accuracy of severe weather warnings. Further, climate research organizations will not obtain the data they need to enhance the science of climate, environmental, and oceanic observations. What GAO Recommends: GAO is recommending that the program take steps to improve management and oversight and determine whether and how to recover certain capabilities that were removed from the program. In commenting on a draft of this report, the Acting Secretary of Commerce agreed with GAO‘ s recommendations and stated that the agency plans to implement them. To view the full product, including the scope and methodology, click on [hyperlink, http://www.gao.gov/products/GAO-09-323]. For more information, contact David A. Powner, (202) 512-9286, pownerd@gao.gov. [End of section] Contents: Letter: Background: GOES-R Is in Development, but Costs Have Increased, Envisioned Functionality Has Been Reduced, and Schedules Have Been Delayed: The GOES-R Program Office Has Taken Steps to Address Lessons Learned from Other Satellite Programs, but Important Actions Remain: NOAA Plans to Address Requirements for Current Products but Has Not Developed Plans for Meeting Requirements for Advanced Products: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Commerce: Appendix III: Comments from the National Aeronautics and Space Administration: Appendix IV: GAO Contact and Staff Acknowledgments: Tables: Table 1: Summary of the Procurement History of GOES: Table 2: Originally Planned GOES-R Series Instruments, as of August 2006: Table 3: Key Changes to the GOES-R Program, as of September 2006: Table 4: Description of Instrument Development Contracts, as of November 2008: Table 5: Status of GOES-R Instruments, as of February 2009: Table 6: Recent Delays in Key GOES-R Milestones: Table 7: Elements of a Sound Cost Estimating Methodology: Table 8: Summary of the GOES-R Cost Estimation Process: Table 9: Key Products from the Hyperspectral Environmental Suite Instrument: Table 10: Comparison of the Advanced Baseline Imager and the Legacy Sounder Instrument: Figures: Figure 1: Approximate GOES Geographic Coverage: Figure 2: Generic GOES Data Relay Pattern: Figure 3: GOES-R Program Office Structure and Staffing: Figure 4: Planned Schedule for GOES-R Program and Key Instruments: Abbreviations: GOES-R: Geostationary Operational Environmental Satellite-R Series: NASA: National Aeronautics and Space Administration: NOAA: National Oceanic and Atmospheric Administration: [End of section] United States Government Accountability Office: Washington, DC 20548: April 2, 2009: Congressional Requesters: Operational geostationary environmental satellites play a critical role in our nation's weather forecasting. These satellites--which are managed by the Department of Commerce's National Oceanic and Atmospheric Administration (NOAA)--provide critical information on atmospheric, oceanic, climatic, and solar conditions that help meteorologists observe and predict global and local weather events. They also provide a means to identify severe storm conditions, such as hurricanes and tornadoes, and to track the movement and intensity of these storms once they develop. NOAA, with the aid of the National Aeronautics and Space Administration (NASA), is procuring the next generation of geostationary satellites, called the Geostationary Operational Environmental Satellite-R (GOES- R) series. The GOES-R series is to replace the current series of satellites, which will likely begin to reach the end of their useful lives in approximately 2014. This new series is expected to mark the first major technological advance in GOES instrumentation since 1994. It is also considered critical to the United States' ability to maintain the continuity of data required for weather forecasting through the year 2028. This report responds to your request that we review NOAA's management of the GOES-R program. Specifically, we were asked to (1) determine the status of the program, (2) evaluate whether plans for the GOES-R acquisition address problems experienced on similar programs, and (3) determine whether NOAA's plan to address the capabilities that were planned for the satellites, but then removed, will be adequate to support current data requirements. To determine GOES-R acquisition status, we evaluated program documents, including cost and schedule estimates, contractor performance reports on instrument development, and executive briefings. To evaluate whether NOAA's acquisition plans address problems experienced on similar programs, we identified lessons learned from other major space acquisitions and compared them with relevant program and contractor documents, including instrument technical reviews, and risk lists. We assessed the GOES-R cost estimate by comparing the process used to develop the estimate with best practices identified in our cost estimating guide.[Footnote 1] To determine the adequacy of NOAA's plan to support key data requirements, we compared original and revised data requirements, determined whether the agency had developed plans for addressing different types of requirements, and discussed agency plans and options for addressing the requirements with key data users. For all objectives, we interviewed the applicable agency and contractor officials and consulted with GAO subject matter experts. We performed our work at NOAA and NASA offices in the Washington, D.C., metropolitan area. We conducted this performance audit from May 2008 to April 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix I contains further details on our objectives, scope, and methodology. Background: Since the 1960s, geostationary and polar-orbiting environmental satellites have been used by the United States to provide meteorological data for weather observation, research, and forecasting. NOAA's National Environmental Satellite, Data, and Information Service is responsible for managing the civilian operational geostationary and polar-orbiting satellite systems as two separate programs, called GOES and the Polar-orbiting Operational Environmental Satellites, respectively. Unlike polar-orbiting satellites, which constantly circle the earth in a relatively low polar orbit, geostationary satellites can maintain a constant view of the earth from a high orbit of about 22,300 miles in space. NOAA operates GOES as a two-satellite system that is primarily focused on the United States (see figure 1). These satellites are uniquely positioned to provide timely environmental data about the earth's atmosphere, its surface, cloud cover, and the space environment to meteorologists and their audiences. They also observe the development of hazardous weather, such as hurricanes and severe thunderstorms, and track their movement and intensity to reduce or avoid major losses of property and life. Furthermore, the satellites' ability to provide broad, continuously updated coverage of atmospheric conditions over land and oceans is important to NOAA's weather forecasting operations. Figure 1: Approximate GOES Geographic Coverage: [Refer to PDF for image: illustration] This illustration depicts the approximate GOES geographic coverage for GOES-11 and GOES-12. Sources: NOAA (data); MapArt (map). [End of figure] To provide continuous satellite coverage, NOAA acquires several satellites at a time as part of a series and launches new satellites every few years (see table 1). NOAA's policy is to have two operational satellites and one backup satellite in orbit at all times. Table 1: Summary of the Procurement History of GOES: Series name: Original GOES[B]; Procurement duration[A]: 1970-1987; Satellites: 1, 2, 3, 4, 5, 6, 7. Series name: GOES I-M; Procurement duration[A]: 1985-2001; Satellites: 8, 9, 10, 11, 12. Series name: GOES-N; Procurement duration[A]: 1998-2010; Satellites: 13, O, P, Q[C]. Series name: GOES-R; Procurement duration[A]: 2008-2016; Satellites: R, S. Source: GAO analysis of NOAA data. [A] Duration includes time from contract award to final satellite launch. [B] The procurement of these satellites consisted of four separate contracts for (1) two early prototype satellites and GOES-1, (2) GOES-2 and -3, (3) GOES-4 through -6, and (4) GOES-G (failed on launch) and GOES-7. [C] NOAA decided not to exercise the option for this satellite. [End of table] Four GOES satellites--GOES-10, GOES-11, GOES-12, and GOES-13- -are currently in orbit. Both GOES-11 and GOES-12 are operational satellites, with GOES-12 covering the east and GOES-11 the west. GOES- 13 is currently in an on-orbit storage mode. It is a backup for the other two satellites should they experience any degradation in service. GOES-10 is at the end of its service life, but it is being used to provide limited coverage of South America. The others in the series, GOES-O and GOES-P, are planned for launch over the next 2 years. [Footnote 2] NOAA is also planning the next generation of satellites, known as the GOES-R series, which are planned for launch beginning in 2015. Each of the operational geostationary satellites continuously transmits raw environmental data to NOAA ground stations. The data are processed at these ground stations and transmitted back to the satellite for broadcast to primary weather services and the global research community in the United States and abroad. Raw and processed data are also distributed to users via ground stations through other communication channels, such as dedicated private communication lines and the Internet. Figure 2 depicts a generic data relay pattern from the geostationary satellites to the ground stations and commercial terminals. Figure 2: Generic GOES Data Relay Pattern: [Refer to PDF for image: illustration] GOES satellite: Raw environmental data sent to ground station; Processed environmental data sent back to GOES; Processed environmental data broadcast from GOES satellite to users; Communications link between ground station and users. Source: GAO analysis of NOAA data. [End of figure] GOES-R Program: An Overview: NOAA plans for the GOES-R program to improve on the technology of prior series, in terms of both system and instrument improvements. The system improvements are expected to fulfill more demanding user requirements by updating the satellite data more often and providing satellite products to users more quickly. The instrument improvements are expected to significantly increase the clarity and precision of the observed environmental data. NOAA originally planned to acquire six different types of instruments. Furthermore, two of these instruments-- the Advanced Baseline Imager and the Hyperspectral Environmental Suite-- were considered to be the most critical because they would provide data for key weather products. Table 2 summarizes the originally planned instruments and their expected capabilities. Table 2: Originally Planned GOES-R Series Instruments, as of August 2006: Planned instrument: Advanced Baseline Imager; Description: Expected to provide variable area imagery and radiometric information about the earth's surface, atmosphere, and cloud cover. Key features include: * monitoring and tracking severe weather; * providing images of clouds to support forecasts; and; * providing higher resolution, faster coverage, and broader coverage simultaneously. Planned instrument: Hyperspectral Environmental Suite[A]; Description: Expected to provide information about the earth's surface to aid in the prediction of weather and climate monitoring. Key features include: * providing atmospheric moisture and temperature profiles to support forecasts and climate monitoring; * monitoring coastal regions for ecosystem health, water quality, coastal erosion, and harmful algal blooms; and; * providing higher resolution and faster coverage. Planned instrument: Geostationary Lightning Mapper; Description: Expected to continuously monitor lightning activity over the United States and adjacent oceans and to provide a more complete dataset than previously possible. Key features include: * detecting lightning strikes as an indicator of severe storms, and; * providing a new capability to GOES that only previously existed on NASA research satellites. Planned instrument: Magnetometer; Description: Expected to provide information on the general level of geomagnetic activity, monitor current systems in space, and permit detection of magnetopause crossings, sudden storm commencements, and substorms. Planned instrument: Space Environmental In-Situ Suite; Description: Expected to provide information on space weather to aid in the prediction of particle precipitation, which causes disturbance and disruption of radio communications and navigation systems. Key features include: * measuring magnetic fields and charged particles; * providing improved heavy ion detection, adding low-energy electrons and protons; and; * enabling early warnings for satellite and power grid operation, telecom services, astronauts, and airlines. Planned instrument: Solar Imaging Suite[B]; Description: Expected to provide coverage of the entire dynamic range of solar X-ray features, from coronal holes to X-class flares, as well as estimate the measure of temperature and emissions. Key features include: * providing images of the sun and measuring solar output to monitor solar storms, and; * providing improved imager capability. Source: GAO analysis of NOAA data. [A] The Hyperspectral Environmental Suite was canceled in September 2006. [B] The Solar Imaging Suite was divided into two separate acquisitions, the Solar Ultraviolet Imager and the Extreme Ultraviolet and X-Ray Irradiance Suite. [End of table] In September 2006, however, NOAA decided to reduce the scope and technical complexity of the GOES-R program because of expectations that total costs, which were originally estimated to be $6.2 billion, could reach $11.4 billion.[Footnote 3] Specifically, NOAA reduced the minimum number of satellites from four to two, canceled plans for developing the Hyperspectral Environmental Suite (which reduced the number of planned satellite products from 81 to 68), and divided the Solar Imaging Suite into two separate acquisitions. The agency estimated that the revised program would cost $7 billion. Table 3 provides a summary of the timeline and scope of these key changes. Table 3: Key Changes to the GOES-R Program, as of September 2006: Number of satellites; Baseline program, as of August 2006: 4; 2 critical instruments and 4 noncritical instruments or instrument suites; Critical instruments: 1. Advanced Baseline Imager; 2. Hyperspectral Environmental Suite; Noncritical instruments/suites: 3. Geostationary Lightning Mapper; 4. Magnetometer; Space Environmental In-Situ Suite; 5. Solar Imaging Suite (which included the Solar Ultraviolet Imager and the Extreme Ultraviolet and X-Ray Irradiance Suite); Revised program, as of September 2006: 2; 1 critical instrument and 5 noncritical instruments or instrument suites; Critical instrument: 1. Advanced Baseline Imager; Noncritical instruments/suites: 2. Geostationary Lightning Mapper; 3. Magnetometer; 4. Space Environmental In-Situ Suite; 5. Solar Ultraviolet Imager (formerly a component of the Solar Imaging Suite); 6. Extreme Ultraviolet and X-Ray Irradiance Suite (formerly a component of the Solar Imaging Suite). Number of satellite products; Baseline program, as of August 2006: 81; Revised program, as of September 2006: 68. Life-cycle cost estimate (in then year dollars); Baseline program, as of August 2006: $6.2-11.4 billion; Revised program, as of September 2006: $7 billion. End of operations and maintenance; Baseline program, as of August 2006: 2034; Revised program, as of September 2006: 2028[A]. Source: GAO analysis of NOAA data. [A] All satellites are expected to have a 15-year life span (5 years in on-orbit storage plus 10 years in operation). [End of table] Acquisition Strategy: NOAA's acquisition strategy was to award contracts for the preliminary design of the GOES-R system to several vendors who would subsequently compete to be the single prime contractor responsible for overall system development and production. In keeping with this strategy, NOAA awarded contracts for the preliminary design of the overall GOES-R system to three vendors in October 2005. However, in March 2007, NOAA revised its acquisition strategy for the development contract. In response to recommendations by independent advisors, the agency decided to separate the overall system development and production contract into two separate contracts--the spacecraft segment and the ground segment. In addition, to reduce the risks associated with developing technically advanced instruments, NASA awarded contracts for the preliminary designs for five of the planned instruments. NASA subsequently awarded development contracts for these instruments and, upon completion, plans to turn them over to the prime contractor responsible for the spacecraft segment of the GOES-R program. The sixth instrument, the Magnetometer, is to be developed as part of the spacecraft contract. Program Office Structure: NOAA is solely responsible for GOES-R program funding and overall mission success. However, since it relies on NASA's acquisition experience and technical expertise to help ensure the success of its programs, NOAA implemented an integrated program management structure with NASA for the GOES-R program (see figure 3). NOAA also located the program office at NASA's Goddard Space Flight Center. Within the program office, there are two project offices that manage key components of the GOES-R system. These are called the flight and ground segment project offices. The flight project office, managed by NASA, is responsible for awarding and managing the spacecraft contract and delivering flight-ready instruments to the spacecraft. The ground segment project office, managed by NOAA, oversees the ground contract, satellite data product development and distribution, and on-orbit operations of the satellites. Figure 3: GOES-R Program Office Structure and Staffing: [Refer to PDF for image: illustration] Top level: * Commerce; * NASA. Second level: * NOAA (reports to Commerce); * National Environmental Satellite, Data, and Information Service (reports to NOAA); * NOAA Program Management Council (communicates with NOAA and National Environmental Satellite, Data, and Information Service); * Goddard Space Flight Center Management Council (reports to NASA; communicates with National Environmental Satellite, Data, and Information Service and GOES-R Program). Third level: * GOES-R Program: System Program Director: NOAA; Deputy System Program Director: NOAA; Assistant System Program Director: NASA; (reports to National Environmental Satellite, Data, and Information Service; communicates with Program Scientist: NOAA and Goddard Space Flight Center Management Council). * Program Scientist: NOAA (reports to National Environmental Satellite, Data, and Information Service; communicates with GOES-R Program). Fourth level (all report to GOES-R Program): Program Control Lead: NOAA; Program Mission Assurance Lead: NASA; Flight Project Project Manager: NASA; Deputy: NOAA; Program Systems Engineering Lead: NASA; Contracts Lead: NOAA; Ground Project; Project Manager: NOAA; Deputy: NASA. Source: NOAA. [End of figure] Prior Report Noted the Likelihood of Continued Cost Growth and Recommended Steps to Mitigate Program Risks: In October 2007, we reported that NOAA had completed preliminary design studies of GOES-R, but that program costs were likely to grow and schedules were likely to be delayed.[Footnote 4] At that time, GOES-R was estimated to cost $7 billion and scheduled to have the first satellite ready for launch in 2014. However, independent studies showed that the program could cost about $2 billion more than the program's cost estimate, and the first satellite launch could be delayed by 2 years. NOAA officials stated that they were working to reconcile the two different cost and schedule estimates. We also reported that while the program had implemented a risk management program, it had multiple risk lists that were not always consistent, and key risks were missing from the risk watch lists-- including risks associated with unfilled executive positions, insufficient reserve funds for unexpected costs, and limitations in NOAA's insight into NASA's deliverables. Specifically, we noted that in past GOES procurements, NOAA did not have the ability to make quick decisions on problems because it lacked insight into the portions of the procurement that were managed by NASA. We recommended that the GOES- R program office manage risks using a program-level risk list and address the additional risks we identified. Over the past year, the program office has improved the integration of its risk management process and taken steps to mitigate the risks we identified. GOES-R Is in Development, but Costs Have Increased, Envisioned Functionality Has Been Reduced, and Schedules Have Been Delayed: The GOES-R program has moved from the preliminary design and definition phase to the development phase of its acquisition life cycle. Program officials have awarded contracts for the five instruments, and they plan to award contracts for the spacecraft[Footnote 5] and ground segments later this year. However, the program's cost, scope, and schedule have changed. Progress Has Been Made on GOES-R Procurement Activities: NOAA and NASA have made progress on the GOES-R program. In January 2008, NOAA approved a key decision milestone that allowed the program to move from the preliminary design and definition phase to the development phase of the acquisition life cycle. This approval also gave the program the authority to issue the requests for proposals for the spacecraft and ground segment projects--which it did in January 2008 and May 2008, respectively. The program office plans to award the prime contract for the spacecraft in May 2009 and the contract for the ground segment in June 2009. In addition, between September 2004 and December 2007, the GOES-R program awarded contracts for the development of five key instruments. [Footnote 6] Table 4 briefly describes each of these instruments, their contract award dates, and their cost and schedule estimates, while figure 4 depicts the schedule for both the program and key instruments. Table 4: Description of Instrument Development Contracts, as of November 2008 (Dollars in millions): Planned instrument: Advanced Baseline Imager; Description: Expected to provide variable area imagery and radiometric information of the earth's surface, atmosphere, and cloud cover; Contract award date: Sept. 2004; Scheduled completion date: June 2012; Baseline cost[A]: $358. Planned instrument: Space Environmental In-Situ Suite; Description: Expected to provide information on space weather to aid in the prediction of disturbances and disruptions of radio communications and navigation systems; Contract award date: Aug. 2006; Scheduled completion date: June 2012; Baseline cost[A]: $67. Planned instrument: Extreme Ultraviolet/X-Ray Irradiance Sensor; Description: Expected to provide real time measurement of solar activity in the Extreme Ultraviolet and X-ray spectrum; Contract award date: Aug. 2007; Scheduled completion date: June 2012; Baseline cost[A]: $55. Planned instrument: Solar Ultraviolet Imager; Description: Expected to observe the sun's ultraviolet emissions and provide early detection and location of flares and coronal mass ejections; Contract award date: Sept. 2007; Scheduled completion date: Oct. 2012; Baseline cost[A]: $113. Planned instrument: Geostationary Lightning Mapper; Description: Expected to continuously monitor lightning activity over the United States; Contract award date: Dec. 2007; Scheduled completion date: June 2012; Baseline cost[A]: $58. Source: GAO analysis of NOAA data. [A] These costs reflect the contractor's baseline cost estimates. In some cases, the program office's cost estimate is higher than the contractor's cost estimate. [End of table] Figure 4: Planned Schedule for GOES-R Program and Key Instruments: [Refer to PDF for image: illustrated schedule] Program start: 2003; Contract award and completion dates for the development of five key instruments: Advanced Baseline Imager: Award: Mid-2004; Completion: Mid-2012. Space Environmental In-Situ Suite: Award: Mid-2006; Completion: Mid-2012. Extreme Ultraviolet/X-Ray Irradiance Sensor: Award: Mid-2007; Completion: Mid-2012. Solar Ultraviolet Imager: Award: Mid-2007; Completion: Late-2012. Geostationary Lightning Mapper: Award: Late-2007 Completion: Mid-2012. Development phase begins: 2008; Space contract award: 2009. Ground contract award: 2009. Key instruments complete: 2012. GOES-R launch: 2015. GOES-S launch: 2018. Program end: 2028. Source: GAO analysis of NOAA data. [End of figure] The five key instruments are currently in varying stages of development. One instrument, the Advanced Baseline Imager, has experienced technical issues leading to cost overruns and schedule delays. The program office rebaselined the cost and schedule of the program in February 2007 and then rebaselined the schedule again in March 2008. Since February 2007, the contractor incurred a cost overrun of approximately $30 million and, since March 2008, the contractor has delayed $11 million worth of work. Program officials reported that they have sufficient management reserves to address the overruns experienced to date. The other instruments are still very early in development. Table 5 describes the status and risk level of each instrument. Table 5: Status of GOES-R Instruments, as of February 2009: Planned instrument: Advanced Baseline Imager; Status: In development since 2004, this instrument has experienced technical issues that led to the need to rebaseline the cost and schedule. A prototype of the unit is under development, and testing is scheduled to begin in May 2009; Program-identified risk level: Cost: high; Schedule: low; Technical: low. Planned instrument: Space Environmental In-Situ Suite; Status: This instrument successfully completed a preliminary design review in December 2008 and is on track for a critical design review in May 2010. It is to be delivered for integration on the GOES-R satellite by June 2012; Program-identified risk level: Cost: low; Schedule: low; Technical: low. Planned instrument: Extreme Ultraviolet/X-Ray Irradiance Sensor; Status: This instrument completed a preliminary design review in November 2008 and is on track for its critical design review in November 2009. Even though the instrument is still relatively early in its development, it is currently at its limit for mass. The contractor has developed a plan to reduce the instrument's mass. This instrument is to be delivered for integration on the GOES-R satellite by June 2012; Program-identified risk level: Cost: low; Schedule: low; Technical: low. Planned instrument: Solar Ultraviolet Imager; Status: This instrument completed a preliminary design review in October 2008 and is scheduled for a critical design review in December 2009. It is to be delivered for integration on the GOES-R satellite by October 2012; Program-identified risk level: Cost: low; Schedule: low; Technical: low. Planned instrument: Geostationary Lightning Mapper; Status: This preliminary design review for this instrument was delayed from January to March 2009 to complete risk reduction activities. In addition, the delivery of the prototype was delayed from July 2010 till February 2011. The instrument has recently experienced cost and schedule variances due to delays in completing key activities, the need for increased coordination with a subcontractor, and additional activities in the design of a component of the prototype; Program-identified risk level: Cost: high; Schedule: high; Technical: low. Sources: NOAA and NASA data. [End of table] GOES-R Cost Estimate Has Increased, Envisioned Functionality Has Been Reduced, and Key Milestones Have Slipped: NOAA has made several important decisions about the cost, scope, and schedule of the GOES-R program. After reconciling the program office's cost estimate of $7 billion with the independent cost estimate of about $9 billion, the agency established a new program cost estimate of $7.67 billion. This is an increase of $670 million from the previous estimate. Program officials plan to revisit this cost estimate after the spacecraft and ground segment contracts are awarded. However, agency officials, including NOAA's Chief Financial Officer and NOAA's National Environmental Satellite, Data, and Information Service Assistant Administrator, stated that this estimate was developed with a relatively high level of confidence and that they believe that any adjustments would be well within the $7.67 billion program budget. To mitigate the risk that costs would rise, program officials decided to remove selected program requirements from the baseline program and treat them as options that could be exercised if funds allow. These requirements include the number of products to be distributed, the time to deliver the remaining products (product latency), and how often these products are updated with new satellite data (refresh rate). Specifically, program officials eliminated the requirement to develop and distribute 34 of the 68 envisioned products, including aircraft icing threat, turbulence, and visibility. Program officials explained that these products are not currently being produced by legacy GOES satellites; they are new products that could be produced from the advanced GOES-R instruments. In addition, the program slowed planned product latency on the remaining products by as much as 10 minutes for hurricane intensity and 6 minutes for volcanic ash detection and height. It also reduced the refresh rates on these products by as much as 55 minutes for sea surface temperatures, cloud top observations, and vertical moisture profiles in the atmosphere. Program officials included the restoration of the products, latency, and refresh rates as options in the ground segment contract--items that could be acquired at a later time. NOAA also delayed GOES-R program milestones, including the dates for issuing the requests for proposals and awarding the contracts for the spacecraft and ground segments. The dates when the satellites would be available for launch have also slipped by 4 months, with the first satellite launch now scheduled for April 2015. Program officials attributed these delays to providing more stringent oversight before releasing the requests for proposals, additional time needed to evaluate the contract proposals, and funding reductions in fiscal year 2008. Table 6 identifies delays in key GOES-R milestones. Table 6: Recent Delays in Key GOES-R Milestones: Event: Request for proposals--spacecraft segment; Scheduled milestones (as of September 2007): Sept. 2007; Actual or current milestones (as of March 2009): Jan. 2008; Change: 4 months. Event: Request for proposals--ground segment; Scheduled milestones (as of September 2007): Nov. 2007; Actual or current milestones (as of March 2009): May 2008; Change: 6 months. Event: Contract award--spacecraft segment; Scheduled milestones (as of September 2007): May 2008; Actual or current milestones (as of March 2009): May 2009; Change: 12 months. Event: Contract award--ground segment; Scheduled milestones (as of September 2007): Aug. 2008; Actual or current milestones (as of March 2009): June 2009; Change: 10 months. Event: First satellite launch (GOES-R); Scheduled milestones (as of September 2007): Dec. 2014; Actual or current milestones (as of March 2009): Apr. 2015; Change: 4 months. Event: Second satellite launch (GOES-S); Scheduled milestones (as of September 2007): Apr. 2016; Actual or current milestones (as of March 2009): Aug. 2016; Change: 4 months. Source: GAO analysis of NOAA data. [End of table] Further delays in the launch of the first GOES-R satellite would run counter to NOAA's policy of having a backup satellite in orbit at all times and could lead to gaps in satellite coverage. Specifically, in 2015, NOAA expects to have two operational satellites in orbit, but it will not have a backup satellite in place until GOES-R is launched. If NOAA experiences a problem with either of its operational satellites before a backup satellite is in orbit, it will need to rely on older decommissioned satellites that may not be fully functional. The GOES-R Program Office Has Taken Steps to Address Lessons Learned from Other Satellite Programs, but Important Actions Remain: GOES-R has taken steps to address lessons from other satellite programs, but important actions remain to be completed. Satellite programs are often technically complex and risky undertakings and, as a result, they often experience technical problems, cost overruns, and schedule delays. We and others have reported on repeated missteps in the acquisition of major satellite systems, including the National Polar-orbiting Operational Environmental Satellite System, the GOES I- M series, the Space Based Infrared System High Program, and the Advanced Extremely High Frequency Satellite System.[Footnote 7] Key lessons learned from these other satellite programs include the importance of (1) ensuring sufficient technical readiness of the system's components prior to key decisions, (2) establishing realistic cost and schedule estimates, (3) providing sufficient management at the program and contractor levels, and (4) performing adequate senior executive oversight to ensure mission success. NOAA Has Taken Steps to Ensure Technical Readiness, but Key Risks Remain: Space programs often experience unforeseen technical problems in the development of critical components as a result of having insufficient knowledge of the components and their supporting technologies prior to key decision points. One key decision point is when an agency decides whether the component is sufficiently ready to proceed from a preliminary study phase into a development phase; this decision point results in the award of the development contract. Another key decision point occurs during the development phase when an agency decides whether the component is ready to proceed from design into production (also called the critical design review). Without sufficient technical readiness at these milestones, agencies could proceed into development contracts for components that are not well understood and enter into the production phase of development with technologies that are not yet mature. Since the late 1990s, NOAA has taken a series of steps to help mitigate technical readiness issues on GOES-R. Specifically, the agency: * conducted preliminary studies on the technologies to be used on the GOES-R instruments; * awarded contracts for the preliminary design of the planned instruments and the overall GOES-R system; * awarded instrument development contracts that include provisions to develop prototypes or engineering models before the flight units for each instrument are developed; * conducted a major review of the Advanced Baseline Imager before the next major milestone;[Footnote 8] * certified that the technology for the spacecraft and ground segments was mature before awarding the contracts;[Footnote 9] * removed the Hyperspectral Environmental Suite from the GOES-R series after preliminary studies showed that it was technically complex; * established independent review teams responsible for assessing the program's technical, programmatic, and management risks on an annual basis to ensure sufficient technical readiness prior to the critical design review milestone; and: * established processes for reviewing the maturity and readiness of algorithms for each of the products. However, key technology risks remain--affecting both the ground segment and the instruments. Specifically, while the hardware that is to be used for the ground segment is mature, key components have not previously been integrated. Consequently, if the components do not work together, the program might have to procure separate antennas, which would impact the program's cost and schedule. The ground segment project office utilizes an integrated product team to manage and mitigate this risk and released a request for information to industry in January 2009. In addition to the ground segment risks, technical risks remain on the development of the instruments. For example, the contractor responsible for developing the Advanced Baseline Imager estimates that the instrument is over 50 percent complete and reports that it has experienced technical issues, including problems with the quality of components in the focal plane module, mirrors, and telescope. As of November 2008, the contractor incurred a cost overrun of approximately $30 million and delayed $11 million worth of work. The other instruments are earlier in their development. Since none has yet been demonstrated in a lab or test environment, the risk remains that the technologies are not sufficiently mature. The program plans to continue efforts to demonstrate technologies before key decision milestones on each instrument. Program Estimates Were Developed Using an Acceptable Methodology, but an Underlying Assumption Is Overly Optimistic: In 2007, we reported that cost-estimating organizations throughout the federal government and industry use 12 key practices--related to planning, conducting, and reporting the estimate--to ensure a sound estimate.[Footnote 10] Table 7 lists these practices. Table 7: Elements of a Sound Cost Estimating Methodology: Activity area: Planning the estimate; Key practice: Define the estimate's purpose; Description: The estimate should define the purpose, describe the level of detail required, and identify the recipient of the estimate and the overall scope. Activity area: Planning the estimate; Key practice: Define the program or system characteristics; Description: The estimate should have a technical baseline description document with, among other things, the program's system and performance characteristics. Activity area: Planning the estimate; Key practice: Identify ground rules and assumptions; Description: The estimate should define what is included and excluded from the estimate and identify global and program-specific assumptions such as the estimate's base year and budget constraints. Activity area: Planning the estimate; Key practice: Determine the estimating approach; Description: The estimate should include a work breakdown structure, the estimating method, and a cost estimating checklist. Activity area: Planning the estimate; Key practice: Develop the estimating plan; Description: The estimate should identify the team, outline the approach, develop an estimate timeline, and identify who will do the independent cost estimate. Activity area: Conducting the estimate; Key practice: Obtain the data; Description: The estimate should include a data collection plan with emphasis on collecting current and relevant technical, programmatic, and risk data. The data sources should be investigated, and data should be normalized for inflation and stored for future estimates. Activity area: Conducting the estimate; Key practice: Perform the estimate; Description: The estimate should be developed in accordance with the identified methodology, ground rules, and assumptions; express costs in constant year dollars; and be validated. Activity area: Conducting the estimate; Key practice: Conduct a risk and uncertainty analysis; Description: The estimate should identify the risks and uncertainties associated with the project's cost, schedule, and technology. There should also be a confidence interval associated with the point estimate. Activity area: Conducting the estimate; Key practice: Conduct a sensitivity analysis; Description: The estimate should test the sensitivity of cost elements to changes in input values and key assumptions and identify the effects of changing the program schedule on the estimate. Activity area: Reporting the estimate; Key practice: Document the estimate; Description: The estimate should document all steps used to develop the estimate so that it can be recreated. Activity area: Reporting the estimate; Key practice: Review and provide results or presentation; Description: The estimate should be presented to management for approval. The presentation should include an explanation of the technical and programmatic uncertainties and comparisons of other cost estimates. Activity area: Reporting the estimate; Key practice: Update the estimate with actual costs and document lessons learned; Description: The estimate should be updated to reflect changes in assumptions and new project phases and milestones. Source: GAO. [End of table] The GOES-R program's cost estimate fully implemented 11 and partially implemented 1 of the 12 best practices for developing a credible cost estimate (see table 8). The practices that were fully implemented include clearly defining the purpose of the estimate and the program's characteristics, establishing and implementing a sound estimating approach, and appropriately assessing the risk and sensitivity of its estimate. The practice that was partially implemented involved ground rules and assumptions. The program defined and documented all of the ground rules and assumptions used in the estimate. However, an independent review team found that the assumed inflation rates used for the ground segment were overly optimistic and could lead to a shortfall of hundreds of millions of dollars. Specifically, the agency used the Department of Defense's inflation rates rather than NASA's historical experiences, which are more conservative. Program officials responded that the agency's long experience in developing ground systems would balance the optimistic inflation rates and that they could adjust the inflation rates, if warranted, in later years. Table 8: Summary of the GOES-R Cost Estimation Process: Best practice: 1. Define the estimate's purpose; Fully met. Best practice: 2. Define the program or system characteristics; Fully met. Best practice: 3. Identify ground rules and assumptions; Partially met. Best practice: 4. Determine the estimating approach; Fully met. Best practice: 5. Develop the estimating plan; Fully met. Best practice: 6. Obtain the data; Fully met. Best practice: 7. Perform the estimate; Fully met. Best practice: 8. Conduct a risk and uncertainty analysis; Fully met. Best practice: 9. Conduct a sensitivity analysis; Fully met. Best practice: 10. Document the estimate; Fully met. Best practice: 11. Review and provide results or presentation; Fully met. Best practice: 12. Update the estimate with actual costs and document lessons learned; Fully met. Source: GAO. [End of table] NOAA Has Improved Program Management, but Work Remains on Key Program Controls: In the past, we have reported on poor performance in program management.[Footnote 11] The key drivers of poor management often include ineffective risk management, insufficient staff to implement earned value management, and inadequate levels of management reserve. In 2006 and 2007, we reported that, while NOAA had taken steps to restructure its management approach on the GOES-R procurement in an effort to improve performance and to avoid past mistakes, key program management areas needed additional attention. We recommended that NOAA improve the consistency of its risk management process, assess and obtain the resources it needed for overseeing the earned value of its contracts, and provide sufficient management reserves to address unexpected issues in instrument development. NOAA subsequently implemented these recommendations by streamlining its risk management processes, supplementing oversight resources, and reassessing its management reserves. Since we last reported on this issue, the program office has made additional progress in earned value management, but more remains to be done. Program's Approach to Earned Value Management Has Shortfalls: Earned value management[Footnote 12] provides a proven means for measuring progress against cost and schedule commitments and thereby identifying potential cost overruns and schedule delays early, when the impact can be minimized. Two key aspects of this process are conducting comprehensive integrated baseline reviews and using monthly variance reports to manage the program. An integrated baseline review is a process used by stakeholders to obtain agreement on the value of planned work and to validate the baseline against which the variances are calculated. These reviews assess the technical scope of the work, key schedule milestones, the adequacy of resources, task and technical planning, and management processes; they are completed whenever a new baseline is established. Once an integrated baseline review has been completed and the project management baseline has been validated, monthly variance reports provide information on the contract status, the reasons for any deviations from cost or schedule plans, and any actions taken to address these deviations. To its credit, the GOES-R program office is using earned value management to oversee the key instrument contracts and plans to use it on the spacecraft and ground segment contracts. To date, the program office has performed integrated baseline reviews on the instruments and obtains and reviews variance reports for each of the instruments. However, there are shortfalls in the program's approach. The program's integrated baseline review for the Advanced Baseline Imager did not include a review of schedule milestones, the adequacy of how tasks are measured, and the contractor's management processes. Further, the variance reports for two instruments--the Advanced Baseline Imager and the Geostationary Lightning Mapper--do not describe all of the significant variances. The imager's reports only describe the five largest cost and schedule variances and do not include variances associated with overhead. For example, the reasons for cost and schedule variances exceeding $1 million were not disclosed in October and November 2008 cost reports.[Footnote 13] Moreover, while the reports identified problems that resulted in cost growth and schedule slippage for the five largest variances, the reports did not identify the actions taken to address them. The mapper's reports also did not disclose the reasons for selected variances, including a $197,000 cost overrun in August 2008 and a $141,000 cost overrun in October 2008. Program officials explained that they meet with the contractor on a monthly basis to discuss all of the variances, but they were unable to provide documentation of these discussions or the reasons, impact of, or mitigation plans for the variances. As a result of these shortfalls, the program office has less assurance that key instruments will be delivered on time and within budget, and it is more difficult for program managers to identify risks and take corrective actions. Executive-Level Involvement Is Occurring: Executive-level involvement is a key aspect of program success, and it is occurring on the GOES-R program. The Office of Management and Budget guidance calls for agencies to establish executive-level oversight boards to regularly track the progress of major system acquisitions. [Footnote 14] In addition, in 2007, NOAA and NASA signed both an interagency agreement and a management control plan that defined the agencies' respective roles and responsibilities. Among other things, the agreements called for the program to provide monthly status review briefings for executives on NOAA's Program Management Council and NASA's Goddard Space Flight Center Management Council. Since these agreements were approved, the program has consistently briefed senior management at monthly meetings and has effectively communicated the program's status and key risks. Additionally, key representatives of NOAA's Program Management Council attend the NASA council's meeting, and senior NASA executives attend NOAA's council meetings. NOAA Plans to Address Requirements for Current Products but Has Not Developed Plans for Meeting Requirements for Advanced Products: The Hyperspectral Environmental Suite instrument was originally planned as part of the GOES-R satellite to meet requirements for products that are currently produced by GOES satellites (such as temperature and moisture profiles at different atmospheric levels), as well as new technically-advanced products (such as moisture fluctuations and ocean color) that are not currently produced by GOES satellites. Table 9 lists the current and new products that were originally planned to be provided by the Hyperspectral Environmental Suite. Table 9: Key Products from the Hyperspectral Environmental Suite Instrument: Current sounding products: * GOES variable data stream format sounder data; * Channel brightness temperature; * Cloud top pressure; * Cloud top temperature; * Effective cloud amount; * Automated Surface Observing System categorical cloud height; * Automated Surface Observing System categorical cloud amount; * Total precipitable water; * Layer precipitable water; * Lifted index (a derived product image); * Surface skin temperature; * Vertical moisture profiles; * Vertical temperature profiles; * Geopotential height; * Water vapor winds; New sounding products: * Dust/aerosol: loading; * Cloud base height; * Capping inversion information; * Moisture flux; * Pressure profile; * Carbon monoxide concentration; * Ozone layers; * Microburst windspeed potential; * Improved temperature and moisture profiles; New coastal waters imaging products: * Ocean color (turbidity/chlorophyll/reflectance); * Ocean turbidity (turbidity/visibility); * Optical properties (particulate absorption, backscatter, fluorescence). Source: GAO analysis of NOAA data. Note: In addition to the products listed here, selected products were to have been jointly produced by the Hyperspectral Environmental Suite and the Advanced Baseline Imager. While these products are expected to be produced by the Advanced Baseline Imager, some will experience a degradation in quality and timeliness from what was originally anticipated. [End of table] NOAA still considers these requirements to be valid. According to National Weather Service meteorologists, users depend on the products that they currently receive from GOES satellites in orbit for hourly and daily weather observations. In addition, NOAA and the science community still have a need for the advanced products. NOAA had planned to use the new sounding products to improve its performance goals, such as helping to increase the lead times associated with severe thunderstorm warnings from an average of 18 minutes in 2000 to as much as 2 hours by 2025, and helping to increase the lead times associated with tornado warnings from an average of 13 minutes in 2007 to as much as 1 hour by 2025.[Footnote 15] In addition, NOAA had planned to use the new coastal waters imaging products to provide more accurate and quantitative understanding of areas along the U.S. East Coast (within 50 miles of the shore) and 130 estuaries throughout the United States- -areas for which NOAA has management responsibilities.[Footnote 16] Similarly, the environmental science communities have continued to express a need for the advanced products. In 2007, the National Research Council recommended that NOAA develop a strategy to restore the planned geostationary advanced sounding capability that was removed from the GOES-R program in order to allow high-temporal and high- vertical resolution measurements of temperature and water vapor. [Footnote 17] As part of that strategy, the report recommended that NOAA work with NASA to complete a demonstration satellite in the near term. NOAA Plans to Fulfill Requirements for Current Products: In light of the cancellation of the Hyperspectral Environmental Suite, NOAA decided to use the planned Advanced Baseline Imager to develop the products that are currently produced by the GOES satellite sounders now in orbit. In mid-2006, NOAA compared the imager's anticipated capabilities with the legacy GOES sounder instrument and reported that the advanced imager would be able to produce the necessary data 20 times faster than the legacy sounder and with comparable or better spatial resolution. However, NOAA also reported that the advanced imager will be less accurate than the legacy sounder for four of the seven product groups. Table 10 compares the capabilities of the Advanced Baseline Imager with the legacy sounder in seven product groups. Table 10: Comparison of the Advanced Baseline Imager and the Legacy Sounder Instrument: Product groups: Radiances; Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager is comparable to the legacy sounder[A][C]; Accuracy: Imager is comparable to the legacy sounder[C]. Product groups: Total precipitable water; Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager is comparable to the legacy sounder[A][C]; Accuracy: Imager is less precise than the legacy sounder[D]. Product groups: Lifted index (atmospheric stability measurement); Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager is comparable to the legacy sounder[A][C]; Accuracy: Imager is less precise than the legacy sounder[D]. Product groups: Skin (land surface) temperature; Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager is comparable to the legacy sounder[A][C]; Accuracy: Imager is comparable to the legacy sounder[C]. Product groups: Profiles (vertical moisture/temperature readings); Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager is comparable to the legacy sounder[A][C]; Accuracy: Imager is less precise than the legacy sounder[D]. Product groups: Clouds; Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager has a finer resolution than the legacy sounder[B]; Accuracy: Imager is less precise than the legacy sounder[D]. Product groups: Moisture winds; Temporal/latency (refresh rates): Imager is approximately 20 times faster than the legacy sounder[B]; Spatial (resolution): Imager has a finer resolution than the legacy sounder[B]; Accuracy: Imager is comparable to the legacy sounder[C]. Source: GAO analysis of NOAA data. [A] The Advanced Baseline Imager is planned to produce these products at a spatial resolution that is finer than that of the legacy sounder. However, in order to maintain continuity with existing products, the finer resolution data from the advanced baseline imager will need to be averaged to produce products with the resolution of the legacy sounder. [B] Improved = the imager is expected to improve this capability. [C] Comparable = the imager is expected to provide comparable capability. [D] Degraded = the imager is expected to degrade this capability. [End of table] In an effort to obtain consensus from the GOES user community, NOAA briefed sounding experts on the Advanced Baseline Imager's ability to develop products currently produced by the legacy GOES sounder. These experts included representatives from the National Weather Service; the National Environmental Satellite, Data, and Information Services; the Department of Defense's satellite data processing centers; academia; and attendees at a weather-related conference.[Footnote 18] NOAA reported that users accepted this plan as a suitable alternative until an advanced sounder could be flown on the GOES series. NOAA noted that users were pleased with the anticipated improvements in refresh rates. While satellite data users were eager to obtain faster refresh rates, recent contract changes have since reduced these expected rates. As previously reported, when the program office reconciled its cost estimate with the independent estimate, the program removed or reduced selected capabilities from the ground segment project. One of the reduced capabilities was the refresh rates for most of the products. Instead of refresh rates that are 20 times faster, current plans call for refresh rates that are only 2 to 4 times faster than current products. The faster refresh rates are now options in the contract. NOAA Assessed Alternatives for Addressing Requirements for Advanced Products but Has Not Yet Defined Its Plans: In addition to efforts to address the requirements for existing products that were removed with the Hyperspectral Environmental Suite, NOAA, NASA, and the Department of Defense assessed alternatives for obtaining advanced sounding and coastal waters imaging products from a geostationary orbit. The options include placing an advanced instrument on a stand-alone satellite or on later GOES satellites. The results of the analysis recommended that NOAA work with NASA to develop a demonstration sounder to fly on an as-yet undetermined satellite in order to build a foundation for an eventual operational advanced sounder on a future GOES satellite. For coastal waters imaging, the analysis recommended that, in the near-term, NOAA evaluate a hyperspectral imager that is planned to be included on the International Space Station and that NOAA and NASA coordinate to identify and evaluate other options for the future. NOAA plans to assess the technical feasibility of various options and to have the National Research Council make recommendations on long-term options for coastal waters imaging. However, NOAA has not defined plans or a timeline for implementing any of the options or for addressing the requirements for advanced products. Further, agency officials were unable to estimate when they would establish plans to fulfill the requirements. Doing so would include justifying the funding for any new initiatives within the agency's investment decision process. Until a decision is made on whether and how to proceed in providing the advanced products, key system users such as the National Weather Service will not be able to meet their goals for improving the lead times or accuracy of severe weather warnings, including warnings for tornadoes and hurricanes. Further, climate research organizations will not obtain the data they need to enhance the science of climate, coastal, environmental, and oceanic observations. Conclusions: The GOES-R satellite series is now in development, but program costs have increased, schedules have been delayed, and the scope of the program has been reduced. Unless the program exercises contract options, key benefits in terms of new products and faster data updates will not be realized. In addition, recent events make it likely that schedules will continue to slip. Any delays in the launch of the first satellite in the GOES-R program increase the risk of gaps in satellite coverage. The program office has made repeated and continuing efforts to learn from problems experienced on other satellite programs, but more can be done in selected areas. Specifically, the program has improved the technical readiness of key components, adopted many sound estimating practices, implemented an earned value management process for overseeing contracts, and is obtaining executive-level oversight. However, the program's approach to earned value management has shortfalls. The program did not perform a comprehensive review after rebaselining a critical instrument--the Advanced Baseline Imager--and has not documented the reasons for all cost overruns. Until these issues are addressed, NOAA faces an increased risk that the GOES-R program will repeat the cost increases, schedule delays, and performance shortfalls that have plagued other satellite programs. In addition, while the GOES-R program office plans to recover existing product capabilities that were lost when a critical sensor was removed from the satellites, NOAA has not yet developed a plan or a timeline for recovering the advanced capabilities that were removed. Doing so would include justifying whether and how to proceed in fulfilling the advanced requirements. Until such decisions and plans are made, the geostationary satellite user community may not be able to make significant improvements in their severe weather forecasts. Recommendations for Executive Action: To improve NOAA's ability to effectively manage the GOES-R program, we recommend that the Secretary of Commerce direct the NOAA Administrator to ensure that the following three actions are taken: * As part of any effort to rebaseline the cost and schedule of the Advanced Baseline Imager, perform an integrated baseline review and ensure that the review includes an assessment of key schedule milestones, the adequacy of resources, task and technical planning, and management processes. * Improve the agency's ability to oversee contractor performance by ensuring that the reasons for cost and schedule variances are fully disclosed and documented. * If feasible and justified, develop a plan and timeline for recovering the advanced capabilities that were removed from the program when the Hyperspectral Environmental Suite was canceled. Agency Comments and Our Evaluation: In written comments on a draft of this report, the Department of Commerce's Acting Secretary stated that the report did a fair and thorough job of assessing the status of the GOES-R program and NOAA's efforts to leverage lessons learned from similar programs. The department agreed with our findings and recommendations and outlined steps it is taking to implement them. For example, the department stated that NOAA will perform an integrated baseline review on the Advanced Baseline Imager, as part of any effort to rebaseline its cost and schedule, and that the GOES-R program office will ensure full disclosure of cost and schedule variances. The department also provided technical comments on the report, which we incorporated as appropriate. The department's comments are provided in appendix II. In addition, NASA's Associate Administrator for the Science Mission Directorate provided written comments on a draft of this report. In those comments, the Associate Administrator stated that the report is complete and accurate in its assessment of NASA's participation in the GOES-R program. The department's comments are provided in appendix III. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to interested congressional committees, the Secretary of Commerce, the Administrator of NASA, the Director of the Office of Management and Budget, and other interested parties. The report also will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you have any questions on matters discussed in this report, please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. Signed by: David A. Powner: Director, Information Technology Management Issues: List of Requesters: The Honorable Brian Baird: Chairman: The Honorable Bob Inglis: Ranking Member: Subcommittee on Energy and Environment: Committee on Science and Technology: House of Representatives: The Honorable Brad Miller: Chairman: The Honorable Paul Broun, Jr. Ranking Member: Subcommittee on Investigations and Oversight: Committee on Science and Technology: House of Representatives: The Honorable F. James Sensenbrenner, Jr. House of Representatives: [End of section] Appendix I: Objectives, Scope, and Methodology: Our objectives were to (1) determine the status of the Geostationary Operational Environmental Satellite-R series (GOES-R) program, (2) evaluate whether the National Oceanic and Atmospheric Administration's (NOAA) plans for the GOES-R acquisition address problems experienced on similar programs, and (3) determine whether NOAA's plan to address the capabilities that were planned for the satellites, but then removed, will be adequate to support current data requirements. To determine the program's status, we evaluated various programmatic and technical plans, management reports, and other program documentation. We reviewed the spacecraft and ground segment requests for proposals, cost and schedule estimates, contractor performance reports on instrument development, planned system requirements, and monthly executive-level management briefings. We also interviewed NOAA and National Aeronautics and Space Administration (NASA) officials from the GOES-R program office. To evaluate whether NOAA's acquisition plans address problems experienced on similar programs, we identified lessons learned from other major space acquisitions by reviewing prior GAO reports and interviewing space acquisition experts. We compared these lessons learned with relevant program and contractor documents and risk lists. Key lessons were related to technical readiness, cost and schedule estimates, program management, and executive-level involvement. Specific steps in each of these areas are as follows: * Technical readiness: We reviewed program, flight project, and ground segment risks, and instrument technical reviews. * Cost and schedule estimates: We identified the process used to develop NOAA's cost estimate by reviewing the program cost estimate, the Cost Analysis Requirements Document, and program cost estimate briefings. We then compared NOAA's process with the 12 steps of a high- quality cost estimating process identified in our cost estimating guide.[Footnote 19] For each step, we assessed whether the GOES-R estimate fully met, partially met, or did not meet the practices associated with a sound cost estimate. We also interviewed government and contractor cost estimating officials. * Program management: We analyzed the program's risk management plan, risk lists, cost performance reports, and integrated baseline reviews. * Executive-level involvement: We analyzed the program's management control plan, attended NOAA and NASA executive council meetings and reviewed the program's briefings to executives at both agencies. We also discussed these topics with appropriate agency officials. To determine the adequacy of NOAA's plans to address the capabilities that were removed from the program, we identified the requirements for existing and advanced products that were associated with the canceled Hyperspectral Environmental Suite instrument. We reviewed agency plans to fulfill requirements for the existing products and the analysis supporting this decision. We compared this analysis with an external scientific publication[Footnote 20] and interviewed satellite data users about the implications of plans to use the Advanced Baseline Imager to provide legacy products. To assess NOAA's plans to fulfill requirements for advanced products, we reviewed NOAA's analysis of options for addressing the advanced products, as well as National Research Council reports on priorities in satellite observation. [Footnote 21] We also interviewed satellite data users, including forecasters and modelers from the National Weather Service, and satellite data processing experts from the National Environmental Satellite, Data, and Information Service. We performed our work at NOAA and NASA offices in the Washington, D.C., metropolitan area. We conducted this performance audit from May 2008 to April 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Commerce: United States Department Of Commerce: The Secretary of Commerce: Washington, D.C. 20230: March 16, 2009: Mr. David A. Powner: Director, Information Technology Management Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Powner: Thank you for the opportunity to review and comment on the Government Accountability Office's draft report entitled Geostationary Operational Environmental Satellites: Acquisition Is Under Way, but Improvements Needed in Management and Contractor Oversight (GAO-09-323). On behalf of the Department of Commerce, I enclose the National Oceanic and Atmospheric Administration's programmatic comments on the draft report. Sincerely, Otto J. Wolff: Acting Secretary: Enclosure: [End of letter] Department of Commerce: National Oceanic and Atmospheric Administration: Comments on the Draft GAO Report Entitled "Geostationary Operational Environmental Satellites: Acquisition Is Under Way, but Improvements Needed in Management and Contractor Oversight" (GAO-09-323/April 2009): General Comments: The Department of Commerce (DOC) appreciates the opportunity to review this report on Geostationary Operational Environmental Satellites, specifically the R-Series (GOES-R). The report does a fair and thorough job assessing the status of the GOES-R program and National Oceanic and Atmospheric Administration's (NOAA) efforts to leverage lessons learned from similar programs. NOAA Response to GAO Recommendations: The draft GAO report states, "To improve NOAA's ability to effectively manage the GOES-R program, we recommend the Secretary of Commerce direct the NOAA Administrator to ensure the following three actions are taken:" Recommendation 1: "As part of any effort to rebaseline the cost and schedule of the Advanced Baseline Imager, perform an integrated baseline review and ensure the review includes an assessment of key schedule milestones, the adequacy of resources, task and technical planning, and management processes." NOAA Response: NOAA agrees with this recommendation. NOAA will perform an integrated baseline review on the Advanced Baseline Imager as part of any effort to rebaseline its cost and schedule. The integrated baseline review will include assessment of the technical scope of the work, key schedule milestones, the adequacy of resources, task and technical planning, and management processes. Recommendation 2: "Improve the agency's ability to oversee contractor performance by ensuring the reasons for cost and schedule variances are fully disclosed and documented." NOAA Response: NOAA agrees with this recommendation. GOES-R contractors submit monthly Cost Performance Reports with itemization of all variances. The GOES-R program office will ensure these cost and schedule variances reported by the contractor are elaborated upon as needed for full understanding and disclosure. Also, the GOES-R program office will fully document the actions taken to address significant cost and schedule variances, along with the reasons for and impact of those actions. Recommendation 3: "If feasible and justified, develop a plan and timeline of recovering the advanced capabilities that were removed from the program when the Hyperspectral Environmental Suite was canceled." NOAA Response: NOAA agrees with this recommendation. NOAA will identify and validate user requirements, evaluate the feasibility and priority of addressing those requirements, and determine the most appropriate methods to meet the requirements. [End of section] Appendix III: Comments from the National Aeronautics and Space Administration: National Aeronautics and Space Administration: Headquarters: Washington, DC 20546-0001: March 16, 2009: Reply to Attention of: Science Mission Directorate: Mr. David A. Powner: Director: Information Technology Management Issues: United States Government Accountability Office: Washington, DC 20548: Dear Mr. Powner: Thank you for the opportunity to review your draft report entitled "Geostationary Operational Environmental Satellites (GOES-R): Acquisition is Under Way, but Improvements Needed in Management and Contractor Oversight," (GAO-09-323). We believe the subject report is complete and accurate in its assessment of the GOES-R program insomuch as it involves NASA's participation in the activities. As the report contains no recommendations to NASA, we do not have any formal comments. We will continue to work closely with the National Oceanic and Atmospheric Administration to improve the areas of concern detailed in your report. If you have any questions or require additional information, please contact Ron Hooker at (202) 358-4508 or Stephen Volz on (202) 358-0364. Sincerely, Signed by: Edward J. Weiler: Associate Administrator for Science Mission Directorate: cc: Science Mission Directorate/C. Gay; M. Freilich; R. Hooker; S. Volz. GSFC/G. Morrow; D. Scheve; M. Brumfield; M. Donnelly. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: David A. Powner, (202) 512-9286, or pownerd@gao.gov: Staff Acknowledgments: In addition to the individual named above, Colleen M. Phillips, Assistant Director; Carol Cha; William Carrigg; Neil Doherty; Franklin Jackson; Kaelin Kuhn; Lee McCracken; Adam Vodraska; and Eric Winter made key contributions to this report. [End of section] Footnotes: [1] GAO, Cost Assessment Guide: Best Practices for Estimating and Managing Program Costs, Exposure Draft, GAO-07-1134SP (Washington, D.C.: July 2, 2007). [2] Satellites in a series are identified by letters of the alphabet when they are on the ground and by numbers once they are in orbit. [3] GAO, Geostationary Operational Environmental Satellites: Additional Action Needed to Incorporate Lessons Learned from Other Satellite Programs, [hyperlink, http://www.gao.gov/products/GAO-06-1129T] (Washington, D.C.: Sept. 29, 2006) and Geostationary Operational Environmental Satellites: Steps Remain in Incorporating Lessons Learned from Other Satellite Programs, [hyperlink, http://www.gao.gov/products/GAO-06-993] (Washington, D.C.: Sept. 6, 2006). [4] GAO, Geostationary Operational Environmental Satellites: Progress Has Been Made, but Improvements Are Needed to Effectively Manage Risks, [hyperlink, http://www.gao.gov/products/GAO-08-18] (Washington, D.C.: Oct. 23, 2007). [5] NASA awarded the spacecraft segment contract in December 2008. However, after a bid protest was filed, NASA decided, in February 2009, to re-evaluate its selection. Based on NASA's decision, the protest was dismissed. [6] A sixth instrument, the Magnetometer, is to be developed as part of the spacecraft segment contract. [7] GAO, Defense Acquisitions: Space System Acquisition Risks and Keys to Addressing Them, [hyperlink, http://www.gao.gov/products/GAO-06-776R] (Washington, D.C.: June 1, 2006); Polar-orbiting Operational Environmental Satellites: Cost Increases Trigger Review and Place Program's Direction on Hold, [hyperlink, http://www.gao.gov/products/GAO-06-573T] (Washington, D.C.: Mar. 30, 2006); Polar-orbiting Operational Environmental Satellites: Technical Problems, Cost Increases, and Schedule Delays Trigger Need for Difficult Trade-off Decisions, [hyperlink, http://www.gao.gov/products/GAO-06-249T] (Washington, D.C.: Nov. 16, 2005); Polar-orbiting Environmental Satellites: Information on Program Cost and Schedule Changes, [hyperlink, http://www.gao.gov/products/GAO-04-1054] (Washington, D.C.: Sept. 30, 2004); Defense Acquisitions: Despite Restructuring, SBIRS High Program Remains at Risk of Cost and Schedule Overruns, [hyperlink, http://www.gao.gov/products/GAO-04-48] (Washington, D.C.: Oct. 31, 2003); Military Space Operations: Common Problems and Their Effects on Satellite and Related Acquisitions, [hyperlink, http://www.gao.gov/products/GAO-03-825R] (Washington, D.C.: June 2, 2003); Defense Acquisitions: Assessments of Major Weapon Programs, [hyperlink, http://www.gao.gov/products/GAO-03-476] (Washington, D.C.: May 15, 2003); Weather Satellites: Action Needed to Resolve Status of the U.S. Geostationary Satellite Program, [hyperlink, http://www.gao.gov/products/GAO/NSIAD-91-252] (Washington, D.C.: July 24, 1991). Department of Defense, Defense Science Board/Air Force Scientific Advisory Board Joint Task Force, Report on the Acquisition of National Security Space Programs (May 2003). [8] In [hyperlink, http://www.gao.gov/products/GAO-06-993], we recommended that NOAA use system engineering experts to perform a comprehensive review of the Advanced Baseline Imager to determine the level of technical maturity before moving the instrument into production. In February 2007, consistent with our recommendation, an integrated independent review team performed a review on the imager. [9] The Consolidated Appropriations Act, 2008 requires NOAA to determine, among other provisions, that key component technologies have been demonstrated in a laboratory or test environment prior to contracting for the development of a major program such as GOES-R. 33 U.S.C. § 878a(b)(1). [10] [hyperlink, http://www.gao.gov/products/GAO-07-1134SP]. [11] [hyperlink, http://www.gao.gov/products/GAO-06-573T], [hyperlink, http://www.gao.gov/products/GAO-06-249T], and GAO, Defense Acquisitions: DOD Has Paid Billions in Award and Incentive Fees Regardless of Acquisition Outcomes, [hyperlink, http://www.gao.gov/products/GAO-06-66] (Washington, D.C.: Dec. 19, 2005). [12] Earned value management is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of a program's total costs. [13] In October 2008, cost overruns totaling $1,083,298 and schedule deviations valued at $1,001,436 were not explained. In November 2008, cost overruns totaling $1,193,559 and schedule deviations valued at $329,091 were not explained. [14] Office of Management and Budget, Supplement to Circular A-11, part 7 (Washington, D.C.: July 1997). Also see, GAO, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, [hyperlink, http://www.gao.gov/products/GAO-04-394G] (Washington, D.C.: March 2004). [15] In addition to advanced sounding, other activities such as improvements in radar technologies are expected to help improve lead times. [16] While current and future satellite systems provide selected coastal waters images, they lack the resolution, sampling frequency, and spectral information (field of vision) needed to monitor coastal areas and estuaries. [17] Committee on Earth Science and Applications from Space: A Community Assessment and Strategy for the Future, Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond (Washington, D.C.: The National Academies Press, 2007). [18] Representatives from the following organizations were also briefed on NOAA's plans: NASA, the University of Wisconsin's Cooperative Institute for Meteorological Satellite Studies, Massachusetts Institute of Technology, Hampton University, Utah State University, and the University of Maryland Baltimore County. [19] [hyperlink, http://www.gao.gov/products/GAO-07-1134SP]. [20] Schmidt, Timothy J., et al., "The GOES-R Advanced Baseline Imager and the Continuation of Current Sounder Products," Journal of Applied Meteorology and Climatology, Vol. 47 (Oct. 2008). [21] Committee on Earth Science and Applications from Space: A Community Assessment and Strategy for the Future, Earth Science and Applications from Space: National Imperatives for the Next Decade and Beyond (Washington, D.C.: The National Academies Press, 2007); and Committee on a Strategy to Mitigate the Impact of Sensor Descopes and Demanifests on the NPOESS and GOES-R Spacecraft, Ensuring the Climate Record from the NPOESS and GOES-R Spacecraft (Washington, D.C.: The National Academies Press, 2008). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.