Aviation Weather

FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, but Face Significant Challenges Gao ID: GAO-09-761 September 9, 2009

The National Weather Service's (NWS) weather products are a vital component of the Federal Aviation Administration's (FAA) air traffic control system. In addition to providing aviation weather products developed at its own facilities, NWS also provides on-site staff at each of FAA's en route centers--the facilities that control high-altitude flight outside the airport tower and terminal areas. Over the last few years, FAA and NWS have been exploring options for enhancing the efficiency of the aviation weather services provided at en route centers. GAO agreed to (1) determine the status and plans of efforts to restructure the center weather service units, (2) evaluate efforts to establish a baseline of the current performance provided by these units, and (3) evaluate challenges to restructuring them. To do so, GAO evaluated agency plans for the restructuring and for establishing performance measures. GAO also compared agency efforts to leading practices and interviewed agency officials.

NWS and FAA are considering plans to restructure the way aviation weather services are provided at en route centers, but it is not yet clear whether and how these changes will be implemented. In 2005, FAA requested that NWS restructure its services by consolidating operations to a smaller number of sites, reducing personnel costs, and providing services 24 hours a day, 7 days a week. NWS developed two successive proposals, both of which were rejected by FAA--most recently because the costs were too high. FAA subsequently requested that NWS develop another proposal by late December 2008. In response, NWS developed a third proposal that involves consolidating 20 of 21 existing center weather service units into two locations. NWS sent this proposal to FAA in early June 2009. FAA responded to NWS in August 2009 by requesting more information regarding NWS's proposal. In response to GAO's prior concerns that NWS and FAA lacked performance measures and a baseline of current performance, the agencies have agreed on five measures and NWS has proposed eight others. In addition, the agencies initiated efforts to establish a performance baseline for 4 of 13 potential performance measures. However, the agencies have not established baseline performance for the other 9 measures. NWS officials stated they are not collecting baseline information on the 9 measures for a variety of reasons, including that some of the measures have not yet been approved by FAA, and that selected measures involve products that have not yet been developed. While 4 of the 9 measures are tied to new products or services that are to be developed if NWS's latest restructuring proposal is accepted, the other 5 could be measured in the current operational environment. For example, both accuracy and customer satisfaction measures are applicable to current operations. It is important to obtain an understanding of the current level of performance in these measures before beginning any efforts to restructure aviation weather services. Without an understanding of the current level of performance, NWS and FAA may not be able to measure the success or failure of changes they make to the center weather service unit operations. As a result, changes to the current structure could degrade aviation operations and safety--and the agencies may not know it. NWS and FAA face challenges in their efforts to improve the current aviation weather structure. These include challenges associated with (1) interagency collaboration, (2) defining FAA's requirements, and (3) aligning any changes with the Next Generation Air Transportation System--a long-term initiative to increase the efficiency of the national airspace system. If the restructuring proposal is accepted, the agencies face three additional challenges in implementing it: (1) developing a feasible schedule that includes adequate time for stakeholder involvement, (2) undertaking a comprehensive demonstration to ensure no services are degraded, and (3) effectively reconfiguring the infrastructure and technologies. Unless and until these challenges are addressed, the proposed restructuring of aviation weather services at en route centers poses new risks and has little chance of success.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-09-761, Aviation Weather: FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, but Face Significant Challenges This is the accessible text file for GAO report number GAO-09-761 entitled 'Aviation Weather: FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, but Face Significant Challenges' which was released on September 9, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Chairman, Committee on Science and Technology, House of Representatives: United States Government Accountability Office: GAO: September 2009: Aviation Weather: FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, but Face Significant Challenges: GAO-09-761: GAO Highlights: Highlights of GAO-09-761, a report to the Chairman, Committee on Science and Technology, House of Representatives. Why GAO Did This Study: The National Weather Service‘s (NWS) weather products are a vital component of the Federal Aviation Administration‘s (FAA) air traffic control system. In addition to providing aviation weather products developed at its own facilities, NWS also provides on-site staff at each of FAA‘s en route centers”the facilities that control high- altitude flight outside the airport tower and terminal areas. Over the last few years, FAA and NWS have been exploring options for enhancing the efficiency of the aviation weather services provided at en route centers. GAO agreed to (1) determine the status and plans of efforts to restructure the center weather service units, (2) evaluate efforts to establish a baseline of the current performance provided by these units, and (3) evaluate challenges to restructuring them. To do so, GAO evaluated agency plans for the restructuring and for establishing performance measures. GAO also compared agency efforts to leading practices and interviewed agency officials. What GAO Found: NWS and FAA are considering plans to restructure the way aviation weather services are provided at en route centers, but it is not yet clear whether and how these changes will be implemented. In 2005, FAA requested that NWS restructure its services by consolidating operations to a smaller number of sites, reducing personnel costs, and providing services 24 hours a day, 7 days a week. NWS developed two successive proposals, both of which were rejected by FAA”most recently because the costs were too high. FAA subsequently requested that NWS develop another proposal by late December 2008. In response, NWS developed a third proposal that involves consolidating 20 of 21 existing center weather service units into two locations. NWS sent this proposal to FAA in early June 2009. FAA responded to NWS in August 2009 by requesting more information regarding NWS‘s proposal. In response to GAO‘s prior concerns that NWS and FAA lacked performance measures and a baseline of current performance, the agencies have agreed on five measures and NWS has proposed eight others. In addition, the agencies initiated efforts to establish a performance baseline for 4 of 13 potential performance measures. However, the agencies have not established baseline performance for the other 9 measures. NWS officials stated they are not collecting baseline information on the 9 measures for a variety of reasons, including that some of the measures have not yet been approved by FAA, and that selected measures involve products that have not yet been developed. While 4 of the 9 measures are tied to new products or services that are to be developed if NWS‘s latest restructuring proposal is accepted, the other 5 could be measured in the current operational environment. For example, both accuracy and customer satisfaction measures are applicable to current operations. It is important to obtain an understanding of the current level of performance in these measures before beginning any efforts to restructure aviation weather services. Without an understanding of the current level of performance, NWS and FAA may not be able to measure the success or failure of changes they make to the center weather service unit operations. As a result, changes to the current structure could degrade aviation operations and safety”and the agencies may not know it. NWS and FAA face challenges in their efforts to improve the current aviation weather structure. These include challenges associated with (1) interagency collaboration, (2) defining FAA‘s requirements, and (3) aligning any changes with the Next Generation Air Transportation System” a long-term initiative to increase the efficiency of the national airspace system. If the restructuring proposal is accepted, the agencies face three additional challenges in implementing it: (1) developing a feasible schedule that includes adequate time for stakeholder involvement, (2) undertaking a comprehensive demonstration to ensure no services are degraded, and (3) effectively reconfiguring the infrastructure and technologies. Unless and until these challenges are addressed, the proposed restructuring of aviation weather services at en route centers poses new risks and has little chance of success. What GAO Recommends: GAO is recommending that the Departments of Commerce and Transportation document baseline performance for several measures, and take steps to address challenges. In commenting on a draft of this report, Commerce officials agreed with the recommendations and identified steps to address them; Transportation officials agreed to consider the recommendations. View [hyperlink, http://www.gao.gov/products/GAO-09-761] or key components. For more information, contact David A. Powner at (202) 512- 9286 or at pownerd@gao.gov. [End of section] Contents: Letter: Background: Proposal to Consolidate Center Weather Service Units Is under Consideration: NWS and FAA Are Working to Establish a Baseline of Current Performance, but Are Not Assessing Key Measures: NWS and FAA Face Challenges in Efforts to Modify the Current Aviation Weather Structure: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Objectives, Scope, and Methodology: Appendix II: Comments from the Department of Commerce: Appendix III: GAO Contact and Staff Acknowledgments: Tables: Table 1: Key Weather Products Provided by the Aviation Weather Center: Table 2: Systems Used in the Center Weather Service Units: Table 3: Key Products and Services Provided by Center Weather Service Units: Table 4: Chronology of Efforts by FAA and NWS: Table 5: Approximate Costs (in millions) for the Transition: Table 6: Performance Measures Identified by FAA and NWS: Table 7: Status of Efforts to Identify Baseline Performance: Table 8: Key Events in FAA and NWS Interactions: Figures: Figure 1: FAA Facilities Involved in Air Traffic Control: Figure 2: Center Weather Service Unit Locations and Service Areas: Figure 3: Proposed Center Weather Service Unit Structure: Abbreviations: FAA: Federal Aviation Administration: NextGen: Next Generation Air Transportation System: NOAA: National Oceanic and Atmospheric Administration: NWS: National Weather Service: [End of section] United States Government Accountability Office: Washington, DC 20548: September 9, 2009: The Honorable Bart Gordon: Chairman: Committee on Science and Technology: House of Representatives: Dear Mr. Chairman: The National Weather Service (NWS) plays a significant role in providing weather services to the aviation community. NWS's weather products and data are vital components of the Federal Aviation Administration's (FAA) air traffic control system, providing weather information to local, regional, and national air traffic management, navigation, and surveillance systems. NWS aviation weather products include forecasts and warnings of meteorological conditions that could affect air traffic, including thunderstorms, air turbulence, and icing. In addition to providing aviation weather products that are developed at its own facilities, NWS also provides staff on site at each of FAA's en route centers--the facilities that control high-altitude flight outside the airport tower and terminal areas. This group of NWS meteorologists--called a center weather service unit--provides air traffic staff with forecasts, advisories, and periodic weather briefings on regional conditions. Over the last few years, FAA and NWS have been exploring options for enhancing the efficiency of the aviation weather services provided at en route centers. In September 2005, FAA asked NWS to restructure its services to be more efficient. Since then, NWS has developed and submitted two proposals to FAA, both of which were rejected. NWS subsequently submitted another proposal. Because of your interest in possible changes to NWS's aviation weather services, we agreed to (1) determine the status and plans of efforts to restructure the center weather service units, (2) evaluate efforts to establish a baseline of the current performance provided by the center weather service units so that FAA and NWS can ensure that any operational changes do not degrade aviation weather services, and (3) evaluate challenges to restructuring the center weather service units. To address our objectives, we reviewed NWS's proposals and transition plans for restructuring the service units and FAA's response to NWS's proposals. We identified both agencies' efforts to establish a baseline of current performance and compared these efforts to government guidance and best practices of leading organizations in performance management. To identify challenges, we compared the agencies' plans with best practices of leading organizations in system development, interagency collaboration, and architecture planning. We also interviewed relevant agency officials. We performed our work at FAA and NWS headquarters offices, and FAA's Air Traffic Control System Command Center in the Washington, D.C., metropolitan area. We performed our work from August 2008 to September 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Additional details on our objectives, scope, and methodology are provided in appendix I. Background: FAA is responsible for ensuring safe, orderly, and efficient air travel in the national airspace system. NWS supports FAA by providing aviation- related forecasts and warnings at air traffic facilities across the country. Among other support and services, NWS provides four meteorologists at each of FAA's 21 en route centers to provide on-site aviation weather services. This arrangement is defined and funded under an interagency agreement. FAA's Mission and Organizational Structure: FAA's primary mission is to ensure safe, orderly, and efficient air travel in the national airspace system. FAA reported that, in 2007, air traffic in the national airspace system exceeded 46 million flights and 776 million passengers. In addition, at any one time, as many as 7,000 aircraft--both civilian and military--could be aloft over the United States. In 2004, FAA's Air Traffic Organization was formed to, among other responsibilities, improve the provision of air traffic services. More than 33,000 employees within FAA's Air Traffic Organization support the operations that help move aircraft through the national airspace system. The agency's ability to fulfill its mission depends on the adequacy and reliability of its air traffic control systems, as well as weather forecasts made available by NWS and automated systems. These resources reside at, or are associated with, several types of facilities: air traffic control towers, terminal radar approach control facilities, air route traffic control centers (en route centers), and the Air Traffic Control System Command Center. The number and functions of these facilities are as follows: * 517 air traffic control towers manage and control the airspace within about 5 miles of an airport. They control departures and landings, as well as ground operations on airport taxiways and runways. * 170 terminal radar approach control facilities provide air traffic control services for airspace within approximately 40 miles of an airport and generally up to 10,000 feet above the airport, where en route centers' control begins. Terminal controllers establish and maintain the sequence and separation of aircraft. * 21 en route centers control planes over the United States--in transit and during approaches to some airports. Each center handles a different region of airspace. En route centers operate the computer suite that processes radar surveillance and flight planning data, reformats them for presentation purposes, and sends them to display equipment used by controllers to track aircraft. The centers control the switching of voice communications between aircraft and the center, as well as between the center and other air traffic control facilities. Three of these en route centers also control air traffic over the oceans. * The Air Traffic Control System Command Center manages the flow of air traffic within the United States. This facility regulates air traffic when weather, equipment, runway closures, or other conditions place stress on the national airspace system. In these instances, traffic management specialists at the command center take action to modify traffic demands in order to keep traffic within system capacity. See figure 1 for a visual summary of the facilities that control and manage air traffic over the United States. Figure 1: FAA Facilities Involved in Air Traffic Control: [Refer to PDF for image: illustration] Air Traffic Control System Command Center: Preflight: Air Traffic Control Tower. Takeoff: Air Traffic Control Tower. Departure: Terminal Radar Approach Control; Air Route Traffic Control Center. En route: Air Route Traffic Control Center. Descent: Air Route Traffic Control Center; Terminal Radar Approach Control. Approach: Terminal Radar Approach Control. Landing: Air Traffic Control Tower. Source: GAO analysis of FAA data. [End of figure] NWS's Mission and Organizational Structure: The mission of NWS--an agency within the Department of Commerce's National Oceanic and Atmospheric Administration (NOAA)--is to provide weather, water, and climate forecasts and warnings for the United States, its territories, and its adjacent waters and oceans to protect life and property and to enhance the national economy. In addition, NWS is the official source of aviation-and marine-related weather forecasts and warnings, as well as warnings about life-threatening weather situations. The coordinated activities of weather facilities throughout the United States allow NWS to deliver a broad spectrum of climate, weather, water, and space weather services in support of its mission. These facilities include 122 weather forecast offices located across the country that provide a wide variety of weather, water, and climate services for their local county warning areas, including advisories, warnings, and forecasts; 9 national prediction centers[Footnote 1] that provide nationwide computer modeling to all NWS field offices; and 21 center weather service units that are located at FAA en route centers across the nation and provide meteorological support to air traffic controllers. NWS Provides Aviation Weather Services to FAA: As an official source of aviation weather forecasts and warnings, several NWS facilities provide aviation weather products and services to FAA and the aviation sector. These facilities include the Aviation Weather Center, weather forecast offices located across the country, and 21 center weather service units located at FAA en route centers across the country. Aviation Weather Center: The Aviation Weather Center located in Kansas City, Missouri, issues warnings, forecasts, and analyses of hazardous weather for aviation. Staffed by 65 personnel, the center develops warnings of hazardous weather for aircraft in flight and forecasts of weather conditions for the next 2 days that could affect both domestic and international aviation. The center also produces a Collaborative Convective Forecast Product, a graphical representation of convective occurrence at 2, 4, and 6 hours. This is used by FAA to manage aviation traffic flow across the country. The Aviation Weather Center's key products are described in table 1. Table 1: Key Weather Products Provided by the Aviation Weather Center: Weather product: Significant Meteorological Information; Description: An advisory concerning the occurrence or expected occurrence of potentially hazardous weather conditions that may affect the safety of aircraft operations in the en route environment. Weather product: Convective Significant Meteorological Information; Description: A text product describing the occurrence or expected occurrence of thunderstorms and related weather conditions over the contiguous United States within 2 hours of issuance time. Weather product: Airman's Meteorological Information; Description: An advisory concerning the occurrence or expected occurrence of certain weather conditions that may affect the safety of aircraft in the en route environment, but at intensities that do not meet the criteria to develop a Significant Meteorological Information product. Weather product: Collaborative Convection Forecast Product; Description: A graphical convection forecast developed for strategic planning and management of en route air traffic. It is produced every 2 hours through collaboration - by way of an online chat room - among the Aviation Weather Center, the Meteorological Services of Canada, airline meteorology departments, FAA's Air Traffic Control System Command Center, and the center weather service units. These collaborative forecasts are produced between March 1 and October 31 every year. Source: GAO analysis of NWS data. [End of table] Weather Forecast Offices: NWS's 122 weather forecast offices issue terminal area forecasts for approximately 625 locations every 6 hours or when conditions change, consisting of the expected weather conditions significant to a given airport or terminal area, and are primarily used by commercial and general aviation pilots. Center Weather Service Units: NWS's center weather service units are located at each of FAA's 21 en route centers and operate 16 hours a day, 7 days a week (see figure 2). Each center weather service unit usually consists of three meteorologists and a meteorologist-in-charge who provide strategic advice and aviation weather forecasts to FAA traffic management personnel. Governed by an interagency agreement, FAA currently reimburses NWS approximately $12 million annually for this support. Figure 2: Center Weather Service Unit Locations and Service Areas: [Refer to PDF for image: illustrated map of the United States] Service Areas are depicted on the map for the following Center Weather Service Unit Locations: Albuquerque, New Mexico; Anchorage, Alaska; Atlanta, Georgia; Boston, Massachusetts; Chicago, Illinois; Cleveland, Ohio; Denver, Colorado; Fort Worth, Texas; Houston, Texas; Indianapolis, Indiana; Jacksonville, Florida; Kansas City, Missouri; Los Angeles, California; Memphis, Tennessee; Miami, Florida; Minneapolis, Minnesota; New York City, New York; Oakland, California; Salt Lake City, Utah; Seattle, Washington; Washington, DC. Sources: NWS (data), Map Resources (map). [End of figure] Center Weather Service Units: An Overview of Systems and Operations: The meteorologists at the center weather service units use a variety of systems to gather and analyze information compiled from NWS and FAA weather sensors. Key systems used to compile weather information include FAA's Weather and Radar Processor, FAA's Integrated Terminal Weather System, FAA's Corridor Integrated Weather System, and a remote display of NWS's Advanced Weather Interactive Processing System. Meteorologists at several center weather service units also use NWS's National Center Advanced Weather Interactive Processing System. Table 2 provides a description of selected systems. Table 2: Systems Used in the Center Weather Service Units: System: Weather and Radar Processor; Description: FAA's Weather and Radar Processor is used in en route centers and receives NWS products and data, information from automated weather sensors located at airports, and data from other sources such as weather satellites and radars. It compiles the information and provides current weather and forecasts to air traffic supervisors, traffic flow managers, and the center weather service unit meteorologists. System: Advanced Weather Interactive Processing System--Remote Display; Description: NWS's Advanced Weather Interactive Processing System integrates hydrometeorological data from a variety of sources and produces graphical displays at NWS weather forecast offices, river forecast centers, and national centers. This system aids forecaster analysis and decision making. Meteorologists at the en route centers have access to this system through a Remote Display system, which provides a dedicated connection to the supporting weather forecast office. The Remote Display is funded by FAA, and maintenance is provided by NWS. System: Integrated Terminal Weather System; Description: FAA's Integrated Terminal Weather System furnishes air traffic controllers and meteorologists with full-color graphic displays of weather information concerning airport terminal airspace within a 60- mile radius. The system also projects movement of severe weather systems up to 1 hour in the future and has been installed at 39 airports. System: Corridor Integrated Weather System; Description: FAA's Corridor Integrated Weather System is a prototype decision support tool that gathers weather information to help controllers select the most efficient routes for diverting traffic to avoid severe weather conditions. This system provides traffic flow managers with comprehensive convective weather data needed for tactical modifications occurring within 2 hours to the operational plan. These tactical modifications to the operational plan may include the weather impacts on air traffic control capacity, a need to modify the mitigation plan, and the execution of a modified mitigation plan. System: National Center Advanced Weather Interactive Processing System; Description: NWS's National Center Advanced Weather Interactive Processing System is the meteorological data visualization and integrated product generation system that provides a national scope of weather information. It is composed of software that ingests, analyzes, displays, and integrates various types of hydrometeorological data including numerical model, surface, upper-air, satellite, radar, and text data. This system is used in only a few center weather service units. Source: GAO analysis of FAA and NWS data. [End of table] NWS meteorologists at the en route centers provide several products and services to the FAA staff, including meteorological impact statements, center weather advisories, periodic briefings, and on-demand consultations. These products and services are described in table 3. In addition, center weather service unit meteorologists receive and disseminate pilot reports, provide input every 2 hours to the Aviation Weather Center's creation of the Collaborative Convective Forecast Product, train FAA personnel on how to interpret weather information, and provide weather briefings to nearby terminal radar approach control facilities and air traffic control towers. Table 3: Key Products and Services Provided by Center Weather Service Units: Product or service: Meteorological impact statement; Description: An unscheduled forecast of weather conditions that are expected to adversely impact the flow of air traffic in the en route center's area of responsibility within 4 to 12 hours. Product or service: Center weather advisory; Description: A short-term, unscheduled warning of hazardous weather conditions used primarily by air crews to anticipate and avoid adverse weather conditions in the en route and terminal environments. It describes current weather conditions or adverse weather conditions-- such as moderate to severe icing or turbulence, thunderstorms, low- level wind shear, and low ceilings and visibility--beginning within the next 2 hours. Product or service: Briefings; Description: Short updates provided by NWS meteorologists to FAA supervisors twice a day; these briefings include current weather warnings and advisories, a summary of forecasted weather across the national airspace, terminal forecasts, and other pertinent meteorological information. Product or service: On-demand consultation; Description: Unscheduled verbal presentations regarding ongoing or expected weather conditions provided to FAA traffic control personnel, supervisors, and other FAA facilities. Source: GAO analysis of FAA and NWS data. [End of table] FAA Seeks to Improve Aviation Weather Services Provided at En Route Centers: In recent years, FAA has undertaken multiple initiatives to assess and improve the performance of the center weather service units.[Footnote 2] Studies conducted in 2003 and 2006 highlighted concerns with the lack of standardization of products and services at NWS's center weather service units. To address these concerns, the agency sponsored studies that determined that weather data could be provided remotely using current technologies, and that private sector vendors could provide these services. In 2005, the agency requested that NWS restructure its aviation weather services by consolidating its center weather service units to a smaller number of sites, reducing personnel costs, and providing products and services 24 hours a day, 7 days a week. NWS subsequently submitted a proposal for restructuring its services, but FAA declined the proposal citing the need to refine its requirements. In December 2007, FAA issued revised requirements and asked NWS to respond with proposals defining the technical and cost implications of three operational concepts. The three concepts involved (1) on-site services provided within the existing configuration of offices located at the 21 en route centers, (2) remote services provided by a reduced number of regional facilities, and (3) remote services provided by a single centralized facility. NWS responded with three proposals, but FAA rejected these proposals in September 2008, noting that while elements of each proposal had merit, the proposed costs were too high. FAA requested that NWS revise its proposal to bring costs down while stating a preference to move toward a single center weather service unit with a back-up site. As a separate initiative, NWS initiated an improvement program for the center weather service units in April 2008. The goal of the program was to improve the consistency of the units' products and services. This program involved standardizing the technology, collaboration, and training for all 21 center weather service units and conducting site visits to evaluate each unit. NWS reported that it has completed its efforts to standardize the service units and plans to complete its site visits by September 2009. Table 4 provides a chronology of the agencies' assessment and improvement efforts. Table 4: Chronology of Efforts by FAA and NWS: Time frame: November 2003; Activity: FAA performed a functional audit of center weather service units and found that the services provided at different en route locations were inconsistent, the products were not standardized, and there was little communication and collaboration between neighboring service units. Time frame: September 2005; Activity: FAA requested that NWS restructure its aviation weather services to provide improved services more efficiently. Specifically, FAA requested that NWS consolidate 20 of the center weather service units (excluding the unit in Alaska) to a smaller number of sites, reduce NWS personnel costs by 20 percent, and deliver forecast products and services 24 hours a day, 7 days a week. Time frame: January 2006; Activity: FAA initiated an analysis of the value of different activities performed by the center weather service units. Similar to the 2003 study, the results of this analysis noted the lack of standardization of products, services, tools, and procedures. In addition, the report found that quality assurance was provided on an informal basis, there was no formal feedback process for products and services, and meteorological training was not standardized. Time frame: August 2006; Activity: NWS conducted a prototype in which center weather service unit products and services were completed and delivered remotely from the closest weather forecast office. This prototype showed that remote operations were possible and effective, but that they would be difficult to implement because of the need for cultural change, technology upgrades, and communication stability. Specifically, forecasters in the prototype were not able to provide dedicated support for the aviation mission because their other duties--including forecasting severe weather at the weather forecast office--took precedence. In addition, a collaboration technology used during the prototype was not operationally ready to use, servers were unstable, critical radar data were inconsistent with weather forecast office data, and communications lines were unstable throughout the prototype. Time frame: September 2006; Activity: An FAA study confirmed that it is possible to deliver weather information, products, and services from one or many remote locations with currently available state-of-the-art technology platforms. Time frame: October 2006; Activity: FAA administered a market survey to determine whether the private sector could provide remote weather services at a lower cost than currently provided. Ten organizations, including private sector firms and government-funded laboratories, responded that they could provide the services that FAA wanted. Separately, NWS presented its proposal for restructuring its aviation weather services to FAA. In this proposal, NWS suggested moving meteorologists from the en route centers to regional weather forecast offices, and providing remote aviation weather services from the weather forecast offices. Time frame: April 2007; Activity: FAA declined NWS's proposal. Instead, FAA reported that it would redefine its requirements for the functions provided by the center weather service units. Time frame: December 2007; Activity: FAA transmitted its redefined requirements to NWS and requested a written response detailing three different operational concepts. Time frame: April 2008; Activity: NWS initiated a short-term improvement program for the center weather service units. The goal of this program was to standardize the technology and training for the units to improve the consistency of products and services. Time frame: May 2008; Activity: In response to the new requirements, NWS provided FAA with three proposals to restructure the center weather service units. Time frame: September 2008; Activity: FAA rejected NWS's three proposals, stating that while elements of each proposal had merit, the agency could not accept them because the proposed costs were too high. Additionally, FAA requested that NWS deliver a revised proposal by December 2008, stating a preference to move toward a single center weather service unit with a back-up site. Source: GAO analysis of NWS and FAA data. [End of table] Prior GAO Report Identified Concerns with Center Weather Service Units; Recommended Steps to Improve Quality Assurance: In January 2008, we reported on concerns about inconsistencies in products and quality among center weather service units.[Footnote 3] We noted that while both NWS and FAA have responsibilities for assuring and controlling the quality of aviation weather observations, neither agency monitored the accuracy and quality of the aviation weather products provided at center weather service units. We recommended that NWS and FAA develop performance measures and metrics for the products and services to be provided by center weather service units, perform annual evaluations of aviation weather services provided at en route centers, and provide feedback to the center weather service units. The Department of Commerce agreed with our recommendations, and the Department of Transportation stated that FAA planned to revise its requirements and that these would establish performance measures and evaluation procedures. Proposal to Consolidate Center Weather Service Units Is under Consideration: NWS and FAA are considering plans to restructure the way aviation weather services are provided at en route centers. After a 6-month delay, NWS sent FAA its latest proposal for restructuring the center weather service units in June 2009.[Footnote 4] NWS's proposal involves consolidating 20 of the 21 existing center weather service units into two locations, with one at the Aviation Weather Center in Kansas City, Missouri, and the other at a new National Centers for Environmental Prediction office planned for the Washington, D.C., metropolitan area. [Footnote 5] The Missouri center is expected to handle the southern half of the United States while the Washington, D.C., center is expected to handle the northern half of the United States. NWS plans for the two new units to be staffed 24 hours a day, 7 days a week, and to function as back-up sites for each other. These new units would continue to use existing forecasting systems and tools to develop products and services. See figure 3 for a visual summary of the proposed consolidated center weather service unit facilities that control and manage air traffic over the United States. Figure 3: Proposed Center Weather Service Unit Structure: [Refer to PDF for image: illustrated map of the United States] The map depicts the following: Current (21) center weather service unit locations at en route centers (refer to figure 2 for specific locations): Albuquerque; Anchorage; Atlanta; Boston; Chicago; Cleveland; Denver; Fort Worth; Houston; Indianapolis; Jacksonville; Kansas City; Los Angeles; Memphis; Miami; Minneapolis; New York; Oakland; Salt Lake City; Seattle; Washington, DC. Proposed (2) center weather service unit locations: Kansas City; Washington, DC metropolitan area. Proposed area of responsibility for center weather service unit North: Boston; Chicago; Cleveland; Denver; Indianapolis; Minneapolis; New York; Salt Lake City; Seattle. Proposed area of responsibility for center weather service unit South: Albuquerque; Atlanta; Fort Worth; Houston; Jacksonville; Kansas City; Los Angeles; Memphis; Miami; Oakland. Center weather service unit location not part of proposed consolidation: Anchorage. Sources: NWS (data), Map Resources (map). [End of figure] While these new units would continue to use existing forecasting systems and tools to develop products and services, NWS has also proposed new products, services, and tools. Two new products are the Collaborative Weather Impact Product and the terminal radar approach control forecast. The former is expected to expand the Aviation Weather Center's existing Collaborative Convective Forecast Product to include convection, turbulence, icing, wind, ceiling/visibility, and precipitation type/intensity. The latter is expected to extract data from the Collaborative Weather Impact Product and include precipitation, winds, and convection for the terminal area; the display will allow the forecaster to layer this information on air traffic management information such as jet routes. In addition, NWS plans to create a Web portal to allow FAA and other users to access its advisories, forecasts, and products as well as national, regional, and local weather briefings. To support on-demand briefings at the new center weather service units, NWS plans to use collaboration tools, such as instant messaging and online collaboration software. Given the reduced number of locations in the revised organizational structure, NWS also proposed reducing the number of personnel needed to support its operations from 84 to 50 full-time staff--a reduction of 34 positions. Specifically, the agency determined that it will require 20 staff members for each new center weather service unit; 4 staff members at the Alaska unit; 5 additional forecasters at the Aviation Weather Center to help prepare the Collaborative Weather Impact Product; and a quality assurance manager at NWS headquarters. NWS anticipates the staff reductions will be achieved through scheduled retirements, resignations, and reassignments. However, the agency has identified the transition of its existing workforce to the new centers as a high- impact risk because staff may decline to move to the new locations. NWS also proposed tentative time frames for transitioning to the new organizational structure over a 3-year period. During the first year after FAA accepts the proposal, NWS plans to develop a transition plan and conduct a 9-month demonstration of the concept in order to ensure that the new structure will not degrade its services. Agency officials estimated that initial operating capability would be achieved by the end of the second year after FAA approval and full operating capability by the end of the third year. NWS estimated the transition costs for this proposal at approximately $12.8 million, which includes approximately $3.3 million for the demonstration. In addition, NWS estimated that the annual recurring costs will be about 21 percent lower than current annual costs. For example, using 2009 prices, NWS estimated that the new structure would cost $9.7 million--about $2.6 million less than the current $12.3 million cost. See table 5 for the estimated costs for transitioning the centers. Table 5: Approximate Costs (in millions) for the Transition: Description: Legacy centers; Year 1: $12.3; Year 2: $12.7; Year 3: $11.7; Year 4: $1.6; Year 5: $0; Total cost: $38.2[A]. Description: Transition costs; Year 1: $4.6; Year 2: $4.0; Year 3: $3.0; Year 4: $1.1; Year 5: $0; Total cost: $12.8[A]. Description: New centers; Year 1: $0; Year 2: $0; Year 3: $4.8; Year 4: $10.8; Year 5: $11.0; Total cost: $26.6. Description: Total; Year 1: $16.9; Year 2: $16.7; Year 3: $19.5; Year 4: $13.5; Year 5: $11.0; Total cost: $77.6. Source: GAO analysis of NWS data. [A] Numbers do not add correctly due to rounding. [End of table] However, it is not clear when and if the agencies will move forward with the proposal. FAA responded to NWS in August 2009 by requesting more information regarding NWS's proposal. One consideration that may affect the proposal involves the current interagency agreement. The most recent agreement between the two agencies, signed in December 2007, is to expire at the end of September 2009. Before it expires, the two agencies could choose to exercise an option to continue this agreement for another year, terminate the agreement, or sign a new agreement. An FAA official reported that the agency wanted to create a new agreement that includes key dates from the proposal, such as those related to the concept demonstration. This official added that such agreements typically take time to develop and coordinate between the agencies. NWS and FAA Are Working to Establish a Baseline of Current Performance, but Are Not Assessing Key Measures: According to best practices in leading organizations, performance should be measured in order to evaluate the success or failure of programs.[Footnote 6] Performance measurement involves identifying performance goals and measures, establishing performance baselines, identifying targets for improving performance, and measuring progress against those targets. Having a clear understanding of an organization's current performance--a baseline--is essential to determining whether new initiatives (like the proposed restructuring) result in improved or degraded products and services. In January 2008, we reported that NWS and FAA lacked performance measures and a baseline of current performance for the center weather service units and recommended that they develop performance measures. [Footnote 7] In response to this recommendation, FAA established 5 performance standards for the center weather service units. FAA also recommended that NWS identify additional performance measures in its proposal for restructuring the center weather service units. While NWS subsequently identified 8 additional performance measures in its proposal, FAA has not yet approved these measures. All 13 performance measures are listed in table 6. Table 6: Performance Measures Identified by FAA and NWS: Performance measure: Service provision (organizational); Description: A measure of time that the unit is operating (either through on-site staff or through off-site staff providing back-up operations); Source: Required by interagency agreement. Performance measure: Product participation; Description: A measure of the unit's participation in the development of the Collaborative Convective Forecast Product when convection is expected to occur within the unit's domain; Source: Required by interagency agreement. Performance measure: Format consistency; Description: A measure of the consistency of product formats, content, and procedures for the issuance of key existing products; Source: Required by interagency agreement. Performance measure: Service provision (briefings); Description: A measure of the unit's provision of twice-daily stand-up briefings; Source: Required by interagency agreement and proposed by NWS. Performance measure: Forecast accuracy; Description: A measure of the accuracy of forecasts used in traffic management decisions; Source: Required by interagency agreement and proposed by NWS. Performance measure: Customer satisfaction; Description: A measure of satisfaction with product quality, timeliness, accuracy, and customer service, as well as the number of complaints received; Source: Proposed by NWS. Performance measure: Service delivery conformity; Description: A measure of the conformity of both standardized and customized services to a checklist of components; Source: Proposed by NWS. Performance measure: Timeliness of on-demand services; Description: A measure of the time taken to respond to requests for on- demand service; Source: Proposed by NWS. Performance measure: Training completion; Description: A measure of completion of standardized training; Source: Proposed by NWS. Performance measure: Product consistency; Description: A measure of the consistency of the proposed Collaborative Weather Impact Product with other products; Source: Proposed by NWS. Performance measure: Timeliness of information updates; Description: A measure of NWS's ability to provide timely updates to the proposed Collaborative Weather Impact Product; Source: Proposed by NWS. Performance measure: Product availability; Description: A measure of the availability of products via a proposed Web portal; Source: Proposed by NWS. Performance measure: Timeliness of management reports on the restructuring; Description: A measure of NWS's ability to provide timely management reports associated with the restructuring; Source: Proposed by NWS. Source: GAO analysis of NWS and FAA data. [End of table] NWS officials reported that they have historical data for 1 of the 13 performance measures--participation in the Collaborative Convective Forecast Product--and are working to obtain a baseline for 3 other performance measures. Specifically, in January 2009, NWS and FAA began evaluating how the center weather service units are performing and, as part of this initiative, are collecting data associated with organizational service provision, format consistency, and briefing service provision. As of June 2009, the agencies had completed evaluations of 13 service units and plan to complete evaluations for all 21 service units by September 2009. However, the agencies have not established a baseline of performance for the 9 other performance measures. NWS officials reported that they are not collecting baseline information for a variety of reasons, including that the measures have not yet been approved by FAA and that selected measures involve products that have not yet been developed. A summary of the status of efforts to establish baselines and reasons for not establishing baselines is provided in table 7. Table 7: Status of Efforts to Identify Baseline Performance: Performance measure: Service provision (organizational); Status of efforts to identify baseline performance: Performance at 21 sites is being documented during site visits; NWS reason for not capturing a performance baseline: Not applicable--is being measured. Performance measure: Product participation; Status of efforts to identify baseline performance: Historical performance is being captured; NWS reason for not capturing a performance baseline: Not applicable--is being measured. Performance measure: Format consistency; Status of efforts to identify baseline performance: Performance at 21 sites is being documented during site visits; NWS reason for not capturing a performance baseline: Not applicable--is being measured. Performance measure: Service provision (briefings); Status of efforts to identify baseline performance: Performance at 21 sites is being documented during site visits; NWS reason for not capturing a performance baseline: Not applicable--is being measured. Performance measure: Forecast accuracy; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: More work is needed to determine how to measure accuracy. Performance measure: Customer satisfaction; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: FAA has not approved this measure; in addition, NWS officials stated they do not currently have the resources to develop and implement this measure. Performance measure: Service delivery conformity; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: FAA has not approved this measure. Performance measure: Timeliness of on-demand services; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: FAA has not approved this measure. Performance measure: Training completion; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: FAA has not approved this measure. Performance measure: Product consistency; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: This product has not yet been developed. Performance measure: Timeliness of information updates; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: This product has not yet been developed. Performance measure: Product availability; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: This product has not yet been developed. Performance measure: Timeliness of management reports on the restructuring; Status of efforts to identify baseline performance: Not measured; NWS reason for not capturing a performance baseline: These reports involve an initiative that has not yet been approved. Source: GAO analysis of NWS and FAA data. [End of table] While 4 of the potential measures are tied to new products or services under the restructuring, the other 5 could be measured using current products and services. For example, accuracy and customer satisfaction are measures that could be tracked for current operations. NWS continually measures the accuracy of a range of weather products-- including hurricane and tornado forecasts. Customer satisfaction measures could be determined by surveying the FAA managers who receive the aviation weather products. It is important to obtain an understanding of the current level of performance in these measures before beginning any efforts to restructure aviation weather services. Without an understanding of the current level of performance, NWS and FAA will not be able to measure the success or failure of any changes they make to the center weather service unit operations. As a result, any changes to the current structure could degrade aviation operations and safety--and the agencies may not know it. NWS and FAA Face Challenges in Efforts to Modify the Current Aviation Weather Structure: NWS and FAA face challenges in their efforts to modify the current aviation weather structure. These include challenges associated with (1) interagency collaboration, (2) defining requirements, and (3) aligning any changes with the Next Generation Air Transportation System (NextGen)--a long-term initiative to increase the efficiency of the national airspace system. Specifically, the two agencies have had difficulties in interagency collaboration and requirements development leading to an inability to reach agreement on a way forward. In addition, the restructuring proposals have not been aligned with the national strategic vision for the future air transportation system. Looking forward, if a proposal is accepted, the agencies could face three additional challenges in implementing the proposal, including (1) developing a feasible schedule that includes adequate time for stakeholder involvement, (2) undertaking a comprehensive demonstration to ensure no services are degraded, and (3) effectively reconfiguring the infrastructure and technologies to the new structure. Unless and until these challenges are addressed, the proposed restructuring of aviation weather services at en route centers poses new risks and has little chance of success. Interagency Collaboration: To date, FAA and NWS have encountered challenges in interagency collaboration. We have previously reported on key practices that can help enhance and sustain interagency collaboration.[Footnote 8] The practices generally consist of two or more agencies defining a common outcome, establishing joint strategies to achieve the outcome, agreeing upon agency roles and responsibilities, establishing compatible policies and procedures to operate across agency boundaries, and developing mechanisms to monitor, evaluate, and report the results of collaborative efforts. While NWS and FAA have established policies and procedures for operating across agencies through the interagency agreement and have initiated efforts to establish a baseline of performance for selected measures through their ongoing site evaluations, the agencies have not defined a common outcome, established joint strategies to achieve the outcome, or agreed upon agency responsibilities. Instead, the agencies have demonstrated an inability to work together to resolve issues and to accomplish meaningful change. Specifically, since 2005, FAA has requested that NWS restructure its aviation weather services three times, and then rejected NWS's proposals twice. Further, after requesting extensions twice, NWS provided its proposal to FAA in June 2009. As a result, it is now almost 4 years since FAA first initiated efforts to improve NWS aviation weather services and the agencies have not yet agreed on what needs to be changed and how it will be changed. Table 8 lists key events. Table 8: Key Events in FAA and NWS Interactions: Time frame: September 2005; Activity: FAA requested that NWS restructure its aviation weather services to consolidate operations in a smaller number of sites at a reduced cost. Time frame: October 2006; Activity: NWS provided a proposal to FAA on how to restructure aviation weather services; also, FAA administered a market study to determine whether the private sector could provide remote aviation weather services. Time frame: April 2007; Activity: FAA rejected NWS's proposal because it did not consolidate the offices to a smaller number of sites and it involved higher training costs. At that time, FAA decided to revise its requirements for aviation weather provided at the center weather service units. Time frame: December 2007; Activity: FAA provided NWS with a new set of aviation weather requirements. Time frame: May 2008; Activity: NWS provided FAA with three proposals to restructure the center weather service units. Time frame: September 2008; Activity: FAA rejected all three proposals and sent NWS back to the drawing board to create a hybrid solution at a lower cost by December 2008. Time frame: December 2008; Activity: NWS requested and FAA approved a 60-day extension on NWS's proposal deadline to address public misperceptions regarding the changes. Time frame: February 2009; Activity: NWS requested a 60-day extension on its proposal deadline to allow the new NOAA administrator time to work with the then-unnamed FAA administrator on the consolidation. FAA approved a 30-day extension. Time frame: June 2009; Activity: NWS provided FAA with a proposal that would consolidate 20 of 21 center weather service units into two locations. Source: GAO analysis of FAA and NWS data. [End of table] Until the agencies agree on a common outcome, establish joint strategies to achieve the outcome, and agree on respective agency responsibilities, they are unlikely to move forward in efforts to restructure weather services. Without sound interagency collaboration, both FAA and NWS will continue to spend time and resources proposing and rejecting options rather than implementing solutions. Defining Requirements: The two agencies' difficulties in determining how to proceed with their restructuring plans are due in part to a lack of stability in FAA's requirements for center weather service units. According to best practices of leading organizations, requirements describe the functionality needed to meet user needs and perform as intended in the operational environment.[Footnote 9] A disciplined process for developing and managing requirements can help reduce the risks associated with developing or acquiring a system or product. FAA released its revised requirements in December 2007 and NWS subsequently provided proposals to meet these requirements. However, FAA rejected all three of NWS's proposals in September 2008 on the basis that the costs of the proposals were too high, even though cost was not specified in FAA's requirements. NWS's latest proposal is based on FAA's December 2007 requirements as well as detailed discussions held between the two agencies in October 2008. However, FAA has not revised its requirements to reflect the guidance provided to NWS in those discussions, including reported guidance on handling the Alaska center and moving to the two-center approach. Without formal requirements developed prior to the development of the new products and services, FAA runs the risk of procuring products and services that do not fully meet its users' needs or perform as intended. In addition, NWS risks continued investments in trying to create a product for FAA without clear information on what the agency wants. Alignment with the Next Generation Air Transportation System: Neither FAA nor NWS has ensured that the restructuring of the center weather service units fits with the national vision for NextGen--a long- term initiative to transition FAA from the current radar-based system to an aircraft-centered, satellite-based system. Our prior work on enterprise architectures shows that connecting strategic planning with program and system solutions can increase the chances that an organization's operational and information technology environments will be configured to optimize mission performance.[Footnote 10] Our experience with federal agencies has shown that investing in information technology without defining these investments in the context of a larger, strategic vision often results in systems that are duplicative, not well integrated, and unnecessarily costly to maintain and interface. The Joint Planning and Development Office[Footnote 11] is responsible for planning and coordinating NextGen. As part of this program, the Joint Planning and Development Office envisions restructuring air traffic facilities, including en route centers, across the country as well as transitioning to new technologies. However, NWS and FAA efforts to restructure the center weather service units have not been aligned with the Joint Planning and Development Office's vision for transforming air traffic control under the NextGen program. Specifically, the chair of NextGen's weather group stated that Joint Planning and Development Office officials have not evaluated NWS's and FAA's plans for restructuring the center weather service units, and have not been asked to do so. Other groups within FAA are responsible for aligning the agency's enterprise architecture with the NextGen vision through annual roadmaps that largely define near-and mid-term initiatives.[Footnote 12] However, recent roadmaps for aviation weather do not include any discussion of plans to restructure the center weather service units or the potential impact that such a change could have on aviation weather systems. Additionally, in its proposal, NWS stated that it followed FAA's guidance to avoid tightly linking the transition schedule to NextGen's expected initial operating capability in 2013, but recommended doing so since the specific role of the center weather service units in NextGen operations is unknown. Until the agencies ensure that changes to the center weather service units fit within the strategic-level and implementation plans for NextGen, any changes to the current structure could result in wasted efforts and resources. Schedule Development: Looking forward, if a proposal is accepted, both agencies could also face challenges in developing a feasible schedule that includes adequate time for stakeholder involvement. NWS estimated a 3-year transition time frame from current operations to the two-center approach. FAA officials commented they would like to have the two- center approach in place by 2012. However, NWS may have difficulty in meeting the transition time frames because activities that need to be conducted serially are planned concurrently within the 3-year schedule. For example, NWS may need to negotiate with its union before implementing changes that affect working conditions--such as moving operations from an en route center to a remote location.[Footnote 13] NWS officials acknowledge the risk that these negotiations can be prolonged and sometimes take years to complete. If the proposal is accepted, it will be important for NWS to identify activities that must be conducted before others in order to build a feasible schedule. Demonstrating No Degradation of Service: If a proposal is accepted, both agencies could face challenges in demonstrating that existing services will not be degraded during the restructuring. In its proposal, NWS identified preliminary plans to demonstrate the new operational concept before implementing it in order to ensure there is no degradation of service. Key steps included establishing a detailed demonstration plan, conducting risk mitigation activities, and implementing a demonstration that is to last at least 9 months. NWS also proposed that the demonstration will include an independent evaluation by a team of government and industry both before the demonstration, to determine if the demonstration is adequate to validate the new concept of operations, and after, to determine the success of the demonstration. In addition, throughout the 9-month demonstration, NWS plans to have the independent team periodically provide feedback, recommendations, and corrective actions. However, NWS has not yet defined all of the performance measures it will use to determine whether the prototype is successful. In its proposal, NWS stated that the agencies will begin to document performance metrics and develop and refine evaluation criteria during the demonstration. If NWS waits to define evaluation criteria during the evaluation, it may not have baseline metrics needed to compare to the demonstration results. Without baseline metrics, NWS may be unable to determine whether the demonstration has degraded service or not. Technology Transition: Both agencies could face challenges in effectively transitioning the infrastructure and technologies to the new consolidated structure, if a proposal is accepted. In its proposal, NWS planned to move its operations from 20 en route centers to two sites within 3 years. However, to do so, the agencies will need to modify their aviation weather systems and develop a communications infrastructure. Specifically, NWS and FAA will need to modify or acquire systems to allow both current and new products for an expanded view of the country. Additionally, NWS will need to develop continuous two-way communications in lieu of having staff on site at each en route center. NWS has recognized the infrastructure as a challenge, and plans to mitigate the risk through continuous dialogue with FAA. However, if interagency collaboration does not improve, attempting to coordinate the systems and technology of the two agencies may prove difficult and further delay the schedule. Conclusions: For several years, FAA and NWS have explored ways to improve the operations of the center weather service units by consolidating operations and providing remote services. Meanwhile, the two agencies have to make a decision on the interagency agreement, which will expire at the end of September 2009. If FAA and NWS are to create a new interagency agreement that incorporates key dates within the proposal, decisions on the proposal will have to be made quickly. An important component of any effort to improve operations is a solid understanding of current performance. However, FAA and NWS are not working to identify the current level of performance in five measures that are applicable to current operations. Until the agencies have an understanding of the current level of performance, they will not be able to measure the success or failure of any changes to the center weather service unit operations. As a result, any changes to the current structure could degrade aviation operations and safety--and the agencies may not know it. If the agencies move forward with plans to restructure aviation weather services, they face significant challenges including a poor record of interagency collaboration, undocumented requirements, and a lack of assurance that this plan fits in the broader vision of NextGen. Moreover, efforts to implement the restructuring will require a feasible schedule, a comprehensive demonstration, and a solid plan for technology transition. Until these challenges are addressed, the proposed restructuring of aviation weather services at en route centers has little chance of success. Recommendations for Executive Action: To improve the aviation weather products and services provided at FAA's en route centers, we are making six recommendations to the Secretaries of Commerce and Transportation. Specifically, we are recommending that the Secretaries direct the NWS and FAA administrators, respectively, to: * immediately identify the current level of performance for the five potential measures that could be identified under current operations (forecast accuracy, customer satisfaction, service delivery conformity, timeliness of on-demand services, and training completion) so that there will be a baseline from which to measure the impact of any proposed operational changes; * establish and approve a set of performance measures for the center weather service units; * improve interagency collaboration by defining a common outcome, establishing joint strategies to achieve the outcome, and agreeing upon each agency's responsibilities; * establish and finalize requirements for aviation weather services at en route centers; * ensure that any proposed organizational changes are aligned with NextGen initiatives by seeking a review by the Joint Program Development Office responsible for developing the NextGen vision; and: * before moving forward with any proposed operational changes, address implementation challenges by: - developing a feasible schedule that includes adequate time for stakeholder involvement, - undertaking a comprehensive demonstration to ensure no services are degraded, and: - effectively transitioning the infrastructure and technologies to the new consolidated structure. Agency Comments and Our Evaluation: The Department of Commerce provided written comments on a draft of this report, signed by the Secretary of Commerce (see appendix II). In the department's letter, NOAA agreed with our recommendations and provided additional information on steps the agency has taken or plans to take to address the recommendations. For example, the agency reported that it is working with FAA to refine its baseline metrics and plans to have baseline metrics in place by the end of fiscal year 2009. NWS also plans to establish a Quality Assurance Manager to work with FAA to develop additional measures and metrics as appropriate. In addition, NOAA reported that since submitting its proposal in May 2008 and receiving a summary of our findings in June 2009, NWS and FAA have made progress in working together. Specifically, NWS has assigned a liaison to the FAA Air Traffic Control System Command Center, met with officials from the Joint Planning and Development Office to discuss the linkage of plans for the center weather service units and NextGen, and held discussions to strengthen the NWS and FAA partnership. The Department of Transportation's Deputy Director of Audit Relations provided comments on a draft of this report via e-mail. In those comments, she noted that the department agreed to consider our recommendations. In addition, she noted a slight concern involving our discussion of the alignment of plans to restructure the center weather service units with NextGen. Specifically, the department noted that current NextGen plans (1) include modifications to the weather systems used by center weather service units and (2) ensure the delivery of the functions currently provided by center weather service units. However, these statements do not alter the agencies' need to align any restructuring plans with the NextGen initiative--and that has not occurred. NextGen roadmaps do not include any discussion of plans to restructure the center weather service units or the potential impact that such a change could have on aviation weather systems. The importance of this alignment is underscored by the NWS Director's recommendation to FAA to provide linkage between restructuring plans and NextGen plans since the specific role of the center weather service units during NextGen operations is not yet known. Both departments also provided technical comments that we incorporated as appropriate. We are sending copies of this report to interested congressional committees, the Secretary of Commerce, the Secretary of Transportation, the Director of the Office of Management and Budget, and other interested parties. In addition, this report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you have any questions about this report, please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Sincerely, Signed by: David A. Powner: Director, Information Technology Management Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: The objectives of our review were to (1) determine the status and plans of efforts to restructure the center weather service units, (2) evaluate efforts to establish a baseline of the current performance provided by center weather service units so that the Federal Aviation Administration (FAA) and National Weather Service (NWS) can ensure that any operational changes do not degrade aviation weather services, and (3) evaluate challenges to restructuring the center weather service units. To determine the status of NWS's plans for restructuring the center weather service units, we reviewed the current interagency agreement, FAA's proposed requirements, and NWS's draft and final proposals for addressing FAA's requirements. We analyzed NWS's draft transition schedules, cost proposals, and evaluation plans. We also interviewed NWS and FAA officials to obtain clarifications on these plans. To evaluate the agencies' efforts to establish a baseline of the current performance provided by center weather service units, we reviewed documentation including FAA's performance standards, the current interagency agreement, NWS's restructuring proposals and Quality Assurance Surveillance Plan, and the agencies' plans for evaluating the centers. We compared the agencies' plans for creating a baseline of current performance with best practices for performance management by the Department of the Navy and General Services Administration.[Footnote 14] We also interviewed NWS and FAA officials involved in establishing a baseline of current performance provided by center weather service units. To evaluate challenges to restructuring the center weather service units, we reviewed agency documentation, including FAA's requirements document and NWS's proposals to restructure the center weather service units. We also reviewed planning documents for the Next Generation Air Transportation System. We compared these documents with best practices for system development and requirements management from the Capability Maturity Model® Integration for Development;[Footnote 15] and with GAO's best practices in interagency collaboration and architecture planning.[Footnote 16] In addition, we interviewed NWS, FAA, and Joint Planning and Development Office officials regarding challenges to restructuring the center weather service units. We performed our work at FAA and NWS headquarters offices, and FAA's Air Traffic Control System Command Center in the Washington, D.C., metropolitan area. We conducted this performance audit from August 2008 to September 2009, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for findings and conclusions based on our audit objectives. [End of section] Appendix II: Comments from the Department of Commerce: United States Department Of Commerce: The Secretary of Commerce: Washington, D.C. 20230: August 11, 2009: Mr. David A. Powner: Director: Information Technology Management Issues: U.S. Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Powner: Thank you for the opportunity to review and comment on the Government Accountability Office's draft report entitled "Aviation Weather: FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, But Face Significant Challenges" (GAO-09-761). On behalf of the Department of Commerce, I have enclosed the National Oceanic and Atmospheric Administration's programmatic comments to the draft report. Sincerely, Signed by: Gary Looke: Enclosure: [End of letter] Department of Commerce: National Oceanic and Atmospheric Administration Comments on the Draft Government Accountability Office Report Entitled "Aviation Weather: FAA and the National Weather Service Are Considering Plans to Consolidate Weather Service Offices, But Face Significant Challenges" (GAO-09-761, September 2009): General Comments: The Department of Commerce and the National Oceanic and Atmospheric Administration (NOAA) appreciate the opportunity to review this report on the restructuring of aviation weather services. We have some general comments regarding collaboration between NOAA's National Weather Service (NWS) and the Department of Transportation's Federal Aviation Administration (FAA), which are presented below. Page 23, second paragraph, second sentence: We recommend modifying the sentence "Instead, the agencies have demonstrated an inability to work together to resolve issues and to accomplish meaningful change" to: "Although there has been an improvement over the last year, the agencies have been unable to work together to resolve issues and to accomplish meaningful change." Since submitting our original response to FAA in May 2008 and based on initial Government Accountability Office (GAO) feedback, NWS and FAA have made significant progress in working together towards a common goal of accomplishing meaningful change. The two agencies are now working collaboratively, as evidenced by: the detailing of a senior NWS liaison to the FAA Command Center; collaboration with officials from the Joint Planning and Development Office (JPDO) on center weather service unit (CWSU) plans and their subsequent linkage to the Next Generation Air Transportation System (NextGen); and collaborative discussions at the senior executive level to strengthen NWS-FAA partnerships. Page 25, last paragraph: NOAA agrees that strategic planning needs to be connected with program and system solutions. To that end, NWS and FAA are working collaboratively with regard to NextGen and are developing the necessary linkages to ensure that the operational and IT environments will be configured to optimize mission performance in the NextGen era. NOAA Response to GAO Recommendations: The draft GAO report states: "To improve the aviation weather products and services provided at FAA's en route centers, we are making six recommendations to the Secretaries of Commerce and Transportation. Specifically, we are recommending that the Secretaries direct the NWS and FAA administrators, respectively, to:" Recommendation 1: "Immediately identify the current level of performance for the five potential measures that could be identified under current operations (forecast accuracy, customer satisfaction, service delivery conformity, timeliness of on-demand services, and training completion) so that there will be a baseline from which to measure the impact of any proposed operational changes." NOAA Response: NOAA concurs with this recommendation and will have baseline metrics developed by the end of fiscal year 2009. NWS will continue to work with FAA to refine these baseline metrics, as appropriate. Recommendation 2: "Establish and approve a set of performance measures for the center weather service units." NOAA Response: NOAA concurs with this recommendation. NWS is working collaboratively with FAA to develop and track the metrics stated in the first GAO recommendation. In addition, NWS is establishing a position, Quality Assurance Manager, to work with FAA to develop additional metrics and measures, as appropriate. Recommendation 3: "Improve interagency collaboration by defining a common outcome, establishing joint strategies to achieve the outcome, and agreeing upon each agency's responsibilities." NOAA Response: NOAA concurs with this recommendation. NWS will work collaboratively with FAA to ensure further clarity in the definition of roles and responsibilities, and will continue to work towards specific objectives and expected outcomes related to the CWSU program. Recommendation 4: "Establish and finalize requirements for aviation weather services at en route centers." NOAA Response: NOAA concurs with this recommendation. NWS has worked with FAA to identify and document requirements and will ensure that any changes that have occurred based on the NWS-FAA collaboration are reflected in updated requirements documents. Recommendation 5: "Ensure that any proposed organizational changes are aligned with NextGen initiatives by seeking a review by the Joint Program Development Office responsible for developing the NextGen vision." NOAA Response: NOAA concurs with this recommendation. NWS has worked with JPDO; however, we recognize that tighter collaboration is needed. The NWS team working on the CWSU response held a meeting on July 14, 2009, with the JPDO official mentioned in the GAO statement of facts. This JPDO official reviewed the current NWS response to FAA and has provided comments. We will continue to work with JPDO to ensure successful alignment with NextGen. Recommendation 6: "Before moving forward with any proposed operational changes, address implementation challenges by: * developing a feasible schedule that includes adequate time for stakeholder involvement; * undertaking a comprehensive demonstration to ensure no services are degraded; and; * effectively transitioning the infrastructure and technologies to the new consolidated structure." NOAA Response: NOAA concurs with this recommendation and notes the following: 1. NWS has proposed a schedule intended to allow for adequate time for stakeholder and independent third-party assessment. 2. NWS has proposed a comprehensive nine-month demonstration and plans to work with FAA and independent and neutral experts to establish sustainable metrics to measure service delivery (including performance and quality). 3. NWS has proposed a transition plan intended to adequately address infrastructure and technology changes. NWS will take appropriate steps to ensure relevant documents clearly articulate these plans. [End of section] Appendix III: GAO Contact and Staff Acknowledgments: GAO Contact: David A. Powner, (202) 512-9286, or pownerd@gao.gov: Staff Acknowledgments: In addition to the contact person named above, Colleen Phillips, Assistant Director; Gerard Aflague; Kate Agatone; Neil Doherty; Rebecca Eyler; and Jessica Waselkow made key contributions to this report. [End of section] Footnotes: [1] These centers include the National Centers for Environmental Prediction Central Operations, Aviation Weather Center, Environmental Modeling Center, Hydrometeorological Prediction Center, Ocean Prediction Center, Storm Prediction Center, Tropical Prediction Center/ National Hurricane Center, Climate Prediction Center, and Space Environment Center. [2] FAA is also involved in a longer-term initiative to increase the efficiency of the national airspace system and to improve its overall safety. This initiative, called the Next Generation Air Transportation System, is a joint effort of the National Aeronautics and Space Administration, the White House Office of Science and Technology Policy, and the Departments of Transportation, Homeland Security, Defense, and Commerce. FAA anticipates that this initiative may lead to major changes in the aviation weather program that would supersede its current restructuring efforts. [3] GAO, Aviation Weather: FAA Is Reevaluating Services at Key Centers; Both FAA and the National Weather Service Need to Better Ensure Product Quality, [hyperlink, http://www.gao.gov/products/GAO-08-258] (Washington, D.C.: Jan. 11, 2008). [4] NWS sought two extensions to the December 2008 deadline in order to allow NWS and FAA a chance to address public misperceptions, to brief the incoming administration, and to arrange discussions between the appropriate NWS and FAA executives. [5] NWS proposed that the center weather service unit located in Anchorage, Alaska, remain unchanged. [6] Department of the Navy, Office of the Chief Information Officer, Guide for Developing and Using Information Technology (IT) Performance Measurements (Washington, D.C., October 2001); General Services Administration, Office of Governmentwide Policy, Performance-Based Management: Eight Steps To Develop and Use Information Technology Performance Measures Effectively (Washington, D.C., 1996). [7] [hyperlink, http://www.gao.gov/products/GAO-08-258]. [8] GAO, Results-Oriented Government: Practices that Can Help Enhance and Sustain Collaboration among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 2005). [9] Carnegie Mellon University Software Engineering Institute, Capability Maturity Model® Integration for Development, Version 1.2 (Pittsburgh, Pa., August 2006). Capability Maturity Model® and Capability Maturity Modeling are registered in the U.S. Patent and Trademark Office. CMM is a service mark of Carnegie Mellon University. [10] GAO, Enterprise Architecture: Leadership Remains Key to Establishing and Leveraging Architectures for Organizational Transformation, [hyperlink, http://www.gao.gov/products/GAO-06-831] (Washington, D.C.: Aug. 14, 2006). [11] The Joint Planning and Development Office has multiple federal partners, including FAA; the Departments of Transportation, Commerce, Defense, and Homeland Security; the National Aeronautics and Space Administration; and the White House Office of Science and Technology Policy. [12] These groups include the NextGen and Operations Planning Service Unit's Aviation Weather Office, Systems Engineering Office, and NextGen Integration and Implementation Office. [13] NWS's agreement with its union includes the need to negotiate on the impact and implementation of any changes affecting working conditions before those changes can be implemented. As such, any effort to realign the center weather service units will involve negotiations between union employees and NWS management. [14] Department of the Navy, Office of the Chief Information Officer, Guide for Developing and Using Information Technology (IT) Performance Measurements (Washington, D.C., October 2001); General Services Administration, Office of Governmentwide Policy, Performance-Based Management: Eight Steps To Develop and Use Information Technology Performance Measures Effectively (Washington, D.C., 1996). [15] Carnegie Mellon University Software Engineering Institute, Capability Maturity Model® Integration for Development, Version 1.2 (Pittsburgh, Pa., August 2006). [16] GAO, Results-Oriented Government: Practices that Can Help Enhance and Sustain Collaboration among Federal Agencies, [hyperlink, http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21, 2005); Enterprise Architecture: Leadership Remains Key to Establishing and Leveraging Architectures for Organizational Transformation, [hyperlink, http://www.gao.gov/products/GAO-06-831] (Washington, D.C.: Aug. 14, 2006). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.