Aviation Safety

Better Data and Targeted FAA Efforts Needed to Identify and Address Safety Issues of Small Air Cargo Carriers Gao ID: GAO-09-614 June 24, 2009

The air cargo industry contributed over $37 billion to the U.S. economy in 2008 and provides government, businesses, and individuals with quick delivery of goods. Although part of an aviation system with an extraordinary safety record, there have been over 400 air cargo accidents and over 900 incidents since 1997, raising concerns about cargo safety. GAO's congressionally requested study addresses (1) recent trends in air cargo safety, (2) factors that have contributed to air cargo accidents, (3) federal government and industry efforts to improve air cargo safety and experts' views on the effectiveness of these efforts, and (4) experts' views on further improving air cargo safety. To perform the study, GAO analyzed agency data, surveyed a panel of experts, reviewed industry and government documents, and interviewed industry and government officials. GAO also conducted site visits to Alaska, Ohio, and Texas.

From 1997 through 2008, 443 accidents involving cargo-only carriers occurred, including 93 fatal accidents. Total accidents declined 63 percent from a high of 62 in 1997 to 23 in 2008. Small cargo carriers were involved in the vast majority of the accidents--79 percent of all accidents and 96 percent of fatal accidents. Although accident rates for large cargo carriers fluctuated during this period, they were comparable to accident rates for large passenger carriers in 2007. GAO could not calculate accident rates based on operations or miles traveled for small carriers because the Federal Aviation Administration (FAA) does not collect the necessary data. Although several factors contributed to these air cargo accidents, our review of National Transportation Safety Board (NTSB) data found that pilot performance was identified as a probable cause for about 80 percent of fatal and about 53 percent of non-fatal cargo accidents. Furthermore, GAO's analysis of NTSB reports for the 93 fatal accidents, using an FAA flight-risk checklist, identified three or more risk factors in 63 of the accidents. Risk factors included low pilot experience, winter weather, and nighttime operations. Alaska's challenging operating conditions and remotely located populations who rely on air cargo are also a contributing factor. Many federal efforts to improve air cargo safety focus on large carriers. Air cargo experts that GAO surveyed ranked FAA's voluntary disclosure programs--in which participating carriers voluntarily disclose safety events to FAA--as the most effective effort to improve air cargo, but two of the three main voluntary disclosure programs are used typically by large carriers. Several industry initiatives, however, focus on carriers with smaller aircraft, such as the Medallion Foundation, which has improved small aircraft safety in Alaska through training and safety audits. The two actions experts cited most often to further improve air cargo safety were installing better technology on cargo aircraft to provide additional tools to pilots and collecting data to track small cargo carrier operations. Using flight risk checklists can also help pilots assess the accumulated risk factors associated with some cargo flights.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-09-614, Aviation Safety: Better Data and Targeted FAA Efforts Needed to Identify and Address Safety Issues of Small Air Cargo Carriers This is the accessible text file for GAO report number GAO-09-614 entitled 'Aviation Safety: Better Data and Targeted FAA Efforts Needed to Identify and Address Safety Issues of Small Air Cargo Carriers' which was released on June 24, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Requesters: United States Government Accountability Office: GAO: June 2009: Aviation Safety: Better Data and Targeted FAA Efforts Needed to Identify and Address Safety Issues of Small Air Cargo Carriers: GAO-09-614: GAO Highlights: Highlights of GAO-09-614, a report to congressional requesters. Why GAO Did This Study: The air cargo industry contributed over $37 billion to the U.S. economy in 2008 and provides government, businesses, and individuals with quick delivery of goods. Although part of an aviation system with an extraordinary safety record, there have been over 400 air cargo accidents and over 900 incidents since 1997, raising concerns about cargo safety. GAO‘s congressionally requested study addresses (1) recent trends in air cargo safety, (2) factors that have contributed to air cargo accidents, (3) federal government and industry efforts to improve air cargo safety and experts‘ views on the effectiveness of these efforts, and (4) experts‘ views on further improving air cargo safety. To perform the study, GAO analyzed agency data, surveyed a panel of experts, reviewed industry and government documents, and interviewed industry and government officials. GAO also conducted site visits to Alaska, Ohio, and Texas. What GAO Found: From 1997 through 2008, 443 accidents involving cargo-only carriers occurred, including 93 fatal accidents. Total accidents declined 63 percent from a high of 62 in 1997 to 23 in 2008. Small cargo carriers were involved in the vast majority of the accidents”79 percent of all accidents and 96 percent of fatal accidents. Although accident rates for large cargo carriers fluctuated during this period, they were comparable to accident rates for large passenger carriers in 2007. GAO could not calculate accident rates based on operations or miles traveled for small carriers because the Federal Aviation Administration (FAA) does not collect the necessary data. Although several factors contributed to these air cargo accidents, our review of National Transportation Safety Board (NTSB) data found that pilot performance was identified as a probable cause for about 80 percent of fatal and about 53 percent of non-fatal cargo accidents. Furthermore, GAO‘s analysis of NTSB reports for the 93 fatal accidents, using an FAA flight-risk checklist, identified three or more risk factors in 63 of the accidents. Risk factors included low pilot experience, winter weather, and nighttime operations. Alaska‘s challenging operating conditions and remotely located populations who rely on air cargo are also a contributing factor. Many federal efforts to improve air cargo safety focus on large carriers. Air cargo experts that GAO surveyed ranked FAA‘s voluntary disclosure programs”in which participating carriers voluntarily disclose safety events to FAA”as the most effective effort to improve air cargo, but two of the three main voluntary disclosure programs are used typically by large carriers. Several industry initiatives, however, focus on carriers with smaller aircraft, such as the Medallion Foundation, which has improved small aircraft safety in Alaska through training and safety audits. The two actions experts cited most often to further improve air cargo safety were installing better technology on cargo aircraft to provide additional tools to pilots and collecting data to track small cargo carrier operations. Using flight risk checklists can also help pilots assess the accumulated risk factors associated with some cargo flights. Figure: Cargo being loaded at Anchorage International Airport, Alaska: [Refer to PDF for image: photograph] Source: GAO. [End of figure] What GAO Recommends: GAO is recommending efforts to enhance small air cargo carrier safety, including data collection of carrier operations, targeted safety programs, and flight risk assessment. FAA and NTSB provided technical comments, which were included as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-09-614] or key components. For more information, contact Gerald Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov. [End of section] Contents: Letter: Background: Air Cargo Accidents and Fatal Accidents Have Decreased, but Small Carriers Have Accounted for Higher Proportions of Both: Pilot Performance, Accumulated Risk, and Other Factors Have Contributed to Air Cargo Accidents: FAA Safety Improvement Efforts Focus Primarily on Large Carriers and, According to Experts, Vary in Effectiveness: Experts Believe Additional Improvements to Air Cargo Safety Could Be Achieved through Better Technology, Data, and Standards: Conclusions: Recommendations: Agency Comments: Appendix I: Objective, Scope, and Methodology: Appendix II: Expert Responses to GAO Survey: Appendix III: FAA's Sample Flight Risk Assessment Tool: Appendix IV: GAO Contact and Staff Acknowledgments: Tables: Table 1: Top Five Flight Risk Factors in Fatal Cargo Accidents, 1997- 2008: Table 2: Selected NTSB Recommendations Related Air Cargo Operations: Table 3: Air Cargo Expert Panelists: Figures: Figure 1: An Illustration of the Interrelationship between Large, Feeder, and Ad Hoc Carriers: Figure 2: Trends in Fatal and Non-fatal Air Cargo Accidents, 1997-2008: Figure 3: Trend in Fatal and Non-fatal Air Cargo Accidents, by Carrier Type, 1997-2008: Figure 4: Air Cargo Accidents, by Carrier Type, 1997-2008: Figure 5: Number of Fatal and Non-fatal Air Cargo Accidents, by Carrier Type, 1997-2008: Figure 6: Factors Associated with Non-fatal and Fatal Air Cargo Accidents, 1997-2008: Figure 7: The Relative Effectiveness of Current Cargo Safety Efforts as Ranked by Air Cargo Experts: Figure 8: The Relative Improvement That Potential Measures Could Have on Air Cargo Safety as Ranked by Air Cargo Experts: Abbreviations: AD: airworthiness directive: ADS-B: Automatic Dependant Surveillance Broadcast: ASAP: Aviation Safety Action Program: ATOS: Air Transport Oversight System: DOT: Department of Transportation: FAA: Federal Aviation Administration: FOQA: Flight Operations Quality Assurance: hazmat: hazardous materials: NATA: National Air Transportation Association: NPG: National Flight Standards Work Program Guidelines: NTSB: National Transportation Safety Board: PHMSA: Pipeline and Hazardous Materials Safety Administration: SMS: safety management system: TCAS: Traffic Alert and Collision Avoidance System: UPS: United Parcel Service: VDRP: Voluntary Disclosure Reporting Program: [End of section] United States Government Accountability Office: Washington, DC 20548: June 24, 2009: The Honorable James L. Oberstar: Chairman: Committee on Transportation and Infrastructure: House of Representatives: The Honorable Jerry F. Costello: Chairman: Subcommittee on Aviation: Committee on Transportation and Infrastructure: House of Representatives: The air cargo industry, which contributed over $37 billion to the U.S. economy in 2008, provides businesses and individuals with quick delivery of everything from small packages to heavy equipment. As part of the U.S. commercial aviation industry, air cargo--along with air passenger service--enjoys an extraordinary safety record. This record reflects the efforts of the Federal Aviation Administration (FAA), airlines, manufacturers, the National Transportation Safety Board (NTSB), and others to maintain one of the safest aviation systems in the world. Nevertheless, over 900 air cargo incidents and over 400 accidents occurred from 1997 to 2008, including 93 fatal accidents with 128 fatalities, raising concerns about air cargo safety. Given the economic importance of the cargo aviation sector and concerns about its safety, you asked that we examine the nature and extent of safety issues in the air cargo industry and determine what is being done to address these issues. While cargo is also carried by passenger air carriers, we focused on all-cargo (cargo-only) air carriers operating fixed-wing aircraft[Footnote 1] and addressed the following questions: 1. What have been recent trends in air cargo safety? 2. What factors have contributed to air cargo accidents in recent years? 3. What have FAA and the industry done to improve air cargo safety, and how do experts view the effectiveness of these efforts? 4. What do experts say FAA and industry could do to further improve air cargo safety? To answer these questions, we obtained and analyzed accident data from NTSB and incident, oversight, and enforcement data from FAA for all- cargo carriers operating fixed-wing aircraft. We assessed the reliability of these data and determined that they were sufficiently reliable for our purposes. We also surveyed a panel of 27 air cargo safety experts and asked them to rate and provide relative rankings on the effectiveness of current efforts to improve air cargo safety, the severity of challenges to air cargo safety, and the potential improvements that additional efforts could have on air cargo safety. With the assistance of the National Academy of Sciences, we selected the panel of experts to represent the perspectives of a cross-section of stakeholders, including pilots, carriers, manufacturers, airports, federal government, and human factors and safety performance. We also conducted site visits in Alaska, Ohio, and Texas because they were geographically diverse and as the states with the largest number of air cargo accidents or because of the relatively large number of air cargo carriers of various sizes located there. Finally, we analyzed documents and interviewed officials from FAA, NTSB, and air cargo industry organizations and an employee group. We conducted our review from December 2007 through June 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. More details of our scope and methodology can be found in appendix I and the experts' responses to our survey can be found in appendix II. Background: The U.S. economy depends on the air cargo industry for the delivery of small, time-sensitive packages under 100 pounds, freight of 100 pounds or more, and mail. Air cargo carriers fall into two distinct categories: (1) all-cargo carriers that transport only cargo and (2) passenger carriers that transport cargo as belly freight in passenger aircraft. For the most part, all-cargo carriers can be categorized according to three business models: (1) large carriers, such as United Parcel Service (UPS) and FedEx, which operate large narrow-body and wide-body aircraft under part 121 of federal aviation regulations; [Footnote 2] (2) feeder carriers, which operate midsize and small aircraft (e.g., Cessna Caravans, Mitsubishi MU-2B-60s) under part 135 or part 121 on regularly scheduled flights in support of large cargo carriers; and (3) ad hoc carriers, which operate small aircraft (e.g., Cessna 401s, Beech Bonanzas) under part 135 and are individually contracted to haul cargo out of smaller airports while not necessarily operating on a regular schedule.[Footnote 3] Throughout this report, we use the term "small carriers" when referring to both feeder carriers and ad hoc carriers. Some carriers operate under both part 121 and part 135, and one large carrier leases aircraft to small carriers to provide feeder operations. (See figure 1 for an illustration of a large carrier feeder-ad hoc relationship.) FAA estimated that as of May 6, 2009, the large all-cargo fleet contained 471 narrow-body and 593 wide-body aircraft and the small carrier all-cargo fleet contained 1,515 aircraft. Figure 1: An Illustration of the Interrelationship between Large, Feeder, and Ad Hoc Carriers: [Refer to PDF for image: three photographs] Photograph #1: Cargo arrives on large carriers into Anchorage International Airport, Alaska. Photograph #2: Cargo travels by large or feeder carriers from Anchorage into Bethel, Alaska. Photograph #3: At Bethel Airport, the cargo is reloaded onto smaller ad- hoc carriers for distribution to surrounding villages. Source: GAO. [End of figure] Several federal transportation agencies play significant roles in air cargo safety. These agencies are FAA, the Department of Transportation's (DOT) Pipeline and Hazardous Materials Safety Administration (PHMSA), and NTSB. Two FAA offices in particular have important responsibility for the oversight of air cargo carriers. First, FAA's Flight Standards Service oversees cargo carrier operations conducted under parts 121 and 135. For each carrier, Flight Standards assembles a team of inspectors (known as a "certificate management team") led by principal inspectors who focus on avionics, maintenance, or operations. For large carriers, dedicated teams of inspectors use the risk-based Air Transport Oversight System (ATOS) to carry out their duties. Under ATOS, inspectors develop surveillance plans for each carrier based on data analysis and risk assessment, and adjust the plans periodically in accordance with inspection results. For feeder and ad hoc carriers operating under part 135, inspectors--who unlike ATOS's dedicated inspection teams may be assigned to multiple air carrier and other certificates--use the National Flight Standards Work Program Guidelines (NPG) to ensure that carriers comply with safety regulations.[Footnote 4] For NPG, Flight Standards annually identifies a minimum set of required inspections to be undertaken. In addition, individual inspectors determine annual sets of planned inspections based on their knowledge of and experience with the carriers they oversee. Second, FAA's Office of Security and Hazardous Materials enforces hazardous materials (hazmat) safety policies and conducts annual inspections of cargo and passenger carriers. Inspectors in this office work exclusively on issues related to compliance with hazmat requirements. When violations of statutory and regulatory requirements are identified, FAA has a variety of enforcement tools at its disposal with which to respond, including administrative and legal sanctions. In addition, PHMSA ensures the safe transport of hazmat by air and other modes. PHMSA promulgates regulations concerning the types and amounts of hazmat that can or cannot be transported by air--often differentiating between what hazmat can be carried on all-cargo versus passenger aircraft--and maintains its own database of hazmat incidents as well as a portal that pulls together hazmat data from other databases. NTSB investigates and determines a probable cause for each U.S. aviation accident, which is defined as "an occurrence associated with the operation of an aircraft which takes place between the time any person boards the aircraft with the intention of flight and all such persons have disembarked, and in which any person suffers death or serious injury, or in which the aircraft receives substantial damage." [Footnote 5] NTSB makes transportation safety recommendations to federal, state, and local agencies and private organizations to reduce the likelihood of recurrences of transportation accidents but has no authority to enforce its recommendations. NTSB also conducts annual reviews of aircraft accident data and determines U.S. aviation accident and fatal accident rates. NTSB also periodically holds public hearings and forums, and issues special studies on various transportation safety topics. Air Cargo Accidents and Fatal Accidents Have Decreased, but Small Carriers Have Accounted for Higher Proportions of Both: From 1997 through 2008, air cargo accidents and fatal accidents each declined by about two-thirds. Despite this decline, small cargo carriers consistently experienced the largest shares of accidents and, especially, fatal accidents. Air Cargo Accidents and Fatal Accidents Declined from 1997 through 2008: Annually, air cargo accidents decreased 63 percent, from 62 in 1997 to 23 in 2008. Average annual air cargo accidents declined from the first to the second half of our review period, from an average of 45 accidents per year from 1997 through 2002 to an average of 28 accidents per year from 2003 through 2008.[Footnote 6] Fatal air cargo accidents also decreased over our 12-year review period, falling from 12 in 1997 to 4 in 2008. In addition, from the first to the second half of our review period, fatal cargo accidents dropped from an average of 10 per year to an average of 6 per year. The fluctuation in annual air cargo accidents could be the result of a number of factors, including the general decline in aviation activity after September 11, 2001; a fluctuation in overall U.S. aviation accidents; and other factors. (See figure 2.) Figure 2: Trends in Fatal and Non-fatal Air Cargo Accidents, 1997-2008: [Refer to PDF for image: stacked vertical bar graph] Calendar year: 1997; Fatal accidents: 12; Non-fatal accidents: 50; Total: 62. Calendar year: 1998; Fatal accidents: 10; Non-fatal accidents: 32; Total: 42. Calendar year: 1999; Fatal accidents: 8; Non-fatal accidents: 31; Total: 39. Calendar year: 2000; Fatal accidents: 11; Non-fatal accidents: 40; Total: 51. Calendar year: 2001; Fatal accidents: 10; Non-fatal accidents: 34; Total: 44. Calendar year: 2002; Fatal accidents: 8; Non-fatal accidents: 25; Total: 33. Calendar year: 2003; Fatal accidents: 6; Non-fatal accidents: 34; Total: 40. Calendar year: 2004; Fatal accidents: 11; Non-fatal accidents: 19; Total: 30. Calendar year: 2005; Fatal accidents: 5; Non-fatal accidents: 28; Total: 33. Calendar year: 2006; Fatal accidents: 5; Non-fatal accidents: 17; Total: 22. Calendar year: 2007; Fatal accidents: 3; Non-fatal accidents: 21; Total: 24. Calendar year: 2008; Fatal accidents: 4; Non-fatal accidents: 19; Total: 23. Source: GAO analysis of NTSB data. Note: During this period, there were 443 accidents, including 93 fatal accidents. [End of figure] Ad hoc carriers experienced the largest decline in accidents, with 28 fewer accidents--dropping from 36 to 8--followed by large carriers, with 9 fewer accidents--dropping from 14 to 5 from 1997 through 2008. Feeder carrier accidents fluctuated during this period, reaching a high of 17 accidents in 2003 compared to a low of 7 accidents both in 1999 and in 2007. We do not know why the spike occurred in 2003. Large carriers had 3 fatal accidents during our review period, which occurred in 1997, 2000, and 2004. (See figure 3.) Without actual data on the number of flight hours, however, we cannot determine an accident rate, and thus we do not know if the decline in ad hoc carrier accidents represents a better safety record for that sector of the air cargo industry. Figure 3: Trend in Fatal and Non-fatal Air Cargo Accidents, by Carrier Type, 1997-2008: [Refer to PDF for image: three line graphs] Year: 1997; Large carrier, fatal: 1; Large carrier, non-fatal: 13; Feeder carriers (small), fatal: 1; Feeder carriers (small), non-fatal: 9; Ad hoc carriers (small), fatal: 8; Ad hoc carriers (small), non-fatal: 28. Year: 1998; Large carrier, fatal: 0; Large carrier, non-fatal: 15; Feeder carriers (small), fatal: 4; Feeder carriers (small), non-fatal: 4; Ad hoc carriers (small), fatal: 6; Ad hoc carriers (small), non-fatal: 12. Year: 1999; Large carrier, fatal: 0; Large carrier, non-fatal: 9; Feeder carriers (small), fatal: 1; Feeder carriers (small), non-fatal: 6; Ad hoc carriers (small), fatal: 7; Ad hoc carriers (small), non-fatal: 16. Year: 2000; Large carrier, fatal: 1; Large carrier, non-fatal: 9;; Feeder carriers (small), fatal: 5; Feeder carriers (small), non-fatal: 8; Ad hoc carriers (small), fatal: 5; Ad hoc carriers (small), non-fatal: 22. Year: 2001; Large carrier, fatal: 0; Large carrier, non-fatal: 11; Feeder carriers (small), fatal: 3; Feeder carriers (small), non-fatal: 11; Ad hoc carriers (small), fatal: 6; Ad hoc carriers (small), non-fatal: 12. Year: 2002; Large carrier, fatal: 0; Large carrier, non-fatal: 5; Feeder carriers (small), fatal: 5; Feeder carriers (small), non-fatal: 5; Ad hoc carriers (small), fatal: 3; Ad hoc carriers (small), non-fatal: 15. Year: 2003; Large carrier, fatal: 0; Large carrier, non-fatal: 4; Feeder carriers (small), fatal: 3; Feeder carriers (small), non-fatal: 14; Ad hoc carriers (small), fatal: 3; Ad hoc carriers (small), non-fatal: 16. Year: 2004; Large carrier, fatal: 1; Large carrier, non-fatal: 5; Feeder carriers (small), fatal: 3; Feeder carriers (small), non-fatal: 7; Ad hoc carriers (small), fatal: 8; Ad hoc carriers (small), non-fatal: 7. Year: 2005; Large carrier, fatal: 0; Large carrier, non-fatal: 7; Feeder carriers (small), fatal: 2; Feeder carriers (small), non-fatal: 7; Ad hoc carriers (small), fatal: 3; Ad hoc carriers (small), non-fatal: 14. Year: 2006; Large carrier, fatal: 0; Large carrier, non-fatal: 4; Feeder carriers (small), fatal: 3; Feeder carriers (small), non-fatal: 5; Ad hoc carriers (small), fatal: 2; Ad hoc carriers (small), non-fatal: 8. Year: 2007; Large carrier, fatal: 0; Large carrier, non-fatal: 2; Feeder carriers (small), fatal: 1; Feeder carriers (small), non-fatal: 6; Ad hoc carriers (small), fatal: 2; Ad hoc carriers (small), non-fatal: 13. Year: 2008; Large carrier, fatal: 0; Large carrier, non-fatal: 5; Feeder carriers (small), fatal: 3; Feeder carriers (small), non-fatal: 7; Ad hoc carriers (small), fatal: 2; Ad hoc carriers (small), non-fatal: 6. Trends, 1997-2008: Large carrier: decrease of 9; Feeder carriers: fluctuated; Ad hoc carriers: decrease of 28. Source: GAO analysis of NTSB data. Note: During this period, there were 443 cargo accidents, including 93 fatal accidents, that included 92 large carrier accidents, 123 feeder carrier accidents, and 224 ad hoc carrier accidents. [End of figure] Small Cargo Carriers Had Higher Proportions of Accidents and Nearly All Fatal Accidents: The small carriers (feeders and ad hoc) in our review experienced 79 percent of the air cargo accidents. Ad hoc carriers accounted for about half of accidents while the feeders were involved in over a quarter of them. (See figure 4.) Feeder and ad hoc carriers averaged 29 accidents per year while large carriers averaged 8 accidents each year. Figure 4: Air Cargo Accidents, by Carrier Type, 1997-2008: [Refer to PDF for image: pie-chart] Ad hoc (small): 51%; Feeders (small): 28%; Large: 21%. Source: GAO analysis of NTSB data. [End of figure] Small air cargo carriers accounted for 96 percent of the fatal air cargo accidents that occurred from 1997 through 2008. Ad hoc carriers accounted for the majority of fatal accidents and feeders for over one- third of fatal accidents. (See figure 5.) Together, feeder and ad hoc carriers averaged 8 fatal accidents per year while large cargo carriers experienced a total of 3 fatal accidents from 1997 through 2008. Figure 5: Number of Fatal and Non-fatal Air Cargo Accidents, by Carrier Type, 1997-2008: [Refer to PDF for image: vertical bar graph] Large, fatal accidents: 3; Large, non-fatal accidents: 89; Feeders (small), fatal accidents: 34; Feeders (small), non-fatal accidents: 89; Ad hoc (small), fatal accidents: 55; Ad hoc (small), non-fatal accidents: 168. Source: GAO analysis of NTSB data. Note: During this period, there were 443 accidents, including 93 fatal accidents, but we were unable to categorize all accidents due to lack of information. [End of figure] Accident Rates Have Fluctuated for Large Cargo Carriers but Are Not Available for Small On-demand Cargo Carriers: The accident rate per departure for large air cargo carriers has fluctuated over the last 25 years, but the overall trend has been downward, and in 2007 was roughly the same as for passenger carriers. [Footnote 7] It is possible to calculate these rates because FAA requires those carriers to report operational data (e.g., flight hours or departures). However, FAA does not require small on-demand carriers operating under part 135 to report operational information, and the majority of feeder and ad hoc cargo carriers fall into this group. In 2003, NTSB recommended that FAA collect this type of data and, according to NTSB, FAA is still reviewing the costs and benefits as well as options for collecting and processing the data.[Footnote 8] However, the lack of data about the flight hours for small on-demand carriers precludes calculation of the industry's current accident or fatality rates or changes in the rates over time, making it difficult to determine whether the industry is becoming more or less prone to accidents. Instead, FAA relies on an annual survey of aircraft owners to form the basis for estimates of small carrier operations, but this survey does not distinguish between passenger and cargo operations, making it impossible to use the survey estimates to calculate cargo or passenger accident rates for on-demand operations or for the cargo industry as a whole. Even though operational data are not available for small air cargo carriers, their fatal accident rates would exceed those of large carriers for the latter part of our review period. From 2005 through 2008, there were no fatal accidents among large cargo carriers, so their fatal accident rate for those years was zero. However, there were 17 fatal feeder and ad hoc accidents over the same period, meaning that their fatal accident rate, if it could be determined, would be higher than zero--though it is unclear how much higher. This logic would not hold for air cargo accidents in general because large carriers had accidents in each year from 1997 through 2008, though they had fewer accidents than the feeder and ad hoc carriers. The lack of data makes it difficult for FAA and industry to target further improvements to the areas with the highest risk. Pilot Performance, Accumulated Risk, and Other Factors Have Contributed to Air Cargo Accidents: Our review of NTSB and FAA air cargo accident and incident data as well as our interviews with industry officials and analyses of industry documents revealed that pilot performance was a prominent factor in air cargo accidents. Additionally, we concluded that accumulated risk, challenging operating conditions in Alaska, and undeclared hazmat were also prominent contributors to air cargo accidents. Pilot Performance Was a Major Contributor to Air Cargo Accidents from 1997 through 2008: Our review of NTSB reports for 417 completed air cargo accident investigations found that pilot performance was cited as the probable cause for about 59 percent of them.[Footnote 9] Specifically, we found that NTSB cited pilot performance as the probable cause for about 53 percent of non-fatal and about 80 percent of fatal air cargo accidents. (See figure 6.) Examples of pilot performance issues in these accidents included the pilot's failure to maintain control of the aircraft or to execute the appropriate procedure. Our review determined that the second most prominent cause of air cargo accidents was some type of equipment failure or malfunction. Figure 6: Factors Associated with Non-fatal and Fatal Air Cargo Accidents, 1997-2008: [Refer to PDF for image: two pie-charts] Non-fatal accidents (330): Pilot performance: 54%; Equipment: 26%; Other: 21%. Fatal accidents (87): Pilot performance: 80%; Equipment: 13%; Other: 7%. Source: GAO analysis of NTSB data. Note: NTSB had not yet completed its investigations for 26 of the 443 air cargo accidents--including 6 fatal accidents. Other causes include weather and wildlife. [End of figure] Pilots of small cargo aircraft have fewer human and other resources available to them to help avoid mistakes or recover from unexpected circumstances. Typically, there is no second pilot to share in the pilot's many duties[Footnote 10] and help respond to emergencies. Eighty-one percent of the fatal air cargo accidents from 1997 through 2008 were single-pilot flights. The lack of a second pilot coupled with the many duties of a single pilot merits a mention of the issue of pilot fatigue. Although NTSB indicated fatigue as a contributing factor--not a probable cause--in just 4 of the 443 accidents in our data, 12 of 27 experts we surveyed ranked pilot fatigue as one of the three most serious challenges to safe air cargo operations. The view of the experts is not out of line with the accident record to the extent that the concern about pilot fatigue has led to vigilance in identifying and addressing fatigue issues. Further compounding the lack of pilot resources, cargo aircraft operated under part 135 are not required to have on-board safety technology such as a traffic collision and avoidance system, a terrain awareness and warning system, or an autopilot,[Footnote 11] which could aid a single pilot in monitoring the environment or responding to changing weather conditions. Most of these systems are required for small passenger aircraft that also operate under part 135. Additionally, small cargo aircraft may fly into airports where FAA does not provide air traffic control services at all hours and the airports offer fewer services than might be required for passenger operations. For example, at the Bethel, Alaska, airport--the transportation hub for the remote villages in the area and the third-busiest airport in Alaska--FAA provides air traffic control services from 7 a.m. to 8 p.m. from November to March and 2 hours later in other months, and the airport clears its runway of snow and staffs its aircraft rescue and fire-fighting equipment only during operations of passenger aircraft with more than 30 passenger seats. Our Analysis Indicates That Most Fatal Accidents Involved Multiple Risk Factors: We analyzed NTSB reports of the 93 fatal air cargo accidents that occurred from 1997 through 2008 using FAA's Flight Risk Assessment Tool and identified three or more risk factors in 63 of the accidents and four or more risk factors in 41 accidents. FAA's tool is located in appendix III and includes 38 risk factors--in the areas of pilot qualifications and experience, operating environment, and equipment-- each with an assigned value ranging from 2 to 5, with 5 indicating the highest risk.[Footnote 12] While we do not know how the presence of these risk factors differs from their occurrence during normal operations, the experts told us that the unrecognized accumulation of multiple risk factors can create a potentially dangerous situation. One 1997 fatal accident, which NTSB attributed to the pilot's disregard of the preflight weather briefing for severe weather, involved six risk factors. The pilot had not flown a minimum number of hours during the previous 90 days, had not accumulated a minimum amount of experience flying the aircraft type involved in the accident, and was flying solo. Additionally, the pilot encountered severe turbulence and icing during the night flight. Table 1 lists the five most common risk factors we identified in the 93 fatal air cargo flights. Table 1: Top Five Flight Risk Factors in Fatal Cargo Accidents, 1997- 2008: Rank: 1; Risk factor: Single-pilot flight; Risk value[A]: 5; Number of fatal air cargo accidents in which factor was present: 82. Rank: 2; Risk factor: Night flight; Risk value[A]: 5; Number of fatal air cargo accidents in which factor was present: 48. Rank: 3; Risk factor: Winter operation; Risk value[A]: 3; Number of fatal air cargo accidents in which factor was present: 37. Rank: 4; Risk factor: Low pilot experience in accident aircraft type (less than 200 hours)[B]; Risk value[A]: 5; Number of fatal air cargo accidents in which factor was present: 23. Rank: 5; Risk factor: Low pilot flying experience last 90 days (less than 100 hours)[B]; Risk value[A]: 3; Number of fatal air cargo accidents in which factor was present: 17. Source: GAO analysis of NTSB. [A] The tool specifies 38 risk factors--in the areas of pilot qualifications and experience, operating environment, and equipment-- and each is assigned a risk value ranging from 2 to 5, with the higher values indicating greater risk. [B] These numbers may actually be higher because the NTSB reports did not list the pilot's flight hours in the last 90 days for 22 of the accident pilots and total flying time in the accident aircraft type for 17 of the accident pilots. [End of table] Alaska's Challenging Operating Conditions Factored Prominently in Air Cargo Accidents: With 18 fatal air cargo accidents from 1997 through 2008, Alaska led all states in this statistic because aviation operations in that state face several unique challenges. Alaska is more dependent on aviation for the transport of goods and people than other states because it lacks a comprehensive road system.[Footnote 13] Less than 10 percent of the state is accessible by road. Therefore, goods must be transported to remote villages via air or barge, and barge transport is not an option during the winter months. These factors make Alaska highly dependent on cargo aircraft, which often fly into poorly maintained airports that often do not meet FAA standards. Consequently, most of the accidents in Alaska involved small aircraft. Alaska is also subject to unusual weather conditions. Taken together, we believe these challenges render Alaska more susceptible to aviation accidents and fatal accidents than other states. Of the 27 experts we surveyed, 5 ranked operating conditions in Alaska as one of the top three challenges to the safe operation of cargo flights; in addition, 12 experts in our panel indicated that Alaskan operating conditions do pose at least moderate challenges to safety, whereas 2 experts (a pilot of large aircraft and a government official) said these conditions were not a challenge. Seven experts said they did not have enough specific knowledge to judge the degree of challenge that Alaskan operating conditions pose to safety. Some Experts Indicated that One of the Most Serious Challenges to Safe Air Cargo Operations Is Undeclared Hazmat, but Few Accidents Have Been Attributed to Hazardous Cargo: Very few of the cargo accidents occurring from year to year were conclusively caused by hazmat. However, 11 experts on our panel ranked undeclared hazmat--materials not noted as hazardous in shipping documents and/or labeled as such on their packaging---among the greatest challenges to safe cargo operations, second only to pilot fatigue. According to our review of the NTSB accident data, only three cargo accidents involved hazmat. They occurred in 1997, 1998, and 2006 to large carriers, and none resulted in fatalities.[Footnote 14] The 2006 accident resulted in an NTSB hearing and recommendations that we discuss later in this report. The problem of undeclared hazmat was cited primarily by government and large carrier experts in our survey. Specifically, three of the four government experts and two of the three large carrier experts cited it as the most serious challenge to air cargo safety. These opinions may stem from the previously cited fires and the relative rarity of a destroyed aircraft among large carriers, as well as government concerns about the transport of lithium batteries on aircraft. FAA hazmat officials told us that undeclared shipments of hazardous materials represent the biggest challenge they face and that lithium batteries are the most challenging type of hazmat in air transportation. We reported in January 2003 that FAA, in the early 1990s, identified a number of incidents associated with batteries, particularly lithium batteries, aboard aircraft in which the batteries caused fires, smoke, or extreme heat.[Footnote 15] In response to these and other concerns, DOT took a number of actions designed to strengthen the regulations for the transportation of lithium batteries.[Footnote 16] In January 2008, NTSB noted that lithium batteries had been involved in at least 9 aviation incidents, and both primary and secondary lithium batteries are regulated as hazardous materials for the purposes of transportation.[Footnote 17] In December 2007, NTSB made six recommendations to PHMSA following a UPS aircraft fire at Philadelphia International Airport in February 2006, in which a number of secondary lithium batteries were found in the accident debris. [Footnote 18] The recommendations included requirements for transporting primary lithium batteries in fire-resistant containers and stowing cargo containing secondary lithium batteries in crew-accessible locations so that any fire hazards can be quickly addressed. These recommendations remain open with acceptable responses from PHMSA because they have not yet been implemented, but according to NTSB, actions are planned that, if satisfactorily completed, may comply with the safety recommendations. FAA and PHMSA have embarked on a lithium battery action plan, which aims to reduce the risk associated with the transport of batteries on aircraft by passengers and as cargo. The primary focus of this plan is all types of lithium batteries. According to DOT, PHMSA and FAA have also initiated a rule-making project to consider additional measures to enhance the safety of lithium battery shipments such as packaging, hazard communication, and stowage requirements. PHMSA plans to publish a notice of proposed rulemaking by December 2009. FAA, the Air Line Pilots Association, and PHMSA also issued safety alerts or advisories in 2007 that addressed smoke and fire hazards, recommended crew actions in the event of a battery fire, noted the availability of guidance for the safe transport of batteries and battery-powered devices on board aircraft, and provided information on proper packing and handling procedures for these batteries.[Footnote 19] Large Numbers of Air Cargo Incidents May Be Precursors to Accidents: Although our analysis for this study included 443 air cargo accidents, cargo carriers were involved in more than twice as many incidents during the first 11 years of our review period, and FAA and others recognize that incidents are potential precursors to more serious accidents. In an analysis of air cargo data for 1997 through 2007 from FAA's Accident/Incident Data System, we identified over 900 air cargo incidents.[Footnote 20] These incidents covered a broad set of events, such as an engine losing power at 7,000 feet; a cargo door opening in flight; and an aircraft engine coming into contact with a fuel truck. FAA does not use incident data to identify precursors to aviation accidents, because the data were not developed for this purpose. However, the agency is moving toward using data to better identify precursors to accidents, but until it does so, it may be missing opportunities to make air cargo operations and aviation, in general, safer.[Footnote 21] For example, from 2000 to 2007, one ad hoc cargo carrier was listed in FAA's database 10 times with incidents that resulted in varying degrees of damage to its aircraft, from none to substantial, and in NTSB's database with one non-fatal accident. This carrier subsequently experienced a fatal accident in 2008. Had this carrier's incident data been used to identify accident precursors, inspectors might have been alerted to underlying problems that might have been addressed, potentially preventing the subsequent fatal accident. In addition, NTSB's accident database also does not track incidents in a way that would allow empirical analysis. The notion that incidents can be precursors to more serious accidents is accepted both inside and outside aviation. NASA's Aviation Safety Reporting System collects, analyzes, and responds to aviation safety incident reports voluntarily submitted by pilots and others to lessen the likelihood of aviation accidents. In its 2005 Safety Management Manual, the International Civil Aviation Organization noted that for accidents, there are precursors evident before the accident, and focusing solely on instances of serious injury or significant damage is a wasted opportunity, since the factors contributing to such accidents may be present in hundreds of incidents. The International Civil Aviation Organization further noted that "effective safety management requires that staff and management identify and analyze hazards before they result in accidents," particularly since there is the opportunity "to identify why the incidents occurred and, equally, how the defenses in place prevented them from becoming accidents." The National Academy of Engineering undertook an accident precursor project in February 2003, which culminated in a report that included aviation accident precursor analysis and management. The report concluded that existing initiatives are not as effective as they could be and encourages government agencies that regulate high-hazard industries to increase their support of research into methods for effectively analyzing and managing precursors.[Footnote 22] FAA Safety Improvement Efforts Focus Primarily on Large Carriers and, According to Experts, Vary in Effectiveness: Many government and industry efforts to improve safety focus primarily on large carriers. Such efforts include programs in which carriers and employees voluntarily disclose potential safety issues, attempts by carriers to institutionalize their safety procedures through safety management systems (SMS)--a proactive, risk-based approach to addressing potential hazards--and FAA's ATOS oversight program even though there is nothing intrinsic to preclude these concepts from being implemented among small cargo carriers. Cargo experts view voluntary disclosure programs, efforts by associations to improve their members' safety procedures, and carrier-implemented SMSs as the most effective current safety programs affecting air cargo (see figure 7). Figure 7: The Relative Effectiveness of Current Cargo Safety Efforts as Ranked by Air Cargo Experts: [Refer to PDF for image: stacked horizontal bar graph] Safety effort: Voluntary disclosure programs; Greatest improvement: 7; Second greatest improvement: 5; Third greatest improvement: 4; Total: 16. Safety effort: Association efforts with members; Greatest improvement: 3; Second greatest improvement: 6; Third greatest improvement: 5; Total: 14. Safety effort: Carrier safety management systems; Greatest improvement: 6; Second greatest improvement: 3; Third greatest improvement: 4; Total: 13. Safety effort: FAA airworthiness directives and operations specifications; Greatest improvement: 7; Second greatest improvement: 1; Third greatest improvement: 2; Total: 10. Safety effort: NTSB recommendations; Greatest improvement: 1; Second greatest improvement: 2; Third greatest improvement: 6; Total: 9. Safety effort: FAA oversight; Greatest improvement: 2; Second greatest improvement: 3; Third greatest improvement: 0; Total: 5. Safety effort: Association lobbying efforts; Greatest improvement: 1; Second greatest improvement: 1; Third greatest improvement: 3; Total: 5. Safety effort: NTSB public meetings; Greatest improvement: 0; Second greatest improvement: 4; Third greatest improvement: 1; Total: 5. Safety effort: Airport safety management systems; Greatest improvement: 0; Second greatest improvement: 1; Third greatest improvement: 2; Total: 3. Safety effort: FAA Infos, seminars, and other communication efforts; Greatest improvement: 0; Second greatest improvement: 1; Third greatest improvement: 0; Total: 1. Source: GAO. [End of figure] Experts Rate Voluntary Disclosure Programs as One of the Most Effective Cargo Safety Programs, but Primarily Large Carriers Participate: The intent of voluntary disclosure programs is to identify and correct safety problems in a nonpunitive way and to provide additional safety information to FAA. Our panel of experts ranked FAA's voluntary disclosure programs as the most effective current program for improving air cargo safety. Specifically, 16 of 27 experts ranked FAA's voluntary disclosure programs as one of the most effective current efforts to improve air cargo safety, and all experts able to judge indicated that the programs were effective on some level. FAA operates multiple voluntary disclosure programs, which use different data sources to help identify safety deficiencies. The three major ones are Flight Operations Quality Assurance (FOQA), the Aviation Safety Action Program (ASAP), and the Voluntary Disclosure Reporting Program (VDRP). At the current time, FOQA is used only by large carriers because of the level of technology required and ASAP is not typically used by small carriers. * FOQA collects and makes available for analysis digital flight data generated during the normal operations of the 23 participating carriers. As of January 2008, the only cargo carriers participating in FOQA were large carriers. Participating carriers pay for the special flight data recorders that can record FOQA data; these recorders cost approximately $20,000 each. Although such an investment can be expensive for some air carriers, some aircraft models come with the data recorder already built in. However, smaller carriers tend to operate older aircraft, which lack the data recorder equipment. * ASAP encourages industry employees to report safety information that may be critical in identifying potential precursors to accidents. Under this program, employees of air cargo carriers and other participating entities report safety events, which a committee that includes the carrier, the employee labor group, and FAA then reviews and determines appropriate corrective actions, such as remedial training. FAA agrees not to pursue enforcement actions for safety violations reported exclusively under this program.[Footnote 23] As of December 2008, 73 carriers participated in ASAP, including 8 cargo carriers. Seven of these 8 cargo carriers were large carriers, possibly because large carriers are more likely than small carriers to have the time and resources required for participation. Officials from an ad hoc carrier we interviewed said it was not practical for their carrier to enter into an agreement with FAA and then organize meetings to discuss disclosures when the carrier could operate an informal safety issue disclosure program internally. * VDRP encourages regulated entities, such as air carriers or repair stations, to voluntarily report instances of regulatory noncompliance. FAA does not take legal action on VDRP disclosures, but a violation with the same root cause can be reported only once by a carrier. Cargo carriers of all types we interviewed indicated that they have participated or would participate in VDRP. However, a 2008 DOT Blue Ribbon Panel found FAA does not routinely analyze VDRP data to identify trends and patterns that could indicate safety risks.[Footnote 24] FAA noted in commenting on a draft of this report that it began conducting regular analysis of VDRP data in January 2009 and that it modified the VDRP data software system and associated guidance to enable the identification of national trends of disclosures that represent the highest risk to safety. Numerous Other Initiatives Including Industry Association Efforts Are Aimed at Improving Safety for Carriers of All Sizes; Some Experts Believe These Are among the Most Effective at Improving Air Cargo Safety: Numerous industry efforts to improve safety are aimed at different sectors of the air cargo industry. These include efforts by membership associations to improve safety among their members, which 14 of the 27 experts on our panel ranked among the top three most effective current efforts to improve air cargo safety--second only to voluntary disclosure. In addition, all experts in a position to comment indicated that these association efforts were at least slightly effective, and 11 of those experts indicated that they were greatly effective. We were unable to identify a central clearinghouse for these association efforts, but the ones we did identify fell into the following six general categories: establishing SMSs, providing fatigue awareness training, providing pilot skills training, adding on-board safety systems, improving flight risk assessments, and providing cargo- specific aircraft rescue and fire-fighting training. One expert said that membership-based efforts are often the most effective because they directly reflect the voluntary priorities of the membership organization and are often directly tailored to the group's specific needs. Examples of industry efforts, including some by industry associations and one joint industry-federal effort, follow. * The Regional Air Cargo Carriers Association--an association of primarily feeder cargo carriers--has developed an SMS template tailored to the needs of smaller, feeder cargo carriers. A Regional Air Cargo Carriers Association official said that the organization's members found FAA's SMS guidance appropriate for large carriers with safety departments, but less useful for feeder or ad hoc carriers that might have only a few employees. SMSs are considered in the international community to be an important way to improve safety in aviation operations and are required by the International Civil Aviation Authority. In commenting on a draft of this report, FAA indicated that it has not endorsed the Regional Air Cargo Carriers Association's SMS program, nor does FAA believe it is consistent with FAA training and program guidelines for SMS. SMS is discussed in more detail later in this report. * The National Air Transportation Association (NATA)--an association primarily of general aviation service companies whose members include some feeder and ad hoc carriers--has tailored flight risk assessments to the needs of its members by automating much of the assessment process. NATA officials said that feeder or ad hoc cargo carriers, like many general aviation operators, do not have the support personnel that larger carriers have to help with preflight checklists and other tasks, and that risk assessments would just add to those tasks unless they were largely automated. NTSB has recommended that a segment of the part 135 community--emergency medical services--utilize flight risk assessments before accepting flights. * In some cases, large carriers act as membership associations by helping their feeder network acquire safety enhancements. For example, the Federal Express feeder program, which helps finance on-board safety enhancements on feeder aircraft, has reduced the number of accidents among its feeder network according to Federal Express officials. * Since 2002, Alaska's Medallion Foundation--a federally-funded, safety promotion organization that is overseen by FAA--offers training for part 135 pilots and has developed safety audits that can lead to carrier certifications in various areas, such as operational risk management, maintenance and ground service, and internal evaluation, to improve air transportation safety in Alaska. According to FAA, it has also approved a modified protocol for ASAP administered centrally by the Medallion Foundation in order to increase the feasibility of ASAP for small operators in Alaska. Meetings with FAA personnel to review reports submitted under the program are conducted by telephone, and all reports are tracked in a central database according to the agency. DOT said that the modified ASAP protocol with the Medallion Foundation has worked well for enabling small, remotely situated operators in Alaska to participate in the program. * The Commercial Aviation Safety Team, a joint FAA-industry effort, has developed an integrated, data-driven strategy to reduce the commercial aviation fatality risk in the United States and promote new government and industry safety initiatives throughout the world. The Team has completed work on 40 of its 65 safety enhancements aimed at eliminating accident causes. Six of the safety enhancements specifically target cargo operations, and each of them is still under way as of the Team's most recent update in May 2007. * The Dallas-Fort Worth International Airport is developing a curriculum and a cargo-specific aircraft rescue and fire-fighting training course. Airport officials said that the course will focus on the unique challenges and approaches to fighting a cargo fire. For example, airport officials said that many aircraft rescue and fire- fighting teams treat passenger/cargo fires and cargo fires similarly when they should be treated differently, because passenger/cargo fires can involve hundreds of people whereas cargo fires typically only endanger the flight crew. The course will also provide hands-on training in the use of cargo-specific fire-fighting tools, such as hull- penetrating tools or devices for locating hot spots--tools that most fire fighters rarely use. Carrier-Based SMSs Are Viewed as Effective by Experts but Are Currently Geared toward Large Carriers: Thirteen cargo experts ranked carrier SMSs as one of the most effective current efforts to improve air cargo safety, making this the third most frequently cited current safety effort. SMSs can differ in their specifics, but FAA defines SMS as a proactive, risk-based approach to addressing potential hazards by categorizing the risk level and taking appropriate mitigating actions to reduce the risk to an acceptable level. Some countries, such as Canada, require carriers to implement an SMS, but the United States does not yet require domestic carriers to do so. FAA and industry officials agreed that FAA will require part 121 carriers to implement SMSs in the next few years. In addition, FAA has issued guidance on developing an SMS, but none of the air cargo carriers we interviewed have implemented one. However, several of the air cargo carriers we interviewed said they had safety programs that are similar to SMSs. An expert in aviation safety said that effective SMSs are good for institutionalizing safety improvements and taking proactive steps to reduce the number of accidents. He further noted that larger companies with more airplanes and more resources are better positioned to do this. By contrast, companies with one airplane and one pilot will not have enough staff time to submit the paper work. As a result, the Regional Air Cargo Carriers Association--a membership organization for feeder carriers--has developed simplified SMS guidance specifically for part 135 cargo carriers. Some airports are also implementing SMSs. Most experts in our panel did not rank airport SMSs among the most effective current efforts, possibly because cargo airports have not implemented them nationwide. FAA Uses Airworthiness Directives and Operations Specifications to Improve Aviation Safety: FAA uses airworthiness directives (AD) and operations specifications to improve aviation safety. An AD is a notification to owners and operators of aircraft that a particular model of aircraft, engine, avionics, or other system has a known safety deficiency that must be corrected. Carriers are prohibited from operating any aircraft that is out of compliance with any applicable AD. Operations specifications are specific limits and requirements developed for individual operators, such as the specific aircraft the carrier is allowed to operate. Ten experts on our panel ranked ADs and operations specifications among the three most effective current efforts to improve air cargo safety. Our interviews with carriers showed that some carriers depend on ADs and operations specifications to learn about safety issues that other carriers have discovered. An official from one feeder carrier said that ADs are like product recalls, and without them, she would never know that there was a problem until there was an accident. FAA also uses other methods for communicating with carriers, such as informational messages, alerts, advisory circulars, and seminars. However, none of the 27 experts on our panel indicated that FAA informational materials and seminars were any more than moderately effective at improving the safety of air cargo operations. FAA Uses Risk-Based Inspection System for Large Carriers Only, and Experts Did Not Rate FAA's Oversight as One of the Most Effective Efforts to Improve Air Cargo Safety: FAA oversees the compliance with safety regulations of all 58 part 121 cargo large carriers[Footnote 25] by using ATOS, which applies a risk- based inspection system tailored to each carrier regulated under part 121. For example, under ATOS, principal inspectors develop surveillance plans for each airline based on data analysis and risk assessment, and adjust the plans periodically to reflect inspection results. Under ATOS, principal inspectors are assigned to just one part 121 carrier. Our interviews with part 121 carriers and FAA inspectors revealed mixed opinions about ATOS. Some of the carriers, particularly the smaller part 121 carriers, indicated that transitioning to ATOS was too complicated and costly and that its focus on administrative reviews has reduced the number of on-site FAA inspections they receive. In addition, some FAA inspectors said that the ATOS paperwork is time consuming and can have the effect of tethering them to their computers. For example, officials from a small part 121 air carrier said that they had to hire a full-time person to work on implementing ATOS as well as spend over $500,000 to hire a company to help revise the carrier's manuals to satisfy FAA requirements under ATOS.[Footnote 26] However, officials from a large carrier indicated that ATOS is a more effective oversight system than NPG once it is fully implemented. In addition, some officials from large carriers said that the bureaucratic nature of ATOS limited the amount of direct oversight they receive. FAA officials said that ATOS does not impose new requirements on carriers, and FAA does not require carriers to set up an ATOS program. FAA officials also said that because ATOS is more robust than the oversight system it replaced--NPG--inspectors may find omissions in manuals that were overlooked before. Carriers may then be required to correct these deficiencies to meet regulatory requirements. In addition, FAA officials said they are exploring options for reducing the time inspectors spend at their computers and increasing the time they spend doing hands-on inspections. For example, FAA officials said that FAA is reducing the number of ATOS program elements in order to make inspection planning and management easier. The 303 part 135 carriers[Footnote 27] remain under the NPG system, which requires all active carriers to be inspected at least once a year and sets the numbers of required inspections nationally and planned inspections at the local level. Some FAA inspectors we interviewed who use the NPG system said they do not base their planned inspections on risk factors but, rather, on what was done the previous year or what they have time to do (NPG inspectors typically oversee several carriers). FAA officials said that FAA is moving toward a risk-based oversight "Safety Assurance System" for part 135 carriers. FAA has completed a gap analysis that compared the existing part 135 oversight system to the system requirements of the new system, and it plans to have the risk-based system developed for part 135 carriers by 2013. The number of inspections that part 135 carriers receive each year varies greatly under NPG. For fiscal years 2004 through 2007, we found 18 part 135 carriers received one inspection in a year, while 6 part 135 carriers received hundreds of inspections in a year. An FAA official said that variation should be expected. For example, carriers with more airplanes, airplanes of more types, and routes to more regions will receive more required inspections under NPG. The number of part 135 carriers that each cargo inspector oversees also varies greatly. On the low end (bottom 10 percent), part 135 cargo inspectors oversee 5 passenger and cargo carriers on average, and on the high end (top 10 percent), inspectors oversee 34 passenger and cargo carriers. While some variation is unavoidable, FAA officials said that FAA has not established guidelines to ensure that the workload is balanced among inspectors. The National Academy of Sciences also found FAA had inadequate staffing standards for its safety inspectors. [Footnote 28] The majority of the experts on our panel did not rank FAA oversight and inspections among the most effective current efforts to improve air cargo safety. Although 5 of the 27 experts rated FAA oversight and inspections among the three most effective current efforts to improve air cargo safety, none of the experts representing carriers' perspectives listed FAA oversight among the most effective current efforts. Officials from an ad hoc cargo carrier said that FAA inspectors do not have enough specific knowledge of cargo operations to effectively oversee cargo operations. Two FAA cargo inspectors that also oversee passenger carriers said that their formal cargo oversight training consisted of a 2-hour online course. FAA officials said part 135 does not differentiate between passenger and cargo operations. However, FAA officials recognized that inspectors that oversee part 135 ad hoc operations may benefit from additional training and are revising an existing multiday course for maintenance inspectors to address cargo operations. Our analysis of oversight data for part 135 carriers showed that cargo inspectors also oversee passenger carriers, often in larger numbers. This could limit inspectors' ability to focus on cargo- specific issues. Officials from several cargo carriers of different types said that FAA inspectors do not do enough on-site inspections to effectively find and correct safety problems. For example, FAA inspectors and carrier officials said that part 135 inspectors often focus on administrative reviews. FAA officials said there needs to be an appropriate balance of on-site inspections and administrative reviews, both of which are important for determining carrier compliance and ensuring safe operations. Ultimately, however, FAA officials said that regulatory compliance is an air carrier responsibility; FAA is responsible for ensuring that air carriers are capable of complying and, in some cases, administrative reviews may be the best way for FAA to do that. FAA Rarely Responds to Cargo Carrier Regulatory Violations with Legal Action or Fines: FAA's oversight includes enforcement efforts, which are designed to promote compliance with statutory and regulatory requirements for aviation safety. When violations are identified, an FAA order calls for inspectors to take the actions most appropriate to achieve future compliance. These actions range from educational and remedial efforts, to administrative actions (such as warning notices), to punitive legal sanctions (such as fines or loss of operation certificate). Violations can be identified by FAA inspectors or by others, such as air traffic controllers or state or local government officials. The relevant FAA inspector prepares a report and recommends an enforcement action. That report and proposed enforcement action are then reviewed and possibly changed at various levels depending on the nature of the recommended enforcement action. FAA closes most air cargo regulatory violations with administrative action, such as a warning notice, or without taking any action. For 1997 through 2008, over half (56 percent) of the 6,564 enforcement actions against cargo carriers were administrative, and another 17 percent involved no action. These were very similar to the percentages for passenger carriers. Within cargo, the ad hoc sector had the lowest percentage of legal actions. Fourteen percent of ad hoc carrier enforcement actions were legal, compared with 24 percent of larger carrier enforcement actions and 16 percent of feeder carrier enforcement actions. Ad hoc cargo carriers also had the largest reductions, on average, among cargo carrier types, with the reduced fines 64 percent below the initially recommended fines. Our previous work found that FAA reduced legal actions for several reasons, including proof that the violator took corrective action to prevent a reoccurrence of the violation or economic hardship that might accrue to the entity that caused the violation. The percentages of cargo carrier violation cases closed with administrative or no action represent a continuation of trends that we observed in our last work on FAA enforcement in 2004.[Footnote 29] At that time, we found that FAA generally closes cases against passenger and cargo carriers with administrative actions, and reducing the amounts of fines may reduce the deterrent effects of those actions. Since then, our analysis of FAA's Enforcement Information System data shows that the share of violations resolved using administrative or no action has increased slightly. Moreover, FAA still lacks information on how these actions have influenced the effectiveness of the enforcement actions, and the recommendation we made in that report--that FAA develop a process for measuring the performance and effectiveness of its enforcement actions--remains open. Some NTSB Recommendations for Improving Cargo Safety Have Not Been Implemented, and the Experts Did Not Consider These Recommendations among the Most Effective Measures: NTSB investigates transportation accidents, including air cargo accidents, and makes recommendations to improve safety. NTSB made numerous recommendations based on air cargo accidents but not all of those were related specifically to cargo issues. For example, after an air cargo accident in 1997, NTSB recommended that FAA require all part 121 air carriers to include additional information and training to flight crews in order to avoid that type of accident in the future. Other NTSB recommendations, however, are cargo-specific recommendations, and all of them remain open. Table 2 summarizes the status of NTSB's cargo-specific recommendations. Table 2: Selected NTSB Recommendations Related Air Cargo Operations: Topic (year): Hazardous materials (1997); Summary of recommendation (NTSB recommendation number): The Air Transport Association should develop, in cooperation with FAA and the U.S. Postal Service, programs to educate passengers, shippers, and postal customers about the dangers of transporting undeclared hazardous materials aboard aircraft. (A-97-082); Current status: Open. NTSB has not yet responded to the Air Transport Association's 2009 correspondence, in which it asserts that the problems with undeclared hazardous materials may no longer exist as they did in 1997 due to the post-9/11 security increases and several hazardous materials awareness efforts by government and industry. Topic (year): Cargo loading (1998); Summary of recommendation (NTSB recommendation number): FAA should require improved training and advisory materials related to weight and balance, cargo handling, cargo restraint, and hazards of misleading. (A-98-047); Current status: Open. FAA has questioned its authority to mandate training for cargo handlers. Topic (year): Effects of dry ice (2001); Summary of recommendation (NTSB recommendation number): FAA should conduct testing to determine acceptable loads of dry ice and its potential effects and revise FAA advisory circulars on the issue. (A-01-014 and A-01-015); Current status: Open. FAA indicated in 2006 that it was preparing a report on its study of dry ice packed in containers commonly used in air cargo services. Topic (year): Cargo handlers (2003); Summary of recommendation (NTSB recommendation number): FAA should modify its list of safety-sensitive functions for part 121 carriers to include all personnel with access to an aircraft, including cargo handlers and others. (A-03-036); Current status: Open. FAA has not issued a final rule on the issue. Topic (year): Fire detection (2007); Summary of recommendation (NTSB recommendation number): FAA should ensure that fire detection systems account for the effects of cargo and cargo containers. (A-07-098); Current status: Open. FAA has not completed its research on the issue or revised its technical standards order. Topic (year): Fire suppression (2007); Summary of recommendation (NTSB recommendation number): FAA should require fire suppression systems be installed in the cargo compartments of all part 121 cargo aircraft. (A-07-099); Current status: Open. FAA has not established the requirement. Topic (year): Aircraft fire-fighting training (2007); Summary of recommendation (NTSB recommendation number): FAA should require airport inspectors to ensure that commercial airports with cargo operations include cargo aircraft in their aircraft rescue and fire-fighting familiarization training programs. (A-07-101); Current status: Open. FAA has not revised the training guidance. Topic (year): Aircraft fire-fighting information (2007); Summary of recommendation (NTSB recommendation number): The Cargo Airline Association in coordination with others should develop and disseminate accurate, complete aircraft emergency response diagrams to aircraft rescue and fire-fighting teams at airports with cargo operations. (A-07-110); Current status: Open. The Cargo Airline Association convened a working group of cargo carriers, aircraft manufacturers, the aircraft rescue and fire-fighting representatives, and others, and, with their input, has decided to provide aircraft rescue and fire-fighter personnel with information through video and other advanced media formats. Cargo Airline Association officials said that the association plans to complete its work and close the recommendation to the satisfaction of NTSB within the year. Topic (year): Emergency exits (2007); Summary of recommendation (NTSB recommendation number): FAA should require improvements to emergency exits on cargo aircraft. (A-07-102 and A-07-103); Current status: Open. FAA has not taken the recommended actions. Source: GAO analysis of NTSB data. [End of table] Most experts did not rank NTSB recommendations among the top efforts to improve air cargo safety. One expert on our panel described NTSB recommendations as the most effective effort to improve air cargo safety, and another 8 of the experts in our panel ranked NTSB recommendations as one of the top three efforts, but 18 experts did not include them among the top three efforts. According to our interviews, NTSB recommendations related to cargo operations are not always practical to implement. For example, one carrier noted that NTSB's recommendation related to aircraft fire-fighting information (cited above) is challenging to implement because there are many cargo aircraft configurations, and even knowledge of the configuration involved in the incident that prompted the recommendation would not have helped firefighters respond to the incident. Despite these reservations, the carrier indicated that it is helping to implement the recommendation because NTSB believes it would improve safety. NTSB also holds public hearings and meetings on topics of particular interest to transportation safety personnel. For example, it held a forum on air cargo safety in 2004, but no explicit recommendations emerged from the forum. Five experts on our panel ranked NTSB's meetings as one of the three most effective current air cargo safety efforts and none of the experts ranked the meetings as the most effective effort. Experts Believe Additional Improvements to Air Cargo Safety Could Be Achieved through Better Technology, Data, and Standards: The 27 experts in our panel commented on numerous additional steps that could further improve air cargo safety. Experts in our panel ranked installing state-of-the-art on-board safety systems on all cargo aircraft and tracking part 135 operations as the potential measures that would most improve air cargo safety (see figure 8). Figure 8: The Relative Improvement That Potential Measures Could Have on Air Cargo Safety as Ranked by Air Cargo Experts: [Refer to PDF for image: stacked horizontal bar graph] Measure: Install onboard safety technology; Greatest improvement: 5; Second greatest improvement: 3; Third greatest improvement: 4; Total: 12. Measure: Track part 135 operations; Greatest improvement: 2; Second greatest improvement: 4; Third greatest improvement: 5; Total: 11. Measure: Align cargo and passenger standards; Greatest improvement: 4; Second greatest improvement: 2; Third greatest improvement: 2; Total: 8. Measure: Improve shipper knowledge of hazmat; Greatest improvement: 3; Second greatest improvement: 4; Third greatest improvement: 1; Total: 8. Measure: Assess the risk of individual flights; Greatest improvement: 2; Second greatest improvement: 3; Third greatest improvement: 3; Total: 8. Measure: Set nongovernment standards ramp employees; Greatest improvement: 2; Second greatest improvement: 2; Third greatest improvement: 3; Total: 7. Measure: Improve FAA inspector training; Greatest improvement: 3; Second greatest improvement: 1; Third greatest improvement: 2; Total: 6. Measure: Improve hazmat enforcement; Greatest improvement: 0; Second greatest improvement: 4; Third greatest improvement: 2; Total: 6. Measure: Increase FAA inspections; Greatest improvement: 3; Second greatest improvement: 1; Third greatest improvement: 1; Total: 5. Measure: Improve pilot training; Greatest improvement: 2; Second greatest improvement: 1; Third greatest improvement: 2; Total: 5. Measure: Set Government standards for ramp employees; Greatest improvement: 1; Second greatest improvement: 2; Third greatest improvement: 1; Total: 4. Source: GAO. [End of figure] Experts Believe Improved On-Board Safety Technology Would Most Improve the Safety of Air Cargo Operations: Adding state-of-the-art on-board safety technology was the potential measure to improve air cargo safety endorsed by the most experts in our panel. Better on-board technology, particularly for smaller aircraft, could provide additional tools to better inform pilots' judgment and decision making. One expert said that the type of technology needed depends on the type of carrier. He noted that large carriers often already have state-of-the-art technology, and that keeping pace with new technologies as they emerge is the challenge for them. However, he said that feeder and ad hoc air cargo carriers are not required to have certain on-board technologies, and that they would benefit from installing better situational awareness technologies, like Traffic Alert and Collision Avoidance Systems (TCAS) or Automatic Dependant Surveillance Broadcast (ADS-B), on their aircraft. TCAS monitors warn the pilot of potential collision dangers, and ADS-B uses satellite- based technology to broadcast aircraft identification, position, and speed with once-per-second updates. Other experts on the panel also indicated that TCAS and ADS-B would most improve safety on the smaller aircraft that lack it. FAA's Capstone Program in Alaska has shown that better technology can reduce aircraft accidents. As described earlier, Alaska's challenging operating conditions factored prominently in air cargo accidents over the last decade. The Capstone Program funded technology upgrades that provide pilots with information on terrain, weather, and air traffic. FAA's goal was to reduce Alaska's higher-than-average aviation accident rate. FAA has stated that an independent study found that, from 2000 through 2004, accidents for Capstone-equipped aircraft were reduced by 47 percent. However, the experts on our panel were not unanimous about the potential for improving safety through better on-board technologies. The experts who represent the part 135 perspective did not rate improving on-board safety systems as highly as the other experts did. Only one of the six part 135 pilots or carriers we surveyed ranked this as one of the top three potential improvements. The other five indicated that improving on-board safety systems is not feasible or would only slightly improve air cargo safety. For example, some part 135 ad hoc and feeder carriers we interviewed indicated that state-of- the-art on-board safety systems are not affordable relative to the value of the aircraft. Specifically, officials from one ad hoc cargo carrier said that traffic collision avoidance systems (such as TCAS or ADS-B) installed on a Cessna, valued at $100,000 to $200,000, would make the biggest improvement in safety, but such a system would cost about $25,000 to install on each aircraft, which the officials said was not practical. Tracking Part 135 Cargo Operations Ranked Second among Top Steps toward Improving Air Cargo Safety among Experts: As stated earlier, we were unable to determine accident rates for small feeder or ad hoc cargo carriers because FAA does not track part 135 operations. However, operational data--such as flight hours or landings--for on-demand part 135 cargo operations would allow analysts to determine accident rates for all cargo carriers as they currently can do for all part 121 (large) carriers. This is important because a higher proportion of air cargo accidents and nearly all fatalities occur to part 135 (small) carriers. Many experts who responded to our survey indicated that better data on part 135 cargo flight and operations could improve air cargo safety.[Footnote 30] Specifically, tracking these data was ranked among the top measures with the greatest potential by the second largest share of experts in our panel. Industry experts and officials we spoke with said there is a need for FAA to have this type of data. For example, NTSB has recommended in 2003 that FAA collect additional operational data from small air carriers in order to generate accident and incident rate information for all sectors of commercial aviation, including air cargo, but the recommendation remains open because an FAA official said that FAA chose not to collect the information. In addition, numerous industry stakeholders told us that not having these data precludes FAA from effectively targeting its safety initiatives. One of the experts on our panel said that people might assume that the part 135 carriers with the most accidents are the ones with the poorest safety records, but that may not be the case when the number of operations is considered. One official from a part 135 carrier said that the industry is generally against greater reporting because it would increase workload. However, he stated that if FAA begins requiring such data, companies will find a way to comply because they already collect the data. We interviewed part 135 carriers of various sizes, and they all indicated that they already track operational data internally and could report these data to FAA without a substantial additional effort, but this was not designed to be a representative sample. Pilot and Carrier Experts Are Divided on How Aligning Cargo Regulations with Passenger Regulations Would Affect Air Cargo Safety: As discussed earlier in this report, the regulations under which most large cargo carriers typically operate (supplemental) differ from the regulations under which most passenger carriers operate (domestic or flag). Although part 121 pilot experts on our panel indicated that aligning cargo regulations with passenger carrier regulations would improve safety, carrier experts generally disagreed. All three pilots, who fly under part 121, ranked alignment of regulations as the top potential measure to improve air cargo safety. As an example, the Air Line Pilots Association, the employee organization for most commercial U.S. pilots, supports the alignment of duty time regulations for all part 121 carriers. The Association believes that longer flight times can increase pilot fatigue and thus increase mistakes and accidents, and, as stated earlier in this report, the experts rated pilot fatigue as a serious challenge to safe cargo operations. On the other hand, none of the seven carrier experts ranked aligning cargo regulations with passenger regulations among their top three potential measures for improving air cargo safety. Five of them indicated that aligning regulations would have a slight or no improvement, and the other two indicated that aligning regulations would not be feasible. Although not specifically related to cargo operations, NTSB has also recommended that FAA revisit its time and duty regulations as they may be related to fatigue. For example, in 2008, it recommended that FAA develop guidance for operators to use in establishing fatigue management systems and then continually assess the effectiveness of the systems. It also recommended in 1995 that FAA review its flight and duty time regulations to include the findings of fatigue and sleep research. These recommendations remain open because FAA has not completed its actions related to these issues. In commenting on a draft of this report, FAA said that it convened a committee to address pilot fatigue issues and that the data it collects may be used in future rule- making efforts regarding pilot flight time, duty, and rest regulations. Conducting Flight Risk Assessments before a Flight Can Help Reduce Accumulated Risk and, According to Experts, Improve Safety: As described earlier, many air cargo accidents over the last 10 years occurred in conditions of accumulated risk--when several risk elements were present, but none was individually significant enough to result in the flight's cancellation. FAA, the Flight Safety Foundation, and NATA have each developed tools that pilots or carriers could voluntarily use to assess accumulated risk factors and determine if the flight should go forward. These tools assign values to various risk elements, such as single pilot operations, night flights, and flights into areas without accurate weather reports. See appendix III for FAA's sample flight risk assessment tool. Eight of the 27 experts on our cargo safety panel ranked flight risk assessment as one of the three potential efforts that could most improve safety. Additionally, 18 experts indicated that incorporating flight risk assessment checklists into air cargo daily operations would have a moderate or great effect on improving the safety of their operations. One expert on our panel said that flight risk assessment cannot prevent all accidents and that not all flights with multiple accumulated risk factors have accidents, but assessing the risk factors may help pilots reduce the number of cargo accidents by recognizing when the accumulated risks become unacceptably high and at which point pilots would either find ways to mitigate those risk factors or delay the flight. Despite the high level of support among our expert panelists for using flight risk assessment checklists, only 1 of the 10 carriers we interviewed used them in their daily operations. Officials from one part 135 carrier said that carriers are busy enough doing all the things that FAA requires to worry too much about ideas that might be very productive but are nonetheless optional. Other experts pointed out that all of the flight risk assessment checklists currently available were designed for passenger operations and that cargo carriers would have to tailor the tools to their needs--potentially a critical obstacle to implementation. Conclusions: Aviation in the United States remains safe and air cargo accidents have declined over the last 10 years, although fatal accidents do occur every year. Most of those accidents and nearly all of the fatal accidents in the last decade have happened to feeder and ad hoc carriers. However, the lack of operational data for part 135 carriers, which make up the bulk of the feeder and ad hoc carriers, makes it impossible to determine accident and fatality rates for small carriers or to track cargo-wide accident or fatality rates over time. FAA's information on small carrier operations is based on its annual survey of aircraft owners, which does not differentiate between passenger and cargo operations, making it impossible to use the survey results for cargo operators. While the numbers of accidents suggest that the fatality rates for feeder and ad hoc carriers are higher than the rates for large carriers, it is impossible to know how much higher the fatality rates are for feeder and ad hoc carriers without data on the numbers of operations for all types of cargo aircraft. It is also difficult for FAA and industry to target further safety improvements to the areas with the highest risk. Despite the higher numbers of accidents and fatal accidents among small cargo carriers, FAA's safety programs have focused primarily on large cargo carriers, the industry segment in which accidents and accident rates have steadily declined. While it makes sense to focus first on large carriers, which operate larger aircraft with larger crews and cargo holds, the safety of the smaller aircraft is also important. There is nothing intrinsic to small carriers that precludes risk-based oversight, voluntary disclosure programs, or the use of SMSs, but these efforts are usually targeted toward, or at least primarily used by, the large cargo carriers. However, cost is a concern for carriers, and poor economic conditions throughout the air cargo sector may mean that few funds will be available in the near term for new safety initiatives. In addition, FAA has increasingly focused on potential accident precursors that would reduce the risk of accidents before a related accident even occurs. However, neither FAA nor NTSB systematically tracks incidents in a way that would allow empirical analysis, even though incidents are widely viewed as accident precursors. Over half of the fatal air cargo accidents since 1997 had multiple risk factors. However, preflight risk assessment checklists are not required to be used within the cargo industry. The concept of assessing and recognizing accumulated risk through flight risk assessment presents an additional low-cost opportunity for identifying and reducing the risk associated with some cargo flights that might otherwise go unnoticed. Recommendations: To help FAA improve the data on and the safety of air cargo operations, we recommend that the Secretary of Transportation direct the FAA Administrator to take the following four actions: * Gather comprehensive and accurate data on all part 135 cargo operations to gain a better understanding of air cargo accident rates and better target safety initiatives. This can be done by separating out cargo activity in FAA's annual survey of aircraft owners or by requiring all part 135 cargo carriers to report operational data as part 121 carriers currently do. * Promote the increased use of safety programs by small (feeder and ad hoc) cargo carriers that use the principles underpinning SMS and voluntary self-disclosure programs. * Evaluate the likelihood that cargo incidents could be precursors to accidents and, if FAA determines they are, create a process for capturing incidents that would allow in-depth analysis of incidents to identify accident precursors related to specific carriers, locations, operations, and equipment. * Create incentives for cargo carriers to use flight risk assessment checklists in their daily operations, including tailoring a sample flight risk assessment checklist for part 135 cargo carriers. Agency Comments: We provided copies of a draft of this report to DOT and NTSB for their review and comment. Both agencies provided technical comments, which we incorporated as appropriate. We are sending copies of this report to interested congressional committees, the Secretary of Transportation, and the Chairman of the National Transportation Safety Board. We are also making copies available to others on request. In addition, this report is available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you or your staffs have any questions about this report, please contact me at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. Signed by: Gerald L. Dillingham, Ph.D. Director, Physical Infrastructure Issues: [End of section] Appendix I: Objective, Scope, and Methodology: Our objective in conducting this study was to review the nature and extent of safety issues in the air cargo industry and what the Federal Aviation Administration (FAA) and others are doing and could do to address them. To accomplish this objective, we established the following research questions: (1) What have been recent trends in air cargo safety? (2) What factors have contributed to air cargo accidents in recent years? (3) What have FAA and the industry done to improve air cargo safety, and how do experts view the effectiveness of these efforts? (4) What do experts say FAA and industry could do to further improve air cargo safety? To determine trends in air cargo safety, we obtained and analyzed accident and incident data for calendar years 1997 through 2008. * From the National Transportation Safety Board (NTSB), we obtained accident data for part 121 and part 135 all-cargo and mail operations that occurred from January 1, 1997, through December 31, 2008. To capture the full extent of cargo operations in Alaska, however, we also included passenger/cargo accidents because Alaska's by-pass mail system, which requires carriers to have a certain share of the passenger market to obtain a by-pass mail contract, resulted in fewer all-cargo carriers in that state. From these data we identified 443 fixed-wing aircraft accidents, including 93 fatal accidents. Six of the accidents involved 2 aircraft, so our data included a total of 449 accident aircraft. * From FAA, we obtained data on part 121 and part 135 fixed-wing all- cargo accidents and incidents that occurred from January 1, 1997, through December 31, 2007. From these data, we eliminated accidents that were also included in the NTSB data, for a total of 937 accidents and incidents. To avoid confusion when discussing the two data sets, we refer to the FAA data as "incidents" and the NTSB data as "accidents." Because we are familiar with and have previously determined that these data were sufficiently reliable for the nationwide descriptive and comparative analyses used in this report, we interviewed agency officials knowledgeable about the databases from which the data were derived to determine that the accident and incident data used in this report continue to be sufficiently reliable for the types of analyses we performed. We also obtained information on industry trends by conducting a literature search and reviewing the resulting documents, conducting a survey of air cargo experts, and interviewing officials and reviewing relevant documents from FAA, the Pipeline and Hazardous Materials Safety Administration (PHMSA), air cargo industry associations, air cargo carriers, airports, an employee group, and others. We also conducted site visits to Alaska, Ohio, and Texas. Those locations were selected to be geographically diverse and as the states with the largest number of air cargo accidents or because of the relatively large number of air cargo carriers of various sizes located there. To assess what factors have contributed to air cargo accidents in recent years, we conducted several analyses. First, to determine prominent accident causes, we analyzed data on probable causes and contributing factors from completed NTSB investigations of 417 air cargo accidents. Second, to assess accumulated risk, we applied FAA's proposed flight risk assessment tool to NTSB's reports on the 93 fatal cargo accidents that occurred during our review period. To do this, we searched each accident report for the 38 risk factors in the tool, such as "pilot flight time less than 100 hours in the last 90 days." For each factor found, we noted its corresponding risk value on an Excel spreadsheet and tabulated the total score as well as the total number of risk factors for each fatal accident. Third, for indications of other factors contributing to air cargo accidents, we surveyed a panel of air cargo experts, which is discussed in more detail later below; analyzed documents and interviewed officials from FAA, PHMSA, NTSB, air cargo industry associations, air cargo carriers, airports, an employee group, and others; and conducted site visits to Alaska, Ohio, and Texas (see the previous paragraph). To determine what FAA and the air cargo industry have done to improve safety, we interviewed FAA and industry officials, reviewed key documents, and analyzed FAA's oversight and enforcement data for all- cargo carriers. We interviewed officials and tested the data and found it sufficiently reliable for our purposes. To obtain experts' opinions about how FAA and the air cargo industry could further air cargo safety, we surveyed a panel of 27 air cargo safety experts. The experts rated and provided relative rankings on the effectiveness of current efforts to improve air cargo safety, the severity of safety challenges faced by the air cargo sector of aviation, and the potential improvements that additional efforts could have on air cargo safety. We selected the panel of experts with the assistance of the National Academy of Sciences to represent the perspectives of a cross-section of air cargo stakeholders. The specific experts, their affiliation, and their expert perspectives are listed below. Table 3: Air Cargo Expert Panelists: Large cargo carriers: 1; Name: Robert Gray; Title: Vice President, Regulatory Compliance and Government Affairs; Affiliation: ABX Air. 2; Name: David A. Prewitt; Title: Managing Director, Air Safety and Regulatory Compliance; Affiliation: Federal Express. 3; Name: Christopher Williams; Title: Director of Airline Safety; Affiliation: UPS Airlines. Feeder and ad hoc cargo carriers: 4; Name: Teak Biondo; Title: Operations Director; Affiliation: Ameristar. 5; Name: Richard Mills; Title: Director of Safety and Compliance; Affiliation: Empire Airlines. 6; Name: Jacqueline Rosser; Title: Executive Director; Affiliation: Air Charter Safety Foundation. 7; Name: Thomas D. Schaner; Title: Director of Operations; Affiliation: AirNet Systems. Larger aircraft cargo pilots: 8; Name: Captain Bill McReynolds; Title: Pilot; Affiliation: Federal Express. 9; Name: Captain Dennis Nugent; Title: Pilot; Affiliation: Kalitta Airlines. 10; Name: Captain Ken Young; Title: Pilot; Affiliation: ASTAR Air Cargo. Smaller aircraft cargo pilots: 11; Name: Michael Looby; Title: Chief Pilot; Affiliation: Castle Aviation. 12; Name: Quinn Hammon; Title: Chief Pilot; Affiliation: AirNet. Federal government: 13; Name: Anthony J. Broderick; Title: Associate Administrator for Regulation and Certification (retired); Affiliation: FAA. 14; Name: Peter Neff; Title: Aviation Safety Inspector & Operations Specialist; Affiliation: FAA. 15; Name: Joseph M. Sedor; Title: Senior Air Safety Investigator; Affiliation: NTSB. Cargo aircraft manufacture and conversion: 16; Name: Nils Lache; Title: Head of Cargo, Freighter, and GHS Definition Cabin & Cargo Customization; Affiliation: Airbus. 17; Name: Paul D. Russell; Title: Chief Engineer, Aviation System Safety; Affiliation: Boeing Commercial Airplanes. 18; Name: David Steinmetz; Title: Vice President and General Manager; Affiliation: Precision Conversions, LLC. Cargo loaders: 19; Name: Robert Kiss; Title: Vice President, Ground Operations; Affiliation: Atlas Air Worldwide Holdings Cargo. Cargo airports: 20; Name: Charles T. Miller; Title: Executive Director; Affiliation: Louisville Regional Airport Authority. 21; Name: Robert W. O'Brien, Jr.; Title: Executive Director; Affiliation: Greater Rockford Airport Authority. 22; Name: Jim Iagulli; Title: Operations Manager; Affiliation: Ted Stevens Anchorage International Airport. Human factors and safety performance: 23; Name: R. Wade Allen; Title: President and Technical Director; Affiliation: Systems Technology, Inc. 24; Name: James Burin; Title: Director, Technical Programs; Affiliation: Flight Safety Foundation. 25; Name: John K. Lauber; Title: Large cargo carriers: Senior Vice President and Chief Product Safety Officer (retired); Affiliation: Airbus. 26; Name: Dr. Terry L. von Thaden; Title: Assistant Professor; Affiliation: University of Illinois Institute of Aviation, Human Factors Division. International: 27; Name: Jean Abouchaar; Title: Director of Cargo, Regulatory and Industrial Affairs (retired); Affiliation: International Air Transport Association. Source: GAO. [End of table] To develop our survey of air cargo experts, we reviewed existing studies about air cargo safety, including previous and ongoing GAO work, and interviewed air cargo safety stakeholders. GAO subject matter experts designed draft questionnaires in close collaboration with a social science survey specialist. We conducted pretests with four people knowledgeable in the field of air cargo (representatives from air carriers, airports, and air transportation associations) to help further refine our questions, develop new questions, clarify any ambiguous portions of the survey, and identify any potentially biased questions. These pretests were conducted in-person and by telephone. We worked with the National Academy of Sciences and internally to develop the panel of experts and obtain contact information. We launched our Web-based survey on August 18, 2008, and received all responses by November 5, 2008. Log-in information to the Web-based survey was e-mailed to participants. We sent one follow-up e-mail message to all nonrespondents a week later, and contacted by telephone all those who had not completed the questionnaire within 3 weeks. We received responses from all 27 of our selected experts. Because our survey was not a sample survey, there are no sampling errors; however, the practical difficulties of conducting any survey may introduce nonsampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, or the types of people who do not respond can introduce unwanted variability into the survey results. We included steps in both the data collection and data analysis stages for the purpose of minimizing such nonsampling errors. As indicated above, GAO subject matter experts collaborated with a social science survey specialist to design draft questionnaires, and versions of the questionnaire were pretested with four knowledgeable people in the air cargo field. From these pretests, we made revisions as necessary. We examined the survey results and performed computer analyses to identify inconsistencies and other indications of error. A second, independent analyst checked the accuracy of all computer analyses. [End of section] Appendix II Expert Responses to GAO Survey: We worked with the National Academy of Sciences to identify air cargo experts that included carrier, pilot, airport, aircraft manufacturer, government, and human factors and safety performance perspectives. We sent our Web-based survey to 27 air cargo experts and received responses from all 27 experts. Our survey was composed of closed-and open-ended questions. In this appendix, we include all the survey questions and aggregate results of responses to the closed-ended questions; we do not provide information on responses provided to the open-ended questions. For a more detailed discussion of our survey methodology, see appendix I. Q1. In your opinion, how effective, if at all, is each of the following current efforts to improve the safety of cargo-only flights? a. Federal Aviation Administration (FAA) airworthiness directives or FAA operational specifications that affect air cargo; Not effective: 1; Slightly effective: 6; Moderately effective: 9; Greatly effective: 8; Unable to judge: 3; Number of respondents: 27. b. FAA informational materials and seminars related to cargo-only safety; Not effective: 2; Slightly effective: 8; Moderately effective: 13; Greatly effective: 0; Unable to judge: 4; Number of respondents: 27. c. FAA oversight and inspections; Not effective: 2; Slightly effective: 7; Moderately effective: 11; Greatly effective: 5; Unable to judge: 2; Number of respondents: 27. d. National Transportation Safety Board's (NTSB) public meetings, hearings, and forums; Not effective: 6; Slightly effective: 6; Moderately effective: 9; Greatly effective: 4; Unable to judge: 2; Number of respondents: 27. e. The implementation of NTSB safety recommendations that affect cargo- only operations; Not effective: 4; Slightly effective: 9; Moderately effective: 6; Greatly effective: 5; Unable to judge: 3; Number of respondents: 27. f. Cargo-only carriers' participation in FAA's voluntary safety disclosure programs; Not effective: 0; Slightly effective: 4; Moderately effective: 5; Greatly effective: 14; Unable to judge: 4; Number of respondents: 27. g. Trade association endeavors to work within their membership to improve cargo safety; Not effective: 0; Slightly effective: 5; Moderately effective: 9; Greatly effective: 11; Unable to judge: 2; Number of respondents: 27. h. Trade association endeavors to advocate that others beyond their members, including FAA and Congress, make cargo safety improvements; Not effective: 1; Slightly effective: 6; Moderately effective: 10; Greatly effective: 6; Unable to judge: 4; Number of respondents: 27. i. Carrier implementation of safety management systems; Not effective: 0; Slightly effective: 7; Moderately effective: 6; Greatly effective: 11; Unable to judge: 3; Number of respondents: 27. j. Airport implementation of safety management systems; Not effective: 1; Slightly effective: 5; Moderately effective: 6; Greatly effective: 6; Unable to judge: 9; Number of respondents: 27. Q2. Considering the above list of efforts (question 1 a-j), which three do you believe are the most effective in improving air cargo safety? a. Federal Aviation Administration (FAA) airworthiness directives or FAA operational specifications that affect air cargo; Most effective: 7; Second most effective: 1; Third most effective: 2. b. FAA informational materials and seminars related to cargo-only safety; Most effective: 0; Second most effective: 1; Third most effective: 0. c. FAA oversight and inspections; Most effective: 2; Second most effective: 3; Third most effective: 0. d. National Transportation Safety Board's (NTSB) public meetings, hearings, and forums; Most effective: 0; Second most effective: 4; Third most effective: 1. e. The implementation of NTSB safety recommendations that affect cargo- only operations; Most effective: 1; Second most effective: 2; Third most effective: 6. f. Cargo-only carriers' participation in FAA's voluntary safety disclosure programs; Most effective: 7; Second most effective: 5; Third most effective: 4. g. Trade association endeavors to work within their membership to improve cargo safety; Most effective: 3; Second most effective: 6; Third most effective: 5. h. Trade association endeavors to advocate that others beyond their members, including FAA and Congress, make cargo safety improvements; Most effective: 1; Second most effective: 1; Third most effective: 3. i. Carrier implementation of safety management systems; Most effective: 6; Second most effective: 3; Third most effective: 4. j. Airport implementation of safety management systems; Most effective: 0; Second most effective: 1; Third most effective: 2. Q3. Besides the efforts listed above, do you know of any other current significant efforts to improve cargo-only aviation safety? If so, please explain. Q4. In your opinion, how much of a challenge, if any, does each of the following issues pose to safely operating cargo-only flights? a. Pilot fatigue related to nighttime flying, ineffective rest periods, or commuting; Not a challenge: 2; Slight challenge: 4; Moderate challenge: 12; Great challenge: 8; Unable to judge: 1; Number of respondents: 27. b. Carrier policies that, by their nature, give pilots economic incentive to fly in less-than-ideal conditions; Not a challenge: 6; Slight challenge: 8; Moderate challenge: 2; Great challenge: 5; Unable to judge: 6; Number of respondents: 27. c. Flights scheduled with less than 4 hours of notice given to the crew; Not a challenge: 6; Slight challenge: 9; Moderate challenge: 3; Great challenge: 4; Unable to judge: 4; Number of respondents: 26. d. Low piloting experience overall, as well as low piloting experience in the specific types of cargo aircraft operated; Not a challenge: 3; Slight challenge: 4; Moderate challenge: 12; Great challenge: 6; Unable to judge: 2; Number of respondents: 27. e. Single-pilot operations; Not a challenge: 2; Slight challenge: 9; Moderate challenge: 5; Great challenge: 6; Unable to judge: 4; Number of respondents: 26. f. Variation within the cargo-only sector regarding the priority given to safety; Not a challenge: 5; Slight challenge: 2; Moderate challenge: 8; Great challenge: 10; Unable to judge: 2; Number of respondents: 27. g. Availability of aircraft rescue and fire-fighting services provided by persons with cargo-specific knowledge and training; Not a challenge: 8; Slight challenge: 6; Moderate challenge: 7; Great challenge: 5; Unable to judge: 1; Number of respondents: 27. h. Difficult cargo-only flight and operating conditions (e.g., airports with limited technology, mountainous terrain, nighttime operations); Not a challenge: 1; Slight challenge: 9; Moderate challenge: 9; Great challenge: 6; Unable to judge: 2; Number of respondents: 27. i. Alaska's aviation and operating environment; Not a challenge: 2; Slight challenge: 5; Moderate challenge: 8; Great challenge: 5; Unable to judge: 7; Number of respondents: 27. j. The handling and transport of undeclared but potentially hazardous materials; Not a challenge: 3; Slight challenge: 5; Moderate challenge: 7; Great challenge: 12; Unable to judge: 0; Number of respondents: 27. k. The handling and transport of declared hazardous materials; Not a challenge: 9; Slight challenge: 10; Moderate challenge: 5; Great challenge: 3; Unable to judge: 0; Number of respondents: 27. l. Cargo loading issues (e.g., weight and balance, shifting); Not a challenge: 6; Slight challenge: 9; Moderate challenge: 7; Great challenge: 3; Unable to judge: 2; Number of respondents: 27. m. Differences in federal safety standards between some cargo-only and passenger flights; Not a challenge: 9; Slight challenge: 8; Moderate challenge: 5; Great challenge: 4; Unable to judge: 1; Number of respondents: 27. n. The ability to obtain quality parts for, and/or maintain the airworthiness of, older cargo-only aircraft; Not a challenge: 7; Slight challenge: 7; Moderate challenge: 5; Great challenge: 6; Unable to judge: 1; Number of respondents: 26. Q5. Considering the above list of challenges (question 4 a-n), which three do you believe are the greatest challenges to air cargo safety? a. Pilot fatigue related to nighttime flying, ineffective rest periods, or commuting; Greatest challenge: 7; Second greatest challenge: 2; Third greatest challenge: 3. b. Carrier policies that, by their nature, give pilots economic incentive to fly in less-than-ideal conditions; Greatest challenge: 0; Second greatest challenge: 3; Third greatest challenge: 1. c. Flights scheduled with less than 4 hours of notice given to the crew; Greatest challenge: 1; Second greatest challenge: 0; Third greatest challenge: 1. d. Low piloting experience overall, as well as low piloting experience in the specific types of cargo aircraft operated; Greatest challenge: 3; Second greatest challenge: 6; Third greatest challenge: 1. e. Single-pilot operations; Greatest challenge: 1; Second greatest challenge: 2; Third greatest challenge: 1. f. Variation within the cargo-only sector regarding the priority given to safety; Greatest challenge: 4; Second greatest challenge: 1; Third greatest challenge: 6. g. Availability of aircraft rescue and fire-fighting services provided by persons with cargo-specific knowledge and training; Greatest challenge: 0; Second greatest challenge: 1; Third greatest challenge: 1. h. Difficult cargo-only flight and operating conditions (e.g., airports with limited technology, mountainous terrain, nighttime operations); Greatest challenge: 1; Second greatest challenge: 2; Third greatest challenge: 6. i. Alaska's aviation and operating environment; Greatest challenge: 1; Second greatest challenge: 4; Third greatest challenge: 0. j. The handling and transport of undeclared but potentially hazardous materials; Greatest challenge: 7; Second greatest challenge: 1; Third greatest challenge: 3. k. The handling and transport of declared hazardous materials; Greatest challenge: 0; Second greatest challenge: 0; Third greatest challenge: 2. l. Cargo loading issues (e.g., weight and balance, shifting); Greatest challenge: 0; Second greatest challenge: 2; Third greatest challenge: 0. m. Differences in federal safety standards between some cargo-only and passenger flights; Greatest challenge: 1; Second greatest challenge: 1; Third greatest challenge: 0. n. The ability to obtain quality parts for, and/or maintain the airworthiness of, older cargo-only aircraft; Greatest challenge: 1; Second greatest challenge: 2; Third greatest challenge: 1. Q6. Besides the challenges listed above, do you know of any other significant challenges to safe cargo-only air operations? If so, please explain. Q7. In your opinion, how much improvement, if any, in the safety of cargo-only air operations would be provided if each of the following measures were implemented? a. FAA increasing the amount of on-site inspections that it conducts; No improvement: 4; Slight improvement: 14; Moderate improvement: 4; Great improvement: 4; Not feasible: 1; Unable to judge: 0; Number of respondents: 27. b. FAA improving inspector training and knowledge of cargo-only air operations; No improvement: 2; Slight improvement: 4; Moderate improvement: 10; Great improvement: 9; Not feasible: 0; Unable to judge: 1; Number of respondents: 26. c. Carriers and flight schools providing better training for cargo-only pilots; No improvement: 3; Slight improvement: 8; Moderate improvement: 11; Great improvement: 3; Not feasible: 0; Unable to judge: 2; Number of respondents: 27. d. The Department of Transportation (DOT) or Transportation Security Administration (TSA) increasing shipper knowledge and declaration of hazardous materials; No improvement: 5; Slight improvement: 6; Moderate improvement: 7; Great improvement: 9; Not feasible: 0; Unable to judge: 0; Number of respondents: 27. e. DOT or TSA promoting better shipper compliance with rules for the handling and transport of declared hazardous materials; No improvement: 4; Slight improvement: 7; Moderate improvement: 7; Great improvement: 8; Not feasible: 0; Unable to judge: 1; Number of respondents: 27. f. FAA setting uniform standards for ramp employees involved in loading and unloading cargo; No improvement: 4; Slight improvement: 11; Moderate improvement: 5; Great improvement: 4; Not feasible: 3; Unable to judge: 0; Number of respondents: 27. g. Industry setting uniform standards for ramp employees involved in loading and unloading cargo; No improvement: 0; Slight improvement: 10; Moderate improvement: 7; Great improvement: 6; Not feasible: 3; Unable to judge: 1; Number of respondents: 27. h. Cargo operators incorporating flight risk assessment checklists into their daily operations; No improvement: 2; Slight improvement: 7; Moderate improvement: 12; Great improvement: 6; Not feasible: 0; Unable to judge: 0; Number of respondents: 27. i. FAA collecting and using part 135 and part 91 cargo flight and operations data to better target safety efforts; No improvement: 2; Slight improvement: 6; Moderate improvement: 6; Great improvement: 9; Not feasible: 0; Unable to judge: 4; Number of respondents: 27. j. FAA bringing cargo-only standards into alignment with those for passenger operations; No improvement: 9; Slight improvement: 5; Moderate improvement: 4; Great improvement: 7; Not feasible: 2; Unable to judge: 0; Number of respondents: 27. k. Owners installing state-of-the-art onboard safety technology on all cargo-only aircraft; No improvement: 0; Slight improvement: 5; Moderate improvement: 7; Great improvement: 9; Not feasible: 5; Unable to judge: 1; Number of respondents: 27. Q8. Considering the above list of possible measures (question 7 a-k), which three do you believe would provide the greatest improvement to air cargo safety? a. FAA increasing the amount of on-site inspections that it conducts; Greatest improvement: 3; Second greatest improvement: 1; Third greatest improvement: 1. b. FAA improving inspector training and knowledge of cargo-only air operations; Greatest improvement: 3; Second greatest improvement: 1; Third greatest improvement: 2. c. Carriers and flight schools providing better training for cargo-only pilots; Greatest improvement: 2; Second greatest improvement: 1; Third greatest improvement: 2. d. The Department of Transportation (DOT) or Transportation Security Administration (TSA) increasing shipper knowledge and declaration of hazardous materials; Greatest improvement: 3; Second greatest improvement: 4; Third greatest improvement: 1. e. DOT or TSA promoting better shipper compliance with rules for the handling and transport of declared hazardous materials; Greatest improvement: 0; Second greatest improvement: 4; Third greatest improvement: 2. f. FAA setting uniform standards for ramp employees involved in loading and unloading cargo; Greatest improvement: 1; Second greatest improvement: 2; Third greatest improvement: 1. g. Industry setting uniform standards for ramp employees involved in loading and unloading cargo; Greatest improvement: 2; Second greatest improvement: 2; Third greatest improvement: 3. h. Cargo operators incorporating flight risk assessment checklists into their daily operations; Greatest improvement: 2; Second greatest improvement: 3; Third greatest improvement: 3. i. FAA collecting and using part 135 and part 91 cargo flight and operations data to better target safety efforts; Greatest improvement: 2; Second greatest improvement: 4; Third greatest improvement: 5. j. FAA bringing cargo-only standards into alignment with those for passenger operations; Greatest improvement: 4; Second greatest improvement: 2; Third greatest improvement: 2. k. Owners installing state-of-the-art onboard safety technology on all cargo-only aircraft; Greatest improvement: 5; Second greatest improvement: 3; Third greatest improvement: 4. Q9. Besides the possible measures listed above, do you have any other suggestions for significantly improving cargo-only aviation safety? If so, please explain. Q10. Please provide any other comments that you have regarding cargo- only safety. [End of section] Appendix III: FAA's Sample Flight Risk Assessment Tool: InFO: Information for Operators: U.S. Department of Transportation: Federal Aviation Administration: InFO 07015 Approved by: AFS-200: Date: 7/3/2007: Flight Standards Service: Washington, DC: [hyperlink, http://www.faa.gov/other_visit/aviation_industry/airline_operators/airli ne_safety/info] An InFO contains valuable information for operators that should help them meet certain administrative, regulatory, or operational requirements with relatively low urgency or impact on safety. Subject: Flight Risk Assessment Tool: Purpose: This InFO describes the proactive identification of possible hazards and the use of risk management tools to mitigate risks as aspects of a Safety Management System (SMS). These tools will provide ways for air operators to determine which flights have more risk and allow operators to intervene and reduce risk when possible. Risk assessment tools are only part of an SMS and should not be considered the whole system. Background: Over the years the Federal Aviation Administration (FAA) and the aviation industry have dramatically increased the safety of air travel by managing and mitigating risks associated with flight. The aviation industry currently provides the safest form of transportation in the United States. However, the industry continues to have some accidents that can be prevented. Therefore, both the FAA and industry are working to continually improve the safety record of turbine-powered aircraft. Over the next few years, the FAA will encourage operators and certificate holders to develop Safety Management Systems (SMS). This safety protocol is described in Advisory Circular (AC) 120-92, Introduction to Safety Management Systems for Air Operators. The Turbine Aircraft Operations Subgroup as part of the General Aviation Joint Steering Committee has developed a risk assessment tool for use in flight operations. In creating this tool, the Turbine Aircraft Operations Subgroup reviewed accident data, identified hazards, and used normal risk assessment development methodology. This tool provides a simple way to implement proactive risk management. An operator can use the risk assessment tool as a standalone tool but incorporating it into an SMS is preferable. Discussion: As discussed in AC 120-92, a hazard is defined as any existing or potential condition that can lead to injury, illness, or death to people; damage to or loss of a system, equipment, or property; or damage to the environment. A hazard is a condition that is a prerequisite of an accident or incident. Every flight has hazards and some level of risk associated with it. It is critical that operators and pilots arc able to differentiate, in advance, between a low risk flight and a high risk flight, and then establish a review process and develop risk mitigation strategies to address flights throughout that range. A risk assessment tool should allow operators and pilots to see the risk profile of a flight in its planning stages. Each operator should determine an acceptable level of risk for its flights based on the type of operation, environment, aircraft used, crew training, and overall operating experience. When the risk for a flight exceeds the acceptable level, the hazards associated with that risk should be further evaluated and the risk reduced. A higher risk flight should not be operated if the hazards cannot be mitigated to an acceptable level. The attached risk assessment tool has been developed for use in understanding different levels of flight risk and to allow operators and pilots to become familiar with this element of an SMS. It is important for operators to understand that risk has several elements that must be considered, including probability, severity, and weighted value. What is the probability of a particular event occurring? If the event does occur, what is the severity likely to be? And what is the weighted value of this type of event compared to other aspects of the operation? In the attached risk assessment tool, this work has been done so the operator has a simplified form of the tool. Each operator may want to add items that are unique to its operation using the additional resources provided. An operator can also change any item currently used in the tool provided it conducts a realistic assessment of the hazard being changed. To use the tool, the operator will need to create numerical thresholds that trigger additional levels of scrutiny prior to a go/no-go decision for the flight. These thresholds should be created to help ensure that the safety standards of each individual operation are maintained. However, it is important that the operator create realistic thresholds. If every flight is within the acceptable range under any condition, it is likely that the thresholds have not been set correctly. Small operations (for example, where the pilot is also the chief pilot and owner) should consider strategies for appropriate consideration of elevated risk that best fit their operation. Recommended Action: The FAA recommends that operators and pilots familiarize themselves with the attached risk assessment tool and AC 120-92. They should then decide whether to use the tool as published or to modify it as needed for their own operations. Once an operator has established the parameters of the tool, it should create operational thresholds and begin using the tool to establish a "risk number" for each flight. This risk number should be used to control risk before a flight takes place. Over time this tool will become unique to each operator and can become a part of its complete SMS. The risk assessment tool cannot guarantee a safe flightsafety is ultimately the responsibility of the pilot and operator. However, it does provide an additional tool to help the pilot and operator make sound safety decisions. Flight Risk Assessment Tool: Date: Departure: Release/Trip #: Tail #: Destination: Pilot Qualifications and Experience: 1. Captain with less than 200 hours in type: Risk Value: 5; Flight Value: 2. First Officer with less than 200 hours in type: Risk Value: 5; Flight Value: 3. Single Pilot Flight: Risk Value: 5; Flight Value: 4. Captain with less than 100 hours last 90 days: Risk Value: 3; Flight Value: 5. First Officer with less than 100 hours last 90 days: Risk Value: 3; Flight Value: 6. Duty day greater than 12 hours: Risk Value: 4; Flight Value: 7. Flight time (Greater than 8 hours in the duty day): Risk Value: 4; Flight Value: 8. Crew Rest (Less than 10 hours prior to the duty day): Risk Value: 5; Flight Value: Total Factor Score - Section 1: Operating Environment: 9. VOR/GPS/LOC/ADF (Best approach available w/o vertical guidance): Risk Value: 3; Flight Value: 10. Circling approach (best available approach): Risk Value: 4; Flight Value: 11. No published approaches: Risk Value: 4; Flight Value: 12. Mountainous airport: Risk Value: 5; Flight Value: 13. Control tower not operational at ETA or ETD: Risk Value: 3; Flight Value: 14. Uncontrolled airport: Risk Value: 5; Flight Value: 15. Alternate airport not selected: Risk Value: 4; Flight Value: 16. Elevation of primary airport greater than 5000 ft. MSL): Risk Value: 3; Flight Value: 17. Wet runway: Risk Value: 3; Flight Value: 18. Contaminated runway: Risk Value: 3; Flight Value: 19. Winter operation: Risk Value: 3; Flight Value: 20. Twilight operation: Risk Value: 2; Flight Value: 21. Night operation: Risk Value: 5; Flight Value: 22. Stopping distance greater than 80% of available runway: Risk Value: 5; Flight Value: 23. Repositioning flight (no passengers or cargo): Risk Value: 5; Flight Value: 24. Pop up trip (Less than 4 hours crew notice): Risk Value: 3; Flight Value: 25. International operation: Risk Value: 2; Flight Value: 26. No weather reporting at destination: Risk Value: 5; Flight Value: 27. Thunderstorms at departure and/or destination: Risk Value: 4; Flight Value: 28. Severe turbulence: Risk Value: 5; Flight Value: 29. Ceiling & visibility at destination less than 500 ft./2 sm: Risk Value: 3; Flight Value: 30. Heavy rain at departure and/or destination: Risk Value: 5; Flight Value: 31. Frozen precipitation at departure and/or destination: Risk Value: 3; Flight Value: 32. Icing (moderate-severe): Risk Value: 5; Flight Value: 33. Surface winds greater than 30 knots: Risk Value: 4; Flight Value: 34. Crosswinds greater than 15 knots: Risk Value: 4; Flight Value: 35. Runway braking action less than good: Risk Value: 5; Flight Value: Total Factor Score - Section 2: Equipment: 36. Special Flight Permit Operation (ferry permit): Risk Value: 3; Flight Value: 37. MEL/COL Items (items related to safety of flight): Risk Value: 2; Flight Value: 38. Special flight limitations based on AFM equipment limitations: Risk Value: 2. Flight Value: Total Factor Score - Section 3: Totals: Example: Use of Flight Risk Assessment Tool: The following discussion provides a practical example of the five step process used to assess risk as outlined in AC 120-92. The example involves the operation of a night flight where the destination airport is experiencing windy, rainy conditions. The captain has fewer than 200 hours in type, and the first officer has flown less than 100 hours in the last 90 days. The company SOPs require the Chief Pilot to evaluate flight risk factor values over 20 from the perspective of accepting the risk, rejecting the risk, or mitigating the risk. Further, the company SOPS prevent the operation of a flight if the risk value exceeds 25. In our example the non-parenthetical numerical value represents the original risk value assigned to the hazard. Risk values in parenthesis represent the reduced risk values assigned after the Chief Pilot acted to mitigate the risks. Step 1. Complete a system and task analysis. * The captain is not highly experienced with less than 200 hours in type. * The first officer has less than 100 hours in the last 90 days. Step 2. Identify the hazards: * The runway is wet. * The flight will operate at night. * The destination crosswinds are greater than 15 knots. Step 3. Analyze the safety risk: * The combination of the risk factors associated with this flight generates a risk value of 20 using the example risk assessment tool. Step 4. Assess the safety risk: * Company policy requires that the Chief Pilot assess and approve any flight risk value greater than 15. Since the risk value of 20 exceeds the company operational threshold risk of 15, the Chief Pilot decides to operate the flight by reducing the flight risk value to a more acceptable level. Step 5. Control the safety risk: * The Chief Pilot focuses on mitigating three hazards: 1. He decides to allow the scheduled captain to operate the flight; 2. However, he assigns the flight to a first officer who is more current and who has flown more than 100 hours in the last 90 days; 3. Further, the Chief pilot changes the destination airport to an airport with no crosswind expected. * By controlling the risk value of these three hazards, the Chief Pilot has reduced the flight overall risk value to 13 and elevates the operational level of safety. Flight Risk Assessment Tool: Date: Any day; Departure: DAL; Release/Trip #: 153; Tail #: N123; Destination: PDK. Pilot Qualifications and Experience: 1. Captain with less than 200 hours in type: Risk Value: 5; Flight Value: 5. 2. First Officer with less than 200 hours in type: Risk Value: 5; Flight Value: 3. Single Pilot Flight: Risk Value: 5; Flight Value: 4. Captain with less than 100 hours last 90 days: Risk Value: 3; Flight Value: 5. First Officer with less than 100 hours last 90 days: Risk Value: 3; Flight Value: 3 (0). 6. Duty day greater than 12 hours: Risk Value: 4; Flight Value: 7. Flight time (Greater than 8 hours in the duty day): Risk Value: 4; Flight Value: 8. Crew Rest (Less than 10 hours prior to the duty day): Risk Value: 5; Flight Value: Total Factor Score - Section 1: 8 (5). Operating Environment: 9. VOR/GPS/LOC/ADF (Best approach available w/o vertical guidance): Risk Value: 3; Flight Value: 10. Circling approach (best available approach): Risk Value: 4; Flight Value: 11. No published approaches: Risk Value: 4; Flight Value: 12. Mountainous airport: Risk Value: 5; Flight Value: 13. Control tower not operational at ETA or ETD: Risk Value: 3; Flight Value: 14. Uncontrolled airport: Risk Value: 5; Flight Value: 15. Alternate airport not selected: Risk Value: 4; Flight Value: 16. Elevation of primary airport greater than 5000 ft. MSL): Risk Value: 3; Flight Value: 17. Wet runway: Risk Value: 3; Flight Value: 3. 18. Contaminated runway: Risk Value: 3; Flight Value: 19. Winter operation: Risk Value: 3; Flight Value: 20. Twilight operation: Risk Value: 2; Flight Value: 21. Night operation: Risk Value: 5; Flight Value: 5. 22. Stopping distance greater than 80% of available runway: Risk Value: 5; Flight Value: 23. Repositioning flight (no passengers or cargo): Risk Value: 5; Flight Value: 24. Pop up trip (Less than 4 hours crew notice): Risk Value: 3; Flight Value: 25. International operation: Risk Value: 2; Flight Value: 26. No weather reporting at destination: Risk Value: 5; Flight Value: 27. Thunderstorms at departure and/or destination: Risk Value: 4; Flight Value: 28. Severe turbulence: Risk Value: 5; Flight Value: 29. Ceiling & visibility at destination less than 500 ft./2 sm: Risk Value: 3; Flight Value: 30. Heavy rain at departure and/or destination: Risk Value: 5; Flight Value: 31. Frozen precipitation at departure and/or destination: Risk Value: 3; Flight Value: 32. Icing (moderate-severe): Risk Value: 5; Flight Value: 33. Surface winds greater than 30 knots: Risk Value: 4; Flight Value: 34. Crosswinds greater than 15 knots: Risk Value: 4; Flight Value: 4 (0). 35. Runway braking action less than good: Risk Value: 5; Flight Value: Total Factor Score - Section 2: 12 (8). Equipment: 36. Special Flight Permit Operation (ferry permit): Risk Value: 3; Flight Value: 37. MEL/COL Items (items related to safety of flight): Risk Value: 2; Flight Value: 38. Special flight limitations based on AFM equipment limitations: Risk Value: 2. Flight Value: Total Factor Score - Section 3: 0. Totals: 20 (13). Questions or comments on this InFO should be addressed to Peter Neff, AFS-820, (202) 493-5400. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: Gerald L. Dillingham, Ph.D., (202) 512-2834 or dillinghamg@gao.gov: Staff Acknowledgments: In addition to those named above, Teresa Spisak (Assistant Director), Richard D. Brown, Keith Cunningham, Elizabeth Eisenstadt, Michele Fejfar, David Hooper, Mitchell Karpman, Valerie Kasindi, Sara Ann Moessbauer, Susie Sachs, Christine San, Pamela Vines, and Crystal Wesco made key contributions to this report. [End of section] Footnotes: [1] A fixed-wing aircraft is a heavier-than-air aircraft capable of flight whose lift is generated not by wing motion relative to the aircraft, but by forward motion through the air. The term is used to distinguish fixed-wing aircraft from rotary-wing aircraft and ornithopters in which lift is generated by blades or wings that move relative to the aircraft. [2] 14 CFR part 121. [3] 14 CFR parts 135 and 121 differ in a number of ways. For example, part 135 cargo aircraft are not required to have on-board safety technology that are required on passenger aircraft such as a traffic collision and avoidance system, a terrain awareness and warning system, or an autopilot. Part 121 operations may be conducted under (1) domestic rules--scheduled operations within the 48 contiguous states conducted by aircraft with 10 or more passenger seats or a payload capacity of over 7,500 pounds; (2) flag rules--scheduled domestic and international operations conducted by aircraft with 10 or more passenger seats or a payload capacity of over 7,500 pounds; or (3) supplemental rules--nonscheduled operations conducted by aircraft with more than 30 passenger seats and a payload capacity of over 7,500 pounds. Large all-cargo carriers typically operate under supplemental rules. Under part 135, FAA regulates a variety of aviation operations, including both "commuter" (scheduled flights with fewer than 10 seats) and "on-demand" (unscheduled air carriers) operations. Most part 135 all-cargo carriers are certificated as on-demand regardless of whether they operate on a regular timetable. Some cargo operations also may be conducted under 14 CFR part 125, which applies to private operations (not available to the public) with 20 or more seats and a payload capacity of 6,000 pounds or more. [4] For additional information on our assessment of NPG, see GAO, Aviation Safety: System Safety Approach Needs Further Integration into FAA's Oversight of Airlines, [hyperlink, http://www.gao.gov/products/GAO-05-726] (Washington, D.C.: Sept. 28, 2005). [5] Aviation accidents are defined at 49 CFR part 830.2. [6] Our analysis of the NTSB data revealed a total of 443 air cargo accidents for calendar years 1997 through 2008 involving 449 aircraft. These numbers include 14 CFR part 121 and 135 cargo and mail flights, except for Alaska, where we included passenger-cargo flights due to circumstances resulting from the by-pass mail program. Of the 443 accidents, 93 were fatal for 128 people. See appendix I for more information on our scope and methodology. [7] These rates do not count "isolated risk" events where risk was isolated to a single person (ramp accidents and turbulence). There were no part 121 cargo nonisolated risk accidents in 2007. [8] NTSB recommendation A-03-037 is classified by NTSB as open with an acceptable response. [9] Since accidents typically result from a combination of circumstances, conditions, and events, NTSB often cites multiple causes and contributing factors in its investigative findings. To determine the incidence of pilot performance among NTSB's determinations of probable cause, we counted pilot performance if it was cited as the initial probable cause of an accident. [10] For pilots of many small cargo aircraft, duties often include loading and unloading the cargo, in addition to readying the aircraft before the flight, flying the cargo to its destination, and servicing the aircraft after completing the flight. [11] Carriers operating under part 135 are required to have a deicing program. FAA is developing a proposed rule change for current and future air carrier aircraft with a maximum take-off weight up to 60,000 pounds to either require the installation of ice-detection equipment or change the procedures for activating ice-protection systems to ensure timely activation of the systems. [12] Other organizations have also developed flight risk assessment checklists. For example, the Flight Safety Foundation is developing a decision-making tool called the flight operations risk assessment system, which will assist managers in determining the relative risk of an accident or incident during a flight operation. [13] For Alaska, we included passenger/cargo accidents as well as cargo and mail accidents in our analysis. We included this additional category of accidents because of the by-pass mail system in Alaska, which requires carriers to have a certain share of the passenger market to obtain a by-pass mail contract. This requirement is intended to decrease cargo-only carriers in Alaska. Therefore, to capture the full scope of cargo operations in Alaska, we decided to include those accidents flown as passenger/cargo operations. [14] The 1997 and 1998 accidents were the result of fumes from the cargo compartments while the 2006 accident was caused by an in-flight cargo fire initiated by an unknown source. [15] GAO, Aviation Safety: Undeclared Air Shipments of Dangerous Goods and DOT's Enforcement Approach, [hyperlink, http://www.gao.gov/products/GAO-03-22] (Washington, D.C.: Jan. 10, 2003). [16] In August 2007, PHMSA banned nonrechargeable lithium battery shipments on passenger aircraft, but the rule does not affect the ability of passengers to carry or use personal devices containing lithium batteries while aboard the aircraft. This ruling does not include cargo-only aircraft. [17] According to NTSB, primary lithium batteries are not rechargeable while secondary lithium batteries are. Primary lithium batteries, which are found in such devices as watches and pocket calculators, contain metallic lithium sealed in a metal casing. These batteries will burn if the lithium is exposed to the air, and halon fire suppressions systems, which are the only systems certified for aviation, are not effective in extinguishing fires involving primary lithium batteries. Secondary lithium batteries are commonly used in such devices as cameras, cellular phones, and laptop computers. These batteries contain lithium ions in a flammable liquid electrolyte. Halon suppression systems are effective in extinguishing secondary lithium battery fires. [18] The NTSB recommendations are numbered A-07-104 through A-10-109. [19] PHMSA guidance entitled "Shipping Batteries Safely by Air: What You Need to Know" provides information on the methods of safely transporting batteries and battery-powered equipment in compliance with the hazardous materials regulations. [20] Although the FAA database does not distinguish between accidents and incidents, our analysis of these data included only air cargo events that were not captured as accidents in NTSB's Aviation Accident Database. Thus, to avoid confusion, we refer to data from the FAA database as "incidents" and data from the NTSB database as "accidents." [21] We will be reporting later this year on our assessment of FAA's use of data for safety oversight. [22] National Academy of Engineering, Accident Precursor Analysis and Management: Reducing Technological Risk Through Diligence (Washington, D.C., August 2004). [23] The ASAP exceptions from Advisory Circular 120-66B include: (1) the noncompliance must be inadvertent and not involve an intentional disregard for safety; (2) the ASAP report must generally be within 24 hours of the end of the flight sequence or within 24 hours of the person becoming aware of possible noncompliance; and (3) the event must not appear to involve criminal activity, substance abuse, controlled substances, alcohol, or intentional falsification. [24] Report of the Independent Review Team: Managing Risks in Civil Aviation: A Review of the FAA's Approach to Safety, A Blue Ribbon Panel Appointed May 1, 2008 by Secretary of Transportation, Mary E. Peters to Examine FAA's Safety Culture and Approach to Safety Management (Washington, D.C.: Sept. 2, 2008). [25] This number includes 16 cargo carriers with part 121 and part 135 certificates. [26] In commenting on a draft of this report, FAA indicated that it believed that operators should not be required to make major revisions or rewrites of their manuals as part of this process. Cargo carriers that we interviewed disagreed. [27] This number includes 16 cargo carriers with part 121 and part 135 certificates. [28] National Research Council of the National Academies, Staffing Standards for Aviation Safety Inspectors, eds. William C. Howell and Susan B. Van Hemel (Washington, D.C., 2007). [29] GAO, Aviation Safety: Better Management Controls are Needed to Improve FAA's Safety Enforcement and Compliance Efforts, [hyperlink, http://www.gao.gov/products/GAO-04-646] (Washington, D.C.: July 6, 2004). [30] In commenting on a draft of this report, FAA said that part 135 operators are encouraged to submit data through FAA's annual survey of aircraft owners, but this survey does not distinguish between passenger and cargo operations. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.