Aviation Safety
System Safety Approach Needs Further Integration into FAA's Oversight of Airlines Gao ID: GAO-05-726 September 28, 2005The Federal Aviation Administration (FAA) uses the Air Transportation Oversight System (ATOS), which was developed around the principles of system safety, to oversee seven "legacy airlines" and nine other airlines. In this report, we refer to airlines that are not in ATOS as non-legacy airlines. Two other processes are used to oversee 99 non-legacy passenger airlines, which represent a fast-growing segment of the commercial aviation passenger industry and carried about 200 million passengers in 2004. The National Work Program Guidelines (NPG) establishes a set of inspection activities for non-legacy airlines. The Surveillance and Evaluation Program (SEP) uses principles of system safety to identify additional risk-based inspections for those airlines. GAO's objective was to assess the processes used by FAA to ensure the safety of non-legacy passenger airlines. GAO reviewed the strengths of FAA's inspection oversight for non-legacy passenger airlines and the issues that hinder its effectiveness.
A key strength of FAA's inspection oversight of non-legacy airlines is the introduction of system safety concepts to some inspections, which FAA accomplished by adding SEP to its traditional inspection process, NPG. Although NPG has risk-based elements, it lacks the structured approach to risk identification found in SEP. Under SEP, data are used to help determine trends or problems. The SEP process uses a team of inspectors to identify inspection activities, which we have previously reported is generally more effective than the use of individuals due to their collective ability to identify risks. Under SEP, inspectors also ascertain risks internal to FAA, such as staffing shortages. FAA's oversight of non-legacy airlines further incorporates processes to ensure that inspectors follow up on airline actions taken in response to inspection findings. These efforts address several past GAO concerns, including that NPG did not allow FAA to identify risks and allocate inspection resources accordingly. The full potential of FAA's inspection program for non-legacy airlines, however, is not being realized due to incomplete implementation of its system safety approach and other challenges. The inspection workload is still heavily oriented to nonrisk-based activities, with 77 percent of inspection activities being identified through the NPG and the remaining relatively small percentage identified through SEP. The emphasis on NPG, including FAA's guidance that inspectors must complete NPG-required inspection activities, acts as a disincentive to identifying further inspection activities through SEP. Inspectors face workload challenges as staff lost through attrition may not be replaced due to a hiring freeze. FAA estimates that over 1,100 inspectors of non-legacy airlines will leave the agency in fiscal years 2005 to 2010. In addition, some FAA inspectors indicated that a lack of technical training on airline systems and equipment posed potential risks to the agency's oversight process. Finally, FAA lacks a process to communicate information to inspectors on how certain internal risks identified through SEP are being resolved. Moreover, FAA has not established a process to evaluate the effectiveness of SEP.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-726, Aviation Safety: System Safety Approach Needs Further Integration into FAA's Oversight of Airlines
This is the accessible text file for GAO report number GAO-05-726
entitled 'Aviation Safety: System Safety Approach Needs Further
Integration into FAA's Oversight of Airlines' which was released on
October 26, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
September 2005:
Aviation Safety:
System Safety Approach Needs Further Integration into FAA's Oversight
of Airlines:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-726]:
GAO Highlights:
Highlights of GAO-05-726, a report to congressional requesters:
Why GAO Did This Study:
The Federal Aviation Administration (FAA) uses the Air Transportation
Oversight System (ATOS), which was developed around the principles of
system safety, to oversee seven ’legacy“ airlines“ and nine other
airlines. In this report, we refer to airlines that are not in ATOS as
non-legacy airlines. Two other processes are used to oversee 99 non-
legacy passenger airlines, which represent a fast-growing segment of
the commercial aviation passenger industry and carried about 200
million passengers in 2004. The National Work Program Guidelines (NPG)
establishes a set of inspection activities for non-legacy airlines. The
Surveillance and Evaluation Program (SEP) uses principles of system
safety to identify additional risk-based inspections for those
airlines.
GAO‘s objective was to assess the processes used by FAA to ensure the
safety of non-legacy passenger airlines. GAO reviewed the strengths of
FAA‘s inspection oversight for non-legacy passenger airlines and the
issues that hinder its effectiveness.
What GAO Found:
A key strength of FAA‘s inspection oversight of non-legacy airlines is
the introduction of system safety concepts to some inspections, which
FAA accomplished by adding SEP to its traditional inspection process,
NPG. Although NPG has risk-based elements, it lacks the structured
approach to risk identification found in SEP. Under SEP, data are used
to help determine trends or problems. The SEP process uses a team of
inspectors to identify inspection activities, which we have previously
reported is generally more effective than the use of individuals due to
their collective ability to identify risks. Under SEP, inspectors also
ascertain risks internal to FAA, such as staffing shortages. FAA‘s
oversight of non-legacy airlines further incorporates processes to
ensure that inspectors follow up on airline actions taken in response
to inspection findings. These efforts address several past GAO
concerns, including that NPG did not allow FAA to identify risks and
allocate inspection resources accordingly.
The full potential of FAA‘s inspection program for non-legacy airlines,
however, is not being realized due to incomplete implementation of its
system safety approach and other challenges. The inspection workload is
still heavily oriented to nonrisk-based activities, with 77 percent of
inspection activities being identified through the NPG and the
remaining relatively small percentage identified through SEP. The
emphasis on NPG, including FAA‘s guidance that inspectors must complete
NPG-required inspection activities, acts as a disincentive to
identifying further inspection activities through SEP. Inspectors face
workload challenges as staff lost through attrition may not be replaced
due to a hiring freeze. FAA estimates that over 1,100 inspectors of non-
legacy airlines will leave the agency in fiscal years 2005 to 2010. In
addition, some FAA inspectors indicated that a lack of technical
training on airline systems and equipment posed potential risks to the
agency‘s oversight process. Finally, FAA lacks a process to communicate
information to inspectors on how certain internal risks identified
through SEP are being resolved. Moreover, FAA has not established a
process to evaluate the effectiveness of SEP.
Aircraft Safety Inspection:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO recommends that FAA develop an evaluative process for SEP and
improve communications and training for inspectors in system safety and
risk management. FAA agreed with the recommendations on training and
communications and will consider developing an evaluative process.
www.gao.gov/cgi-bin/getrpt?GAO-05-726.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Gerald L. Dillingham at
(202) 512-2834 or dillinghamg@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
FAA's Oversight Process Uses System Safety, Which Is Designed to
Identify and Control Risks and Improve Resource Utilization:
Most Inspection Activities Are Not Prioritized Based on a Structured
Risk Assessment Process:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: FAA's Surveillance and Evaluation Process:
SEP Incorporates System Safety into FAA's Inspection Oversight of Non-
legacy Airlines:
SEP Incorporates Some Elements of ATOS, While NPG Relies on a Set
Number of Inspections:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Number of Times Risks Internal to FAA Were Identified by
Inspectors through SEP, Fiscal Years 2002-2004:
Table 2: Severity of Internal Risks Identified by FAA Inspectors,
Fiscal Years 2002-2004:
Table 3: SEP-and NPG-Initiated Required Inspections for the Top 25 Non-
legacy Airlines, Fiscal Years 2002-2004:
Table 4: Allocation of FAA Inspection Staff under SEP and ATOS for Four
Airlines:
Table 5: Number of Enplanements for the Top 25 Non-legacy Airlines,
2004
Table 6: FAA Offices and Airlines Interviewed by GAO:
Table 7: Organizations Interviewed by GAO:
Table 8: Safety Systems and Examples of Risk Indicators in SEAT:
Table 9: Various Elements of ATOS, NPG, and SEP:
Figures:
Figure 1: Percentage of Enplaned Passengers on Non-legacy Airlines and
Legacy Airlines, 1995-2004:
Figure 2: Number of FAA Aviation Safety Inspectors for Commercial
Airlines, Fiscal Years 2002-2004, and Estimates for Fiscal Years 2005-
2007:
Figure 3: Non-legacy Airline Inspectors' Views on the Extent Technical
Training Received in the Last 2 Years Has Helped Them Do Their Jobs:
Figure 4: Views of Inspectors of Non-legacy Airlines on the Extent
Inspectors Have Received Technical Training in a Timely Manner during
Their FAA Careers:
Figure 5: View of Inspectors of Non-legacy Airlines on the Extent to
Which They Received Training in Automated Systems Used in the
Inspection Process in a Timely Manner:
Abbreviations:
ATOS: Air Transportation Oversight System:
DOT: Department of Transportation:
FAA: Federal Aviation Administration:
NPG: National Work Program Guidelines:
PTRS: Program Tracking and Reporting Subsystem:
SEAT: Surveillance and Evaluation Assessment Tool:
SEP: Surveillance and Evaluation Program:
Letter September 28, 2005:
The Honorable James L. Oberstar:
Ranking Democratic Member:
Committee on Transportation and Infrastructure:
House of Representatives:
The Honorable Jerry F. Costello:
Ranking Democratic Member:
Subcommittee on Aviation:
Committee on Transportation and Infrastructure:
House of Representatives:
The Honorable Peter A. DeFazio:
House of Representatives:
The U.S. commercial aviation industry, with less than one fatal
accident per 5 million flights from 2002 through 2004, has an
extraordinary safety record. However, when passenger airlines have
accidents or serious incidents, regardless of their rarity, the
consequences can be tragic. In order to maintain a high level of
aviation safety, it is critical to have well-established, efficient,
and effective systems in place to provide an early warning of hazards
that can lead to accidents. The Federal Aviation Administration (FAA)
has established a number of systems and processes to inspect and
oversee various aspects of passenger airline safety, such as aircraft
maintenance and flight operations. About 585 of FAA's approximately
3,200 inspectors are dedicated to overseeing the largest commercial
passenger airlines, including the seven "legacy airlines," and nine
other airlines through FAA's Air Transportation Oversight System
(ATOS), which was developed around the principles of system
safety.[Footnote 1] System safety involves the continual evaluation of
all of an airline's operations for the purpose of identifying and
mitigating risks. Approximately 1,100 inspectors[Footnote 2] oversee
other entities and individuals, including 99 smaller commercial
passenger airlines--which we refer to as non-legacy passenger airlines-
-about 5,200 aircraft repair stations, and approximately 625,000
pilots. Non-legacy passenger airlines, a fast-growing segment of the
commercial aviation passenger industry, carried about 200 million
passengers in 2004.
FAA's inspection process for airlines not covered by ATOS has two
components. The National Work Program Guidelines (NPG) is the baseline
component of the oversight program for these airlines. In 2002, FAA
added another component, the Surveillance and Evaluation Program (SEP),
to the inspection process to incorporate principles of ATOS into its
oversight of non-legacy passenger airlines. The two components are used
together to establish the number of annual inspections for non-legacy
airlines. Inspections can encompass many different activities, such as
visually spot-checking an airplane at a gate, monitoring procedures on
a scheduled flight, or observing maintenance being performed on an
aircraft. Each year, FAA headquarters establishes baseline inspections
for each airline through NPG, while through SEP, teams of FAA
inspectors analyze the results of an airline's prior inspections at
periodic meetings and, based on their assessment of specific risks,
establish other inspections that may be needed.
In response to your request, we assessed FAA's processes for ensuring
the safety of non-legacy passenger airlines. Specifically, we addressed
the following questions: (1) What are the strengths of FAA's inspection
approach for non-legacy passenger airlines? and (2) What issues hinder
the effectiveness of FAA's inspection approach?
To address these questions, we obtained and analyzed information from a
variety of sources. We examined FAA documents about SEP, NPG, and ATOS.
We also reviewed prior reports prepared by us and others on SEP and NPG
to determine significant issues involving those programs and how they
were resolved. In addition, we surveyed a statistical sample of FAA
safety inspectors to obtain their views about the training they
receive. We had no practical way to assess information on the amount of
training necessary for inspector proficiency or the timeliness of the
training provided. We conducted semistructured interviews and analyzed
relevant documents from FAA headquarters officials, and from field
managers and inspectors in 7 regional and 13 field offices,[Footnote 3]
which were selected because they oversee the top 25 non-legacy
airlines[Footnote 4] ranked by the number of enplanements in 2004. At
these locations, we collected information on the inspection process and
inspector staffing levels, workload, and training. In addition, we
conducted semistructured interviews with safety officials at 16 of
those top 25 non-legacy airlines, and with officials at seven industry
organizations that represent airlines, inspectors, pilots, mechanics,
and maintenance facilities. We analyzed data on required NPG and SEP
inspections for fiscal years 2002 through 2004 from FAA's nationwide
inspection database--the Program Tracking and Reporting Subsystem
(PTRS).[Footnote 5] We assessed the reliability of the database and
found the data sufficiently reliable for the types of analyses that we
conducted for this report. We also tested for the presence of several
management controls, including the processes for verifying inspection
results, establishing a process to evaluate FAA's inspection oversight
process, managing the PTRS database, and communicating among managers
and inspectors relating to NPG and SEP. In addition, we reviewed
literature on system safety and compared FAA's system safety framework
with that presented in the literature. We conducted our work from
August 2004 through September 2005 in accordance with generally
accepted government auditing standards. Additional information on our
methodology is found in appendix I.
Results in Brief:
A key strength of FAA's inspection oversight of non-legacy airlines is
the introduction of system safety concepts to some inspections, which
FAA accomplished by adding SEP to its traditional inspection process,
NPG. SEP presents a shift in concept from FAA's customary method that
relied on conducting a set number of inspections of an airline's
operations to an approach that allows for the efficient use of
inspection staff and resources by prioritizing workload based on areas
of highest risk. To facilitate the implementation of a system safety
approach, FAA has made an effort to train its staff in system safety.
In addition, FAA utilizes teams of inspectors in SEP, which we have
previously reported is generally more effective than the use of
individuals because of the team's collective ability to identify risks.
SEP also allows inspectors to identify risks internal to FAA, such as
staffing shortages or training deficiencies, and FAA has established a
chain of command to address these risks. FAA's oversight of non-legacy
airlines further incorporates processes to ensure that inspectors
follow up on airline actions taken in response to inspection findings.
These efforts address several past concerns noted by us--that FAA
needed to better identify risks and allocate inspection resources
accordingly, and verify and monitor inspection findings to ensure that
priorities were achieved.
The full potential of FAA's inspection program for non-legacy airlines,
however, is not being realized due to incomplete implementation of the
agency's system safety approach and other challenges. The inspection
workload is still heavily oriented to the NPG's nonrisk-based
activities. For fiscal years 2002 through 2004, 77 percent of
inspection activities required for the top 25 non-legacy airlines were
identified through NPG, so that only the remaining relatively small
percentage of inspection activities were identified based on risk
through SEP. We found that the large percentage of NPG-identified
activities occurred because FAA's guidance places greater emphasis on
the NPG-identified activities, even though the guidance establishes a
process--referred to as retargeting--whereby inspectors can replace NPG-
identified activities with SEP-identified activities that they deem
constitute a greater safety risk. For example, the guidance emphasizes
the importance of the NPG-identified activities by requiring inspectors
to complete all of the NPG-identified activities by the end of each
fiscal year but permitting SEP-identified activities to be rescheduled
to the following fiscal year. In addition, very few activities are
being retargeted, partly because field offices have interpreted FAA's
emphasis on NPG activities as discouraging retargeting. Inspector
workload also presents a challenge to FAA's oversight, as the number of
inspector staff available to oversee non-legacy airlines has declined
due to attrition and workload shifts related to transferring staff to
ATOS, and most losses were not replaced due to a hiring freeze. In 11
of the 13 FAA field offices that we contacted, officials indicated a
shortage of different types of inspectors needed to oversee the non-
legacy airlines, which has sometimes resulted in inspections being
delayed or eliminated. We also identified limitations concerning
technical and SEP-specific training for inspectors of non-legacy
airlines. For example, inspectors identified a lack of technical
training on airline systems and equipment as an internal risk to the
agency. In addition, FAA does not provide SEP-specific procedural
training to certain types of inspectors, which some inspectors told us
was needed to maximize their usefulness.[Footnote 6] Further, there is
inadequate communication from headquarters to inspectors on the
resolution of internal risks identified under SEP, according to some
inspectors. Headquarters officials acknowledged that there is no formal
feedback process to inform the inspectors about issues they raised
concerning internal risks. FAA has not established a way to evaluate
its inspection oversight process for non-legacy airlines. Moreover, the
agency's ability to evaluate this process is hindered by the lack of
important inspection-related information--such as whether the risks
identified through SEP have been mitigated--in its nationwide
inspection database.
To improve the effectiveness of the agency's oversight of non-legacy
airlines, we recommend that the Secretary of the Department of
Transportation (DOT) direct the FAA Administrator to develop a
continuous evaluative process for FAA's activities under SEP and link
SEP to the performance-related goals and measures developed by the
agency, track performance toward these goals, and determine appropriate
program changes. We also recommend that the agency improve
communication and training to ensure inspectors understand FAA's
policies and procedures in areas such as system safety and risk
management. DOT generally agreed with our recommendations to improve
communication and training. DOT said that it would consider our
recommendation to develop a continuous evaluative process for SEP and
link SEP to agency goals, but that its plan to put the remaining non-
legacy airlines in the ATOS program by the end of fiscal year 2007 may
make this recommendation unnecessary. In the past, FAA's efforts to
move airlines to ATOS have experienced delays, therefore, we retained
this recommendation. The department also provided clarifying comments
and technical corrections, which we incorporated as appropriate.
Background:
Non-legacy airlines represent a fast-growing segment of the passenger
airline industry. From 2002 through 2004, the annual enplanements for
these airlines grew from 122 million to about 200 million passengers,
or from about 20 percent to 28 percent of all passenger air travel.
During this same period, the percentage of people flying on legacy
airlines declined from 80 percent to about 72 percent of all passengers
flown (see fig. 1).
Figure 1: Percentage of Enplaned Passengers on Non-legacy Airlines and
Legacy Airlines, 1995-2004:
[See PDF for image]
[End of figure]
FAA's Inspector Workforce Carries Out Oversight:
FAA's safety oversight of non-legacy airlines is carried out by
inspectors located at 109 field offices throughout the world that are
part of 9 regional offices. For each airline, FAA puts together a team
led by principal inspectors who maintain primary responsibility for
managing the airline's certificate requirements[Footnote 7] and focus
on one of three disciplines: avionics,[Footnote 8] maintenance, or
operations. Additional team members include those based at the FAA
office that holds the airline's operating certificate--typically an
aircraft dispatch inspector, a cabin safety inspector, and assistants.
FAA locates its principal inspectors close to their respective
airlines' primary operational base. For example, the principal
inspectors for Independence Air (formerly Atlantic Coast Airlines) are
located at FAA's field office at Dulles International Airport, where
the airline has its headquarters. In addition, FAA has geographic
inspectors based at the 109 field offices to conduct additional
inspections. Rather than being designated to particular airlines,
geographic inspectors may conduct inspections of aircraft of any non-
legacy airlines that land in their area.
FAA's safety inspector workforce for all commercial airlines has
remained steady, averaging about 1,780 inspectors over fiscal years
2002 through 2004. FAA cannot determine how many of these inspectors
were assigned to non-legacy passenger airlines during those 3 years,
because the agency did not collect that information, according to an
FAA headquarters' official. FAA expects the number of inspectors
assigned to non-legacy airlines to remain steady for fiscal years 2005
and 2006 and decline slightly in fiscal year 2007, as shown in figure
2. Inspectors record information about these inspections in PTRS--a
nationwide computerized database that maintains such information as
inspector findings and airline activities in response to the findings.
Figure 2: Number of FAA Aviation Safety Inspectors for Commercial
Airlines, Fiscal Years 2002-2004, and Estimates for Fiscal Years 2005-
2007:
[See PDF for image]
Note: FAA does not have information to separate out the number of ATOS
inspectors and inspectors of non-legacy airlines prior to fiscal year
2005.
[End of figure]
FAA Uses NPG and SEP to Oversee Non-legacy Airlines:
NPG and SEP are the main inspection processes that FAA uses to oversee
the safety of non-legacy airlines. Since 1985, FAA has used NPG, which
includes both required and planned inspections, as its primary means of
ensuring that airlines comply with safety regulations. In NPG, an FAA
committee of program managers identifies an annual minimum set of
required inspections that are to be undertaken to ensure that airlines
are in compliance with their operating certificates. In addition,
inspectors determine annual sets of planned inspections based on their
knowledge and experience with the particular airlines they oversee.
Typically, inspections would include ramp inspections, in which
inspectors examine an aircraft while it is parked at the airport, and
maintenance inspections. However, we found problems with NPG throughout
the 1990s, including: (1) FAA's routine inspections were ineffective in
identifying serious safety problems, (2) critical airline inspections
had not been conducted, (3) FAA's follow-up actions often did not
ensure that problems were corrected once identified, and (4) FAA did
not have a methodology for estimating airline safety risks so that it
could target limited inspection resources to high-risk inspections. We
also found that FAA's inspection database was of limited use in
providing early warning of potential risks or targeting inspection
resources.
In response to these findings, and in the aftermath of the 1996 ValuJet
crash, an FAA task force reviewed the agency's safety inspection
process and recommended in part that the agency initiate a project to
make surveillance of airlines more systematic and targeted to deal with
identified risks. This recommendation resulted in the agency's
development and implementation of ATOS in 1998 at the nation's 10
largest commercial passenger airlines, with the goal of eventually
including all commercial passenger and cargo airlines.[Footnote 9] ATOS
emphasizes a system safety approach that extends beyond periodically
checking airlines for compliance with regulations to the use of
technical and managerial skills to identify, analyze, and control
hazards and risks.[Footnote 10] The goal of ATOS is to identify safety
trends in order to spot and correct problems at their root cause before
an accident occurs. This program allows FAA inspectors to look at an
airline as a whole, to see how the many elements of its operations,
including aircraft, pilots, maintenance facilities, flight operations,
and cabin safety, interact to meet federal standards. Collectively, the
airlines under ATOS had a dedicated inspector staff of 585 inspectors
as of July 2005. The number assigned to each airline depends on the
size of the airline's operations. ATOS uses special checklists and
databases that are intended to cover all areas of airline operations.
Part of FAA's oversight is expected to include an in-depth look at an
airline's policies and procedures and whether the airline is following
them. There are 16 airlines in the ATOS program, including two cargo
carriers as of September 2005. Due to resource constraints, FAA
determined it would not be able to immediately place the remaining
passenger airlines in the ATOS program. FAA developed SEP as a bridge
to introduce safety risk concepts used in ATOS into the oversight
process for non-legacy airlines in order to facilitate the ultimate
transition of these airlines to ATOS.As of September 2005, FAA
estimates that it will move the remaining non-legacy airlines to ATOS
by the end of fiscal year 2007.
SEP Implemented as a Transition to ATOS:
Since the introduction of ATOS, FAA has been moving toward integrating
system safety into its oversight activities. However, the agency has
been delayed in placing a significant number of non-legacy airlines in
the ATOS program, resulting in those airlines continuing to be overseen
through NPG, a process that is not system safety oriented. FAA agrees
that NPG is not a system safety process, but explained that the planned
NPG inspections, which inspectors identify based on their expertise,
provide a risk-based element to the process. To address the delay in
moving airlines to ATOS, FAA, in 2002, added SEP to NPG as a way to
introduce airlines to a system safety oversight process until those
airlines were transitioned to ATOS.[Footnote 11] Although originally
envisioned by FAA as a transitional program for airlines that were
awaiting placement in ATOS, SEP has become a more permanent oversight
process as budget and staff constraints have prevented the agency from
moving all airlines to ATOS.
SEP is complementary to NPG and permits the risk-based, data-driven
alteration of required NPG inspections. SEP provides a formal structure
to risk identification that is absent from the NPG process. Under SEP,
the principal FAA inspectors for each airline meet periodically during
the year to discuss the results of their inspections and identify risks
using the same risk assessment principles and checklists used in
ATOS.[Footnote 12] An outcome of the meetings is the identification of
inspections to augment the NPG baseline of required inspections, which
FAA views as the minimum number of inspections that need to occur to
ensure that certain areas of all airlines are reviewed.[Footnote 13]
During the SEP process, FAA headquarters allows principal inspectors to
replace NPG- required inspections with inspections targeting higher-
risk areas identified through the SEP meetings. This process is known
as retargeting. At the end of the meeting, entries are created in the
PTRS database that indicate all inspections to be performed for that
airline. SEP-initiated inspections that are designated as priorities
are required to be completed.[Footnote 14] See appendix II for
additional information on SEP, ATOS, and NPG.
FAA Also Uses Industry Partnership Programs to Provide Safety Oversight
of Airlines:
FAA also oversees the safety of both legacy and non-legacy passenger
airlines through participation in industry partnership programs. Two of
these programs--the Aviation Safety Action Program and Voluntary
Disclosure Reporting Program--encourage certain airline employees, such
as pilots and mechanics, or airlines to voluntarily report safety
information that might be critical to identifying potential precursors
to accidents without fear that FAA or their companies will use reports
accepted under the programs to take legal enforcement or disciplinary
actions against them. The Aviation Safety Action Program provides for
the voluntary self-reporting of safety incidents under procedures set
out in memorandums of understanding between FAA, airlines, and
participating employee groups such as aircraft mechanics. As of June
2005, 20 of the top 25 non-legacy airlines (in terms of the number of
enplanements) were participating in the Aviation Safety Action Program.
The Voluntary Disclosure Reporting Program allows airlines to
voluntarily report safety incidents to FAA. However, our prior work
found that FAA has not analyzed violation data derived from these two
programs to monitor national trends in airline operations, so that it
can target resources to address operational risks.[Footnote 15] A third
program, the Internal Evaluation Program, requires airlines to
continuously monitor and evaluate their practices and procedures.
FAA's Oversight Process Uses System Safety, Which Is Designed to
Identify and Control Risks and Improve Resource Utilization:
The incorporation of system safety into FAA's oversight presents a
shift in concept from the agency's traditional oversight method, which
relied upon periodic inspections, to an approach that allows for the
prioritization of inspections based on areas of highest risks. The
development of SEP, a key element in FAA's system safety oversight
process, allows for the efficient use of inspection staff and resources
by incorporating this risk-based approach. FAA's incorporation of SEP
into its oversight addresses a past concern by us that the agency was
not prioritizing inspectors' workload. Our review of literature by
government and private organizations supported FAA's concept of system
safety because it makes apparent the risks that are the basis for
changes. In addition, FAA's oversight process includes a requirement
that inspectors verify that corrective actions have occurred, providing
a level of assurance that the safety problem has been mitigated.
FAA Has Provided System Safety Training for Inspectors:
To facilitate the implementation of a system safety approach, FAA has
made an effort to train its staff in system safety, which is a strength
of its oversight approach. FAA has recognized that inspectors need
training in the system safety concept in order to effectively
incorporate this approach into airline oversight. Specifically, FAA
believed that its inspectors have gaps in their needed competencies and
skills that could affect the agency's system safety approach to
inspections. Among the largest training gaps for inspectors, according
to FAA, were training in risk analyses and systems thinking. As a
result of these findings, FAA has undertaken training activities to
improve inspector competencies in system safety.[Footnote 16] According
to FAA, between fiscal years 2002 and 2004, it provided basic system
safety training to almost 3,675 staff, including non-legacy and ATOS
inspectors.
SEP Incorporates System Safety by Focusing on the Identification and
Prioritization of Inspections Based on Risks:
By incorporating risk assessment into the inspection process, SEP
addresses a long-standing problem. Since the mid-1980s, our reports
have shown that FAA did not have a methodology for assessing airline
safety risks so that it could target limited inspection resources to
high-risk conditions.[Footnote 17] SEP's system safety concept of
reducing risk through the identification, analysis, and control of
hazards is also consistent with the views presented in literature of
other government and private organizations, such as the Department of
Defense, Canada's civil aviation authority (Transport Canada), and the
Flight Safety Foundation. For example, DOD looks at system safety as a
means of reducing risk through early identification, analysis,
elimination, and control of hazards. For the airlines it oversees,
Transport Canada requests that each airline incorporate system safety
into its operations by integrating safety into its policies, management
and employee practices, and operating procedures. FAA incorporates such
concepts in its system safety oversight approach.
SEP's system safety process has a number of strengths. At periodic
meetings, the principal inspectors of an airline identify potential
risks that they believe should be addressed. These inspectors consider
a variety of information that they have obtained through personal
observation and from reports prepared by geographic inspectors and
located in the PTRS database. A strength of this approach, consistent
with findings in our past reports,[Footnote 18] is that teams of
inspectors are generally more effective than individual inspectors in
their ability to collectively identify concerns. Another strength is
that FAA has developed risk assessment worksheets aligned with key
airline systems that guide inspectors through identifying and
prioritizing risks. The worksheets guide inspectors to organize the
results of their previous inspections and surveillance into a number of
areas, such as flight operations, personnel training, and cabin safety,
in order to identify specific risks in each area and target the
office's resources to mitigating those risks. (See app. II for more
details on the risk assessment process.)
SEP Also Allows for the Identification of Internal Risks That Might
Hinder Oversight:
During their periodic meetings, principal inspectors also have the
opportunity to identify risks in FAA's internal operations that could
adversely affect the inspectors' ability to conduct safety oversight of
non-legacy airlines, which is another strength of FAA's oversight
approach. FAA inspectors and managers agreed that identifying internal
risks is helpful, as they considered it an efficient and effective way
of identifying needed program improvements. Inspectors can identify
deficiencies in a number of categories, including inspector staffing,
training, availability of geographic inspectors, and resources such as
travel funds. In addition, the inspectors can quantify the degree of
the risk that each of these categories poses. Being able to identify
and make officials aware of situations that can hinder an agency's
ability to perform its mission is a key management control.
This process has resulted in FAA headquarters receiving significant
information on areas that inspectors believe are deficient. For fiscal
years 2002 through 2004, inspectors identified about 560 risks that, in
their opinion, could have an impact on how FAA managed the inspection
process of airlines, as shown in table 1. Inadequate training (aircraft-
specific training and other training) was the most frequently
identified concern of the inspectors, accounting for about 40 percent
of the total risks identified during the 3-year period--27 percent of
the risks were related to inadequate or nonexistent aircraft-specific
training and 15 percent were related to the lack of other training. We
discuss this issue in detail later in this report. The total number of
risks identified by FAA inspectors declined over the 3 years.
Inspectors from two FAA field offices opined that the decline may be
due to better staff understanding of risk identification and to the
risks being addressed expeditiously so that inspectors did not need to
restate them the following year.
Table 1: Number of Times Risks Internal to FAA Were Identified by
Inspectors through SEP, Fiscal Years 2002-2004:
Risk: Inadequate staffing on the certificate management team[A];
Number of risks identified: 2002: 67;
Number of risks identified: 2003: 53;
Number of risks identified: 2004: 55;
Number of risks identified: 175 (31%).
Risk: Inadequate or nonexistent aircraft-specific training;
Number of risks identified: 2002: 58;
Number of risks identified: 2003: 57;
Number of risks identified: 2004: 33;
Number of risks identified: 148 (27%).
Risk: Lack of other training;
Number of risks identified: 2002: 35;
Number of risks identified: 2003: 21;
Number of risks identified: 2004: 29;
Number of risks identified: 85 (15%).
Risk: Inadequate geographic staffing ;
Number of risks identified: 2002: 30;
Number of risks identified: 2003: 9;
Number of risks identified: 2004: 5;
Number of risks identified: 44 (8%).
Risk: Inadequate resources (such as travel funds) for the certificate
management team;
Number of risks identified: 2002: 24;
Number of risks identified: 2003: 19;
Number of risks identified: 2004: 10;
Number of risks identified: 53 (9%).
Risk: Other;
Number of risks identified: 2002: 28;
Number of risks identified: 2003: 11;
Number of risks identified: 2004: 13;
Number of risks identified: 52 (9%).
Total;
Number of risks identified: 2002: 242;
Number of risks identified: 2003: 170;
Number of risks identified: 2004: 145;
Number of risks identified: 557 (99%)[B].
Source: GAO analysis of FAA data.
[A] The certificate management team may include the principal avionics,
maintenance, and operations inspectors; aircraft dispatch inspector;
and cabin safety inspector.
[B] Percentages do not add to 100 due to rounding.
[End of table]
The process not only allows inspectors to identify the risk, but also
to describe the severity of the risk in order to assist FAA
headquarters in prioritizing its consideration of internal risks.
Inspectors rate each risk they identify in terms of severity
(negligible to catastrophic) and likelihood of occurrence (improbable
to frequent). Risks are then categorized using a scale of high, medium,
or low depending on the severity and likelihood of occurrences. For
example, risks rated high are those that have high levels of likelihood
and severity. As shown in table 2, over a 3-year period, 125 out of 557
risks were rated as high severity, which could help FAA determine which
issues most need targeting. Among the risks rated as high, training and
inadequate staffing were identified most frequently, accounting for 34
percent and 31 percent, respectively. As of May 2005, about 87 percent
of identified risks had been closed, including about 90 percent of the
risks identified as high. Risks are closed either because action has
been taken to mitigate the risk or FAA has determined that the level of
risk is acceptable and no action is warranted.
Table 2: Severity of Internal Risks Identified by FAA Inspectors,
Fiscal Years 2002-2004:
Severity of risk: High;
Number of risks: 2002: 37;
Number of risks: 2003: 58;
Number of risks: 2004: 30;
Number of risks: Total: 125 (22%).
Severity of risk: Medium;
Number of risks: 2002: 183;
Number of risks: 2003: 88;
Number of risks: 2004: 107;
Number of risks: Total: 378 (68%).
Severity of risk: Low;
Number of risks: 2002: 17;
Number of risks: 2003: 22;
Number of risks: 2004: 8;
Number of risks: Total: 47 (8%).
Severity of risk: Unknown;
Number of risks: 2002: 5;
Number of risks: 2003: 2;
Number of risks: 2004: 0;
Number of risks: Total: 7 (1%).
Total;
Number of risks: 2002: 242;
Number of risks: 2003: 170;
Number of risks: 2004: 145;
Number of risks: Total: 557 (99%)[A].
Source: GAO analysis of FAA data.
[A] Percentages do not add to 100 due to rounding.
[End of table]
An additional strength of the internal risk identification process is
that FAA has established a chain of command for responding to these
internal risks at the appropriate level. For example, local training
issues are likely to be handled by field offices, geographic support
issues would normally be handled by regional offices, and issues that
cannot be resolved by these field and regional offices would be
referred to headquarters.
FAA's Oversight Process Includes Monitoring to Follow Up on Airline
Actions Taken in Response to Findings:
Another strength of FAA's oversight of non-legacy airlines is that
inspectors monitor the actions that those airlines have taken in
response to inspection findings through subsequent inspections and
participation in safety partnership programs with the airlines.
However, the inspectors' monitoring process could be improved by better
data, as we discuss later in this report. SEP requires inspectors to
monitor the actions taken by non-legacy airlines that will address the
risks identified during an inspection and to verify that the actions
taken have resulted or will result in resolution of the problems. This
requirement addresses a concern of ours that the NPG inspection program
did not often ensure that the agency follow-up actions corrected
problems once they were identified.[Footnote 19] In addition, FAA's
requirement for monitoring and verifying corrective action conforms to
a management control standard on the need for federal agencies to have
monitoring and verification policies or procedures for ensuring that
the findings of audits and other reviews are promptly resolved. FAA
inspectors monitor and confirm that corrective action has been taken by
either conducting a special inspection of the area, following up during
a subsequent inspection, or reviewing airline documentation that
addresses the deficiency. According to inspectors and field office
managers, the verification method used is based in many instances on
the severity of the deficiency, with special inspections conducted for
areas that could affect air safety.
According to some FAA inspectors and airline officials, the regular
meetings held to discuss concerns about airlines have opened up
communications both within FAA and between FAA and the airlines. Lack
of communication among the airlines and FAA was, according to the
director of FAA's Flight Standards Service, historically a significant
cause of airline safety problems. FAA inspectors and managers noted
that they maintain an open line of communication with airline
management by meeting regularly with key airline officials (such as
directors of operations and directors of maintenance) to discuss the
results of the SEP risk analyses and the airline's proposed
resolutions.
FAA's Oversight Process Is Hindered by Certain Program Weaknesses and
Human Capital Management Challenges:
FAA's oversight of non-legacy airlines is hindered by the incomplete
implementation of its system safety approach and other challenges.
FAA's incorporation of system safety into its oversight process is
incomplete, as the agency continues to emphasize a nonsafety system-
based process, NPG, to identify most of its inspection activities.
Human capital management challenges also impede FAA's oversight. These
challenges include a hiring freeze that began in January 2005;
attrition; in certain cases, the reallocation of inspectors' duties as
airlines are moved to the ATOS program; insufficient training for
inspectors on risk management, a key element of SEP; and insufficient
communication between FAA headquarters and field offices. In addition,
FAA lacks a process to continuously evaluate SEP. Finally, the agency's
PTRS database lacks some important inspection-related information, such
as whether risks have been mitigated, that would aid in targeting
further oversight activities as well as a nationwide analysis of
inspection results.
Most Inspection Activities Are Not Prioritized Based on a Structured
Risk Assessment Process:
We found that NPG remains the primary basis for FAA's inspection of non-
legacy airlines. For fiscal years 2002 through 2004, about 77 percent
of the required inspection activities for the top 25 non-legacy
airlines were initiated through the NPG annual planning session,
compared with 23 percent that were SEP-initiated. As a result, most of
the required inspection activities are not prioritized based on risk.
(See table 3.) In addition, the total number of required inspections
for non-legacy airlines declined during the 3-year period, while the
number of passengers on those airlines grew from 122 million to 200
million annual enplanements. The decline in inspections during this
period may be due, in part, to the movement of three airlines from SEP
to ATOS. FAA headquarters officials did not know why this decline
occurred, but told us that it may be due to agency efforts to eliminate
unnecessary activities and shift from a "quality control" oversight
approach, in which individual products or outputs are inspected to
determine if they meet specifications, to a "quality assurance"
approach, in which airlines' processes are inspected. While NPG also
includes planned inspections, which have a risk-based element, these
inspections lack the structured approach to risk identification that
SEP has.[Footnote 20] FAA officials acknowledged that the NPG planned
inspections would benefit from the use of risk management tools.
Table 3: SEP-and NPG-Initiated Required Inspections for the Top 25 Non-
legacy Airlines, Fiscal Years 2002-2004:
SEP-initiated;
2002: 1,261;
2003: 1,567;
2004: 927;
Total: 3,755 (23%).
NPG-initiated;
2002: 5,470;
2003: 3,623;
2004: 3,338;
Total: 12,431 (77%).
Total;
2002: 6,731;
2003: 5,190;
2004: 4,265;
Total: 16,186 (100%).
Source: GAO analysis of information from FAA's PTRS database.
[End of table]
According to FAA officials, there are no minimum numbers or percentages
of required SEP inspections. Therefore, a large percentage of NPG-
initiated activities is not unexpected or negative in their view. FAA
officials said NPG-initiated activities are an important part of the
inspection process because they provide information to the agency on an
annual basis for certain types of inspections and enable periodic
assessments of key programs in an airline's operations. According to
FAA, NPG is organized to systematically validate on an annual basis the
performance of all of an airline's safety-critical programs. FAA
considers this type of recurring assessment to be an important system
safety principle and believes that it provides an opportunity to
discover and correct latent failures before they cause safety problems.
In addition, according to FAA officials, NPG identifies a minimum
number of inspections that they believe need to occur to ensure that
certain areas of an airline are reviewed, unlike SEP, which does not
provide the built-in assurance that these areas will be inspected
eventually.
We agree with FAA officials that the agency should not establish
minimum numbers or expected percentages of SEP inspections. However, we
found that the large percentage of required NPG activities compared
with SEP activities occurs, in part, because FAA's guidance places a
greater emphasis on completing required NPG activities. This situation,
along with other factors we discuss below, may deter inspectors from
identifying additional inspections through SEP. FAA's guidance includes
the following:
* Field offices must complete 100 percent of the required NPG-
identified activities by the end of each fiscal year[Footnote 21]
(unless they are retargeted).[Footnote 22]
* Field offices are not required to complete SEP-initiated activities
by the end of the fiscal year and may reschedule them to the following
fiscal year.
In addition, the budget given to field offices for inspections is based
on required NPG-initiated activities, and funding to cover SEP-
initiated activities must come out of that budget. Inspectors have
indicated concern regarding resources for inspections; for example, a
lack of resources, including a lack of travel funds to conduct
inspection activities, made up about 10 percent of the internal risks
identified by inspectors through SEP during fiscal years 2002 through
2004. According to headquarters officials, requests for funding high-
risk SEP-identified inspections are always approved; however, some
noncritical activities may be delayed due to budget constraints. For
example, they said that some certification work--such as certifying
that new aircraft or aircraft parts meet FAA standards--is being
delayed until an inspector travels to the airline to perform another
activity and can do the certification at little or no extra cost.
Under FAA's guidance, if inspectors identify a risk under SEP that they
wish to mitigate through an additional inspection activity, they must
either add the SEP-initiated activity to the list of required NPG-
initiated activities or retarget a required NPG activity and replace it
with the SEP-initiated activity. Adding SEP-initiated activities to the
required NPG-identified activities may increase the overall workload of
inspectors. This situation may affect the number of additional
activities that inspectors identify during the SEP meetings. In fact,
we found that in some cases, no SEP-initiated activities were included
with NPG-initiated activities as part of an airline's overall required
inspections. For example, for one airline overseen by FAA's Central
Region, PTRS data show that, in fiscal year 2002, inspectors conducted
about 400 required NPG-initiated activities and no SEP-initiated
activities. According to FAA headquarters officials, many inspectors
view SEP-initiated activities as additional work. This view was also
held by some inspectors we spoke with.
Moreover, our analysis of PTRS data indicates that retargeting occurs
infrequently. In fiscal year 2002, about 5 percent of all NPG-required
inspections were retargeted by inspectors and 3 percent were retargeted
in fiscal years 2003 and 2004. This low rate occurs, in part, because
field offices have interpreted FAA's emphasis on NPG inspections as
discouraging retargeting. For example, three of the principal
inspection teams for the 16 airlines told us they do not retarget any
NPG activities. Further, in 5 of the 13 field offices we visited, we
were told by both managers and inspectors that they believe NPG
activities take priority over all but the most risky situations
identified through SEP. Similarly, an official with the Professional
Airways Systems Specialists, a union that represents many FAA safety
inspectors, told us that its members believe that FAA's management does
not give as much attention and priority to SEP as it does to ATOS.
FAA's emphasis on required NPG-initiated inspections is also shown
through its data-tracking efforts. According to FAA officials, for
fiscal years 2002 through 2004, all required NPG activities had been
completed by the end of the year. FAA did not know how many completed
SEP-initiated activities were mitigated or required further inspection
activities that were then carried over to the next fiscal year because
it does not collect such information nationwide in PTRS. Moreover,
field office managers also rely on PTRS, which lacks information on the
number of activities that are carried over. For example, one field
office supervisor told us that one inspector had carried over 19
activities from fiscal year 2004 to fiscal year 2005. However, the
field office supervisor said it was difficult to determine how many
additional activities were carried over because PTRS does not track
this information.
Shifts in Inspectors' Workload Present a Challenge to FAA's Inspection
Oversight Process:
The number of FAA inspectors available to oversee non-legacy airlines
will be affected by recent and anticipated trends in attrition and a
hiring freeze. First, for fiscal year 2004, the number of inspectors
(those assigned to non-legacy airlines, ATOS, and general aviation) who
left FAA was greater than the number of inspectors hired, resulting in
a net loss of 67 inspectors. Second, the number of inspectors of
commercial airlines (both ATOS and non-legacy) is expected to continue
to decline from 1,810 at the end of fiscal year 2004 to 1,704 at the
end of fiscal year 2005. Third, from fiscal years 2006 to 2007, the
agency anticipates the number of inspectors of non-legacy airlines will
decline from 1,119 to 1,088. Fourth, for fiscal years 2005 through
2010, FAA estimated that over 1,100 inspectors of non-legacy airlines
and general aviation will leave the agency, with an average loss due to
attrition of about 195 inspectors per year.
FAA will have to hire inspectors to offset the anticipated losses.
However, the agency put a hiring freeze in place in January 2005 for
budgetary reasons, which prevents the agency from hiring inspectors to
replace those lost due to attrition unless they are deemed critical.
Furthermore, in order to absorb a budget reduction in fiscal year
2005,[Footnote 23] FAA expects to eliminate about 200 inspector
positions in the Office of Aviation Safety. These reductions will be
accomplished through attrition. In fiscal year 2006, the office expects
the staffing level to increase by about 80, which would still be below
the fiscal year 2004 staffing level.[Footnote 24] FAA is filling safety-
critical positions that become vacant through internal appointments.
According to FAA officials, principal inspectors and managers are
considered safety critical. As other safety inspectors, including
geographic inspectors, leave the agency or are appointed to safety-
critical positions, they are not being replaced at this time, according
to those officials. Moreover, their workload is being divided among the
remaining inspectors, thereby increasing the inspectors' workload,
which has resulted in other, less critical, work being delayed or
deferred. In 11 of the 13 FAA field offices we contacted, officials
indicated a shortage of different types of inspectors needed to oversee
the non-legacy airlines. For example, the Washington, D.C., field
office did not have a principal avionics inspector. Officials with FAA
field and regional offices said they have developed ways to deal with
the inadequate staffing of inspectors in their offices. For example,
the Southern Regional Office told us that it uses geographic inspectors
more frequently than in the past to help conduct inspections rather
than having the inspectors assigned to the non-legacy airline travel to
conduct them. However, two field offices---Chicago and Minneapolis---
indicated the need for more geographic inspectors. Another way that FAA
offices are dealing with inspector shortages is by having an inspector
perform inspections for several offices. For example, the Atlanta field
office has been without a cabin safety inspector for a year and uses
the inspector assigned to the Southern Regional Office. According to an
FAA inspector, some inspections are delayed due to lack of staff
availability. In addition, according to our analysis of FAA data
(discussed earlier in this report), for fiscal years 2002 through 2004,
principal inspectors identified lack of staff for the airlines'
certificate management teams as among the top two risks for each year,
and overall, more than one-third of the risks identified were related
to lack of inspectors.
The move of additional airlines into ATOS, which requires more
inspectors per airline than SEP, also affects the workload of
inspectors for non-legacy airlines. In addition, unlike SEP inspectors,
ATOS inspectors are dedicated to an airline and generally cannot be
used to conduct inspections of other entities. When four airlines were
recently transitioned into ATOS, the total size of the four inspection
teams increased 30 percent, from 73 to 95 inspectors, as shown in table
4. As a result, the number of inspectors available to oversee non-
legacy airlines was reduced not only by those 73 who had covered the
four airlines while they were under the NPG and SEP processes (who
would have also had responsibilities for other entities) but by an
additional 22 inspectors as well.
Table 4: Allocation of FAA Inspection Staff under SEP and ATOS for Four
Airlines:
Airline: Champion;
Number of FAA inspectors under SEP: 4;
Number of FAA inspectors under ATOS: 6;
Change in number of inspectors: +2.
Airline: American Eagle;
Number of FAA inspectors under SEP: 31;
Number of FAA inspectors under ATOS: 35;
Change in number of inspectors: +4.
Airline: ExpressJet;
Number of FAA inspectors under SEP: 18;
Number of FAA inspectors under ATOS: 24;
Change in number of inspectors: +6.
Airline: SkyWest;
Number of FAA inspectors under SEP: 20;
Number of FAA inspectors under ATOS: 30;
Change in number of inspectors: +10.
Total;
Number of FAA inspectors under SEP: 73;
Number of FAA inspectors under ATOS: 95;
Change in number of inspectors: +22.
Source: FAA.
[End of table]
FAA is aware that transitioning additional inspectors to airlines in
ATOS is putting a burden on the workload of those inspectors who
continue to inspect non-legacy airlines, and the agency has, in one
case, revised its procedures for dedicating staff to individual ATOS
airlines. When Champion Airlines became part of ATOS in January 2005,
FAA changed ATOS policy to allow the Northwest Airlines inspection team
to share its data analyst and manager with the Champion inspection
team. FAA officials believe that sharing staff resources will address
the issue of staff shortages.
All of the inspectors at the offices we contacted had other duties in
addition to overseeing non-legacy airlines. The inspectors performed
activities such as certifying and approving new aircraft types;
overseeing repair stations and aviation schools; reviewing FAA
directives and aircraft updates; responding to complaints;
investigating accidents, incidents, and complaints; overseeing
designees;[Footnote 25] and attending training. Although primarily
responsible for a particular airline, some principal inspectors
conducted inspections of other airlines and entities. For example, a
principal inspector in the Great Lakes Region was responsible for
overseeing three repair stations in addition to a non-legacy airline.
In addition, a principal operations inspector in the Southern Region
was also responsible for overseeing designees such as pilot examiners.
Our analysis of the inspectors' workload contained in the PTRS database
showed that for fiscal years 2002 through 2004, about 75 percent of FAA
inspectors had responsibility for more than 3 entities, and about half
had responsibility for more than 15. While the size and complexity of
the various entities will dictate the inspector workload, having
responsibility for a large number of entities can increase an
inspector's workload because of the need to be familiar with the
operating procedures of each entity as well as to spend time physically
inspecting each entity.
Inspectors Are Concerned That Training Limitations Present a Challenge
to Inspection Oversight Process:
While FAA provided system safety training to its inspector workforce,
as mentioned earlier in this report, the lack of sufficient training in
other areas was identified by principal inspectors during their risk
analysis meetings and by inspectors we spoke with.[Footnote 26] Our
analysis of FAA's database of internal risks identified by principal
inspectors of non-legacy airlines indicated that 42 percent (233 out of
558) of the internal risks identified for fiscal years 2002 through
2004 related to training. Although FAA policy requires that principal
inspectors assigned to non-legacy airlines complete technical training
on the airline's specific policies, procedures, and equipment before
they can conduct airline inspection activities, principal inspectors
identified risks associated with the lack of such training. For
example, our analysis of the internal risk database found that
principal inspectors identified a lack of technical training for
themselves or other inspectors on their team on specific aircraft,
engine types, navigation equipment, and avionics that pertained to the
airlines they were overseeing.[Footnote 27] Others cited a lack of
training in cargo loading, hazardous material identification, or weight
and balance. Still others indicated the need for training in FAA's
systems or processes. For example, several indicated that they had not
received the basic courses for principal inspectors (such as
"foundation for principal inspectors"). As another example, several
principal inspectors reported that certain inspectors on their team
lacked training in using the computer system that analyzes information
collected in PTRS, noting that without this training they were unable
to directly access the system. Even though inspectors expressed these
concerns, approximately 87 percent of identified internal risks have
been closed by FAA, according to our analysis of FAA data. However, we
could not determine how many of those risks were actually mitigated, as
FAA closes risks either when action has been taken to mitigate the risk
or because the agency has determined that the level of risk is
acceptable and no action is warranted.
During our field office visits, inspectors also spoke about the lack of
technical training in certain areas. For example, inspectors told us
they had not received training on some newer aircraft they were
responsible for inspecting and that they were not always able to attend
free training provided by the manufacturers, which they felt would help
them obtain needed technical knowledge. We found instances where
management acknowledged that such training was needed. In the past,
concerns had been raised that FAA's acceptance of free training from a
regulated entity might have the appearance of a conflict of interest.
In September 2005, we reported that FAA accepts such training under
limited circumstances.[Footnote 28] FAA has established safeguards to
help preclude the appearance of a conflict of interest when FAA accepts
training from the entities it regulates at no cost to the agency or in
exchange for an in-kind service.[Footnote 29]
In addition, the principal inspectors we spoke with indicated that some
geographic inspectors who assist them in providing oversight of non-
legacy airlines lack the technical and SEP-specific procedural training
to maximize their usefulness. Several field office managers and
principal inspectors told us that geographic inspectors needed to be
knowledgeable about the operations of airlines they inspect in order to
provide effective assistance. Several principal inspectors indicated
that misperceptions about an airline's operating requirements are not
uncommon because geographic inspectors often lack training on the
aircraft they inspect.[Footnote 30] As a result, the inspectors
indicated that geographic inspectors had at times incorrectly coded
inspection information in PTRS and erroneously identified risks, which
resulted in principal inspectors having to spend time determining that
a problem did not actually exist. In addition, several airline
officials told us that geographic inspectors had incorrectly identified
problems and appeared to not understand airlines' operational
procedures. For example, we were told that a geographic inspector
improperly cited an airline for not complying with a deicing program
that the airline had previously informed FAA inspectors it was
changing. FAA headquarters officials told us that it is not feasible to
train all geographic inspectors on all airlines they may encounter.
Further, they noted it would be good for geographic inspectors to
contact principal inspectors when they are unsure of whether an action
or condition they observe is a problem. However, FAA headquarters has
not provided that instruction to the geographic inspectors. Industry
representatives also told us this was a problem that could be
alleviated by inspectors having access to airlines' operating manuals
online.
Another issue identified as a problem by principal inspectors we spoke
with is that not all geographic inspectors are provided training in
SEP, including how to use the uniform coding scheme that was developed
for SEP to facilitate the identification and analysis of risks.
Geographic inspectors may therefore note the results of their
inspections through narrative in the comments section of PTRS rather
than through SEP codes, so that the principal inspectors must read
through all the comments in order to identify geographic inspectors'
concerns. As PTRS may contain thousands of entries on one airline each
year, reading every narrative comment can be time consuming and
difficult to interpret. The lack of coding of geographic inspectors'
findings could therefore result in principal inspectors missing
problems or risks identified by geographic inspectors. We do not know
how often this problem occurs, as FAA has not assessed the reliability
of this data in PTRS. However, several FAA staff brought this issue to
our attention. One field office manager and SEP coordinator told us
that PTRS entries by geographic inspectors are not helpful because they
do not provide meaningful comments for FAA's risk assessments, and
several principal inspectors told us that training for geographic
inspectors on SEP coding would help them more easily identify risks
during their periodic meetings.
These training issues may arise in part because FAA has not
systematically identified technical training needs for principal or
other inspectors. Our previous report found that FAA has not
systematically identified overall training needs of its inspectors to
ensure that the curriculum addresses the unique training needs of each
type of inspector.[Footnote 31] Instead, course development focuses on
individual courses. FAA said it recognizes that it manages courses as
individual components of an overall curriculum that is only loosely
defined and that it needs to develop courses and address training needs
as part of an overall curriculum. It has established a curriculum
transformation plan that it estimates it will fully implement in 2008.
In responding to our survey, inspectors had mixed views on the
usefulness of technical training.[Footnote 32] For about three-
quarters[Footnote 33] of the responses, roughly equal percentages of
inspectors of non-legacy airlines responded that the technical training
they received in the last 2 years[Footnote 34] helped them do their
current jobs to some extent, to a moderate extent, or to a great
extent. On the other hand, 6 percent indicated that the technical
training had not helped them do their jobs and another 26 percent
indicated it had only helped to some extent. (See fig. 3.)
Figure 3: Non-legacy Airline Inspectors' Views on the Extent Technical
Training Received in the Last 2 Years Has Helped Them Do Their Jobs:
[See PDF for image]
Note: Approximately 10 percent of inspectors responded that they had no
basis to judge or did not know.
[End of figure]
Further, the timeliness of certain training was identified as a concern
by both principal inspectors we spoke with and inspectors responding to
our survey. Specifically, some principal inspectors we spoke with
indicated a lack of timeliness for SEP training. Since FAA's
introduction of the SEP training for non-legacy inspectors in 2001, a
total of 700 inspectors have received the training---just over 300
inspectors between 2001 and 2002 and almost 400 inspectors between 2003
and 2004. This means that less than 40 percent of the approximately
1,800 inspectors of both ATOS[Footnote 35] and non-legacy airlines (as
of fiscal year 2004) could have received SEP training. FAA requires all
principal inspectors and other inspectors who are part of certificate
management teams to receive SEP training. Geographic inspectors are not
part of those teams and may not receive that training. While all of the
principal inspectors who were on staff when SEP was implemented in 2002
received initial SEP training, according to FAA, since that time, newly
assigned inspectors may not have received the training before beginning
their responsibilities. Four principal inspectors who were assigned
relatively recently to non-legacy airlines and were not provided the
initial SEP training told us they had received subsequent training
given to all inspectors to explain revisions to the SEP program. They
said that the lack of initial SEP training hindered their ability to
understand SEP terms and processes and participate fully in the
periodic meetings to analyze information from inspections to identify
risks. Those inspectors also believed that the subsequent SEP training
was not as helpful without the initial SEP training.
The views of the inspectors we spoke with are consistent with our
survey results, which indicated that many inspectors of non-legacy
airlines are not highly satisfied with the timeliness of technical
training they say they need to do their jobs. Our survey found that
only about 16 percent of non-legacy inspectors believe to a great or
very great extent that they have received technical training in a
timely manner to do their jobs. Moreover, about the same percentage (15
percent) of the inspectors reported that the timing of their training
had not been useful to their jobs at all. (See fig. 4.)
Figure 4: Views of Inspectors of Non-legacy Airlines on the Extent
Inspectors Have Received Technical Training in a Timely Manner during
Their FAA Careers:
[See PDF for image]
Note: Approximately 1 percent of inspectors responded that they did not
know.
[End of figure]
About one-third of the inspectors[Footnote 36] in our survey indicated
that the training they received in the automated systems used in the
inspection process--including PTRS--was provided in a timely manner to
a moderate extent. Another 30 percent thought the training was only
somewhat timely.(See fig. 5.) Timely training on automated systems is
important in order to have accurate data. Inaccurate PTRS data hinder
the identification of risks by principal inspectors, as we discussed
previously in this report.
Figure 5: View of Inspectors of Non-legacy Airlines on the Extent to
Which They Received Training in Automated Systems Used in the
Inspection Process in a Timely Manner:
[See PDF for image]
Note: Approximately 1 percent of inspectors responded that they did not
know.
[End of figure]
FAA Lacks Effective Communication in How Internal Risks Are Resolved:
Some inspectors told us that when they identified internal risks to
FAA, they were able to see the results of actions taken by their field
and regional offices to address the issues, but that they did not know
what actions, if any, were taken by FAA headquarters. This situation
indicates the lack of a key management control that calls for
information to be recorded and communicated to individuals within an
entity who need the information in a form and within a time frame that
enable them to carry out their responsibilities. However, unless the
inspectors have access to a secure intra-agency Web site and know how
to locate the issue they submitted, they do not know how FAA
headquarters has responded to the risks because this information is not
directly provided to them by headquarters. Headquarters officials
acknowledged that there is no formal feedback loop by which to inform
the inspectors about the issues that they raised. In prior work, we
found that a lack of communication with inspector staff on the
resolution of enforcement actions that resulted from inspections is
frustrating to inspectors and might be a disincentive to reporting
violations they find during inspections.[Footnote 37] This suggests
that a similar lack of communication could reduce inspectors' interest
in identifying internal risks to FAA. We found that the overall number
of times internal risks were identified by inspectors declined for
fiscal years 2002 through 2004.
PTRS Data Are Missing Elements That Would Make the Data More Useful:
The PTRS database has limitations that reduce the usefulness of the
data in helping inspectors ensure that they are effectively conducting
inspections. PTRS does not facilitate FAA's ability to monitor whether
risks identified through SEP have been mitigated. While the SEP process
generates required inspection activities based on risks that have been
identified through analysis of prior inspections, inspectors record
these inspection activities as closed in the PTRS database once they
have completed the inspection, whether or not the risks identified in
SEP have been mitigated, according to some principal inspectors we
spoke with. While inspectors are expected to use the comments section
in PTRS to record additional information, such as whether identified
risks have been mitigated, they do so infrequently. Our analysis of
about 1.8 million PTRS records showed that fewer than 400 records
included any form of the words "mitigate," "fix," or "resolve."
Moreover, such analysis is time consuming to conduct because a uniform
coding scheme is not employed for entering the information. As a
result, managers are not easily able to use PTRS data to monitor
whether risks identified through SEP have been mitigated and lack data
that would be useful in evaluating SEP. It is left to inspectors to
determine subsequently if the risks have been mitigated, usually
through follow-up on-site inspections.
In addition, while a specific risk identified through SEP might result
in multiple inspections, there is no clear linkage between these
inspections when they are recorded in PTRS, according to a regional SEP
manager. For example, if a risk was identified through SEP-generated
inspection activities that included looking at manuals, checking
aircraft records, and performing a ramp check, inspectors would have to
review all comment records to identify the resulting activities in PTRS
that are related to that particular risk. This lack of linkage in PTRS
may reduce inspectors' overall understanding of an airline's situation.
Inspectors who do not know why activities have been generated may not
target or fit their work to assess these identified risks, while a
greater understanding of how inspection activities are connected to
identified risks could help ensure that inspections are conducted in a
way that maximizes the benefits of a risk-based approach.
FAA Has Not Evaluated Its Inspection Oversight Process or Linked It to
Agency Goals:
FAA has not evaluated its inspection oversight process for non-legacy
airlines to determine how the process contributes to the agency's
mission and overall safety goals. In addition, FAA has not explicitly
linked SEP to the overall safety goals. According to management control
standards for federal agencies and our prior reports on results-
oriented management,[Footnote 38] federal agencies should establish
measurable performance goals for their programs and operations.
Agencies should also have an evaluation process for their programs so
that agency officials, Members of Congress, and others will be able to
determine whether goals are being achieved.
While FAA has not established a specific goal for SEP, it has an
overall goal to achieve the lowest possible accident rate and
constantly improve safety. To address this goal, FAA's strategy
includes expanding cost-effective safety oversight and surveillance by
targeting its inspection resources better. However, FAA's strategy does
not explicitly show how SEP contributes to the safety goal. In
addition, FAA has not yet evaluated SEP to determine if it is achieving
the agency's goal. As a consequence of not having an evaluative process
for SEP, FAA does not have the information it needs to determine what
changes should be made to improve its system safety process. Also, FAA
does not have a process to examine the nationwide implications of or
trends in the risks that inspectors have identified through their risk
assessments. Consequently, FAA does not have the information it needs
to proactively determine on a continuous basis risk trends at the
national level. While FAA has an evaluation office, this office is not
doing this type of analysis; rather, the office conducts analyses of
the types of inspections generated under SEP by airline and FAA region,
according to the manager of that office. One FAA field office we
contacted had taken the initiative to analyze risks identified during
the SEP meetings for trends. In FAA's Eastern Region, a manager
routinely analyzes trends in types of findings within an airline and
across airlines and provides this information to the inspectors. This
is an informal process, and it is not certain that it will continue
since the manager has been reassigned to another office. Moreover, such
analyses at the local or regional level do not fulfill the management
control to have a process in place to determine if program goals are
being achieved.
In addition, we found that while FAA has created specific national
goals for NPG that call for completing all required inspections each
year, in some cases these goals impede the agency's ability to
effectively implement its system safety approach. For example, in order
to meet the annual goal, the regional offices have established interim
goals to complete 25 percent of the required inspections each quarter.
Inspectors in one field office told us that this situation created a
disincentive for identifying additional activities under SEP due to
concerns about completing the quarterly requirements. Those inspectors
also noted that these goals encouraged them to prioritize their work
based on what they can most easily accomplish, rather than on what
represents the most significant risks--the antithesis of a system
safety approach.
Conclusions:
Our review of FAA's oversight of non-legacy airlines suggests that the
full benefits of a system safety approach can be realized only if the
approach is more fully implemented and utilized. FAA's articulation of
its system safety approach, its application of system safety principles
to its oversight process through SEP, and its widely distributed
training on system safety are positive steps toward improving oversight
by using the advantages of system safety--particularly the ability to
identify and prioritize inspections based on risk. Fully developing SEP
is important since FAA has been unable to move significant numbers of
airlines to ATOS and has only recently established the goal to do so by
the end of fiscal year 2007. A process such as SEP is needed to
identify risks in and among non-legacy airlines and system-wide
problems. However, the usefulness of FAA's system safety approach is
reduced by limitations in the implementation of SEP--such as FAA
headquarters' predominant focus on NPG, which has led to only a small
percentage of inspection activities being SEP-initiated; the lack of
training of geographic inspectors on SEP codes that could make their
inspections more useful to identifying risks during the SEP process;
and the lack of linkage to national goals and evaluations of SEP. Until
SEP is more fully implemented, it is clear that FAA's approach to
overseeing non-legacy airlines is not largely risk based. Unless SEP is
more thoroughly integrated into its oversight, FAA may not be fully
maximizing the efficiency of its inspections in identifying and
mitigating risks in order to ensure the safety of non-legacy airlines.
With FAA operating under a hiring freeze and the number of inspectors
available for non-legacy airlines possibly further reduced by attrition
and the move of inspection staff to the ATOS program, the need to
maximize the effectiveness of inspection activities in ensuring the
safety of non-legacy airlines is even more critical.
Recommendations for Executive Action:
To improve the effectiveness of the agency's oversight of non-legacy
airlines, we recommend that the Secretary of Transportation direct the
FAA Administrator to implement the following four recommendations:
* To improve its safety oversight of airlines, FAA should (1) develop a
continuous evaluative process for its activities under SEP and link SEP
to the performance-related goals and measures developed by the agency,
track performance towards agency goals, and determine appropriate
program changes. The evaluation should include an analysis of
inspection findings to identify trends and risks at the national level.
* In order to ensure that all regional and field offices have a
complete and timely understanding of FAA's policies relating to the
inspection process, FAA needs to (2) improve communication in areas
such as whether and how internal risks identified by inspectors have
been resolved and (3) improve training in areas such as risk
management, coding items in the PTRS database, and how and under what
circumstances SEP-identified activities can replace NPG-identified
activities through retargeting.
* To better utilize geographic inspectors' support, FAA needs to (4)
improve the geographic inspectors' understanding of the system safety
approach and operations of the airlines they inspect. FAA should
consider actions such as additional training, additional oversight in
particular areas, having airlines' operating manuals available online
for review by inspectors, and improving communication between
geographic inspectors and principal inspectors on issues related to
identifying safety violations.
Agency Comments and Our Evaluation:
We provided a draft of this report to DOT for review and comment. We
obtained oral comments from FAA and DOT officials, including FAA's
Deputy Associate Administrator for Aviation Safety. In particular, FAA
officials made the point that planned NPG activities have an element of
risk identification. We agree and revised the report. The FAA officials
generally agreed with our recommendations to improve communication,
training, and geographic inspectors' understanding of the system safety
approach. The officials said that FAA would consider our recommendation
to develop a continuous evaluative process for SEP and link it to the
agency's goals, but that its plan to eventually place the remaining non-
legacy airlines in the ATOS program might make this recommendation
unnecessary. In the past, FAA's efforts to move airlines to ATOS have
experienced delays, therefore, we retained this recommendation. FAA
officials also provided clarifying comments and technical corrections,
which we incorporated as appropriate.
As agreed with your office, unless you announce the contents of this
report earlier, we plan no further distribution until 14 days from the
report date. At that time, we will send copies of this report to other
congressional committees, the Secretary of Transportation, and the
Administrator, FAA. We will also make copies available to others upon
request. In addition, the report will be available at no cost on GAO's
Web site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-2834 or [Hyperlink, dillinghamg@gao.gov].
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix III.
Signed by:
Gerald L. Dillingham, Ph.D.
Director, Physical Infrastructure Issues:
[End of section]
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
The objective of this report is to assess the Federal Aviation
Administration's (FAA) processes for ensuring the safety of a fast-
growing portion of the commercial airline industry--the non-legacy
passenger airlines.[Footnote 39] Specifically, this report responds to
the following questions: (1) What are the strengths of FAA's inspection
oversight of non-legacy passenger airlines? (2) What issues hinder the
effectiveness of FAA's inspection approach?
The scope of our review included FAA's oversight activities from fiscal
year 2002--when FAA implemented a new process (the Surveillance and
Evaluation Program or SEP) to its oversight of non-legacy airlines--
through fiscal year 2004. To address the two questions, we obtained and
analyzed information about FAA's oversight of the top 25 non-legacy
airlines ranked according to the number of enplanements in 2004. Those
25 airlines are identified in table 5, and each was covered by SEP
during the entire 3-year period or a portion of the period.[Footnote
40] Together, these 25 airlines accounted for about 90 percent of all
non-legacy passenger enplanements in 2004.
Table 5: Number of Enplanements for the Top 25 Non-legacy Airlines,
2004:
Rank: 1;
Airline: American Eagle[A];
Enplanements: 14,869,258.
Rank: 2;
Airline: ExpressJet[A];
Enplanements: 13,664,642.
Rank: 3;
Airline: SkyWest[A];
Enplanements: 13,417,720.
Rank: 4;
Airline: Airtran;
Enplanements: 13,178,118.
Rank: 5;
Airline: Comair;
Enplanements: 12,637,210.
Rank: 6;
Airline: JetBlue;
Enplanements: 11,731,733.
Rank: 7;
Airline: Atlantic Southeast;
Enplanements: 10,427,885.
Rank: 8;
Airline: American Trans Air;
Enplanements: 10,340,914.
Rank: 9;
Airline: Mesa;
Enplanements: 9,122,237.
Rank: 10;
Airline: Atlantic Coast;
Enplanements: 7,046,971.
Rank: 11;
Airline: Air Wisconsin;
Enplanements: 6,954,187.
Rank: 12;
Airline: Frontier;
Enplanements: 6,437,921.
Rank: 13;
Airline: Pinnacle;
Enplanements: 6,362,805.
Rank: 14;
Airline: Horizon;
Enplanements: 5,930,448.
Rank: 15;
Airline: Chautauqua;
Enplanements: 5,608,947.
Rank: 16;
Airline: Mesaba;
Enplanements: 5,427,694.
Rank: 17;
Airline: Hawaiian;
Enplanements: 5,234,766.
Rank: 18;
Airline: Spirit;
Enplanements: 4,592,640.
Rank: 19;
Airline: Aloha;
Enplanements: 4,187,019.
Rank: 20;
Airline: Trans States[A];
Enplanements: 3,462,869.
Rank: 21;
Airline: Executive;
Enplanements: 2,796,163.
Rank: 22;
Airline: Midwest Express;
Enplanements: 2,376,304.
Rank: 23;
Airline: PSA;
Enplanements: 2,030,870.
Rank: 24;
Airline: Piedmont;
Enplanements: 1,948,292.
Rank: 25;
Airline: Ryan International;
Enplanements: 1,626,437.
Source: GAO analysis of U.S. Department of Transportation information.
[A] These airlines were transitioned into the ATOS program from 2003
through 2005.
[End of table]
To determine FAA's legal oversight responsibility for commercial
passenger airlines, we obtained and analyzed regulations that govern
FAA oversight of these airlines. We also obtained and reviewed FAA
handbooks, procedures, and orders that describe the role and
responsibilities of FAA managers and inspectors[Footnote 41] in
implementing the two inspection oversight processes for non-legacy
airlines--SEP and the National Work Program Guidelines (NPG). We
conducted literature searches and reviewed prior reports and articles
on the oversight processes, including those prepared by the Department
of Transportation's Inspector General and us. We reviewed documentation
provided by FAA, and contacted officials there and at GAO, to determine
whether findings identified in those reports have been addressed. We
also reviewed literature on system safety and compared it with FAA's
system safety framework, which is incorporated in SEP.
We interviewed FAA headquarters officials from the Office of Aviation
Safety and its Flight Standards Service to obtain descriptions of NPG
and SEP. To understand how these processes were implemented, we
collected information from 7 regional offices and 13 field and
certificate management offices. We selected these offices because they
have oversight responsibility for most of the top 25 non-legacy
airlines ranked by the number of enplanements in 2004. We also
interviewed officials from 16 of the 25 non-legacy airlines. The
regional offices, field and certificate management offices, and
airlines we interviewed are shown in table 6. For each FAA office, we
conducted semistructured interviews with managers, SEP coordinators,
and inspectors to obtain information on how they implemented NPG and
SEP. We also collected information on inspector staffing levels,
workload, and training. In addition, we reviewed whether management
controls have been established for NPG and SEP,[Footnote 42] and
determined if they were linked to FAA's overall safety performance
goals. These controls included the (1) establishment of performance
goals, (2) verification of inspection results, (3) management of the
inspection database (Program Tracking and Reporting Subsystem or PTRS),
and (4) communication among managers and inspectors relating to NPG and
SEP. We also interviewed airline safety officials to obtain their views
on FAA's inspection and surveillance activities and their participation
in FAA-industry partnership programs.
Table 6: FAA Offices and Airlines Interviewed by GAO:
Regional office: Central;
FAA field office: None;
Airline: None.
Regional office: Eastern;
FAA field office: Baltimore Flight Standards District Office (FSDO);
Airline: Piedmont.
FAA field office: Garden City FSDO;
Airline: JetBlue.
FAA field office: Washington FSDO;
Airline: Independence Air[A].
Regional office: Great Lakes;
FAA field office: Detroit FSDO;
Airline: Spirit.
FAA field office: Indianapolis FSDO;
Airline: American Trans Air.
Airline: Chautauqua.
FAA field office: Minneapolis FSDO;
Airline: Mesaba.
FAA field office: Chicago FSDO;
Airline: Air Wisconsin.
Airline: Midwest Express.
Regional office: Northwest Mountain;
FAA field office: None;
Airline: None.
Regional office: Southern;
FAA field office: Atlantic Southeast (Atlanta) Certificate Management
Unit;
Airline: Atlantic Southeast.
FAA field office: Comair (Louisville) Certificate Management Office
(CMO);
Airline: Comair.
FAA field office: Orlando FSDO;
Airline: Executive.
FAA field office: Airtran (Orlando) CMO;
Airline: Airtran.
FAA field office: Memphis FSDO;
Airline: Pinnacle.
Regional office: Southwest;
FAA field office: None;
Airline: None.
Regional office: Western Pacific;
FAA field office: Honolulu FSDO;
Airline: Aloha.
Airline: Hawaiian.
Source: GAO.
[A] Formerly Atlantic Coast Airlines.
[End of table]
We also conducted interviews with seven industry organizations to
obtain their views on FAA's oversight processes. (See table 7.) We
selected these organizations because they represented diverse segments
of the aviation industry. In addition, we selected organizations that
met one or more of the following criteria: membership inspected by FAA,
familiarity with FAA's safety oversight of airlines, familiarity with
FAA's SEP or Air Transportation Oversight System processes, and
familiarity with inspector training.
Table 7: Organizations Interviewed by GAO:
Type of organization: Trade associations and unions;
Organization contacted: Aircraft Mechanics Fraternal Association;
Professional Aviation Maintenance Association; Professional Airways
Systems Specialists.
Type of organization: Transportation safety organization;
Organization contacted: National Transportation Safety Board.
Type of organization: Organizations whose members are certificated by
FAA and subject to FAA oversight;
Organization contacted: Regional Airline Association; Aeronautical
Repair Station Association; Airline Pilots Association International.
Source: GAO.
[End of table]
We analyzed information about FAA's inspection activities for the top
25 non-legacy airlines using the agency's PTRS database for fiscal
years 2002 through 2004. Specifically, we analyzed information on the
number of required NPG and SEP inspections, inspector workload (i.e.,
number and type of entities inspected), and number of NPG activities
that were retargeted (i.e., NPG-identified activities that were
replaced with SEP-identified activities). This analysis excluded
enroute activities, which take place when an inspector monitors an
aircraft as it travels from one destination to another. These
inspections were excluded because inspectors often use these as
secondary inspections as a means to travel to a location where their
primary inspection is such as a repair station. In addition, we
analyzed data that FAA maintains on the internal risks identified by
inspectors through regular SEP meetings. For fiscal years 2002 through
2004, we examined the type, severity, and closure status of the
identified risks. We also analyzed comment fields in that data to
determine if they indicated that the risks had been mitigated.
To assess the reliability of the data used in this report, we
interviewed knowledgeable agency officials about the data, performed
electronic testing of relevant data fields for obvious errors in
accuracy and completeness, and collected and reviewed documentation
from data system managers about the data and the systems that produced
them. We determined that the data were sufficiently reliable for the
purposes of this report.
To gather information about inspectors' perspectives on the technical
training available to them, we conducted a Web-based survey of a
representative sample of FAA safety inspectors. The survey asked a
combination of questions concerning the amount and timeliness of
training that allowed for open-ended and close-ended responses. We had
no practical way to assess information on the amount of training
necessary for inspector proficiency or the timeliness of the training
provided. We drew a stratified random probability sample of 496
inspectors from the population of 2,989 FAA aviation safety
inspectors.[Footnote 43] We stratified the population into 12 groups on
the basis of the type of work the inspector performed. Each sample
element was subsequently weighted in the analysis to account
statistically for all members of the population. For this report, we
used a subgroup sample of 205 non-legacy safety inspectors.
Because we followed a probability procedure based on random selections,
our sample is only one of a large number of samples that we might have
drawn. Since each sample could have provided different estimates, we
express our confidence in the precision of our overall sample's results
as a 95 percent confidence interval (e.g., plus or minus 7.1 percent).
This is the interval that would contain the actual population value for
95 percent of the samples we could have drawn. As a result, we are 95
percent confident that each of the confidence intervals in this report
will include the true values in the study population. The percentage
estimates for the non-legacy subgroup has a margin of error of plus or
minus 7.1 percent or less. Survey estimates presented as comparisons
between groups are statistically significant when the 95 percent
confidence intervals do not overlap.
The survey was conducted using self-administered electronic
questionnaires accessible on the Internet through a secure Web browser.
We sent e-mail notifications to 496 inspectors (205 of which were in
our subgroup of inspectors of non-legacy airlines) beginning on
December 6, 2004. We then sent each potential respondent a unique
password and user name to ensure that only members of the target
population could participate in the survey. To encourage respondents to
complete the questionnaire, we sent a subsequent e-mail message to
further prompt each nonrespondent approximately 2 weeks after the
initial e-mail message. We sent nonrespondents two more notices and
closed the survey on February 4, 2005. Of the 496 inspectors we
surveyed, we received 392 usable responses (79 percent). Among our
subgroup of 205 non-legacy inspectors, we received 161 usable responses
(79 percent).
In addition to sampling errors, the practical difficulties in
conducting surveys of this type may introduce other types of errors,
commonly referred to as nonsampling errors. For example, questions may
be misinterpreted, or the respondents' answers may differ from those of
the inspectors who did not respond. We took steps to reduce these
errors.
Finally, we pretested the content and format of the questionnaire with
safety inspectors at local FAA offices in Baltimore, Los Angeles, and
Seattle. During the pretests we asked the inspectors questions to
determine whether (1) the survey questions were clear, (2) the terms
used were precise, (3) the questionnaire placed an undue burden on the
respondents, and (4) the questions were unbiased. We made changes to
the content and format of the final questionnaire based on the pretest
results.
We conducted our work from August 2004 through September 2005 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix II: FAA's Surveillance and Evaluation Process:
SEP Incorporates System Safety into FAA's Inspection Oversight of Non-
legacy Airlines:
The Surveillance and Evaluation Program (SEP) is a process designed to
introduce a data driven risk analysis system for non-legacy airlines
and is guided by a Surveillance and Evaluation Assessment Tool (SEAT).
SEP models itself on a system safety approach that is incorporated in
the oversight of legacy airlines through the Air Transportation
Oversight System (ATOS). SEP allows teams of inspectors to identify
suspected trends through a data based system rather than relying
totally on past experiences. A certificate management team oversees
each of the 99 airlines under SEP. Each team is led by three principal
inspectors, one for each major area of inspections (operations,
maintenance, and avionics). Additional team members include those based
at the FAA office that holds the airline's operating certificate--
typically an aircraft dispatch inspector, a cabin safety inspector, and
assistants.
The principal inspectors are required to meet as a team at least twice
a year to assess the safety risks associated with the airlines they
oversee and develop a surveillance plan for the airline, including
designating inspection duties to other FAA offices at locations to
which the airline flies or conducts business. In some field offices,
teams meet each quarter. The teams use a planning tool--SEAT--to assess
the systems in place at an airline and to identify any potential
internal and external risks. Using SEAT, the team analyzes 10 systems,
shown in table 8.
Table 8: Safety Systems and Examples of Risk Indicators in SEAT:
System and purpose: Aircraft configuration and control: maintains the
physical condition of the aircraft and associated components;
Selected examples of risk indicators in SEAT:
1. Does the airline have the proper maintenance and inspection programs
in place?
2. Do the airline and vendor follow their fueling procedures, policies,
and controls?
3. Is the airline following its approved deicing procedures and
policies?
System and purpose: Manuals: controls the information and instructions
that define and govern an airline's activities;
Selected examples of risk indicators in SEAT:
1. Is content consistent and complete across manuals?
2. Are manuals up-to-date and available?
System and purpose: Flight operations: governs aircraft movement;
Selected examples of risk indicators in SEAT:
1. Are passengers boarding in a safe environment?
2. Are airlines ensuring that cargo is handled and carried safely
according to their policies?
3. Does the airline adhere to its approved weight and balance program?
System and purpose: Personnel training and qualifications: ensures that
an airline's personnel are trained and qualified;
Selected examples of risk indicators in SEAT:
1. Does the airline adhere to its training programs for crew members
(attendants, dispatchers, station personnel, check airman and
instructors, and maintenance personnel)?
2. Are current and appropriate certifications of personnel available
upon request, and does the airline have the programs, policies, and
procedures in place to ensure certificates are valid?
System and purpose: Route structures: maintains an airline's facilities
on approved routes;
Selected examples of risk indicators in SEAT:
1. Does the airline adhere to its policies on weather-reporting
facilities?
2. Do the airline's maintenance and service facilities comply with its
policies, procedures, and controls?
System and purpose: Airman/crew member flight, rest, and duty time:
prescribes time limitations for airline employees;
Selected examples of risk indicators in SEAT:
1. Does the airline adhere to its policies on flight crew (attendant or
dispatcher) flight/duty/rest time?
2. Does the airline adhere to its procedures and controls for its
scheduling and reporting system?
System and purpose: Technical administration: addresses other aspects
of an airline's certification and operations, including key safety
personnel and programs;
Selected examples of risk indicators in SEAT:
1. Does the director of maintenance (chief inspector, director of
operations, and chief pilot) accomplish assigned duties and
responsibilities?
2. Does the director of safety effectively administer the safety
program?
System and purpose: Risk indicators: reflects the impact external and
internal events have on an airline's system safety and stability;
Selected examples of risk indicators in SEAT:
1. Consider the impact of changes in required management personnel
(airline management, turnover in personnel, reduction in workforce,
layoffs, buyout, etc.); 2. Consider the complaints that affect
surveillance planning and how that airline responded to them; 3.
Consider the age of the airline's fleet, its process to survey and
inspect aging aircraft, and the effectiveness of its aging aircraft
program.
System and purpose: Other: records the presence and implementation of
airline developed security and substance abuse programs;
Selected examples of risk indicators in SEAT:
1. Does the airline understand FAA's authority to conduct drug tests
and demonstrations?
2. Does the airline have a security (drug and alcohol) program and is
the airline in compliance with the program?
System and purpose: Certificate management risks: reflects the impact
of FAA's resources on providing oversight of an airline;
Selected examples of risk indicators in SEAT:
1. Consider if there is an adequate number and type of inspectors
assigned to the certificate; 2. Consider if there is an adequate amount
of geographic surveillance; 3. Consider the resources (travel budget,
country clearance, etc.) of the certificate holding district office.
Source: FAA.
[End of table]
To complete SEAT, the principal inspectors rely on their knowledge of
the airline and on the data available through FAA's Safety Performance
Analysis System database, which contains the Program Tracking and
Reporting Subsystem and 12 additional databases with safety and
performance information. SEAT includes a set of risk indicators for
each of the elements (often in the form of a question) to be discussed
by the team to indicate concerns about any real or potential problem
that could contribute to the failure of one of the airline's elements,
subsystems, or systems. Inspectors rate each risk they identify in
terms of severity (negligible to catastrophic) and likelihood of
occurrence (improbable to frequent), and SEAT calculates an overall
risk rating. Risks rated high have high levels of likelihood and
severity.
Using this information, the team identifies a set of required
inspections that must be completed during that fiscal year and planned
inspections. The required inspections are added to the NPG-identified
inspections, and the combined list becomes the annual surveillance plan
for the airline. During the SEAT process, principal inspectors are
allowed to substitute NPG-required inspections with SEP-identified
inspections targeting higher-risk areas.
SEP Incorporates Some Elements of ATOS, While NPG Relies on a Set
Number of Inspections:
Table 9 describes FAA's three inspection processes for overseeing
legacy and non-legacy airlines: ATOS, NPG, and SEP. Many of the
elements of ATOS, such as the use of data to identify risks and the
development of surveillance plans by inspectors, are incorporated in
the SEP process. The NPG process, in contrast, is not focused on the
use of data and relies on an established set of inspections that are
not risk based.
Table 9: Various Elements of ATOS, NPG, and SEP:
Description of program;
ATOS:
* Focuses on safety vulnerabilities rather than regulatory compliance;
* Analysts and inspectors review airline data to identify areas of
safety risk;
* Inspectors develop surveillance plans for each airline, based on data
analysis and assessment of risks, and adjust the plans periodically
based on inspection results;
NPG:
* Focuses on inspectors completing a prescribed number of inspection
activities;
* Primarily based on checking airline compliance with regulations;
* Relies on inspectors' expertise to identify trends and risks;
SEP:
* Focuses on inspectors conducting a risk assessment of various areas;
* Inspectors review data to identify areas of safety risk and use
Flight Standards Safety Analysis Information Center and the Safety
Performance Analysis System as analytical tools;
* Inspectors develop surveillance plans for each airline, based on data
analysis and assessment of risks, and adjust plans periodically based
on inspection results;
* Inspectors can also verify that planned NPG activities meet the
surveillance needs for a particular year.
Type of commercial passenger airline inspected;
ATOS: Legacy commercial airlines;
NPG: Non-legacy commercial airlines;
SEP: Non-legacy commercial airlines.
Frequency of inspections;
ATOS: Continuous safety oversight;
NPG: Periodic; regular inspections are established annually by an FAA
headquarters committee;
SEP: Periodic; inspections are established during meetings held at
least twice a year using risk-based criteria.
Approximate number of aviation safety inspectors conducting
inspections[A];
ATOS: 585;
NPG: 1,100[B];
SEP: 1,100[B].
Number of commercial passenger airlines under the program[A];
ATOS: 14[C];
NPG: 99;
SEP: 99.
Sources: GAO and FAA.
[A] As of July 2005.
[B] There are a total of about 1,100 inspectors for both NPG and SEP
inspections.
[C] FedEx and United Parcel Service, two cargo air carriers, are also
in the ATOS program.
[End of table]
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Gerald L. Dillingham, (202) 512-2834:
Staff Acknowledgments:
In addition to the above, Teresa Spisak, Assistant Director; Carl
Barden; Bari Bendell; Curtis Groves; Bert Japikse; Catherine Kim;
Minette Richardson; Phillis Riley; Larry Thomas; Alwynne Wilbur; and
Matt Zisman made key contributions to this report.
(542039):
FOOTNOTES
[1] The legacy airlines are Alaska, American, Continental, Delta,
Northwest, United, and US Airways. As of July 2005, the following non-
legacy passenger airlines and cargo airlines were also part of ATOS:
America West, American Eagle, Champion, ExpressJet, SkyWest, Southwest,
Trans States, FedEx, and United Parcel Service. In this report, we
refer to all passenger airlines that are not in the ATOS program as non-
legacy airlines.
[2] The remaining approximately 1,500 inspectors oversee general
aviation.
[3] For this report, we use "field office" to refer to FAA field,
regional, and certificate management offices, unless otherwise noted.
[4] Four of the 25 non-legacy airlines in our review are now in the
ATOS program---American Eagle, ExpressJet, SkyWest, and Trans States.
[5] PTRS also includes information on "planned" inspection activities
and other surveillance activities, which are not inspections, such as
conducting telephone conversations with airline officials and reading
documents related to an airline. However, the database does not
distinguish between planned inspections and other activities. As a
result, we excluded these activities from our analyses.
[6] Geographic inspectors, who are based around the country and not
assigned to any particular airline, do not normally receive training in
the SEP process.
[7] Before commencing operations, an airline must obtain an operating
certificate from FAA. FAA issues the certificate after determining that
an airline's manuals, aircraft, facilities, and personnel meet federal
safety standards. FAA subsequently monitors the airline's operations,
primarily through safety inspections to ensure that it continues to
meet the terms of its certificate.
[8] Avionics focuses on the electronic components of an aircraft.
[9] Our review of ATOS's early implementation found problems, which FAA
addressed by improving guidance to inspectors and increasing data
usefulness. Since then, DOT's Inspector General has found additional
problems with FAA's implementation of ATOS. GAO, Aviation Safety: FAA's
New Inspection System Offers Promise, but Problems Need to be
Addressed, GAO/RCED-99-183 (Washington, D.C.: June 28, 1999); and DOT
Office of Inspector General, Safety Oversight of an Air Carrier
Industry in Transition, AV-2005-062 (Washington, D.C.: June 3, 2005).
[10] System safety is a process by which inspectors perform continual
evaluations of an airline's operations for the purpose of identifying
safety hazards and assessing the severity, frequency, and probability
of the hazards and monitoring their resolution.
[11] SEP was also developed to increase surveillance of new entrant
airlines that are less than 5 years old.
[12] The checklists are required under ATOS and are suggested but not
mandatory for use under SEP.
[13] Required NPG inspections consist of activities to help validate
that critical air carrier subsystems do not have potential latent
failures. According to FAA, periodic validation of these critical
subsystems is important due to the inherent risks associated with their
failures.
[14] Both NPG and SEP consist of required and planned activities.
According to FAA, the required NPG activities account for 12 percent of
the total inspections performed in a given year and the planned NPG
activities account for about 45 percent. The remaining 43 percent of
NPG activities are created by unplanned requirements, according to the
agency. SEP inspections are also designated as required or planned. Any
planned SEP inspections that are not completed by the end of the fiscal
year become required inspections for the following year.
[15] GAO, Aviation Safety: Better Management Controls Are Needed to
Improve FAA's Safety Enforcement and Compliance Efforts, GAO-04-646
(Washington, D.C.: July 6, 2004).
[16] GAO, Aviation Safety: FAA Management Practices for Technical
Training Generally Effective; Further Actions Could Enhance Results,
GAO-05-728 (Washington, D.C.: Sept. 7, 2005).
[17] GAO/RCED-98-6; GAO/T-RCED-92-25.
[18] GAO/RCED-99-183 and GAO, Aviation Safety: FAA Oversight of Repair
Stations Needs Improvement, GAO/RCED-98-21 (Washington, D.C.: Oct. 24,
1997).
[19] GAO/RCED-92-14.
[20] In addition to required NPG and SEP inspections, FAA inspectors
conduct planned inspections and unplanned activities, such as
enforcement investigations. While we recognize that unplanned
activities account for a portion of an inspector's workload, we did not
include them in our analysis because they are generated on an "as
discovered" basis and, therefore, lack comparability with other
inspections that are generated through the NPG or SEP processes. For
our analysis, we looked only at required inspections because they have
higher priority than the planned inspections and because PTRS does not
distinguish between planned inspections and other planned activities.
According to FAA, the required NPG inspections account for about 12
percent of the total inspections performed in a given year, planned NPG
activities account for about 45 percent, and unplanned activities
account for the remaining 43 percent of NPG activities. According to
data provided by FAA, for the planned inspections in fiscal years 2002-
2004, 6 percent were SEP-initiated and 94 percent were NPG-initiated,
indicating that an even larger percentage of these inspections compared
with required inspections are not prioritized based on a structured
risk identification process.
[21] FAA has no similar requirement for planned NPG activities, the
completion of which depends upon available resources.
[22] Inspectors can substitute or retarget required NPG-initiated
activities with activities identified during the periodic SEP meetings
that they deem constitute a greater safety risk.
[23] The Office of Aviation Safety was expected to absorb about a $17.4
million reduction in its fiscal year 2005 budget. The reductions were
required by Congress and FAA.
[24] In September 2005, FAA headquarters officials told us they
expected Congress to appropriate an additional $8 million in fiscal
year 2006, which would allow them to hire more inspectors than the
previously planned 80.
[25] Designees are private persons and organizations that handle the
vast majority of FAA's safety certification activities, such as testing
pilots and mechanics for FAA-issued certificates.
[26] We have reported that FAA has followed or is taking many effective
management practices in planning, developing, and delivering technical
training, yet inspectors express widespread dissatisfaction with this
training. See GAO-05-728.
[27] We did not verify these reports that inspectors made in the
internal risk database. However, FAA procedures call for all risks to
be reviewed by managers, who can also enter comments indicating
concurrence or disagreement with the risk, among other things. The
examples we cite in this report do not contain information indicating
that management questioned the accuracy of the inspectors' statements.
[28] GAO-05-728.
[29] The safeguards include (1) executing an agreement with the
aviation industry training provider outlining the conditions under
which the training will be accepted, (2) conducting a legal review of
the agreement to ensure that there are no conflict of interest issues,
and (3) obtaining the approval of the FAA Administrator for the
acceptance of the training.
[30] We were not provided sufficient information to verify these
statements.
[31] GAO-05-728.
[32] We defined "technical training" as training that develops
knowledge of the production, maintenance and operation of aircraft
(including currency training), aircraft parts, and systems. For the
purposes of our survey, technical training did not include automation
training. Our definition differs somewhat from FAA's use of the term.
FAA defines "technical training" to include aviation technologies as
well as topics such as inspector job skills, risk analysis, data
analysis, and training in software packages, such as spreadsheets.
[33] The actual estimate is 77 percent. All percentage estimates from
the survey have a margin of error of plus or minus 7.1 percentage
points or less. Survey estimates presented are statistically
significant when the 95 percent confidence intervals do not overlap.
[34] FAA officials told us that training occurs over the span of a
career and cautioned that asking inspectors' views about 2 years'
experience would present a distorted view. We recognize FAA's point and
the fact that FAA requires candidates for safety inspector positions to
have extensive technical qualifications and experience. However, it is
not reasonable to expect inspectors' to recall their views on training
received over a large time span, as doing so could lead to unreliable
results.
[35] FAA does not have information to separate out the number of ATOS
inspectors and inspectors of non-legacy airlines prior to fiscal year
2005.
[36] The actual estimate is 33 percent.
[37] GAO/RCED-98-6.
[38] GAO, Managing for Results: Strengthening Regulatory Agencies'
Performance Management Practices, GAO/GGD-00-10 (Washington, D.C.: Oct.
28, 1999); and Executive Guide: Effectively Implementing the Government
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June
1996).
[39] In this report, we refer to all passenger airlines overseen by
FAA's Air Transportation Oversight System (ATOS) as legacy airlines and
those that are not as non-legacy airlines. As of July 2005, the
following passenger airlines and cargo airlines were part of ATOS:
Alaska, America West, American, American Eagle, Champion, Continental,
Delta, ExpressJet, FedEx, Northwest, SkyWest, Southwest, Trans States,
United, United Parcel Service, and US Airways.
[40] Four of the 25 airlines--American Eagle, ExpressJet, SkyWest, and
Trans States--were transitioned into the ATOS program from 2003 through
2005. Prior to then, they were covered under SEP.
[41] In this report, we refer to FAA staff who perform safety audits,
inspections, and surveillance as inspectors.
[42] Management controls are the continuous processes that federal
agencies are required by law--i.e., the Federal Managers' Financial
Integrity Act of 1982, the Government Performance and Results Act of
1993, the Chief Financial Officers Act of 1990, and the Federal
Financial Management Improvement Act of 1996--and by the Office of
Management and Budget to use to provide reasonable assurance that their
goals, objectives, and missions are being met. We identified, with the
assistance of GAO specialists in that area, those control standards
established by the Office of Management and Budget and us that are
appropriate to FAA's inspection process. See GAO, Standards for
Internal Control in the Federal Government, GAO/AIMD-00-21.3.1
(Washington, D.C.: November 1999); and Internal Control Management and
Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 2001).
[43] Our population included only those inspectors that actively
participate in inspection activities as part of their regular job
duties. It did not include managers, supervisors, or inspectors
detailed to headquarters or regional offices.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
