Aviation Safety
FAA's Safety Oversight System Is Effective but Could Benefit from Better Evaluation of Its Programs' Performance Gao ID: GAO-06-266T November 17, 2005The U.S. commercial aviation industry has an extraordinary safety record. However, when passenger airlines have accidents or serious incidents, regardless of their rarity, the consequences can be tragic. The Federal Aviation Administration (FAA) works to maintain a high level of safety through an effective safety oversight system. Keys to this system are to: (1) establish programs that focus resources on areas of highest safety risk and on mitigating risks; (2) provide training and communication to ensure that inspectors can consistently carry out the agency's oversight programs; and (3) have processes and data to continuously monitor, evaluate, and improve the numerous oversight programs that make up the safety oversight system. This statement focuses on these three key areas and is based on recent GAO reports on FAA's inspection oversight programs, industry partnership programs, enforcement program, and training program.
FAA's safety oversight system includes programs that focus on risk identification and mitigation through a risk-based system safety approach, leveraging of resources through designee and partnership relationships, and enforcement of safety regulations, but the benefits of these programs are not being fully realized. For example, FAA's system safety approach includes the addition of a program that emphasizes risk identification to its traditional inspection program for overseeing some airlines, which is not based on risk. However, it is likely that the benefits of this approach could be enhanced if the inspection workload was not as heavily oriented to the traditional inspection program's non-risk based activities. FAA leverages its resources through its designee programs, in which designated individuals and organizations perform about 90 percent of certification-related activities, and through its industry partnership programs, which are designed to assist the agency in receiving safety information. An outgrowth of FAA's inspection process is its enforcement program, which is intended to ensure industry compliance with safety regulations. However, GAO has expressed concerns that this program may not be as effective as it could be in deterring violations. FAA has made training an integral part of its safety oversight system, but several actions could improve the results of its training efforts, including ensuring that inspectors are well-trained in FAA's system safety approach and have sufficient knowledge of increasingly complex aircraft and systems to effectively identify safety risks. FAA has established mandatory training requirements for its workforce and designees. We have reported that FAA has generally followed effective management practices for planning, developing, delivering, and assessing the impact of its technical training for safety inspectors. GAO has found inadequate evaluative processes and limitations with data for FAA's inspection programs, designee programs, industry partnership programs, and enforcement program. For example, FAA lacked requirements or criteria for evaluating its designee programs. In another example, FAA's nationwide enforcement database is not as useful as it could be because of missing or incomplete historical information about enforcement cases.
GAO-06-266T, Aviation Safety: FAA's Safety Oversight System Is Effective but Could Benefit from Better Evaluation of Its Programs' Performance
This is the accessible text file for GAO report number GAO-06-266T
entitled 'Aviation Safety: FAA's Safety Oversight System Is Effective
but Could Benefit from Better Evaluation of Its Programs' Performance'
which was released on November 17, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Aviation, Committee on Commerce, Science,
and Transportation, U.S. Senate:
United States Government Accountability Office:
GAO:
Not to Be Released Before10:00 a.m. EST:
Thursday, November 17, 2005:
Aviation Safety:
FAA's Safety Oversight System Is Effective but Could Benefit from
Better Evaluation of Its Programs' Performance:
Statement for the Record by Gerald L. Dillingham, Ph.D., Director:
Physical Infrastructure Issues:
GAO-06-266T:
GAO Highlights:
Highlights of GAO-06-266T, a testimony to the Subcommittee on Aviation,
Committee on Commerce, Science, and Transportation, U.S. Senate:
Why GAO Did This Study:
The U.S. commercial aviation industry has an extraordinary safety
record. However, when passenger airlines have accidents or serious
incidents, regardless of their rarity, the consequences can be tragic.
The Federal Aviation Administration (FAA) works to maintain a high
level of safety through an effective safety oversight system. Keys to
this system are to: (1) establish programs that focus resources on
areas of highest safety risk and on mitigating risks; (2) provide
training and communication to ensure that inspectors can consistently
carry out the agency‘s oversight programs; and (3) have processes and
data to continuously monitor, evaluate, and improve the numerous
oversight programs that make up the safety oversight system. This
statement focuses on these three key areas and is based on recent GAO
reports on FAA‘s inspection oversight programs, industry partnership
programs, enforcement program, and training program.
What GAO Found:
FAA‘s safety oversight system includes programs that focus on risk
identification and mitigation through a risk-based system safety
approach, leveraging of resources through designee and partnership
relationships, and enforcement of safety regulations, but the benefits
of these programs are not being fully realized. For example, FAA‘s
system safety approach includes the addition of a program that
emphasizes risk identification to its traditional inspection program
for overseeing some airlines, which is not based on risk. However, it
is likely that the benefits of this approach could be enhanced if the
inspection workload was not as heavily oriented to the traditional
inspection program‘s non-risk based activities. FAA leverages its
resources through its designee programs, in which designated
individuals and organizations perform about 90 percent of certification-
related activities, and through its industry partnership programs,
which are designed to assist the agency in receiving safety
information. An outgrowth of FAA‘s inspection process is its
enforcement program, which is intended to ensure industry compliance
with safety regulations. However, GAO has expressed concerns that this
program may not be as effective as it could be in deterring violations.
FAA has made training an integral part of its safety oversight system,
but several actions could improve the results of its training efforts,
including ensuring that inspectors are well-trained in FAA‘s system
safety approach and have sufficient knowledge of increasingly complex
aircraft and systems to effectively identify safety risks. FAA has
established mandatory training requirements for its workforce and
designees. We have reported that FAA has generally followed effective
management practices for planning, developing, delivering, and
assessing the impact of its technical training for safety inspectors.
GAO has found inadequate evaluative processes and limitations with data
for FAA‘s inspection programs, designee programs, industry partnership
programs, and enforcement program. For example, FAA lacked requirements
or criteria for evaluating its designee programs. In another example,
FAA‘s nationwide enforcement database is not as useful as it could be
because of missing or incomplete historical information about
enforcement cases.
FAA‘s Safety Oversight System Includes Inspections, Training, and Other
Activities:
[See PDF for image]
[End of figure]
What GAO Recommends:
In order to help FAA fully realize the benefits of its safety oversight
system, GAO has made a number of recommendations to address weaknesses
identified in previous reviews, such as improving program evaluation
and data. These recommendations have not been fully implemented,
although in some cases FAA has taken steps towards addressing them.
www.gao.gov/cgi-bin/getrpt?GAO-06-266T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Dr. Gerald Dillingham at
(202) 512-2834 or dillinghamg@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
The U.S. commercial aviation industry, with less than one fatal
accident per 5 million flights from 2002 through 2005, has an
extraordinary safety record. This record is a result of the efforts of
the Federal Aviation Administration (FAA), airlines, manufacturers, and
others in the aviation industry to maintain one of the safest aviation
systems in the world. However, when passenger airlines have accidents
or serious incidents, regardless of their rarity, the consequences can
be tragic, as a single accident can result in hundreds of deaths. In
order to maintain a high level of safety, it is critical to have a
safety oversight system that is comprehensive, efficient, and effective
and can provide an early warning of hazards that can lead to accidents.
It is equally important to have a skilled, well-trained workforce to
implement and monitor this safety oversight system. FAA's workforce of
about 3,200 inspectors stationed at more than 100 field offices
throughout the world is responsible for carrying out the agency's
processes to certify, inspect, and enforce safety regulations for all
aspects of the aviation industry, including the aircraft and its
component parts, over 100 commercial airlines, about 5,000 aircraft
repair stations, and hundreds of thousands of pilots and mechanics. FAA
augments its inspector workforce with nearly 13,600 designated
organizations and individuals (designees) that conduct the more routine
aspects of industry oversight, such as administering tests to pilots
and mechanics as part of their certification requirements, and augments
the safety information that it obtains from inspections with industry
partnership programs. Keys to making this safety oversight system work
are to: (1) establish programs that focus resources on areas of highest
safety risk and on mitigating risks; (2) provide training and other
communication to ensure that inspectors maintain the skills and
knowledge to consistently carry out the agency's oversight programs;
and (3) have processes and data to continuously monitor, evaluate, and
improve the numerous oversight programs that make up the safety
oversight system. This statement focuses on these three key areas of
FAA's "early warning system" and is based on our recent reports on
FAA's inspection oversight programs, industry partnership programs,
enforcement program, and training program. We will also discuss our
recommendations that FAA has not fully addressed in these areas.
In summary:
* FAA's aviation safety oversight system includes programs that focus
on risk identification and mitigation through a system safety approach,
the leveraging of resources, and enforcement of safety regulations, but
the benefits of these programs are not being fully realized. FAA's
system safety approach has many strengths, including the addition of a
program that emphasizes risk identification to its traditional
inspection program for overseeing non-legacy airlines,[Footnote 1]
which is not based on risk. However, it is likely that the benefits of
this approach could be enhanced if the inspection workload for non-
legacy airlines was not still heavily oriented to the traditional
inspection program's non-risk based activities. FAA leverages resources
through its designee programs, in which designees perform about 90
percent of certification-related activities, thus allowing FAA to
better concentrate its limited staff resources on the most safety-
critical functions. However, concerns about the consistency and
adequacy of designee oversight that FAA field offices provide have been
raised by experts and other individuals we interviewed. FAA also
leverages its resources through industry partnership programs, which
are designed to assist the agency in receiving safety information. For
example, FAA encourages voluntary reports of safety violations by
responding to them by issuing a warning letter rather than a fine or
other legal sanction. FAA's enforcement program, which is an outgrowth
of its inspection process, is intended to ensure industry compliance
with safety regulations and is another important element of its safety
oversight system. FAA's policy for assessing legal sanctions against
entities or individuals that do not comply with aviation safety
regulations is intended to deter future violations. However, we found
that recommendations for sanctions are sometimes reduced on the basis
of factors that are not associated with the merits of the case, and the
economic literature on deterrence suggests that the goal of preventing
future violations is weakened when the penalties for violations are
lowered for reasons not related to the merits of the case.
* FAA has made training an integral part of its safety oversight
system, but several actions could improve the results of its training
efforts. FAA's use of a risk-based system safety approach to
inspections requires inspectors to apply data analysis and auditing
skills to identify and control potential risks. Therefore, it is
important that inspectors are well-trained in this approach and have
sufficient knowledge of increasingly complex aircraft and systems to
effectively identify safety risks. FAA has established mandatory
training requirements for its workforce as well as designees. We have
reported that FAA has generally followed effective management practices
for planning, developing, delivering, and assessing the impact of its
technical training for safety inspectors, although some practices have
yet to be fully implemented. For example, in developing its training
curriculum for inspectors, FAA followed effective management practices,
such as developing courses that support changes in inspection
procedures resulting from regulatory changes or agency initiatives. On
the other hand, FAA develops technical courses on an ad hoc basis
rather than as part of an overall curriculum for each type of
inspector, such as inspectors of operations or cabin safety, because
the agency has not systematically identified the technical skills and
competencies each type of inspector needs to effectively perform
inspections. FAA has recognized the need for improvements to its
training program in this and other areas.
* It is important for FAA to have effective evaluative processes and
accurate nationwide data for its numerous safety oversight programs so
that program managers and other officials have assurance that the
safety programs are having their intended effect. Such processes and
data are especially important because FAA's workforce is so dispersed
throughout the world--with thousands of staff working out of more than
100 offices worldwide--and because FAA's use of a system safety
approach represents a cultural shift from its traditional inspection
program. Evaluation is important for understanding if the cultural
shift has effectively occurred. Our most recent work has shown the lack
of evaluative processes and limitations with data for FAA's inspection
program for non-legacy airlines, designee programs, industry
partnership programs, and enforcement program. For example, we found
that FAA lacked requirements or criteria for evaluating its designee
programs. In another example, FAA's enforcement policy calls for the
assessment of sanctions that would potentially deter future violations.
However, FAA lacks an evaluative process, so it is not known whether
the agency's enforcement practices, such as at times reducing
sanctions, may weaken any deterrent effect that would be expected from
such sanctions. Furthermore, FAA's ability to evaluate its programs is
hindered by the lack of useful nationwide data. For example, FAA's
nationwide enforcement database is not as useful as it could be because
of missing or incomplete historical information about enforcement
cases.
* In order to help FAA fully realize the benefits of its safety
oversight system, we have made a number of recommendations to address
the weaknesses that we identified in our reviews. These recommendations
have not been fully implemented, although in some cases FAA has taken
steps towards addressing them. Evaluative processes and relevant data
are particularly important as FAA works to change its culture by
incorporating a system safety approach into its oversight, and we have
recommended that FAA develop continuous evaluative processes for its
oversight of non-legacy airlines, its designee programs, and its
enforcement program, and systematically assess inspectors' technical
training needs. In addition, FAA's nationwide databases are in need of
improvements in their comprehensiveness and ease of use. We have
recommended that FAA improve the consistency and completeness of its
designee and enforcement databases. Continuous improvements in these
areas are critical to FAA's ability to have a robust "early warning
system" in order to maintain one of the safest aviation systems in the
world.
Background:
FAA's safety oversight system is made up of a number of programs for
airlines and other entities. Safety oversight programs for airlines
provide for their initial certification, periodic surveillance, and
inspection. Since 1985, FAA has used National Work Program Guidelines
(NPG), its traditional inspection program for airlines, as a primary
means of ensuring that airlines comply with safety regulations. In NPG,
an FAA committee of program managers identifies an annual minimum set
of required inspections that are to be undertaken to ensure that
airlines are in compliance with their operating certificates. In 1998,
the agency implemented the Air Transportation Oversight System (ATOS),
which currently oversees the nation's largest 15 commercial airlines
and cargo carriers, with the goal of eventually including all
commercial passenger and cargo airlines in it. ATOS emphasizes a system
safety approach that extends beyond periodically checking airlines for
compliance with regulations to the use of technical and managerial
skills to identify, analyze, and control hazards and risks. For
example, under ATOS, inspectors develop surveillance plans for each
airline, based on data analysis and assessment of risks, and adjust the
plans periodically based on inspection results. However, the agency has
been delayed in placing a significant number of other passenger
airlines in ATOS, resulting in 99 passenger airlines, which we refer to
as non-legacy airlines, continuing to be overseen through NPG, a
process that is not risk-based or system safety oriented. In 2002, FAA
added the Surveillance and Evaluation Program (SEP) to the NPG
inspection program to incorporate principles of ATOS into its oversight
of non-legacy passenger airlines. The two programs are used together to
establish the number of annual inspections for non-legacy airlines.
(Appendix 1 describes each inspection program.) Figure 1 illustrates
some typical activities covered during inspections.
Figure 1: FAA's Safety Inspections Cover a Wide Range of Activities:
[See PDF for image]
Note: As a workforce, FAA inspectors conduct a wide variety of
inspections, including ensuring that pilots are qualified to operate
aircraft, inspecting aircraft for safety, and overseeing FAA-certified
mechanics.
[End of figure]
FAA's safety oversight programs for other aspects of the aviation
industry--including manufacturers of aircraft and aircraft parts,
repair stations, flight schools, aviation maintenance technician
schools, pilots, and mechanics--involve certification, surveillance,
and inspection by FAA's safety inspectors, engineers, flight surgeons,
and designated representatives. FAA authorizes about 13,400 private
individuals and about 180 organizations (called "designees") to act as
its representatives to conduct many safety certification activities,
such as administering flight tests to pilots, inspecting repair work by
maintenance facilities, conducting medical examinations of pilots, and
approving designs for aircraft parts. These designees are grouped into
18 different programs and are overseen by three FAA offices--Flight
Standards Service, Aerospace Medicine, and Aircraft Certification
Service--all of which are under the Office of Aviation Safety (see fig.
2).
Figure 2: FAA Offices That Manage the Different Designee Programs and
Numbers of Designees (as of May 2004):
[See PDF for image]
[End of figure]
Since 1990, FAA has emphasized gaining compliance from the aviation
industry through cooperative means by establishing industry partnership
programs with the aviation community that allow participants, such as
airlines and pilots, to self-report violations of safety regulations
and help identify safety deficiencies, and potentially mitigate or
avoid fines or other legal action. For example, the Voluntary
Disclosure Program encourages the self-reporting of manufacturing
problems and safety incidents by participants that can include air
carriers and repair stations. Appendix II describes the industry
partnership programs.
When violations of statutory and regulatory requirements are identified
through inspections, through the partnership programs in certain cases,
or through other methods, FAA has a variety of enforcement tools that
it may use to respond to them, including administrative actions (such
as issuing a warning notice or a letter of correction that includes the
corrective actions the violator will take) and legal sanctions (such as
levying a fine or suspending or revoking a pilot or other FAA-issued
certificate).
FAA's Safety Oversight System Focuses on Risk Identification and
Mitigation Through System Safety, Leveraging of Resources, and
Enforcement of Safety Regulations, but Benefits Are Not Being Fully
Realized:
In recent reports, we found that FAA's safety oversight system has
programs that focus on risk identification and mitigation through a
system safety approach, the leveraging of resources, and enforcement of
safety regulations, but that the benefits of these programs are not
being fully realized. In our recent report on FAA's oversight of non-
legacy airlines, we found that the focus on risk identification through
the addition of SEP has many strengths and allows for enhancing the
efficiency of FAA's oversight activities.[Footnote 2] Rather than
relying on NPG's customary method of conducting a set number of
inspections of an airline's operations, SEP emphasizes a system safety
approach of using risk analysis techniques. SEP allows for the
efficient use of inspection staff and resources by prioritizing
workload based on areas of highest risk, and it includes a requirement
that inspectors verify that corrective actions have occurred. For
example, FAA has developed risk assessment worksheets for SEP that are
aligned with key airline systems that guide inspectors through
identifying and prioritizing risks. The worksheets guide inspectors to
organize the results of their previous inspections and surveillance
into a number of areas such as flight operations and personnel training
in order to identify specific risks in each area and target the
office's resources to mitigating those risks. The development of a
system safety approach addresses a long-standing concern by us that FAA
did not have a methodology for assessing airline safety risks so that
it could target limited inspection resources to high-risk
conditions.[Footnote 3] Another strength of SEP, consistent with
findings in our past reports, is that SEP relies on teams of
inspectors, which are generally more effective than individual
inspectors in their ability to collectively identify concerns.[Footnote
4]
However, the benefits of FAA's system safety approach for the
inspection of non-legacy airlines could be enhanced by a more complete
implementation of SEP and addressing other challenges. The inspection
workload for non-legacy airlines is still heavily oriented to the NPG's
non-risk based activities. For example, as shown in table 1, from
fiscal years 2002 through 2004, 77 percent of inspection activities
required for the top 25 non-legacy airlines in terms of the number of
enplanements were identified through NPG, and the remaining percentage
of inspection activities were identified based on risk through SEP.
Although inspectors can replace NPG-identified activities with SEP-
identified activities that they deem constitute a greater safety risk,
we found that FAA inspectors interpret agency emphasis on NPG as
discouraging this practice. In order to ensure that all inspectors who
oversee non-legacy airlines have a complete and timely understanding of
the agency's policies relating to the inspection process, we
recommended in September 2005 that FAA improve communication with and
training of inspectors in this area.
Table 1: SEP-and NPG-Initiated Required Inspections for the Top 25 Non-
legacy Airlines, Fiscal Years 2002-2004:
Type of inspection: SEP-initiated;
2002: 1,261;
2003: 1,567;
2004: 927;
Total: 3,755 (23%).
Type of inspection: NPG-initiated;
2002: 5,470;
2003: 3,623;
2004: 3,338;
Total: 12,431 (77%).
Type of inspection: Total;
2002: 6,731;
2003: 5,190;
2004: 4,265;
Total: 16,186 (100%).
Source: GAO analysis of FAA information.
[End of table]
Another way that FAA attempts to enhance the efficiency of its
oversight activities is through its designee programs. We reported that
FAA maximizes its resources by allowing designees to perform about 90
percent of certification-related activities, thus allowing FAA to
better concentrate its limited staff resources on the most safety-
critical functions.[Footnote 5] For example, while designees conduct
routine certification functions, such as approvals of aircraft
technologies that the agency and designees have had previous experience
with, FAA focuses on new and complex aircraft designs or design
changes. In addition, the use of designees expands FAA's access to
technical expertise within the aviation community. For the aviation
industry, the designee programs enable individuals and organizations to
obtain required FAA certifications--such as approvals of the design,
production, and airworthiness of aircraft--in a timely manner, thus
reducing delays and costs to the industry that might result from
scheduling direct reviews by FAA staff. For example, officials from an
aircraft manufacturer told us that the use of designees has added
significantly to the company's ability to enhance and improve daily
operations by decreasing certification delivery time and increasing the
flexibility and utilization of company resources. In addition,
designees are convenient to the aviation industry due to their wide
dispersal throughout the United States.
However, concerns about the consistency and adequacy of designee
oversight that FAA field offices provide have been raised by experts
and other individuals we interviewed. For example, designees and
industry officials that we spoke with indicated that FAA's level of
oversight and interpretation of rules differ among regions and among
offices within a region, which limits FAA's assurance that designees'
work is performed uniformly in accordance with FAA's standards and
policy. Experts also ranked this issue as a top weakness.[Footnote 6]
Table 2 shows the top five weaknesses identified by our experts.
Experts also made a number of suggestions to strengthen the designee
program, including clearly defining and following agency criteria for
selecting designees and increasing penalties for designees found to
violate standards or who do not exercise proper judgment. To improve
management control of the designee programs, and thus increase
assurance that designees meet FAA's performance standards, we
recommended that FAA develop mechanisms to improve the compliance of
FAA program and field offices with existing policies and incorporate,
as appropriate, suggestions from our expert panel. In response to our
recommendations, FAA is planning, among other things, to form a team to
identify and share best practices for overseeing designee programs.
Table 2: Experts' Ranking of Top 5 Oversight Weaknesses:
Ranking: 1;
Weakness: FAA offices level of oversight and interpretation of rules
are inconsistent.
Ranking: 2;
Weakness: Inactive, unqualified, or poor performing designees are not
identified and removed expeditiously.
Ranking: 3;
Weakness: It is difficult to terminate poor performing designees.
Ranking: 4;
Weakness: Inadequate surveillance and oversight of designees.
Ranking: 5;
Weakness: FAA has not made oversight of designees a high enough
priority.
Source: GAO analysis of expert panel information.
Note: Rankings based on responses from 62 experts and the frequency of
responses indicating a "great" or "very great" weakness.
[End of table]
FAA also leverages its resources through its industry partnership
programs. These partnership programs are designed to assist the agency
in receiving safety information, including reports of safety
violations. According to FAA officials, the Aviation Safety Action
Program, Aviation Safety Reporting Program, and Voluntary Disclosure
Reporting Program augment FAA's enforcement activities and allow FAA to
be aware of many more safety incidents than are discovered during
inspections and surveillance. In addition, the Flight Operational
Quality Assurance Program provides safety information in the form of
recorded flight data from participating airlines. FAA has established
some management controls over its partnership programs, such as
procedures to track actions taken to correct safety incidents reported
under the programs, but the agency lacks management controls to measure
and evaluate the performance of these programs, an issue that we will
discuss later in the testimony.
FAA's enforcement process, which is intended to ensure industry
compliance with safety regulations, is another important element of its
safety oversight system. FAA's policy for assessing legal sanctions
against entities or individuals that do not comply with aviation safety
regulations is intended to deter future violations. FAA has established
some management controls over its enforcement efforts, with procedures
that provide guidance on identifying regulated entities and individuals
that are subject to inspections or surveillance actions, determining
workload priorities on the basis of the timing and type of inspection
to be performed, detecting violations of safety regulations, tracking
the actions that are taken by the entities and individuals to correct
the violations and achieve compliance with regulations, and imposing
punitive sanctions or remedial conditions on the violators. These
procedures provide FAA inspectors, managers, and attorneys with a
process to handle violations of safety regulations that are found
during routine inspections.
However, we found that the effect of FAA's legal sanctions on
deterrence is unclear, and that recommendations for sanctions are
sometimes changed on the basis of factors that are not associated with
the merits of the case. We found that from fiscal years 1993 through
2003, attorneys in FAA's Office of the Chief Counsel authorized a 52
percent reduction in the civil monetary penalties assessed from a total
of $334 million to $162 million. FAA officials told us that the agency
sometimes reduces sanctions in order to prioritize attorneys' caseloads
by closing the cases more quickly through negotiating a lower fine.
Economic literature on deterrence suggests that although negative
sanctions (such as fines and certificate suspensions) can deter
violations, if the violator expects sanctions to be reduced, he or she
may have less incentive to comply with regulations. In effect, the goal
of preventing future violations is weakened when the penalties for
present violations are lowered for reasons not related to the merits of
the case. In addition, FAA lacks management controls to measure and
evaluate its enforcement process, which we discuss later in this
testimony.
FAA Has Made Training an Integral Part of Its Safety Oversight System
but Several Actions Could Improve Results:
FAA's use of a risk-based system safety approach to inspections
requires inspectors to apply data analysis and auditing skills to
identify, analyze, assess, and control potential hazards and risks.
Therefore, it is important that inspectors are well-trained in this
approach and have sufficient knowledge of increasingly complex
aircraft, aircraft parts, and systems to effectively identify safety
risks. It is also important that FAA's large cadre of designees is well-
trained in federal aviation regulations and FAA policies. FAA has made
training an integral part of its safety inspection system and has
established mandatory training requirements for its workforce as well
as designees. FAA provides inspectors with extensive training in
federal aviation regulations; inspection and investigative techniques;
and technical skills, such as flight training for operations
inspectors. The agency provides its designees with an initial
indoctrination that covers federal regulations and agency policies, and
refresher training every 2 to 3 years.
We have reported that FAA has generally followed effective management
practices for planning, developing, delivering, and assessing the
impact of its technical training[Footnote 7] for safety inspectors,
although some practices have yet to be fully implemented.[Footnote 8]
In its planning activities for training, FAA has linked technical
training efforts to its goal of safer air travel and has identified
technical proficiencies needed to improve safety inspectors'
performance in meeting this goal. For example, FAA's Offices of Flight
Standards and Aircraft Certification have identified gaps in several of
the competencies required to conduct system safety inspections,
including risk assessment, data analysis, systems thinking, and
designee oversight. According to FAA, it is working to correct these
gaps. We have also identified gaps in the training provided to
inspectors in the Office of Flight Standards who oversee non-legacy
airlines, and have recommended that FAA improve inspectors' training in
areas such as system safety and risk management to ensure that these
inspectors have a complete and timely understanding of FAA's inspection
policies. We have identified similar competency gaps related to
designee oversight. For example, FAA does not require refresher
training concerning designee oversight, which increases the risk that
staff do not retain the information, skills, and competencies required
to perform their oversight responsibilities. We recommended that FAA
provide additional training for staff who directly oversee
designees.[Footnote 9] We did not identify any specific gaps in the
competencies of designees.[Footnote 10] In prioritizing funding for
course development activities, FAA does not explicitly consider which
projects are most critical. Figure 3 describes the extent to which FAA
follows effective management practices in planning training.
Figure 3: Extent that FAA Follows Effective Management Practices in
Planning Technical Training:
[See PDF for image]
[End of figure]
In developing its training curriculum for inspectors, FAA also for the
most part follows effective management practices, such as developing
courses that support changes in inspection procedures resulting from
regulatory changes or agency initiatives. On the other hand, FAA
develops technical courses on an ad hoc basis rather than as part of an
overall curriculum for each inspector specialty--such as air carrier
operations, maintenance, and cabin safety--because the agency has not
systematically identified the technical skills and competencies each
type of inspector needs to effectively perform inspections. Figure 4
describes the extent to which FAA follows effective management
practices in developing training.
Figure 4: Extent that FAA Follows Effective Management Practices in
Developing Technical Training:
[See PDF for image]
[A] This management practice is not specifically identified in our
assessment guide. However, a management approach that assesses training
needs holistically rather than on a course-by-course basis can provide
for a more systematic assessment of whether and how training will help
meet organizational needs.
[End of figure]
In delivering training, FAA has also generally followed effective
management practices. (See fig. 5.) For example, FAA has established
clear accountability for ensuring that inspectors have access to
technical training, developed a way for inspectors to choose courses
that meet job needs and further professional development, and offers a
wide array of technical and other courses. However, both FAA and its
inspectors recognize the need for more timely selection of inspectors
for technical training. In addition, FAA acknowledges the need to
increase communication between inspectors and management with respect
to the training program, especially to ensure that inspectors have
bought into the system safety approach to inspections.
Figure 5: Extent that FAA Follows Effective Management Practices in
Delivering Technical Training:
[See PDF for image]
[End of figure]
FAA offers numerous technical courses from which inspectors can select
to meet job needs. However, from our survey of FAA's inspectors, we
estimate that only about half think that they have the technical
knowledge needed for their jobs.[Footnote 11] FAA officials told us
that inspectors' negative views stem from their wanting to acquire
proficiencies that are not as crucial in a system safety environment.
We also found a disparity between inspectors and FAA concerning the
receipt of requested training. We estimated that 28 percent of
inspectors believe that they get the technical training that they
request. However, FAA's records show that FAA approves about 90 percent
of these requests, and inspectors are making good progress in receiving
training. Over half of the inspectors have completed at least 75
percent of technical training that FAA considers essential. FAA
officials told us that inspectors' negative views on their technical
knowledge and the training they have received stem from their not
accepting FAA's move to a system safety approach. That is, the
inspectors are concerned about acquiring individual technical
proficiency that is not as crucial in a system safety environment.
Given that it has not completed assessing whether training for each
inspector specialty meets performance requirements, FAA is not in a
position to make definitive conclusions concerning the adequacy of
inspector technical training.
FAA also generally followed effective management practices in
evaluating training. The agency requires that each training course
receive a systematic evaluation every 3 years to determine if the
course is up to date and relevant to inspectors' jobs, although
training officials noted that many courses have yet to undergo such an
evaluation. However, FAA collects limited information on the
effectiveness of training, and its evaluations have not measured the
impact of training on FAA's mission goals, such as reducing accidents.
Training experts acknowledge that isolating performance improvements
resulting from training programs is difficult for any organization.
(See fig. 6.)
Figure 6: Extent that FAA Follows Effective Management Practices in
Evaluating Its Training Program:
[See PDF for image]
[End of figure]
While FAA follows many effective management practices in its training
program, the agency also recognizes the need for improvements,
including (1) systematically assessing inspectors' needs for technical
and other training, (2) better timing of technical training so that
inspectors receive it when it is needed to perform their jobs, and (3)
better linking the training provided to achieving agency goals of
improving aviation safety. FAA has begun to act in these areas, and we
believe that if effectively implemented, the actions should improve the
delivery of training and ultimately improve aviation safety. Therefore,
it is important for FAA to follow through with its efforts. As a
result, we recommended in September 2005, among other things, that in
order to ensure that inspector technical training needs are identified
and met in a timely manner, FAA systematically assess inspectors'
technical training needs, better align the timeliness of training to
when inspectors need the training to do their jobs, and gain
inspectors' acceptance for changes made or planned to their training.
It is important that both FAA's inspection workforce and FAA-certified
aviation mechanics are knowledgeable about increasingly complex
aircraft, aircraft parts, and systems. While we did not attempt to
assess the technical proficiency that FAA's workforce requires and will
require in the near future, FAA officials said that inspectors do not
need a substantial amount of technical training courses because
inspectors are hired with a high degree of technical knowledge of
aircraft and aircraft systems. They further indicated that inspectors
can sufficiently keep abreast of many of the changes in aviation
technology through FAA and industry training courses and on-the-job
training. However, in its certification program for aviation mechanics,
we found that FAA standards for minimum requirements for aviation
courses at FAA-approved aviation maintenance technician schools and its
requirements for FAA-issued mechanics certificates do not keep abreast
with the latest technologies. In 2003, we reported that those standards
had not been updated in more than 50 years.[Footnote 12] We recommended
that FAA review the curriculum and certification requirements and
update both. FAA plans to make changes in the curriculum for FAA
approved aviation maintenance technicians that reflect up-to-date
aviation technologies and finalize and distribute a revised Advisory
Circular in March 2006 that describes the curriculum changes. FAA then
plans to allow the aviation industry time to implement the recommended
curriculum changes before changing the requirements for FAA-issued
mechanics certificates.
FAA Has Evaluated Some Safety Programs, but the Lack of Evaluative
Systems and Nationwide Data Impedes FAA's Ability to Continuously
Monitor Its Safety Programs:
It is important for FAA to have effective evaluative processes and
accurate nationwide data on its numerous safety oversight programs so
that program managers and other officials have assurance that the
safety programs are having their intended effect. Such processes and
data are especially important because FAA's workforce is so dispersed
worldwide--with thousands of staff working out of more than 100 local
offices--and because FAA's use of a risk-based system safety approach
represents a cultural shift from its traditional inspection program.
Evaluation is important to understanding if the cultural shift has
effectively occurred. Our most recent work has shown the lack of such
processes and limitations with data for FAA's inspection programs for
non-legacy airlines, designee programs, industry partnership programs,
and enforcement program. In response to recommendations that we have
made regarding these programs, some improvements are being made. On the
positive side, as we mentioned earlier, our most recent work found that
FAA generally follows effective management practices in evaluating
individual technical training courses.
FAA has not evaluated its inspection oversight programs for non-legacy
airlines--which include SEP and NPG--to determine how the programs
contribute to the agency's mission and overall safety goals, and its
nationwide inspection database lacks important information that could
help it perform such evaluations--such as whether risks identified
through SEP have been mitigated. In addition, the agency does not have
a process to examine the nationwide implications of or trends in the
risks that inspectors have identified through their risk assessments--
information it would need to proactively determine risk trends at the
national level on a continuous basis. FAA's evaluation office instead
conducts analyses of the types of inspections generated under SEP by
airline and FAA region, according to FAA. We recommended that FAA
develop a continuous evaluative process for activities under SEP and
link SEP to the performance-related goals and measures developed by the
agency, track performance toward these goals, and determine appropriate
program changes. FAA is considering our recommendation, but its plan to
place the remaining non-legacy airlines in the ATOS program by the end
of fiscal year 2007 might make this recommendation unnecessary,
according to the agency. Since FAA's past efforts to move airlines to
ATOS have experienced delays, we believe that this recommendation is
still valid.
We also found that FAA lacked requirements or criteria for periodically
evaluating its designee programs. In 2004, we reported that the agency
had evaluated 6 of its 18 designee programs over the previous 7 years
and had plans to evaluate 2 more, although it had no plans to evaluate
the remaining 10 programs because of limited resources.[Footnote 13]
FAA conducted these evaluations on an ad hoc basis usually at the
request of headquarters directors or regional office managers. In
addition, we found that FAA's oversight of designees is hampered, in
part, by the limited information on designees' performance contained in
the various designee databases.[Footnote 14] These databases contain
descriptive information on designees, such as their types of
designations and status (i.e., active or terminated). More complete
information would allow the agency to gain a comprehensive picture of
whether staff are carrying out their responsibilities to oversee
designees. To improve management control of the designee programs, and
thus increase assurance that designees meet the agency's performance
standards, we recommended that FAA establish a process to evaluate all
designee programs and strengthen the effectiveness of its designee
databases by improving the consistency and completeness of information
in them. To address our recommendations, FAA expects to develop a plan
to evaluate all designee programs on a recurring basis and intends to
establish a team that will examine ways to improve automated
information related to designees.
In addition, we found that FAA does not evaluate the effects of its
industry partnership and enforcement programs to determine if stated
program goals, such as deterrence of future violations, are being
achieved. For example, little is known about nationwide trends in the
types of violations reported under the partnership programs or whether
systemic, nationwide causes of those violations are identified and
addressed. Furthermore, FAA's enforcement policy calls for inspectors
and legal counsel staff to recommend or assess enforcement sanctions
that would potentially deter future violations. However, without an
evaluative process, it is not known whether the agency's practice of
generally closing cases with administrative actions rather than legal
sanctions[Footnote 15] and at times reducing the amount of the fines,
as mentioned earlier in this testimony, may weaken any deterrent effect
that would be expected from sanctions.
FAA's ability to evaluate the impact of its enforcement efforts is also
hindered by the lack of useful nationwide data. FAA inspection offices
maintain independent, site-specific databases because they do not find
the nationwide enforcement database--the Enforcement Information System
(EIS)--as useful as it could be because of missing or incomplete
historical information about enforcement cases. As a result of
incomplete data on individual cases, FAA inspectors lack the complete
compliance history of violators when assessing sanctions. We
recommended that FAA develop evaluative processes for its enforcement
activities and partnership programs and use them to create performance
goals, track performance towards those goals, and determine appropriate
program changes. We also recommended that FAA take steps to improve the
usefulness of the EIS database by enhancing the completeness of
enforcement information. FAA expects to address some of these issues as
it revises its enforcement policy, which is expected to be issued later
in fiscal year 2006. In addition, FAA has established a database
workgroup that is developing long-and short-term solutions to address
the problems with EIS.
Recommendations We Have Made to Improve FAA's Safety Oversight System:
In order to help FAA fully realize the benefits from its safety
oversight system, we have made a number of recommendations to address
weaknesses that we identified in our reviews. These recommendations
have not been fully implemented, although in some cases FAA has taken
steps towards addressing them. Evaluative processes and relevant data
are particularly important as FAA works to change its culture by
incorporating a system safety approach into its oversight, and we have
recommended that FAA develop continuous evaluative processes for its
oversight programs for non-legacy airlines, its designee programs, and
its industry partnership and enforcement programs, and systematically
assess inspectors' technical training needs. In addition, FAA's
nationwide databases are in need of improvements in their
comprehensiveness and ease of use. Without comprehensive nationwide
data, FAA does not have the information needed to evaluate its safety
programs and have assurance that they are having the intended results.
We have recommended that FAA improve the completeness of its designee
and enforcement databases. Continuous improvements in these areas are
critical to FAA's ability to have a robust "early warning system" and
maintain one of the safest aviation systems in the world.
Contacts and Acknowledgments:
For further information on this testimony, please contact Dr. Gerald
Dillingham at (202) 512-2834 or by email at dillinghamg@gao.gov.
Individuals making key contributions to this testimony include Brad
Dubbs, Phillis Riley, Teresa Spisak, and Alwynne Wilbur.
[End of section]
Appendix I: Description of FAA's Inspection Programs:
Table 3 describes the Federal Aviation Administration's (FAA) three
inspection processes for overseeing airlines: Air Transportation
Oversight System (ATOS), National Work Program Guidelines (NPG), and
Surveillance and Evaluation Program (SEP). Many of the elements of
ATOS, such as the use of data to identify risks and the development of
surveillance plans by inspectors, are incorporated in the SEP process.
The NPG process, in contrast, is not focused on the use of data and
relies on an established set of inspections that are not risk based.
Table 3: Various Elements of ATOS, NPG, and SEP:
Description of program;
ATOS:
* Focuses on safety vulnerabilities rather than regulatory compliance;
* Analysts and inspectors review airline data to identify areas of
safety risk;
* Inspectors develop surveillance plans for each airline, based on data
analysis and assessment of risks, and adjust the plans periodically
based on inspection results;
NPG:
* Focuses on inspectors completing a prescribed number of inspection
activities;
* Primarily based on checking airline compliance with regulations;
* Relies on inspectors' expertise to identify trends and risks;
SEP:
* Focuses on inspectors conducting a risk assessment of various areas;
* Inspectors review data to identify areas of safety risk and use
Flight Standards Safety Analysis Information Center and the Safety
Performance Analysis System as analytical tools;
* Inspectors develop surveillance plans for each airline, based on data
analysis and assessment of risks, and adjust plans periodically based
on inspection results;
* Inspectors can also verify that planned NPG activities meet the
surveillance needs for a particular year.
Type of commercial passenger airline inspected;
ATOS: Legacy commercial airlines;
NPG: Non-legacy commercial airlines;
SEP: Non-legacy commercial airlines.
Frequency of inspections;
ATOS: Continuous safety oversight;
NPG: Periodic; regular inspections are established annually by an FAA
headquarters committee;
SEP: Periodic; inspections are established during meetings held at
least twice a year using risk-based criteria.
Approximate number of aviation safety inspectors conducting
inspections[A];
ATOS: 585;
NPG: 1,100[B];
SEP: 1,100[B].
Number of commercial passenger airlines under the program[B];
ATOS: 13[C];
NPG: 99;
SEP: 99.
Source: GAO and FAA:
[A] As of July 2005.
[B] There are a total of about 1,100 inspectors for both the NPG and
SEP inspections.
[C] FedEx and United Parcel Service, two cargo air carriers, are also
in the ATOS program. This number reflects the recent merger of US
Airways and America West, which were both in ATOS prior to the merger.
[End of table]
[End of section]
Appendix II: Description of FAA's Partnership Programs:
Aviation Safety Action Program (ASAP):
Year Established: 1997:
Participation: Participants include employees of air carriers and
repair stations that have entered into a memorandum of understanding
with the Federal Aviation Administration (FAA). The memoranda can cover
employee groups, such as pilots, maintenance employees, dispatchers, or
flight attendants. Each employee group is covered by a separate
memorandum of understanding. As of June 2004, FAA had accepted 54
memoranda of understanding and received over 80,000 ASAP reports, which
may or may not include safety violations, according to FAA officials.
Purpose: ASAP seeks to improve aviation safety through the voluntary
self-reporting of safety incidents under the procedures set forth in
the memorandum of understanding. Under the program, FAA does not take
enforcement action against employees who voluntarily self-reported
safety violations for reports that are sole-source (the report is the
only way FAA would have learned about the incident) and will pursue
administrative action only for reports that are not sole-source.
Incidents that involve alcohol, drugs, criminal activity, or an
intentional disregard for safety are not eligible for self-reporting
under ASAP.
Process: Each memorandum of understanding is a voluntary partnership
between FAA, the airline, and an employee group. Although employee
groups are not always included, FAA encourages their participation. The
memorandum of understanding ensures that employees who voluntarily
disclose FAA safety violations in accordance with the procedures and
guidelines of ASAP will receive administrative action or no action in
lieu of legal enforcement action.
Once a memorandum of understanding is approved, employees can begin
reporting violations that fall under the agreement. When a violation
occurs, an employee notifies the Event Review Committee, which includes
representatives from FAA and the airline or the repair station and
generally includes the appropriate employee association. The committee
must be notified in writing within the time limit specified in the
memorandum of understanding. The committee then determines whether to
accept the report under the ASAP program. If the report is accepted (it
meets the acceptance criteria in the memorandum and does not involve
criminal activity, substance abuse, controlled substances, or alcohol),
then the committee determines the action to take. That action may
include remedial training or administrative action, but it will not
include a legal sanction.
Results: FAA does not know the overall program results because it does
not have a national, systematic process in place to evaluate the
overall success of ASAP. However, FAA cites examples that describe
ASAP's contribution to enhanced aviation safety. These examples include
identifying deficiencies in aircraft operations manuals, airport
equipment, and runways. In July 2003, FAA's Compliance and Enforcement
Review recommended that FAA evaluate the use and effectiveness of this
program.
Aviation Safety Reporting Program (ASRP):
Year Established: 1975:
Participation: Participants are all users of the national airspace
system, including air traffic controllers and employees of air carriers
and repair stations.
Purpose: The program is designed to improve aviation safety by offering
limited immunity for individuals who voluntarily report safety
incidents. ASRP was founded after TWA Flight 514 crashed on approach to
landing in December 1974 after the crew misinterpreted information on
the approach chart. This accident occurred only 6 weeks after another
plane experienced the same error.
Process: The National Aeronautics and Space Administration (NASA)
administers this program. When a safety incident occurs, a person may
submit a form and incident report to NASA. There are four types of
forms that can be submitted to NASA: (1) Air Traffic Control, (2)
General Reports (includes Pilots), (3) Flight Attendants, and (4)
Maintenance Personnel.
At least two aviation safety analysts read these forms and the incident
reports that accompany them. The analysts at NASA screen the incident
reports for urgent safety issues, which will be marked for immediate
action to the appropriate FAA office or aviation authority. NASA
analysts also edit the report's narrative to eliminate any identifying
information. In addition, each report has a tear-off portion, which is
separated and returned to the individual who reported the incident as a
receipt of the incident report's acceptance into the ASRP. When a
safety violation that has been previously reported under ASRP comes to
the attention of FAA, the agency issues a legal sanction, which is then
waived. Reports that would not be eligible to have a legal sanction
waived include deliberate violations, violations involving a criminal
offense, or accident; reports filed by participants who have committed
a violation of federal aviation regulations or law within the last 5
years and reports filed later than 10 days following an incident.
Results: While FAA and NASA do not know the overall program results
because they do not have a formal national evaluation program to
measure the overall effectiveness of the program, the agencies widely
disseminate information generated from the program to aircraft
manufacturers and others. ASRP reports are compiled into a database
known as the Aviation Safety Reporting System. When a potentially
hazardous condition is reported, such as a defect in a navigational aid
or a confusing procedure, NASA will send a safety alert to aircraft
manufacturers, the FAA, airport representatives, and other aviation
groups. The database is used for a monthly safety bulletin that
includes excerpts from incident reports with supporting commentary by
FAA safety experts. NASA officials estimate that the bulletin is read
by over 150,000 people. In addition, individuals and organizations can
request a search of the database for information on particular aircraft
aviation safety subjects, including human performance errors and safety
deficiencies. Further, NASA has used the database to analyze
operational safety issues, such as general aviation incidents, pilot
and controller communications, and runway incursions.
Flight Operational Quality Assurance (FOQA):
Year Established: 1995:
Participation: Participants include air carriers that equip their
airplanes to record flight data. As of March 2004, 13 airlines had FAA-
approved FOQA programs, and approximately 1,400 airplanes were equipped
for the program.
Purpose: FOQA is designed to enhance aviation safety through the
analysis of digital flight data generated during routine flights.
Process: Air carriers that participate in the program equip their
aircraft with special acquisition devices or use the airplanes' flight
data recorders to collect data and determine if the aircraft are
deviating from standard procedures. These data include engine
temperatures, descent rate, and deviations from the flight path. When
the aircraft lands, data are transmitted from the aircraft to the
airline's FOQA station, where they are analyzed for flight trends and
possible safety problems.
Once the data are transmitted to the FOQA ground station, the data are
extracted and analyzed by software programs. The FOQA data are combined
with data from maintenance databases, weather conditions, and other
safety reporting systems, such as ASAP, in order to identify trends in
flight operations. The analysis typically focuses on events that fall
outside normal boundaries specified by the manufacturer's operational
limitations and the air carrier's operational standards.
FOQA data are collected and analyzed by individual air carriers. The
data on safety trends are made available to FAA in an aggregated form
with no identification of individual carriers. According to FAA
officials, air carriers do not want to release this data to any outside
party (including FAA) because of concerns that the data could then be
publicly released. Air carriers pay for the special flight data
recorders that can record FOQA data, which cost approximately $20,000
each. Although this can be an expensive investment for some air
carriers, most newer aircraft models come with the data recorder built
into the airplane. The International Civil Aviation Organization (ICAO)
has recommended that airlines from member countries implement a FOQA
program. FAA has notified ICAO that the program will remain voluntary
in the United States.
Results: Although FAA has no formal national evaluation program to
measure the overall results or effectiveness of FOQA programs, FAA
cites examples that describe FOQA's contribution to enhanced aviation
safety. For example, one FOQA program highlighted a high rate of
descent when airplanes land at a particular airport. On the basis of
the information provided from FOQA, air traffic controllers at the
airport were able to develop alternative approach procedures to
decrease the rate of descent.
Voluntary Disclosure Reporting Program (VDRP):
Year Established: 1990:
Participation: Participants include air carriers, repair stations, and
production approval holders.[Footnote 16]
Purpose: FAA initiated the program to promote aviation safety by
encouraging the voluntary self-reporting of manufacturing, and quality
control problems and safety incidents involving FAA requirements for
maintenance, flight operations, drug and alcohol prevention programs,
and security functions.
Process: Upon discovering a safety violation, participants can
voluntarily disclose the violation to FAA within 24 hours. The initial
notification should include a description of the violation, how and
when the violation was discovered, and the corrective steps necessary
to prevent repeat violations. Within 10 days of filing the initial
notification to FAA, the entity is required to provide a written report
that cites the regulations violated, describes how the violation was
detected, provides an explanation of how the violation was inadvertent,
and provides a description of the proposed comprehensive fix. FAA may
pursue legal action if the participant discloses violations during, or
in anticipation of, an FAA inspection.
The violation must be reported immediately after being detected, must
be inadvertent, must not indicate that a certificate holder is
unqualified, and must include the immediate steps that were taken to
terminate the apparent violation. If these conditions are met, and the
FAA inspector has approved the comprehensive fix, then the FAA
inspector will prepare a letter of correction and the case is
considered closed with the possibility of being reopened if the
comprehensive fix is not completed.
Results: FAA does not know the overall program results because it does
not have a process to measure the overall effectiveness of the program
nationwide. A 2003 internal FAA report recommended that the agency
evaluate the use and effectiveness of this program.
[End of section]
Selected GAO Reports on Aviation Safety:
Aviation Safety: System Safety Approach Needs Further Integration into
FAA's Oversight of Airlines. GAO-05-726. Washington, D.C.: September
28, 2005.
Aviation Safety: FAA Management Practices for Technical Training Mostly
Effective; Further Actions Could Enhance Results. GAO-05-728.
Washington, D.C.: September 7, 2005.
Aviation Safety: Oversight of Foreign Code-Share Safety Program Should
Be Strengthened. GAO-05-930. Washington, D.C.: August 5, 2005.
Aviation Safety: FAA Needs to Strengthen the Management of Its Designee
Programs. GAO-05-40. Washington, D.C.: October 8, 2004.
Aviation Safety: Better Management Controls are Needed to Improve FAA's
Safety Enforcement and Compliance Efforts. GAO-04-646. Washington,
D.C.: July 6, 2004.
Aviation Safety: Information on FAA's Data on Operational Errors at Air
Traffic Control Towers. GAO-03-1175R. Washington, D.C.: September 23,
2003.
Aviation Safety: FAA Needs to Update the Curriculum and Certification
Requirements for Aviation Mechanics. GAO-03-317. Washington, D.C.:
March 6, 2003.
Aviation Safety: FAA and DOD Response to Similar Safety Concerns. GAO-
02-77. Washington. D.C.: January 22, 2002.
Aviation Safety: Safer Skies Initiative Has Taken Initial Steps to
Reduce Accident Rates by 2007. GAO/RCED-00-111. Washington, D.C.: June
30, 2000.
Aviation Safety: FAA's New Inspection System Offers Promise, but
Problems Need to Be Addressed. GAO/RCED-99-183. Washington, D.C.: June
28, 1999.
Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in
Identifying and Responding to Risks. GAO/RCED-98-6. Washington, D.C.:
February 27, 1998.
FOOTNOTES
[1] We refer to all passenger airlines that are not in FAA's Air
Transportation Oversight System (ATOS) as non-legacy airlines. The
seven "legacy" airlines and eight other airlines are overseen through
ATOS. The air carriers in the ATOS program are Alaska; American;
Continental; Delta; Northwest; United; American Eagle; Champion;
ExpressJet; SkyWest; Southwest; Trans States; FedEx; United Parcel
Service; and US Airways, which recently merged with America West.
[2] GAO, Aviation Safety: System Safety Approach Needs Further
Integration into FAA's Oversight of Airlines, GAO-05-726 (Washington,
D.C.: Sept. 28, 2005).
[3] GAO, Aviation Safety: Weaknesses in Inspection and Enforcement
Limit FAA in Identifying and Responding to Risks, GAO/RCED-98-6
(Washington, D.C.: Feb. 27, 1998); GAO, Aviation Safety: FAA Needs to
More Aggressively Manage Its Inspection Program, GAO/T-RCED-92-25
(Washington, D.C.: Feb. 6, 1992).
[4] GAO/RCED-98-6; GAO/T-RCED-92-25.
[5] GAO, Aviation Safety: FAA Needs to Strengthen the Management of Its
Designee Programs, GAO-05-40 (Washington, D.C.: Oct. 8, 2004).
[6] We identified 62 aviation experts with knowledge and expertise in
FAA's designee programs, who participated on a Web-based panel that
provided the group's views on the strengths and weaknesses of the
designee programs and ways to improve the programs. The experts
included designees, FAA inspectors and engineers, independent experts
and university academics, and private sector and aviation industry
associations. We obtained the experts' views by employing an iterative
and controlled feedback process for obtaining individual views and then
allowing each participant to respond to the entire panel's comments.
[7] We define technical training as training in aviation technologies.
FAA includes in its definition of technical training topics such as
system safety and risk analysis, inspector job skills, data analysis,
and training in software packages.
[8] GAO, Aviation Safety: FAA Management Practices for Technical
Training Mostly Effective; Further Actions Could Enhance Results, GAO-
05-728 (Washington, D.C.: Sept. 7, 2005). We compared FAA's management
of its inspector technical training efforts with effective management
practices outlined in GAO, Human Capital: A Guide for Assessing
Strategic Training and Development Efforts in the Federal Government,
GAO-04-546G (Washington, D.C.: Mar. 1, 2004).
[9] GAO-05-40.
[10] However, many experts on our panel indicated it was of high or
highest importance to ensure standard training of designees within
specific specialties to improve the consistency of their work, and to
increase the number of subject-matter workshops for designees.
[11] Because of the statistical survey techniques we employed in
surveying FAA's inspectors, we are 95 percent confident that the
results we present are within 4.6 percentage points of the results that
we would have obtained if we had surveyed all 3,000 inspectors. That
is, we are 95 percent confident that had we surveyed all inspectors,
between 48 and 57 percent of them would have told us that, to a great
or very great extent, they have the technical knowledge to do their
jobs. All percentage estimates from the survey have a margin of error
of plus or minus 4.6 percentage points or less, unless otherwise noted.
[12] GAO, Aviation Safety: FAA Needs to Update the Curriculum and
Certification Requirements for Aviation Mechanics, GAO-03-317
(Washington, D.C.: Mar. 6, 2003).
[13] GAO-05-40.
[14] These databases are Flight Standards Service's Program Tracking
and Reporting Subsystem and National Vital Information Subsystem,
Aircraft Certification Service's Designee Information Network, and
Office of Aerospace Medicine's Airmen Medical Certification Information
Subsystem.
[15] We found that during fiscal years 1993 through 2003, FAA closed
about 53 percent of the nearly 200,000 enforcement actions with
administrative actions (such as warning notices). About 28 percent of
the actions were closed with legal sanctions (such as fines) and about
18 percent were closed with no enforcement action.
[16] A production approval holder is an entity that holds a
certificate, approval, or authorization from FAA to manufacture
aircraft, aircraft engines, propellers, and related parts and articles.
