Aviation Security
Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed Gao ID: GAO-05-781 September 6, 2005Training flight and cabin crew members to handle potential threats against domestic aircraft is an important element in securing our nation's aviation system. The responsibility for ensuring that crew members are prepared to handle these threats is a shared responsibility between the private sector--air carriers--and the federal government, primarily the Transportation Security Administration (TSA). This report addresses (1) actions TSA has taken to develop guidance and standards for flight and cabin crew member security training and to measure the effectiveness of the training, (2) how TSA ensures domestic air carriers comply with the training guidance and standards, and (3) efforts TSA has taken to develop and assess the effectiveness of its voluntary self-defense training program.
Since the terrorist attacks of September 11, 2001, TSA enhanced guidance and standards for flight and cabin crew member security training with input from stakeholders. Specifically, TSA revised the guidance and standards to include additional training elements required by law and to improve the organization and clarity of the guidance and standards. Some stakeholders we interviewed and our own review generally found that the revised guidance and standards improved upon previous versions in terms of organization and clarity of the information provided. However, some stakeholders identified concerns about, for example, the reasonableness of applying parts of the guidance and standards to both flight and cabin crew members and the difficulty in implementing some of the standards without additional information or training tools from TSA. Additionally, TSA has not established strategic goals and performance measures for assessing the effectiveness of the training because it considers its role in the training program as regulatory. In this regard, TSA views the individual air carriers as responsible for establishing performance goals and measures for their training programs, but has not required them to do so. Without goals and measures, TSA and air carriers will be limited in their ability to fully assess accomplishments and target associated improvements. TSA recently took steps to strengthen its efforts to oversee air carriers' flight and cabin crew security training to ensure they are complying with the required guidance and standards. For example, in January 2005, TSA added staff with expertise in designing training programs to review air carriers' crew member security training curriculums and developed a standard form for staff to use to conduct their reviews. However, TSA lacks adequate controls for monitoring and reviewing air carriers' crew member security training, including written procedures for conducting and documenting these reviews. TSA plans to develop written procedures, but has not established a timeframe for completing this effort. TSA has developed an advanced voluntary self-defense training program with input from stakeholders and implemented the program in December 2004, as required by law. However, stakeholders and our own analysis identified concerns about the training design and delivery, such as the lack of recurrent training and the lack of a realistic training environment. Also, TSA has not yet established performance measures for the program or established a time frame for evaluating the program's effectiveness.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-781, Aviation Security: Flight and Cabin Crew Member Security Training Strengthened, but Better Planning and Internal Controls Needed
This is the accessible text file for GAO report number GAO-05-781
entitled 'Aviation Security: Flight and Cabin Crew Member Security
Training Strengthened, but Better Planning and Internal Controls
Needed' which was released on September 8, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
September 2005:
Aviation Security:
Flight and Cabin Crew Member Security Training Strengthened, but Better
Planning and Internal Controls Needed:
GAO-05-781:
GAO Highlights:
Highlights of GAO-05-781, a report to congressional requesters:
Why GAO Did This Study:
Training flight and cabin crew members to handle potential threats
against domestic aircraft is an important element in securing our
nation‘s aviation system. The responsibility for ensuring that crew
members are prepared to handle these threats is a shared responsibility
between the private sector”air carriers”and the federal government,
primarily the Transportation Security Administration (TSA). This report
addresses (1) actions TSA has taken to develop guidance and standards
for flight and cabin crew member security training and to measure the
effectiveness of the training, (2) how TSA ensures domestic air
carriers comply with the training guidance and standards, and (3)
efforts TSA has taken to develop and assess the effectiveness of its
voluntary self-defense training program.
What GAO Found:
Since the terrorist attacks of September 11, 2001, TSA enhanced
guidance and standards for flight and cabin crew member security
training with input from stakeholders. Specifically, TSA revised the
guidance and standards to include additional training elements required
by law and to improve the organization and clarity of the guidance and
standards. Some stakeholders we interviewed and our own review
generally found that the revised guidance and standards improved upon
previous versions in terms of organization and clarity of the
information provided. However, some stakeholders identified concerns
about, for example, the reasonableness of applying parts of the
guidance and standards to both flight and cabin crew members and the
difficulty in implementing some of the standards without additional
information or training tools from TSA. Additionally, TSA has not
established strategic goals and performance measures for assessing the
effectiveness of the training because it considers its role in the
training program as regulatory. In this regard, TSA views the
individual air carriers as responsible for establishing performance
goals and measures for their training programs, but has not required
them to do so. Without goals and measures, TSA and air carriers will be
limited in their ability to fully assess accomplishments and target
associated improvements.
TSA recently took steps to strengthen its efforts to oversee air
carriers‘ flight and cabin crew security training to ensure they are
complying with the required guidance and standards. For example, in
January 2005, TSA added staff with expertise in designing training
programs to review air carriers‘ crew member security training
curriculums and developed a standard form for staff to use to conduct
their reviews. However, TSA lacks adequate controls for monitoring and
reviewing air carriers‘ crew member security training, including
written procedures for conducting and documenting these reviews. TSA
plans to develop written procedures, but has not established a
timeframe for completing this effort.
TSA has developed an advanced voluntary self-defense training program
with input from stakeholders and implemented the program in December
2004, as required by law. However, stakeholders and our own analysis
identified concerns about the training design and delivery, such as the
lack of recurrent training and the lack of a realistic training
environment. Also, TSA has not yet established performance measures for
the program or established a time frame for evaluating the program‘s
effectiveness.
What GAO Recommends:
GAO is recommending that the Secretary of Homeland Security direct TSA
to (1) establish strategic goals for crew security training, develop
guidance and standards for air carriers to use to develop goals and
measures for their training, and review air carriers‘ goals and
measures, (2) develop written procedures for monitoring air carriers‘
crew security training, and (3) establish performance measures and a
time frame for evaluating the effectiveness of the voluntary self-
defense training. TSA reviewed a draft of this report and generally
agreed with GAO‘s findings and recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-05-781.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cathleen Berrick at (202)
512-8777 or berrickc@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
TSA Has Enhanced Guidance and Standards for Flight and Cabin Crew
Security Training with Input from Stakeholders, but Better Planning Is
Needed:
Recent Progress Made to Strengthen Oversight of Flight and Cabin Crew
Member Security Training Program, but Weaknesses Remain:
TSA Developed and Implemented Crew Member Self-Defense Training, but
Has Not Established Performance Measures or Evaluated Training
Effectiveness:
Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: TSA's Deployment of an Advanced Voluntary Crew Member
Self- Defense Training from December 2004 through September 2005, as of
July 2005:
Appendix III: Comments from the Department of Homeland Security:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: List of Basic Crew Member Security Training Elements Required
By Law:
Table 2: List of Advanced Voluntary Crew Member Self-Defense Training
Elements Required By Law:
Table 3: Summary of Stakeholders' Concerns on TSA's Prototype Advanced
Voluntary Self-Defense Training and Actions Taken by TSA in Response to
the Concerns:
Table 4: Crew Member Participation in Advanced Voluntary Crew Member
Self-Defense Training from December 2004 through June 2005:
Figure:
Figure 1: Number of Domestic Air Carriers by Carrier Group and
Percentage of Passengers Flown Domestically by Carrier Group:
Abbreviations:
ATSA: Aviation and Transportation Security Act:
DHS: Department of Homeland Security:
DOD: Department of Defense:
DOT: Department of Transportation:
FAA: Federal Aviation Administration:
FAMS: Federal Air Marshal Service:
FBI: Federal Bureau of Investigation:
GPRA: Government Performance and Results Act:
OMB: Office of Management and Budget:
TSA: Transportation Security Administration:
Vision 100Vision100-Century of Aviation Reauthorization Act:
United States Government Accountability Office:
Washington, DC 20548:
September 6, 2005:
The Honorable Daniel K. Inouye:
Co-Chairman:
Committee on Commerce, Science, and Transportation:
United States Senate:
The Honorable John D. Rockefeller, IV:
Ranking Minority Member:
Subcommittee on Aviation:
Committee on Commerce, Science, and Transportation:
United States Senate:
Training flight and cabin crew members (pilots and flight attendants)
to handle potential threats against domestic commercial aircraft is an
important element in securing our nation's aviation system. These
flight and cabin crew members play a key role in ensuring the security
of commercial aircraft and the safety of airline passengers and are the
last line of defense in the event of an attempted terrorist attack
onboard a commercial aircraft. The responsibility for ensuring that
flight and cabin crew members are prepared to respond to these threats
is a shared responsibility between the private sector--air carriers--
and the federal government, primarily the Transportation Security
Administration (TSA). In an effort to strengthen aviation security,
title 49 of the United States Code, section 44918, requires that air
carriers develop and deliver a basic security training program for
flight and cabin crew members in accordance with training elements
prescribed in the law. The law also requires that TSA monitor and
periodically review air carriers' training programs to ensure they
adequately prepare crew members for potential threat conditions.
Furthermore, the law requires TSA to develop and provide an advanced
voluntary self-defense training program for flight and cabin crew
members. Although there has not been another successful terrorist
attack against commercial aircraft in the United States since September
11, 2001, concerns remain about whether flight and cabin crew members
are prepared to handle a potential hijacking and other threats to
commercial aircraft.
To determine the progress TSA has made in developing and monitoring
flight and cabin crew security training, we examined TSA's efforts to
develop guidance and standards for air carriers' flight and cabin crew
security training, monitor air carriers' compliance with the guidance
and standards, and develop and deliver advanced voluntary self-defense
training for crew members. Specifically, this report addresses the
following questions: (1) What actions has TSA taken to develop guidance
and standards for flight and cabin crew security training and to
measure the effectiveness of the training? (2) How does TSA ensure
domestic air carriers comply with required training guidance and
standards? (3) What efforts has TSA taken to develop, implement, and
measure the effectiveness of advanced voluntary self-defense training
for flight and cabin crew members?
In conducting our work, we reviewed TSA and Federal Aviation
Administration (FAA) documentation related to flight and cabin crew
member security training guidance and standards and advanced voluntary
crew member self-defense training. The security training guidance
provides detailed guidance from which air carriers must develop their
flight and cabin crew security training programs. The security training
standards set forth the specific requirements for crew member security
training, such as the training elements that must be included in air
carriers' basic and recurrent (refresher) crew member security training
programs. We also interviewed officials from 19 domestic air carriers,
2 air carrier associations, and 6 crew member labor organizations
regarding security training guidance and standards, and advanced
voluntary self-defense training.[Footnote 1] We selected domestic air
carriers based on whether they were currently offering initial and/or
recurrent security training[Footnote 2] and on the size of the air
carrier in an effort to include a mixture of various domestic air
carriers and air carriers of varying sizes. The size of an air carrier
is based on the annual operating revenues and the number of revenue
passenger boardings. We visited 8 of these air carriers to observe
their training and to interview flight and cabin crew members about the
training. Because we selected a nonprobability sample of domestic air
carriers, the information we obtained from these interviews and visits
cannot be generalized to all domestic air carriers. We also observed
two sessions of TSA's advanced voluntary crew member self-defense
training. Information obtained from our interviews and visits cannot be
generalized to all air carriers and stakeholders. Additionally, we
interviewed officials responsible for developing security training
guidance and standards and reviewing air carriers security training
programs at TSA headquarters. We also interviewed officials at the FAA;
Federal Bureau of Investigation (FBI); Bureau of Immigration and
Customs Enforcement, Federal Air Marshal Service (FAMS); Department of
Defense (DOD); crew member labor organizations; and associations
representing air carriers to obtain information on their involvement in
the development of crew member security training guidance and standards
and advanced voluntary crew member self-defense training. A more
detailed description of our scope and methodology is contained in
appendix I.
We conducted our work from June 2004 through August 2005 in accordance
with generally accepted government auditing standards.
Results in Brief:
Building on the legislatively mandated guidance developed by FAA in
January 2002 and corresponding standards, TSA enhanced guidance and
standards for flight and cabin crew member security training with input
from various stakeholders. However, TSA has not established strategic
goals and performance measures for assessing the effectiveness of crew
member security training, nor required air carriers to do so. TSA
officials stated that they revised the guidance and standards for two
main reasons. First, the law, as amended by the Vision 100---Century of
Aviation Reauthorization Act (Vision 100), enacted in December 2003,
required that air carriers include additional training elements in
their basic crew member security training programs to prepare flight
and cabin crew members for potential threat conditions.[Footnote 3]
Second, TSA determined that the guidance and standards needed to be
better organized and to more clearly define security training elements,
in part due to feedback from air carriers, flight and cabin crew member
labor organizations, and associations representing air carriers. Some
stakeholders stated and our own review found that the organization of
the previous security training standards were difficult to follow and
lacked clarity in some areas. For example, the previous guidance did
not define what constitutes life-threatening behavior, whereas the
revised guidance provides both a definition of this behavior and
examples. Although the revised guidance and standards were an
improvement over the previous versions in terms of organization and
clarity, some stakeholders identified concerns about the reasonableness
of applying parts of the guidance and standards to both flight and
cabin crew members, difficulty in implementing some of the standards
without additional information or training tools from TSA, and the
vagueness of some of the guidance and standards. For example, some of
the training standards remain generalized to both flight attendants and
pilots, rather than targeted to their specific job functions in
responding to a security threat. Additionally, TSA has not established
strategic goals and performance measures for the flight and cabin crew
member security training program. The Government Performance and
Results Act (GPRA) of 1993 requires, among other things, that agencies
use outcome-oriented goals and measures that assess results of a
program or activity compared to its intended purpose. TSA training
officials stated that they decided not to develop strategic goals or
performance measures for flight and cabin crew security training
because the officials view their role in the training program as
regulatory--that is, monitoring air carriers' compliance with the
training guidance and standards established by TSA. In this regard,
officials stated that it is the individual air carriers' responsibility
to establish performance goals and measures specific to their security
training programs. However, without overall strategic goals developed
by TSA, air carriers do not have a framework from which to develop
their individual performance goals and measures. Further, TSA has not
explicitly required air carriers to develop performance goals and
measures or provided air carriers with guidance and standards for doing
so. The absence of performance goals and measures for flight and cabin
crew security training limits the ability of TSA and air carriers to
fully assess the accomplishments of the flight and cabin crew member
security training program in adequately training crew members, and to
target appropriate improvements.
Although TSA has recently taken steps to strengthen its efforts to
oversee air carriers' flight and cabin crew security training to ensure
carriers are complying with required guidance and standards, TSA lacks
adequate internal controls for monitoring and reviewing air carriers'
flight and cabin crew member security training. The law requires TSA to
monitor and periodically review air carriers' security training to
ensure that the training is adequately preparing crew members for
potential threat conditions. The law also requires TSA to consider
complaints from crew members in determining when to review an air
carriers' crew member security training program. Further, the
Comptroller General's Standards for Internal Control in the Federal
Government, which describes the minimum level of quality acceptable for
internal controls in government and the basis against which internal
controls should be evaluated, calls for, among other things, that
controls generally be designed to assure that ongoing monitoring occurs
during the course of normal operations, transactions and other
significant events be documented clearly, and documentation be readily
available for examination.[Footnote 4] TSA has recently taken several
steps to strengthen its review of air carriers' crew member security
training curriculum, including (1) adding staff with expertise in
designing training programs to review the overall design of the air
carriers' crew member security training curriculums, rather than solely
ensuring that each of the training elements is discussed in the
training curriculums; (2) developing a standard form for TSA inspectors
and training staff to use to conduct and document their reviews of air
carrier security training curriculums to ensure training requirements
stated in the revised security training standards are included in the
air carriers' security training and to enhance consistency in the
review process; (3) requiring air carriers to obtain written
participant feedback at the end of flight and cabin crew member
security training to capture information on participants' overall
satisfaction with the training; and (4) reorganizing the inspection
staff into a newly created Office of Compliance and issuing position
announcements to fill vacant TSA inspector positions, which should
allow for greater monitoring of air carriers' flight and cabin crew
security training programs. Although TSA made these recent
improvements, the agency lacks adequate internal controls for
monitoring and reviewing air carriers' crew member security training.
Specifically, TSA does not have written procedures for (1) completing
the standard form used by TSA inspectors and training staff in
reviewing air carriers' flight and cabin crew security training
curriculum; (2) conducting and documenting observations of air
carriers' classroom delivery of flight and cabin crew security
training--a function performed by TSA inspectors--and selecting the air
carriers to visit; (3) ensuring that air carriers track flight and
cabin crew members' completion of required security training; and (4)
considering flight and cabin crew members' written feedback on security
training and security-related complaints from flight and cabin crew
members in determining when to review air carriers' flight and cabin
crew security training. TSA officials stated that they plan to develop
a handbook that documents procedures for TSA inspectors to use in
reviewing air carriers' flight and cabin crew members security training
as well as guidance for the training staff to use during their reviews.
However, officials stated that they have not established a time frame
for completing these efforts. Until TSA documents procedures for
monitoring and reviewing air carriers' flight and cabin crew member
security training, TSA will continue to lack a structure that provides
reasonable assurance that TSA inspectors and training staff are
performing thorough assessments of air carriers' security training.
TSA has developed an advanced voluntary self-defense training program
for flight and cabin crew members with input from key stakeholders and
implemented the program in December 2004 as required by law. However,
some stakeholders and our own analysis identified concerns about the
training design and delivery, and TSA has not yet developed performance
measures for the program or established a time frame for evaluating the
program's overall effectiveness. In developing the advanced voluntary
self-defense training program, TSA consulted with law enforcement
experts, the Federal Air Marshal Service,[Footnote 5] air carriers, air
carrier associations, crew member labor organizations, and other
subject matter experts and self-defense experts. TSA also piloted the
training course in five cities and made refinements to the course based
on feedback from participants. For example, training participants
stated that the course included too much lecture time and that the
training course was too long, particularly given that crew members must
attend the training on their own time and must pay for their travel,
lodging, and meals. In response, TSA increased the amount of hands-on
training and shortened the training program from 4 days to 3 days.
During the first 7 months of the training program, participation in the
training was relatively low, with only 474 flight and cabin crew
members (39 percent of total capacity) attending the training. TSA
officials attributed the low participation to crew members having a
difficult time obtaining 3 consecutive days of leave to attend the
training. Although TSA incorporated some stakeholder input into the
course design, some stakeholders, including individuals identified as
experts by TSA, and our own analysis identified several concerns
regarding the training design and delivery, such as the lack of
recurrent training, the lack of a realistic setting in which to conduct
the training, and instructors' lack of knowledge of crew members'
actual work environment. Our prior human capital work has found that in
implementing a training program, an agency should ensure that
implementation involves effective and efficient delivery of
training[Footnote 6]--that is, the training should be conducted in a
setting that approximates the participants' working conditions and be
taught by individuals who are knowledgeable about the subject matter
and work environment. Furthermore, TSA has not yet developed
performance measures for the program or established a time frame for
evaluating the program's overall effectiveness. TSA training officials
stated that although they recognize the importance of measuring the
effectiveness of the self-defense training program, they have not been
able to undertake this effort due to resource constraints within which
they are operating and numerous internal process improvements currently
under way in TSA. Without performance measures and an evaluation of the
program's effectiveness, TSA will not have meaningful information from
which to determine whether the voluntary training program is achieving
its intended results and to make any needed improvements.
To assist TSA in further strengthening its flight and cabin crew
security training program and to provide the tools necessary to monitor
the delivery and accomplishments of the training, we are making a
number of recommendations to the Secretary of the Department of
Homeland Security (DHS). These recommendations include directing the
Assistant Secretary, Transportation Security Administration, to develop
a framework from which to assess the accomplishments of flight and
cabin crew member security training, including establishing strategic
goals for the training; developing guidance for air carriers to use in
establishing performance goals and measures for their individual
training programs; and reviewing air carriers' goals and measures as
part of its monitoring efforts. We are also recommending that the
Assistant Secretary, Transportation Security Administration, strengthen
internal controls for monitoring and reviewing air carriers' flight and
cabin crew member security training programs and for documenting the
results of its monitoring efforts; and establish performance measures
for the advanced voluntary crew member self- defense training program
and a time frame for evaluating the effectiveness of the training.
We provided a draft of this report to the Department of Homeland
Security for its review and comment. DHS, in its written comments,
generally concurred with the findings and recommendations in the
report, and agreed that efforts to implement our recommendations are
critical to a successful flight and cabin crew member security training
program. DHS described some actions TSA has taken or planned to take to
implement these recommendations. For example, DHS stated that TSA has
begun to establish strategic goals for the flight and cabin crew member
security training program. DHS also stated that TSA is in the process
of developing a monitoring plan, to the extent that resources permit,
and a handbook for reviewing air carriers' flight and cabin crew member
security training programs. Additionally, DHS stated that TSA is
currently working with the Office of Management and Budget (OMB) to
establish performance measures for use in OMB's Performance Assessment
Rating Tool for TSA's flight security training. The full text of DHS's
comments is included in appendix III.
Background:
After the terrorist attacks of September 11, 2001, the President signed
the Aviation and Transportation Security Act (ATSA) into law on
November 19, 2001, with the primary goal of strengthening the security
of the nation's aviation system. ATSA created TSA as the agency
responsible for securing all modes of transportation, including
aviation.[Footnote 7] The President also issued the National Strategy
for Homeland Security in July 2002. The National Strategy for Homeland
Security sets forth a plan to strengthen homeland security through the
cooperation of federal, state, local, and private-sector organizations
in various areas. The National Strategy for Homeland Security aligns
and focuses homeland security functions into six critical mission
areas: (1) intelligence and warning, (2) border and transportation
security, (3) domestic counterterrorism, (4) protecting critical
infrastructures and key assets, (5) defending against catastrophic
threats, and (6) emergency preparedness and response. A theme of the
national strategy is that homeland security is a shared responsibility
among these stakeholders, not solely the responsibility of the federal
government. In the case of flight and cabin crew member security
training, air carriers and TSA both play an important role. Air
carriers are responsible for developing and delivering security
training programs for their crew members. TSA (and previously FAA) is
responsible for developing the guidance and standards that air carriers
are to use to design and deliver their security training and for
monitoring air carriers' flight and cabin crew member security training
programs for compliance with the guidance and standards.[Footnote 8] If
TSA finds an air carrier to be noncompliant with developing and
conducting the required flight and cabin crew member security training,
TSA has a range of actions it can take, including imposing fines, and
in extreme circumstances, force the air carrier to shut down its
operations.
The Bureau of Transportation Statistics reported that 105 domestic
passenger air carriers were operating in the United States in 2004. Of
the 105 air carriers, 12 (11 percent) are major air carriers that
carried over 76 percent of the passengers in 2004. With a few
exceptions for small aircraft, every commercial flight in the United
States has at least two flight crew members and one cabin crew member
onboard. These crew members are viewed as the last line of defense in
what TSA describes as its layered security system, which includes
perimeter security (e.g., airport security fencing), 100 percent
passenger and checked baggage screening, hardened flight deck doors,
armed federal air marshals, and armed pilots.[Footnote 9] Figure 1
provides the number of domestic air carriers by carrier group (major,
national, and regional) and the percentage of passengers flown
domestically by carrier group during fiscal year 2004.[Footnote 10]
Figure 1: Number of Domestic Air Carriers by Carrier Group and
Percentage of Passengers Flown Domestically by Carrier Group:
[See PDF for image]
[End of figure]
Federal guidance for air carriers to use to develop their flight and
cabin crew security training programs has been in place for over 20
years. FAA developed the crew member security training guidance,
referred to as Common Strategy I, in the early 1980's in response to
numerous hijacking incidents in the late 1970's. Common Strategy I
generally instructed air carriers to develop training programs that
called for flight and cabin crew members to cooperate with threatening
passengers or hijackers and slow compliance with their demands. Based
on this guidance, FAA also developed corresponding security training
standards that set forth the requirements for flight and cabin crew
member security training. Air carriers were required to incorporate the
guidance and standards into their security training programs. FAA
principal security inspectors and principal operations inspectors were
responsible for monitoring air carriers' compliance with the security
training standards. The nature of the terrorist attacks on September
11, 2001, however, demonstrated that the philosophy of Common Strategy
I--to cooperate with hijackers--was flawed as it presumed that
hijackers would not use aircraft as weapons of mass destruction.
Following the events of September 11, 2001, section 107 of ATSA
required FAA, in consultation with TSA and other stakeholders, to
develop detailed guidance for flight and cabin crew security training
programs within 60 days after the enactment of the act. FAA developed
and issued security training guidance, in accordance with the
requirements of ATSA, on January 19, 2002. In February 2002, TSA
assumed responsibility for monitoring air carriers' security training
for United States passenger air carriers and the air carrier security
inspections function was transferred from FAA to TSA.[Footnote 11]
Following the enactment of ATSA, the President signed into law two acts
that amended the flight and cabin crew training requirements codified
at title 49 of the U.S. Code, section 44918--the Homeland Security Act
of 2002 and Vision 100. The Homeland Security Act, enacted on November
25, 2002, amended the law by, among other things, mandating that, if
TSA updated training guidance, it must issue a rule to include elements
of self-defense in the training programs.[Footnote 12] Vision 100,
subsequently enacted on December 12, 2003, amended the flight and cabin
crew security training law in its entirety to require that:
* air carriers providing scheduled passenger air transportation carry
out a training program that addresses the 10 elements listed in table 1;
* TSA approve the air carrier's training programs;
* TSA, in consultation with FAA, monitor air carrier training programs
and periodically review an air carrier's training program to ensure the
program is adequately preparing crew members for potential threat
conditions;
* TSA, in consultation with FAA, order air carriers to modify training
programs to reflect new or different security threats; and:
* TSA develop and provide an advanced voluntary self-defense training
program to provide both classroom and effective hands-on training in,
at least, the six training elements listed in table 2.
Table 1 lists the minimum training elements required by law, as enacted
by ATSA and as amended by Vision 100, for basic crew member security
training.
Table 1: List of Basic Crew Member Security Training Elements Required
By Law:
Legislative requirements for crew member security training:
Determination of the seriousness of any occurrence;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Crew
communication and coordination;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Appropriate
responses to defend oneself;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Use of
protective devices assigned to crew members;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Psychology
of terrorists to cope with hijacker behavior and passenger responses;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: (Live)
situational training exercises regarding various threat conditions;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Flight deck
procedures or aircraft maneuvers to defend the aircraft;
ATSA: Yes;
Vision 100: Yes.
Legislative requirements for crew member security training: Recognizing
suspicious activities;
ATSA: No;
Vision 100: Yes.
Legislative requirements for crew member security training: The proper
commands to give passengers and attackers;
ATSA: No;
Vision 100: Yes.
Legislative requirements for crew member security training: The proper
conduct of a cabin search, including explosive device recognition;
ATSA: No;
Vision 100: Yes.
Source: ATSA and Vision 100.
[End of table]
Table 2 lists the training elements that TSA must include in an
advanced voluntary self-defense training program for flight and cabin
crew members under the law, as amended by Vision 100.
Table 2: List of Advanced Voluntary Crew Member Self-Defense Training
Elements Required By Law:
Legislative requirements for crew member security training:
Deterring a passenger who might present a threat.
Advanced control, striking, and restraint techniques.
Training to defend oneself against edged or contact weapons.
Methods to subdue and restrain an attacker.
Use of available items aboard the aircraft for self-defense.
Appropriate and effective responses to defend oneself including the use
of force against an attacker.
Source: Vision 100.
[End of table]
Over the years, our work on best practices in training has found that
generally high-performing organizations follow certain key steps in
developing and measuring the effectiveness of training
programs.[Footnote 13] These steps include:
* planning--developing a strategic approach that establishes priorities
and leverages investments in training to achieve agency results and
identify the competencies--commonly referred to as knowledge, skills,
abilities, and behaviors--needed to achieve organizational missions and
goals, and measure the extent to which their employees possess these
competencies;
* design and development--identifying specific initiatives that the
agency will use, along with other strategies, to include individual and
organizational performance;
* implementation--ensuring effective and efficient delivery of training
opportunities in an environment that supports learning and change; and:
* evaluation--assessing the extent to which training efforts contribute
to improved performance and results.
TSA Has Enhanced Guidance and Standards for Flight and Cabin Crew
Security Training with Input from Stakeholders, but Better Planning Is
Needed:
TSA Enhanced Flight and Cabin Crew Member Security Training Guidance
and Standards with Input from Stakeholders, but Stakeholders Continue
to Have Concerns:
Building on the legislatively mandated guidance developed by FAA and
the corresponding standards, TSA enhanced crew member security training
guidance and standards with input from stakeholders in accordance with
the law, as amended by Vision 100. TSA policy and training officials
stated that they revised the guidance and standards for two main
reasons. First, the law required that air carriers include additional
training elements in their basic crew member security training programs
to prepare flight and cabin crew members for potential threat
conditions.[Footnote 14] Second, TSA determined that the guidance and
standards needed to be better organized and to more clearly define
security training elements, in part due to feedback from air carriers,
flight and cabin crew member labor organizations, and associations
representing air carriers.[Footnote 15] For example, stakeholders we
interviewed and our own review found that the organization of the
previous security training standards was difficult to follow in that
several requirements were addressed in multiple sections of the
document rather than focused in a single section. During the summer of
2003 and May 2004, TSA established two internal working groups
comprised of representatives of its policy, training, regulatory, and/
or legal offices. One working group was responsible for revising the
security training guidance, and the other working group was responsible
for revising the corresponding security training standards--the
standards from which air carriers must train their flight and cabin
crew members. TSA officials stated that these working groups determined
the reasonableness and appropriateness of the security training
elements contained in the existing guidance and standards in place at
that time and what additional training elements were needed.
During the development of the revised guidance and standards, TSA
provided external stakeholders[Footnote 16] with two opportunities to
provide comments. In July 2004, the first comment period, TSA convened
a meeting of external stakeholders to present an overview of the draft
revised guidance and standards and to provide copies of the documents
for their review and comment. TSA initially requested that stakeholders
provide comments on the draft revised guidance and standards within 2
weeks. However, in response to stakeholder concerns about the short
comment period, TSA extended the comment period for an additional 2
weeks. After consolidating all stakeholder comments, TSA's internal
working group reviewed the comments to determine which to incorporate
in the guidance and standards. In August 2004, the second comment
period, TSA convened additional meetings with external stakeholders--
one meeting with air carrier associations and another with crew member
labor organizations--to review each of the stakeholders' comments and
to discuss changes made to the revised guidance and standards in
response to these comments.[Footnote 17] In September 2004, TSA
provided the stakeholders with a 30-day comment period on the revised
guidance and standards. After receiving comments and determining which
of the suggested changes to include in the revised guidance and
standards, TSA issued the finalized guidance and standards to air
carriers on January 3, 2005.
Stakeholders we interviewed and our own analysis of revisions made to
the guidance and standards generally found the revised guidance and
standards to be better organized and to provide some additional clarity
on security training requirements for crew members. For example, we
found that the previous standards only implicitly addressed the
requirement for training on the psychology of terrorists and addressed
it in multiple sections in the document. In contrast, the revised
standards organized information on this requirement in a single section
and clearly identified the requirement as "psychology of terrorists."
Additionally, the previous guidance did not define what constitutes
life-threatening behavior, whereas the revised guidance provides both a
definition of this behavior and examples.
Although TSA made these enhancements, stakeholders we interviewed and
stakeholders identified by TSA provided concerns about the
reasonableness of applying parts of the guidance and standards to both
flight and cabin crew members, the difficulty in implementing some of
the standards without additional information or training tools from
TSA, and the vagueness of some of the guidance and standards. Our
interviews with officials from 19 air carriers and 8 representatives
from aviation associations and crew member labor organizations, after
the revised guidance and standards were finalized in January 2005, also
identified similar concerns. Regarding the applicability of the
standards to flight and cabin crew members, officials from 9 of the 19
air carriers that we interviewed stated that some of the training
standards remained generalized to both pilots and flight attendants,
rather than targeted to their specific job functions in responding to a
security threat. For example, TSA requires both pilots and flight
attendants to have annual hands-on training on how to use restraining
devices. However, 2 of the 19 air carriers we interviewed stated that
training pilots annually on how to use restraining devices is not
necessary because pilots are trained to stay inside the flight deck at
all times, even when an incident occurs in the aircraft cabin. TSA
officials stated that all crew members need annual hands-on training on
how to use restraining devices because off-duty flight crew members
frequently fly, and if an incident occurs in the aircraft cabin, they
will know how to use the devices. One crew member labor organization
agreed with TSA's position, stating that incidents could occur in which
pilots may need to apply the restraints. Additionally, the crew member
labor organization official stated since pilots in command are the
security coordinators on flights, they must be familiar with the
strategies, tactics, and techniques that flight attendants may use in
defense of themselves, the passengers, and the aircraft.
Additionally, some stakeholders expressed concerns about the difficulty
in implementing some of the standards without additional information or
training tools from TSA. For example, officials from 12 of the 19 air
carriers we interviewed stated that TSA had not provided sufficient
training materials or tools to enable them to deliver certain elements
of the security training. These air carriers stated that although they
requested the additional information or tools, TSA responded that air
carriers were responsible for identifying and providing the required
tools needed to deliver the security training. A labor union
organization official stated that relying on training organizations and
air carriers to develop the training materials "perpetuates the
disparate quality and breadth of training available throughout the
industry, which does little to assure a common strategy approach to
securing United States skies." Additionally, officials from 4 of the 19
air carriers we interviewed expressed concerns that TSA did not take
into consideration that some air carriers do not have the expertise and
personnel to conduct the annual basic self-defense training. TSA
responded that basic self-defense training is legislatively required
and the Federal Air Marshal Service, FBI, and other agencies are
willing to work with the air carriers on their overall flight and cabin
crew security training. TSA officials further stated that the air
carriers should have an established line of communication with these
agencies, but if the air carriers are seeking a point of contact, TSA
would provide agency contact information. According to a Federal Air
Marshal Service official, a Federal Air Marshal Service liaison meets
periodically with the air carriers and aviation industry associations
representing the air carriers and crew members to discuss overall
communications including flight and cabin crew training issues.
Furthermore, 9 of 27 stakeholders (air carriers, associations
representing air carriers, and crew member labor organizations) we
interviewed were concerned about the lack of definition, guidance, and
clarity for parts of the revised security training guidance and
standards. For example, the crew member security training standards
require that crew members demonstrate proficiency in various security
training elements, such as the use of protective and restraining
devices and proper conduct of a cabin search. However, the standards do
not define proficiency. Officials from a crew member labor organization
stated that without clear, measurable training objectives for
individual air carrier training departments to determine crew member
proficiency and training objectives, the likelihood that training
quality and content will vary from air carrier to air carrier
increases. TSA training officials stated that air carriers, in
conjunction with their training departments, are required to develop a
method for determining crew member proficiency in the required training
elements. TSA officials stated that the air carriers developed the
training program, not TSA, and are therefore in the best position to
define proficiency. TSA officials stated that their training staff's
review of the training materials include verifying that there are
opportunities built into the training for flight and cabin crew members
to demonstrate proficiency in the required elements. TSA officials
further stated that air carriers should have the latitude to tailor
their desired level of proficiency for the various standards to their
individual operations. We found, however, that without standards for
proficiency, which commonly serve as criteria for success in training
programs, TSA will only be able to document training activity, and not
the results of the training, i.e., whether the intended knowledge was
in fact transferred to the training participants at a level acceptable
to TSA.
TSA Has Not Established Strategic Goals or Performance Measures for
Crew Member Security Training:
TSA has not established strategic goals or performance measures for
flight and cabin crew member security training, nor required air
carriers to do so. GPRA requires that agencies use outcome-oriented
goals and measures that assess results, effects, or impacts of a
program or activity compared to its intended purpose. GPRA also
requires federal agencies to consult with key stakeholders--those with
a direct interest in the success of the program--in developing goals
and measures. Strategic goals explain the results that are expected
from a program and when to expect those results. These goals should be
expressed in a manner that could be used to gauge success in the
future. Performance measures (indicators used to gauge performance) are
meant to cover key facets of performance and help decision makers
assess program accomplishments and improve program performance. With
respect to flight and cabin crew security training, strategic goals
would represent the key outcomes that TSA expects air carriers to
achieve in providing flight and cabin crew member security training,
and performance measures would gauge to what extent air carriers are
achieving these outcomes.
TSA training officials stated that they decided not to develop
strategic goals or performance measures because they view their role in
the crew member security training program as purely regulatory--that
is, monitoring air carriers' compliance with the training guidance and
standards established by TSA. In this regard, TSA is the regulatory
agency responsible for determining whether the security training
program is adequately preparing flight and cabin crew members for
potential threat conditions. TSA training officials also stated that
due to the varying nature of the air carriers' training programs, TSA
believes that it is the individual air carriers that are responsible
for establishing goals and performance measures specific to their
security training programs and for using the results to make program
improvements.[Footnote 18] However, without overall strategic goals
established by TSA in collaboration with air carriers, air carriers do
not have a framework from which to develop their individual performance
goals and measures. Furthermore, TSA has not explicitly required air
carriers to develop performance goals and measures or provided them
with guidance and standards for doing so. Without guidance and
standards, the 84 individual air carriers may establish inconsistent
performance goals and measures. Additionally, the absence of
performance goals and measures for flight and cabin crew security
training limits the ability of TSA and air carriers to fully assess the
accomplishments of the flight and cabin crew member security training
and to target program improvements.
Recent Progress Made to Strengthen Oversight of Flight and Cabin Crew
Member Security Training Program, but Weaknesses Remain:
TSA Has Recently Taken Steps to Strengthen Oversight Efforts of Air
Carriers Security Training:
TSA has recently taken steps to improve its oversight of air carriers'
crew member security training. One step includes adding staff with
training expertise to review air carriers' crew member security
training curriculums to determine whether there is evidence that each
applicable training standard is being met. When we began our review,
TSA's review of air carriers' crew member security training programs
was solely the responsibility of the principal security
inspectors.[Footnote 19] These TSA inspectors were responsible for
conducting a regulatory review to determine whether air carriers' crew
member security training curriculums met the requirements set forth in
the standards. Beginning in January 2005, TSA began using training
staff with expertise in designing training programs to review the
overall design of the air carriers' crew member security training
curriculum, how the information is to be conveyed, the expected setting
of the practice environment, and the way in which the information is to
be presented--and to ensure that the security training curriculum
satisfies the required security training standards. TSA inspectors are
responsible for identifying which standards apply to each of the air
carriers, based on their knowledge of the air carrier's flight
operations, size of aircraft, and presence or absence of international
routes. TSA officials stated that between January 2005, when the
revised guidance and standards were issued, and August 2, 2005, the
training staff were involved in the review of the 71 security training
curriculums that had been submitted to TSA.[Footnote 20]
In January 2005, TSA took another step to strengthen its review of air
carriers' flight and cabin crew member security training by developing
a standard form for TSA inspectors and training staff to use to conduct
and document their reviews of air carriers' security training
curriculums. Also, TSA developed an internal memorandum, dated January
5, 2005, that generally describes the review process TSA inspectors and
training staff are to use when reviewing air carriers' crew member
security training curriculums. The standard form, which lists the
required training elements, is used by TSA inspectors to document the
requirements stated in the revised security training standards that
apply to a particular air carrier, and by the training staff to verify
that air carrier's initial and recurrent training plans include the
applicable requirements and to document their comments.[Footnote 21]
Prior to the development of this form, there were no documented
procedures for how the inspections were to be conducted or a standard
form for TSA inspectors to use to document their reviews of air
carriers' crew member security training curriculums. Additionally, TSA
lacked complete documentation of its reviews of air carriers' security
training. Specifically, although TSA officials stated that TSA
inspectors reviewed all 84 air carriers' revised security training
curriculums in response to January 2002 guidance and the corresponding
standards, TSA was only able to provide us documentation related to 11
reviews.[Footnote 22] The Comptroller General's Standards for Internal
Control in the Federal Government states that agencies should document
all transactions and other significant events and should be able to
make this documentation readily available for examination. With the
development of a standard form for reviewing air carriers' security
training curriculums in January 2005, TSA was able to provide us with
documentation for all 18 of the reviews of air carriers' security
training curriculums that TSA inspectors and training staff had
conducted between January 2005 and April 20, 2005.[Footnote 23]
Additionally, in January 2005, TSA began requiring air carriers to
obtain participant feedback at the end of crew member security
training.[Footnote 24] According to our human capital work,[Footnote
25] participant feedback can be useful in providing the agency with
varied perspectives on the effect of the training. However, TSA
training officials stated that they are not certain how, if at all,
they will use the participant feedback in conducting oversight of air
carriers' crew member security training programs. TSA officials stated
that it is the responsibility of the individual air carriers to assess
the results of participant feedback and to make changes to improve the
security training as necessary. In May 2005, TSA training officials
acknowledged that it would be useful for its inspectors to review
participant feedback on an annual basis to assess flight and cabin crew
members' views of their air carriers' security training programs and to
identify trends within and across air carriers. The official
acknowledged that these results could provide TSA inspectors
information they could use to prioritize their reviews of air carriers'
crew member security training. However, the official stated that
reviewing the participant feedback is a resource intensive process that
also requires a certain level of expertise and is not feasible for TSA
to undertake at this time. Without plans for reviewing participant
feedback, TSA is not making use of available information on possible
deficiencies in the quality of air carriers' security training programs
or identifying best practices that could be shared.
Furthermore, TSA is taking steps to address a staffing shortage among
its TSA inspector workforce to enable greater monitoring of air
carriers' flight and cabin crew member security training. Specifically,
on April 1, 2005, TSA reorganized its inspection staff into a newly
created Office of Compliance.[Footnote 26] TSA officials stated that
this reorganization should help address the staffing shortfalls that
previously existed. TSA also issued position announcements in an effort
to fill vacant inspector positions. TSA officials stated that they had
about 23 TSA inspectors onboard when the inspection function
transferred from FAA to TSA in February 2002. As of February 2005, TSA
had 15 inspectors onboard, 5 of whom were in the position for less than
5 months. Between January 2004 and September 2004, the TSA inspector
workforce ranged from about 7 to 14 inspectors. TSA officials stated
that a number of these staff subsequently left TSA because of
advancement opportunities within the Department of Homeland Security
and personal reasons. As part of TSA's monitoring efforts, TSA
inspectors periodically visit air carriers to observe classroom
delivery of flight and cabin crew member security training and to
review air carrier records documenting flight and cabin crew member
completion of required security training. TSA officials stated that
with the existing inspector workforce, they were only able to conduct
observations of about 25 of air carriers' classroom delivery of flight
and cabin crew member security training during fiscal year
2004.[Footnote 27] Although TSA is not required to observe the
classroom delivery of all air carriers' flight and cabin crew member
security training on an annual basis, TSA officials stated that these
observations allow them to determine whether security training is being
delivered consistent with air carriers' approved security training
curriculums and to identify potential problems with the training
delivery.
TSA Has Not Fully Developed Procedures for Monitoring Air Carriers'
Crew Member Security Training:
While TSA has taken steps to strengthen its oversight of air carriers'
crew member security training, TSA has not fully developed procedures
for monitoring this training. TSA is required by law to monitor and
periodically review air carriers' security training to ensure that the
training is adequately preparing crew members for potential threat
conditions. The Comptroller General's Standards for Internal Control in
the Federal Government calls for controls generally to be designed to
assure that ongoing monitoring occurs during the course of normal
operations and that transactions and other significant events be
documented clearly and the documentation be readily available for
examination.[Footnote 28] We identified weaknesses in TSA's controls in
these areas with regard to monitoring and reviewing air carriers'
flight and cabin crew security training.
First, although TSA recently developed a standard form for its
inspectors and training staff to use in reviewing air carriers' flight
and cabin crew member security training, TSA has not developed
procedures for completing this form. TSA officials acknowledged that
there are no documented procedures or criteria for staff to use to
complete the standard form or for determining which standards apply to
individual air carriers and whether or not to approve an air carrier's
security training curriculum.[Footnote 29] The lack of written
procedures may result in inconsistent assessments of the air carriers'
security training curriculums and inconsistent application of the
standards to air carriers. Formal procedures for reviewing air
carriers' flight and cabin crew security training could provide
standardization when TSA inspectors and training staff assess the air
carriers' security training curriculum.
Second, TSA does not have documented procedures for conducting and
documenting observations of air carriers' classroom delivery of flight
and cabin crew member security training. During fiscal year 2004,
according to TSA officials, TSA inspectors visited about 25 air
carriers to observe crew member security training and review files,
such as records documenting crew members' completion of required
security training.[Footnote 30] TSA officials stated that they did not
have sufficient resources to visit all 84 air carriers to observe their
security training. We requested records documenting TSA inspectors'
visits to air carriers to assess the completeness and consistency of
these reviews. However, TSA officials stated that they were unable to
provide us with the requested documentation. Without written procedures
to guide TSA inspectors in observing security training and assessing
the results of their observations, its inspectors may not conduct
comprehensive and consistent reviews. Additionally, without a mechanism
for documenting and maintaining TSA inspectors' reviews of air
carriers' security training delivery in a standard format, TSA lacks
the ability to track the results of these reviews and identify
patterns, including strengths and weaknesses, in training delivery
within and across air carriers. In June 2005, a TSA official stated
that TSA inspectors will monitor at least one flight and cabin crew
member training class per year to ensure the curriculum is being
followed. TSA inspectors are to provide the results of the monitoring
to the principal operations inspector via memo or email. TSA officials
stated that inspections of monitoring crew member security training
will be maintained in a database, but TSA has not established a time
frame for completing this database or documented procedures for this
process.
Additionally, although the law requires TSA to consider complaints from
crew members in determining when to review air carriers' flight and
cabin crew member security training programs, TSA does not have
procedures for considering such complaints. TSA inspection officials
stated that they were not aware of any instances in which crew members
had complained to TSA about security. However, in the event that TSA
does receive complaints from crew members in the future, it is
important that TSA have established and documented procedures to inform
its inspectors of how to consider the complaints in reviewing air
carriers' security training programs. TSA officials stated that
complaints from flight and cabin crew members will be directed to their
Office of Transportation Security Policy for review and all decisions
regarding flight and cabin crew member security training program
modifications or policy changes will be evaluated and disseminated by
this office. The officials also stated that if the complaints involve
the training delivery process, the TSA inspectors may be required to
increase the frequency of on-site inspections based upon an evaluation
of the seriousness of the complaints that are received.
TSA officials stated that they plan to develop a handbook for its
inspectors and guidance for its training staff to use in monitoring and
reviewing air carriers' flight and cabin crew member security training
to help provide assurance that standardized monitoring occurs. However,
TSA has not established a time frame for completing these efforts.
TSA Developed and Implemented Crew Member Self-Defense Training, but
Has Not Established Performance Measures or Evaluated Training
Effectiveness:
In December 2004, as required by law, TSA implemented an advanced
voluntary crew member self-defense training program for flight and
cabin crew members after obtaining stakeholder input.[Footnote 31]
Participation in the voluntary training course has been relatively low,
with only 474 flight and cabin crew members (39 percent of total
capacity) attending the training during the first 7 months of the
program. TSA training officials attributed the low participation to
crew members having a difficult time in obtaining 3 consecutive days of
leave to enable them to participate in the training. Additionally,
although TSA incorporated some stakeholder concerns into the course
design, some stakeholders, including individuals identified as experts
by TSA and our own analysis identified concerns regarding the training
design and delivery, including the training's voluntary nature, the
setting's lack of realism, the training's lack of recurrence, and the
instructor's lack of knowledge of crew members' actual work
environment. TSA has not developed performance measures for the program
or established a time frame for evaluating the program's overall
effectiveness, including the effectiveness of the training design and
delivery.
TSA Developed and Implemented Advanced Voluntary Self-Defense Training
in Consultation with Stakeholders, but Participation in Training
Remains Low:
TSA developed and implemented an advanced voluntary self-defense
training program for flight and cabin crew members in consultation with
key stakeholders by December 12, 2004, as required by law. TSA
consulted with law enforcement personnel, security experts with
expertise in self-defense training, representatives of air carriers,
flight attendants, labor organizations representing flight attendants,
terrorism experts, Federal Air Marshal Service officials, and
educational institutions offering law enforcement training programs, in
developing the self-defense training program and determining how to
apply the training elements specified by law. [Footnote 32] According
to TSA officials, in 2002, in anticipation of having to develop a
mandatory self-defense training program as required by the Homeland
Security Act,[Footnote 33] TSA established a working group comprised of
law enforcement experts, Federal Air Marshals,[Footnote 34] and other
subject matter experts, such as aviation security experts and self-
defense/martial arts training experts, to assess what elements should
be included in the training. This working group collaborated on the
overall program design and delivery, including the program goals and
objectives and the course content and delivery method. The working
group's efforts were placed on hold in 2003 when TSA was advised that
legislation would be enacted to make the training a voluntary program
to be provided by TSA, rather than a mandatory training program to be
delivered by individual air carriers. After the enactment of Vision 100
in December 2003, TSA continued its efforts to develop an advanced
voluntary self-defense training program until the program's official
implementation in December 2004, building on the input of the initial
working group. The overall goal of the advanced voluntary crew member
self-defense training, as defined by TSA, is to enable crew members to
develop a higher level of competency in self-defense tactics to prevent
or reduce the possibility of injury or death to one's person or the
takeover of an aircraft. TSA also established several objectives for
the training, including:
* recognize potential threats before an act of violence occurs;
* interpret behaviors that lead to potential hostile acts;
* conclude appropriate courses of action crew members must take to
avert hostile actions intended to injure crew members or passengers or
to take over an aircraft; and:
* apply appropriate individual self-protection measures and self-
defense tactics to prevent or reduce the possibility of injury or death
to one's person or the takeover of an aircraft.
Prior to implementing the voluntary training in December 2004, TSA
piloted the prototype advanced voluntary self-defense training in
August and September 2004 in five cities with major airline hubs and
refined the training based on comments from participants.[Footnote 35]
The participants provided positive feedback in four areas, including
(1) the repetitive moves taught throughout the course made the self-
defense tactics easy to learn; (2) the training prepared them mentally
and physically to defend themselves and provided a good foundation in
self-defense; (3) the small class size and instructor to student ratio
of 1 to 8 was conducive to a productive learning environment;[Footnote
36] and (4) the location of the training facility and lodging was well
received. TSA also received feedback on changes that could be made to
enhance the training. Table 3 provides a summary of the stakeholders'
concerns on TSA's prototype advanced voluntary self-defense training
and actions taken by TSA in response to the concerns.
Table 3: Summary of Stakeholders' Concerns on TSA's Prototype Advanced
Voluntary Self-Defense Training and Actions Taken by TSA in Response to
the Concerns:
Comments: Training course was too long;
Action taken by TSA in response to stakeholder concerns: Decreased the
number of training days from 4 to 3.
Comments: Too much lecture time;
Action taken by TSA in response to stakeholder concerns: Increased the
amount of hands-on training from about 70 percent to 85 percent of the
total course time.
Comments: Lack of aircraft cabin or cockpit to promote realistic
training scenarios;
Action taken by TSA in response to stakeholder concerns: Encouraged
instructors to attempt to simulate the aircraft cabin through creative
means, such as creating an aisle using chairs and having participants
perform the tactics within the confines of the chairs.
Comments: Some instructors were not knowledgeable in aviation
terminology, and aircraft environment, or the airline industry;
Action taken by TSA in response to stakeholder concerns: Advised the
contractor to consider hiring instructors with flight attendant and
pilot experience.
Comments: Lack of communication in getting the notice out about the
class scheduling;
Action taken by TSA in response to stakeholder concerns: Committed to
providing more advance notice in announcing the official training.
Comments: Lack of information on recurrent training;
Action taken by TSA in response to stakeholder concerns: Although TSA
has not ruled out recurrent training in the future, TSA is currently
not planning to conduct such training because the law only requires TSA
to provide the advanced voluntary crew member self-defense training.
Under the basic flight and cabin crew member security training program,
air carriers are to provide instructions on basic defensive techniques.
Comments: Crew members' exposure to potential liability if a crew
member hurts someone when using techniques taught in the self-defense
training to address an incident onboard an aircraft;
Action taken by TSA in response to stakeholder concerns: TSA advises
crew members participating in the advanced voluntary crew member self-
defense training program of the liability protection offered by 49
U.S.C. § 44903(k), which shields individuals, including crew members,
from liability in the event they take action to defend themselves or
others based upon a reasonable belief that an act of criminal violence
or aircraft piracy was occurring or about to occur on board the
aircraft.
Source: GAO analysis of interviews with stakeholders.
[End of table]
As of June 2005, a total of 474 crew members had participated in the
training in 51 classes. During the initial deployment of the advanced
voluntary crew member self-defense training in December 2004,
participation was only about 14 percent of the total capacity being
utilized, and about 38 percent of total enrolled participants actually
attended the training course. Participation increased in January
through March 2005, but declined in April through June 2005, with only
23 percent of total capacity utilized in June 2005. TSA plans to offer
46 additional advanced voluntary self-defense training courses during
the remainder of fiscal year 2005 in 10 cities.[Footnote 37] According
to TSA officials, TSA estimated that approximately 21,700 crew members
will participate in the training, based on information obtained from
air carrier associations and crew member labor organizations. Table 4
provides information on flight and cabin crew member participation in
the advanced voluntary self-defense training from December 2004 through
June 2005.[Footnote 38]
Table 4: Crew Member Participation in Advanced Voluntary Crew Member
Self-Defense Training from December 2004 through June 2005:
Month: December 2004;
Number of classes in which training took place: 4;
Class capacity: 96;
Number enrolled: 34;
Percentage of capacity enrolled: 35%;
Number attended: 13;
Percentage of enrolled attended: 38%;
Percentage of total capacity attended: 14%.
Month: January 2005;
Number of classes in which training took place: 8;
Class capacity: 192;
Number enrolled: 115;
Percentage of capacity enrolled: 60%;
Number attended: 75;
Percentage of enrolled attended: 65%;
Percentage of total capacity attended: 39%.
Month: February 2005;
Number of classes in which training took place: 10;
Class capacity: 240;
Number enrolled: 159;
Percentage of capacity enrolled: 66%;
Number attended: 144;
Percentage of enrolled attended: 91%;
Percentage of total capacity attended: 60%.
Month: March 2005;
Number of classes in which training took place: 10;
Class capacity: 240;
Number enrolled: 135;
Percentage of capacity enrolled: 56%;
Number attended: 123;
Percentage of enrolled attended: 91%;
Percentage of total capacity attended: 51%.
Month: April 2005;
Number of classes in which training took place: 7;
Class capacity: 168;
Number enrolled: 63;
Percentage of capacity enrolled: 38%;
Number attended: 49;
Percentage of enrolled attended: 78%;
Percentage of total capacity attended: 29%.
Month: May 2005;
Number of classes in which training took place: 8;
Class capacity: 192;
Number enrolled: 61;
Percentage of capacity enrolled: 32%;
Number attended: 48;
Percentage of enrolled attended: 79%;
Percentage of total capacity attended: 25%.
Month: June 2005;
Number of classes in which training took place: 4;
Class capacity: 96;
Number enrolled: 41;
Percentage of capacity enrolled: 43%;
Number attended: 22;
Percentage of enrolled attended: 54%;
Percentage of total capacity attended: 23%.
Total;
Number of classes in which training took place: 51;
Class capacity: 1224;
Number enrolled: 608;
Percentage of capacity enrolled: 50%;
Number attended: 474;
Percentage of enrolled attended: 78%;
Percentage of total capacity attended: 39%.
Source: GAO analysis of TSA data.
[End of table]
TSA officials stated that the low participation rate in December 2004
was largely due to the short advance notice they provided stakeholders
regarding the training. TSA announced the availability of the December
2004 training courses 6 days before the training was to take place.
This short notice significantly restricted the ability of flight and
cabin crew members to participate in the training because they
generally submit their scheduling bids 30 days prior to their work
schedule.[Footnote 39] TSA attributed the short notice provided to the
length of time it took to reallocate funds from other TSA programs to
enable implementation of the advanced voluntary self-defense training
program.[Footnote 40] TSA provided crew members with more than 30 days
notice for the remaining sessions. However, TSA training officials
stated that participation continued to remain low in January 2005 due
to crew members calling in sick, crew members being called for flight
duty at the last moment, and inclement weather. TSA training officials
stated that although they projected that crew member participation
would increase over time, crew member participation decreased in June
2005 due to crew members having a difficult time obtaining 3
consecutive days of leave to attend training. TSA training officials
also stated that based on their experience with the Federal Flight Deck
Officer training program, crew members' ability to obtain leave for the
purpose of attending training may be associated with seasonal
variances, with low participation usually occurring during the spring
and summer months. Stakeholders, including air carriers and crew member
labor organizations, attributed the low participation to crew members
having to attend the training on their own time and pay the cost of
travel, lodging, and meals. TSA training officials stated that they
were continuing to gather information from flight and cabin crew
members through the training contractor in an effort to identify the
causes for the low participation and, ultimately, to try to address
these causes.
Stakeholders Identified Concerns about the Self-Defense Training Design
and Delivery:
Stakeholders, including individuals that TSA identified as subject
matter experts, and our own analysis identified concerns with the
design and delivery of the advanced voluntary crew member self-defense
training.[Footnote 41] These concerns include the training's voluntary
nature, the setting's lack of realism, the training's lack of
recurrence, and the instructor's lack of knowledge of crew members'
actual work environment. These same concerns were identified by
stakeholders in response to the prototype self-defense training. As
previously stated, our prior human capital work has found that in
implementing a training program, an agency should ensure that
implementation involves effective and efficient delivery of training--
that is, the training should be conducted in a setting that
approximates the participants' working conditions and taught by
individuals who are knowledgeable about the subject matter and work
environment.
In the case of the advanced voluntary self-defense training program, 13
of the 33 stakeholders expressed concerns about the voluntary nature of
the training and stated that the training should be mandatory. Six of
these 13 stakeholders stated that the program's voluntary nature is
inconsistent with TSA's revised security training guidance that seeks
to establish a common strategy that would enable individuals involved
in an incident onboard an aircraft to know what others involved will be
thinking and doing. These same stakeholders stated that because the
training is not mandatory, if some crew members have had the self-
defense training while others have not, a breakdown in communication
could occur. TSA training officials stated that because the security
training standards require all crew members to receive training on how
to communicate and coordinate during a disturbance, they are not
concerned about the voluntary nature of the self-defense training
program.
Additionally, 14 of the 33 stakeholders expressed concerns about the
lack of a realistic training setting during the delivery of advanced
voluntary crew member self-defense training. The self-defense
techniques are taught in an open-space setting, unlike the narrow
aisles crew members have to work within on an actual aircraft. In the
two training sessions we observed in two different cities, participants
had to be constantly reminded by the instructors of the restricted
training space because participants repeatedly made defensive moves,
such as spins and wide kicks, which could not be performed inside an
aircraft cabin. Our prior human capital work has found that for
learning to be successful, the training environment--training facility
and equipment--should be favorable to successful learning.[Footnote 42]
TSA officials stated they examined the possibility of purchasing
aircraft simulators for the self-defense training and found that it
would cost TSA about $100,000 per simulator. Officials stated that they
have advised the instructors to try to create a setting, using chairs,
tape, or other means, to simulate the narrow aisles on an aircraft. We
informed TSA that in the two training sessions we attended, instructors
did not use these techniques. TSA officials stated that they would
follow up with the instructors to ensure they use these techniques.
Fifteen of the 33 stakeholders also expressed concerns about the lack
of recurrent self-defense training given that self-defense skills are
difficult to sustain if not consistently practiced over time.
Stakeholders stated that a 3-day, one-time self-defense training course
would not enable crew members to develop proficiency in self-defense.
TSA officials responsible for developing the advanced voluntary
training program stated that the self-defense training is not intended
to make participants proficient in self-defense. Rather, the training
is intended to enable crew members to develop a higher level of
competency in self-defense tactics by extending their knowledge and
skills in the use of self-defense techniques and improvised
weapons.[Footnote 43] TSA officials also stated that the key benefit of
the training is a change in the mindset of participants that enables a
greater awareness of threat conditions onboard an aircraft and in their
daily lives. Additionally, they stated that it is the responsibility of
the individual participants to practice the various self-defense
techniques they were taught. Furthermore, they stated that although TSA
is not currently offering a recurrent training program, it has not
ruled out the possibility of recurrent training in the future. Eleven
of the 33 stakeholders also expressed concerns that the self-defense
training could give participants a false sense of security. For
example, two stakeholders stated that the false sense of security lies
in the participants who take the course once and expect to be skilled
and proficient using the self-defense techniques without realizing that
they may not be capable of following through when an incident occurs.
Finally, 6 of the 33 stakeholders, including subject matter experts and
crew member labor organizations, identified concerns about the lack of
knowledge instructors had about the crew members' actual working
environment. While some stakeholders commended the instructors for
their technical knowledge of crew members' actual work environment,
others expressed concerns that some instructors lacked technical
knowledge and expertise of the aviation industry. For example, a
training participant we interviewed stated that the instructor did not
understand how safety devices onboard an aircraft operate. The
instructor suggested inflating an emergency raft while inflight to
protect the flight deck. However, according to the training
participant, inflating an emergency raft in flight could injure or kill
passengers and crew members. Our prior human capital work found that
the use of instructors who are knowledgeable of the subject matter and
experienced in aviation industry issues can help provide assurance that
instructors can effectively transfer these skills and knowledge to
others. TSA officials stated that they advised the training contractor
to hire instructors with law enforcement, martial arts/self-defense,
and aviation backgrounds. Additionally, TSA provided the instructors
with training on aviation terminology so instructors could better
communicate with the students throughout the course.
TSA officials stated that they were aware of the stakeholder concerns
regarding the self-defense training course. The officials stated that
their ability to address these concerns is limited by funding
constraints and competing priorities. TSA officials further stated that
they will continue to work with the contractor that is delivering the
training to obtain any information that would be beneficial to the
design and implementation of the training program.
TSA Has Not Established Performance Measures or a Time Frame for
Evaluating Training Effectiveness:
TSA has not yet developed performance measures for the advanced
voluntary crew member self-defense training program or established a
time frame for evaluating the program's overall effectiveness. Our
prior human capital work on best practices in training has found that
generally, high-performing organizations evaluate the effectiveness of
their training programs and use the results to target performance
improvements.[Footnote 44] In February 2005, TSA began conducting end-
of-course evaluations--participant feedback--of the training and is
planning to assess these evaluations to ensure the training is
consistently achieving results over time. Additionally, TSA will use
the results to modify the training, if appropriate. Although these
evaluations should enable TSA to assess participants' views on the
training facilities, materials, and instructors, they will not enable
TSA to determine whether the training increased the participants'
knowledge and skills. TSA officials stated that they recognize the
importance of developing performance measures and evaluating the
effectiveness of the program to ensure that it is consistently
achieving the goals and target performance improvements. Although TSA
plans to undertake these efforts, it has not established time frames
for doing so. TSA officials stated that the numerous internal process
improvements currently under way in TSA that compete for time and
resources will affect how soon the agency can establish performance
measures and conduct an evaluation of the training program. Without
performance measures and an evaluation of the program's overall
effectiveness, TSA will not have meaningful information with which to
determine whether the training program is actually enabling crew
members to develop a higher level of competency in self-defense
tactics--the intended goal of the training program.
Conclusions:
It has been less than 4 years since TSA assumed responsibility for
aviation security. During this period, TSA implemented numerous
initiatives to strengthen the various layers of security in commercial
aviation. These efforts have largely focused on passenger and checked-
baggage screening--among the first lines of defense in preventing
terrorist attacks on commercial aircraft. TSA has recently taken steps
to ensure that flight and cabin crew members--the last line of defense-
-are prepared to handle potential threat conditions onboard commercial
aircraft. The revised guidance and standards TSA developed for air
carriers to follow in developing and delivering their flight and cabin
crew member security training is a positive step forward in
strengthening security onboard commercial aircraft. However, guidance
and standards alone do not provide assurances that the training
delivered by air carriers is achieving TSA's intended results. TSA
views its role in flight and cabin crew member security training as
regulatory, and that air carriers are responsible for measuring the
success of their individual training programs. We agree that air
carriers have responsibility for assessing the effectiveness of their
training programs. However, we believe that overall responsibility for
ensuring that flight and cabin crew members are prepared to respond to
terrorist threats must be shared between the air carriers and TSA. In
supporting this partnership, TSA should establish strategic goals for
the flight and cabin crew security training program so that air
carriers can develop their security programs, and measure the
effectiveness of these programs, based on desired results, or goals,
clearly defined by TSA. Without strategic goals to inform air carriers
of what is expected from their training programs, and in the absence of
guidance and standards to help ensure that air carriers establish
consistent performance goals and measures, it will be difficult for TSA
and the air carriers to gauge the success of training programs over
time and to determine how to direct improvement efforts most
effectively.
Additionally, while we are encouraged by the recent steps TSA has taken
to improve its monitoring and review of air carriers' security training
programs, without enhanced controls, such as written procedures for TSA
staff to follow in conducting and documenting their reviews, TSA lacks
reasonable assurance that its monitoring and review efforts will be
conducted in a consistent and complete manner. Furthermore, a key
source of information on the effectiveness of air carriers' security
training is participant feedback on the training. TSA's recent
requirement that air carriers obtain written feedback from flight and
cabin crew members at the end of security training is a step in the
right direction. However, without a process in place for considering
this information during its oversight efforts, TSA is not effectively
utilizing available information that could assist it in prioritizing
and focusing its monitoring and review activities.
Through developing and implementing the advanced voluntary self-defense
training program, TSA took another step forward in its efforts to
prepare flight and cabin crew members to handle potential threat
conditions onboard commercial aircraft. However, TSA has not yet
established performance measures or a timeframe for evaluating the
effectiveness of the training program, including the training design
and delivery. Congress enacted the Government Performance and Results
Act of 1993 to focus the federal government on achieving results and
providing objective, results-oriented information to improve
congressional decision making. Without performance measures or a method
for evaluating the effectiveness of the training, TSA may not have
information with which to systematically assess the program's
strengths, weaknesses, and performance. Performance measures and an
evaluation of the program's effectiveness can assist TSA in focusing
its improvement efforts and provide Congress with information to assess
the impact of an advanced voluntary self-defense training program.
Recommendations:
To help provide TSA management with reasonable assurance that its
security training guidance and standards for flight and cabin crew
members are preparing crew members for potential threat conditions, and
to enable TSA and air carriers to assess the accomplishments of the
security training and target program improvements, we recommend that
the Secretary of the Department of Homeland Security direct the
Assistant Secretary, Transportation Security Administration, to take
the following three actions:
* establish strategic goals for the flight and cabin crew member
security training program, in collaboration with air carriers, and
communicate these goals to air carriers to explain the results that are
expected from the training;
* develop guidance and standards for air carriers to use in
establishing performance goals and measures for their individual flight
and cabin crew member security training programs to help ensure
consistency in the development of goals and measures; and:
* review air carriers' goals and measures as part of its monitoring
efforts to help ensure that they are linked to strategic goals
established by TSA and to assess whether the training programs are
achieving their intended results.
To strengthen TSA's internal controls and help ensure that air carriers
are complying with TSA's guidance and standards, we also recommend that
the Assistant Secretary, Transportation Security Administration,
establish a time frame for finalizing written procedures for monitoring
and reviewing air carriers' flight and cabin crew security training.
These procedures should address the process for completing flight and
cabin crew member curriculum review forms, determining which standards
apply to individual air carriers and whether or not to approve an air
carrier's training curriculum, conducting and documenting observations
of air carriers' classroom delivery of security training, reviewing air
carriers' security training goals and measures, and considering
security related complaints from flight and cabin crew members. As part
of its efforts to develop written procedures, TSA should examine ways
to incorporate participant feedback into its monitoring and review
efforts.
In addition, to help ensure that the advanced voluntary crew member
self-defense training is achieving its intended results, we recommend
that the Assistant Secretary, Transportation Security Administration,
establish performance measures for the advanced voluntary crew member
self-defense training program and a time frame for evaluating the
effectiveness of the training, including the effectiveness of the
training design and delivery.
Agency Comments and Our Evaluation:
We provided a draft of this report to DHS for review and comment. On
August 29, 2005, we received written comments on the draft report,
which are reproduced in full in appendix III. DHS generally concurred
with the findings and recommendations in the report, and agreed that
efforts to implement our recommendations are critical to a successful
flight and cabin crew member security training program. With regard to
our recommendations that TSA establish strategic goals for the flight
and cabin crew member security training program and develop guidance
and standards for air carriers to use in establishing performance goals
and measures for their individual flight and cabin crew member security
training programs, DHS stated that TSA has begun efforts to establish
strategic goals for the program and, air carriers would benefit from
additional guidance--that is, guidance in addition to the flight and
cabin crew security training standards--to use in establishing
performance goals and measures for their individual flight and cabin
crew security training programs. While TSA has established standards
for air carriers' to use in developing their flight and cabin crew
security training, these standards do not include strategic goals for
the training nor provide any guidance for establishing performance
goals and measures. In addition, at the time of our review, TSA had not
begun developing strategic goals for flight and cabin crew security
training. Therefore, we cannot assess the extent to which the goals TSA
is currently developing satisfy our recommendation. With respect to our
recommendation that TSA establish a time frame for finalizing written
procedures for monitoring and reviewing air carriers' flight and cabin
crew security training, DHS stated that TSA is in the process of
developing a monitoring plan, to the extent that resources permit, and
a handbook for reviewing air carriers' flight and cabin crew member
security training programs. DHS further stated that the handbook is
currently under development and will be completed and ready for
implementation in fiscal year 2006. Finally, regarding our
recommendation that TSA establish performance measures for the advanced
voluntary crew member self-defense training program and a time frame
for evaluating the effectiveness of the training, DHS stated that TSA
is working with OMB to establish performance measures for use in OMB's
Performance Assessment Rating Tool for flight security training and
will finalize these measures using fiscal year 2005 data as the
baseline. According to DHS, these measures will provide TSA with
information that can be used in evaluating the effectiveness of the
advanced voluntary crew member self-defense training. DHS also stated
that TSA has begun to reach out to stakeholders to obtain feedback on
this training. TSA's successful completion of these ongoing and planned
activities should address the concerns we raised in this report. We
also provided relevant sections of this report to FAA, FBI, and DOD for
their review, and incorporated their technical comments into the report
as appropriate.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 2 days
from the date of this report. At that time, we will send copies of this
report to the Secretary of the Department of Homeland Security and the
Administrator of the Transportation Security Administration and
interested congressional committees. We will also make copies available
to others upon request. In addition, the report will be made available
at no charge on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please
contact me at (202) 512-8777 or berrickc@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major contributions
to this report are listed in appendix IV.
Signed by:
Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To determine the progress TSA has made in developing and monitoring
flight and cabin crew security training, we examined TSA's efforts to
develop guidance and standards for air carriers' flight and cabin crew
security training, monitor air carriers' compliance with the guidance
and standards, and develop and deliver advanced voluntary self-defense
training for crew members. Specifically, this report addresses the
following questions: (1) What actions has TSA taken to develop guidance
and standards for flight and cabin crew security training and to
measure the effectiveness of the training? (2) How does TSA ensure
domestic air carriers comply with required training guidance and
standards? (3) What efforts has TSA taken to develop, implement, and
measure the effectiveness of advanced voluntary self-defense training
for flight and cabin crew members?
To determine the actions TSA has taken to develop guidance and
standards for flight and cabin crew security training and to measure
the effectiveness of the training as well as how TSA ensures domestic
air carriers comply with required training guidance and standards, we
obtained and analyzed relevant legislation, guidance, and standards
developed by TSA and FAA, and TSA records documenting its reviews of
air carriers' security training programs. We reviewed the security
training guidance and standards to determine whether they contained the
statutory requirements for flight and cabin crew security training of
49 U.S.C. § 44918, as established by the Aviation and Transportation
Security Act, and as amended by the Homeland Security Act of 2002 and
Vision 100--Century of Aviation Reauthorization Act. We also
interviewed TSA training, policy, and inspections officials to identify
their reasons for revising the guidance and standards and the process
they used to revise these documents and ensure air carriers' compliance
with the guidance and standards. Additionally, we compared TSA's
process for monitoring and reviewing air carrier compliance with flight
and cabin crew security training guidance and standards to standards
for internal control in the federal government. To assess stakeholder
involvement in the development of the guidance and standards and
identify any stakeholder concerns, we interviewed officials from the
FAA, FBI, FAMS, DOD, crew member labor organizations, and associations
representing air carriers. At eight domestic air carriers we visited,
we interviewed air carrier officials to obtain their views on the
security training guidance and standards and TSA's efforts to ensure
air carriers' compliance with the guidance and standards and to observe
the flight and cabin crew initial or recurrent (refresher) security
training. We selected these domestic air carriers based on whether they
were currently offering initial and/or recurrent security training and
on the size of the air carrier in an effort to include a mixture of
various domestic air carriers and air carriers of varying sizes. The
size of an air carrier is based on the annual operating revenues and
the number of revenue passenger boardings. Finally, we conducted phone
interviews with representatives of 11 additional domestic air carriers-
-which we selected using the same criteria we used to select the 8 air
carriers to visit--to obtain their views on the flight and cabin crew
member guidance and standards and TSA's monitoring of air carriers'
compliance with these standards. Because we selected a nonprobability
sample of domestic air carriers, the information we obtained from these
interviews and visits cannot be generalized to all domestic air
carriers.
To determine the efforts TSA has taken to develop, implement, and
measure the effectiveness of advanced voluntary self-defense training
for flight and cabin crew members, we obtained and analyzed relevant
legislation, TSA's course training manual for the self-defense
training, and feedback provided by flight and cabin crew members who
participated in the prototype training. We also interviewed TSA
training officials responsible for designing and implementing the
voluntary advanced crew member self-defense training program.
Additionally, we observed the final training in two cities.
Furthermore, we interviewed relevant stakeholders, including
representatives of air carriers; labor organizations representing
flight attendants and pilots as well as individual flight attendants
and pilots; aviation industry associations representing air carriers;
individuals identified as subject matter experts or self-defense
training experts; and federal officials at the FBI, FAMS, and FAA to
determine whether TSA consulted them when developing the crew member
self-defense training. We identified subject matter experts or self-
defense training experts based on recommendations from TSA and crew
member labor organizations. We also interviewed representatives of the
19 domestic air carriers mentioned above to obtain their views on the
design and delivery of the advanced voluntary crew member self-defense
training. We assessed the extent to which TSA incorporated stakeholder
input into the training program, and the basis for TSA's decisions on
which stakeholder input to incorporate into the training. Finally, we
assessed TSA's efforts to develop the training programs relative to our
guidance for assessing training and development efforts in the federal
government.
We conducted our work from June 2004 through August 2005 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: TSA's Deployment of an Advanced Voluntary Crew Member Self-
Defense Training from December 2004 through September 2005, as of July
2005:
Note: c = cancelled class. Crew member self-defense training was
cancelled due to low participation.
Training location: Alexandria, Virginia;
December 2004;
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: August;
2005: September.
Training location: Miami, Florida;
December 2004;
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Chicago, Illinois;
December 2004: [C];
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Fort Worth, Texas;
December 2004;
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Wilmington, California;
December 2004;
2005: January;
2005: February;
2005: March;
2005: April: [C];
2005: May: [C];
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Aurora, Colorado;
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Broomall, Pennsylvania;
2005: January: [C];
2005: February;
2005: March;
2005: April: [C];
2005: May: [C];
2005: June: [C];
2005: July: [C];
2005: August;
2005: September: No class currently scheduled.
Training location: Decatur/Clarkston, Georgia;
2005: January;
2005: February;
2005: March;
2005: April: [C];
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: San Francisco, California;
2005: January;
2005: February;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Training location: Tempe, Arizona;
2005: March;
2005: April;
2005: May;
2005: June: [C];
2005: July: [C];
2005: August;
2005: September.
Source: GAO analysis of TSA data.
[End of table]
[End of section]
Appendix III: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
August 26, 2005:
Ms. Cathleen A. Berrick:
Director, Homeland Security & Justice Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, D.C. 20548:
Dear Ms. Berrick:
Thank you for the opportunity to comment on draft report GAO-05-781 SU,
Aviation Security: Flight and Cabin Crew Member Security Training
Strengthened, but Better Planning and Internal Controls Needed. The
Department of Homeland Security (DHS) appreciates the work done to
identify areas for improvement in the Transportation Security
Administration's (TSA's) review of flight and cabin crew security
training. We generally concur with the report and its recommendations
and appreciate the discussion of challenges, and related on-going and
planned work designed to fully meet our responsibilities.
The report acknowledges the substantial progress TSA has made in
reviewing the cabin crew security training plans of numerous air
carriers, in addition to developing and delivering a voluntary crew
member self-defense training program. However, there are areas within
the report about which DHS would like to comment.
DHS agrees that efforts to implement the U.S. Government Accountability
Office's five recommendations are critical to a successful flight and
cabin crew security program. Strategic goals and objectives provide the
framework guiding the actions that make up the flight and cabin crew
member security training program. TSA has already begun efforts to
establish strategic goals for this program.
While Appendix XIII of the Aircraft Operator Standard Security Program
(AOSSP) outlines the standards for the program, DHS concurs that air
carriers would benefit from additional guidance to use in establishing
performance goals and measures for their individual flight and cabin
crew security training programs.
The TSA Office of Compliance is in the process of developing a
monitoring plan for new and revised Air Carrier Basic Crew Member
Security Programs, to the extent that resources permit. A Principal
Security Inspector's (PSI) handbook, currently under development, will
define specific objectives, required paperwork, PSI assessment
protocols, and guidance for curricula reviews. It is anticipated that
the handbook will be completed and ready for implementation commencing
in FY 2006.
With regard to voluntary crew member self-defense training, TSA's
Office of Workforce Performance and Training is working in concert with
the Office of Management and Budget to establish performance measures
for use in the Performance Assessment Rating Tool (PART) for Flight
Security Training. The program is currently finalizing its performance
measures and will use FY 2005 data as the baseline for those measures.
These measures will provide the program with information that can be
used in evaluating the effectiveness of the crew member self-defense
training. Additionally, TSA has begun efforts to reach out to
stakeholders for feedback. Combined with the compilation of data from
Level 1 evaluations, these two efforts are significant first steps in
gathering valuable information concerning effectiveness, design, and
delivery.
In summary, DHS appreciates your review of the flight and cabin crew
member security training program and thanks you for the thorough
analysis and discussion that comprises this report. We continue to be
cognizant of the areas upon which we can improve.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Director:
Departmental GAO/OIG Liaison Office:
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Cathleen A. Berrick (202) 512-8777:
Acknowledgements:
In addition to the contact named above, Katherine Davis, Kimberly
Gianopoulos, Sally Gilley, Stan Kostyla, Tom Lombardi, Gary Malavenda,
Maria Strudwick, Carol Willett, and Su Jin Yon made key contributions
to this report.
FOOTNOTES
[1] Foreign air carriers operating to and from the United States are
not regulated under the mandates of the Aircraft Operator Security
Program (under which domestic air carriers operate) but are subject to
the security requirements of 49 C.F.R. part 1546. Foreign air carriers
operating under part 1546 must maintain a security program that TSA
accepts as providing passengers a level of protection similar to the
level of protection provided by U.S. air carriers serving the same
airports. See 49 U.S.C. § 44906. Further, foreign air carriers must
abide by the standards and recommended practices established by the
International Civil Aviation Organization, which adopted expanded
security training requirements for crew members in March 2002 that are
similar to those found in the Aviation and Transportation Security Act
(ATSA), Pub. L. No. 107-71, § 107, 115 Stat. 611, 610-11 (2001).
[2] Initial security training is provided to newly hired flight and
cabin crew members as well as crew members returning from a leave of
absence of more than 3 years. Recurrent security training is annual
refresher training provided to all flight and cabin crew members.
[3] 49 U.S.C. § 44918. The Aviation and Transportation Security Act
(ATSA), enacted in November 2001, created the Transportation Security
Administration (TSA) and mandated the Federal Aviation Administration
(FAA) to develop a detailed guidance for a scheduled passenger air
carrier flight and cabin crew member training program. FAA continued to
be responsible for overseeing flight and cabin crew members' security
training until TSA assumed that responsibility pursuant to ATSA.
Subsequently, the Vision 100--Century of Aviation Reauthorization Act
(Vision 100), enacted in December 2003, revised the basic security
training program requirements and mandated that TSA develop and
implement an advanced voluntary self-defense training program for crew
members. Pub. L. No. 108-176, § 603, 117 Stat. 2490, 2563-65. Together,
these acts mandated actions for TSA, FAA, and air carriers to
strengthen flight and cabin crew security training under § 44918,
including the development of security training guidance and standards.
[4] GAO, Internal Control: Standards for Internal Control in the
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November
1999).
[5] The Federal Air Marshal Service provides air security and promotes
public confidence in our nation's civil aviation system through the
deployment of Federal Air Marshals who are specially trained civil
aviation security specialists that are deployed onboard aircraft to
protect passengers, crew, and aircraft from terrorist activities on
both domestic and international flights.
[6] GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, GAO-04-546G (Washington,
D.C.: March 2004).
[7] ATSA created TSA as an agency within the Department of
Transportation (DOT). The Homeland Security Act of 2002, Pub. L. No.
107-296, 116 Stat. 2135, signed into law on November 25, 2002,
transferred TSA from the DOT to the new Department of Homeland
Security. FAA continued to be responsible for overseeing flight and
cabin crew members' security training until TSA assumed that
responsibility pursuant to ATSA. On or about February 17, 2002, TSA
assumed responsibility for flight and cabin crew members' security
training. Initially, however, TSA was neither staffed nor organized to
meet that responsibility on that date. Accordingly, while TSA "ramped
up" its operations FAA's Office of Aviation Security, while being
largely absorbed into TSA, continued to assist TSA in its new roles as
regulator and overseer.
[8] The flight and cabin crew security training implemented by air
carriers pursuant to their security programs, the corresponding
guidance and standards, and the related oversight discussed in this
report are the responsibility of TSA, and not of FAA, which is
responsible for approved training programs required under 14 C.F.R.
part 121.
[9] Pilots (commercial, charter, and all cargo aircraft) who volunteer
to participate in the Federal Flight Deck Officers Program are trained
and armed to protect the aircraft cockpit. The pilots are deputized as
federal flight deck officers.
[10] The Bureau of Transportation Statistics defines major carriers as
those with over $1 billion in annual operating revenues, national
carriers as those with $100 million to $1 billion in annual operating
revenues, and regional (large and medium air carriers) as those with up
to $100 million in annual operating revenues.
[11] TSA currently oversees about 84 air carriers. Bureau of
Transportation Statistics data show that there were 105 commercial
passenger air carriers in fiscal year 2004. TSA officials stated that
this difference of 21 air carriers may be due to air carriers merging
or going out of business.
[12] TSA, however, took no action pursuant to § 1403 of the Homeland
Security Act. TSA maintained the standards prescribed by ATSA until the
passage of Vision 100, which prompted TSA to revise crew member
security training guidance and standards.
[13] GAO-04-546G and GAO, Human Capital: Selected Agencies' Experiences
and Lessons Learned in Designing Training and Development Program, GAO-
04-291 (Washington, D.C.: January 30, 2004).
[14] The additional elements that Vision 100 required air carriers to
include in their basic crew member security training programs are (1)
proper commands to give passenger and attackers, (2) recognizing
suspicious activities, and (3) proper conduct of a cabin search,
including explosive device recognition.
[15] TSA convened a 2-day working session in the summer of 2003 with
representatives from all the associations on a number of topics related
to flight and cabin crew security training. Information collected
during these sessions, and ultimately the legislative elements set
forth in Vision 100, triggered TSA to revise the common strategy and
the standards.
[16] These external stakeholders included representatives of air
carriers, associations representing air carriers (Air Transport
Association and Regional Airline Association), crew member labor
organizations (Association of Flight Attendants and Air Line Pilots
Association), and federal agencies, such as the Federal Aviation
Administration, Federal Air Marshal Service, the Federal Bureau of
Investigation, and the Department of Defense North American Airspace
Defense Command. According to a Federal Air Marshal Service official,
Federal Air Marshal Service officials attended meetings at TSA related
to the flight and cabin crew security training to ensure that the TSA
training was not in conflict with the Federal Air Marshal Service
operations.
[17] TSA officials stated that in the past crew member labor
organizations had not been provided the opportunity to review and
provide comments on the security training standards.
[18] In 2004, the House Appropriations Committee, in its report
accompanying H.R. 4567--the fiscal year 2005 DHS appropriations bill--
directed that TSA expeditiously promulgate basic, performance-based
training requirements for flight attendant security training. In May
2005, the House Appropriations Committee's report accompanying H.R.
2360--the fiscal year 2006 DHS appropriations bill--expressed concern
with TSA's lack of progress in developing minimum regulatory standards
for basic, industrywide flight attendant security training programs and
stated that it expects TSA to report on the status of these performance-
based training requirements no later than July 16, 2006.
[19] For this report, we refer to TSA principal security inspectors as
TSA inspectors.
[20] As of August 2, 2005, TSA officials stated that TSA received 73
air carriers' security training curriculums. Of the 73, 71 curriculums
were reviewed by the training staff and 63 were approved by the TSA
inspectors. Ten security training curriculums remain with the TSA
inspectors for modifications.
[21] According to TSA officials, not all of the requirements apply to
all air carriers. For example, if an air carrier is not hiring new crew
members, the requirements for initial security training would not apply
to that air carrier.
[22] Officials from 15 of the 19 air carriers we interviewed reported
that their training curriculums developed in response to the January
2002 standards were reviewed and approved by TSA.
[23] TSA officials stated that they had received 18 of the 84 air
carriers' security training curriculums as of April 20, 2005.
[24] TSA provided air carriers with a sample form they could use to
develop their end of the course participant evaluation. The sample form
solicited feedback in a number of areas, such as the appropriateness of
the security training course material and design, the instructor's
knowledge of the content of the course material, whether training
exercises simulate tasks needed to perform the job, and the relevancy
of the training to improve knowledge or skills to accomplish crew
members' job functions.
[25] GAO-04-546G.
[26] TSA inspectors were previously located in the Aviation Regulatory
Inspections Division, which has been merged into the newly established
Office of Compliance.
[27] TSA officials also stated that overseeing air carriers' crew
member security training is a small function relative to other TSA
inspector responsibilities. For example, TSA inspectors spend much of
their time responding to air carriers' security incidents such as air
carriers' failure to conduct required cabin searches or security
situations involving hostile passengers.
[28] GAO/AIMD-00-21.3.1.
[29] TSA's inspectors generally determine which standards apply to a
particular air carrier. If air carriers are not hiring new crew
members, then they will not submit an initial security training
curriculum but only submit the recurrent security training curriculum
to TSA for review and approval. The training staff review the air
carriers' security training curriculum to ensure that the curriculum
satisfies the security training standards. According to air carrier
officials, initial security training is provided to newly hired flight
and cabin crew members as well as crew members returning from leave of
absence of more than 3 years. Recurrent security training is annual
training provided to all flight and cabin crew members.
[30] TSA relies on air carriers to track flight and cabin crew members'
completion of required security training. TSA officials stated that
when conducting site visits to observe air carriers' crew member
security training, they may, but are not required to, review air
carriers' records of crew member training completion, although not
required to do so by law. However, officials stated that air carriers
are accountable for their crew members' completion of required security
training. All 19 air carriers we interviewed stated that they maintain
information on their flight and cabin crew members' security training
completion using either training records and/or a database to track the
information electronically.
[31] TSA refers to this program as Crew Member Self-Defense Training.
As with the TSA's basic flight and cabin crew security training
program, this advanced voluntary self-defense training program and its
related oversight is not the responsibility of FAA.
[32] Section 44918, as amended by Vision 100, required TSA to address
several training elements in the advanced voluntary crew member self-
defense training, including (1) deterring a passenger who might present
a threat; (2) advanced control, striking, and restraint techniques; (3)
training to defend oneself against edged or contact weapons; (4)
methods to subdue and restrain an attacker; (5) use of available items
aboard the aircraft for self-defense; and (6) appropriate and effective
responses to defend oneself including the use of force against an
attacker.
[33] The amendments to § 44918 made by Vision 100 superseded those of
the Homeland Security Act, which required TSA, in updating its training
guidance, to issue a rule mandating the inclusion of self-defense
elements into the training program delivered by air carriers.
[34] According to Federal Air Marshal Service officials, Federal Air
Marshal Service officials attended meetings at TSA related to the crew
member self-defense training to ensure that the TSA training was not in
conflict with the Federal Air Marshal Service procedures.
[35] The prototype training was held in Alexandria, Virginia;
Wilmington, California; Fort Worth, Texas; Miami, Florida; and Chicago,
Illinois. Crew member self-defense training in Chicago was cancelled
due to low participation. TSA used a training contractor to deliver the
training and provide the training facilities.
[36] The classes are limited to 24 volunteer crew members per site to
ensure an instructor to student ratio of 1 to 8.
[37] The ten cities include Alexandria, Virginia; Miami, Florida;
Chicago, Illinois; Fort Worth, Texas; Wilmington, California; Broomall,
Pennsylvania; Decatur/Clarkston, Georgia; San Francisco, California;
Aurora, Colorado; and Tempe, Arizona. TSA officials stated that they
selected the cities in which to deliver the training based on whether
there was a major airline hub within or nearby the city. Appendix II
provides information on TSA actual and planned deployment of advanced
voluntary crew member self-defense training from December 2004 through
September 2005.
[38] Section 44918, as amended by Vision 100, requires that TSA develop
and deliver an advanced voluntary crew member self-defense program to
all flight and cabin crew members. TSA officials defined voluntary as
not only allowing crew member to choose whether they want to attend the
training, but also allowing them to observe and not participate in the
hands-on portion of the training.
[39] Scheduling bids are tentative crew member flight schedules, leave,
and vacation times.
[40] Congress has not appropriated funds for the advanced voluntary
crew member self-defense training program. In fiscal year 2004, TSA
reallocated $500,000 from other training programs to support the
development of advanced voluntary crew member self-defense training. In
fiscal year 2005, TSA reallocated $2 million from other programs to
support the delivery of this training. The Senate Appropriations
Committee Report, which accompanied H.R. 2360--the fiscal year 2006 DHS
appropriations bill, as passed by the Senate--would provide $3 million
for voluntary flight crew training.
[41] We interviewed 33 stakeholders regarding TSA's advanced voluntary
self-defense training program, including individuals TSA identified as
subject matter experts and representatives of air carriers, air carrier
associations, crew member labor organizations, and four federal
agencies.
[42] GAO-04-546G.
[43] TSA defines improvised weapons as items commonly found aboard an
aircraft, such as coffee pots, beverage carts, wine bottles, and fire
extinguishers.
[44] GAO-04-546G and GAO-04-291.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
