Aviation Security
Risk, Experience, and Customer Concerns Drive Changes to Airline Passenger Screening Procedures, but Evaluation and Documentation of Proposed Changes Could Be Improved Gao ID: GAO-07-634 April 16, 2007The Transportation Security Administration's (TSA) most visible layer of commercial aviation security is the screening of airline passengers at airport checkpoints, where travelers and their carry-on items are screened for explosives and other dangerous items by transportation security officers (TSO). Several revisions made to checkpoint screening procedures have been scrutinized and questioned by the traveling public and Congress in recent years. For this review, GAO evaluated (1) TSA's decisions to modify passenger screening procedures between April 2005 and December 2005 and in response to the alleged August 2006 liquid explosives terrorist plot, and (2) how TSA monitored TSO compliance with passenger screening procedures. To conduct this work, GAO reviewed TSA documents, interviewed TSA officials and aviation security experts, and visited 25 airports of varying sizes and locations.
Between April 2005 and December 2005, proposed modifications to passenger checkpoint screening standard operating procedures (SOP) were made for a variety of reasons, and while a majority of the proposed modifications--48 of 92--were ultimately implemented at airports, TSA's methods for evaluating and documenting them could be improved. SOP modifications were proposed based on the professional judgment of TSA senior-level officials and program-level staff. TSA considered the daily experiences of airport staff, complaints and concerns raised by the traveling public, and analysis of risks to the aviation system when proposing SOP modifications. TSA also made efforts to balance the impact on security, efficiency, and customer service when deciding which proposed modifications to implement, as in the case of the SOP changes made in response to the alleged August 2006 liquid explosives terrorist plot. In some cases, TSA tested proposed modifications at selected airports to help determine whether the changes would achieve their intended purpose. However, TSA's data collection and analyses could be improved to help TSA determine whether proposed procedures that are operationally tested would achieve their intended purpose. For example, TSA officials decided to allow passengers to carry small scissors and tools onto aircraft based on their review of threat information, which indicated that these items do not pose a high risk to the aviation system. However, TSA did not conduct the necessary analysis of data it collected to assess whether this screening change would free up TSOs to focus on screening for high-risk threats, as intended. TSA officials acknowledged the importance of evaluating whether proposed screening procedures would achieve their intended purpose, but cited difficulties in doing so, including time pressures to implement needed security measures quickly. Finally, TSA's documentation on proposed modifications to screening procedures was not complete. TSA documented the basis--that is, the information, experience, or event that encouraged TSA officials to propose the modifications--for 72 of the 92 proposed modifications. In addition, TSA documented the reasoning behind its decisions for half (26 of 44) of the proposed modifications that were not implemented. Without more complete documentation, TSA may not be able to justify key modifications to passenger screening procedures to Congress and the traveling public. TSA monitors TSO compliance with passenger checkpoint screening procedures through its performance accountability and standards system and through covert testing. Compliance assessments include quarterly observations of TSOs' ability to perform particular screening functions in the operating environment, quarterly quizzes to assess TSOs' knowledge of procedures, and an annual knowledge and skills assessment. TSA uses covert tests to evaluate, in part, the extent to which TSOs' noncompliance with procedures affects their ability to detect simulated threat items hidden in accessible property or concealed on a person. TSA airport officials have experienced resource challenges in implementing these compliance monitoring methods. TSA headquarters officials stated that they are taking steps to address these challenges.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-634, Aviation Security: Risk, Experience, and Customer Concerns Drive Changes to Airline Passenger Screening Procedures, but Evaluation and Documentation of Proposed Changes Could Be Improved
This is the accessible text file for GAO report number GAO-07-634
entitled 'Aviation Security: Risk, Experience, and Customer Concerns
Drive Changes to Airline Passenger Screening Procedures, but Evaluation
and Documentation of Proposed Changes Could Be Improved' which was
released on May 7, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
April 2007:
Aviation Security:
Risk, Experience, and Customer Concerns Drive Changes to Airline
Passenger Screening Procedures, but Evaluation and Documentation of
Proposed Changes Could Be Improved:
GAO-07-634:
GAO Highlights:
Highlights of GAO-07-634, a report to congressional requesters
Why GAO Did This Study:
The Transportation Security Administration‘s (TSA) most visible layer
of commercial aviation security is the screening of airline passengers
at airport checkpoints, where travelers and their carry-on items are
screened for explosives and other dangerous items by transportation
security officers (TSO). Several revisions made to checkpoint screening
procedures have been scrutinized and questioned by the traveling public
and Congress in recent years.
For this review, GAO evaluated
(1) TSA‘s decisions to modify passenger screening procedures between
April 2005 and December 2005 and in response to the alleged August 2006
liquid explosives terrorist plot, and (2) how TSA monitored TSO
compliance with passenger screening procedures. To conduct this work,
GAO reviewed TSA documents, interviewed TSA officials and aviation
security experts, and visited 25 airports of varying sizes and
locations.
What GAO Found:
Between April 2005 and December 2005, proposed modifications to
passenger checkpoint screening standard operating procedures (SOP) were
made for a variety of reasons, and while a majority of the proposed
modifications”48 of 92”were ultimately implemented at airports, TSA‘s
methods for evaluating and documenting them could be improved. SOP
modifications were proposed based on the professional judgment of TSA
senior-level officials and program-level staff. TSA considered the
daily experiences of airport staff, complaints and concerns raised by
the traveling public, and analysis of risks to the aviation system when
proposing SOP modifications. TSA also made efforts to balance the
impact on security, efficiency, and customer service when deciding
which proposed modifications to implement, as in the case of the SOP
changes made in response to the alleged August 2006 liquid explosives
terrorist plot. In some cases, TSA tested proposed modifications at
selected airports to help determine whether the changes would achieve
their intended purpose. However, TSA‘s data collection and analyses
could be improved to help TSA determine whether proposed procedures
that are operationally tested would achieve their intended purpose. For
example, TSA officials decided to allow passengers to carry small
scissors and tools onto aircraft based on their review of threat
information, which indicated that these items do not pose a high risk
to the aviation system. However, TSA did not conduct the necessary
analysis of data it collected to assess whether this screening change
would free up TSOs to focus on screening for high-risk threats, as
intended. TSA officials acknowledged the importance of evaluating
whether proposed screening procedures would achieve their intended
purpose, but cited difficulties in doing so, including time pressures
to implement needed security measures quickly. Finally, TSA‘s
documentation on proposed modifications to screening procedures was not
complete. TSA documented the basis”that is, the information,
experience, or event that encouraged TSA officials to propose the
modifications”for 72 of the 92 proposed modifications. In addition, TSA
documented the reasoning behind its decisions for half (26 of 44) of
the proposed modifications that were not implemented. Without more
complete documentation, TSA may not be able to justify key
modifications to passenger screening procedures to Congress and the
traveling public. TSA monitors TSO compliance with passenger checkpoint
screening procedures through its performance accountability and
standards system and through covert testing. Compliance assessments
include quarterly observations of TSOs‘ ability to perform particular
screening functions in the operating environment, quarterly quizzes to
assess TSOs‘ knowledge of procedures, and an annual knowledge and
skills assessment. TSA uses covert tests to evaluate, in part, the
extent to which TSOs‘ noncompliance with procedures affects their
ability to detect simulated threat items hidden in accessible property
or concealed on a person. TSA airport officials have experienced
resource challenges in implementing these compliance monitoring
methods. TSA headquarters officials stated that they are taking steps
to address these challenges.
What GAO Recommends:
In the March 2007 report that contained sensitive security information,
GAO recommended, and the Department of Homeland Security concurred,
that TSA develop sound methods to assess whether proposed screening
changes would achieve their intended purpose and generate complete
documentation on proposed screening changes that are deemed
significant.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-634].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cathleen Berrick at
(202)512-3404 or berrickc@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
TSA Considered Risk, Experience, and Customer Concerns when Modifying
Passenger Screening Procedures, but Could Improve Its Evaluation and
Documentation of Proposed Procedures:
TSA Has Several Methods in Place to Monitor TSO Compliance with
Passenger Checkpoint Screening SOPs:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope and Methodology:
Appendix II: Sources of SOP Changes:
Appendix III: Comments from the Department of Homeland Security:
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Related Products:
Tables:
Table 1: Categories of Proposed and Implemented Passenger Checkpoint
Screening Changes Considered between April 2005 and December 2005:
Table 2: Factors Considered by TSA When Deciding How to Modify
Passenger Checkpoint Screening SOPs in Response to the Alleged August
2006 Terrorist Plot to Detonate Liquid Explosives on U.S.-Bound
Aircraft:
Table 3: Proposed Procedures Operationally Tested by the Explosives
Detection Improvement Task Force, October 2005-January 2006:
Table 4: TSA Evaluation of Documentation of Agency Decisions Made
between August 7 and August 13, 2006, Regarding the Alleged Liquid
Explosives Terrorist Plot:
Table 5: Modules Included in Recertification Knowledge and Skills
Assessment:
Figures:
Figure 1: Passenger Checkpoint Screening Functions:
Figure 2: TSA Airport Screening Positions:
Abbreviations:
ATSA: Aviation and Transportation Security Act:
CAPPS: computer-assisted passenger prescreening system:
CBP: Customs and Border Protection:
COMPEX: Compliance Examination:
DHS: Department of Homeland Security:
ETD: explosive trace detection:
ETP: explosives trace portal:
FBI: Federal Bureau of Investigation:
FEMA: Federal Emergency Management Agency:
FFDO: Federal Flight Deck Officer:
FSD: Federal Security Director:
HHMD: hand-held metal detector:
IED: improvised explosive device:
PASS: Performance Accountability and Standards System:
PMIS: Performance Management Information System:
PWD: person with disabilities:
SOP: standard operating procedure:
SPOT: Screening Passengers by Observation Technique:
STEA: Screener Training Exercises and Assessments:
TIP: Threat Image Projection:
TSA: Transportation Security Administration:
TSO: transportation security officer:
USP: Unpredictable Screening Process:
WTMD: walk-through metal detector:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 16, 2007:
The Honorable Bennie Thompson:
Chairman,
Committee on Homeland Security:
House of Representatives:
The Honorable John Mica:
Ranking Republican Member:
Committee on Transportation and Infrastructure:
House of Representatives:
The alleged August 2006 terrorist plot to detonate liquid explosives
onboard multiple commercial aircraft bound for the United States from
the United Kingdom has highlighted the continued importance of securing
the commercial aviation system. The Transportation Security
Administration (TSA) is responsible for, among other things, securing
the nation's commercial aviation system while also facilitating the
movement of passengers. To protect this system after the September 11,
2001, terrorist attacks, TSA implemented a multilayered system of
security--the most publicly visible layer being the physical screening
of passengers and their carry-on bags at airport screening checkpoints,
which all passengers must pass through prior to entering an airport's
sterile area, the area within the terminal that provides passengers
access to boarding aircraft.[Footnote 1]
The passenger checkpoint screening system is composed of three
elements: (1) the people responsible for conducting the screening of
airline passengers and their carry-on items--Transportation Security
Officers (TSO) (formerly known as screeners), (2) the procedures TSOs
are to follow to conduct screening, and (3) the technology used in the
screening process. Collectively, these elements help to determine the
effectiveness and efficiency of passenger checkpoint screening. TSA has
made efforts to enhance each of the three elements of the passenger
checkpoint screening system.
Since its inception, TSA has issued 25 versions of the passenger
checkpoint screening standard operating procedures (SOP), to include
new screening procedures as well as changes to existing screening
procedures. Several of these revisions have been criticized and
scrutinized by the traveling public and Congress. For example, in
September 2004, TSA modified its passenger screening procedures in
response to the August 2004 midair explosions of two Russian airliners,
believed to have been caused by Chechen women who concealed explosive
devices under their clothing. Specifically, the revision entailed a
more invasive technique for patting down the torso area of passengers.
According to TSA officials, in response to complaints raised by airline
passengers and TSA's review of additional threat information, TSA
further modified the pat-down procedure in December 2004 to entail a
more targeted, less intrusive pat-down procedure. In December 2005, TSA
allowed passengers to carry small scissors and small tools onto
aircraft, resulting in concern by some industry representatives that
allowing sharp objects onto planes would put flight crew at risk of
attack.[Footnote 2] This procedural change also resulted in the TSA
Assistant Secretary being asked to testify before Congress on the
agency's rationale for allowing passengers to carry small scissors and
small tools onto planes and a legislative mandate for us to assess the
impact of the prohibited items list change on public safety and
screening operations.[Footnote 3]
In light of the potential impact of changes to passenger checkpoint
screening procedures, you asked that we assess TSA's process for
determining whether and how screening procedures should be modified, to
include the creation of new screening procedures and changes to
existing screening procedures. Specifically, this report addresses the
following questions: (1) How and on what basis did TSA modify passenger
screening procedures and what factors guided the decisions to do so?
(2) How does TSA determine whether TSOs are complying with the standard
procedures for screening passengers and their carry-on items? In March
2007, we issued a report that contained sensitive security information
regarding TSA's passenger checkpoint screening procedures, including
the factors TSA considered when modifying passenger screening
procedures and TSA's efforts to monitor TSO compliance with standard
passenger screening procedures. This report provides the results of our
March 2007 report with sensitive security information removed.
To obtain information on the process used to modify passenger
checkpoint screening procedures, we reviewed and analyzed available TSA
documentation on proposed procedure modifications that were considered
between April 2005 and December 2005, as well as threat assessments and
operational studies that supported SOP modifications.[Footnote 4] We
also reviewed and analyzed similar documentation for proposed
modifications considered between August 2006 and November 2006 in
response to the alleged terrorist plot to detonate liquid explosives
onboard multiple aircraft en route from the United Kingdom to the
United States. We included modifications to passenger checkpoint
screening procedures related to this particular event because they
provided the most recent information available of TSA's approach to
modifying screening procedures in response to an immediate perceived
threat to civil aviation. To assess TSO compliance with standard
operating procedures, our work also involved a review of available
documentation, including guidance, checklists, and other evaluation
tools used by TSA. In addition, we met with TSA headquarters officials
who were involved in the process of determining whether proposed
changes to passenger checkpoint screening procedures should be
implemented, and who were responsible for overseeing efforts to monitor
TSO compliance with screening procedures. We also visited or conducted
phone interviews with staff at 25 airports, which we selected based on
variation in size, geographic location, and level of performance on
compliance-related assessments. At each airport, we interviewed Federal
Security Directors (FSD),[Footnote 5] members of their management
teams, and TSOs with passenger screening responsibilities. Information
from these interviews cannot be generalized to all airports nationwide.
Two of the airports we visited were also participants in TSA's
Screening Partnership Program.[Footnote 6] We also met with officials
from the Department of Homeland Security (DHS) Science and Technology
Directorate as well as the Federal Bureau of Investigation (FBI) to
discuss the methodology and results of their liquid explosives tests,
which were used to support TSA's decisions to modify the SOP in
September 2006. We also interviewed five experts in the field of
aviation security to obtain their perspectives on TSA's approach for
deciding whether to implement proposed checkpoint screening
procedures.[Footnote 7] We compared TSA's approach for implementing and
revising passenger checkpoint screening procedures, and for monitoring
TSO compliance, with the Comptroller General's standards for internal
control in the federal government[Footnote 8] and with risk management
guidance. We assessed the reliability of the data we acquired from TSA
regarding TSO compliance and found the data to be sufficiently reliable
for our purposes.
We conducted our work from March 2005 through January 2007 in
accordance with generally accepted government auditing standards. More
details about the scope and methodology of our work are presented in
appendix I.
Results in Brief:
During our 9-month review period, proposed modifications to passenger
checkpoint screening procedures were made in various ways and for a
variety of reasons, and while a majority of the proposed modifications-
-48 of 92--were ultimately implemented at airports, TSA's methods for
evaluating and documenting them could be improved. Some SOP
modifications were proposed based on the professional judgment of TSA
senior-level officials and program-level staff at headquarters and at
airports nationwide, while other modifications were proposed by members
of a TSA task force charged with enhancing TSA's ability to detect
improvised explosive devices at checkpoints. TSA officials proposed SOP
modifications based on risk information (threat and vulnerability
information), daily experiences of staff working at airports, and
complaints and concerns raised by the traveling public. In addition to
these factors, TSA senior leadership made efforts to balance the impact
that proposed SOP modifications--such as the changes to the liquids,
gels, and aerosols screening procedures--would have on security,
efficiency, and customer service when deciding whether proposed SOP
modifications should be implemented. In some cases, TSA first tested
proposed modifications to screening procedures at selected airports to
help determine whether the changes would achieve their intended
purpose, such as to enhance detection of prohibited items or to free up
TSO resources to perform screening activities focused on threats
considered to pose a high risk, such as explosives. However, TSA's data
collection and analyses could be improved to help TSA determine whether
proposed procedures that are operationally tested would achieve their
intended purpose. Specifically, for the seven tests of proposed
screening procedures TSA conducted during our review period, although
TSA collected some data on the efficiency of and customer response to
the procedures at selected airports, the agency generally did not
collect the type of data or conduct the necessary analysis that would
yield information on whether proposed procedures would achieve their
intended purpose. TSA officials acknowledged that they could have made
some improvements in the various analyses they conducted related to the
decision to allow small scissors and tools onboard aircraft, but cited
several difficulties in doing so. Nevertheless, until TSA takes steps
to improve its ability to evaluate the potential impact of screening
changes on security and TSO resource availability, it may be difficult
for TSA to determine how best to allocate limited TSO resources, help
ensure the screeners' ability to detect explosives and other high-
threat objects, and evaluate whether proposed modifications to
screening procedures would have the intended effect. Finally, TSA's
documentation on proposed modifications to screening procedures was not
always complete. TSA documented the basis--that is, the information,
experience, or event that encouraged TSA officials to propose the
modifications--for 72 of the 92 proposed modifications. In addition,
TSA only documented the reasoning behind its decisions for about half
(26 of 44) of the proposed modifications that were not implemented. Our
standards for governmental internal controls and associated guidance
suggest that agencies should document key decisions in a way that is
complete and accurate. Without such information, TSA cannot always
justify significant SOP modifications to Congress and the traveling
public. TSA officials acknowledged that it is beneficial to maintain
documentation on the reasoning behind decisions to implement or reject
proposed SOP modifications deemed significant, particularly given the
organizational restructuring and staff turnover within TSA.
TSA monitors TSO compliance with passenger checkpoint screening SOPs
through its performance accountability and standards system and through
local and national covert testing.[Footnote 9] According to TSA
officials, the agency developed the performance accountability and
standards system in response to our 2003 report that recommended that
TSA establish a performance management system that makes meaningful
distinctions in employee performance,[Footnote 10] and in response to
input from TSA airport staff on how to improve passenger and checked
baggage screening measures. This system is used by TSA to measure TSO
compliance with passenger checkpoint screening procedures. Of the 24
FSDs we interviewed about compliance assessments, 9 cited difficulties
in implementing the performance accountability and standards system
because of a lack of available staff to conduct observations and
administer SOP quizzes. When asked whether they planned to address
FSDs' concerns regarding a lack of available staff to evaluate TSO
compliance with SOPs, TSA headquarters officials said that they have
automated many of the data entry functions of the performance
accountability and standards system to relieve the field of the burden
of manually entering this information into the online system.
Furthermore, the TSA Assistant Secretary stated that FSDs were given
the option of delaying implementation of the performance accountability
and standards system if they were experiencing resource challenges. In
addition to implementing the performance accountability and standards
system, TSA conducts local and national covert tests to evaluate, in
part, the extent to which TSOs' noncompliance with the SOPs affects
their ability to detect simulated threat items hidden in accessible
property or concealed on a person.[Footnote 11] Even though all 24 FSDs
said that they have conducted local covert tests, 10 of these FSDs said
that lack of available staff made it difficult to conduct these tests.
TSA officials told us that they are considering resource alternatives
for implementing these tests, but did not provide us with specific
details of these plans. Based on the results of national covert tests
conducted between September 2005 and July 2006, which showed that some
TSOs did not identify threat objects, in part because they did not
comply with SOPs, TSA's Office of Inspection recommended, among other
things, that the Office of Security Operations ensure that TSOs adhere
to the current passenger checkpoint screening SOPs. However, until the
resource limitations that have restricted TSA's use of its compliance
monitoring tools have been fully addressed, TSA may not have assurance
that TSOs are screening passengers according to standard procedures.
To help improve TSA's ability to evaluate proposed SOP modifications
and to justify its decisions regarding whether proposed SOP
modifications should be implemented, in the March 2007 report that
contained sensitive security information, we recommended that the
Secretary of the Department of Homeland Security direct the Assistant
Secretary of Homeland Security for TSA to (1) develop sound evaluation
methods, when possible, that can help TSA determine whether proposed
procedures that are operationally tested would achieve their intended
purpose, and (2) generate and maintain complete documentation of
proposed modifications deemed significant by TSA. DHS generally
concurred with our findings and recommendations and outlined actions
TSA plans to take to implement the recommendations. For example, TSA
intends to improve its methods for evaluating proposed SOP
modifications, which may entail randomly selecting the airports that
will participate in a study to better isolate the impact of proposed
SOP modifications on passenger screening. DHS also stated that TSA is
in the process of developing protocols that will require documentation
of the source and intent of proposed SOP modifications, as well as
documentation of TSA officials' reasoning for implementing or rejecting
proposed modifications. The full text of DHS's comments is included in
appendix III.
Background:
Passenger Checkpoint Screening System:
Passenger screening is a process by which personnel authorized by TSA
inspect individuals and property to deter and prevent the carriage of
any unauthorized explosive, incendiary, weapon, or other dangerous item
onboard an aircraft or into a sterile area.[Footnote 12] Passenger
screening personnel must inspect individuals for prohibited items at
designated screening locations.[Footnote 13] As shown in figure 1, the
four passenger screening functions are:
* X-ray screening of property,
* walk-through metal detector screening of individuals,
* hand-wand or pat-down screening of individuals, and:
* physical search of property and trace detection for explosives.
Typically, passengers are only subjected to X-ray screening of their
carry-on items and screening by the walk-through metal detector.
Passengers whose carry-on baggage alarms the X-ray machine, who alarm
the walk-through metal detector, or who are designated as selectees--
that is, passengers selected by the Computer-Assisted Passenger
Prescreening System (CAPPS[Footnote 14]) or other TSA-approved
processes to receive additional screening--are screened by hand-wand or
pat-down and have their carry-on items screened for explosives traces
or physically searched.
Figure 1: Passenger Checkpoint Screening Functions:
[See PDF for image]
Source: GAO and Nova Development Corporation.
Note: Explosive trace detection (ETD) works by detecting vapors and
residues of explosives. Human operators collect samples by rubbing
swabs along the interior and exterior of an object that TSOs determine
to be suspicious, and place the swabs in the ETD machine, which then
chemically analyzes the swab to identify any traces of explosive
materials.
Bomb Appraisal Officers (BAO) are available to respond to unresolved
alarms at the checkpoint that involve possible explosive devices. The
BAO may contact appropriate law enforcement or bomb squad officials if
review indicates possible or imminent danger, in which case the BAO
ensures that the security checkpoint is cleared. The BAO approves
reopening of security lane(s) if no threat is posed.
[A] BDOs are TSOs specially trained to detect suspicious behavior in
individuals approaching the checkpoint. Should the BDO observe such
behavior, he or she may refer the individual for additional screening
or to a law enforcement officer.
[B] The hand-wand or pat-down is conducted if a passenger is identified
or randomly selected for additional screening because he or she met
certain criteria or alarmed the walk-through metal detector.
[C] Manual or ETD searches of accessible property occur if the
passenger is identified or randomly selected for additional screening
or if the screener identified a potential prohibited item on X-ray.
[End of figure]
The passenger checkpoint screening system is composed of three
elements: the people responsible for conducting the screening of
airline passengers and their carry-on items--TSOs, the technology used
during the screening process, and the procedures TSOs are to follow to
conduct screening. Collectively, these elements help to determine the
effectiveness and efficiency of passenger checkpoint screening.
Transportation Security Officers:
TSOs screen all passengers and their carry-on baggage prior to allowing
passengers access to their departure gates. There are several positions
within TSA that perform and directly supervise passenger screening
functions. Figure 2 provides a description of these positions.
Figure 2: TSA Airport Screening Positions:
[See PDF for image]
Source: GAO analysis of TSA data.
[A] Number of annualized TSA screening positions for fiscal year 2006.
These positions do not include private screener positions at the six
airports that participated in the Screening Partnership Program during
fiscal year 2006.
[End of figure]
In May 2005, we reported on TSA's efforts to train TSOs and to measure
and enhance TSO performance.[Footnote 15] We found that TSA had
initiated a number of actions designed to enhance passenger TSO,
checked baggage TSO, and supervisory TSO training. However, at some
airports TSOs encountered difficulty accessing and completing recurrent
(refresher) training because of technological and staffing constraints.
We also found that TSA lacked adequate internal controls to provide
reasonable assurance that TSOs were receiving legislatively mandated
basic and remedial training, and to monitor the status of its recurrent
training program. Further, we reported that TSA had implemented and
strengthened efforts to collect TSO performance data as part of its
overall effort to enhance TSO performance. We recommended that TSA
develop a plan for completing the deployment of high-speed Internet/
intranet connectivity to all TSA airport training facilities, and
establish appropriate responsibilities and other internal controls for
monitoring and documenting TSO compliance with training requirements.
DHS generally concurred with our recommendations and stated that TSA
has taken steps to implement them.
Screening Technology:
There are typically four types of technology used to screen airline
passengers and their carry-on baggage at the checkpoint:
* walk-through metal detectors,
* X-ray machines,
* hand-held metal detectors, and:
* explosive trace detection (ETD) equipment.
The President's fiscal year 2007 budget request noted that emerging
checkpoint technology will enhance the detection of prohibited items,
especially firearms and explosives, on passengers. As of December 2006,
TSA plans to conduct operational tests of three types of passenger
screening technologies within the next year. TSA has conducted other
tests in the past; for example, during fiscal year 2005, TSA
operationally tested document scanners, which use explosive trace
detection technology to detect explosives residue on passengers'
boarding passes or identification cards. TSA decided not to expand the
use of the document scanner, in part because of the extent to which
explosives traces had to be sampled manually. TSA also plans to begin
operational tests of technology that would screen bottles for liquid
explosives. We are currently evaluating the Department of Homeland
Security's and TSA's progress in planning for, managing, and deploying
research and development programs in support of airport checkpoint
screening operations. We expect to report our results in August 2007.
Standard Operating Procedures:
TSA has developed checkpoint screening standard operating procedures,
which are the focus of this report, that establish the process and
standards by which TSOs are to screen passengers and their carry-on
items at screening checkpoints.[Footnote 16] Between April 2005 and
December 2005, based on available documentation, TSA deliberated 189
proposed changes to passenger checkpoint screening SOPs, 92 of which
were intended to modify the way in which passengers and their carry-on
items are screened.[Footnote 17] TSA issued six versions of the
passenger checkpoint screening SOPs during this period.[Footnote 18]
TSA modified passenger checkpoint screening SOPs to enhance the
traveling public's perception of the screening process, improve the
efficiency of the screening process, and enhance the detection of
prohibited items and suspicious persons. As shown in table 1, 48 of the
92 proposed modifications to passenger checkpoint screening SOPs were
implemented, and the types of modifications made or proposed generally
fell into one of three categories--customer satisfaction, screening
efficiency, and security.
Table 1: Categories of Proposed and Implemented Passenger Checkpoint
Screening Changes Considered between April 2005 and December 2005:
Category of proposed changes: Customer satisfaction;
Description of category: Changes that will improve the traveling
public's perception of the screening process or reduce or exempt
categories of authorized individuals from certain aspects of the
screening process;
Proposed SOP changes: 42;
Implemented SOP changes: 22.
Category of proposed changes: Screening efficiency;
Description of category: Changes that will improve screening flow,
clarify screener duties, update equipment procedures, or enhance the
working environment of screening locations;
Proposed SOP changes: 31;
Implemented SOP changes: 17.
Category of proposed changes: Security;
Description of category: Changes that will improve TSA's ability to
detect prohibited items and suspicious persons;
Proposed SOP changes: 19;
Implemented SOP changes: 9.
Category of proposed changes: Total;
Description of category: [Empty];
Proposed SOP changes: 92;
Implemented SOP changes: 48.
Source: GAO analysis of TSA data.
[End of table]
TSA Considered Risk, Experience, and Customer Concerns when Modifying
Passenger Screening Procedures, but Could Improve Its Evaluation and
Documentation of Proposed Procedures:
TSA used various processes between April 2005 and December 2005 to
modify passenger checkpoint screening SOPs, and a variety of factors
guided TSA's decisions to modify SOPs. TSA's processes for modifying
SOPs generally involved TSA staff recommending proposed modifications,
reviewing and commenting on proposed modifications, and TSA senior
leadership making final decisions as to whether proposed modifications
should be implemented. During our 9-month review period, TSA officials
considered 92 proposed modifications to the way in which passengers and
their carry-on items were screened, and 48 were implemented.[Footnote
19] TSA officials proposed SOP modifications based on risk factors
(threat and vulnerability information), day-to-day experiences of
airport staff, and concerns and complaints raised by passengers. TSA
then made efforts to balance security, efficiency, and customer service
when deciding which proposed SOP modifications to implement. Consistent
with our prior work that has shown the importance of data collection
and analyses to support agency decision making, TSA conducted data
collection and analysis for certain proposed SOP modifications that
were tested before they were implemented at all airports. Nevertheless,
we found that TSA could improve its data collection and analysis to
assist the agency in determining whether the proposed procedures would
enhance detection or free up TSO resources, when intended. In addition,
TSA did not maintain complete documentation of proposed SOP
modifications; therefore, we could not fully assess the basis for
proposed SOP modifications or the reasons why certain proposed
modifications were not implemented. TSA officials acknowledged that it
is beneficial to maintain documentation on the reasoning behind
decisions to implement or reject SOP modifications deemed significant.
TSA's Processes for Modifying SOPs Were Driven by Input from TSA Field
and Headquarters Staff:
Proposed SOP modifications were submitted and reviewed under two
processes during our 9-month review period, and for each process, TSA
senior leadership made the final decision as to whether the proposed
modifications would be implemented. One of the processes TSA used to
modify passenger checkpoint screening SOPs involved TSA field staff or
headquarters officials, and, to a lesser extent, TSA senior leadership,
suggesting ways in which passenger checkpoint screening SOPs could be
modified. These suggestions were submitted through various mechanisms,
including electronic mail and an SOP panel review conducted by TSA
airport personnel. (These methods are described in more detail in app.
II.) Eighty-two of the 92 proposed modifications were considered under
this process.
If TSA officials determined, based on their professional judgment, that
the recommended SOP modifications--whether from headquarters or the
field--merited further consideration, or if a specific modification was
proposed by TSA senior leadership, the following chain of events
occurred:
* First, the procedures branch of the Office of Security Operations
drafted SOP language for each of the proposed modifications.[Footnote
20]
* Second, the draft language for each proposed modification was
disseminated to representatives of various TSA divisions for review,
and the language was revised as needed.
* Third, TSA officials tested proposed modifications in the airport
operating environment if they found it necessary to:
- assess the security impact of the proposed modification,
- evaluate the impact of the modification on the amount of time taken
for passengers to clear the checkpoint,
- measure the impact of the proposed modification on passengers and
industry partners, or:
- determine training needs created by the proposed modification.
* Fourth, the revised SOP language for proposed modifications was sent
to the heads of several TSA divisions for comment.
* Fifth, considering the comments of the TSA division heads, the head
of the Office of Security Operations or other TSA senior leadership
made the final decision as to whether proposed modifications would be
implemented.
Another process for modifying passenger checkpoint screening SOPs
during our 9-month review period was carried out by TSA's Explosives
Detection Improvement Task Force. The task force was established in
October 2005 by the TSA Assistant Secretary to respond to the threat of
improvised explosive devices (IED) being carried through the
checkpoint. The goal of the task force was to apply a risk-based
approach to screening passengers and their baggage in order to enhance
TSA's ability to detect IEDs.[Footnote 21] The task force developed 13
of the 92 proposed SOP modifications that were considered by TSA
between April 2005 and December 2005.[Footnote 22] The task force
solicited and incorporated feedback from representatives of various TSA
divisions on these proposed modifications and presented them to TSA
senior leadership for review and approval. TSA senior leadership
decided that 8 of the 13 proposed modifications should be operationally
tested--that is, temporarily implemented in the airport environment for
the purposes of data collection and evaluation--to better inform
decisions regarding whether the proposed modifications should be
implemented. Following the testing of these proposed modifications in
the airport environment, TSA senior leadership decided to implement 7
of the 8 operationally tested changes.[Footnote 23] (The task force's
approach to testing these procedures is discussed in more detail
below.) Following our 9-month period of review, the changes that TSA
made to its passenger checkpoint screening SOPs in response to the
alleged August 2006 liquid explosives terror plot were decided upon by
DHS and TSA senior leadership, with some input from TSA field staff,
aviation industry representatives, and officials from other federal
agencies.
Risk Factors, Day-to-Day Experiences, and Customer Concerns Were the
Basis for Proposed SOP Modifications:
Based on available documentation,[Footnote 24] risk factors (i.e.,
threats to commercial aviation and vulnerability to those threats), day-
to-day experiences of airport staff, and complaints and concerns raised
by passengers were the basis for TSA staff and officials proposing
modifications to passenger checkpoint screening SOPs.
Fourteen of the 92 procedure modifications recommended by TSA staff and
officials were based on reported or perceived threats to commercial
aviation, and existing vulnerabilities to those threats. For example,
the Explosives Detection Improvement Task Force proposed SOP
modifications based on threat reports developed by TSA's Intelligence
and Analysis division. Specifically, in an August 2005 civil aviation
threat assessment, the division reported that terrorists are likely to
seek novel ways to evade U.S. airport security screening.[Footnote 25]
Subsequently, the task force proposed that the pat-down procedure
performed on passengers selected for additional screening be revised to
include not only the torso area, which is what the previous pat-down
procedure entailed, but additional areas of the body such as the
legs.[Footnote 26] The August 2005 threat assessment also stated that
terrorists may attempt to carry separate components of an IED through
the checkpoint, then assemble the components while onboard the
aircraft. To address this threat, the task force proposed a new
procedure to enhance TSOs' ability to search for components of
improvised explosive devices. According to TSA officials, threat
reports have also indicated that terrorists rely on the routine nature
of security measures in order to plan their attacks. To address this
threat, the task force proposed a procedure that incorporated
unpredictability into the screening process by requiring designated
TSOs to randomly select passengers to receive additional search
procedures. Following our 9-month review period, TSA continued to use
threat information as the basis for proposed modifications to passenger
checkpoint screening SOPs. In August 2006, TSA proposed modifications
to passenger checkpoint screening SOPs after receiving threat
information regarding an alleged terrorist plot to detonate liquid
explosives onboard multiple aircraft en route from the United Kingdom
to the United States. Regarding vulnerabilities to reported threats,
based on the results of TSA's own covert tests (undercover, unannounced
tests), TSA's Office of Inspection recommended[Footnote 27] SOP
modifications to enhance the detection of explosives at the passenger
screening checkpoint.[Footnote 28]
TSA officials also proposed modifications to passenger checkpoint
screening SOPs based on their professional judgment regarding perceived
threats to aviation security. For example, an FSD recommended changes
to the screening of funeral urns based on a perceived threat. In some
cases, proposed SOP modifications appeared to reflect threat
information analyzed by TSA officials. For example, TSOs are provided
with Threat in the Spotlight, a weekly report that identifies new
threats to commercial aviation, examples of innovative ways in which
passengers may conceal prohibited items, and pictures of items that may
not appear to be prohibited items but actually are. TSOs are also
provided relevant threat information during briefings that take place
before and after their shifts. In addition, FSDs are provided
classified intelligence summaries on a daily and weekly basis, as well
as monthly reports of suspicious incidents that occurred at airports
nationwide. TSA's consideration of threat and vulnerability--through
analysis of current documentation and by exercising professional
judgment--is consistent with a risk-based decision-making
approach.[Footnote 29] As we have reported previously, and DHS and TSA
have advocated, a risk-based approach, as applied in the homeland
security context, can help to more effectively and efficiently prepare
defenses against acts of terrorism and other threats.
TSA headquarters and field staff also based proposed SOP modifications-
-specifically, 36 of the 92 proposed modifications--on experience in
the airport environment. For example, TSA headquarters officials
conduct reviews at airports to identify best practices and deficiencies
in the checkpoint screening process. During one of these reviews,
headquarters officials observed that TSOs were not fully complying with
the pat-down procedure. After discussions with TSOs, TSA headquarters
officials determined that the way in which TSOs were conducting the
procedure was more effective. In addition, TSA senior leadership, after
learning that small airports had staffing challenges that precluded
them from ensuring that passengers are patted down by TSOs of the same
gender, proposed that opposite-gender pat-down screening be allowed at
small airports.
Passenger complaints and concerns shared with TSA also served as a
basis for proposed modifications during our 9-month review period.
Specifically, of the 92 proposed SOP modifications considered during
this period, TSA staff and officials recommended 29 modifications based
on complaints and concerns raised by passengers. For example, TSA
headquarters staff recommended allowing passengers to hold their hair
while being screened by the Explosives Trace Portal,[Footnote 30] after
receiving complaints from passengers about eye injuries from hair
blowing in their eyes and hair being caught in the doors of the portal.
TSA Balanced Security, Efficiency, and Customer Service when Deciding
whether to Implement Proposed SOP Modifications:
When deciding whether to implement proposed SOP modifications, TSA
officials also made efforts to balance the impact of proposed
modifications on security, efficiency,[Footnote 31] and customer
service. TSA's consideration of these factors reflects the agency's
mission to protect transportation systems while also ensuring the free
movement of people and commerce. As previously discussed, TSA sought to
improve the security of the commercial aviation system by modifying the
SOP for conducting the pat-down search. (TSA identified the modified
pat-down procedure as the "bulk-item" pat-down.) When deciding whether
to implement the proposed modification, TSA officials considered not
only the impact that the bulk-item pat-down procedure would have on
security, but also the impact that the procedure would have on
screening efficiency and customer service. For example, TSA officials
determined that the bulk-item pat-down procedure would not
significantly affect efficiency because it would only add a few seconds
to the screening process. Following our 9-month review period, TSA
continued to make efforts to balance security, efficiency, and customer
service when deciding whether to implement proposed SOP modifications,
as illustrated by TSA senior leadership's deliberation on proposed SOP
modifications in response to the alleged August 2006 liquid explosives
terrorist plot. TSA modified the passenger checkpoint screening SOP
four times between August 2006 and November 2006 in an effort to defend
against the threat of terrorists' use of liquid explosives onboard
commercial aircraft.[Footnote 32] While the basis for these
modifications was to mitigate risk, as shown in table 2, TSA senior
leadership considered several other factors when deciding whether to
implement the modifications.
Table 2: Factors Considered by TSA When Deciding How to Modify
Passenger Checkpoint Screening SOPs in Response to the Alleged August
2006 Terrorist Plot to Detonate Liquid Explosives on U.S.-Bound
Aircraft:
Procedures: August 10, 2006: Total ban on liquids and gels in
accessible property or onboard aircraft. Exceptions:
* baby formula/ milk if infant is traveling;
* prescription medication with name matching passenger's ticket;
* insulin and other essential nonprescription medications;
* liquids and gels carried by passengers with disabilities, after
screening for explosive materials, with Supervisory TSO/Screening
Manager concurrence;
* supplies brought into retail area by approved vendors for restocking
of retail operations.
Passengers required to remove shoes at checkpoints for X-ray screening;
Impact on security: Benefits;
* Terrorists less likely to successfully carry liquid explosives onto
aircraft using container;
* Terrorists less likely to successfully carry liquid explosives onto
aircraft in shoes (e.g., gel-based insoles).
Drawbacks;
* None identified;
Impact on efficiency of screening process: Benefits;
* Requiring passengers to remove footwear will speed the screening
process by reducing the need to ETD and physically inspect footwear.
Footwear now only needs to be subjected to physical search if something
suspicious appears on the X-ray of the shoes;
Drawbacks;
* Total ban on liquids and gels may be unsustainable for long term
because more passengers would check their baggage rather than carry it
on, which would cause a strain on the checked baggage screening system;
Impact on customer service: Benefits;
* Exceptions allow passengers with legitimate medical and other needs
to bring essential liquids onboard aircraft;
* Passengers less confused about whether to remove shoes for X-ray
screening;
Drawbacks;
* Inconvenient for passengers to not be able to carry toiletries and
similar liquids and gels onto planes;
Other considerations:
* Threat was a specific type of liquid explosive;
* There was no checkpoint screening technology available for deployment
that could detect the specific liquid explosive.
Procedures: August 12, 2006[A] : Aerosols prohibited. Following
additional items allowed past checkpoints:
* baby food in small containers, if baby/small child is traveling;
* essential nonprescription medications (e.g., contact lens saline
solution, eye care products), not to exceed 4 fluid ounces per
container;
* liquids and gels for diabetic passengers, no greater than 8 fluid
ounces per container;
* gels, saline solutions, and other liquids used to augment portions of
body for medical/cosmetic reasons;
* life support/life sustaining liquids (e.g., bone marrow and blood
products;
Impact on security: Benefits;
* Terrorists less likely to successfully carry liquid explosives onto
aircraft using container;
* Terrorists less likely to successfully carry liquid explosives onto
aircraft in shoes (e.g., gel-based insoles).
Drawbacks;
* None identified;
Impact on efficiency of screening process: Benefits;
* Requiring passengers to remove footwear will speed the screening
process by reducing the need to ETD and physically inspect footwear.
Footwear now only needs to be subjected to physical search if something
suspicious appears on the X-ray of the shoes;
Drawbacks;
* Total ban on liquids and gels may be unsustainable for long term
because more passengers would check their baggage rather than carry it
on, which would cause a strain to the checked baggage screening system;
Impact on customer service: Benefits;
* Clarified for TSOs types and amounts of liquids and gels exempt from
ban;
* Created smoother process at checkpoint, minimizing impact upon
travelers;
* Gave diabetic passengers access to essential liquids;
* Lifted prohibition against critical life saving fluids;
Drawbacks;
* None identified;
Other considerations:
* Feedback from TSA field staff and industry representatives regarding
exemptions associated with liquids, gels, and aerosols restrictions and
specific information on the quantities of certain types of liquids,
gels, and aerosols that should be exempted from the restrictions;
* Additional information was obtained about the alleged terrorist plot,
including information from the United Kingdom and U.S. intelligence
communities and discussions with explosives experts.
Procedures: September 26, 2006: Liquids, gels, aerosols (not on
prohibited items list or considered hazardous materials) permitted in
accessible property in 3-fluid ounce bottles fit comfortably in one
quart-size, clear plastic, zip-top bag per passenger. Plastic bags
screened by X-ray. Items purchased in sterile area of airports
permitted onboard aircraft.
Items allowed past checkpoints in amounts larger than 3 fluid ounces;
must be declared and cleared by TSO:
* baby formula/milk /food in small containers if baby/ small child is
traveling,;
* medications (liquid, gel, aerosol),;
* liquids and gels for passengers indicating need to address diabetic
or other medical condition;
TSOs conducting declaration process positioned ahead of checkpoint to
assess liquids, gels, and aerosols to determine reasonable quantity for
passenger's itinerary, and to advise passengers on procedures related
to liquids, gels and aerosols that are either prohibited (requiring
disposal or abandonment of items) or permitted but outside of plastic
bag (TSO marks boarding pass or travel document to indicate items).
Items newly permitted past checkpoints in any amount:
* liquid, gel, and aerosol cleaning supplies required by airport
employees servicing sterile area,;
* gels and frozen liquids required to cool any other items permitted
past checkpoints, provided no unresolved suspicious items or
activities.
Random ETD sampling of plastic bags, containers within plastic bags,
and other containers holding liquids, gels, and aerosols;
Impact on security: Benefits;
* Plastic bags present deterrent and operational complexities for
terrorists--attempts to combine liquids increase probability of
detection;
* Requirement to remove and submit plastic bags for X-ray screening
serves as deterrent to terrorists, and provides TSOs opportunity to
view and examine all liquids, gels, and aerosols;
* Plastic bags hinder terrorists from carrying large enough amounts of
liquid explosives that could potentially cause catastrophic damage to
an aircraft;
* Declaration process thought to deter terrorists from attempting to
carry liquid explosives onboard aircraft;
* Random ETD sampling enables TSOs to determine whether the small
amounts of liquids and gels being carried through the checkpoint are,
in fact, explosives. This procedure may also deter terrorists from
attempting to carry liquid explosives onboard aircraft;
Drawbacks;
* Possibility that terrorists could combine liquids in small bottles to
generate an amount large enough to potentially cause catastrophic
damage to an aircraft; * The additional drawbacks related to the impact
on security are sensitive security information. Therefore, we do not
discuss those drawbacks in this report;
Impact on efficiency of screening process: Benefits;
* Enables TSOs to focus resources on detecting explosives, rather than
small amounts of liquids and gels that do not represent serious threat;
* Checked baggage screening expected to return to sustainable levels;
* Requirement to remove and submit plastic bags for X-ray screening
encourages passengers to reduce clutter in bags, making it easier for
TSOs to screen for prohibited and threat items.
Drawbacks;
* Increase in number of items X-rayed per passenger, which may slow
down screening process;
Impact on customer service: Benefits;
* Procedures easily learned by public and TSOs;
* Accommodates many passengers with legitimate needs for small
quantities of liquids during flights.
Drawbacks;
* Possible negative public reaction to passengers having to provide
their own plastic bags;
Other considerations:
* The results of liquid explosives tests conducted by DHS and the FBI.
The results of these tests are sensitive security information and are
not discussed in this report;
* TSA gathered data to test its assumption regarding sustainability of
the total ban on liquids, gels, and aerosols and found that following
the total ban, there was approximately a 27 percent increase in the
number of bags checked per passenger.
Procedures: November 21, 2006: Same as the procedures implemented on
September 26, 2006, with the exception of the following:
Liquids, gels, and aerosols allowed in 3.4-fluid- ounce (100-
milliliter) "travel size" bottles.
Declaration process eliminated; TSA employee ahead of checkpoint offers
public advisements and assessments on procedures; (Additional
modifications were made to the liquids, gels, and aerosols screening
procedures. However, these additional modifications are sensitive
security information. Therefore, we do not discuss these modifications
in this report.);
Impact on security: Benefits; No additional security benefits
identified.
Drawbacks;
No additional drawbacks to security identified;
Impact on efficiency of screening process: Benefits;
* Elimination of the declaration process will reduce unnecessary
redundancy in the examination of exempted liquids and gels, which
previously occurred both prior to and following x-ray screening; (TSA
identified additional efficiency benefits of this modification to the
liquids, gels, and aerosols screening procedures. These additional
benefits are sensitive security information. Therefore, we do not
discuss these benefits in this report.);
Drawbacks; No additional efficiency drawbacks identified;
Impact on customer service: Benefits;
* Allowing for risk-based discretion on the part of Supervisory TSOs
enhances customer service for passengers who have legitimate reasons
for carrying liquids, gels, or aerosols onboard planes.
Drawbacks; No additional customer service drawbacks identified;
Other considerations:
* The European Union allowed passengers to carry liquids, gels, and
aerosols in travel sized containers up to 100 milliliters,
approximately 3.4 fluid ounces;
* The results of liquid explosive testing conducted by FBI and DHS;
* TSA recognized that no procedure could be written to address every
possible scenario involving liquids, gels, and aerosols. Therefore, TSA
enabled Supervisory TSOs to use their discretion, while also
considering security risks.
Source: GAO analysis of TSA documentation:
[A] The August 12, 2006, SOP change incorporates clarifications
implemented on August 16, 2006.
[End of table]
As TSA senior leadership obtained more information about the particular
threat posed by the liquid explosives through tests conducted by DHS's
Science and Technology Directorate and FBI, TSA relaxed the
restrictions to allow passengers to carry liquids, gels, and aerosols
onboard aircraft in 3-fluid-ounce bottles--and as of November 2006, 3.4-
fluid-ounce bottles--that would easily fit in a quart-sized, clear
plastic, zip-top bag. TSA senior leadership identified both benefits
and drawbacks to this SOP modification, but determined that the balance
of security, efficiency, and customer service that would result from
these SOP changes was appropriate. As shown in table 2, TSA officials
recognize that there are security drawbacks--or vulnerabilities--
associated with allowing passengers to carry even small amounts of
liquids and gels onboard aircraft. For example, two or more terrorists
could combine small amounts of liquid explosives after they pass
through the checkpoint to generate an amount large enough to possibly
cause catastrophic damage to an aircraft. However, TSA officials stated
that doing so would be logistically challenging given the physical harm
that the specific explosives could cause to the person handling them,
and that suspicion among travelers, law enforcement officials, and
airport employees would likely be raised if an individual was seen
combining the liquid contents of small containers stored in two or more
quart-sized plastic bags. TSA officials stated that at the time of the
modifications to the liquid, gels, and aerosols screening procedures,
there was consensus among explosives detection experts, both
domestically and abroad, regarding TSA's assumptions about how the
explosives could be used and the damage they could cause to an
aircraft.[Footnote 33] TSA officials also stated that after reviewing
the intelligence information related to the alleged August 2006 London
terror plot--particularly with regard to the capability and intent of
the terrorists--TSA determined that allowing small amounts of liquids,
gels, and aerosols onboard aircraft posed an acceptable level of risk
to the commercial aviation system.[Footnote 34] Moreover, TSA officials
acknowledged that there are vulnerabilities with allowing passengers to
carry liquids that are exempted from the 3.4-fluid-ounce limit--such as
baby formula and medication--onboard aircraft.
TSA officials stated that the enhancements TSA is making to the various
other layers of aviation security will help address the security
vulnerabilities identified above. For example, TSA has increased
explosives detection canine patrols, deployed Federal Air Marshals on
additional international flights, increased random screening of
passengers at boarding gates, and increased random screening of airport
and TSA employees who pass through the checkpoint. TSA also plans to
expand implementation of its Screening Passengers by Observation
Technique (SPOT) to additional airports. SPOT involves specially
trained TSOs observing the behavior of passengers and resolving any
suspicious behavior through casual conversation with passengers and
referring suspicious passengers to selectee screening.[Footnote 35] TSA
intends for SPOT to provide a flexible, adaptable, risk-based layer of
security that can be deployed to detect potentially high-risk
passengers based on certain behavioral cues.
TSA's Analysis of the Impact of Certain Proposed Screening Changes on
Security and TSO Resources Could Be Strengthened:
While professional judgment regarding risk factors, experience in the
operating environment, and customer feedback have guided many of the
decisions TSA leadership made about which screening procedures to
implement, TSA also sought to use empirical data as a basis for
evaluating the impact some screening changes could have on security and
TSO resources. The TSA Assistant Secretary stated in December 2005 that
TSA sought to make decisions about screening changes based on data and
metrics--a practice he said TSA would continue. The use of data and
metrics to inform TSA's decision making regarding implementing proposed
screening procedures is consistent with our prior work that has shown
the importance of data collection and analyses to support agency
decision making. Between October 2005 and January 2006, TSA's
Explosives Detection Improvement Task Force sought to collect data as
part of an effort to test the impact of seven proposed procedures at
selected airports, as noted earlier.[Footnote 36] These seven proposed
procedures were selected because officials believed they would have a
significant impact on how TSOs perform daily screening functions, TSO
training, and customer acceptability. According to TSA's chief of
security operations, the purpose of testing these procedures in the
airport environment was to ensure that TSA was "on the right path" in
implementing them. These particular procedures were considered by
senior TSA officials as especially important for enhancing the
detection of explosives and for deterring terrorists from attempting to
carry out an attack. According to TSA, some of the proposed procedures
could also free up TSOs so that they could spend more time on
procedures for detecting explosives and less time on procedures
associated with low security risks, such as identifying small scissors
in carry-on bags. The seven proposed procedures tested by the task
force reflect both new procedures and modifications to existing
procedures, as shown in table 3.
Table 3: Proposed Procedures Operationally Tested by the Explosives
Detection Improvement Task Force, October 2005-January 2006:
Title of proposed procedure: Screening Passengers by Observation
Technique[A];
New or revised procedure: New;
Previous procedure: N/A;
Proposed procedure: Designated TSOs will observe the behavioral
patterns of passengers, and based on their observations, TSOs will
conduct casual conversations, refer suspicious passengers to secondary
screening, and in some cases refer some individuals to law enforcement
officers.
Title of proposed procedure: Unpredictable Screening Process (USP);
New or revised procedure: Revised;
Previous procedure: Selectee, or additional, screening of passengers
must be conducted continuously. If the number of individuals that alarm
the walk-through metal detector or if the number of bags that alarm is
not enough to ensure continual additional screening, individuals and
bags must be randomly selected to meet this requirement;
Proposed procedure: Random selectee screening is to be replaced by the
USP, which entails random selection of passengers across two screening
lanes to be subjected to a predetermined element of the selectee
screening process. The specific elements are sensitive security
information and are not discussed in this report.
Title of proposed procedure: Bulk-item pat-down search;
New or revised procedure: Revised;
Previous procedure: The pat-down procedure included only the torso area
of the body;
Proposed procedure: The pat-down is to include not only the torso, but
also from the waistline down.
Title of proposed procedure: IED components search;
New or revised procedure: New;
Previous procedure: N/A;
Proposed procedure: TSOs are to implement additional measures if they
find an IED component, such as a battery, when screening.
Title of proposed procedure: Selectee screening changes;
New or revised procedure: Revised;
Previous procedure: There was a rigid set of procedures for resolving
alarms set off by selectees;
Proposed procedure: More flexibility is to be provided for resolving
alarms set off by selectees.
Title of proposed procedure: Threat area search;
New or revised procedure: Revised; Previous procedure: For bags that
appear to pose a security threat, various searches were conducted,
where some of the searches were not directly focused on the reason for
suspicion;
Proposed procedure: For bags that appear to pose a security threat, the
searches that are conducted are intended to focus more on the reason
for suspicion.
Title of proposed procedure: Prohibited items list changes;
New or revised procedure: Revised;
Previous procedure: Scissors (metal with pointed tips, except ostomy
scissors with pointed tips with an overall length, including blades and
handle, of 4 inches or less, when accompanied by an ostomate supply kit
containing related supplies, such as collection pouches, wafers,
positioning plates, tubing, or adhesives) and tools (including, but not
limited to, wrenches and pliers) were not permitted on aircraft;
Proposed procedure: Allow scissors with pointed tips and blades less
than 4 inches and tools less than 7 inches in length onto aircraft.
Source: TSA:
Note: N/A stands for "not applicable," meaning that no previous
procedure existed prior to the new procedure.
[A] Implementation of SPOT did not involve a revision to the passenger
checkpoint screening SOP; rather, TSA developed a separate set of
standard operating procedures for SPOT. However, we included SPOT in
our review because it modifies the way in which TSOs screen passengers
and their carry-on items at the checkpoint.
[End of table]
Our analysis of TSA's data collection and data analysis for the seven
procedures that were operationally tested identified several problems
that affected TSA's ability to determine whether these procedures, as
designed and implemented by TSA, would have the intended effect--to
enhance the detection of explosives during the passenger screening
process or to free up resources so that explosives detection procedures
could be implemented. Although the deterrence of persons intending to
do harm is also an intended effect of some proposed SOP modifications,
TSA officials said that it is difficult to assess the extent to which
implementation of proposed procedures would deter terrorists. The
Office of Management and Budget has also acknowledged the difficulty in
measuring deterrence, particularly for procedures intended to prevent
acts of terrorism. While we agree that measuring deterrence is
difficult, opportunities exist for TSA to strengthen its analyses to
help provide information on whether the proposed procedures would
enhance detection or free up TSO resources, when intended.
Screening Passengers by Observation Technique. TSA officials stated
that SPOT is intended to both deter terrorists and identify suspicious
persons who intend to cause harm while on an aircraft. While we
recognize that it is difficult to assess the extent to which terrorists
are deterred by the presence of designated TSOs conducting behavioral
observations at the checkpoint, we believe that there is an opportunity
to assess whether SPOT contributes to enhancing TSA's ability to detect
suspicious persons that may intend to cause harm on an aircraft. One
factor that may serve as an indicator that a person intends to do harm
on an aircraft is whether that individual is carrying a prohibited
item. TSA collected and assessed data at 14 airports for various time
periods on the number of prohibited items found on passengers who were
targeted under SPOT and referred to secondary screening or law
enforcement officials.[Footnote 37] However, these data collection
efforts, alone, did not enable TSA to determine whether the detection
of prohibited items would be enhanced if SPOT were implemented because
TSA had no means of comparing whether persons targeted by SPOT were
more likely to carry prohibited items than persons not targeted by
SPOT. To obtain this information, the task force would have had to
collect data on the number of passengers not targeted by SPOT that had
prohibited items on them. This information could be used to determine
whether a greater percentage of passengers targeted under SPOT are
found to have prohibited items than those passengers who are not
targeted by SPOT, which could serve as one indicator of the extent to
which SPOT would contribute to the detection of passengers intending to
cause harm on an aircraft.
Although it has not yet done so, it may be possible for TSA to evaluate
the impact of SPOT on identifying passengers carrying prohibited items.
There is precedent in other federal agencies for evaluating the
security benefits of similar procedures. For instance, U.S. Customs and
Border Protection (CBP) within DHS developed the Compliance Examination
(COMPEX) system to evaluate the effectiveness of its procedures for
selecting international airline passengers for secondary screening.
Specifically, COMPEX compares the percentage of targeted passengers on
which prohibited items are found to the percentage of randomly selected
passengers on which prohibited items are found. The premise is that
targeting is considered to be effective if a greater percentage of
targeted passengers are found to possess prohibited items than the
percentage of randomly selected passengers, and the difference between
the two percentages is statistically significant.[Footnote 38] CBP
officials told us in May 2006 that they continue to use COMPEX to
assess the effectiveness of their targeting of international airline
passengers.[Footnote 39] When asked about using a method such as COMPEX
to assess SPOT, TSA officials stated that CBP and TSA are seeking to
identify different types of threats through their targeting programs.
CBP, through its targeting efforts, is attempting to identify
passengers with contraband and unauthorized aliens, whereas TSA,
through SPOT, is attempting to identify potential high-risk passengers.
Additionally, in commenting on a draft of this report, DHS stated that,
according to TSA, the possession of a prohibited item is not a good
measure of SPOT effectiveness because an individual may not intend to
use a prohibited item to cause harm or hijack an aircraft. While it may
be possible for a terrorist to cause harm or hijack an aircraft without
using a prohibited item, as in the case of the September 11 terrorist
attacks,[Footnote 40] other terrorist incidents and threat information
identify that terrorists who carried out or planned to carry out an
attack on a commercial aircraft intended to do so by using prohibited
items, including explosives and weapons. Therefore, we continue to
believe that comparing the percentage of individuals targeted and not
targeted under SPOT on which prohibited items are found could be one of
several potential indicators of the effectiveness of SPOT. Such a
measure may be most useful with regard to the prohibited items that
could be used to bring down or hijack an aircraft. TSA officials stated
that the agency agrees in principle that measuring SPOT effectiveness,
if possible, may provide valuable insights.
Unpredictable Screening Process, Bulk-Item Pat-Down Search, and IED
Component Search. We found that the task force also could have
strengthened its efforts to evaluate the security impact of other
proposed procedures--specifically, USP, the bulk-item pat-down search,
and the IED component search. For all three of these procedures, the
task force did not collect any data during the operational testing that
would help determine whether they would enhance detection capability.
TSA officials told us that they did not collect these data because they
had a limited amount of time to test the procedures because they had to
make SOP modifications quickly as part of the agency's efforts to focus
on higher threats, such as explosives, and the TSA Assistant
Secretary's goal of implementing the SOP modifications before the 2005
Thanksgiving holiday travel season. Nevertheless, TSA officials
acknowledged the importance of evaluating whether proposed screening
procedures, including USP and the bulk-item pat-down, would enhance
detection capability. TSA officials stated that covert testing has been
used to assess TSOs' ability to detect prohibited items, but covert
testing was not implemented during operational testing of proposed
procedures. Office of Inspection officials questioned whether covert
testing could be used to test, exclusively, the security benefit of
proposed procedures, because TSO proficiency and the capability of
screening technology also factor into whether threat objects are
detected during covert tests. Four of the five aviation security
experts we interviewed acknowledged this limitation but stated that
covert testing is the best way to assess the effectiveness of passenger
checkpoint screening.[Footnote 41] In commenting on a draft of this
report, DHS stated that, according to TSA, USP is intended to disrupt
terrorists' planning of an attack by introducing unpredictability into
the passenger checkpoint screening process, and tools such as covert
testing could not be used to measure the effectiveness of USP to this
end. While we agree that covert testing may not be a useful tool to
assess the impact USP has on disrupting terrorists' plans and deterring
terrorists from attempting to carry out an attack, we continue to
believe that covert testing could have been used to assess whether USP
would have helped to enhance detection capability during the passenger
screening process, which TSA officials stated was another intended
result of USP.
Although TSA did not collect data on the security impact of the USP and
bulk-item pat-down procedures, the task force did collect data on the
impact these procedures had on screening efficiency--the time required
to perform procedures--and on the reaction of TSOs, FSDs, and
passengers to the proposed procedures. These data indicated that the
USP procedure took less time, on average, for TSOs to conduct than the
procedure it replaced (the random continuous selectee screening
process); the revised pat-down procedure took TSOs about 25 seconds to
conduct; and that passengers generally did not complain about the way
in which both procedures were conducted.
With respect to operational testing of the IED component search
procedure, TSA was unable to collect any data during the testing period
because no IEDs were detected by TSOs at the airports where the testing
took place. As with the USP and bulk-item pat-down procedures, TSA
could have conducted covert tests during the operational testing period
to gather simulated data for the IED search procedure, in the absence
of actual data.
Selectee Screening Changes and Threat Area Search. Recognizing that
some of the proposed procedures intended to enhance detection would
require additional TSO resources, TSA implemented several measures
aimed collectively at freeing up TSOs' time so that they could focus on
conducting more procedures associated with higher threats--identifying
explosives and suspicious persons. For example, TSA modified the
selectee screening procedure and the procedure for searching carry-on
items--the threat area search--in order to reduce screening time.
During an informal pilot of these proposed procedures at 3 airports in
November 2005, TSA determined that the proposed selectee screening
procedure would reduce search time of each selectee passenger, on
average, by about 1.17 minutes at these airports. TSA also determined
through this study that the proposed threat area search, on average,
took 1.83 minutes to conduct at the participating airports, as compared
to the existing target object search that took, on average, 1.89
minutes, and the existing whole bag search that took, on average, 2.37
minutes.
Prohibited Items List Changes. Another measure TSA implemented to free
up TSO resources to focus on higher threats involved changes to the
list of items prohibited onboard aircraft. According to TSA, TSOs were
spending a disproportionate amount of TSA's limited screening resources
searching for small scissors and small tools, even though, based on
threat information and TSA officials' professional judgment, such items
no longer posed a significant security risk given the multiple layers
of aviation security. TSA officials surmised that by not having to
spend time and resources physically searching passengers' bags for low-
threat items, such as small scissors and tools, TSOs could focus their
efforts on implementing more effective and robust screening procedures
that can be targeted at screening for explosives.
To test its assumption that a disproportionate amount of TSO resources
was being spent searching for small scissors and tools, TSA collected
information from several sources. First, TSA reviewed data maintained
in TSA's Performance Management Information System (PMIS),[Footnote 42]
which showed that during the third and fourth quarters of fiscal year
2005 (a 6-month period), TSOs confiscated a total of about 1.8 million
sharp objects other than knives or box cutters. These sharp objects
constituted 19 percent of all prohibited items confiscated at the
checkpoint. Second, based on information provided by FSDs, TSOs, and
other screening experts, TSA determined that scissors constituted a
large majority of the total number of sharp objects found at passenger
screening checkpoints. Third, TSA headquarters officials searched
through confiscated items bins at 4 airports and found that most of the
scissors that were confiscated had blades less than 4 inches in length.
Based on these collective efforts, TSA concluded that a significant
number of items found at the checkpoint were low-threat, easily
identified items, such as small scissors and tools, and that a
disproportionate amount of time was spent searching for these items--
time that could have been spent searching for high-threat items, such
as explosives. TSA also concluded that because TSOs can generally
easily identify scissors, if small scissors were no longer on the
prohibited items list, TSOs could avoid conducting time-consuming
physical bag searches to locate and remove these items.
While we commend TSA's efforts to supplement professional judgment with
data and metrics in its decision to modify passenger checkpoint
screening procedures, TSA did not conduct the necessary analysis of the
data collected to determine the extent to which the removal of small
scissors and tools from the prohibited items list could free up TSO
resources. Specifically, TSA did not analyze the data on sharp objects
confiscated at the checkpoint along with other relevant factors, such
as the amount of time taken to search for scissors and the number of
TSOs at the checkpoint conducting these searches, to determine the
extent to which TSO resources could actually be freed up. Based on our
analysis of TSA's data for the 6-month period, where we considered
these other relevant factors, we determined that TSOs spent, on
average, less than 1 percent of their time--about 1 minute per day over
the 6-month period--searching for the approximately 1.8 million sharp
objects, other than knives and box cutters, that were found at
passenger screening checkpoints between April 2005 and September
2005.[Footnote 43] If the average amount of time TSOs spent searching
for sharp objects per day over a 6-month period was less than 1 minute
per TSO, and sharp objects constituted just 19 percent of all
prohibited items confiscated at checkpoints over this period, then it
may not be accurate to assume that no longer requiring TSOs to search
for small scissors and tools would significantly contribute to TSA's
efforts to free up TSO resources that could be used to implement other
security measures.
To further support its assertion that significant TSO resources would
be freed up as a result of removing small scissors and tools from the
list of prohibited items, TSA officials cited the results of an
informal study conducted in October 2005--which was intended to provide
a general idea of the types of prohibited items TSOs were finding as a
result of their searches and how long various types of searches were
taking TSOs to conduct. Specifically, according to the study conducted
at 9 airports over a 14-day period, TSA determined that 24 percent of
items found during carry-on bag searches were scissors. However, based
on data regarding the number of bags searched, removing scissors may
not significantly contribute to TSA's efforts to free up TSO
resources.[Footnote 44]
TSA conducted additional informal studies 30, 60, and 90 days after the
prohibited items list change went into effect to determine whether the
change had resulted in reductions in the percentage of carry-on bags
that were searched and overall screening time. However, we identified
limitations in TSA's methodology for conducting these studies.[Footnote
45] In February 2007, a TSA official stated that some FSDs interviewed
several TSOs after the prohibited items list change went into effect,
and these TSOs reported that the change did save screening time.
However, TSA could not identify how many TSOs were interviewed, at
which airports the TSOs were located, and how the TSOs were selected
for the interview; nor did TSA document the results of these
interviews. TSA also did not use random selection or representative
sampling when determining which TSOs should be interviewed. Therefore,
the interview results cannot be generalized.
TSA officials acknowledged that they could have made some improvements
in the various analyses they conducted on the prohibited items list
change. However, they stated that they had to make SOP modifications
quickly as part of the agency's efforts to focus on higher threats,
such as explosives, and the TSA Assistant Secretary's goal of
implementing the SOP modifications before the 2005 Thanksgiving holiday
travel season. Additionally, officials stated that they continue to
view their decision to remove small scissors and tools from the
prohibited items list as sound, particularly because they believe small
scissors and tools do not pose a significant threat to aviation
security. TSA officials also stated that they believe the prohibited
items list change would free up resources based on various sources of
information, including the professional judgment of TSA airport staff,
and their analysis of PMIS data on prohibited items confiscated at
checkpoints. The TSA Assistant Secretary told us that even if TSA
determined that the proposed SOP modifications would not free up
existing TSO resources to conduct explosives detection procedures, he
would have implemented the modifications anyway considering the added
security benefit of the explosives detection procedures. Additionally,
a TSA headquarters official responsible for airport security operations
stated that to help strengthen the agency's analysis of future proposed
SOP changes, the agency plans to provide the Explosives Detection
Improvement Task Force with the necessary resources to help improve its
data collection and analysis.
An additional measure intended to free up TSO resources[Footnote 46]
involved changes to CAPPS rules.[Footnote 47] TSA's assumption is that
these changes could allow TSOs who were normally assigned to selectee
screening duties to be reassigned to new procedures, such as USP, which
may require new screening positions. (Both USP and SPOT require TSO
positions: USP requires one screening position for every two screening
lanes, while SPOT typically uses more than one screening position per
ticket checker at the checkpoint.[Footnote 48]):
According to FSDs we interviewed, the changes made to the prohibited
items list and the CAPPS rules had not freed up existing TSO resources,
as intended. Specifically, as of August 2006, 13 of 19 FSDs we
interviewed at airports that tested USP or SPOT said that TSO resources
were not freed up as a result of these changes. In addition, 9 of the
19 FSDs said that in order to operationally test USP or SPOT, TSOs had
to work overtime, switch from other functions (such as checked baggage
screening), or a screening lane had to be closed. TSA's Explosives
Detection Improvement Task Force reported that nearly all of the FSDs
at airports participating in operational testing of USP believed that
the procedure had security value,[Footnote 49] though the task force
also reported that 1 FSD dropped out of the operational testing program
for USP due to insufficient staffing resources and another could only
implement the procedure during off-peak travel periods. Additionally,
most of the FSDs we interviewed stated that the changes to the
prohibited items list and CAPPS rules did not free up TSOs, as
intended, to better enable TSOs to take required explosives detection
training. Specifically, as of August 2006, of the 19 FSDs we
interviewed at airports that implemented USP and SPOT, 13 said that
they did not experience more time to conduct explosives training as a
result of changes to the prohibited items list and CAPPS
rules.[Footnote 50] Three of the 13 FSDs said that they used overtime
to enable TSOs to take the explosives training. As previously stated,
the TSA Assistant Secretary stated that even if existing TSO resources
are not freed up to conduct explosives detection procedures, longer
lines and wait times at airport checkpoints are an acceptable
consequence, considering their added security benefit. With regard to
explosives training, he stated that it is acceptable for FSDs to use
overtime or other methods to ensure that all TSOs participated in the
required explosives detection training. He further noted that even if
one screening change does not free up TSO resources, all of the changes
intended to accomplish this--when taken together--should ultimately
help to redirect TSO resources to where they are most needed.
TSA's efforts to add data and metrics to its tool kit for evaluating
the impact of screener changes are a good way to supplement the use of
professional judgment and input from other experts and sources in
making decisions about modifying screening procedures. However, TSA's
methods for data collection and analysis could be improved. We
recognize the challenges TSA faces in evaluating the effectiveness of
proposed procedures, particularly when faced with time pressures to
implement procedures. However, by attempting to evaluate the potential
impact of screening changes on security and resource availability, TSA
could help support its decision making on how best to allocate limited
TSO resources and ensure that the ability to detect explosives and
other high-threat objects during the passenger screening process is
enhanced.
Documentation of the Reasoning behind Proposed SOP Modifications Was
Incomplete:
While we were able to assess TSA's reasoning behind certain proposed
SOP modifications considered during our review period, our analysis was
limited because TSA did not maintain complete documentation of proposed
SOP modifications. Documentation of the reasoning behind decisions to
implement or reject proposed modifications was maintained in various
formats, including spreadsheets developed by TSA officials, internal
electronic mail discussions among TSA officials, internal memorandums,
briefing slides, and reports generated based on the results of
operational testing. TSA did improve its documentation of the proposed
SOP modifications that were considered during the latter part of our 9-
month review period. Specifically, the documentation for the SOP
modifications proposed under the Explosives Detection Improvement Task
Force provided more details regarding the basis of the proposed
modifications and the reasoning behind decisions to implement or reject
the proposed modifications.
Of the 92 proposed SOP modifications considered during our 9-month
review period that TSA documented, TSA provided the basis for 72. More
specifically, TSA documented the basis--that is, the information,
experience, or event that encouraged TSA officials to propose an SOP
modification--for 35 of the 48 that were implemented and for 37 of the
44 that were not implemented. However, TSA only documented the
reasoning behind TSA senior leadership's decisions to implement or not
implement proposed SOP modifications for 43 of 92 proposed
modifications. According to TSA officials, documentation that explains
the basis for recommending proposed modifications can also be used to
explain TSA's reasoning behind its decisions to implement proposed
modifications. However, the basis on which an SOP modification was
proposed cannot always be used to explain TSA senior leadership's
decisions not to implement a proposed modification. In these cases,
additional documentation would be needed to understand TSA's decision
making. However, TSA only documented the reasoning behind its decisions
for about half (26 of 44) of the proposed modifications that were not
implemented. TSA officials told us that they did not intend to document
all SOP modifications that were proposed during our review period.
Officials stated that, in some cases, the reasoning behind TSA's
decision to implement or not implement a proposed SOP modification is
obvious and documentation is not needed. TSA officials acknowledged
that it is beneficial to maintain documentation on the reasoning behind
decisions to implement or reject proposed SOP modifications deemed
significant, particularly given the organizational restructuring and
staff turnover within TSA.[Footnote 51] However, TSA officials could
not identify which of the 92 proposed SOP modifications they consider
to be significant because they do not categorize proposed modifications
in this way.
Our standards for governmental internal controls and associated
guidance suggest that agencies should document key decisions in a way
that is complete and accurate, and that allows decisions to be traced
from initiation, through processing, to after completion.[Footnote 52]
These standards further state that documentation of key decisions
should be readily available for review. Without documenting this type
of information, TSA cannot always justify significant modifications to
passenger checkpoint screening procedures to internal or external
stakeholders, including Congress and the traveling public. In addition,
considering the ongoing personnel changes, without sufficient
documentation, future decision makers in TSA may not know on what basis
the agency historically made decisions to develop new or revise
existing screening procedures.
Following our 9-month review period, TSA continued to make efforts to
improve documentation of agency decision making, as evidenced by
decisions regarding the August 2006 and September 2006 SOP
modifications related to the screening of liquids and gels. For
example, TSA senior leadership evaluated the actions taken by the
agency between August 7 and August 13, 2006, in response to the alleged
liquid explosives terrorist plot, in order to identify lessons learned
and improve the agency's reaction to future security incidents. As a
result of this evaluation, as shown in table 4, TSA made several
observations and recommendations for improving documentation of agency
decision making when considering modifications to screening procedures.
Table 5: TSA Evaluation of Documentation of Agency Decisions Made
between August 7 and August 13, 2006, Regarding the Alleged Liquid
Explosives Terrorist Plot:
Observations: There was no tracking of the overall timing and progress
of deliberations of the various decision options;
Recommendations for improvement: Track and record key issues raised and
the timing of deliberations.
Observations: There was no formal tracking of the decision options that
were discussed or the rationale that was used when selecting among the
various decision options;
Recommendations for improvement: Formally document options discussed,
decisions made, and the rationale behind the decisions.
Observations: There were no formal requirements for the type of
information that needed to be documented or the format used to document
the information on agency decisions;
Recommendations for improvement: For each decision that is made,
standardize the type of information that should be documented and
develop an appropriate mechanism to store the information.
Observations: The documentation that was used to support agency
decisions did not contain basic audit trail information, such as the
origin of the document and how the document was used. This may prevent
decision makers from understanding the relevancy of the documentation
to agency decisions;
Recommendations for improvement: For each document used to support
agency decisions, identify the origin of the document and how the
document was used by decision makers.
Source: TSA.
[End of table]
Documentation of TSA's decisions regarding the September 26, 2006,
modifications to the liquid screening procedures showed that TSA had
begun implementing the recommendations in table 4. TSA's documentation
identified the various proposed liquid screening procedures that were
considered by TSA, the benefits and drawbacks of each proposal, and the
rationale behind TSA's final decision regarding which proposal to
implement. The documentation also tracked the timing of TSA's
deliberations of each of the proposed liquid screening procedures.
However, the documentation of TSA's decisions was not always presented
in a standard format, nor was the origin and use of supporting
documentation always identified. TSA officials acknowledged that
documentation of the September 2006 SOP modifications could have been
improved and stated that efforts to improve documentation, through
implementation of the recommendations in table 4, will continue to be a
high priority.
TSA Has Several Methods in Place to Monitor TSO Compliance with
Passenger Checkpoint Screening SOPs:
A New Performance Accountability System Helps TSA Monitor TSO
Compliance with SOPs:
TSA implemented a performance accountability system in part to
strengthen its monitoring of TSO compliance with passenger checkpoint
screening SOPs. Specifically, in April 2006, TSA implemented the
Performance Accountability and Standards System (PASS) to assess the
performance of all TSA employees, including TSOs.[Footnote 53]
According to TSA officials, PASS was developed in response to our 2003
report that recommended that TSA establish a performance management
system that makes meaningful distinctions in employee
performance,[Footnote 54] and in response to input from TSA airport
staff on how to improve passenger and checked baggage screening
measures. With regard to TSOs, PASS is not intended solely to measure
TSO compliance with SOPs. Rather, PASS will be used by TSA to assess
agency personnel at all levels on various competencies, including
training and development, readiness for duty, management skills, and
technical proficiency.
There are three elements of the TSO technical proficiency component of
PASS that are intended to measure TSO compliance with passenger
checkpoint screening procedures: (1) quarterly observations conducted
by FSD management staff of TSOs' ability to perform particular
screening functions in the operational environment, such as pat-down
searches and use of the hand-held metal detector, to ensure they are
complying with checkpoint screening SOPs; (2) quarterly quizzes given
to TSOs to assess their knowledge of the SOPs; and (3) an annual,
multipart knowledge and skills assessment. While the first two elements
are newly developed, the third element--the knowledge and skills
assessment--is part of the annual TSO recertification program that is
required by the Aviation and Transportation Security Act (ATSA) and has
been in place since October 2003.[Footnote 55] Collectively, these
three elements of PASS are intended to provide a systematic method for
monitoring whether TSOs are screening passengers and their carry-on
items according to SOPs. TSA's implementation of PASS is consistent
with our internal control standards, which state that agencies should
ensure that policies and procedures are applied properly.[Footnote 56]
The first component of PASS (quarterly observations) is conducted by
screening supervisors or screening managers, using a standard checklist
developed by TSA headquarters, with input from TSA airport staff. There
is one checklist used for each screening function, and TSOs are
evaluated on one screening function per quarter. For example, the hand-
held metal detector skills observation checklist includes 37 tasks to
be observed, such as whether the TSO conducted a pat-down search to
resolve any suspect areas. The second component of PASS (quarterly
quizzes) consists of multiple-choice questions on the standard
operating procedures. For example, one of the questions on the PASS
quiz is "What is the correct place to start an HHMD outline [a hand-
held metal detector search] on an individual: (a) top of the head, (b)
top of the feet, or (c) top of the shoulder?"
The third component of PASS is the annual knowledge and skills
assessment, a component of the annual recertification program that
evaluates the technical proficiency of TSOs. This assessment is
composed of three modules: (1) knowledge of standard operating
procedures, (2) recognition of threat objects on an X-ray image, and
(3) demonstration of screening functions. According to TSA officials,
while recertification testing is not a direct measure of operational
compliance with passenger checkpoint screening SOPs, recertification
testing, particularly module 1 and module 3, is an indicator of whether
TSOs are capable of complying with SOPs. TSA officials stated that if a
TSO does not have knowledge of SOPs and if the TSO cannot demonstrate
basic screening functions as outlined in the SOPs, then the TSO will
likely not be able to comply with SOPs when performing in the operating
environment. Table 5 provides a summary of each of these modules.
Table 6: Modules Included in Recertification Knowledge and Skills
Assessment:
Testing module: Knowledge of standard operating procedures;
Description: Computerized 50-question multiple-choice test. It is
either passenger-or baggage-specific.
Testing module: Image recognition;
Description: Computerized test that consists of 100 images and is used
to evaluate a TSO's skill and ability in detecting threat or prohibited
objects within X-ray images.
Testing module: Practical demonstration of skills;
Description: Hands- on simulated work sample to evaluate a TSO's
knowledge, skills, and ability when performing specific screening tasks
along with ability to provide customer service.
Source: TSA.
[End of table]
FSDs we interviewed reported that they have faced resource challenges
in implementing PASS. Specifically, as of July 2006, 9 of 24 FSDs we
interviewed said they experienced difficulties in implementing PASS due
to lack of available staff to conduct the compliance-related
evaluations. TSA officials stated that they have automated many of the
data-entry functions of PASS to relieve the field of the burden of
manually entering this information into the PASS online system. For
example, all scores related to the quarterly quiz and skill observation
components are automatically uploaded, and PASS is linked to TSA's
online learning center database to eliminate the need to manually enter
TSOs' learning history. In addition, the TSA Assistant Secretary said
that FSDs were given the option of delaying implementation of PASS if
they were experiencing resource challenges.
TSA Uses Local and National Covert Testing, in Part, to Assess TSO
Compliance with SOPs:
TSA also conducts local and national covert tests, which are used to
evaluate, in part, the extent to which noncompliance with the SOPs
affects TSOs' ability to detect simulated threat items hidden in
accessible property or concealed on a person. TSA first issued guidance
on its local covert testing program--known as Screener Training
Exercises and Assessments (STEA)--in February 2004. STEA testing is
conducted by FSD staff at airports, who determine the frequency at
which STEA tests are conducted as well as which type of STEA tests are
conducted. According to the STEA results reported by TSA between March
2004 and February 2006,[Footnote 57] TSOs' noncompliance with the SOP
accounted for some of the STEA test failures.[Footnote 58] TSOs' lack
of proficiency in skills or procedures, which may affect TSOs' ability
to comply with procedures, was also cited as the reason for some of the
STEA test failures. TSOs who fail STEA tests are required to take
remedial training to help them address the reasons for their failure.
FSDs we interviewed reported that they have faced resource challenges
in conducting STEA tests. Specifically, even though all 24 FSDs we
interviewed as of July 2006 said that they have conducted STEA tests,
10 of these FSDs said that the lack of available staff made it
difficult to conduct these tests. When asked how they planned to
address FSDs' concerns regarding a lack of available staff to complete
STEA tests, TSA headquarters officials told us that they are
considering resource alternatives for implementing the STEA program,
but could not provide us with the specific details of these
plans.[Footnote 59] Until the resource limitations that have restricted
TSA's use of its compliance monitoring tools have been fully addressed,
TSA may not have assurance that TSOs are screening passengers according
to the SOP.
As previously discussed, TSA's Office of Inspection initiated its
national covert testing program in September 2002. National covert
tests are conducted by TSA headquarters-based inspectors who carry
simulated threat objects hidden in accessible property or concealed on
their person through airport checkpoints, and in cases where TSOs fail
to detect threat objects, the inspectors identify the reasons for
failure. During September 2005, TSA implemented a revised covert
testing program to focus more on catastrophic threats--threats that can
bring down or destroy an aircraft. According to Office of Inspection
officials, TSOs may fail to detect threat objects during covert testing
for various reasons, including limitations in screening technology,
lack of training, limitations in the procedures TSOs must follow to
conduct passenger and bag searches, and TSOs' noncompliance with
screening checkpoint SOPs. Office of Inspection officials also said
that one test could be failed due to multiple factors, and that it is
difficult to determine the extent to which any one factor contributed
to the failure. TSOs who fail national covert tests, like those who
fail STEA tests, are also required to take remedial training to help
them address the reasons for failure.[Footnote 60]
Conclusions:
The alleged August 2006 terrorist plot to detonate liquid explosives
onboard multiple U.S.-bound aircraft highlighted the need for TSA to
continuously reassess and revise, when deemed appropriate, existing
passenger checkpoint screening procedures to address threats against
the commercial aviation system. In doing so, TSA faces the challenge of
securing the aviation system while facilitating the free movement of
people. Passenger screening procedures are only one element that
affects the effectiveness and efficiency of the passenger checkpoint
screening system. Securing the passenger checkpoint screening system
also involves the TSOs who are responsible for conducting the screening
of airline passengers and their carry-on items, and the technology used
to screen passengers and their carry-on items.
We believe that TSA has implemented a reasonable approach to modifying
passenger checkpoint screening procedures through its consideration of
risk factors (threat and vulnerability information), day-to-day
experience of TSA airport staff, and complaints and concerns raised by
passengers and by making efforts to balance security, efficiency, and
customer service. We are also encouraged by TSA's efforts to conduct
operational testing and use data and metrics to support its decisions
to modify screening procedures. We acknowledge the difficulties in
assessing the impact of proposed screening procedures, particularly
with regard to the extent to which proposed procedures would deter
terrorists from attempting to carry out an attack onboard a commercial
aircraft. However, there are existing methods, such as covert testing
and CBP's COMPEX--a method that evaluates the effectiveness of CBP's
procedures for selecting international airline passengers for secondary
screening--that could be used by TSA to assess whether proposed
screening procedures enhance detection capability. It is also important
for TSA to fully assess available data to determine the extent to which
TSO resources would be freed up to perform higher-priority procedures,
when this is the intended effect. Without collecting the necessary data
or conducting the necessary analysis that would enable the agency to
assess whether proposed SOP modifications would have the intended
effect, it may be difficult for TSA to determine how best to improve
TSOs' ability to detect explosives and other high-threat items and to
allocate limited TSO resources. With such data and analysis, TSA would
be in a better position to justify its SOP modifications and to have a
better understanding of how the changes affect TSO resources.
Additionally, because TSA did not always document the basis on which
SOP modifications were proposed or the reasoning behind decisions to
implement or not implement proposed modifications, TSA may not be able
to justify SOP modifications to Congress and the traveling public.
While we are encouraged that TSA's documentation of its decisions
regarding the SOP modifications made in response to the alleged August
2006 liquid explosives terrorist plot was improved compared to earlier
documentation, it is important for TSA to continue to work to
strengthen its documentation efforts. Such improvements would enable
TSA officials responsible for making SOP decisions in the future to
understand how significant SOP decisions were made historically--a
particular concern considering the restructuring and staff turnover
experienced by TSA.
As shown by TSA's covert testing results, the effectiveness of
passenger checkpoint screening relies, in part, on TSOs' compliance
with screening procedures. We are, therefore, encouraged by TSA's
efforts to strengthen its monitoring of TSO compliance with passenger
screening procedures. We believe that TSA has implemented a reasonable
process for monitoring TSO compliance and that this effort should
assist TSA in providing reasonable assurance that TSOs are screening
passengers and their carry-on items according to screening procedures.
Given the resource challenges FSDs identified in implementing the
various methods for monitoring TSO compliance, it will be important for
TSA to take steps, such as automating PASS data entry functions, to
address such challenges.
Recommendations for Executive Action:
To help strengthen TSA's evaluation of proposed modifications to
passenger checkpoint screening SOPs and TSA's ability to justify its
decisions to implement or not implement proposed SOP modifications, in
the March 2007 report that contained sensitive security information, we
recommended that the Secretary of Homeland Security direct the
Assistant Secretary of Homeland Security for TSA to take the following
two actions:
* when operationally testing proposed SOP modifications, develop sound
evaluation methods, when possible, that can be used to assist TSA in
determining whether proposed procedures would achieve their intended
result, such as enhancing TSA's ability to detect prohibited items and
suspicious persons and freeing up existing TSO resources that could be
used to implement proposed procedures, and:
* for future proposed SOP modifications that TSA senior leadership
determines are significant, generate and maintain documentation to
include, at minimum, the source, intended purpose, and reasoning behind
decisions to implement or not implement proposed modifications.
Agency Comments and Our Evaluation:
On March 6, 2007, we received written comments on the draft report,
which are reproduced in full in appendix III. DHS generally concurred
with our recommendations and outlined actions TSA plans to take to
implement the recommendations.
DHS stated that it appreciates GAO's conclusion that TSA has
implemented a reasonable approach to modifying passenger checkpoint
screening procedures through its assessment of risk factors, the
expertise of TSA employees, and input from the traveling public and
other stakeholders, as well as TSA's efforts to balance security,
operational efficiency, and customer service while evaluating proposed
changes.
With regard to our recommendation to develop sound evaluation methods,
when possible, to help determine whether proposed SOP modifications
would achieve their intended result, DHS stated that TSA plans to make
better use of generally accepted research design principles and
techniques when operationally testing proposed SOP modifications. For
example, TSA will consider using random selection, representative
sampling, and control groups in order to isolate the impact of proposed
SOP modifications from the impact of other variables. DHS also stated
that TSA's Office of Security Operations is working with subject matter
experts to ensure that operational tests are well designed and
executed, and produce results that are scientifically valid and
reliable. As discussed in this report, employing sound evaluation
methods for operationally testing proposed SOP modifications will
enable TSA to have better assurance that new passenger checkpoint
screening procedures will achieve their intended purpose, which may
include improved allocation of limited TSO resources and enhancing
detection of explosives and other high-threat objects during the
passenger screening process. However, DHS stated, and we agree, that
the need to make immediate SOP modifications in response to imminent
terrorist threats may preclude operational testing of some proposed
modifications.
Concerning our recommendation regarding improved documentation of
proposed SOP modifications, DHS stated that TSA intends to document the
source, intent, and reasoning behind decisions to implement or reject
proposed SOP modifications that TSA senior leadership deems
significant. Documenting this type of information will enable TSA to
justify significant modifications to passenger checkpoint screening
procedures to internal and external stakeholders, including Congress
and the traveling public. In addition, considering the ongoing
personnel changes TSA has experienced, such documentation should enable
future decision makers in TSA to understand on what basis the agency
historically made decisions to develop new or revise existing screening
procedures.
In addition to commenting on our recommendations, DHS provided comments
on some of our findings, which we considered and incorporated in the
report where appropriate. One of DHS's comments pertained to TSA's
evaluation of the prohibited items list change. Specifically, while TSA
agrees that the agency could have conducted a more methodologically
sound evaluation of the impact of the prohibited items list change, TSA
disagrees with our assessment that the prohibited items list change may
not have significantly contributed to TSA's efforts to free up TSO
resources to focus on detection of high-threat items, such as
explosives. As we identified in this report, based on interviews with
FSDs, airport visits to determine the types of items confiscated at
checkpoints, and a study to determine the amount of time taken to
conduct bag searches and the number of sharp objects collected as a
result of these searches, TSA concluded that the prohibited items list
change would free up TSO resources. DHS also stated that interviews
with TSOs following the prohibited items list change confirmed that the
change had freed up TSO resources. However, based on our analysis of
the data TSA collected both prior to and following the prohibited items
list change, we continue to believe that TSA did not conduct the
necessary analysis to determine the extent to which the removal of
small scissors and tools from the prohibited items list would free up
TSA resources.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 21 days
from the date of this report. At that time, we will send copies of the
report to the Secretary of the Department of Homeland Security, the TSA
Assistant Secretary, and interested congressional committees as
appropriate. We will also make copies available to others on request.
If you or your staff have any questions about this report, please
contact me at (202) 512-3404 or berrickc@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff that made major
contributions to this report are listed in appendix IV.
Signed by:
Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
[End of section]
Appendix I: Objectives, Scope and Methodology:
To assess the Transportation Security Administration's (TSA) process
for modifying passenger checkpoint screening procedures and how TSA
monitors compliance with these procedures, we addressed the following
questions: (1) How and on what basis did TSA modify passenger screening
procedures and what factors guided the decisions to do so? (2) How does
TSA determine whether TSOs are complying with the standard procedures
for screening passengers and their carry-on items?
To address how TSA modified passenger screening procedures and what
factors guided the decisions to do so, we obtained and analyzed
documentation of proposed standard operating procedures (SOP) changes
considered between April 2005 and September 2005, as well as threat
assessments and operational studies that supported SOP
modifications.[Footnote 61] The documentation included a list of
proposed changes considered, as well as the source, the intended
purpose, and in some cases the basis for recommending the SOP
modification--that is, the information, experience, or event that
encouraged TSA officials to propose the modifications--and the
reasoning behind decisions to implement or reject proposed SOP
modifications. We also obtained documentation of the proposed SOP
changes considered by TSA's Explosives Detection Improvement Task
Force, which was the deliberating body for proposed changes that were
considered between October 2005 and December 2005. We also reviewed and
analyzed similar documentation for proposed SOP modifications
considered between August 2006 and November 2006 in response to the
alleged terrorist plot to detonate liquid explosives onboard multiple
aircraft en route from the United Kingdom to the United States. We
included modifications to passenger checkpoint screening procedures
related to this particular event because they provided the most recent
information available of TSA's approach to modifying screening
procedures in response to an immediate perceived threat to civil
aviation. The documentation included notes from internal meetings,
slides for internal and external briefings on proposed SOP
modifications, data on customer complaints and screening efficiency,
and the results of liquid explosives testing conducted by the
Department of Homeland Security (DHS) Science and Technology
Directorate and the Federal Bureau of Investigation (FBI). We also
obtained each revision of the passenger checkpoint screening SOP that
was generated between April 2005 and December 2005 and August 2006 and
November 2006,[Footnote 62] as well as accompanying documentation that
highlighted all of the changes made in each revision. In addition, we
met with TSA headquarters officials who were involved in the process
for determining whether proposed passenger checkpoint screening
procedures should be implemented. We also met with officials in the DHS
Science and Technology Directorate as well as the FBI to discuss the
methodology and results of their liquid explosives tests, which were
used to support TSA's decisions to modify the SOP in September 2006. We
also met with TSA Office of Inspection and DHS Office of Inspector
General staff to discuss their covert testing at passenger checkpoints
and the recommended changes to the passenger checkpoint screening SOP
that were generated based on testing results. We also obtained and
analyzed data and information collected by TSA on the proposed
procedures that were evaluated in the operational environment. In
addition, we met or conducted phone interviews with Federal Security
Directors (FSD) and their management staff, including Assistant FSDs
and Screening Managers, and Transportation Security Officers (TSO) with
passenger screening responsibilities, at 25 commercial airports to gain
their perspectives on TSA's approach to revising the passenger
checkpoint screening SOP. We also met with officials from four aviation
associations--the American Association of Airport Executives, Airports
Council International, the Air Transport Association, and the Regional
Airline Association--to gain their perspectives on this objective.
Finally, we met with five aviation security experts to obtain their
views on methods for assessing the impact of proposed passenger
checkpoint screening procedures. We selected these experts based on
their depth of experience in the field of aviation security, employment
history, and their recognition in the aviation security community.
However, the views of these experts may not necessarily represent the
general view of other experts in the field of aviation security. We
compared TSA's approach to revising its passenger checkpoint screening
SOP with the Comptroller General's standards for internal control in
the federal government[Footnote 63] and risk management guidance.
To address how TSA determines whether TSOs are complying with the
standard procedures for screening passengers and their carry-on items,
we obtained documentation of compliance-related initiatives, including
guidance, checklists, and SOP quizzes used to assess TSO compliance
under the Performance Accountability and Standards System (PASS), and
guidance provided to FSDs for developing local compliance audit
programs. We also obtained the fiscal year 2005 recertification and
Screener Training Exercises and Assessments (STEA) test results, which
were used, in part, to assess TSO compliance with and knowledge of the
passenger checkpoint screening SOP. In addition, we reviewed the
results of covert testing conducted by TSA's Office of Inspection,
which were also used, in part, to assess TSO compliance with the
passenger checkpoint screening SOP. We assessed the reliability of the
compliance-related data we received from TSA, and found the data to be
sufficiently reliable for our purposes. In addition, we interviewed TSA
headquarters officials who were responsible for overseeing efforts to
monitor TSO compliance with standard operating procedures. This
included officials in the Office of Security Operations, Office of
Human Capital, and the Office of Operational Process and Technology.
Our audit work also included visits to or phone conferences with 25
airports, where we interviewed FSDs, members of their management teams,
and Transportation Security Officers with passenger screening
responsibilities.[Footnote 64] However, the perspectives of these FSDs
and their staff cannot be generalized across all airports. In July
2006, we submitted two sets of follow-up questions to FSD staff,
related to their experiences with implementing PASS and STEA tests. We
also obtained documentation of local compliance audit programs from the
FSD staff at several of these airports. We compared TSA's approach for
monitoring TSO compliance with the Comptroller General's standards for
internal control in the federal government.[Footnote 65]
As previously mentioned, we conducted site visits and/or phone
interviews at 25 airports[Footnote 66] (8 category X airports, 7
category I airports, 4 category II airports, 4 category III airports,
and 2 category IV airports) to discuss issues related to TSA's approach
to revising the passenger checkpoint screening SOP, and the agency's
approach to monitoring TSO compliance with the SOP.[Footnote 67] We
visited 7 of these airports during the design phase of our study. These
airports were selected based on variations in size and geographic
location, and whether they were operationally testing any proposed
passenger checkpoint screening procedures or passenger screening
technology. We also selected 2 airports that participated in the
Screening Partnership Program.[Footnote 68]
After visiting the 7 airports during the design phase of our review, we
selected an additional 15 airports to visit based on variations in
size, geographic distribution, and performance on compliance-related
assessments. Specifically, we obtained and analyzed fiscal year 2005
Screener Training Exercise and Assessments results and fiscal year 2005
recertification testing results to identify airports across a range of
STEA and recertification scores. Additionally, we visited 3 additional
airports that operationally tested the proposed Unpredictable Screening
Process (USP) and the Screening Passengers by Observation Technique
(SPOT) procedure.
In July 2006, we received from 19 FSDs answers to follow-up questions
on their experiences with implementing pilot testing of SPOT or USP.
This included 14 FSDs that were not part of our initial rounds of
interviews. Nine of these 14 FSDs were from airports that participated
in SPOT pilots. The remaining 5 of 14 FSDs that were not part of our
initial rounds of interviews were from airports that were participants
in USP pilots.
We conducted our work from March 2005 through March 2007 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Sources of SOP Changes:
Of the 92 proposed screening changes considered by TSA between April
2005 and December 2005, 63 were submitted by TSA field staff, including
Federal Security Directors and Transportation Security
Officers.[Footnote 69] Thirty proposed screening changes were submitted
by TSA headquarters officials. Last, TSA senior leadership, such as the
TSA Assistant Secretary, recommended 5 of the 92 proposed screening
changes considered during this time period. One SOP modification was
also proposed through a congressional inquiry. TSA's solicitation of
input from both field and headquarters officials regarding changes to
the passenger checkpoint screening SOP was consistent with internal
control standards,[Footnote 70] which suggest that there be mechanisms
in place for employees to recommend improvements in operations.
The FSDs with whom we met most frequently identified periodic
conference calls with the Assistant Secretary, the SOP Question and
Answer mailbox, or electronic mail to Security Operations officials as
the mechanisms by which they recommended changes to the SOP. The TSOs
with whom we met identified their chain of command and the SOP Question
and Answer mailbox as the primary mechanisms by which they submitted
suggestions for new or revised procedures. According to TSA officials,
the SOP mailbox entails FSDs and their staff, including TSOs,
submitting suggestions, questions, or comments to TSA's Security
Operations division via electronic mail, either directly or through
their supervisors. Submissions are then compiled and reviewed by a
single Security Operations official, who generates responses to the
questions that have clear answers. However, for submissions for which
the appropriate response is not obvious or for submissions that include
a suggestion to revise the SOP, this official forwards the submissions
to other Security Operations officials for further deliberation. SOP
mailbox responses are provided to all TSA airport officials. If TSA
headquarters revised a screening procedure based on a mailbox
submission, the revision is noted in the mailbox response.
Thirty of the screening changes considered by TSA between April 2005
and December 2005 were proposed by TSA headquarters officials,
including Security Operations officials, who are responsible for
overseeing implementation of checkpoint screening. According to
Security Operations officials, they recommended changes to checkpoint
screening procedures based on communications with TSA field officials
and airport optimization reviews. Security Operations officials conduct
optimization reviews to identify best practices and deficiencies in the
checkpoint screening and checked baggage screening processes. As part
of these reviews, Security Operations officials may also assess
screening efficiency and whether TSOs are implementing screening
procedures correctly.
Other TSA headquarters divisions also suggested changes to passenger
checkpoint screening procedures. For example, the Office of Law
Enforcement recommended that there be an alternative screening
procedure for law enforcement officials who are escorting prisoners or
protectees. Previously, all armed law enforcement officers were
required to sign a logbook at the screening checkpoint, prior to
entering the sterile area of the airport. The officials in the Office
of Passengers with Disabilities also recommended changes to checkpoint
screening procedures. For example, in the interest of disabled
passengers, they suggested that TSOs be required to refasten all
wheelchair straps and buckles undone during the screening process.
Last, TSA senior leadership suggested 5 of the 92 procedural changes
considered by TSA between April 2005 and December 2005. TSA senior
leadership also proposed a procedure that would allow TSOs to conduct
the pat-down procedure on passengers of the opposite gender at airports
with a disproportionate ratio of male and female TSOs.
[End of section]
Appendix III: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
March 6, 2007:
Ms. Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
U.S. Government Accountability Office:
441 G. Street, NW:
Washington, DC 20548:
Dear Ms. Berrick:
Thank you for the opportunity to comment on the draft report: "Risk,
Experience, and Customer Concerns Drive Changes to Airline Passenger
Screening Procedures, but Evaluation and Documentation of Proposed
Changes Could be Improved (GAO-07-57SU)." The Department of Homeland
Security (DHS) and Transportation Security Administration (TSA)
appreciate GAO's work in planning, conducting, and issuing this report.
The Department and TSA appreciates GAO's conclusion that we have
implemented a reasonable approach to modifying passenger checkpoint
screening procedures through our assessment of risk factors, the
expertise of TSA employees, and input from the traveling public and
other stakeholders, as well as our efforts to balance security,
operational efficiency, and customer service while evaluating proposed
changes. In its report, GAO recommends that TSA improve its methods to
evaluate and document changes to passenger screening procedures. TSA
agrees that there is opportunity for improvement and will continue to
take steps to use sound evaluation methods, when possible, to assess
the impact of proposed changes. However, the urgency and nature of some
procedural changes may not always lend themselves to a resource
consuming (time and personnel) evaluation.
Above all, TSA must remain flexible and able to respond quickly to
disrupt new terrorist threats. TSA demonstrated such flexibility in
August 2006, when the challenge presented by the United Kingdom
Terrorist Bomb Plot caused us to quickly and effectively implement
important changes to airport security. In just a few hours, literally
overnight, TSA rolled out a new airport security checkpoint process at
every airport in the Nation. We trained tens of thousands of
Transportation Security Officers (TSOs) and rewrote dozens of
regulations affecting aviation security around the globe. In addition,
TSA effected changes world-wide for every flight bound for the United
States and silently (as always) deployed hundreds of Federal Air
Marshals to saturate affected flights flown by U.S. carriers. TSA
accomplished all of this while maintaining effective security - that
demonstrates flexibility. TSA will continue to be challenged by other
terrorist plots and will be prepared for the unknown new threat as well
as address all the known threats; this puts a priority on flexibility.
In December 2005, TSA also demonstrated the flexibility necessary to
protect the Nation's transportation system by making small but
important changes to the Prohibited Items List, which allowed TSOs to
focus more effort on detecting high-risk threats which have the ability
to cause catastrophic damage to an airplane in flight (e.g., Improvised
Explosive Devices [IEDs]). GAO, in its report, states that TSA's data
collection and analysis efforts, in this case, do not allow TSA to
conclude with certainty that these changes did indeed free-up TSO
resources. TSA disagrees as our decision to remove small scissors and
small tools from the Prohibited Items List was not only based on an
analysis of data but was also firmly rooted in our assessment of risk,
professional judgment, and experience. For example, when considering
the changes, experienced Federal Security Directors (FSDs) who were
interviewed unanimously indicated the changes would free up TSO
resources.
To further support this theory, TSA Headquarters officials visited
several airports, to observe first hand, the type of items surrendered
by passengers. TSA also considered standard values for how long it
takes to conduct a bag search along with data indicating how many sharp
items are collected. While this particular data collection and analysis
effort may not have been methodologically rigorous, it did serve to
provide us with insights regarding the type and quantity of items
collected at the passenger checkpoint. Perhaps most importantly, post-
change interviews with TSOs (TSA personnel with the most hands-on
experience concerning prohibited items) confirmed this change did
indeed save time. In sum, data collection and analysis, along with
several other factors, played a role in TSA's decision-making process.
GAO further states in its report that TSA could use a methodology
employed by Customs and Border Patrol (CBP) to assess whether proposed
screening procedures enhance detection. GAO suggests that, like CBP,
TSA could compare items collected from passengers identified by TSA's
program for Screening Passengers by Observation Technique (SPOT) to
those collected from a random sample of passengers. TSA agrees in
principle, that measuring SPOT effectiveness, if possible, may provide
valuable insights. However, CBP's behavior detection program seems to
be fundamentally different from TSA's SPOT program; consequently, such
a comparison may not be valid.
CBP and TSA are seeking to identify two completely different types of
threats: (1) CBP hopes to identify passengers who are attempting to
smuggle contraband or attempting to enter the country illegally; while
(2) TSA, on the other hand, seeks to identify passengers who intend to
cause catastrophic harm onboard an aircraft. In CPB's case, possession
of illegal contraband or fraudulent documentation is the target
variable. Possession of a prohibited item is not a good measure of SPOT
effectiveness, in TSA's case, since it does not include those
individuals who would plot to cause harm or commandeer an aircraft
without the benefit of a prohibited item. In addition, possession of a
prohibited item is often an oversight and not an intentional act. TSA's
SPOT program makes a significant contribution to passenger security as
a deterrent. As GAO indicates in its report, deterrence is not readily
measured.
GAO also indicates that TSA could have used covert testing to measure
the effectiveness of the Unpredictable Screening Process (USP). The
basic premise of USP is based on risk and current understanding of
terrorist tradecraft. For example, it is thought that terrorists wait
for opportunities where the likelihood of success is greater. Based on
historical intelligence, we know that terrorists conduct pre-
operational surveillance prior to an attack in an attempt to understand
all aspects of the security processes and use that knowledge against
their enemy. USP is a mechanism which effectively adds a layer of
unpredictability to the screening process and ensures the terrorists
cannot use standardized and predictable security protocols against us.
By its very design, the TSO understands exactly what process they
should use at a given time, but the terrorist will not be able to
exactly predict the type of screening they will undergo. Because the
security process is now unpredictable, terrorists are prevented from
ever developing the necessary confidence they require for their efforts
to succeed. Deterrence is not measured through covert testing.
Finally, TSA appreciates GAO's conclusion that it is important for TSA
to continue taking steps, such as automating Performance Accountability
and Standards System (PASS) data-entry functions, to address resource
challenges. TSA has organized a multi-disciplined team of subject
matter experts (SMEs) under one umbrella, the Optimization Program, to
help FSDs improve screening efficiency, effectiveness, and address
other resource challenges. SMEs scrutinize aspects of the operation,
such as screening processes, manpower allocation, equipment mix, and
safety programs. The optimization team develops recommendations and
then works with the FSD to ensure implementation achieves the desired
efficiency or effectiveness goal.
In summary, TSA is committed to protecting the security of the
traveling public. To this end, TSA must be flexible and able to adapt
quickly to changes in terrorist tactics. When possible, TSA will
continue to supplement our assessment of risk, professional judgment,
and experience with empirical data and analysis, while optimizing
resources and screening operations.
The following represents our responses to the recommendations.
Recommendation 1: When operationally testing proposed SOP
modifications, develop sound evaluation methods, when possible, that
can be used to assist TSA in determining whether proposed SOP
procedures would achieve their intended result, such as enhancing TSA's
ability to detect prohibited items and suspicious persons and freeing
up existing TSO resources that could be used to implement proposed
procedures:
Concur: TSA will continue to perform operational testing of proposed
SOP modifications, when practicable, as a method for determining their
impact on passenger checkpoint security effectiveness and efficiency.
Moving forward, TSA intends to make better use of generally accepted
research design principles and techniques. Some examples include:
* clearly defined testing protocols and success criteria (independent
and dependent variables);
* test objectives which are directly linked to program objectives;
* pre- test/ post-test and longitudinal studies;
* control groups to isolate the effect of chance and other confounding
variables;
* use of-:
- representative samples,
- randomization, and:
- statistical analysis.
TSA Office of Security Operations has already begun partnering with
other agency SMEs to ensure operational tests are well designed,
executed, and produce results that are scientifically valid and
reliable. Of course, the need to dispatch SOP changes to combat an
immediate terrorist threat may, at times, preclude operational testing
of new procedures.
Recommendation 2: For future proposed SOP modifications that TSA senior
leadership determines are significant, generate and maintain
documentation to include, at minimum, the source, intended purpose, and
reasoning behind decisions to implement or not implement proposed
modifications.
Concurs: As noted in this report, TSA maintained the recommended
documentation for the majority of the SOP issues studied by GAO. TSA
intends to raise this standard by establishing additional processes and
controls to ensure that all SOP change proposals that could have a
significant impact on resources or the integrity of the process of
ensuring security are documented as recommended. These controls are
being implemented in the current SOP change cycle. For each level of
review (consolidation, interagency staff, and senior decision making),
documentation includes the source, intent, and reasoning for
implementing or rejecting those SOP change proposals determined to be
significant by senior leadership.
TSA has already begun making progress implementing GAO's
recommendations. This progress demonstrates our commitment to continual
improvement to ensure the security of the traveling public.
Thank you for the opportunity to provide comments to the draft report.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Director:
Departmental GAO/OIG Liaison Office:
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Cathleen A. Berrick, (202) 512-3404 or berrickc@gao.gov:
Acknowledgments:
In addition to the person named above, Maria Strudwick, Assistant
Director; David Alexander; Christopher W. Backley; Amy Bernstein;
Kristy Brown; Yvette Gutierrez-Thomas; Katherine N. Haeberle; Robert D.
Herring; Richard Hung; Christopher Jones, Stanley Kostyla; and Laina
Poon made key contributions to this report.
[End of section]
GAO Related Products:
Aviation Security: TSA's Staffing Allocation Model Is Useful for
Allocating Staff among Airports, but Its Assumptions Should Be
Systematically Reassessed. GAO-07-299. Washington, D.C.: February 28,
2007:
Aviation Security: Progress Made in Systematic Planning to Guide Key
Investment Decisions, but More Work Remains. GAO-07-448T. Washington,
D.C.: February 13, 2007.
Homeland Security: Progress Has Been Made to Address the
Vulnerabilities Exposed by 9/11, but Continued Federal Action Is Needed
to Further Mitigate Security Risks. GAO-07-375. Washington, D.C.:
January 24, 2007.
Aviation Security: TSA Oversight of Checked Baggage Screening
Procedures Could Be Strengthened GAO-06-869. Washington, D.C.: July 28,
2006.
Aviation Security: TSA Has Strengthened Efforts to Plan for the Optimal
Deployment of Checked Baggage Screening Systems, but Funding
Uncertainties Remain GAO-06-875T. Washington, D.C.: June 29, 2006.
Aviation Security: Management Challenges Remain for the Transportation
Security Administration's Secure Flight Program. GAO-06-864T.
Washington, D.C.: June 14, 2006.
Aviation Security: Further Study of Safety and Effectiveness and Better
Management Controls Needed if Air Carriers Resume Interest in Deploying
Less-than-Lethal Weapons. GAO-06-475. Washington, D.C.: May 26, 2006.
Aviation Security: Enhancements Made in Passenger and Checked Baggage
Screening, but Challenges Remain. GAO-06-371T. Washington, D.C.: April
4, 2006.
Aviation Security: Transportation Security Administration Has Made
Progress in Managing a Federal Security Workforce and Ensuring Security
at U.S. Airports, but Challenges Remain. GAO-06-597T. Washington, D.C.:
April 4, 2006.
Aviation Security: Progress Made to Set Up Program Using Private-Sector
Airport Screeners, but More Work Remains. GAO-06-166. Washington, D.C.:
March 31, 2006.
Aviation Security: Significant Management Challenges May Adversely
Affect Implementation of the Transportation Security Administration's
Secure Flight Program. GAO-06-374T. Washington, D.C.: February 9, 2006.
Aviation Security: Federal Air Marshal Service Could Benefit from
Improved Planning and Controls. GAO-06-203. Washington, D.C.: November
28, 2005.
Aviation Security: Federal Action Needed to Strengthen Domestic Air
Cargo Security. GAO-06-76. Washington, D.C.: October 17, 2005.
Transportation Security Administration: More Clarity on the Authority
of Federal Security Directors Is Needed. GAO-05-935. Washington, D.C.:
September 23, 2005.
Aviation Security: Flight and Cabin Crew Member Security Training
Strengthened, but Better Planning and Internal Controls Needed. GAO-05-
781. Washington, D.C.: September 6, 2005.
Aviation Security: Transportation Security Administration Did Not Fully
Disclose Uses of Personal Information during Secure Flight Program
Testing in Initial Privacy Notes, but Has Recently Taken Steps to More
Fully Inform the Public. GAO-05-864R. Washington, D.C.: July 22, 2005.
Aviation Security: Better Planning Needed to Optimize Deployment of
Checked Baggage Screening Systems. GAO-05-896T. Washington, D.C.: July
13, 2005.
Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains. GAO-05-457. Washington, D.C.: May
2, 2005.
Aviation Security: Secure Flight Development and Testing Under Way, but
Risks Should Be Managed as System Is Further Developed. GAO-05-356.
Washington, D.C.: March 28, 2005.
Aviation Security: Systematic Planning Needed to Optimize the
Deployment of Checked Baggage Screening Systems. GAO-05-365.
Washington, D.C.: March 15, 2005.
Aviation Security: Measures for Testing the Effect of Using Commercial
Data for the Secure Flight Program. GAO-05-324. Washington, D.C.:
February 23, 2005.
Transportation Security: Systematic Planning Needed to Optimize
Resources. GAO-05-357T. Washington, D.C.: February 15, 2005.
Aviation Security: Preliminary Observations on TSA's Progress to Allow
Airports to Use Private Passenger and Baggage Screening Services. GAO-
05-126. Washington, D.C.: November 19, 2004.
General Aviation Security: Increased Federal Oversight Is Needed, but
Continued Partnership with the Private Sector Is Critical to Long-Term
Success. GAO-05-144. Washington, D.C.: November 10, 2004.
Aviation Security: Further Steps Needed to Strengthen the Security of
Commercial Airport Perimeters and Access Controls. GAO-04-728.
Washington, D.C.: June 4, 2004.
Transportation Security Administration: High-Level Attention Needed to
Strengthen Acquisition Function. GAO-04-544. Washington, D.C.: May 28,
2004.
Aviation Security: Challenges in Using Biometric Technologies. GAO-04-
785T. Washington, D.C.: May 19, 2004.
Nonproliferation: Further Improvements Needed in U.S. Efforts to
Counter Threats from Man-Portable Air Defense Systems. GAO-04-519.
Washington, D.C.: May 13, 2004.
Aviation Security: Private Screening Contractors Have Little
Flexibility to Implement Innovative Approaches. GAO-04-505T.
Washington, D.C.: April 22, 2004.
Aviation Security: Improvement Still Needed in Federal Aviation
Security Efforts. GAO-04-592T. Washington, D.C.: March 30, 2004.
Aviation Security: Challenges Delay Implementation of Computer-
Assisted Passenger Prescreening System. GAO-04-504T. Washington, D.C.:
March 17, 2004.
Aviation Security: Factors Could Limit the Effectiveness of the
Transportation Security Administration's Efforts to Secure Aerial
Advertising Operations. GAO-04-499R. Washington, D.C.: March 5, 2004.
Aviation Security: Computer-Assisted Passenger Prescreening System
Faces Significant Implementation Challenges. GAO-04-385. Washington,
D.C.: February 13, 2004.
Aviation Security: Challenges Exist in Stabilizing and Enhancing
Passenger and Baggage Screening Operations. GAO-04-440T. Washington,
D.C.: February 12, 2004.
The Department of Homeland Security Needs to Fully Adopt a Knowledge-
based Approach to Its Counter-MANPADS Development Program. GAO-04-341R.
Washington, D.C.: January 30, 2004.
Aviation Security: Efforts to Measure Effectiveness and Strengthen
Security Programs. GAO-04-285T. Washington, D.C.: November 20, 2003.
Aviation Security: Federal Air Marshal Service Is Addressing Challenges
of Its Expanded Mission and Workforce, but Additional Actions Needed.
GAO-04-242. Washington, D.C.: November 19, 2003.
Aviation Security: Efforts to Measure Effectiveness and Address
Challenges. GAO-04-232T. Washington, D.C.: November 5, 2003.
Airport Passenger Screening: Preliminary Observations on Progress Made
and Challenges Remaining. GAO-03-1173. Washington, D.C.: September 24,
2003.
Aviation Security: Progress since September 11, 2001, and the
Challenges Ahead. GAO-03-1150T. Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Enhance Security
Efforts. GAO-03-1154T. Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Help Address Security
Challenges. GAO-03-843. Washington, D.C.: June 30, 2003.
Federal Aviation Administration: Reauthorization Provides Opportunities
to Address Key Agency Challenges. GAO-03-653T. Washington, D.C.: April
10, 2003.
Transportation Security: Post-September 11th Initiatives and Long-Term
Challenges. GAO-03-616T. Washington, D.C.: April 1, 2003.
Airport Finance: Past Funding Levels May Not Be Sufficient to Cover
Airports' Planned Capital Development. GAO-03-497T. Washington, D.C.:
February 25, 2003.
Transportation Security Administration: Actions and Plans to Build a
Results-Oriented Culture. GAO-03-190. Washington, D.C.: January 17,
2003.
Aviation Safety: Undeclared Air Shipments of Dangerous Goods and DOT's
Enforcement Approach. GAO-03-22. Washington, D.C.: January 10, 2003.
Aviation Security: Vulnerabilities and Potential Improvements for the
Air Cargo System. GAO-03-344. Washington, D.C.: December 20, 2002.
Aviation Security: Registered Traveler Program Policy and
Implementation Issues. GAO-03-253. Washington, D.C.: November 22, 2002.
Airport Finance: Using Airport Grant Funds for Security Projects Has
Affected Some Development Projects. GAO-03-27. Washington, D.C.:
October 15, 2002.
Commercial Aviation: Financial Condition and Industry Responses Affect
Competition. GAO-03-171T. Washington, D.C.: October 2, 2002.
Aviation Security: Transportation Security Administration Faces
Immediate and Long-Term Challenges. GAO-02-971T. Washington, D.C.: July
25, 2002.
Aviation Security: Information Concerning the Arming of Commercial
Pilots. GAO-02-822R. Washington, D.C.: June 28, 2002.
Aviation Security: Vulnerabilities in, and Alternatives for, Preboard
Screening Security Operations. GAO-01-1171T. Washington, D.C.:
September 25, 2001.
Aviation Security: Weaknesses in Airport Security and Options for
Assigning Screening Responsibilities. GAO-01-1165T. Washington, D.C.:
September 21, 2001.
Homeland Security: A Framework for Addressing the Nation's Efforts. GAO-
01-1158T. Washington, D.C.: September 21, 2001.
Aviation Security: Terrorist Acts Demonstrate Urgent Need to Improve
Security at the Nation's Airports. GAO-01-1162T. Washington, D.C.:
September 20, 2001.
Aviation Security: Terrorist Acts Illustrate Severe Weaknesses in
Aviation Security. GAO-01-1166T. Washington, D.C.: September 20, 2001.
FOOTNOTES
[1] In addition to passenger checkpoint screening, TSA's layers of
aviation security include, among other things, the screening of all
checked baggage for explosives and the deployment of Federal Air
Marshals on designated high-risk flights.
[2] Specifically, TSA modified the list of items prohibited and
permitted on aircraft by allowing metal scissors with pointed tips and
a cutting edge of 4 inches or less, as measured from the fulcrum, and
small tools of 7 inches or less, including screwdrivers, wrenches, and
pliers, to pass through the passenger screening checkpoint. See 70 Fed.
Reg. 72,930 (Dec. 8, 2005).
[3] We plan to issue a report on the impact of the prohibited items
list changes on public safety and screening operations later this year.
[4] We began our review period in April 2005 to coincide with TSA's
consideration of proposed SOP modifications related to the second major
revision of the passenger checkpoint screening SOP since TSA's
inception.
[5] TSA security activities at airports are overseen by FSDs. Each FSD
is responsible for overseeing security activities, including passenger
screening, at one or more commercial airports. We visited or conducted
phone interviews with officials at 25 airports. However, we met with
only 24 FSDs, as 1 FSD was responsible for 2 of the airports we
visited.
[6] The Aviation and Transportation Security Act (ATSA), Pub. L. No.
107-71, 115 Stat. 597 (2001), established TSA and assigned TSA with the
responsibility of building a federal workforce to conduct screening of
airline passengers and their checked baggage. See 49 U.S.C. §§ 114(a),
44901(a). ATSA also required that TSA allow commercial airports to
apply to TSA to transition from a federal to a private screener
workforce. See 49 U.S.C § 44920. To support this effort, TSA created
the Screening Partnership Program to allow all commercial airports an
opportunity to apply to TSA for permission to use qualified private
screening contractors and private screeners. There are currently 6
airports participating in the Screening Partnership Program, including
Jackson Hole, Kansas City International, Greater Rochester
International, San Francisco International, Sioux Falls Regional, and
Tupelo Regional.
[7] We used the following criteria to identify aviation security
experts: present and past employment in aviation security, depth of
experience in aviation security, and recognition in the aviation
industry.
[8] GAO, Internal Control: Standards for Internal Control in the
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: August 2001).
[9] Covert testing involves TSA headquarters officials (national
testing) or TSA field staff and other federal employees (local testing)
attempting to carry simulated threat objects through the checkpoint
without the objects being detected by TSOs. The results of the national
covert tests are classified and therefore are not included in this
report.)
[10] GAO, Transportation Security Administration: Actions and Plans to
Build a Results Oriented Culture, GAO-03-190 (Washington, D.C.: January
2003).
[11] The results of local covert testing are sensitive security
information and, therefore, are not included in this report.
[12] Sterile areas are located within the terminal where passengers are
provided access to boarding aircraft. Access to these areas is
controlled by Transportation Security Officers (or by nonfederal
screeners at airports participating in the Screener Partnership
Program) at checkpoints where they conduct physical screening of
individuals and their carry-on baggage for weapons, explosives, and
other prohibited items.
[13] Transportation Security Officers must deny passage beyond the
screening location to any individual or property that has not been
screened or inspected in accordance with passenger screening standard
operating procedures. If an individual refuses to permit inspection of
any item, that item must not be allowed into the sterile area or
onboard an aircraft.
[14] CAPPS is a computer-assisted system that, based on information
obtained from airline reservation systems, identifies passengers that
may pose a high risk to aviation security. These high-risk passengers
and their carry-on baggage are subject to additional and more thorough
screening.
[15] GAO, Aviation Security: Screener Training and Performance
Measurement Strengthened, but More Work Remains, GAO-05-457
(Washington, D.C.: May 2, 2005).
[16] Private screeners conduct passenger and checked baggage screening
at six airports as part of TSA's Screening Partnership Program. TSA
requires that private screeners screen passengers using the same
standard operating procedures as TSOs.
[17] Between April 2005 and December 2005, TSA considered a total of
189 proposed modifications to passenger checkpoint screening SOPs.
However, 97 of the proposed modifications were not intended to alter
the way in which passengers and their carry-on items are screened;
rather, these modifications were generally intended to correct, edit,
or clarify SOP language. For example, TSA modified SOP language to
ensure that TSA field staff were aware that tribal law enforcement
officers should be granted the same screening exemptions as other law
enforcement officers. TSA also amended the SOP to help ensure the
occupational safety of TSOs. For example, TSA headquarters officials
proposed that procedures for reporting potential radiation hazards
regarding X-ray equipment be incorporated into the SOP. The remaining
92 proposed SOP modifications were intended to alter the way in which
passengers and their carry-on items were screened, and 48 of those
proposed modifications were subsequently implemented.
[18] TSA issued six revised versions of the passenger checkpoint
screening SOP during the 9-month period under review: April 7, 2005;
July 7, 2005; August 26, 2005; September 12, 2005; October 25, 2005;
and December 7, 2005. However, we did not include the April 2005
revised SOP in our review since the changes incorporated in that
revision were deliberated by TSA officials outside of our 9-month
period of review.
[19] Of the 48 proposed modifications that were implemented, TSA made
the decision to implement 16 of these modifications following our 9-
month review period. However, because much of TSA's deliberation of
these 16 procedures occurred during our review period, we included
these procedures among those that were implemented.
[20] The Office of Security Operations is the TSA division responsible
for overseeing the implementation of passenger and property screening
at airport checkpoints.
[21] In order to achieve its goal of improving IED detection, in
addition to modifying passenger checkpoint screening SOPs, the task
force established several initiatives, including enhanced bomb
detection training for TSOs and increased use of explosives detection
canine teams.
[22] Three of the 92 proposed SOP modifications were considered by TSA
under both processes.
[23] The number of airports at which any one proposed change was pilot
tested ranged from 3 to 14, and the duration of the pilot testing
ranged from 5 days to several weeks.
[24] The number of proposed SOP modifications that fall under the
various "basis" categories (e.g., threat and vulnerability information)
does not total 92 because documentation was not available for all
proposed modifications and some of the proposed modifications had more
than one basis.
[25] We did not assess the quality of the intelligence information used
by TSA's Office of Intelligence and Analysis to generate its civil
aviation threat assessments.
[26] The pat-down procedure is performed for three purposes: (1) as a
substitute for walk-through metal detector screening, (2) to resolve
walk-through metal detector alarms, and (3) as a standard procedure for
screening passengers selected for additional screening. The details of
the pat-down procedures are sensitive security information and are not
discussed in this report.
[27] The recommendations made by the Office of Inspection are sensitive
security information or classified information. Therefore, they are not
discussed in this report.
[28] The DHS Office of Inspector General conducts similar covert tests,
and historically has recommended changes to the passenger checkpoint
screening SOP as a result of these tests. However, the Office of
Inspector General did not make any recommendations that resulted in
procedural changes between April 2005 and December 2005.
[29] A risk-based approach generally involves consideration of the
following when making decisions: threat--capability and intent of
terrorists to carry out an attack, vulnerability--weakness that may be
exploited by identified threats, and criticality or consequence--the
impact of an attack if it were to be carried out.
[30] Explosives Trace Portal screening entails a passenger stepping
into the portal, after which puffs of air are emitted onto the
passenger. The portal then draws in any residue that was loosened as a
result of the puffs of air, and analyzes the residue to determine if
there are explosive traces.
[31] TSA defines SOP modifications related to efficiency as changes
that will improve screening flow, clarify TSO duties, update equipment
procedures, or enhance the working environment of screening locations.
[32] The SOP modifications made by TSA on August 10, 2006, August 12,
2006, September 26, 2006, and November 21, 2006, were designed to
address only one particular hydrogen peroxide-based liquid explosives
mixture, which, according to TSA officials, was the same mixture that
the alleged terrorists had planned to detonate on U.S.-bound flights
originating in the United Kingdom. DHS and FBI have identified
additional liquid explosives mixtures that could pose a threat to
commercial aviation. DHS has ongoing evaluations of the additional
mixtures to determine their explosive potential and the extent of
damage that detonation of these mixtures could cause to an aircraft.
DHS is also evaluating explosives detection technology to determine the
extent to which it can be used at the checkpoint to defend against the
liquid explosives threat. We are currently evaluating DHS's and TSA's
progress in planning for, managing, and deploying research and
development programs in support of airport checkpoint screening
operations. We expect to report on our results in August 2007.
[33] In February 2007, DHS Science and Technology directorate conducted
aircraft vulnerability tests to determine the extent of damage the
liquid explosives that were to be used in the alleged August 2006
London terror plot would cause to an aircraft. The results of these
tests, however, are sensitive security information and are not included
in this report.
[34] The intelligence information regarding the August 2006 London
terror plot is classified and, therefore, is not included in this
report.
[35] In the event that TSOs cannot determine the reason for a
passenger's suspicious behavior, the TSO refers the passenger to law
enforcement officials. TSA officials responsible for SPOT told us that
in designing the implementation of SPOT, they worked closely with FBI
staff, Secret Service staff, Israeli security experts, and state police
with experience in recognizing suspicious behaviors.
[36] Another SOP change was operationally tested and subsequently
rejected. TSA did not provide documentation or other information on the
reason it was rejected.
[37] SPOT was operationally tested at 1 airport beginning in December
2003, at 2 additional airports beginning in October 2004, and at 2
other airports beginning in October 2005. The remaining 9 airports
began participating in the operational testing of SPOT in December
2005.
[38] Statistically significant means that it is highly unlikely to
obtain a difference of a given size or more by chance, assuming that
there is actually no difference in the probability of finding
prohibited items between targeted and randomly selected passengers.
[39] CBP officials could not comment on whether a similar methodology
could be used by TSA, since they were not familiar with the SPOT
procedure.
[40] Following the September 11 terrorist attacks, the items terrorists
reportedly used to carry out the attacks--box cutters--were
subsequently prohibited onboard aircraft.
[41] The fifth expert we interviewed said that he was uncertain how to
assess the effectiveness of passenger checkpoint screening procedures.
[42] TSA's Performance Management Information System is designed to
collect, analyze, and report passenger and baggage screening
performance data, such as wait times at selected airports, workload
data, and the performance and utilization of passenger and baggage
screening equipment. TSA headquarters uses PMIS data to support
external reporting on performance and internal decision-making
processes.
[43] To conduct our analysis we used TSA data that showed (1) it takes,
on average, about 1.89 minutes to conduct a bag search that was
initiated because a TSO identified a prohibited item (such as a pair of
scissors) in the X-ray image of a carry-on bag--this average search
time was derived from an informal TSA property search time study
conducted at 9 airports--and (2) there were 28,785 actual full-time-
equivalent (FTE) passenger screening TSOs during fiscal year 2005. One
FTE is equal to 1 work year or 2,080 nonovertime hours. To determine
the number of minutes per day, on average, each TSO spent searching for
sharp objects found during the 6-month period, we took the following
steps. First, we calculated the total amount of time (in minutes) taken
to conduct the searches by multiplying the number of sharp objects
found (1,762,571) by the average time to conduct targeted searches
(1.89 minutes), assuming that one item was found per search. This
totaled 55,521 hours. Next, we calculated the amount of time, on
average, each TSO spent searching for the sharp objects found by
dividing 55,521 hours by 28,785 TSO FTEs. The result was 1.93 hours per
TSO. Finally, we converted average hours per TSO to minutes and divided
by 130 days--the number of days worked by a TSO for 26 weeks over a 6-
month period (assuming 5 work days per week at 8 hours per day). The
result was an average of 0.89 minutes per day per TSO over the 6-month
period.
[44] The number of bags searched is sensitive security information.
[45] The results of the informal follow-on studies, which were
conducted at 6 to 9 airports, show that the percentage of carry-on bags
searched increased slightly at the time of the 30-day study, then
decreased slightly at the time of the 60-day and 90-day studies,
respectively. However, the results of these informal studies may not be
reliable due to the limitations in the methodology TSA used to conduct
the studies. Specifically, TSA did not use a methodology that would
control for factors other than the prohibited items list change that
may influence the percentage of carry-on bags searched by TSOs. To do
this, TSA would have had to develop a formal, systematic methodology
for randomly selecting various times of day, location of checkpoints,
number of checkpoints, and so on for data collection. By not
controlling for such factors, TSA may not know the extent to which a
reduction in the percentage of carry-on bags searched is due to the
prohibited items list changes.
[46] TSA officials told us that TSA's Office of Intelligence assessed
the potential impact each of these CAPPS changes would have on security
and, based in its analysis, determined that none of the CAPPS changes
would compromise security.
[47] Passengers can be selected for secondary screening through CAPPS
or other TSA-approved processes, such as the Selectee List. CAPPS rules
are sensitive security information and, therefore, are not discussed in
this report.
[48] Ticket checkers are aircraft operator or TSA employees who are
positioned before the screening checkpoint to perform identification
check and sterile area access responsibilities as required by TSA. For
passengers, ticket checkers verify travel documents and make sure the
identifying information on the travel document is consistent with the
information on the individual's personal identification documents
(e.g., licenses, passport, etc.) Ticket checkers are also responsible
for directing passengers designated as selectees to the appropriate
screening lane. For nonpassengers, ticket checkers verify required
identification before allowing access to the sterile area.
[49] The task force reported that 1 FSD was unsure of the security
benefits provided by USP, though this FSD did support the concept of
introducing unpredictability into the screening process.
[50] Of the remaining 6 FSDs, 5 said that TSO resources were freed up
as a result of the prohibited items list and CAPPS rules changes, and 1
was uncertain whether TSO resources were actually freed up.
[51] Since its inception in November 2001, TSA has had multiple
Assistant Secretaries (originally titled Under Secretaries of
Transportation for Security). In addition, between January 2005 and
August 2006, TSA issued seven press releases regarding senior-level
personnel changes within the agency.
[52] GAO, Internal Control: Internal Control Management and Evaluation
Tool, GAO-01-1008G (Washington, D.C.: August 2001).
[53] In July 2005, prior to the implementation of PASS, TSA required
all FSDs to implement an audit program of screening checkpoint
operations, primarily focused on assessing TSO compliance with
checkpoint screening SOPs. Specifically, each airport is to have an
audit program that evaluates TSOs' ability to detect threat objects
taken through the checkpoint, as well as TSOs' compliance with SOPs for
screening passengers and their accessible property. The audit program
is also intended to evaluate screening supervisors' and lead TSOs'
compliance with the SOP.
[54] GAO-03-190.
[55] ATSA requires that each TSO receive an annual proficiency review
to ensure he or she continues to meet all qualifications and standards
required to perform screening functions. See 49 U.S.C. § 44935(f)(5).
[56] GAO-01-1008G.
[57] As of February 2006, STEA test results had been recorded for a
total of 417 airports.
[58] The results of STEA testing are sensitive security information
and, therefore, are not included in this report.
[59] As of December 2006, TSA was in the process of modifying STEA into
a performance measurement program. TSA plans to implement the new STEA
program during the second quarter of fiscal year 2007.
[60] The covert testing results, including the reasons for failure, and
the recommendations made by the Office of Inspection are classified and
cannot be discussed in this report.
[61] We began our review period in April 2005 to coincide with TSA's
consideration of proposed SOP modifications related to the second major
revision of the passenger checkpoint screening SOP since TSA's
inception.
[62] We did not assess all of the proposed SOP modifications associated
with the SOP revisions issued between August 2006 and November 2006;
rather, we only reviewed the proposed modifications associated with
screening for liquids, gels, and aerosols.
[63] GAO/AIMD-00-21.3.1.
[64] We visited 25 airports. However, we met with only 24 FSDs, as 1
FSD was responsible for 2 of the airports we visited.
[65] GAO/AIMD-00-21.3.1.
[66] The list of airports we visited is sensitive security information.
Therefore, we do not identify those airports in this report.
[67] TSA classifies the more than 400 commercial airports in the United
States into one of five categories--X, I, II, III, and IV. Generally,
category X airports have the largest number of passenger boardings and
category IV airports have the smallest number.
[68] The Aviation and Transportation Security Act (ATSA) required that
TSA begin allowing commercial airports to apply to TSA to transition
from a federal to a private screener workforce. See 49 U.S.C. § 44920.
To support this effort, TSA created the Screening Partnership Program
to allow all commercial airports an opportunity to apply to TSA for
permission to use qualified private screening contractors and private
screeners. There are currently 6 airports participating in the
Screening Partnership Program, including Jackson Hole, Kansas City
International, Greater Rochester International, San Francisco
International, Sioux Falls Regional, and Tupelo Regional.
[69] There were 10 SOP modifications that were proposed both by
multiple sources. We attributed 9 of these proposed modifications to
each of the relevant sources. TSA did not identify the sources for the
1 remaining modification that was proposed by multiple sources.
[70] GAO/AIMD-00-21.3.1.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
