Aviation Security
TSA Oversight of Checked Baggage Screening Procedures Could Be Strengthened Gao ID: GAO-06-869 July 28, 2006The Transportation Security Administration (TSA) is responsible for screening all checked baggage in U.S. airports for explosives and has deployed explosive detection systems and developed standard procedures for their use. TSA also allows alternative screening procedures to be used for short-term, special circumstances. This report addresses (1) how TSA prioritized the use of checked baggage screening procedures and identified trade-offs in security effectiveness and operational efficiencies; (2) how TSA reported use of the procedures and ensured that standard procedures are used whenever possible; and (3) what steps TSA took to reduce airports' need to use alternative screening procedures and to establish performance measures to monitor their use. To address these issues, GAO interviewed TSA officials, reviewed information from TSA's database on checked baggage screening operations; and conducted airport site visits.
TSA has prioritized standard and alternative checked baggage screening procedures based on legislative requirements and TSA officials' judgment of the procedures' effectiveness. Use of various procedures to screen checked baggage has involved trade-offs in security effectiveness, which vary by the type of procedure used and the circumstances of its use. It is TSA's policy to use standard procedures whenever possible because TSA officials determined that these procedures provide the most effective detection of explosives. TSA policy also allows the use of alternative screening procedures when volumes of baggage awaiting screening pose security vulnerabilities or when TSA airport officials determine that there is a security risk associated with large concentrations of passengers in an area waiting for their baggage to be screened. Regarding operational efficiencies, TSA has not fully determined the throughput and costs of the various alternative screening procedures in part because it does not count the number of bags screened using these procedures. TSA has conducted covert tests (undercover, unannounced) of standard procedures, but has not conducted this testing for alternative screening procedures. TSA cited logistical difficulties in conducting covert tests for alternative screening procedures. However, by not doing so, TSA is not collecting data that could provide useful information in determining the security effectiveness of the procedures in an operational setting and how to improve their effectiveness. TSA cannot identify the percentage of checked baggage screened using standard versus alternative screening procedures because TSA records standard procedures in terms of the number of bags screened in its management information system, but records alternative procedures in terms of the number of occasions and hours of use. However, TSA officials estimated that a low percentage of checked baggage is screened using alternative screening procedures. To assess the extent that standard screening procedures are used whenever possible, TSA has established internal controls to monitor the use of standard and alternative screening procedures; however, these controls were not always implemented to ensure the gathering of complete and accurate information. This may limit TSA managers' ability to assess the effect of using alternative screening procedures and determine what should be done to minimize the use of the procedures. TSA headquarters officials stated that they are working with TSA airport staff to correct such reporting problems. TSA has taken steps to reduce the need to use alternative screening procedures at airports, including anticipating factors that could increase passenger and baggage volume and acting to address these factors. However, TSA has not developed performance measures and targets to assess its progress in minimizing the need to use the procedures. By creating performance measures, TSA could gauge whether it is making progress toward minimizing the need to use alternative screening procedures at airports. Performance targets for the procedures would be an indicator of how much risk TSA is willing to accept in using the procedures.
GAO-06-869, Aviation Security: TSA Oversight of Checked Baggage Screening Procedures Could Be Strengthened
This is the accessible text file for GAO report number GAO-06-869
entitled 'Aviation Security: TSA Oversight of Checked Baggage Screening
Procedures Could Be Strengthened' which was released on July 28, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Ranking Democratic Member, Committee on Transportation
and Infrastructure, House of Representatives:
United States Government Accountability Office:
GAO:
July 2006:
Aviation Security:
TSA Oversight of Checked Baggage Screening Procedures Could Be
Strengthened:
Aviation Security:
GAO-06-869:
GAO Highlights:
Highlights of GAO-06-869, a report to the Ranking Democratic Member,
Committee on Transportation and Infrastructure, House of
Representatives
Why GAO Did This Study:
The Transportation Security Administration (TSA) is responsible for
screening all checked baggage in U.S. airports for explosives and has
deployed explosive detection systems and developed standard procedures
for their use. TSA also allows alternative screening procedures to be
used for short-term, special circumstances. This report addresses (1)
how TSA prioritized the use of checked baggage screening procedures and
identified trade-offs in security effectiveness and operational
efficiencies; (2) how TSA reported use of the procedures and ensured
that standard procedures are used whenever possible; and (3) what steps
TSA took to reduce airports‘ need to use alternative screening
procedures and to establish performance measures to monitor their use.
To address these issues, GAO interviewed TSA officials, reviewed
information from TSA‘s database on checked baggage screening
operations; and conducted airport site visits.
What GAO Found:
TSA has prioritized standard and alternative checked baggage screening
procedures based on legislative requirements and TSA officials‘
judgment of the procedures‘ effectiveness. Use of various procedures to
screen checked baggage has involved trade-offs in security
effectiveness, which vary by the type of procedure used and the
circumstances of its use. It is TSA‘s policy to use standard procedures
whenever possible because TSA officials determined that these
procedures provide the most effective detection of explosives. TSA
policy also allows the use of alternative screening procedures when
volumes of baggage awaiting screening pose security vulnerabilities or
when TSA airport officials determine that there is a security risk
associated with large concentrations of passengers in an area waiting
for their baggage to be screened. Regarding operational efficiencies,
TSA has not fully determined the throughput and costs of the various
alternative screening procedures in part because it does not count the
number of bags screened using these procedures. TSA has conducted
covert tests (undercover, unannounced) of standard procedures, but has
not conducted this testing for alternative screening procedures. TSA
cited logistical difficulties in conducting covert tests for
alternative screening procedures. However, by not doing so, TSA is not
collecting data that could provide useful information in determining
the security effectiveness of the procedures in an operational setting
and how to improve their effectiveness.
TSA cannot identify the percentage of checked baggage screened using
standard versus alternative screening procedures because TSA records
standard procedures in terms of the number of bags screened in its
management information system, but records alternative procedures in
terms of the number of occasions and hours of use. However, TSA
officials estimated that a low percentage of checked baggage is
screened using alternative screening procedures. To assess the extent
that standard screening procedures are used whenever possible, TSA has
established internal controls to monitor the use of standard and
alternative screening procedures; however, these controls were not
always implemented to ensure the gathering of complete and accurate
information. This may limit TSA managers‘ ability to assess the effect
of using alternative screening procedures and determine what should be
done to minimize the use of the procedures. TSA headquarters officials
stated that they are working with TSA airport staff to correct such
reporting problems.
TSA has taken steps to reduce the need to use alternative screening
procedures at airports, including anticipating factors that could
increase passenger and baggage volume and acting to address these
factors. However, TSA has not developed performance measures and
targets to assess its progress in minimizing the need to use the
procedures. By creating performance measures, TSA could gauge whether
it is making progress toward minimizing the need to use alternative
screening procedures at airports. Performance targets for the
procedures would be an indicator of how much risk TSA is willing to
accept in using the procedures.
What GAO Recommends:
GAO is recommending that TSA use information on airport usage of
alternative screening procedures in conducting covert testing;
strengthen TSA‘s monitoring and tracking of the use of alternative
screening procedures; and develop performance measures and targets for
the use of alternative screening procedures. DHS reviewed a draft of
this report and generally concurred with GAO‘s findings and
recommendations.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-869].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cathleen Berrick at (202)
512-3404 or berrickc@gao.gov.
[End of Section]
Contents:
Letter:
Results in Brief:
Background:
TSA Prioritized Screening Procedures Based on Legislative Requirements
and Judgment of Effectiveness but Has Not Tested the Security
Effectiveness of Alternative Screening Procedures in an Operational
Environment:
The Full Extent of the Usage of Alternative Screening Procedures Is Not
Known, and Internal Controls for Monitoring the Usage of Baggage
Screening Procedures Could Be Improved:
TSA Has Taken Action to Reduce the Need to Use Alternative Screening
Procedures, but Has Not Implemented Performance Measures or Targets:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Homeland Security:
Appendix III: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Table:
Table 1: Bags per Hour Screened Using Standard Screening Procedures for
Stand-alone and In-line EDS Machines and ETD Machines:
Figures:
Figure 1: EDS Machines In a Stand-alone Configuration Used by TSA to
Screen Checked Baggage:
Figure 2: ETD Machine Used by TSA to Screen Checked Baggage:
Abbreviations:
ATSA: Aviation and Transportation Security Act:
DHS: Department of Homeland Security:
EDS: Explosive Detection System:
ETD: Explosive Trace Detection:
FAA: Federal Aviation Administration:
FSD: Federal Security Director:
GPRA: Government Performance and Results Act:
OI: Office of Inspections:
PMIS: Performance Management Information System:
TSA: Transportation Security Administration:
TSO: Transportation Security Officer:
United States Government Accountability Office:
Washington, DC 20548:
July 28, 2006:
The Honorable James L. Oberstar:
Ranking Democratic Member:
Committee on Transportation and Infrastructure:
House of Representatives:
Dear Mr. Oberstar:
Commercial U.S. aircraft have long been a target for terrorist attacks
through the use of explosives carried in checked baggage, as
demonstrated by the 1988 bombing of a U.S. aircraft over Lockerbie,
Scotland. After the terrorist attacks of September 11, 2001, which
further highlighted the vulnerability of U.S. aircraft to acts of
terrorism, Congress passed and the President signed into law the
Aviation and Transportation Security Act (ATSA), mandating, among other
things, that all checked baggage at U.S. airports be screened using
explosive detection systems by December 31, 2002.[Footnote 1] In
response to this mandate, the Transportation Security Administration
(TSA) has deployed two types of systems and has established standard
procedures for their use: (1) explosive detection systems (EDS) that
use specialized X-rays to detect characteristics of explosives that may
be contained in baggage as it moves along a conveyor belt and (2)
explosive trace detection (ETD) systems, whereby an individual (i.e., a
baggage screener) swabs a bag and then inserts the swab into the ETD
machine, which, in turn, can detect chemical residues that may indicate
the presence of explosives within a bag.[Footnote 2] TSA also allows
alternative screening procedures to be used when volumes of baggage
awaiting screening pose security vulnerabilities or when TSA officials
determine that there is a security risk associated with large
concentrations of passengers in an area. These alternative screening
procedures include the use of EDS and ETD machines in nonstandard
ways,[Footnote 3] and also include three procedures that do not use EDS
or ETD--screening with explosives detection canines, physical bag
searches, and matching baggage to passenger manifests to confirm that
the passenger and his or her baggage are on the same plane. It is TSA's
policy to use standard EDS and ETD screening procedures whenever
possible and, when necessary, to use EDS or ETD-based alternative
screening procedures before using non-EDS or ETD alternative screening
procedures. TSA has also established operating procedures for checked
baggage screening that instruct Transportation Security Officers (TSO-
-formerly known as screeners) in how to use these standard and
alternative screening procedures.
In February 2004, we testified that TSA was unable to fully utilize
explosive detection systems to satisfy the ATSA mandate to screen 100
percent of checked baggage for explosives because of TSO shortages and
problems with screening equipment, among other factors.[Footnote 4]
Further, in February 2005, we reported that TSA considers one baggage
screening method--the use of EDS machines--to be the superior baggage
screening procedure in terms of efficiency compared to ETD because EDS
automatically detects explosives without direct human involvement and
screens more bags for explosives per hour. We also reported that at
most smaller airports, where EDS machines are not installed, TSA
screens solely with ETD machines. Finally, we reported that while TSA
had made progress in deploying EDS and ETD machines, it had not
conducted a systematic, prospective analysis of the optimal deployment
of these machines to achieve long-term savings and enhanced
efficiencies and security. In February 2006, TSA issued a report to
Congress detailing its strategic planning framework for its checked
baggage screening program. According to TSA, the framework, which
focuses on identifying optimal checked baggage screening solutions for
airports, will be used to establish a comprehensive strategic plan for
TSA's checked baggage screening program. TSA expects to complete the
strategic plan in early fall 2006.
You asked that we continue to assess TSA's progress in enhancing the
effectiveness of checked baggage screening operations. In February
2006, we issued a report that contained sensitive security information
regarding TSA's use of standard and alternative checked baggage
screening procedures, including the extent to which the procedures were
used by TSA and the trade-offs in security effectiveness of using
standard and alternative screening procedures to screen checked baggage
for explosives.[Footnote 5] We concurrently issued a classified
correspondence on our analysis of the results of TSA's checked baggage
screening covert testing.[Footnote 6] This report provides the results
of our February 2006 report with sensitive security information
removed. In this report, we address the following questions: (1) How
did TSA prioritize the use of standard and alternative checked baggage
screening procedures and what security effectiveness trade-offs and
operational efficiencies has TSA identified in using these procedures
to screen checked baggage for explosives? (2) To what extent has TSA
used standard and alternative screening procedures to screen checked
baggage for explosives and how does TSA ensure that standard screening
procedures are used whenever possible? (3) What steps has TSA taken to
reduce airports' need to use alternative screening procedures and to
establish performance measures and targets for the use of the
procedures?
To address how TSA prioritized the use of checked baggage screening
procedures and identified trade-offs in security effectiveness and
operational efficiencies of using standard and alternative screening
procedures to screen checked baggage for explosives, we assessed TSA's
standard operating procedures, obtained and analyzed relevant
legislation, and conducted a literature search to obtain information on
screening procedures, technologies, and related aviation trends. We
also reviewed studies conducted by TSA's Transportation Security
Laboratory regarding checked baggage screening. We interviewed
officials from various TSA offices as well as air carriers, explosive
detection systems equipment manufacturers, and an airport industry
association to obtain information regarding TSA's checked baggage
screening procedures and the relative priority they were given. To
determine the extent to which TSA used standard and alternative
screening procedures to screen checked baggage and how TSA ensured that
standard screening procedures are used whenever possible, we reviewed
TSA's checked baggage standard operating procedures manual and its
Performance Management Information System (PMIS) database, which
contains information on checked baggage screening operations.[Footnote
7] We compared TSA procedures for use of standard and alternative
screening procedures with the Comptroller General's Standards for
Internal Control in the Federal Government. We interviewed TSA
officials to determine the reliability of the information contained in
the PMIS database and reviewed selected database records. Although we
determined that the database did not always contain accurate data on
the duration of each occurrence of alternative baggage screening
procedures, we nonetheless considered the database to be sufficiently
reliable for purposes of this report since the data identified overall
trends in the use of alternative screening procedures. To assess the
steps TSA has taken to reduce airports' need to use alternative
screening procedures and the measures and targets TSA has set for
alternative screening procedures, we analyzed TSA directives regarding
use of the procedures, PMIS data, standard operating procedures for
checked baggage, and the inventory of explosive detection systems. When
we found discrepancies in the inventory data of explosive detection
systems, we worked with TSA to resolve the discrepancies. TSA also
completed a reconciliation of the inventory database with data
collected manually by TSA officials. We also examined TSA's checked
baggage screening performance measures and targets as they relate to
the Government Performance and Results Act of 1993 (GPRA) requirements.
We visited nine airports during this review to observe baggage
screening operations. We chose these airports based on reporting in
PMIS, selecting some airports that had used alternative screening
procedures and some that had not used the procedures. We also conducted
structured interviews with TSA Federal Security Directors
(FSD)[Footnote 8] and their staffs who were responsible for the
security of 29 randomly selected airports. Because we selected a
nonprobability sample of airports to visit, the information we obtained
during these visits cannot be generalized to all airports
nationwide.[Footnote 9] Also, while the interviews we conducted with
FSDs were random samples of airports, the samples were too small to
generalize the interview results with a high degree of statistical
confidence to all airports nationwide.
We conducted our work from August 2004 through July 2006 in accordance
with generally accepted government auditing standards. A more detailed
discussion of our scope and methodology is contained in appendix I.
Results in Brief:
TSA prioritized the use of standard and alternative checked baggage
screening procedures based on legislative requirements and TSA
officials' judgment of the effectiveness of the procedures, and TSA's
use of these procedures has involved trade-offs in security
effectiveness. It is TSA's policy to use standard EDS and ETD screening
procedures whenever possible because of legislative requirements to do
so and because TSA has concluded that these procedures provide the most
effective detection of explosives at a checked baggage screening
station. TSA considers screening with EDS to be superior to screening
with ETD because EDS machines process more bags per hour and
automatically detect explosives without direct human involvement. Given
the high volumes of checked baggage processed at some airports and
unforeseen events such as equipment failures and severe weather, TSA
has determined that it will have to continue to at times use
alternative screening procedures to screen checked baggage for
explosives. TSA officials prioritized the use of these procedures based
on their professional judgment of the effectiveness of the procedures
and the classified probabilities of detection of EDS and ETD machines
obtained from the TSA Transportation Security Laboratory. Alternative
screening procedures include physical bag searches; alternative hybrid
procedures, which involve a combination of the standard checked baggage
procedures for EDS and ETD;[Footnote 10] matching checked bags to
passenger manifests to deter bombers who try to load a bag on a plane
without boarding; explosive detection canines; and another screening
method TSA has determined to be sensitive security information. With
regard to operational efficiencies, TSA has not determined the
throughput (number of checked bags screened per hour) and costs of the
various alternative screening procedures in part because it does not
count the number of bags screened using these procedures. While TSA has
used alternative screening procedures for more than 3 years and expects
to continue to use them, it has not tested the security effectiveness
of these procedures in an operational environment. TSA has conducted
national covert (undercover, unannounced) testing of standard screening
procedures since September 2002 to assess checked baggage screening
technologies and procedures, and TSO performance in detecting
explosives in an operational environment. TSA has also authorized FSDs
and their designates to conduct local covert testing on standard
screening procedures at airports since March 2005 to determine if TSOs
can detect simulated improvised explosive devices hidden in checked
baggage. However, TSA has not conducted national or local covert
testing specifically focused on alternative screening procedures. TSA
officials stated that conducting national testing of alternative
screening procedures would be challenging, particularly since testing
is planned ahead of time and it is difficult to predict when an airport
might use alternative screening procedures. While it may be
logistically difficult to conduct national covert testing on
alternative screening procedures, TSA data on which airports most
frequently use the procedures and the reasons for the usage could
provide TSA with information to use in selecting airports for covert
testing as part of the risk-based approach to covert testing that it is
developing. TSA officials also cited challenges in conducting local
covert testing of alternative screening procedures, including the
difficulty in predicting the need to use the procedures and the lack of
available federal staff to conduct the testing, particularly at smaller
airports. Because FSDs and their staffs authorize and initiate the use
of alternative screening procedures, they could schedule some of their
ongoing local covert testing for checked baggage screening to coincide
with the use of these procedures. In not assessing the alternative
screening procedures through covert testing, TSA is not collecting data
that could help determine how effective the procedures are in an
operational setting or how to improve the security effectiveness of the
procedures.
TSA reported using both standard and alternative screening procedures
in PMIS but could not identify the percentage of all checked baggage
screened using EDS and ETD with standard or alternative screening
procedures. TSA could not identify the percentage of usage of standard
or alternative screening procedures because for standard screening
procedures, PMIS contains data on the number of bags screened; whereas
for alternative screening procedures, it contains data on the number of
occasions and hours of use. However, TSA officials estimated that a
high percentage of checked baggage is screened using EDS and ETD
machines with standard screening procedures[Footnote 11] and a low
percentage is screened using alternative screening procedures.[Footnote
12] TSA also established internal controls to monitor and track the use
of standard and alternative screening procedures, including requiring
FSDs to report the use of alternative screening procedures into PMIS
and to call headquarters for permission to use the alternative
screening procedures for more than 2 hours. According to the
Comptroller General's Standards for Internal Control, all transactions
and other significant events need to be completely and accurately
documented. However, we identified several areas where TSA's
information on its use of standard and alternative screening procedures
was not complete or accurate for the period covered by our review.
First, TSA's method for estimating the number of bags screened with ETD
using standard screening procedures led to inaccurate counts of baggage
screened. Improved counting of bags would provide TSA management with
better information to use in making decisions related to its baggage
screening operations including where to deploy screening equipment.
Second, FSDs and their staffs did not always accurately report the
occurrences when a particular alternative baggage screening procedure
was used, impeding TSA's ability to reliably determine how often and
for how long the alternative screening procedures were used. Third,
FSDs and their staffs did not always report the use of alternative
screening procedures as required. Inaccurate and incomplete reporting
on how often alternative screening procedures are used or for how long,
combined with not tracking the number of bags screened using the
procedures, may limit TSA managers' ability to assess the effect of
using alternative screening procedures on aviation security and to
determine the types of actions that should be taken at airports to help
minimize the use of these procedures. TSA officials stated they are
working with FSDs to correct these reporting problems and have issued
guidance clarifying requirements for reporting alternative screening
procedures.
TSA has taken steps to reduce airports' need to use alternative
screening procedures, but has not established performance measures or
targets regarding the use of these procedures. According to our review
of PMIS data, the use of alternative screening procedures between
October 2004 and September 2005--measured in terms of the total hours
of use reported by FSDs--initially increased and then
declined.[Footnote 13] TSA attributed the reported overall decline in
the use of alternative screening procedures to a number of factors,
including better coordination with groups such as tour operators;
deploying "optimization teams" to airports that were frequently using
alternative screening procedures to determine why the procedures were
being used so often and to suggest remedies; and deploying additional
EDS machines.[Footnote 14] As we reported in March 2005, additional EDS
systems integrated into the airport baggage handling system ("in-line"
EDS systems) could reduce by 78 percent the number of baggage TSOs and
supervisors needed to screen checked baggage at airports with the
systems.[Footnote 15] TSA's February 2006 checked baggage screening
planning framework includes a prioritization of which additional
airports should receive new funding for in-line systems and a
description of how stand-alone EDS machines from those airports should
be redistributed to other airports. After in-line EDS systems are
installed and any staffing reductions are achieved, redistributing the
screening positions to other airports with staffing shortages could
also reduce the need to use alternative screening procedures at these
airports. While TSA data indicate that the use of alternative screening
procedures is declining, the strategic planning framework states that
at some airports alternative screening procedures will increasingly be
used because of rising passenger traffic. TSA has projected that the
number of originating domestic and international passengers will rise
by about 127 million passengers over current levels by 2010, which
could increase airports' need to rely on alternative screening
procedures in the future in the absence of additional or more efficient
EDS machines. Furthermore, while TSA has taken steps to reduce the need
to use alternative screening procedures at airports, it has not created
performance measures or targets regarding its progress in minimizing
the need to use alternative screening procedures at airports. By
creating performance measures for the use of alternative screening
procedures and corresponding targets, TSA could gauge whether it is
making progress in working to minimize the need to use alternative
screening procedures at airports and to consider the necessity to take
further steps to minimize the need for their use.
To help strengthen TSA's management of checked baggage screening
operations, including screening with alternative screening procedures,
we are recommending that the Secretary of the Department of Homeland
Security (DHS) direct the Assistant Secretary, TSA, to use PMIS data on
the use of alternative screening procedures at airports to help
determine which airports to conduct national covert testing at and when
to conduct such testing, to conduct local covert testing of alternative
screening procedures, to strengthen its monitoring and tracking of the
use of alternative screening procedures, and to develop performance
measures and performance targets for the use of alternative screening
procedures.
We provided a draft copy of this report to DHS for review. DHS, in its
written comments, generally concurred with our findings and
recommendations and stated that the recommendations and findings will
help strengthen TSA's management of checked baggage screening
operations. The full text of DHS's comments is included in appendix II.
Background:
Standard Procedures for Using EDS and ETD:
ATSA mandated that the screening of all checked baggage at commercial
airports be done using explosive detection systems by December 31,
2002. To satisfy this mandate, TSA deployed two types of screening
equipment to all airports in the United States where screening is
required: (1) explosive detection systems, which use computer-aided
tomography[Footnote 16] X-rays adapted from the medical field to
automatically recognize the characteristic signatures of threat
explosives, and (2) explosives trace detection systems, which use
chemical analysis to detect traces of explosive materials' vapors and
residues. As we reported in February 2004, largely because of shortages
of equipment and insufficient time to modify airports to accommodate
EDS machines, TSA was unable, at certain airports, to meet this
deadline. Recognizing the obstacles encountered by TSA, the Homeland
Security Act of 2002, in effect, subsequently extended the deadline for
screening all checked baggage for explosives until December 31, 2003,
for airports at which TSA was unable to meet the earlier deadline
established by ATSA. We also reported that TSA fell short of fully
satisfying the extended 2003 mandate and continued to face challenges
in screening checked baggage because of (1) an insufficient number of
TSOs to operate the EDS and ETD machines, TSO absenteeism, and a lack
of TSO training in how to operate the machines and (2) a lack of EDS
and ETD equipment and inoperable equipment.[Footnote 17]
By taking the equivalent of hundreds of X-ray pictures of a bag from
different angles, the EDS machine examines the objects inside of the
baggage to identify the characteristic signatures of threat explosives
such as density and atomic number. TSA has certified, acquired, and
deployed EDS machines manufactured by three companies. EDS machines can
be installed in airports either in stand-alone mode (not integrated
with baggage handling systems) or in-line (integrated with baggage
handling systems). TSA has developed standard procedures for using EDS.
Figure 1 shows EDS machines in use at an airport.
Figure 1: EDS Machines In a Stand-alone Configuration Used by TSA to
Screen Checked Baggage:
[See PDF for image]
Source: GAO.
[End of figure]
At airports that do not have EDS machines and at airports where certain
screening stations do not have EDS--for example at curbside check-in
stations--TSA uses ETD machines to conduct primary screening. ETD
machines are also used for secondary screening, which resolves alarms
from EDS machines that indicate the possible presence of explosives
inside a bag. TSA has certified, acquired, and deployed ETD machines
from three manufacturers. Figure 2 shows an ETD machine in use at an
airport.
Figure 2: ETD Machine Used by TSA to Screen Checked Baggage:
[See PDF for image]
Source: GAO.
[End of figure]
At some screening stations, TSA has also allowed primary screening with
both EDS and ETD machines simultaneously. In this hybrid configuration,
the EDS machine is used to maximum capacity before the ETD machines are
used. Additionally, when the EDS machine alarms, the ETD machines are
also used for secondary screening.
Alternative Screening Procedures:
TSA also uses alternative screening procedures to screen checked
baggage for explosives under certain short-term special circumstances,
when the standard procedures using EDS and ETD are not used. Two of
these procedures involve the use of EDS and ETD. The first of these is
an alternative hybrid procedure that is used at a screening station
configured only for EDS primary screening. Under this procedure, the
EDS is to be used to capacity and the remainder of the bags are
screened with ETD. The specifics of the second procedure that involves
the use of EDS and ETD are sensitive security information.[Footnote 18]
If one of these EDS-or ETD-based alternative screening procedure is not
available, TSA will resort to one of the procedures that does not use
EDS or ETD--canine screening; physical inspections of baggage; and
positive passenger bag match, which requires that passengers be on the
same aircraft as their checked baggage.[Footnote 19]
FSDs and their designates not lower than the Assistant FSD for
Screening may authorize the use of alternative screening procedures
under two circumstances: (1) when the FSD or his or her designate
determines that there is a security threat created by large
concentrations of passengers waiting to have their baggage screened or
(2) volumes of baggage awaiting screening in a confined baggage
screening area pose an explosive or other security vulnerability. These
circumstances may arise for reasons such as high passenger volumes,
screening machine breakdowns, or unusual weather events such as
hurricanes. After alternative screening procedures have been used, TSA
requires that information on each occurrence be recorded by FSD staff
into the PMIS database, including circumstances leading to the use of
the procedure, type of procedure used, and duration. This information
on the use of alternative screening procedures from PMIS is to be
included in daily briefing reports for TSA senior management.
TSA Prioritized Screening Procedures Based on Legislative Requirements
and Judgment of Effectiveness but Has Not Tested the Security
Effectiveness of Alternative Screening Procedures in an Operational
Environment:
TSA prioritized standard and alternative checked baggage screening
procedures based on legislative requirements and TSA officials'
judgment of the security effectiveness of the procedures. TSA's use of
these various procedures has involved trade-offs in security
effectiveness. TSA officials determined that in general, standard
screening procedures are more effective than alternative screening
procedures. TSA has estimated that in terms of efficiency, EDS
processes more bags per hour than ETD. With regard to operational
efficiencies, TSA has not determined the throughput and costs of the
various alternative screening procedures, in part because it does not
count the number of bags screened using the procedures. Additionally,
while TSA has assessed the security effectiveness of screening with
standard procedures in an operational environment through covert
testing, it has not conducted similar testing of alternative screening
procedures.
TSA Prioritized Standard and Alternative Screening Procedures Based on
Legislative Requirements and TSA Officials' Judgment of Security
Effectiveness:
TSA is required by legislation to screen all checked baggage using
explosive detection systems, and TSA officials concluded that standard
screening procedures that use EDS or ETD provided the most effective
detection of explosives at a baggage screening station and that
alternative screening procedures should be used only for short-term,
special circumstances. These circumstances include times when security
targets are created by large volumes of passengers awaiting baggage
screening or when security vulnerabilities are created by volumes of
bags awaiting screening. According to TSA, a group of officials from
its Chief Technologist, Chief Counsel, Aviation Operations, and
Operations Policy offices met to prioritize the use of standard and
alternative screening procedures. TSA officials stated that this group
did not use formal criteria to prioritize the procedures but instead
prioritized them based on their professional judgment of the
effectiveness of the procedures, including the classified probabilities
of detection of EDS and ETD machines obtained from the Transportation
Security Laboratory. This prioritization was subsequently included in
TSA's standard operating procedures for checked baggage screening. TSA
has determined that details on the prioritization of alternative
screening procedures constitute sensitive security information.
Standard Baggage Screening Procedures Vary in Operational Efficiency:
TSA has estimated that, in terms of efficiency, EDS processes more bags
per hour than ETD--EDS ranges from a minimum of 80 bags per hour for
one model of a stand-alone machine up to 500 bags per hour for an in-
line system, compared to 36 bags per hour by the operator of an ETD
machine.[Footnote 20] Whenever EDS machines have been installed at a
screening station, TSA requires airports to use them as the primary
method to screen checked baggage. At some screening stations, TSA has
also allowed the use of a hybrid configuration as a standard screening
procedure that involves colocated EDS and ETD machines for primary
screening, with the EDS machine used to maximum capacity before the ETD
machines are used. When the EDS machine alarms, the ETD machines are
also used for secondary screening. Table 1 shows the bags per hour
screened by EDS and ETD machines.
Table 1: Bags per Hour Screened Using Standard Screening Procedures for
Stand-alone and In-line EDS Machines and ETD Machines:
Type of equipment: EDS machines: CTX 2500--stand-alone only;
Maximum bags per hour: Stand-alone: 120;
Maximum bags per hour: In-line: NA[A].
Type of equipment: EDS machines: CTX 5500;
Maximum bags per hour: Stand-alone: 180;
Maximum bags per hour: In-line: 250.
Type of equipment: EDS machines: CTX 9000--in-line only;
Maximum bags per hour: Stand-alone: NA;
Maximum bags per hour: In-line: 500.
Type of equipment: EDS machines: L3 6000;
Maximum bags per hour: Stand- alone: 140; Maximum bags per hour: In-
line: 500.
Type of equipment: EDS machines: CT-80--stand-alone only;
Maximum bags per hour: Stand-alone: 80;
Maximum bags per hour: In-line: NA.
Type of equipment: EDS machines: ETD machines--stand-alone only;
Maximum bags per hour: Stand-alone: 36;
Maximum bags per hour: In-line: MA.
Source: TSA.
[A] NA: Not applicable.
[End of table]
At 312 mostly smaller airports and at some airport screening stations
such as curbside check-in stations, TSA has installed ETD instead of
EDS for primary screening because of the configuration of screening
stations, the costs associated with procuring EDS, and the low
passenger volume at smaller airports. In our March 2005 report, we
recommended that TSA assess the feasibility, expected benefits, and
cost to replace ETD machines with stand-alone EDS machines for the
primary screening of checked baggage at those airports where in-line
EDS systems would not be either economically justified or justified for
other reasons. [Footnote 21] DHS stated that TSA was conducting an
analysis of the airports that rely heavily on ETD machines as the
primary checked baggage screening technology in order to identify
airports that would benefit from replacing ETD machines with stand-
alone EDS equipment.[Footnote 22] In February 2006, in response to
GAO's recommendation and a legislative requirement to submit a schedule
for expediting the installation and use of in-line systems and
replacement of ETD equipment with EDS machines,[Footnote 23] TSA
provided its strategic planning framework for its checked baggage
screening program to Congress. This framework introduces a strategy
intended to increase efficiency through deploying EDS to as many
airports as practicable, lower life-cycle costs for the program,
minimize impacts to TSA and airport/airline operations, and provide a
flexible security infrastructure for accommodating growing airline
traffic and potential new threats.[Footnote 24] The framework is an
initial step in addressing the following areas:
* optimized checked baggage screening solutions--finding the ideal mix
of higher-performance and lower-cost alternative screening solutions
for the 250 airports with the highest checked baggage volumes and:
* funding prioritization schedule by airport--identifying the top 25
airports that should first receive federal funding for projects related
to the installation of explosive detection systems based on
quantitative modeling of security, economic, and other factors.
TSA's strategic plan for the checked baggage screening program, which
TSA expects to complete by early fall 2006, is to include funding and
cost-sharing strategies for the installation of in-line baggage
screening systems.
Use of Alternative Baggage Screening Procedures Involves Trade-offs in
Security Effectiveness, while Trade-offs in Operational Efficiencies
Have Not Been Determined:
TSA has determined that the use of alternative screening procedures at
airports has created trade-offs in security effectiveness, but it has
not determined the operational efficiencies of these procedures in
terms of throughput and costs.[Footnote 25] TSA based its
prioritization of the alternative screening procedures on its judgment
of the procedures' security effectiveness and classified probabilities
of detection of EDS and ETD machines obtained from the TSA
Transportation Security Laboratory. TSA has not determined the
operational efficiencies of the various alternative screening
procedures in terms of throughput and costs in part because it does not
count the number of bags screened using the procedures. If the higher-
prioritized alternative screening procedure is not available at a
screening station, the FSD may authorize a lower-prioritized
procedure.[Footnote 26]
Positive Passenger Bag Match:
Under the positive passenger bag match alternative screening procedure,
TSA coordinates with airlines to ensure that passengers are on the same
aircraft as their checked baggage. If a passenger checks a bag but does
not board the airplane, the bag is removed before departure. This
procedure was first implemented based on the premise that a terrorist
would seek to place a bomb on an airplane without sacrificing his or
her life by boarding the airplane. In light of the suicide terrorist
attacks of September 11, this premise is now considered flawed. An
airline trade association and airline officials representing two
airlines we interviewed also stated that the procedure creates
operational inefficiencies for airlines. According to these officials,
the process of matching bags with passengers can delay flights because
the flight cannot take off until all baggage is matched to an on-board
passenger. These officials also stated that implementing the procedure
increases the workload of airline personnel, who are responsible for
conducting the procedures at the direction of TSA.
Canine Screening:
TSA also uses canine units as an alternative screening procedure. These
units are composed of trained explosives detection canines and
handlers. In terms of efficiency, TSA officials reported that it can be
difficult to mobilize canine units in sufficient time to screen checked
baggage when alternative screening procedures are needed, especially
since the need to use the procedures can arise without warning.
Officials also reported that screening checked baggage using canines
requires enough open floor space to lay out the baggage as well as a
sufficient number of personnel to move the bags into position for
canine screening.
Physical Inspection:
The physical inspection alternative screening procedure requires human
intervention to detect explosives, weapons, and improvised explosive
devices and their components, and does not involve use of EDS or ETD
machines. While TSOs are trained to detect improvised explosive devices
and their components and to detect signs of tampering, the success of
the TSOs in finding these items depends on their skill in detecting
such items through manual searches and their adherence to TSA's
standard operating procedures for checked baggage regarding physical
inspection. Since human TSOs are involved, the efficiency of physical
inspection in terms of baggage throughput rate can vary depending on
the contents of the bag and how quickly the TSO conducts the search.
Alternative Hybrid Procedures:
Alternative hybrid procedures involve using a combination of EDS and
ETD at a screening station normally configured only for EDS. The
efficiency of alternative hybrid procedures can vary because each use
of the procedure can involve a different proportion of EDS and ETD
screening, with greater use of EDS leading to more efficient screening
in terms of number of bags screened per hour.
TSA Has Not Tested the Operational Security Effectiveness of
Alternative Screening Procedures through Covert Testing:
While TSA has reported using alternative screening procedures for more
than 3 years, it has not tested the security effectiveness of the
procedures in detecting explosives in an operational environment. TSA
has conducted national covert testing of standard screening procedures
since September 2002, and local covert testing of standard screening
procedures since March 2005. However, it has not specifically focused
national or local covert testing on alternative screening procedures to
determine the security effectiveness of the procedures. TSA's Office of
Inspections (OI--formerly the Office of Internal Affairs and Program
Review) conducts national covert tests at airports to assess the
security effectiveness of checked baggage screening technology,
procedures, and TSO performance in detecting explosives in an
operational environment. These tests, in which undercover inspectors
attempt to pass threat objects through passenger screening checkpoints
and in checked baggage, are designed to identify vulnerabilities in
passenger and checked baggage screening systems and to identify
systematic problems affecting screening in the areas of training,
procedures, and technology.[Footnote 27] The schedule for this testing
called for inspectors to test all category X airports once a year,
category I and II airports once every 2 years, and category III and IV
airports at least once every 3 years.[Footnote 28] In August, 2005, TSA
suspended this cycle of testing. In April 2006, TSA officials stated
that OI was moving to a testing schedule to include observations of
screening stations and concentrated testing for improvised explosive
devices at the screening checkpoint. The schedule is based on risk-
based factors such as current intelligence information, high-
vulnerability airports, procedural changes, training initiatives, and
introduction of new technologies. According to Office of Inspections
officials, during the 3-year testing cycle, inspectors tested the
procedures being used by TSOs at the time of the test; alternative
screening procedures were tested only if inspectors coincidentally
conducted a test at a screening station while one of the procedures was
in use. Office of Inspections officials stated that they did not
schedule tests of alternative screening procedures because their
resources were dedicated to conducting testing on standard screening
procedures. Furthermore, the officials stated that since covert testing
visits are planned in advance of the tests and the need to use
alternative screening procedures is not always known in advance, it
would be logistically difficult to plan a covert testing visit to
coincide with an airport's use of alternative screening procedures
because of airports' intermittent and often short-term use of the
procedures. While it may be logistically difficult to conduct national
covert testing on alternative screening procedures, PMIS data on which
airports most frequently use the procedures and the reasons for the
usage could provide the Office of Inspections with information to
select airports for covert testing as part of the risk-based approach
to covert testing that it is developing.
In addition to its national covert testing program, in March 2005, TSA
also began an airport-based local covert testing program to determine
if TSA checked baggage TSOs can detect a simulated improvised explosive
device that is hidden in a test bag. Participation in this program is
at the discretion of the FSD. FSD staff test screening at EDS stations
by placing simulant explosives in baggage and surreptitiously running
the bags through the machines. According to TSA, between March 2005 and
February 2006, 2,526 local tests of EDS screening were conducted at 108
airports. When we asked TSA headquarters officials in charge of the
local covert testing program about the feasibility and usefulness of
testing the use of alternative screening procedures, they stated that
they had not previously considered testing the procedures through the
local covert testing program. These officials also cited challenges in
conducting such testing, including the difficulty in predicting the
need to use the procedures and the lack of available federal staff to
conduct the testing, particularly at smaller airports. Because FSDs and
their staffs authorize and initiate the use of alternative screening
procedures, they could schedule some of their ongoing local covert
testing for checked baggage screening to coincide with the use of these
procedures. In not testing the alternative screening procedures through
national or local covert testing, TSA is not collecting data that could
provide useful information on how to improve the security effectiveness
of these procedures in detecting explosives.
The Full Extent of the Usage of Alternative Screening Procedures Is Not
Known, and Internal Controls for Monitoring the Usage of Baggage
Screening Procedures Could Be Improved:
TSA Does Not Collect Consistent Data on the Use of Standard and
Alternative Screening Procedures to Enable an Accurate Determination of
the Full Extent of Their Use:
TSA reported using both standard and alternative screening procedures
in PMIS but could not identify the percentage of all checked baggage
screening using EDS and ETD with standard or alternative screening
procedures. TSA cannot identify the percentage of usage of standard or
alternative screening procedures because for standard screening
procedures, PMIS contains data on the number of bags screened, whereas
for alternative screening procedures it contains data on the number of
occasions and hours of use.[Footnote 29] However, TSA officials
estimated that a high percentage of checked baggage is screened using
EDS and ETD machines with standard screening procedures[Footnote 30]
and a low percentage is screened using alternative screening
procedures.[Footnote 31] TSA determined that the number of bags
screened using EDS and ETD with standard screening procedures between
October 2004 and September 2005 as well as data on the use of
alternative screening procedures reported into PMIS during this same
period are sensitive security information.
TSA Established Internal Controls to Monitor the Usage of Standard and
Alternative Screening Procedures, but Some Controls Have Not Been
Adequately Implemented:
TSA established internal controls to monitor and track the usage of
standard and alternative screening procedures, but has not adequately
implemented some of these controls. An internal control is an integral
component of an organization's management and is designed to provide
reasonable assurance that agencies achieve effectiveness and efficiency
of operations and compliance with applicable laws and regulations. The
Comptroller General's Standards for Internal Controls require that
transactions and events be completely and accurately recorded in order
to ensure that information is available for management to guide
operations and make decisions.[Footnote 32] While TSA has established
internal controls to monitor and track use of baggage screening
procedures, such as requiring FSDs and their designates to report the
use of alternative screening procedures into PMIS and to call TSA
headquarters for permission to use the procedures for more than 2
hours, some of the controls have not been adequately implemented
because events have not been completely and accurately recorded.
Consequently, TSA does not have complete information on the extent of
the use of alternative screening procedures that would be helpful for
TSA management in making decisions on actions to minimize the need to
use alternative screening procedures at airports, such as deploying
screening equipment.
Recording of ETD Baggage Screening in PMIS:
Information that FSDs and their staffs report in PMIS regarding the
number of bags screened using ETD machines may not be accurate because
of the way in which the number of bags screened is estimated. While EDS
machines automatically count each bag screened, ETD machines count each
swab analyzed, rather than each bag screened. TSA uses this count of
analyses to estimate the number of bags screened using ETD. The number
of analyses for a bag screened using ETD may vary depending on how many
times the machine alarms during the screening process and other
factors, which may lead to overreporting of baggage screened. TSA
officials stated that they are aware of these discrepancies and are
working to improve counting of baggage screened using ETD.[Footnote 33]
Improved counting of bags would provide TSA management with better
information to use in making decisions related to its baggage screening
operations, including where to deploy screening equipment.
Recording Occurrences of Alternative Screening Procedures:
FSDs and their staffs did not always completely and accurately record
information in PMIS on the use of various alternative screening
procedures. On the basis of our review of PMIS data from October 2004
through September 2005, we found that FSDs and their designates did not
always accurately report the occurrences when a particular baggage
screening procedure was used. For example, some of the airports that
reported using alternative screening procedures voluntarily reported in
a PMIS comments field that they used the procedures intermittently over
the course of several hours, even though in PMIS they reported only one
occurrence that lasted several hours.[Footnote 34] FSD staff at one of
these airports reported in PMIS one occurrence of using alternative
screening procedures for 15.5 hours straight but reported in the
comments field that the procedures were used during 24 different
occurrences during the 15.5 hours. According to TSA guidance, these
data should have been recorded as 24 separate occurrences in the PMIS
database, not simply noted in the comments field. TSA officials stated
that they were aware that many airports were reporting the use of
alternative screening procedures for extended periods of time rather
than recording each time the use of the procedures was started and
stopped within the reported time. In May 2005, the TSA Assistant
Administrator for Aviation Programs sent a memo to FSDs noting that the
start and stop time of each individual use of an alternative screening
procedure at each screening station should be reported into PMIS. In
our analysis of PMIS data from May 2005 through September 2005,
subsequent to the issuance of this memo, some of the airports continued
to report intermittent use of alternative screening procedures in the
comments fields.
The design of PMIS also contributed to incomplete and inaccurate
recording of information because it does not allow FSDs and their
designates to report two or more alternative screening procedures used
during the same occurrence. TSA officials have instructed FSDs and
their designates to record the alternative screening procedure that is
used the most during the occurrence. One airport voluntarily reported
in the PMIS comments field that it used three alternative screening
procedures throughout the occurrence. However, the PMIS database only
allowed reporting of the use of one of the procedures during this time.
Because of these reporting limitations, TSA managers do not receive
complete information on how often or for how long the various
alternative screening procedures are actually used--information that
could affect their decisions on what actions to take to minimize the
need to use alternative screening procedures at airports.
Another factor that could contribute to incomplete and inaccurate
reporting of alternative screening procedures in PMIS is that although
FSDs and their staffs are required to report every occurrence of the
procedures in PMIS, they may not have always done so. Until August
2005, when this requirement was eliminated, FSDs and their staffs were
required to report to the Transportation Security Operations Center--
TSA's command, control, communications and intelligence center--
whenever they were about to begin using alternative screening
procedures or to switch back to standard EDS or ETD screening after
using alternative screening procedures.[Footnote 35] While TSA
officials stated that they did not keep formal records of the calls,
they kept what TSA termed "informal notes" on sheets that included
times when the use of the alternative screening procedures began and
ended and the type of procedure used. When we compared a select
number[Footnote 36] of these sheets completed between February and
March 2005 to PMIS reporting for the same period, we found that 21
percent of the occurrences of use of alternative screening procedures
recorded on the sheets were not recorded into PMIS as required by TSA's
standard operating procedures.[Footnote 37] Inaccurate reporting on the
frequency of use of alternative screening procedures may hinder
management decision making on how best to minimize airport need to use
these procedures.
Permission to Use Alternative Screening Procedures for More than 2
Hours:
While TSA's standard operating procedures require FSDs or their
designates to call headquarters for permission to use alternative
screening procedures that are used for more than 2 hours in order to
ensure that the procedures are used only for short-term, special
circumstances, TSA does not require headquarters to maintain a record
of these calls. Because these calls are not recorded, TSA management is
not able to ensure that the requirement in the standard operating
procedures is being followed. Recording these calls would enable TSA to
compare the records to the hours of use of alternative screening
procedures data maintained in PMIS. This comparison would provide TSA
with information to help provide reasonable assurance that FSDs and
their staffs are complying with the standard operating procedures'
requirement to call for permission to exceed 2 hours' use of the
procedures. TSA headquarters officials stated that there had not been
any instances in which airports were denied permission to exceed 2
hours' use of alternative screening procedures. However, without
records of the calls, we were not able to verify that permission was
granted for the occurrences that exceeded 2 hours.[Footnote 38]
TSA Has Taken Action to Reduce the Need to Use Alternative Screening
Procedures, but Has Not Implemented Performance Measures or Targets:
TSA Has Taken Steps to Reduce the Need to Use Alternative Screening
Procedures at Airports:
TSA has taken steps to reduce airports' need to rely on the use of
alternative baggage screening procedures and is working to minimize the
need to use these procedures. According to our review of PMIS data, the
use of alternative screening procedures between October 2004 and
September 2005--measured in terms of the total hours these procedures
were employed--initially increased and then declined. TSA attributed
the reported overall decline in the usage of alternative screening
procedures in part to improved coordination among FSDs, airlines, and
local organizations. According to TSA officials, this coordination
helps FSDs and their staffs anticipate surges in passenger traffic so
that they can adequately staff screening stations. In our structured
interviews with FSDs and their staffs responsible for 29 airports,
several FSDs also cited the importance of coordination with local
organizations and how this reduced their need to use alternative
screening procedures. For example, the FSD for one airport said that he
coordinated with local summer camps to have campers' baggage screened
the day before their flights to reduce the amount of baggage that has
to be screened when campers arrive at the airport. The FSD for another
airport communicated with cruise ship management about the scheduling
of cruises in order to anticipate any surges in passenger traffic that
may have created the need to use alternative screening procedures.
TSA officials have also taken action to reduce airports' need to use
alternative screening procedures through the use of "optimization team"
visits to airports. These visits are conducted at the request of TSA
senior leadership or an FSD with the goal to observe screening
operations and maximize efficiencies by applying practices learned at
other airports. According to TSA officials, recurring use of
alternative screening procedures triggered some of the optimization
team visits TSA has conducted, and the optimization team visits may
have led to a reduction in the number of occasions in which these
procedures needed to be used.[Footnote 39] For example, on May 18-19,
2005, an optimization team visited one airport and recommended
procuring one ETD machine and changing the location of another to
reduce the airport's need to use alternative screening procedures. On
July 6-7, 2005, another optimization team visited another airport. The
team suggested reconfiguring EDS machines at the airport's screening
stations, which resulted in an increase in baggage throughput from 120
to 150 bags per hour. At both of these airports, alternative screening
procedures were used more frequently prior to the optimization team
visit than they were after the visit.
TSA officials also stated that as additional equipment is deployed and
enhanced to enable TSA to increase checked baggage screening
throughputs--bags screened per hour--TSA will be in a better position
to reduce the need for use of alternative screening procedures. We
reported in March 2005 that as of June 2004, TSA had deployed 1,228 EDS
machines.[Footnote 40] Between June 2004 and June 2006, TSA had
deployed 399 additional EDS machines in both in-line (integrated into
the airport baggage system) and stand-alone (in airport lobbies or
baggage makeup areas) configurations. EDS machines in an in-line
configuration are able to screen up to 500 bags per hour, as compared
to EDS machines in a stand-alone configuration that screen between 80
and 180 bags per hour. The superior efficiency of screening with in-
line EDS compared to screening with stand-alone EDS may have been a
factor in reducing the need to use alternative screening procedures at
airports where in-line systems were installed. TSA reported that, as of
June 2006, 25 airports had operational in-line EDS systems and an
additional 24 airports had in-line systems under construction. Although
in-line EDS systems can create improvements in operational efficiencies
of an airport's checked baggage screening system, baggage volumes that
exceed the system's capacity and equipment breakdowns still sometimes
occur, necessitating the use of alternative screening procedures. For
example, some of the airports that have installed airportwide in-line
systems reported using alternative screening procedures because of
equipment failures and high passenger and baggage volumes after their
systems were operational.[Footnote 41] Since stand-alone EDS machines
screen between 80 and 180 bags per hour compared to ETD machines, which
allow for screening of 36 bags per hour, additional stand-alone EDS
machines also may have helped the airports where they were installed to
screen baggage with standard screening procedures rather than
alternative screening procedures. Additionally, in May 2005, TSA
certified software and hardware upgrades for 519 out of 1,322 EDS
machines, which are used in both in-line and stand-alone
configurations. These upgrades are being tested in a pilot program. TSA
officials anticipate that the upgrades could lead to increased baggage
throughput for the machines, which could further reduce need to use
alternative screening procedures.
Installation of in-line EDS systems at airports that currently use
stand-alone EDS and ETD for primary screening has further potential to
reduce the need for alternative screening procedures to be used at
these airports. In March 2005, we reported that TSA had estimated that
in-line checked baggage systems would reduce by 78 percent the number
of baggage TSOs and supervisors required to screen checked baggage at
nine airports that had signed agreements to develop the
systems.[Footnote 42] Under the congressionally imposed 45,000 TSO full-
time-equivalent limit, when staff requirements are reduced at one
airport through increased efficiencies, full-time equivalent positions
will become available to address TSO shortages at other
airports.[Footnote 43] TSA's February 2006 checked baggage strategic
planning framework included a prioritization of which additional
airports should receive funding for in-line systems and a description
of how stand-alone EDS machines from those airports should be
redistributed to other airports.[Footnote 44] In the framework, TSA
also reported that many of the initial in-line systems had produced a
level of TSO labor savings insufficient to offset up-front capital
costs of constructing the systems. According to TSA, the facility and
baggage handling system modifications have been higher than expected at
the nine airports that have signed agreements to fund the systems. TSA
stated that the keys to reducing future costs are establishing
guidelines outlining best practices and a set of efficient design
choices and using newer EDS technology that best matches each optimally
scaled design solution. In February 2006, TSA reported that recent
improvements in the design of the in-line EDS checked baggage screening
systems and the EDS screening technology now offer the opportunity for
higher-performance and lower-cost screening systems. The final
strategic plan will include the results of TSA's cost sharing study
that it is currently conducting in consultation with airport operators,
airlines, and other key stakeholders to identify ways to fund in-line
EDS systems.[Footnote 45] After in-line EDS systems are installed and
staffing reductions are achieved, redistributing the TSO positions to
other airports with staffing shortages may reduce airport need to use
alternative screening procedures.
Technology developments may also help TSA to reduce the use of
alternative screening procedures. In March 2005, we reported that TSA
was working to develop a computer-aided tomography explosives detection
system that is smaller and lighter than systems currently deployed in
airport lobbies and that the new system was intended to replace systems
currently in use, including larger and heavier EDS machines and ETD
equipment.[Footnote 46] The smaller size of the system would create
opportunities for TSA to transfer screening operations to other
locations, such as airport check-in counters. The machine would also be
an option for airports that currently rely on ETD machines since it
would be cheaper than other certified machines and it would have higher
baggage throughput than screening using ETD machines, potentially
reducing the need to use alternative screening procedures at airports
where it is installed. In March 2005, TSA began to pilot this machine
at three airports. At one of these pilot airports, the FSD stated that
he anticipates that the smaller EDS machine will reduce staffing needs,
reduce workers compensation claims, and ultimately enable the airport
to incorporate the machines in-line behind the ticket counters. TSA
reported that the machine achieved throughput rates of up to 80 bags
per hour, higher than the throughput rate of up to 36 bags per hour for
an ETD operated by one TSO or up to 72 bags per hour for an ETD
operated by two TSOs. In September 2005, TSA entered into a $24.8
million contract to purchase 72 of these machines that will be
installed at 24 airports.
While TSA data indicate that the use of alternative screening
procedures is declining, TSA reported in its February 2006 framework
that at some airports alternative screening procedures will
increasingly be used because of rising passenger traffic. TSA has
projected that the number of originating domestic and international
passengers will rise by about 127 million passengers over current
levels by 2010. If TSA's current estimate of an average of 0.76 checked
bags per passenger were to remain constant through 2010, TSA would be
screening about 96 million more bags than it now screens. This could
increase airports' need to rely on alternative screening procedures in
the future in the absence of additional or more efficient EDS machines.
TSA headquarters officials stated that while TSA is working to minimize
the need to use alternative screening procedures, it intends to
maintain the procedures as part of its standard operating procedures so
that FSDs will have options to respond to events such as unforeseen
equipment failures, surges in passenger traffic, and weather-related
incidents such as hurricanes. Additionally, some of the FSDs that we
interviewed stated that they anticipate continuing to need to use
alternative screening procedures because of screening capacity limits
and rising passenger volume, and some of these FSDs anticipated
increasing their use of the procedures as their airport passenger
traffic rises because of limitations in the physical layout of their
airports that contribute to overcrowding.[Footnote 47]
TSA Has Not Established Performance Measures or Targets Related to the
Use of Alternative Screening Procedures:
Although TSA is working to minimize the need to use alternative
screening procedures at airports, it has not established performance
measures or targets related to the use of these procedures. The
Government Performance and Results Act of 1993 provides, among other
things, that federal agencies establish program performance measures,
including the assessment of relevant outputs and outcomes of
measures.[Footnote 48] Performance measures are meant to cover key
aspects of performance and help decision makers to assess program
accomplishments and improve program performance. A performance target
is a desired level of performance expressed as a tangible, measurable
objective, against which actual achievement will be compared. By
analyzing the gap between target and actual levels of performance,
management can target those processes that are most in need of
improvement, set improvement goals, and identify appropriate process
improvements or other actions.
TSA has established four performance measures for the checked baggage
screening program. Three of these measures make up TSA's checked
baggage screening performance index. This index measures the overall
performance of the system through a composite of indicators that are
derived by combining specific performance measures related to checked
baggage screening. Specifically, this index measures the effectiveness
of screening systems through machine probability of detection and
covert testing results, efficiency through a calculation of dollars
spent per bag screened, and customer complaints at both airports and
TSA's national call center. TSA considers the final performance
measure--compliance with the ATSA requirement to screen all checked
baggage using explosive detection systems (EDS and ETD)--to be obsolete
since it reported all airports as capable of screening with EDS or ETD
in January 2005. The use of alternative screening procedures is not
included in the index, nor does TSA have stand-alone measures or
targets for the use of alternative screening procedures.
TSA officials stated that they did not want to implement performance
measures or targets for alternative screening procedures because they
are already working to minimize the need to use the procedures at
airports. However, TSA officials also acknowledged that they will
continue to rely on alternative screening procedures because of
unforeseen circumstances such as high baggage volumes or weather-
related incidents. By creating a performance measure for the use of
alternative screening procedures as part of the checked baggage
screening index or as a stand-alone measure, TSA could gauge whether it
is making progress toward minimizing the need to use these procedures
at airports and have more complete information on how well the overall
checked baggage screening system is performing. Furthermore,
performance targets for the use of alternative screening procedures
would provide an indicator of how much risk TSA is willing to accept in
using these procedures, and TSA's monitoring of this indicator would
identify when it has exceeded the level of risk that it has determined
is acceptable. For example, if TSA were to determine the percentage of
checked baggage that should be screened using alternative screening
procedures, and if its performance data showed that it was currently
screening a higher percentage than the target, TSA would be able to
decide whether to take steps to bring the use of these procedures into
line with its desired level of use.
Finally, the extent to which performance measures and targets will
assist TSA in minimizing the need to use these procedures at airports
is dependent upon the accuracy and completeness of the reporting of
alternative screening procedures in PMIS, including the percentage of
bags screened using the procedures, as previously discussed.
Conclusions:
It has been over 4 years since Congress issued the mandate for TSA to
screen all checked baggage at commercial airports using explosive
detection systems. During this time, TSA has deployed EDS or ETD
machines at more than 400 commercial airports and reported achieving
the capability to screen 100 percent of checked baggage using these
machines. As part of this effort, TSA developed standard and
alternative checked baggage screening procedures. While TSA
acknowledges that screening with alternative screening procedures is
less effective than screening with standard screening procedures, it
has also recognized the need for continued use of alternative screening
procedures because of high passenger and baggage volumes resulting from
unpredictable and unforeseen circumstances, such as equipment
breakdowns and unusual weather events. Given TSA's plans to continue to
use alternative screening procedures and the trade-offs in security
effectiveness involved in their use, it will be important for TSA to
test the effectiveness of these procedures in an operating environment.
One such way for TSA to test the security effectiveness of the various
alternative screening procedures is through the covert testing
conducted by the Office of Inspections. While we recognize the
logistical challenges that the Office of Inspections faces in
conducting checked baggage covert testing on alternative screening
procedures, using PMIS data on the use of these procedures--including
data on the airports that use the procedures the most frequently or for
extended periods of time--could help the Office of Inspections in
selecting airports for testing as part of the risk-based approach to
covert testing that it is currently developing. By not assessing
alternative screening procedures through national or local covert
testing, TSA is missing an opportunity to gather information to help
determine the security effectiveness of alternative screening
procedures in an operational setting.
Additionally, TSA headquarters has established internal controls to
monitor and track the use of alternative screening procedures at
airports and has taken steps to improve reporting of these procedures
in the PMIS database. However, without strengthening its controls, such
as providing a means for measuring the number of bags screened using
alternative screening procedures and enabling TSA airport staff to
report the concurrent use of more than one alternative screening
procedure, TSA lacks reasonable assurance that it has complete and
accurate information on the use of these procedures.
Furthermore, TSA has taken steps to reduce the need to use alternative
screening procedures at airports, but does not expect to eliminate the
use of these procedures. Increasing air travel and TSA's effort to
operate within or below the current 45,000 TSO full-time-equivalent
limit could add to the need for alternative screening procedures,
unless more or more efficient EDS machines are deployed. Given TSA's
continuing use of the procedures, performance measures and targets
would provide TSA and Congress with objective information to assess
TSA's progress in minimizing the need to use the procedures at
airports, and would help inform TSA decision making on whether and when
mitigating steps are needed to achieve its desired level of use.
Recommendations for Executive Action:
To help inform TSA of the security effectiveness of alternative
screening procedures in an operational setting, and to help TSA
strengthen its monitoring of the use of alternative screening
procedures, we recommend that the Secretary of the Department of
Homeland Security direct the Assistant Secretary, Transportation
Security Administration, to take the following four actions:
* Use PMIS data on use of alternative screening procedures in
determining at which airports to conduct covert testing and when to
conduct testing at these airports as part of the Office of Inspections'
new risk-based approach to covert testing.
* Conduct local covert testing of alternative screening procedures to
determine whether checked baggage TSOs can detect simulated improvised
explosives when using these procedures.
* Strengthen the monitoring and tracking of the use of alternative
screening procedures to help determine the progress the agency is
making in minimizing its need to use these procedures. This effort
would include continuing to address reporting problems in the PMIS
database system, keeping a record of calls requesting permission to
exceed 2 hours' use of the procedures, and providing a means for
measuring the use of alternative screening procedures compared to the
use of standard procedures, such as counting baggage screened with
alternative screening procedures.
* Develop performance measures and performance targets for the use of
alternative screening procedures in checked baggage screening, perhaps
as part of the checked baggage screening program performance index, to
help TSA measure its progress in working toward minimizing the need to
use alternative screening procedures at airports and to have more
complete information on the overall performance of the checked baggage
screening system.
Agency Comments and Our Evaluation:
We provided a draft of this report to DHS for review and comment. On
July 25, 2006, we received written comments on the draft report, which
are reproduced in full in appendix II. DHS concurred with our findings
and recommendations and stated that the report will help strengthen
TSA's management of checked baggage screening operations.
Regarding our recommendation that TSA use Performance Management
Information System data on the use of alternative screening procedures
in determining at which airports to conduct covert testing and when to
conduct testing at these airports, DHS concurred and stated that TSA's
Office of Inspections will consider PMIS information on alternative
screening procedures as part of its new risk-based approach to covert
testing and will develop new checked baggage screening testing
protocols. Concerning our recommendation that TSA conduct local covert
testing of alternative screening procedures, DHS concurred and stated
that TSA is currently modifying its local covert testing program to
strengthen the program and expects that these modifications will better
prepare TSOs to detect simulated improvised explosives. We are pleased
that TSA is making efforts to strengthen its local covert testing
program. We continue to believe that testing of alternative screening
procedures would provide TSA with an opportunity to gather information
to help identify and improve the security effectiveness of alternative
screening procedures in an operational setting.
DHS concurred with our recommendation to strengthen the monitoring and
tracking of the use of alternative screening procedures to help
determine the progress the agency is making in minimizing its need to
use the procedures. In response to our recommendation that TSA address
reporting problems in its PMIS database system, DHS stated that PMIS
has been enhanced with both functionality and data quality-related
processes to ensure data reliability. According to TSA, the system
alerts the user when a data field is filled in with a value that falls
outside the operational norms for a particular airport. Additionally,
according to TSA, PMIS training and functionality reviews occur on a
regular basis and user manuals and best practices are updated
consistently. While these efforts should help improve the data
reliability of PMIS, they will not fully address the reporting problems
highlighted in our report. Specifically, the steps TSA has taken do not
address inaccurate counts of baggage screened resulting from TSA's
method for estimating the number of bags screened with ETD using
standard screening procedures or inaccurate reporting of occurrences
when a particular alternative screening procedure is used. Without
addressing these reporting problems, TSA will continue to lack
reasonable assurance that it has complete and accurate information on
the use of these procedures.
In response to our recommendation on keeping a record of calls
requesting permission to exceed 2 hours use of the procedures, DHS
stated that because the amount of time that alternative screening
procedures are used is recorded in PMIS, there is no further
documentation required for exceeding the 2 hour threshold. However,
while documentation is entered into PMIS on the amount of time the
procedures are used, recording the length of time that the procedures
are used does not allow TSA to verify that FSDs are actually requesting
permission to use the procedures for more than 2 hours as required. In
response to our recommendation on providing a means for measuring the
use of alternative screening procedures compared to the use of standard
procedures, such as counting baggage screened with alternative
screening procedures, DHS stated that TSA will evaluate the necessity
of requiring the recording of the number of bags screened by
alternative screening procedures, and if it finds it to be a useful
metric, it will require FSDs to include the number of bags in their
report in PMIS. We are encouraged that TSA will undertake this
evaluation, as we believe that it will allow TSA to have more assurance
that it has complete and accurate information on the use of these
procedures.
In response to our recommendation on developing performance measures
and targets for the use of alternative screening procedures, DHS
concurred and stated that TSA is currently meeting the intent of this
recommendation by monitoring and tracking the use of alternative
screening procedures through PMIS. DHS stated that using this system
has assisted TSA in identifying areas for improvement nationwide and
addressing local issues to minimize the need for alternative screening
procedures. DHS also stated that TSA intends to continue monitoring and
tracking the use of alternative screening procedures and to implement
the recommendations in this report for refining the data and evaluating
the need to make adjustments based on the current performance level.
While we support TSA's efforts to ensure the use of alternative
screening procedures is accurately reported in PMIS, given the security
effectiveness trade-offs associated with alternative screening
procedures, we do not believe that tracking the use of the procedures
with PMIS is sufficient to provide congressional and other decision
makers with an indication of the progress the agency expects to make in
minimizing the need to use the procedures at airports. Performance
measures and targets would provide this information and would help to
reinforce accountability and to ensure that managers focus on the
results they are striving to achieve regarding minimizing the use of
alternative screening procedures in their day-to-day activities.
We will send copies of the report to the Secretary of the Department of
Homeland Security; the Assistant Secretary, TSA; and interested
congressional committees as appropriate. We will also make copies
available to others on request. In addition, the report will be
available at no charge on GAO's Web site at [Hyperlink,
http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-3404 or berrickc@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff that made major
contributions to this report are listed in appendix III.
Sincerely yours,
Signed by:
Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To assess the Transportation Security Administration's (TSA) efforts to
screen all checked baggage using explosive detection systems (EDS) and
explosive trace detection machines (ETD), we addressed the following
questions: (1) How did TSA prioritize the use of standard and
alternative checked baggage screening procedures and what security
effectiveness trade-offs and operational efficiencies has TSA
identified in using these procedures to screen checked baggage for
explosives? (2) To what extent has TSA used standard and alternative
screening procedures to screen checked baggage for explosives and how
does TSA ensure that standard screening procedures are used whenever
possible? (3) What steps has TSA taken to reduce airports' need to use
alternative screening procedures and to establish performance measures
and targets for the use of the procedures?
To assess how TSA prioritized the use of checked baggage screening
procedures and to assess the trade-offs in security effectiveness and
operational efficiencies associated with various baggage screening
procedures, we analyzed TSA's standard operating procedures for using
these procedures. We also obtained and analyzed relevant legislation
and conducted a literature search to obtain information on screening
procedures, technologies, and related aviation trends. This search
identified various TSA reports, Department of Homeland Security
Inspector General reports, and aviation industry reports. We also
reviewed studies from the TSA Transportation Security Laboratory
regarding checked baggage screening. We interviewed officials from
various TSA offices, including the Chief Technologist's Office,
Aviation Programs, the Transportation Security Operations Center, the
Transportation Security Laboratory, Chief Operating Officer's Office,
and Office of Planning to learn about checked baggage screening
procedures and how they were given relative priority. We also
interviewed officials from air carriers, explosive detection systems
equipment manufacturers, and an airport industry association to obtain
information regarding TSA's checked baggage screening procedures. We
assessed the results from unannounced, undercover covert testing of
checked baggage screening operations conducted by TSA's Office of
Inspections and questioned TSA officials about the procedures used to
ensure the reliability of the covert test data. On the basis of their
answers, we believe that the covert test data are sufficiently reliable
for the purposes of our review. We also reviewed results of
unannounced, undercover covert testing of checked baggage screening
operations conducted at airports by Federal Security Directors (FSD)
and their staffs and collected as part of TSA's Screener Training
Exercises and Assessments program. After reviewing documentation from
TSA, we found the data from the Screener Training Exercises and
Assessments program were sufficiently reliable for the purposes of our
review.
To assess the extent to which TSA has used standard and alternative
screening procedures to screen checked baggage and how TSA ensures that
standard checked baggage screening procedures are used whenever
possible, we reviewed and analyzed TSA's Performance Management
Information System (PMIS) database, which contains information on
baggage screening operations and the use of alternative screening
procedures. We found several issues with these data, including, in some
cases, multiple occurrences of the use of alternative screening
procedures recorded as one occurrence and also, in some cases, more
than one procedure being used during an occurrence but the occurrence
was entered into the database as only one procedure because of the
constraints of the database. When we interviewed TSA officials about
these data reliability issues, officials acknowledged that airports may
have inaccurately reported some occurrences of the use of alternative
screening procedures. However, the officials stated that they were
working to correct the reporting problems and consider the data
generally reliable. On the basis of these discussions and our review of
the database, we found the data to be sufficiently reliable for the
purposes of this report, since the data provide overall trends in the
use of the procedures. To determine what controls are in place to track
and report the use of baggage screening procedures, we analyzed the
PMIS database and the PMIS user guide. We also analyzed TSA's operating
procedures for checked baggage and policy guidance and compared TSA's
procedures for ensuring that airports correctly report the use of
alternative screening procedures to the Comptroller General's Standards
for Internal Controls in the Federal Government. We also interviewed
officials from TSA's Office of Planning, Chief Operating Office,
Transportation Security Operations Center, and Inspections offices
concerning checked baggage screening procedures.
To assess the steps TSA has taken to reduce airports' need to use
alternative screening procedures and the measures and targets TSA has
set for alternative screening procedures, we analyzed TSA's PMIS data
and its standard operating procedures for checked baggage screening and
TSA's inventory of explosive detection systems. We found discrepancies
in the inventory data of explosive detection systems and worked with
TSA to resolve the discrepancies. TSA also completed a reconciliation
of the inventory database with data collected manually by TSA
officials. We also analyzed documentation from the TSA Transportation
Security Operations Center and interviewed TSA officials from the Chief
Operating Officer's Office, Office of Planning, the Office of Assistant
Secretary, and Chief Technology Office. Additionally, we examined TSA's
checked baggage performance measures and targets in the context of the
Government Performance and Results Act of 1993 (GPRA) requirements.
In addressing these objectives, we conducted site visits at nine
airports--three category X, one category I, four category II, and one
category IV airport. We chose these airports based on one or more of
the following factors: use of alternative screening procedures at the
airport as reported in PMIS, testing of screening equipment at the
airport, proximity to another airport being visited by GAO, and size of
airport. The results from our airport visits provided examples of
checked baggage screening operations and issues but cannot be
generalized beyond the airports visited because we did not use
statistical sampling in selecting the airports. We also conducted
structured interviews with FSDs and their staffs who were responsible
for 29 randomly selected airports. One FSD we interviewed was
responsible for two airports in our sample. We conducted all but one of
these interviews over the telephone. Using information from PMIS, we
selected airports that had reported using alternative screening
procedures and airports that had not reported using alternative
screening procedures between October 18, 2004, and December 21,
2004.[Footnote 49] Although the interviews were conducted with FSDs and
their staffs at random samples of airports, the samples are too small
to generalize the interview results with a high degree of statistical
confidence to all airports nationwide. The results from these
interviews do provide information about checked baggage screening
operations at the airports for which the FSDs and their staffs are
responsible.
We conducted our work from September 2004 through July 2006 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington. DC 20528:
July 25, 2006:
Ms. Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Ms. Berrick:
Thank you for the opportunity to comment on the Draft Report, "Aviation
Security: TSA Oversight of Checked Baggage Screening Procedures Could
Be Strengthened," GAO-06-869, the public version of GAO-06-291 SU,
which was issued on February 28, 2006. The recommendations and findings
of this report should help strengthen TSA's management of checked
baggage screening operations.
We are pleased to report that TSA has already undertaken initiatives to
improve checked baggage screening operations. For example, TSA has
refined its definitions and requirements for using alternative
screening procedures (ASP). As such, the amount of total checked
baggage screening time using ASP is very low. In addition, TSA has
deployed sufficient screening equipment to meet its requirement to
screen all checked baggage using explosives detection equipment,
refined staffing levels, and optimized screening procedures.
Covert checked baggage tests conducted by the TSA Office of Inspection
(01), formerly the Office of Internal Affairs and Program Review
(OIAPR), are designed to identify vulnerabilities in the checked
baggage screening system. OI covert checked baggage tests were
originally designed to evaluate the effectiveness of TSA's newly hired
and trained screener workforce and to assess the adequacy of security
systems and controls at all airports nationwide. OI committed to
complete covert checkpoint and checked baggage testing of all TSA and
Screening Partnership Program airports in a 3-year timeframe. During
the period of this review, 01 did not focus on ASP. As part of TSA's
new risked-based approach to covert testing, 01 will consider new
checked baggage screening testing protocols.
The following represents the Department of Homeland Security (DHS) and
TSA's responses to the recommendations contained in the draft report.
Recommendation 1: Use Performance Management Information System (PMIS)
data on use of alternative screening procedures in determining at which
airports to conduct covert testing and when to conduct testing at these
airports as part of the Office of Inspection's new risk based approach
to covert testing.
Concur: The Office of Inspection will consider PMIS information on
alternative screening procedures as part of TSA's new risk-based
approach to covert testing. Accordingly, we will develop new checked
baggage screening testing protocols that will assist TSA in
strengthening aviation security.
Recommendation 2: Conduct local covert testing of alternative screening
procedures to determine whether checked baggage Transportation Security
Officers (TSO) can detect simulated improvised explosives when using
these procedures.
Concur. TSA is currently making modifications to strengthen the local
covert testing programs. TSA expects these modifications will better
prepare TSOs to detect simulated improvised explosives.
Recommendation 3: Strengthen the monitoring and tracking of the use of
alternative screening procedures to help determine the progress the
agency is making in minimizing its need to use these procedures. This
effort would include continuing to address reporting problems in the
PMIS database system, keeping a record of calls requesting permission
to exceed 2 hours' use of the procedures, and providing a means for
measuring the use of alternative screening procedures compared to
standard procedures, such as counting baggage screened with alternative
screening procedures.
Concur: The Performance Management Information System (PMIS) is
enhanced with both functionality and data quality related processes to
ensure data reliability. The system alerts the user when a data field
is filled in with a value that falls outside of the operational norms
for a particular airport. Additionally, PMIS training and functionality
reviews occur on regular bases and user manuals and best practices are
updated consistently.
As stated in the report, TSA requires all Federal Security Directors
(FSD) to report any instance and type of alternative screening
procedures. Currently, all instances, types, and length of time using
alternative screening procedures are documented in PMIS. If the
procedures are needed for more than 2 hours, the Assistant
Administrator of Security Operations is notified and speaks to the FSD
requesting permission to extend the procedures for more than 2 hours.
Since the amount of time alternative screening procedures are used is
recorded in PMIS, there is no further documentation required for
exceeding the 2-hour threshold. TSA will evaluate the necessity of
requiring the recording of the number of bags screened by ASP and if
found to be a useful metric, will require FSD's to include the number
of bags in their report in PMIS.
Recommendation 4: Develop performance measures and performance targets
for the use of alternative screening procedures in checked baggage
screening, perhaps as part of the checked baggage screening program
performance index, to help TSA measure its progress in working toward
minimizing the need to use alternative screening procedures at airports
and to have more complete information on the overall performance of the
checked baggage screening system.
Concur: TSA is currently meeting the intent of this recommendation by
monitoring and tracking the use of alternative screening procedures
through PMIS. Using this system has assisted TSA in identifying areas
of improvement nationwide and addressing local issues to minimize the
need for alternative screening procedures. Consequently, TSA is
spending a very low percent of its total checked baggage screening time
using alternative screening procedures.
TSA intends to continue monitoring and tracking the use of alternative
screening procedures and to implement the recommendation in this report
for refining the data we capture to assist us in evaluating the need to
make adjustments based on our current performance level.
Thank you for the opportunity to provide comments on your draft report.
The Department and TSA appreciate the amount of time and work invested
in this type of study.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Director Departmental GAO/OIG Liaison Office:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Cathleen A. Berrick (202) 512-3404:
Acknowledgments:
In addition to the contact named above, David Alexander, Leo Barbour,
Chuck Bausell Jr., Amy Bernstein, Kevin Copping, Katherine Davis, Josh
Diosomito, Christine Fossett, Richard Hung, Benjamin Jordan, Thomas
Lombardi, Lisa Shibata, Maria Strudwick, and Alper Tunca made key
contributions to this report.
[End of section]
Related GAO Products:
Aviation Security: TSA Has Strengthened Efforts to Plan for the Optimal
Deployment of Checked Baggage Screening Systems but Funding
Uncertainties Remain GAO-06-875T, June 29, 2006:
Aviation Security: Enhancements Made in Passenger and Checked Baggage
Screening, but Challenges Remain. GAO-06-371T. Washington, D.C.: April
4, 2006.
Aviation Security: Transportation Security Administration Has Made
Progress in Managing a Federal Security Workforce and Ensuring Security
at U.S. Airports, but Challenges Remain. GAO-06-597T. Washington, D.C.:
April 4, 2006.
Aviation Security: Progress Made to Set Up Program Using Private-Sector
Airport Screeners, but More Work Remains. GAO-06-166. Washington, D.C.:
March 31, 2006.
Aviation Security: Significant Management Challenges May Adversely
Affect Implementation of the Transportation Security Administration's
Secure Flight Program. GAO-06-374T. Washington, D.C.: February 9, 2006.
Aviation Security: Federal Air Marshal Service Could Benefit from
Improved Planning and Controls. GAO-06-203. Washington, D.C.: November
28, 2005.
Aviation Security: Federal Action Needed to Strengthen Domestic Air
Cargo Security. GAO-06-76. Washington, D.C.: October 17, 2005.
Transportation Security Administration: More Clarity on the Authority
of Federal Security Directors Is Needed. GAO-05-935. Washington, D.C.:
September 23, 2005.
Aviation Security: Flight and Cabin Crew Member Security Training
Strengthened, but Better Planning and Internal Controls Needed. GAO-05-
781. Washington, D.C.: September 6, 2005.
Aviation Security: Transportation Security Administration Did Not Fully
Disclose Uses of Personal Information during Secure Flight Program
Testing in Initial Privacy Notes, but Has Recently Taken Steps to More
Fully Inform the Public. GAO-05-864R. Washington, D.C.: July 22, 2005.
Aviation Security: Better Planning Needed to Optimize Deployment of
Checked Baggage Screening Systems. GAO-05-896T. Washington, D.C.: July
13, 2005.
Aviation Security: Screener Training and Performance Measurement
Strengthened, but More Work Remains. GAO-05-457. Washington, D.C.: May
2, 2005.
Aviation Security: Secure Flight Development and Testing Under Way, but
Risks Should Be Managed as System Is Further Developed. GAO-05-356.
Washington, D.C.: March 28, 2005.
Aviation Security: Systematic Planning Needed to Optimize the
Deployment of Checked Baggage Screening Systems. GAO-05-365.
Washington, D.C.: March 15, 2005.
Aviation Security: Measures for Testing the Effect of Using Commercial
Data for the Secure Flight Program. GAO-05-324. Washington, D.C.:
February 23, 2005.
Transportation Security: Systematic Planning Needed to Optimize
Resources. GAO-05-357T. Washington, D.C.: February 15, 2005.
Aviation Security: Preliminary Observations on TSA's Progress to Allow
Airports to Use Private Passenger and Baggage Screening Services. GAO-
05-126. Washington, D.C.: November 19, 2004.
General Aviation Security: Increased Federal Oversight Is Needed, but
Continued Partnership with the Private Sector Is Critical to Long-Term
Success. GAO-05-144. Washington, D.C.: November 10, 2004.
Aviation Security: Further Steps Needed to Strengthen the Security of
Commercial Airport Perimeters and Access Controls. GAO-04-728.
Washington, D.C.: June 4, 2004.
Transportation Security Administration: High-Level Attention Needed to
Strengthen Acquisition Function. GAO-04-544. Washington, D.C.: May 28,
2004.
Aviation Security: Challenges in Using Biometric Technologies. GAO-04-
785T. Washington, D.C.: May 19, 2004.
Nonproliferation: Further Improvements Needed in U.S. Efforts to
Counter Threats from Man-Portable Air Defense Systems. GAO-04-519.
Washington, D.C.: May 13, 2004.
Aviation Security: Private Screening Contractors Have Little
Flexibility to Implement Innovative Approaches. GAO-04-505T.
Washington, D.C.: April 22, 2004.
Aviation Security: Improvement Still Needed in Federal Aviation
Security Efforts. GAO-04-592T. Washington, D.C.: March 30, 2004.
Aviation Security: Challenges Delay Implementation of Computer-
Assisted Passenger Prescreening System. GAO-04-504T. Washington, D.C.:
March 17, 2004.
Aviation Security: Factors Could Limit the Effectiveness of the
Transportation Security Administration's Efforts to Secure Aerial
Advertising Operations. GAO-04-499R. Washington, D.C.: March 5, 2004.
Aviation Security: Computer-Assisted Passenger Prescreening System
Faces Significant Implementation Challenges. GAO-04-385. Washington,
D.C.: February 13, 2004.
Aviation Security: Challenges Exist in Stabilizing and Enhancing
Passenger and Baggage Screening Operations. GAO-04-440T. Washington,
D.C.: February 12, 2004.
The Department of Homeland Security Needs to Fully Adopt a Knowledge-
based Approach to Its Counter-MANPADS Development Program. GAO-04-341R.
Washington, D.C.: January 30, 2004.
Aviation Security: Efforts to Measure Effectiveness and Strengthen
Security Programs. GAO-04-285T. Washington, D.C.: November 20, 2003.
Aviation Security: Federal Air Marshal Service Is Addressing Challenges
of Its Expanded Mission and Workforce, but Additional Actions Needed.
GAO-04-242. Washington, D.C.: November 19, 2003.
Aviation Security: Efforts to Measure Effectiveness and Address
Challenges. GAO-04-232T. Washington, D.C.: November 5, 2003.
Airport Passenger Screening: Preliminary Observations on Progress Made
and Challenges Remaining. GAO-03-1173. Washington, D.C.: September 24,
2003.
Aviation Security: Progress Since September 11, 2001, and the
Challenges Ahead. GAO-03-1150T. Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Enhance Security
Efforts. GAO-03-1154T. Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Help Address Security
Challenges. GAO-03-843. Washington, D.C.: June 30, 2003.
Federal Aviation Administration: Reauthorization Provides Opportunities
to Address Key Agency Challenges. GAO-03-653T. Washington, D.C.: April
10, 2003.
Transportation Security: Post-September 11th Initiatives and Long-Term
Challenges. GAO-03-616T. Washington, D.C.: April 1, 2003.
Airport Finance: Past Funding Levels May Not Be Sufficient to Cover
Airports' Planned Capital Development. GAO-03-497T. Washington, D.C.:
February 25, 2003.
Transportation Security Administration: Actions and Plans to Build a
Results-Oriented Culture. GAO-03-190. Washington, D.C.: January 17,
2003.
Aviation Safety: Undeclared Air Shipments of Dangerous Goods and DOT's
Enforcement Approach. GAO-03-22. Washington, D.C.: January 10, 2003.
Aviation Security: Vulnerabilities and Potential Improvements for the
Air Cargo System. GAO-03-344. Washington, D.C.: December 20, 2002.
Aviation Security: Registered Traveler Program Policy and
Implementation Issues. GAO-03-253. Washington, D.C.: November 22, 2002.
Airport Finance: Using Airport Grant Funds for Security Projects Has
Affected Some Development Projects. GAO-03-27. Washington, D.C.:
October 15, 2002.
Commercial Aviation: Financial Condition and Industry Responses Affect
Competition. GAO-03-171T. Washington, D.C.: October 2, 2002.
Aviation Security: Transportation Security Administration Faces
Immediate and Long-Term Challenges. GAO-02-971T. Washington, D.C.: July
25, 2002.
Aviation Security: Information Concerning the Arming of Commercial
Pilots. GAO-02-822R. Washington, D.C.: June 28, 2002.
Aviation Security: Vulnerabilities in, and Alternatives for, Preboard
Screening Security Operations. GAO-01-1171T. Washington, D.C.:
September 25, 2001.
Aviation Security: Weaknesses in Airport Security and Options for
Assigning Screening Responsibilities. GAO-01-1165T. Washington, D.C.:
September 21, 2001.
Homeland Security: A Framework for Addressing the Nation's Efforts. GAO-
01-1158T. Washington, D.C.: September 21, 2001.
Aviation Security: Terrorist Acts Demonstrate Urgent Need to Improve
Security at the Nation's Airports. GAO-01-1162T. Washington, D.C.:
September 20, 2001.
Aviation Security: Terrorist Acts Illustrate Severe Weaknesses in
Aviation Security. GAO-01-1166T. Washington, D.C.: September 20, 2001.
FOOTNOTES
[1] Congress subsequently extended this deadline by 1 year.
[2] TSA interpreted ATSA's reference to "explosive detection systems"
to allow for the deployment of EDS and ETD to satisfy the mandate.
[3] The nonstandard ways that the machines are used is sensitive
security information.
[4] GAO, Aviation Security: Challenges Exist in Stabilizing and
Enhancing Passenger and Baggage Screening Operations, GAO-04-440T
(Washington, D.C.: Feb. 12, 2004).
[5] GAO, Aviation Security: TSA Management of Checked Baggage Screening
Procedures Could Be Improved, GAO-06-291SU (Washington, D.C.: Feb. 28,
2006).
[6] GAO, Analysis of TSA's Covert Testing for Checked Baggage
Screening, GAO-06-317C (Washington, D.C.: Feb. 28, 2006).
[7] PMIS is a Web-based application used by TSA field staff to submit
data to headquarters on operations and performance.
[8] The Federal Security Director is the ranking TSA authority
responsible for the leadership and coordination of TSA security
activities at the nation's commercial airports.
[9] Nonprobability sampling is a method of sampling where observations
are selected in a manner that is not completely random, usually using
specific characteristics of the population as criteria. Results from
nonprobability samples cannot be used to make inferences about a
population because in a nonprobability sample, some elements of the
population being studied have no chance or an unknown chance of being
selected as part of the sample.
[10] Alternative hybrid procedures use a combination of EDS and ETD at
a screening station that is usually configured only for EDS. At some
screening stations, TSA has also allowed the use of a hybrid
configuration as a standard screening procedure.
[11] TSA does not require airports to report information on the number
of bags screened using alternative screening procedures. To derive its
estimate of use of alternative screening procedures across the system,
TSA used alternative screening procedures baggage counts only from
those airports that voluntarily reported the information in order to
calculate the average number of bags screened per hour. TSA then used
this calculation of the average bags per hour and the total number of
screening hours using the procedures to make this estimate. Because TSA
did not have baggage counts for all of the occurrences of alternative
screening procedures, this estimate may be inaccurate.
[12] TSA determined that the estimated use of alternative screening
procedures is sensitive security information.
[13] The specific details on the number of hours alternative screening
procedures were used are sensitive security information.
[14] At 46 airports, a combined total of 154 EDS machines were added;
at 22 airports, a combined total of 62 EDS machines were removed; and
47 airports maintained the same number of EDS machines. One airport
with two EDS machines in inventory in 2005 was not on the 2004 or 2002
inventory lists.
[15] GAO, Aviation Security: Systematic Planning Needed to Optimize the
Deployment of Checked Baggage Screening Systems, GAO-05-365
(Washington, D.C.: Mar.15, 2005).
[16] Computer-aided tomography is a method of producing a three-
dimensional image of the internal structures of a solid object by the
observation and recording of the differences in the effects on the
passage of waves of energy impinging on those structures.
[17] GAO-04-440T.
[18] TSA also moved additional ETD machines to screening stations to
allow for ETD screening. Prior to March 2005, TSA had categorized this
procedure, "additional ETDs," as an alternative screening procedure.
Beginning in March 2005, TSA began to categorize this procedure as a
standard screening procedure. We did not include this procedure in our
analysis of PMIS data on alternative screening procedures.
[19] ATSA, as codified at 49 U.S.C. §44901(d)-(e), authorizes TSA to
screen checked baggage using canine screening, physical inspection, or
a bag match program if explosive detection equipment is unavailable.
[20] TSA officials stated that two TSOs can use an ETD machine at the
same time, raising the baggage screened throughput to 72 bags per hour
[21] GAO-05-365.
[22] The stand-alone EDS equipment TSA is considering for these
airports includes surplus machines no longer needed once airports
installed in-line EDS machines and a newly certified EDS machine
appropriate for baggage screening operations that require a lower
throughput (bags screened per hour).
[23] Intelligence Reform and Terrorism Prevention Act of 2004, Pub. L.
No. 108-458, § 4019, 118 Stat. 3638, 3721-22.
[24] TSA has determined that the details of its analysis of the optimal
checked baggage screening solutions are sensitive security information.
[25] The specifics of the trade-offs in security effectiveness have
been determined to be sensitive security information.
[26] One alternative screening procedure that involves the use of EDS
and ETD is not discussed in this report because TSA designated the
procedure as sensitive security information.
[27] GAO, Aviation Security: Screener Training and Performance
Measurement Strengthened, but More Work Remains, GAO-05-457
(Washington, D.C.: May 2, 2005).
[28] TSA classifies the over 400 airports in the United States into one
of five categories--X, I, II, III, and IV. Generally, category X
airports have the largest number of passenger boardings and category IV
airports have the smallest number.
[29] Although FSDs and their staffs are not required to report the
number of bags screened with alternative screening procedures into
PMIS, some FSDs and staff from airports that reported using alternative
screening procedures reported this information in comment fields on
occurrences of use of alternative screening procedures between October
2004 and September 2005. The number of FSDs and the number of
occurrences have been determined to be sensitive security information.
[30] TSA does not require airports to report information on the number
of bags screened using alternative screening procedures. To derive an
estimate of use of alternative screening procedures across the system,
TSA used alternative screening procedures baggage counts only from
those airports that voluntarily reported the information in order to
calculate the average number of bags screened per hour. TSA then used
this calculation of the average bags per hour and the total number of
screening hours using the procedures to make this estimate. Because TSA
did not have baggage counts for all of the occurrences of alternative
screening procedures, this estimate may be inaccurate.
[31] TSA determined that its estimate of the use of alternative
screening procedures is sensitive security information.
[32] GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).
[33] TSA determined that the other factors that may influence the
number of analyses completed for a bag screened using ETD are sensitive
security information.
[34] The comments field is a PMIS database field used to add
descriptive data on the alternative screening procedures occurrence.
TSA determined that the exact number of airports that voluntarily
reported intermittent use of alternative screening procedures in the
comments field is sensitive security information.
[35] TSA officials stated that they eliminated this requirement because
headquarters could get information on use of alternative screening
procedures through PMIS.
[36] TSA classified the number of sheets that we reviewed as sensitive
security information.
[37] We selected sheets to analyze based on the completeness of the
information contained on the sheets. Each sheet selected had a date, an
airport code or name, a beginning and end time of use of alternative
screening procedures, type of alternative screening procedure used, and
reason for using the alternative screening procedure.
[38] TSA classified the number of occurrences that were more than 2
hours as sensitive security information.
[39] The optimization visits are also used to improve the design of
passenger and baggage checkpoints, validate the TSO staffing model at
the airport, evaluate staffing and scheduling practices, and determine
compliance with the standard operating procedures.
[40] GAO-05-365.
[41] Each of the airports' in-line systems became operational on a
different date. The number of airports that reported using alternative
screening procedures due to equipment failures and high passenger and
baggage volumes after their systems were operational has been
determined to be sensitive security information.
[42] GAO-05-365.
[43] Section 4023 of the Intelligence Reform and Terrorism Prevention
Act of 2004 requires TSA to develop and submit to the appropriate
congressional committees, standards for determining aviation security
staffing at commercial airports no later than 90 days after December
17, 2004, the date of the act's enactment, and GAO to conduct an
analysis of these standards. These standards were submitted to Congress
on June 22, 2005, and GAO is currently reviewing these standards.
[44] The details of the strategic planning framework for the checked
baggage screening program constitute sensitive security information.
[45] Section 4019(d) of the Intelligence Reform and Terrorism
Prevention Act of 2004 requires TSA to complete a cost-sharing study in
collaboration with industry stakeholders to review the benefits and
cost of in-line checked baggage screening systems, innovative financing
approaches, formulas for cost sharing between different government
entities and the private sector, and potential cost-saving approaches.
[46] GAO-05-365.
[47] TSA determined that the exact number of FSDs that anticipate
continued or increased use of alternative screening procedures is
sensitive security information.
[48] According to the Government Performance and Results Act, the
Office of Management and Budget, and GAO, outcomes assess actual
results as compared with the intended results that occur from carrying
out a program or activity. Outcomes are the results of a program or
activity. For further information, see GAO, Results-Oriented
Government: GPRA Has Established a Solid Foundation for Achieving
Greater Results, GAO-04-38 (Washington, D.C.: Mar. 10, 2004).
[49] TSA has determined that the exact number of airports we selected
is sensitive security information.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
