Aviation Security
TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify Passengers on the No Fly and Selectee Lists, but Expects Ultimate Solution to Be Implementation of Secure Flight Gao ID: GAO-08-992 September 9, 2008Air carriers remain a front-line defense against acts of terrorism that target the nation's civil aviation system. A key responsibility of air carriers is to check passengers' names against terrorist watch-list records to identify persons who should be prevented from boarding (the No Fly List) or who should undergo additional security scrutiny (the Selectee List). Eventually, the Transportation Security Administration (TSA) is to assume this responsibility through its Secure Flight program. However, due to program delays, air carriers retain this role. You asked GAO to review domestic air carriers' watch-list-matching processes. GAO examined (1) the watch-list-matching requirements air carriers must follow that have been established by TSA, and (2) the extent to which TSA has assessed air carriers' compliance with these requirements. GAO reviewed TSA's security directives, internal guidance used by TSA's inspectors to assess air carriers' compliance with requirements, and inspection results, as well as interviewed staff from 14 of 95 domestic air carriers (selected to reflect a range in operational sizes). This report is the public version of a restricted report (GAO-08-453SU) issued in July 2008.
TSA's requirements for domestic air carriers to conduct watch-list matching include a requirement to identify passengers whose names are either identical or similar to those on the No Fly and Selectee lists. Similar-name matching is important because individuals on the watch list may try to avoid detection by making travel reservations using name variations. According to TSA's Office of Intelligence, there have been incidents of air carriers failing to identify potential matches by not successfully conducting similar-name matching. However, until revisions were initiated in April 2008, TSA's security directives did not specify what types of similar-name variations were to be considered by air carriers. Thus, in interviews with 14 air carriers GAO found inconsistent approaches to conducting similar-name matching. Due to such inconsistency, a passenger could be identified as a match by one air carrier and not by another. In addition, not every air carrier reported conducting similar name comparisons. Further, in January 2008, TSA conducted an evaluation of air carriers and found deficiencies in their capability to conduct similar-name matching. Shortly thereafter, in April 2008, TSA revised the No Fly List security directive to specify a baseline capability for conducting watch-list matching, and TSA reported that it planned to similarly revise the Selectee List security directive. Because the baseline capability requires that air carriers compare only the types of name variations specified in the directive, TSA recognizes that the new baseline capability will not address all vulnerabilities. However, TSA emphasized that establishing the baseline capability should improve air carriers' performance of watch-list matching and, in TSA's view, is the best interim solution pending the implementation of Secure Flight. TSA has undertaken various efforts to assess domestic air carriers' compliance with watch-list matching requirements; however, until 2008, TSA had conducted limited testing of air carriers' similar-name-matching capability. In 2005, for instance, TSA conducted an evaluation to determine whether air carriers had the capability to identify names that were identical--but not similar--to those on the No Fly List. Also, regarding regularly conducted inspections, TSA's guidance did not specifically direct inspectors to test air carriers' similar-name-matching capability, nor did the guidance specify the number or types of name variations to be assessed. Records in TSA's database for regular inspections conducted during 2007 made reference to name-match testing in 61 of the 1,145 watch-list-related inspections that GAO reviewed. Without criteria or standards for air carriers to follow in comparing name variations, TSA did not have a uniform basis for assessing compliance and addressing deficiencies. However, during the course of GAO's review and prompted by findings of the evaluation conducted in January 2008, TSA reported that its guidance for inspectors would be revised to help ensure air carriers' compliance with security directives. Although TSA has plans to strengthen its oversight of air carriers' compliance with the revised security directives, it is too early to assess the extent of such oversight since TSA's efforts are ongoing and not completed.
GAO-08-992, Aviation Security: TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify Passengers on the No Fly and Selectee Lists, but Expects Ultimate Solution to Be Implementation of Secure Flight
This is the accessible text file for GAO report number GAO-08-992
entitled 'Aviation Security: TSA Is Enhancing Its Oversight of Air
Carrier Efforts to Identify Passengers on the No Fly and Selectee
Lists, but Expects Ultimate Solution to Be Implementation of Secure
Flight' which was released on September 10, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
September 2008:
Aviation Security:
TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify
Passengers on the No Fly and Selectee Lists, but Expects Ultimate
Solution to Be Implementation of Secure Flight:
Aviation Security and Watch List Matching:
GAO-08-992:
GAO Highlights:
Highlights of GAO-08-992, a report to congressional committees.
Why GAO Did This Study:
Air carriers remain a front-line defense against acts of terrorism that
target the nation‘s civil aviation system. A key responsibility of air
carriers is to check passengers‘ names against terrorist watch-list
records to identify persons who should be prevented from boarding (the
No Fly List) or who should undergo additional security scrutiny (the
Selectee List). Eventually, the Transportation Security Administration
(TSA) is to assume this responsibility through its Secure Flight
program. However, due to program delays, air carriers retain this role.
You asked GAO to review domestic air carriers‘ watch-list-matching
processes. GAO examined (1) the watch-list-matching requirements air
carriers must follow that have been established by TSA, and (2) the
extent to which TSA has assessed air carriers‘ compliance with these
requirements. GAO reviewed TSA‘s security directives, internal guidance
used by TSA‘s inspectors to assess air carriers‘ compliance with
requirements, and inspection results, as well as interviewed staff from
14 of 95 domestic air carriers (selected to reflect a range in
operational sizes). This report is the public version of a restricted
report (GAO-08-453SU) issued in July 2008.
What GAO Found:
TSA‘s requirements for domestic air carriers to conduct watch-list
matching include a requirement to identify passengers whose names are
either identical or similar to those on the No Fly and Selectee lists.
Similar-name matching is important because individuals on the watch
list may try to avoid detection by making travel reservations using
name variations. According to TSA‘s Office of Intelligence, there have
been incidents of air carriers failing to identify potential matches by
not successfully conducting similar-name matching. However, until
revisions were initiated in April 2008, TSA‘s security directives did
not specify what types of similar-name variations were to be considered
by air carriers. Thus, in interviews with 14 air carriers GAO found
inconsistent approaches to conducting similar-name matching. Due to
such inconsistency, a passenger could be identified as a match by one
air carrier and not by another. In addition, not every air carrier
reported conducting similar name comparisons. Further, in January 2008,
TSA conducted an evaluation of air carriers and found deficiencies in
their capability to conduct similar-name matching. Shortly thereafter,
in April 2008, TSA revised the No Fly List security directive to
specify a baseline capability for conducting watch-list matching, and
TSA reported that it planned to similarly revise the Selectee List
security directive. Because the baseline capability requires that air
carriers compare only the types of name variations specified in the
directive, TSA recognizes that the new baseline capability will not
address all vulnerabilities. However, TSA emphasized that establishing
the baseline capability should improve air carriers‘ performance of
watch-list matching and, in TSA‘s view, is the best interim solution
pending the implementation of Secure Flight.
TSA has undertaken various efforts to assess domestic air carriers‘
compliance with watch-list matching requirements; however, until 2008,
TSA had conducted limited testing of air carriers‘ similar-name-
matching capability. In 2005, for instance, TSA conducted an evaluation
to determine whether air carriers had the capability to identify names
that were identical”but not similar”to those on the No Fly List. Also,
regarding regularly conducted inspections, TSA‘s guidance did not
specifically direct inspectors to test air carriers‘ similar-name-
matching capability, nor did the guidance specify the number or types
of name variations to be assessed. Records in TSA‘s database for
regular inspections conducted during 2007 made reference to name-match
testing in 61 of the 1,145 watch-list-related inspections that GAO
reviewed. Without criteria or standards for air carriers to follow in
comparing name variations, TSA did not have a uniform basis for
assessing compliance and addressing deficiencies. However, during the
course of GAO‘s review and prompted by findings of the evaluation
conducted in January 2008, TSA reported that its guidance for
inspectors would be revised to help ensure air carriers‘ compliance
with security directives. Although TSA has plans to strengthen its
oversight of air carriers‘ compliance with the revised security
directives, it is too early to assess the extent of such oversight
since TSA‘s efforts are ongoing and not completed.
What GAO Recommends:
GAO is not making any recommendations because TSA initiated actions in
April 2008 to strengthen watch-list-matching requirements and its
oversight of air carriers‘ implementation of these requirements.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-992]. For more
information, contact Cathleen A. Berrick at (202) 512-3404 or
berrickc@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
TSA Took Action in 2008 to Enhance Watch-List Matching Conducted by Air
Carriers but Believes the Ultimate Solution Will Be Implementation of
Secure Flight:
Until a 2008 Special Emphasis Inspection, TSA Had Conducted Limited
Testing of Air Carriers' Capability to Perform Similar-Name Matching:
Concluding Observations:
Agency Comments:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Overview of Selected Domestic Air Carriers' Watch-List-
Matching Processes:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: TSA Watch-List-Matching Requirements Prior to the April 2008
Revision to the No Fly List Security Directive:
Table 2: Requirements for Matching Passenger Data to No Fly and
Selectee Lists and Inspection Guidelines Used to Assess Compliance with
the Requirements:
Table 3: Watch-List-Matching Requirements and the Related Inspection
Guidelines (Fiscal Year 2007):
Figure:
Figure 1: Overview of the Current Passenger Watch-List-Matching
Process:
Abbreviations:
CAPPS: Computer Assisted Passenger Prescreening System:
DHS: Department of Homeland Security:
FBI: Federal Bureau of Investigation:
PARIS: Performance and Results Information System:
PNR: passenger name record:
TRIP: Traveler Redress Inquiry Program:
TSA: Transportation Security Administration:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 9, 2008:
Congressional Committees:
Currently, more than 6 years after the terrorist attacks on September
11, 2001, air carriers remain a front-line defense against acts of
terrorism that target the nation's civil aviation system. A key aspect
of air carriers' security responsibilities is to conduct preboarding
checks of all passengers' personal information against terrorist watch-
list records that contain information on thousands of individuals with
known or potential links to terrorism. This process, referred to
hereafter as watch-list matching, involves comparing passenger data--
most prominently name and date of birth--against the No Fly List to
identify individuals who should be prevented from boarding an aircraft,
and against the Selectee List to identify individuals who must undergo
enhanced screening at the checkpoint prior to boarding.[Footnote 1]
The Transportation Security Administration (TSA) requires that domestic
air carriers operating to, from, and within the United States conduct
watch-list matching.[Footnote 2] Data compiled by TSA's Office of
Intelligence indicate that, at times, these air carriers have failed to
identify individuals who are on the No Fly List. For instance, for the
3-year period from January 2005 through December 2007, TSA documented
several known incidents involving individuals on the No Fly List who,
because of failures of domestic air carriers' watch-list-matching
processes, were allowed to board international flights traveling to or
from the United States.[Footnote 3] Data for these types of incidents,
referred to as false negative watch-list-matching results, generally
are not available for domestic flights--that is, domestic air carrier
operations between two points within the United States or its
territories.[Footnote 4] Nevertheless, because the requirements for air
carriers to conduct watch-list matching are generally the same
irrespective of the departure or arrival location, false negative
incidents may be occurring on domestic flights if watch-listed
individuals attempt to fly domestically.
At present, domestic air carriers generally conduct watch-list matching
in accordance with requirements that TSA sets forth in security
directives--a regulatory tool through which TSA may impose security
measures on a regulated entity, in this case air carriers, generally in
response to an immediate or imminent threat.[Footnote 5] For example,
security directives require that air carriers execute comparisons of
passenger information with No Fly and Selectee list information within
24 hours of a flight's scheduled departure. TSA also has responsibility
for overseeing how air carriers implement the requirements set forth in
security directives. Critical to this effort are the agency's aviation
security inspectors, who oversee air carrier efforts at air carriers'
corporate security offices (principal security inspectors) and at
airport locations (transportation security inspectors).
As required by law, TSA is to take over from air carriers the function
of matching passenger information to the No Fly and Selectee lists for
domestic flights.[Footnote 6] Since 2003, we have been assessing TSA's
efforts to develop such a watch-list-matching program, currently known
as Secure Flight, and have reported that significant challenges,
including the need to follow a more structured systems development
approach and to fully address how the program would protect passengers'
privacy rights, have delayed its implementation.[Footnote 7] In April
2008, we reported that TSA has made significant progress in developing
Secure Flight, but that challenges remained in a number of areas,
including the need to develop more robust cost and schedule estimates.
[Footnote 8] We are continuing to review TSA's development and
implementation of Secure Flight in response to requests from the U.S.
Senate (Committee on Commerce, Science, and Transportation, and its
Subcommittee on Aviation Operations, Safety, and Security; Committee on
Appropriations, Subcommittee on Homeland Security; Committee on
Homeland Security and Governmental Affairs; and Committee on the
Judiciary) and the U.S. House of Representatives (Committee on
Transportation and Infrastructure, Committee on Homeland Security, and
the Committee on Oversight and Government Reform). In addition, the
Consolidated Appropriations Act, 2008, requires that we report to the
Committees on Appropriations of the Senate and House of Representatives
on the Department of Homeland Security's (DHS) certification of 10
conditions outlined in section 522(a) of the Department of Homeland
Security Appropriations Act, 2005, related to the development and
implementation of the Secure Flight program.[Footnote 9] The report is
to be submitted 90 days after the DHS's Secretary certifies that all 10
conditions have been successfully met.
Pending Secure Flight's implementation, air carriers will continue to
have primary responsibility for the watch-list-matching function. In
conjunction with our ongoing evaluation of Secure Flight, we testified
in June 2006 that due to delays and uncertainty surrounding Secure
Flight's implementation, some air carriers were enhancing their watch-
list-matching processes. We further identified that these improvements,
though beneficial to the respective air carrier's operations, could
further exacerbate differences that currently exist among the various
air carriers, and could result in varying levels of effectiveness
across air carriers in matching passenger information to the No Fly and
Selectee lists.[Footnote 10]
Due to the importance of identifying passengers who may pose a threat
to commercial aviation, we were asked to review the current processes
that domestic air carriers use to conduct watch-list matching for
domestic flights.[Footnote 11] Accordingly, this report addresses the
following questions:
* What are TSA's requirements for domestic air carriers to conduct
watch-list matching for domestic flights?
* To what extent has TSA assessed domestic air carriers' compliance
with watch-list-matching requirements?
This report is a public version of the restricted report (GAO-08-453SU)
that we provided to you on July 10, 2008. DHS and TSA deemed some of
the information in the restricted report as Sensitive Security
Information, which must be protected from public disclosure. Therefore,
this report omits this information, such as the specific details
associated with the current processes that domestic air carriers use to
conduct watch-list matching. Although the information provided in this
report is more limited in scope, it addresses the same principal
questions as the restricted report. Also, the overall methodology used
for both reports is generally the same.
To determine TSA's requirements for matching passenger information
against the No Fly and Selectee lists for domestic flights, we reviewed
TSA's security directives, policies, and other guidance applicable to
watch-list matching. We also interviewed officials at TSA's Office of
Transportation Sector Network Management, Office of Security
Operations, Office of Intelligence, and Office of Chief Counsel. We
also reviewed key policy documents for Secure Flight, as well as our
most recent reports and testimonies on the program to determine the
planned matching process. In addition, to identify the composition and
use of the No Fly and Selectee lists, we interviewed officials with the
Department of Justice, Federal Bureau of Investigation's (FBI)
Terrorist Screening Center, which has responsibility for managing the
use of terrorist information in screening processes.[Footnote 12] We
also contacted officials from a federally sponsored working group on
identity matching to discuss the challenges associated with name-based
matching. Moreover, to understand how air carriers have responded to
watch-list-matching requirements, we conducted telephone interviews
with officials from 14 domestic air carriers.[Footnote 13] Our
selection of air carriers was based, in part, on operational size with
the goal of obtaining a range of sizes based on operating revenue. For
example, the Department of Transportation classifies eight of the air
carriers in our review as major air carriers that provide service to
locations across the nation and, with the exception of one air carrier,
around the world.[Footnote 14] The remaining six air carriers had
comparatively smaller business operations that generally provided
service covering a geographical area, such as the Pacific Northwest, or
commuter service.[Footnote 15] Although the 14 air carriers we spoke
with represent a range in the types of air carriers that conduct watch-
list matching, and, according to our calculations, accounted for
approximately 70 percent of all passengers that boarded domestic
flights in 2005, the results of our telephone interviews are not
generalizable to the domestic operations of all domestic air carriers.
However, our selection allowed us to understand how watch-list matching
was performed for the majority of passengers flying domestically in
2005. In addition, although our work summarizes the 14 air carriers'
watch-list-matching capabilities as described to us in interviews, we
did not independently verify each air carrier's reported method of
implementation to determine the reliability of the data.
To determine the extent to which TSA has assessed domestic air
carriers' compliance with watch-list-matching requirements in the No
Fly and Selectee list security directives,[Footnote 16] we first
assessed TSA's inspection process, including the focus of inspections
and inspection methods. We also examined TSA's national inspection
plans and related guidance and policy documents. Further, at TSA
headquarters, we interviewed officials responsible for developing and
implementing inspection guidance and compiling and analyzing inspection
results. Specifically, we interviewed representatives from the Office
of Security Operations and the Office of Transportation Sector Network
Management. We analyzed the results of both regular inspections (i.e.,
inspections conducted in conjunction with annual inspection plans) and
nonroutine watch-list-related inspections that TSA conducted. For
instance, we analyzed regular watch-list-related inspections that TSA
conducted during fiscal year 2007 to ensure that air carriers were in
compliance with applicable requirements. Although we concluded that
these regular inspection data were sufficiently reliable for the
purposes of this report, we have concerns about the potential for error
based on TSA's process for querying its inspection database (we discuss
these concerns in more detail in app. I). To assess data reliability,
we performed electronic testing, discussed the data system and any data
inconsistencies we found with knowledgeable TSA officials, and reviewed
existing information about the data system. We also reviewed results
from a special emphasis assessment that TSA conducted in 2005, and a
special emphasis inspection it conducted in January 2008, both of which
addressed air carriers' capability to conduct watch-list matching.
[Footnote 17] We determined that the sampling and related procedures
used for the special emphasis assessment were insufficient for
providing a reliable estimate of the success rate of all attempted
matches by air carriers. We did not assess the initial data TSA
provided in February 2008 for the special emphasis inspection it
conducted the previous month.[Footnote 18]
We conducted this performance audit from July 2006 to September 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on the audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on the audit objectives. More details about the scope
and methodology of our work are presented in appendix I.
Results in Brief:
TSA has issued two security directives (one for the No Fly List and
another for the Selectee List) that delineate requirements related to
air carrier watch-list matching, including the identification of
passengers with names similar to those on the lists. Identifying
passengers with names similar to those on the No Fly and Selectee
lists--a process TSA refers to as similar-name matching--is a critical
component of watch-list matching because individuals may travel using
abbreviated name forms or other variations of their names. Therefore,
searching for only an exact match of the passenger's name may not
result in identifying all watch-listed individuals. There have been
incidents, according to TSA's Office of Intelligence, of air carriers
failing to identify potential matches by not successfully conducting
similar-name matching. Before revisions to the security directives were
initiated in 2008, TSA expected air carriers to find similar names but
provided no specificity on the extent to which air carriers should make
these comparisons. The 14 air carriers we interviewed reported
implementing varied approaches to similar-name matching. Because air
carriers used different approaches, a passenger could be identified as
a match to a watch-list record by one carrier and not by another
carrier, which results in uneven effectiveness of watch-list matching.
Generally, TSA had been aware that air carriers were not using
equivalent processes to compare passenger names with names on the No
Fly and Selectee lists. However, in early 2008 the significance of such
differences was crystallized during the course of our review and
following TSA's special emphasis inspection of air carriers' name-
matching capability. On the basis of these inspection results, TSA
issued a revised security directive governing the use of the No Fly
List in April 2008 to establish a baseline capability for similar-name
matching to which all air carriers must conform. Also, TSA announced
that it plans to revise the Selectee List security directive to
similarly require the new baseline capability.[Footnote 19] According
to TSA officials, the new baseline capability is intended to improve
the effectiveness of watch-list matching, particularly for those air
carriers that did not compare the types of name variations specified by
the new baseline capability or that compared none at all. However, TSA
officials noted that the new baseline is not intended to address all
possible types of name variations and the related security
vulnerabilities. Agency officials explained that based on their
analysis of the No Fly and Selectee lists and interviews with
intelligence community officials, the newly established baseline covers
the most critical types of name variations. TSA officials further
stated that this is an interim solution that will strengthen security
while not requiring air carriers to invest in significant modifications
to their watch-list-matching processes, given TSA's expected
implementation of Secure Flight beginning in 2009. These officials
added that when implemented, Secure Flight will be better able to use
passenger names and other identifying information to more accurately
match passengers to the subjects of watch-list records.
TSA has undertaken various efforts to assess domestic air carriers'
compliance with watch-list-matching requirements in the No Fly and
Selectee list security directives; however, until 2008, TSA had
conducted limited testing of air carriers' similar-name-matching
capability. In 2005, for instance, TSA conducted a special emphasis
assessment that focused on air carriers' capability to prescreen
passengers for exact-name matches with the No Fly List, but did not
address the air carriers' capability to conduct similar-name
comparisons. Regarding inspections conducted as part of regular
inspection cycles, TSA's guidance establishes that regulatory
requirements encompassing critical layers of security need intensive
oversight, and that testing is the preferred method for validating
compliance. However, before being revised in 2008, TSA's inspection
guidelines (called PARIS prompts)[Footnote 20] for watch-list-related
inspections were broadly stated and did not specifically direct
inspectors to test air carriers' similar-name-matching capability.
Moreover, TSA's guidance provided no baseline criteria or standards
regarding the number or types of such variations that must be assessed.
In response to our inquiry, 6 of TSA's 9 principal security inspectors
told us that their assessments during annual inspection cycles have not
included examining air carriers' capability to conduct certain basic
types of similar-name comparisons. Also, in reviewing documentation of
the results of the most recent inspection cycle (fiscal year 2007), we
found that available records in TSA's database made references to name-
matching tests in 6 of the 36 watch-list-related inspections that
principal security inspectors conducted, and in 55 of the 1,109
inspections that transportation security inspectors conducted.[Footnote
21] Without baseline criteria or standards for air carriers to follow
in conducting similar-name comparisons, TSA has not had a uniform basis
for assessing compliance. Further, without routinely and uniformly
testing how effectively air carriers are conducting similar-name
matching, TSA may not have had an accurate understanding of the quality
of air carriers' watch-list-matching processes. However, TSA began
taking corrective actions during the course of our review and after it
found deficiencies in the capability of air carriers to conduct similar-
name matching during a January 2008 special emphasis inspection.
[Footnote 22] More specifically, following the January 2008 inspection,
TSA officials reported that TSA immediately began working with
individual air carriers to address deficiencies. Also, officials
reported that, following the issuance of TSA's revised No Fly List
security directive in April 2008, the agency had plans to assess air
carriers' progress in meeting the baseline capability specified in the
new security directive after 30 days, and that the annual inspection
plan for transportation security inspectors would be revised to help
ensure compliance by air carriers with requirements in the new security
directive. In September 2008, TSA provided us with results from a May
2008 special emphasis assessment of seven air carriers' compliance with
the revised No Fly List security directive. Although the details of
this special emphasis assessment are classified, TSA generally
characterized the results as positive. Further, the TSA officials noted
that the agency's internal handbook-- which provides guidance to
transportation security inspectors on how to inspect air carriers'
performance of various requirements, including watch-list-matching
requirements--was being revised and was expected to be released later
this year. Thus, the TSA officials stated that the new inspection
guidance would be used in conjunction with the nationwide regulatory
activities plan for fiscal year 2009. While these actions and plans are
positive developments, it is too early to determine the extent to which
TSA will assess air carriers' compliance with watch-list-matching
requirements moving forward since these efforts are still underway.
We provided a draft of our restricted report to DHS and the Department
of Justice for review and comment. DHS had no comments. The Department
of Justice provided technical comments to the restricted version of
this report, which we incorporated where appropriate.
Background:
TSA uses a layered system of defense to secure civil aviation whereby
additional layers provide security when any one security measure may
fail. Watch-list matching is one such layer of defense. Air carriers
began checking passenger names against government-supplied terrorist
watch lists (compiled by the FBI and distributed by the Federal
Aviation Administration) in the early 1990s. After the attacks of
September 11, 2001, and the subsequent establishment of TSA during the
same year, primary responsibility for civil aviation security,
including overseeing the watch-list-matching process, fell to TSA.
[Footnote 23] The Aviation and Transportation Security Act, enacted in
November 2001, requires that a system be used to evaluate all
passengers before they board an aircraft and ensure that selected
individuals and their carry-on and checked baggage are adequately
screened.[Footnote 24] TSA fulfilled this mandate by continuing to
require and oversee air carrier operation of the Computer Assisted
Passenger Prescreening System (CAPPS)--an electronic application that
selects individuals for enhanced screening at the passenger checkpoint
based on certain travel characteristics identified by TSA as indicating
potential risk--and by issuing security directives in April 2002 that
continued and amended the requirements that domestic air carriers match
passenger information against the No Fly and Selectee lists. These
security directives are the No Fly List Procedures security directive,
requiring domestic air carriers to conduct checks of passenger
information against the No Fly List to identify individuals who should
be precluded from boarding flights, and the Selectee List Procedures
security directive, directing domestic air carriers to conduct checks
of passenger information against the Selectee List to identify
individuals who should receive enhanced screening (e.g., additional
physical screening or a hand-search of carry-on baggage) before
proceeding through the security checkpoint.[Footnote 25] Since 2002,
TSA has issued numerous revisions to the No Fly and Selectee list
security directives to strengthen and clarify requirements, and has
issued guidance to assist air carriers in implementing their watch-
list-matching processes.[Footnote 26]
So that they may carry out watch-list-matching requirements, TSA
provides air carriers with access to the No Fly and Selectee lists--
subsets of the terrorist screening database managed by the FBI's
Terrorist Screening Center. The terrorist screening database is
composed of records that contain identifying information (e.g., name
and date of birth) on both foreign and U.S. citizens with known or
appropriately suspected links to terrorism. Only those nominations in
the terrorist screening database submitted by elements within the
intelligence community, including the FBI, that meet criteria specified
by the Homeland Security Council[Footnote 27] relating to the threat
that an individual poses to civil aviation are exported as records to
be included on the No Fly or Selectee lists.[Footnote 28] At present,
the Terrorist Screening Center forwards the No Fly and Selectee lists
to TSA's Office of Intelligence, which generally posts new lists daily
to a secure Web board that air carriers may access to retrieve the
lists.[Footnote 29] The Terrorist Screening Center provides TSA's
Office of Intelligence with new No Fly and Selectee lists on a daily
basis as well as any time a nominating entity submits additions and
deletions that require immediate notification to the aviation
community.
TSA's Regulatory Inspection Framework:
TSA is responsible for ensuring air carriers' compliance with
regulatory requirements, including requirements reflected in TSA
security directives and TSA-approved security programs. According to
TSA inspection guidance, compliance with regulatory requirements may be
validated in various ways, depending on the risk associated with the
requirements. For example, when regulatory requirements are largely
administrative and encompass the least critical layers of security,
compliance may be validated largely through inspections based on
documentation reviews. However, when regulatory requirements encompass
more critical layers of security, more intensive oversight is needed,
and compliance typically is to be validated through testing,
inspections, surveillance, special emphasis assessments, and special
emphasis inspections.
TSA conducts inspections of air carriers throughout the year as part of
regular inspection cycles based on annual inspection plans. These
inspections are based on inspection guidelines known as PARIS prompts,
which address a broad range of regulatory requirements (including
airport perimeter security and cargo security, as well as screening of
employees, baggage, and passengers). With respect to watch-list
matching, an inspection guideline (PARIS prompt) instructs inspectors
to determine, for example, whether the air carrier is comparing the
names of all passengers against names on the most current No Fly and
Selectee lists in accordance with the procedures outlined in TSA's
security directives.
TSA conducts watch-list-related inspections at air carriers' corporate
security offices (where policies and procedures are established on how
watch-list matching is to be performed) and at airports (where policies
and procedures for responding to a potential match are implemented).
TSA's principal security inspectors are responsible for conducting
inspections at domestic air carriers' corporate headquarters. These
inspectors assess air carriers' compliance with security requirements
and provide direct oversight of air carriers' implementation of and
compliance with TSA-approved security programs. TSA considers principal
security inspectors to be subject-matter experts for the air carrier
community concerning implementation of and compliance with security
programs and other requirements. As of January 2008, nine principal
security inspectors were responsible for assessing the compliance of
domestic air carriers with requirements in the No Fly and Selectee list
security directives (as well as with other regulatory requirements
pertaining to commercial aviation). Each of these inspectors has
responsibility for one or more domestic air carriers. For fiscal year
2007, there were 72 domestic air carriers to which the No Fly and
Selectee list security directives applied.
Field inspectors--known as transportation security inspectors--conduct
watch-list-related inspections at airports. They are responsible for a
multitude of TSA-related activities, including conducting inspections
and investigations of airports and air carriers, monitoring compliance
with applicable civil aviation security policies and regulations,
resolving routine situations that may be encountered in the assessment
of airport security, participating in testing of security systems in
connection with compliance inspections, identifying when enforcement
actions should be initiated, and providing input on the type of action
and level of penalty commensurate with the nature and severity of a
violation that is ultimately recommended to TSA's Office of Chief
Counsel. As of June 2008, there were 681 transportation security
inspectors responsible for 459 commercial airports across the United
States.
Secure Flight: Development of a Government-Run Watch-List-Matching
Process:
TSA began developing a program to take over watch-list-matching
capability from air carriers in March 2003.[Footnote 30] TSA cancelled
this earlier effort, known as CAPPS II, due to development challenges
and privacy concerns. In July 2004, the National Commission on
Terrorist Attacks Upon the United States (the 9/11 Commission)
recommended that the federal government take over the watch-list-
matching function from air carriers.[Footnote 31] Subsequently, the
Intelligence Reform and Terrorism Prevention Act of 2004 required that
TSA develop such a watch-list-matching capability.[Footnote 32] Shortly
after suspending work on the CAPPS II program in August 2004, TSA
initiated development of Secure Flight, a program that the agency
expects will allow the federal government to perform watch-list
matching for passengers on all flights within the United States and
ultimately for international flights with departures from or arrivals
in the United States.
In February 2006, we testified that although some progress had been
made in developing Secure Flight, long-standing issues related to
systems development and testing, program management, privacy
protections, and redress remained.[Footnote 33] We reported in
testimony that as a result of these deficiencies the program was at
risk of failure. Following our February 2006 testimony, TSA announced a
temporary suspension of Secure Flight's development to reassess program
goals and capabilities. TSA completed this reassessment in January
2007, moved forward to complete its concept-of-operations plan for the
Secure Flight program and strengthen systems development efforts, and,
in August 2007, issued a notice of proposed rulemaking describing the
requirements TSA will expect air carriers to implement to facilitate
the government-run prescreening process.[Footnote 34] TSA expects that,
beginning in early calendar year 2009, the Secure Flight program will
begin assuming from air carriers the watch-list-matching responsibility
for domestic flights. At some point following this assumption for
domestic flights, TSA plans to assume from U.S. Customs and Border
Protection this watch-list-matching function for international flights
that depart from or arrive in the United States. However, we testified
in February 2008 that despite significant progress in the development
of Secure Flight, TSA did not fully follow best practices for
developing Secure Flight's life-cycle cost and schedule estimates, and
that failure to do so put the program at risk of cost overruns, missed
deadlines, and performance shortfalls, among other issues.[Footnote 35]
TSA Took Action in 2008 to Enhance Watch-List Matching Conducted by Air
Carriers but Believes the Ultimate Solution Will Be Implementation of
Secure Flight:
Through its security directives, TSA has issued requirements for watch-
list matching, which include identifying passengers with names similar
to those on the No Fly and Selectee lists--a process TSA refers to as
similar-name matching. Before undertaking revisions of the relevant
security directives in 2008, TSA expected air carriers to conduct
similar-name matching but TSA's security directives did not specify how
many and what types of such name variations air carriers should
compare. Consequently, some of the 14 air carriers we interviewed
reported that they compared more name variations than others. Air
carriers that do not conduct similar-name comparisons and carriers that
conduct relatively limited comparisons are less effective in
identifying watch-listed individuals who travel under name variations.
Also, due to inconsistent air carrier processes, a passenger could be
identified as a match by one carrier and not by another. In April 2008,
during the course of our review, TSA revised and issued the No Fly List
security directive to specify a baseline capability for similar-name
matching to which all air carriers must conform. Also, in April 2008,
TSA officials reported that the agency had plans to similarly revise
the Selectee List security directive to require the same baseline
capability.[Footnote 36] TSA officials acknowledged that the new
baseline capability will not address all vulnerabilities identified by
TSA. However, the officials stated that the new baseline capability was
their best interim approach for improving air carriers' matching
efforts because, among other reasons, it will strengthen watch-list
matching without requiring considerable investment in a solution that
will be replaced when Secure Flight is implemented. TSA officials
further stated that the longer term solution for watch-list matching is
Secure Flight, which will have the capability to undertake more
advanced searches for individuals on the No Fly and Selectee lists.
Prior to April 2008, TSA Watch-List-Matching Requirements Were Broad
and Allowed Air Carriers to Implement Less Effective Processes:
Prior to a revision of the No Fly List security directive in April
2008--and a similar revision planned for the Selectee List security
directive--TSA's watch-list-matching requirements for domestic flights
(summarized in table 1) addressed five key processes: (1) retrieval of
the No Fly and Selectee lists, (2) the matching of passenger and list
information, (3) the use of TSA's Cleared List,[Footnote 37] (4)
notification procedures, and (5) record-keeping activities.[Footnote
38] In April 2008, TSA revised the No Fly List security directive for
watch-list matching and also reported plans for similarly revising the
Selectee List security directive. The security directive revisions--
discussed later in this section--still address the five key process
areas, but provide greater specificity on TSA's requirements for
matching passenger and watch-list information (the second key process
shown in table 1).[Footnote 39] Prior to the April 2008 revision of the
No Fly List security directive, TSA's requirements in this area lacked
specificity for purposes of implementation, although the then-current
security directives addressed the need for air carriers to identify
passengers with names that are either identical or similar to those on
the No Fly List or the Selectee List. To identify passengers with
similar names--an activity known as similar-name matching--air
carriers' automated programs or manual reviews were expected to capture
No Fly and Selectee list names that are variations of the name on the
passenger's reservation.
Table 1: TSA Watch-List-Matching Requirements Prior to the April 2008
Revision to the No Fly List Security Directive:
Requirements (key processes):
(1) Retrieving the No Fly and Selectee lists;
Discussion:
* Air carriers must monitor the TSA Web board throughout the day for
the most recent postings of the No Fly and Selectee lists.
Requirements (key processes):
(2) Matching passenger data to No Fly and Selectee lists;
Discussion:
* Within 24 hours of scheduled flight departure time, but no later than
passenger check-in, air carriers are to compare records from the most
recently issued No Fly and Selectee lists with identifying information
on passengers found in the respective air carrier's reservation system
and offered by passengers at the time of check-in;
* When comparing data, air carriers must identify name matches to the
No Fly and Selectee lists. To identify similar-name matches, automated
and manual processes are expected to have the capability to compare
name variations;
* To determine which passengers are matches, a passenger's name and one
piece of identifying information (found either within the air carrier's
reservation system or supplied by the passenger at check-in) must match
with corresponding information provided on the No Fly or Selectee
lists.
Requirements (key processes):
(3) Using the TSA Cleared List[A];
Discussion:
* When making determinations on matches, air carriers must use the TSA
Cleared List, which is composed of names and other personal-identifying
information on individuals whom the Department of Homeland Security has
reviewed and determined are not individuals on the No Fly or Selectee
lists. Individuals determined to be on the TSA Cleared List should be
accepted for travel and not be subject to further procedures for
handling matches to No Fly or Selectee lists identified in the security
directives.
Requirements (key processes):
(4) Notifying authorities; Discussion:
* Upon identifying a passenger whose information matches with the No
Fly or Selectee lists and who is not on the TSA Cleared List, air
carriers must follow certain notification procedures, such as to
contact the federal security director and the appropriate local law
enforcement officer (for matches to the No Fly List) or to designate
the passenger as a selectee for enhanced checkpoint screening
procedures (for matches to the Selectee List).
Requirements (key processes):
(5) Keeping records; Discussion:
* Air carriers must keep records on the results of watch-list matching
for specified time periods--for example, air carriers must keep a
record of all flights operated with passengers designated as selectees
for 7 calendar days from the date of the flight's departure.
Sources: GAO analysis of TSA's No Fly List Procedures security
directive (SD 1544-01-20 series) and Selectee List Procedures security
directive (SD 1544-01-21 series), versions dated July 8, 2004, and
March 8, 2007.
[A] Security directives in effect prior to the April 2008 revision of
the No Fly List Procedures security directive referenced a "cleared
column," a format for clearing passengers. TSA eventually replaced this
format with the Cleared List, and revised language for the April 2008
No Fly List security directive.
[End of table]
Air carriers must conduct similar-name matching because watch-listed
individuals may travel using variations of the names attributed to them
on the No Fly or Selectee lists and, thus, would not be identified if
air carriers searched only for an exact-name match. At present, TSA
does not require that air carriers collect the full name from
passengers making travel reservations, thus, passengers may travel
using variations of their legally documented names; for example,
abbreviated name forms or portions of their names. Such name variations
may arise due to unintentional errors--for example, a travel agent
mistakenly books travel for "Jon" when the name spelling is actually
"John," or the agent accidentally transposes a passenger's first and
middle names for a flight reservation. Traveling under a name variation
could also represent a watch-listed individual's intentional effort to
evade detection. For example, an individual identified as John Robert
Smith on his driver's license may make a travel reservation using a
common name variation--such as using his middle and last names (Robert
Smith) or his initials and last name (J.R. Smith). If the John Robert
Smith in this example were a name on the No Fly List, an exact, letter-
for-letter comparison of the passenger's reservation name (either
Robert Smith or J.R. Smith) with the No Fly List would fail to identify
the watch-listed individual. However, a comparison of possible
variations of the watch-list name (John Robert Smith) could identify
either Robert Smith or J.R. Smith as a potential match--that is, an
individual who is a possible match to the No Fly List or Selectee List
and whose personal identifying information requires further review
before a match can be determined.
Before 2008, TSA's Security Directives Allowed Air Carriers More
Discretion in Comparing Name Variations:
Regarding similar-name matching, before 2008, TSA's security directives
had broad requirements that allowed air carriers discretion in
determining the extent to which they compared name variations. For
instance, to identify watch-listed individuals who travel using
variations of their name, TSA's security directives did not specify how
many possible combinations of name elements should be compared. TSA
officials explained that the agency initially issued broad security
directives to allow air carriers flexibility in implementing
requirements and--until the April 2008 revision of the No Fly List
security directive--left the directives relatively unchanged because
the agency was developing a government-run capability to take over this
function. The operations of those air carriers that are subject to the
watch-list-matching requirements of TSA's security directives range
from commuter providers to international-service providers. According
to TSA officials, broad security directive requirements permit air
carriers with such diverse operations to implement processes that best
meet their operational needs and technological capabilities.
Officials further explained that TSA's focus has been on developing its
own watch-list-matching capability (now Secure Flight) since 2003. TSA
officials noted that, though not an impetus for making requirements
broad when first articulated in 2002, this focus on developing a
government-run watch-list-matching program is one reason why these
requirements remained relatively unchanged until April 2008.
Failure to Conduct Similar-Name Matching or Comparing Name Variations
to a Lesser Extent Reduces the Effectiveness of Watch-List Matching:
The 14 air carriers we interviewed reported adopting different
approaches to name matching. Although each of the 14 air carriers we
spoke with during our review reported conducting comparisons to
identify exact-name matches of passengers and names on the No Fly List
or the Selectee List, not every air carrier reported conducting similar-
name comparisons.[Footnote 40] Those air carriers that conducted
similar-name comparisons reported using various approaches, some of
which compared more name variations than others.
According to air carriers, a critical factor affecting their
implementation of similar-name-matching requirements was their
observation that conducting more comparisons for variations results in
longer lines at ticket counters and passenger inconvenience.
Specifically, 10 air carriers commented that conducting similar-name
comparisons resulted in more passengers being identified as potential
matches. At the time of check-in, air carriers must perform additional
checks at the ticket counter of each potentially matched passenger's
government-issued identification against data on the No Fly and
Selectee lists. Therefore, according to 12 of the 14 air carriers we
spoke with, a large number of potential matches can lead to congestion
at the ticket counter and longer wait times for all passengers.
Inconsistent approaches to conducting similar-name matching could lead
a passenger to be identified as a match by one air carrier and not by
another. Further, not conducting similar-name matching--or conducting
such matching to only a very limited extent--compromises the usefulness
of the No Fly List and Selectee List. There have been incidents,
according to TSA's Office of Intelligence, of air carriers failing to
identify potential matches by not effectively conducting similar-name
matching. In these incidents, the air carriers' processes led to false
negative watch-list-matching results--that is, individuals who were on
the No Fly List and were not identified by the respective air carrier's
watch-list-matching process. In some of these incidents, the
individual's flight reservation contained a name that varied somewhat
from the name on the No Fly List, and the air carrier's watch-list-
matching process did not identify the name as a possible match.
In most of these cases, the failures of the air carriers to identify
the potential matches were discovered as a result of the U.S. Customs
and Border Protection's comparison of passenger and watch-list data for
international flights. Specifically, TSA learned of the failures
through U.S. Customs and Border Protection, which identified the No Fly
listed individual when conducting its own comparison of passenger
information against the No Fly and Selectee lists for international
flights.[Footnote 41] These comparisons, performed as part of U.S.
Customs and Border Protection's border security mission, took place
after the air carriers completed their comparisons, in effect
constituting a second check of passenger and watch-list information.
U.S. Customs and Border Protection does not screen passengers on
domestic flights; thus, there is no opportunity for a second comparison
of passenger information against the No Fly and Selectee lists for
domestic flights. Therefore, it is difficult to determine the extent to
which domestic air carriers may be failing to identify watch-listed
individuals who are able to board domestic flights.
In October 2007, we reported that of the known cases in which
individuals on the No Fly List flew on international flights bound to
or from the United States, some were allowed to fly because the
respective air carrier's process failed to identify the passenger's
name as a match.[Footnote 42] Although these individuals were
subsequently identified in-flight by other means, the onboard security
threats required an immediate counterterrorism response, which in some
instances resulted in diverting the aircraft to a location other than
its original destination.[Footnote 43] According to TSA's Office of
Intelligence, some of these incidents may be attributed to air
carriers' inability to identify similar-name matches when passengers
travel using variations of their name.
TSA had been aware that air carriers were not using equivalent
processes to compare passenger names with names on the No Fly and
Selectee lists. For instance, in June 2006, we reported that the
improvements air carriers were making to their individual watch-list-
matching processes, though beneficial to the respective air carrier's
operations, could further exacerbate differences that currently exist
among the various air carriers and could result in varying levels of
effectiveness across air carriers in matching passenger information to
the No Fly and Selectee lists.[Footnote 44] Furthermore, TSA's March
2007 Secure Flight Program Baseline explained "because each aircraft
operator conducts its own matching process, the ability to conduct
watch-list matching and coordinate law enforcement responses is not
consistent across the aviation industry."[Footnote 45] Moreover, in
several interviews over the course of our work, TSA officials
acknowledged that in general, some air carriers were performing more
similar-name comparisons than other air carriers. TSA's understanding
of the significance of these differences was crystallized in January
2008, when results of a special emphasis inspection identified
deficiencies in air carriers' similar-name-matching capability.
To Address Deficiencies in Air Carriers' Similar-Name-Matching
Capability, TSA Issued a Revised No Fly List Security Directive in
April 2008 to Provide More Specific Requirements:
During the course of our work and in response to findings of the
January 2008 special emphasis inspection that identified deficiencies
in air carriers' similar-name-matching capability, TSA officials
reported that the agency immediately began to assess options for
corrective actions to implement across the aviation industry. In doing
so, officials noted that they consulted with representatives from the
intelligence community, the Secure Flight program, and the aviation
industry. On the basis of its assessment, TSA revised the No Fly List
security directive in April 2008 to establish a specific baseline
capability for air carriers in conducting similar-name matching. Also,
in April 2008, TSA officials reported that the agency had plans to
similarly revise the Selectee List security directive to require the
same baseline capability.[Footnote 46]
TSA officials acknowledged that the new baseline capability will not
address all vulnerabilities identified by TSA. However, TSA officials
explained that they expect the new similar-name matching baseline
capability to strengthen the watch-list matching currently performed by
air carriers. In particular, the officials expect the newly established
baseline capability to improve the matching processes of those air
carriers that do not compare the kinds of variations required by the
new baseline or that compare none at all. Furthermore, according to
agency officials, the variations specified by the new baseline address
the types of situations air carriers will encounter due to passengers
making their own reservations. Accordingly, TSA concluded that
requiring air carriers to conduct similar-name comparisons beyond the
baseline capability specified in the revised No Fly List security
directive was not warranted for the interim period pending the
implementation of Secure Flight. TSA was not able to provide us with
data or analysis to support this assertion, and we did not undertake an
independent analysis to determine the sufficiency of the newly
established baseline.
TSA officials also explained they determined that revising the security
directives to be the most feasible approach for strengthening the
current watch-list-matching process over other options because it was
expedient and would have the least negative impact on air carriers'
operations. Specifically, TSA officials determined that upon issuing
the revised No Fly List security directive, air carriers would need
only 2 to 4 weeks to implement new requirements. When considering how
this option would affect air carrier operations, TSA officials
explained that they considered the number of potential matches that
likely would be generated by the new baseline capability. As previously
discussed, air carriers reported that comparing more name variations
results in more passengers being identified as potential matches, who
then must go to the ticket counter to obtain their boarding passes.
Thus, large numbers of potential matches could overwhelm air carriers'
check-in operations. TSA officials explained that the industry
officials with whom they consulted in developing the new baseline
capability believed it would produce a manageable number of potential
matches.
In exploring actions to strengthen the watch-list-matching process, TSA
considered two other options--one that would have required each air
carrier to contract with third-party providers to develop customized
watch-list-matching software, and another that involved the creation of
an expanded version of the No Fly and Selectee lists to include name
variations so that air carriers need only conduct comparisons to
identify an identical match. TSA identified significant obstacles to
implementing these options. Specifically, TSA determined that
contracting with third-party vendors was impracticable due to
availability and timing concerns. For instance, identifying appropriate
vendors and implementing vendor-provided solutions could take almost 2
years--an unrealistic time frame given that Secure Flight's
implementation is scheduled to begin in 2009. In this regard, TSA
officials also expressed reluctance to requiring air carriers to
undertake the expense of contracting with third-party vendors for an
interim approach, while at the same time requiring that air carriers
invest in system changes for Secure Flight. With regard to the option
of adding name variations to the No Fly and Selectee lists, according
to TSA officials, creating these variations would have greatly expanded
the total size of the No Fly List, which could overwhelm the name-
matching capability of some air carriers and could potentially send an
unmanageable number of potential matches to the ticket counters of air
carriers. As previously discussed, in our air carrier interviews, 10 of
the 14 air carriers reported that searching for more name variations
leads to the identification of more potential matches. In this regard,
there is some support for TSA's determination that expansion of the No
Fly and Selectee lists could produce an unmanageable number of
potential matches. However, we did not independently assess this issue.
Although TSA officials characterized the new baseline capability as a
good interim solution for strengthening watch-list matching--one that
balances TSA's need to strengthen watch-list matching with the air
carriers' need for efficient operations--they stressed that the Secure
Flight program is ultimately the solution. For example, in its
development of Secure Flight, TSA plans to develop a name-matching
process that will have the capability to identify name variations
beyond those specified by the new baseline. Further, according to TSA,
Secure Flight will be better able to use passenger names and other
identifying information (such as date of birth and gender) to more
accurately match passengers to the subjects of watch-list records and,
thereby, further reduce the risks of false negatives without
unacceptably increasing the number of false positives (mistakenly
identifying a passenger's name as a potential match with watch-list
records).
Until a 2008 Special Emphasis Inspection, TSA Had Conducted Limited
Testing of Air Carriers' Capability to Perform Similar-Name Matching:
Although TSA assessed air carriers' compliance with watch-list-matching
requirements through a special emphasis assessment conducted in 2005
and through planned inspections conducted in conjunction with annual
inspection cycles, the agency had tested similar-name matching to a
limited extent until 2008. For instance, the 2005 special emphasis
assessment focused on air carriers' capability to identify passenger
names that were exact matches with names on the No Fly List, but did
not address the capability to conduct similar-name matching. Also,
during the most recent annual inspection cycle (fiscal year 2007),
although some TSA inspectors tested air carriers' effectiveness in
conducting similar-name matching, the inspectors did so at their own
discretion and without specific evaluation criteria. However, during a
special emphasis inspection conducted in January 2008, TSA found
deficiencies in the capability of air carriers to conduct similar-name
matching.[Footnote 47] Thereafter, following TSA's revision of the No
Fly List security directive in April 2008, officials planned to issue
new guidance for inspectors to better ensure compliance by air carriers
with requirements in the new security directive (e.g., by providing
uniform evaluation criteria consistent with the new requirements). In
response to our request for updated information on its oversight
efforts, TSA provided us the results of a special emphasis assessment
(conducted in May 2008) of seven air carriers' compliance with the
revised No Fly List security directive. Although the details of this
special emphasis assessment are classified, TSA officials generally
characterized the results as positive. Further, TSA's noted that the
agency's internal handbook--which provides guidance to transportation
security inspectors on how to inspect air carriers' performance of
various requirements, including watch-list-matching requirements--was
being revised and was expected to be released later this year. Thus,
TSA indicated that the new inspection guidance would be used in
conjunction with the nationwide regulatory activities plan for fiscal
year 2009. While these actions and plans are positive developments, it
is too soon to determine the extent to which air carriers' compliance
with watch-list-matching requirements will be assessed based on the new
security directives since these efforts are still underway.
TSA's Special Emphasis Assessment in 2005 Focused on Air Carriers'
Exact-Name-Matching Capability:
TSA conducted a special emphasis assessment in 2005 that tested the
capability of domestic air carriers to find passenger names that were
exact matches to names on the No Fly List. The 2005 special emphasis
assessment was undertaken at the request of the TSA Administrator due
to serious failures in air carriers' watch-list-matching processes,
according to a senior TSA official. To conduct the assessment, TSA
inspectors made flight reservations using the exact name of an
individual who was on the No Fly List and not on the TSA Cleared List.
If the air carrier identified the name on the reservation as a
potential match to the individual on the No Fly List--and the check-in
agent identified through the reservation system that further assistance
was needed to finish the check-in process (e.g., to call security)--the
test was considered to be successfully completed. According to TSA
data:
* air carriers passed a large majority of the initial tests conducted
in June and July 2005, although several air carriers failed one or more
tests and:
* those air carriers that failed a test were retested in September
2005, and a large majority of these air carriers passed the tests.
[Footnote 48]
Although TSA conducted a large number of tests, TSA officials stated--
and our own analyses confirmed--that results from this special emphasis
assessment would not produce a reliable estimate of the success rate of
all attempted matches by air carriers because TSA did not randomly
select the air carriers, airports, or individual flights for review. As
a result, the findings from this assessment cannot be used to infer
overall or individual rates of success in identifying exact name
matches in accordance with the No Fly and Selectee list security
directives. That is, although the 2005 special emphasis assessment
provided insight into air carriers' effectiveness in conducting a basic
form of name matching, the picture provided was incomplete. Moreover,
the air carriers' failure rates may have been considerably higher had
the special emphasis assessment tested similar-name-matching
capability, given that this capability involves more than finding a
name that is a letter-for-letter match to another name. However, TSA
officials told us that at the time of the special emphasis assessment
in 2005, exact-name matching was the agency's focus.
TSA Conducted Planned Watch-List-Related Inspections throughout the
Year, but Inspectors Tested Air Carriers' Effectiveness at Similar-Name
Matching at Their Own Discretion and without Baseline Evaluation
Criteria:
Since issuing the No Fly and Selectee list security directives in 2002,
TSA has incorporated watch-list-related inspections into its regular
inspection cycle, but inspectors tested air carriers' effectiveness in
similar-name matching during these planned inspections to a limited
extent and without specific evaluation criteria. In the most recent
annual inspection cycle (fiscal year 2007), TSA conducted 1,145
inspections of air carriers' compliance with watch-list-related
requirements in the No Fly and Selectee security directives; 1,109 of
these inspections were conducted at air carriers' airport locations by
transportation security inspectors and 36 at air carriers' corporate
security offices by principal security inspectors.[Footnote 49] The
1,145 inspections covered 60 of the 72 domestic air carriers to which
the security directives applied during fiscal year 2007, and most of
the carriers were inspected multiple times that year.[Footnote 50] TSA
found air carriers in compliance with required procedures in 1,133 (99
percent) of the 1,145 inspections.[Footnote 51]
These inspections were based on one or more inspection guidelines
(called PARIS prompts) and were sometimes conducted in combination with
inspections related to other regulatory requirements, such as
performing criminal history record checks on employees or implementing
CAPPS procedures. Table 2 presents the inspection guidelines TSA used
to assess a key security directive requirement that we reviewed--
matching passenger names to the No Fly and Selectee lists.[Footnote 52]
Additional guidelines used to assess other requirements in our review
are presented in appendix I.[Footnote 53]
Table 2: Requirements for Matching Passenger Data to No Fly and
Selectee Lists and Inspection Guidelines Used to Assess Compliance with
the Requirements:
Requirements for matching passenger data to No Fly and Selectee lists:
* Within 24 hours of scheduled flight departure time, air carriers are
to compare records from the most recently issued No Fly and Selectee
lists with identifying information on passengers found in the
respective air carrier's reservation system and offered by passengers
at the time of check-in;
* When comparing data, air carriers must identify name matches
(including similar-name matches) to the No Fly and Selectee lists;
* To determine which passengers are matches, a passenger's name and one
piece of identifying information (found either within the air carriers'
reservation system or supplied by the passenger at check-in) must match
with corresponding information provided on the No Fly or Selectee
lists;
Inspection guidelines: Transportation security inspectors:
* All passenger names are compared to the most current No Fly and
Selectee lists;
* The aircraft operator is comparing all passenger names to the most
current No Fly and Selectee lists in accordance with the procedures
outlined in Security Directive 1544-01-20 series (No Fly) and Security
Directive 1544-01-21 series (Selectee);
Inspection guidelines: Principal security inspectors:
* Procedures are in place to ensure the most recently issued No Fly
List is utilized within 24 hours of receipt;
* Procedures are in place to ensure the most recently issued Selectee
List is utilized within 24 hours of receipt;
* Procedures are in place to contact the Federal Security Director,
local law enforcement, the FBI, and TSA Office of Intelligence for
matches to the No Fly List;
* Records are maintained of all flights operated with passengers who
were determined by local law enforcement, U.S. legal attaché, or TSA
Office of Intelligence not to be a match.
Sources: GAO analysis of TSA's No Fly List Procedures security
directive (SD 1544-01-20 series) and Selectee List Procedures security
directive (SD 1544-01-21 series), versions dated July 8, 2004, and
March 8, 2007, and inspection guidelines applicable during fiscal year
2007.
[End of table]
The inspections conducted by transportation security inspectors at
airports used the guidelines in table 2 to assess air carriers'
compliance in matching passenger data to the No Fly and Selectee lists
in fiscal year 2007. However, these inspectors tested exact-name and
similar-name matching during these inspections at their own discretion;
moreover, an official in TSA's Office of Security Operations,
Compliance Division, stated that, generally, transportation security
inspectors test exact-name-matching capability only. This inspection
guideline is broadly written and does not specify the methods for
validating compliance with the requirement to perform name comparisons.
According to a TSA official in the Office of Security Operations, field
inspectors may validate compliance by asking check-in agents to
demonstrate that they have access to the current No Fly and Selectee
lists and that any hard copies of the lists are properly protected;
they may also interview check-in agents to ensure that they understand
the security directive requirements, observe them as they process
passengers who have been identified as Selectee or No Fly individuals,
and/or test the air carriers' system by requesting a gate pass in the
name of an individual on the watch list. We found evidence of field
inspectors testing air carriers' name matching systems in 55 of the
1,109 inspections they conducted in fiscal year 2007 (such tests may
have been administered during the other inspections conducted in fiscal
year 2007 but were not documented).
For the 36 inspections conducted by principal security inspectors at
air carriers' corporate security offices, we found 6 inspection records
that referred to tests of exact-name and similar-name matching
capability (they may have administered such tests during the other
inspections they conducted that year but did not document the tests).
Principal security inspectors did not have an inspection guideline
directing them to assess exact-name and similar-name matching
capability specifically--thus they tested this capability at their own
initiative, and then reported their methods and results in conjunction
with one of the four guidelines presented in table 2. Further, in
response to our inquiry, 6 of TSA's 9 principal security inspectors
told us that their assessments have not included examining air
carriers' capability to conduct certain basic types of similar-name
comparisons.
TSA establishes in guidance for inspections (including watch-list-
related inspections) that testing is the preferred method for assessing
air carriers' compliance with regulations whenever possible and that it
is only through testing that security can be assured.[Footnote 54] TSA
further establishes in inspection guidance that when regulatory
requirements encompass critical layers of security, more intensive
oversight is needed, and compliance typically is to be validated
through testing, inspections, surveillance, special emphasis
assessments, and special emphasis inspections.[Footnote 55] Without
routinely testing air carriers' compliance with the similar-name-
matching requirement, TSA may not have reliable data on the
effectiveness of air carriers' watch-list-matching processes and could
be hindered in taking timely action to address any deficiencies.
Inspectors who have tested air carriers' effectiveness in performing
similar-name matching have done so without specific evaluation
criteria. As discussed earlier, for any given name there are a number
of possible name variations that could be used for travel, but TSA
inspectors did not have baseline criteria on the number or types of
such variations that must be evaluated. In the absence of specific
standards for similar-name matching that all air carriers must follow,
TSA has had no assurance that its inspections are based on uniform
evaluation criteria. The inspections may not have been conducted
uniformly and may have produced inconsistent results, given the absence
of specific standards. In fall 2007, TSA began to review the adequacy
of inspection guidance used by principal security inspectors, including
guidance for watch-list-related inspections. As discussed in the
following section, TSA expects to provide baseline criteria on the
number and types of such variations inspectors must evaluate, but had
not completed these efforts as of early September 2008.
A Special Emphasis Inspection Conducted in 2008 Found Deficiencies in
Air Carriers' Similar-Name-Matching Capabilities, and TSA Has Plans for
Corrective Actions:
During the course of our review and following TSA's discovery of a
major air carrier's inability to effectively conduct both exact-name
and similar-name-matching against the No Fly List, TSA initiated a 3-
day, special emphasis inspection in January 2008 that tested the
capability of 83 air carriers to conduct watch-list matching.[Footnote
56] According to TSA officials, this inspection covered 52 domestic air
carriers and 31 foreign air carriers. To implement the special emphasis
inspection, TSA used 100 names on the No Fly List to test the 83 air
carriers' capability to identify both exact-name and similar-name
matches based on various types of possible name variations. On the
basis of test results, a senior TSA official stated that the agency has
confidence in air carriers' capability to identify exact-name matches.
Regarding the capability to identify similar-name matches, TSA found
that no air carrier was successful in identifying matches involving all
types of name variations, although some carriers were more effective
than others.
On the basis of this inspection, TSA officials stated that they began
to strengthen oversight of air carriers' similar-name-matching
capability. For example, the TSA officials explained that--after a 30-
day period following issuance of the revised No Fly List security
directive in April 2008--the agency's inspectors would begin to
evaluate air carriers' performance in complying with the new
requirements. TSA officials explained that these initial inspections
would be conducted at air carriers' corporate security offices and at
airports. Officials further stated that after these initial
inspections, others would be conducted periodically and, if applicable,
TSA would impose progressively stronger enforcement actions against air
carriers that are not successful in meeting the new standards.
In September 2008, in response to our request for updated information
on the status of its oversight efforts, TSA provided us the results of
a special emphasis assessment (conducted during May 20-29, 2008) of
seven air carriers' compliance with new requirements in the No Fly List
security directive. Although the details of this special emphasis
assessment are classified, TSA generally characterized the results as
positive. Also, TSA plans to work with individual air carriers, as
applicable, to analyze specific failures, improve system performance,
and conduct follow-up testing as needed.
In further reference to revision of the No Fly List security directive
in April 2008, TSA officials stated that the agency's internal guidance
is being updated to align inspection guidance with the revised
directive. The officials elaborated that the new inspection guidance
will place more emphasis on testing the effectiveness of security
measures rather than using a checklist approach to determine whether an
air carrier has a particular procedure in place. Regarding the emphasis
on testing, our review noted that the draft guidance being developed
for principal security inspectors included testing scenarios based on
the types of name variations that air carriers must be capable of
conducting in accordance with the revised watch-list-matching
requirements. Also, according to TSA, guidance for transportation
security inspectors is being developed (as part of the 2009 Regulatory
Activities Plan) to provide more specific direction to inspectors for
assessing name-matching capability. In September 2008, in response to
our inquiry, TSA noted that the agency's internal handbook--which
provides guidance to transportation security inspectors on how to
inspect air carriers' performance of various requirements, including
watch-list-matching requirements--was being revised and was expected to
be released later this year. Thus, TSA indicated that the new
inspection guidance would be used in conjunction with the nationwide
regulatory activities plan for fiscal year 2009. Overall, the actions
taken (and planned to be taken) by TSA are positive developments,
although it is too soon to determine the extent to which TSA will
assess air carriers' compliance with the revised watch-list-matching
requirements.
According to TSA officials, there were other benefits stemming from the
January 2008 special emphasis inspection. For example, officials stated
that in considering options for corrective actions, TSA consulted with
representatives from the intelligence community, which is responsible
for identifying names (and variations of names)[Footnote 57] for
inclusion on the No Fly and Selectee lists. According to TSA, these
discussions enhanced the intelligence community's understanding of how
air carriers use the No Fly and Selectee lists, and as a result, the
intelligence community is better positioned to carefully consider which
name variations are appropriate for being added to the lists and
whether these variations would be helpful for the purposes of watch-
list matching. Further, TSA officials noted that such considerations,
in turn, could benefit air carriers and the public by limiting the
number of passengers who are misidentified as being potential matches
with watch-list records. TSA officials added that insights regarding
the extent to which name variations exist on the No Fly and Selectee
lists also have benefited ongoing efforts to design and implement the
Secure Flight program. Specifically, officials explained that TSA now
has a fuller understanding of the types of name variations presently
contained in watch-list records and, in turn, a fuller understanding of
what types of comparisons Secure Flight should be capable of
performing.
Concluding Observations:
Shortcomings that have national security implications exist in the
watch-list-matching capability of domestic air carriers, as confirmed
by the results of TSA's recent special emphasis inspection.
Specifically, TSA found differences among air carriers in the
thoroughness and effectiveness of their processes for comparing
passengers' names with those on the No Fly List. A particular concern
involves similar-name comparisons. However, TSA's April 2008 revision
of the No Fly List security directive establishes a baseline name-
matching capability by specifying the types of name variations that air
carriers' processes must be capable of identifying. Effective
implementation of the baseline capability should strengthen watch-list-
matching processes, especially for those air carriers that had been
using less thorough approaches for identifying similar-name matches.
Concurrently, revised internal guidance for TSA's inspectors can help
ensure that compliance decisions are based upon testing and that these
tests are carried out regularly, using the standards specified within
the security directives as evaluation criteria. Also, if properly
documented in inspection reports, the results of these tests could give
TSA management better information on the quality of watch-list matching
being conducted by air carriers, thereby improving TSA's monitoring of
the overall security posture of the aviation sector. At the time of our
review, TSA's process for revising its guidance was in the initial
stages; thus it is too early to determine the extent to which updated
guidance for principal security inspectors and transportation security
inspectors would strengthen oversight of air carriers' compliance with
the security directive requirements. Given continued delays in the
implementation of the Secure Flight program, TSA's oversight of air
carriers' compliance with watch-list-matching requirements remains an
important responsibility. TSA officials acknowledge that the baseline
capability specified in the revised No Fly List security directive and
the similar revision planned for the Selectee List security directive-
-while an improvement--does not address all vulnerabilities identified
by TSA and does not provide the level of risk mitigation that is
expected to be achieved from Secure Flight. Thus, TSA intends to deploy
the Secure Flight program beginning in January 2009 so that it may
implement this more robust matching capability.
Agency Comments:
We provided a draft of our restricted report (GAO-08-453SU) to the
Department of Homeland Security and the Department of Justice for
review and comment. The Department of Homeland Security had no
comments. The Department of Justice provided technical comments on the
restricted version of this report, which we incorporated where
appropriate.
We will send copies of this report to the appropriate congressional
committees; the Secretary of Homeland Security; and the U.S. Attorney
General. We will make copies available to others upon request. The
report will also be available at no charge on our Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report or wish to
discuss the matter further, please contact me at (202) 512-3404 or
berrickc@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix III.
Signed by:
Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
List of Congressional Committees:
The Honorable Robert C. Byrd:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Committee on Appropriations:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Kay Bailey Hutchison:
Ranking Member:
Committee on Commerce, Science, and Transportation:
United States Senate:
The Honorable Joseph I. Lieberman:
Chairman:
The Honorable Susan M. Collins:
Ranking Member:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Patrick J. Leahy:
Chairman:
The Honorable Arlen Specter:
Ranking Member:
Committee on the Judiciary:
United States Senate:
The Honorable John D. Rockefeller, IV:
Chairman:
The Honorable Kay Bailey Hutchison:
Ranking Member:
Subcommittee on Aviation Operations, Safety, and Security:
Committee on Commerce, Science, and Transportation:
United States Senate:
The Honorable Dave Obey:
Chairman:
The Honorable Jerry Lewis:
Ranking Member:
Committee on Appropriations:
House of Representatives:
The Honorable Bennie G. Thompson:
Chairman:
The Honorable Peter T. King:
Ranking Member:
Committee on Homeland Security:
House of Representatives:
The Honorable Henry A. Waxman:
Chairman:
The Honorable Tom Davis:
Ranking Member:
Committee on Oversight and Government Reform:
House of Representatives:
The Honorable James L. Oberstar:
Chairman:
The Honorable John L. Mica:
Ranking Republican Member:
Committee on Transportation and Infrastructure:
House of Representatives:
The Honorable David E. Price:
Chairman:
The Honorable Harold Rogers:
Ranking Member:
Subcommittee on Homeland Security:
Committee on Appropriations:
House of Representatives:
The Honorable Judd Gregg:
United States Senate:
The Honorable Don Young:
House of Representatives:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
Objectives:
To examine the current processes that domestic air carriers use to
conduct watch-list matching for domestic flights, we addressed the
following questions: (1) What are TSA's requirements for domestic air
carriers to conduct watch-list matching for domestic flights? (2) To
what extent has TSA assessed domestic air carriers' compliance with
watch-list-matching requirements?
Scope and Methodology:
In addressing the principal questions, we drew upon our previous work
and reports on aviation security--specifically, reports covering TSA's
inspection process, Secure Flight, and other passenger prescreening
programs. We also consulted our most recent reports and testimonies on
terrorist watch lists. In addition, we reviewed relevant studies
conducted by other governmental agencies, including the Congressional
Research Service and the Department of Justice's Office of Inspector
General. This report is a public version of the restricted report that
we provided to congressional committees in July 2008.[Footnote 58]
More details about the scope and methodology of our work to address
each of the principal questions are presented in the following
sections, respectively.
TSA's Requirements for Air Carriers to Conduct Watch-List Matching for
Domestic Flights:
To determine TSA's requirements for air carriers to match passenger
information against the No Fly List and the Selectee List for domestic
flights, we assessed two key TSA documents--the No Fly List Procedures
security directive and the Selectee List Procedures security directive.
[Footnote 59] We reviewed versions of these security directives--
including the revisions made in April 2008--to identify applicable
requirements for watch-list matching. For the purposes of this report,
we considered applicable requirements to be those that, according to
TSA, would be assumed by the Secure Flight program, once operational,
and those that TSA had itself identified for its oversight
activities.[Footnote 60] Thus, we identified the following requirements
(or key processes) as being within this scope (see table 1, which is
presented earlier in this report): (1) the retrieval of the No Fly and
Selectee lists, (2) the matching of passenger and watch-list
information, (3) the use of the TSA Cleared List, (4) procedures for
notifying authorities, and (5) keeping appropriate records.[Footnote
61]
To further our understanding of these requirements, we reviewed TSA
policies and other guidance applicable to watch-list matching. We also
interviewed officials from TSA's Office of Security Operations, which
had primary responsibility for writing the security directives, and
officials from two TSA offices that collaborated with the Office of
Security Operations in crafting critical sections of the directives--
the Office of Transportation Sector Network Management and the Office
of Intelligence. To better understand TSA's rationale for similar-name-
matching requirements as well as the challenges associated with name-
based matching, we attended meetings of the interagency Federal
Identity Match Search Engine Performance Standards Working Group, which
was organized by the Terrorist Screening Center to help ensure
awareness of best practices with regard to identity matching among
federal agencies, and spoke with one of the group's experts working in
the field of name matching.[Footnote 62] To obtain information on the
composition and use of the No Fly and Selectee lists, we spoke with
officials from the Department of Justice's Terrorist Screening Center
and TSA's Office of Intelligence. Further, to understand how TSA
compiles and disseminates its Cleared List to air carriers, we spoke
with officials from the Department of Homeland Security's Traveler
Redress Inquiry Program (TRIP) and TSA's Office of Transportation
Security Redress, which share responsibility for managing the TSA
Cleared List for the current watch-list-matching process. Finally, to
compare the current watch-list-matching process with that proposed once
the federal government performs watch-list matching, we reviewed recent
Secure Flight program documents.[Footnote 63]
To generally understand how domestic air carriers have responded to
TSA's requirements, we selected for interviews a nonprobability sample
of 14 air carriers from a TSA-provided list of 95 air carriers that
were subject to the watch-list-matching security directives for fiscal
year 2005. To ensure that our sample of air carriers reflected a range
of operational sizes, we based our selections partly on data from the
U.S. Department of Transportation, which places air carriers in size
categories based on operating revenue. Specifically, we selected 8 that
were considered "major" air carriers, each having more than $1 billion
in operating revenue in 2005; all but one of these 8 major air carriers
flew internationally. In addition, we selected 3 air carriers the
Department of Transportation identified as "national" air carriers,
each having more $100 million to $1 billion in operating revenue in
2005, and 1 air carrier the department identified as a "regional" air
carrier, with $100 million or less in operating revenue. We also
selected two air carriers from the list that were not included in the
Department of Transportation's revenue groupings, given the small scale
of their operations, but were identified by the department as air
carriers that provide commuter service. National, regional, and
commuter air carriers--which generally provided service covering a
geographical area, such as the Pacific Northwest--had comparatively
smaller business operations.
In selecting the 14 air carriers, we also considered the number of
passengers transported. To determine this number, we used the
Department of Transportation's data for number of revenue passengers
who enplaned (boarded) domestic air carriers during calendar year 2005-
-the most recent year for which data were available when making our
selections in 2006.[Footnote 64] To the extent possible, we identified
the number of domestic enplanements for those air carriers required to
perform watch-list matching in 2005, identified within the previously
cited TSA list. According to our calculations, the 14 air carriers in
our study accounted for approximately 70 percent of all passengers who
boarded domestic air carriers' flights during calendar year 2005, and
thus, our selection allowed us to understand how watch-list matching
was performed for the majority of passengers flying domestically in
2005. Although the 14 domestic air carriers we selected represent a
range in size of air carrier operations and transported a majority of
passengers that boarded domestic flights in calendar year 2005, the
results of our interviews are not generalizable to all domestic air
carriers.
To help ensure consistency in conducting our interviews with air
carriers, we developed a data collection instrument with questions
focusing on air carriers' implementation of certain requirements of the
No Fly and Selectee list security directives. We conducted four of
these interviews in person at the air carriers' headquarters and the
rest via telephone. In addition, to clarify our understanding of air
carriers' processes, we conducted follow-up phone interviews with four
selected air carriers and received written answers to our follow-up
questions from an additional four selected air carriers. The air
carrier officials who answered our questions generally held positions
in corporate security and regulatory affairs; however, half of the air
carriers also had information technology systems specialists
participate to answer technical questions related to automated name-
matching systems. We did not audit or independently verify each air
carrier's implementation of TSA's security directive requirements;
rather, our work summarizes the capabilities as reported by officials
at the 14 air carriers.
Finally, to understand challenges air carriers have experienced in
implementing watch-list-matching requirements, we examined TSA's case
files on all regulatory violations of the No Fly List Procedures and
the Selectee List Procedures security directives reported since the
directives were first issued by TSA in 2002 to the time TSA provided us
with the data in November 2007--a total of 32 cases.[Footnote 65] We
reviewed these case files, which contained documentation and other
legal analyses pertaining to TSA's inspection findings following the
discovery of the violation, to determine the nature and causes (i.e.,
human or electronic) of the violations and to identify any patterns
among the cases. Finally, to clarify the agency's process for
investigating and adjudicating security directive violations, we spoke
with officials from TSA's Office of Chief Counsel.
Extent to Which TSA Has Assessed Domestic Air Carriers' Compliance with
Watch-List-Matching Requirements for Prescreening Passengers:
To address this objective, we first obtained an overview of TSA's plans
and guidance for assessing air carriers' compliance with regulatory
requirements. For instance, to understand the inspection process, the
focus of inspections, and inspection methods, we reviewed TSA's
National Inspection Manual, the Principal Security Inspector Handbook,
and related implementing guidance and policy documents. Further, we
interviewed or received written responses to our submitted questions
from the general manager of TSA's Office of Transportation Sector
Network Management, the two branch chiefs in the office's Commercial
Aviation Sector, and all nine of the office's principal security
inspectors. We particularly focused on contacting the principal
security inspectors because they are responsible for conducting
inspections at air carriers' corporate security offices (where watch-
list-matching policies and procedures are formulated) that apply across
an air carrier's operations. In addition, to obtain information on the
creation of inspection plans and guidance and the compilation and
analysis of inspection data, we spoke with individuals in the Office of
Security Operations and the Office of Transportation Sector Network
Management. Also, to obtain management's perspectives on inspections,
we spoke with the assistant general managers of the Office of Security
Operations' Compliance Division and its Procedures Division. We also
interviewed two federal security directors[Footnote 66] and two
transportation security inspectors, also within TSA's Office of
Security Operations and who were located in the Washington, D.C.,
metropolitan area, on planning and conducting inspections.
After obtaining an understanding of TSA's plans and guidance for
assessing air carriers' compliance with regulatory requirements, we
reviewed the results of TSA inspections that are scheduled on a regular
basis in conjunction with annual inspection plans. In conducting
inspections each year, TSA's inspectors use an extensive list of
inspection guidelines (known as PARIS prompts)[Footnote 67] that cover
a broad range of applicable topics--including topics outside the scope
of our review, such as airport perimeter security and cargo security,
as well as screening of employees and baggage.[Footnote 68] As
presented in table 3, we determined that TSA used 11 inspection
guidelines during fiscal year 2007 that were relevant to the objectives
of our review.[Footnote 69] Of these, guidelines 1, 2, and 6 through 11
were applicable to inspections conducted by principal security
inspectors, while guidelines 3 through 5 were applicable to inspections
conducted by transportation security inspectors.
Table 3: Watch-List-Matching Requirements and the Related Inspection
Guidelines (Fiscal Year 2007):
Requirements (key processes): Retrieving the No Fly and Selectee lists;
Inspection guidelines (prompts):
1. Procedures are in place to ensure the most recently issued No Fly
List is utilized within 24 hours of receipt;
2. Procedures are in place to ensure the most recently issued Selectee
List is utilized within 24 hours of receipt.
Requirements (key processes): Matching passenger data to No Fly and
Selectee lists;
Inspection guidelines (prompts):
3. All passenger names are compared to the most current No Fly and
Selectee lists in accordance with the Private Charter Standard Security
Program;
4. The aircraft operator is comparing all passenger names to the most
current No Fly and Selectee lists in accordance with the procedures
outlined in Security Directive 1544-01-20 series (No Fly) and Security
Directive 1544-01-21 series (Selectee).
Requirements (key processes): Using the TSA Cleared List;
Inspection guidelines (prompts):
5. A passenger identified as a match on the Selectee List is cleared,
along with his or her accessible property.
Requirements (key processes): Notifying authorities;
Inspection guidelines (prompts):
6. Procedures are in place to contact the federal security director,
local law enforcement, FBI, and TSA Office of Intelligence for matches
to the No Fly List;
7. Procedures are in place to contact the TSA Office of Intelligence
for matches to the Selectee List.
Requirements (key processes): Keeping records[A];
Inspection guidelines (prompts):
8. Records are maintained of all flights operated with passengers who
were determined by a local law enforcement, U.S. legal attaché, or TSA
Office of Intelligence not to be a match;
9. Records are maintained of every flight operated with passengers who
are designated as selectees;
10. Records are maintained of every flight with an individual who is
cleared to fly utilizing data in the TSA Cleared List including the
name of the cleared individual and the accepting aircraft operator
representative[B] (No Fly List);
11. Records are maintained of every flight with an individual who is
cleared to fly utilizing data in the TSA Cleared List including the
name of the cleared individual and the accepting aircraft operator
representative[B] (Selectee List).
Sources: GAO analysis of TSA's security directives and related
guidance.
[A] Maintaining accurate records, according to TSA officials, provides
a starting point for an investigation in the event of a terrorist
incident.
[B] This inspection guideline reflects the current process, which is to
use the TSA Cleared List. Security directives in effect prior to April
2008 referenced a "cleared column," a format for clearing passengers.
TSA eventually replaced this format with the Cleared List and updated
language in the April 2008 revision of the No Fly List Procedures
security directive to reflect the new process.
[End of table]
In reference to the 12 inspection guidelines--the 11 guidelines listed
in table 3 and the 1 guideline discussed in footnote 12 of this
appendix--TSA queried its PARIS database to identify all inspections of
domestic air carriers conducted during fiscal year 2007 that used at
least one of these guidelines. In addition to determining the number of
inspections, we reviewed the fiscal year 2007 inspection data to
calculate compliance rates.[Footnote 70] We did not evaluate the
substantive basis for the inspectors' assessment decisions regarding
compliance with requirements.
To determine whether and to what extent TSA's inspectors tested the air
carriers' capability to conduct exact-name and similar-name matching,
we also reviewed documentation of testing in a data field (in the PARIS
database) that allowed inspectors to enter narrative comments regarding
similar-name matching, among other inspection activities. In doing so,
we conducted a formal content analysis by having two analysts
independently review comments in the data field and then resolve any
inconsistencies between the two sets of analytical observations.
Moreover, we submitted written questions to each of TSA's nine
principal security inspectors asking them to describe their practices
for testing air carriers' capability to identify similar-name
variations.
In contrast to these regular inspections, TSA also conducted a special
emphasis assessment and a special emphasis inspection, nonroutine
activities conducted at the direction of TSA headquarters. A special
emphasis assessment addresses a vulnerability that generally is not
tied to a regulation, while a special emphasis inspection is tied to a
regulatory requirement. TSA provided us information on the scope,
methodology, and results of a special emphasis assessment that TSA
conducted during June, July, and September 2005. We reviewed the scope,
methodology, and results of this assessment with our methodologists and
with TSA officials. We determined that the sampling and related
procedures used for the special emphasis assessment were insufficient
for providing a reliable estimate of the success rate of all attempted
matches by air carriers; thus, the results cannot be used to infer
overall or individual rates of compliance with the name-matching
requirements in TSA's security directives.
In February 2008, TSA provided us a briefing on the scope and
methodology of a special emphasis inspection conducted the month before
in which the similar-name-matching capability of 52 domestic air
carriers and 31 foreign air carriers was tested. The briefing also
covered analyses of the results to date of the special emphasis
inspection and a discussion of the corrective actions that TSA was
planning to implement to address deficiencies. In April 2008, TSA
provided us with an updated briefing on its plans for corrective
actions. In September 2008, we requested information on TSA's progress
with these corrective actions. In response, TSA provided us the results
of a special emphasis assessment (conducted during May 20-29, 2008) of
seven air carriers' compliance with requirements in the April 2008 No
Fly List security directive. We did not assess the reliability of the
data TSA collected during the January 2008 special emphasis inspection
nor the May 2008 special emphasis assessment.
Reliability of Fiscal Year 2007 Inspections Data:
In assessing the reliability of the fiscal year 2007 data that TSA
provided us for watch-list-related inspections based on annual
inspection cycles, we performed electronic testing, discussed the data
system and any data inconsistencies we found with knowledgeable TSA
officials, and reviewed existing information about the system. Although
we determined that the data were reliable for the purposes of this
report, we have concerns about TSA's process for querying its
inspection database, and the potential for faulty output. The process
is cumbersome and prone to user error due, in part, to differences that
occur in the verbiage of inspection guidelines and types of inspections
as they are revised over time.
We conducted this performance audit from July 2006 to September 2008 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on the audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on the audit objectives.
[End of section]
Appendix II: Overview of Selected Domestic Air Carriers' Watch-List-
Matching Processes:
TSA's watch-list-matching requirements for domestic flights address
five key process areas: (1) retrieval of the No Fly and Selectee lists,
(2) the matching of passenger and list information, (3) the use of
TSA's Cleared List, (4) notification procedures, and (5) record-keeping
activities (see table 1).[Footnote 71]
To generally understand how TSA's requirements for watch-list matching
were being implemented, we reviewed documents in which TSA provided
general information on air carriers' processes. We also interviewed 14
domestic air carriers with operations ranging in size from
international to commuter service about their watch-list-matching
processes. All 14 air carriers were subject to TSA's requirements for
comparing passenger information with records on the No Fly and Selectee
lists and the TSA Cleared List.[Footnote 72] We asked each of the 14 to
describe their processes for meeting TSA's requirements.[Footnote 73]
The air carriers' implementation of these requirements can be discussed
in reference to three time periods--before passenger check-in, at
passenger check-in, and after passenger check-in--as reflected in the
following sections, respectively, and as illustrated in figure 1.
Before Passenger Check-in: Retrieving the No Fly and Selectee Lists and
Executing Name Comparisons:
The 14 air carriers told us that they obtain new versions of the No Fly
and Selectee lists through one or both of the following methods (1)
assigning an employee to monitor TSA's Web board for new postings at
certain intervals throughout the day, and (2) receiving an e-mail
message from TSA to the respective air carrier's security staff
informing them of new No Fly and Selectee lists. Also, all 14 air
carriers reported using passenger name record (PNR) data--data
collected from the passenger at the time a reservation is made--to make
comparisons against the No Fly and Selectee lists. Specifically, the
air carriers said that they have implemented procedures to execute
comparisons of PNR and watch-list data prior to scheduled flight
departure. Most of the air carriers told us they do this by using
computerized matching programs that automatically execute comparisons.
Because the 14 air carriers we interviewed did not collect date of
birth (an identifying data element that air carriers receive on the No
Fly and Selectee lists) within PNR data, this information generally was
not available for matching purposes prior to check-in. However, as
discussed later in this appendix, several air carriers reported
developing systems capable of accessing passenger date-of-birth
information collected and stored outside of PNR data for use in
comparisons conducted prior to check-in, but this information was not
available for all of their passengers. Thus, the 14 air carriers we
spoke with were limited to performing name-only comparisons--that is,
comparisons of passenger names with names on the No Fly and Selectee
lists--prior to check-in for at least some, if not all, passengers. All
14 air carriers we spoke with reported conducting comparisons to
identify exact-name matches of passengers and watch-list names.
However, not every air carrier reported conducting comparisons to
identify similar-name matches.
At Passenger Check-in: Completing Comparisons of Passenger and Watch-
List Information and Using TSA's Cleared List:
In accordance with TSA requirements, air carriers are to collect
additional identifying information at check-in to assist in identifying
passengers who are matches with information on the No Fly or Selectee
lists. Air carriers collect additional identifying information at check-
in only for those passengers identified as potential matches to the No
Fly or Selectee lists through the name-only comparisons they conduct
prior to check-in. To prevent individuals who are potential matches
from checking in by other means, such as using Internet or airport
kiosk check-in, air carriers with automated systems place an automatic
"lock" on boarding passes (see fig. 1).[Footnote 74] By doing so, the
air carriers force all potentially matched passengers to check in at
the ticket counter, where an agent is to collect a valid form of
identification with date of birth (typically, a government-issued
identification document such as a driver's license or passport) to
complete the comparison of passenger and watch-list information.
To check the potentially matched passenger's date of birth information
against the No Fly and Selectee lists, most of the 14 air carriers we
interviewed reported comparing the two dates manually, and the other
air carriers reported keying the passenger's date of birth into a
computer system that would automatically execute the comparison.
[Footnote 75] The 14 air carriers reported that if they determine that
the dates of birth do not match, they unlock the boarding pass without
consulting TSA, in accordance with TSA requirements, thereby allowing
the passenger to continue the boarding process (see fig. 1, post-check-
in number 1).[Footnote 76] However, if a passenger's date of birth
matches with that of an individual on the No Fly or Selectee lists, the
14 air carriers said that they consider the passenger to be a match and
followed the procedures outlined in TSA's security directives for
handling matches to the No Fly or Selectee lists (see fig. 1, post-
check-in numbers 2 and 3).
Figure 1: Overview of the Current Passenger Watch-List-Matching
Process:
[See PDF for image]
This figure is an illustration of the current passenger watch-list-
matching process, as follows:
Precheck-in:
* Passenger makes reservation; PNR is created;
* TSA posts No Fly, Selectee, and TSA Cleared lists to secure Web
board;
* Air carrier retrieves list data;
* Air carrier system compares PNR data to List data.
* Potential match: Yes, proceed to check-in.
Check-in:
* Clearance process for locked PNRs (system match during comparison)?
- Passengers present government ID at ticket counter;
Agent compares passenger‘s ID to No Fly, Selectee, and TSA Cleared list
data.
* Check-in process for Nonlocked PNRs (not a potential match in system
comparison): Internet, kiosk or ticket counter.
Post Check-in:
* Matching results:
1. Cleared: Passenger identified as not being on No Fly or Selectee
list; Passenger identified as a match to cleared list:
* PNR unlocked; Checkpoint screening; Passenger proceeds to flight.
2. Selectee match: Passenger identified as a match to Selectee list:
* PNR unlocked; Additional screening; Checkpoint screening; Passenger
proceeds to flight.
3. No Fly: Passenger identified as a match to No Fly list:
* Air carrier contacts appropriate officials;
- Not cleared; PNR locked; Boarding pass denied; Or:
- PNR unlocked; Cleared; PNR unlocked; downgraded to selectee;
Additional screening; Checkpoint screening; Passenger proceeds to
flight; Or:
- PNR unlocked; Cleared; Additional screening; Checkpoint screening;
Passenger proceeds to flight.
Source: GAO analysis.
[End of figure]
Also, 10 air carriers reported using the TSA Cleared List to identify
and clear passengers misidentified as a match to the No Fly List or the
Selectee List, generally at the time of check in. The other 4 air
carriers reported not using the list--despite TSA's requirement that
all air carriers do so. In addition, of the 10 air carriers that
reported using the cleared list, 2 reported using the list in
conjunction with their independently developed processes to "pre-clear"
individuals (discussed below). Development of such processes was
undertaken to allow air carriers to identify and clear misidentified
passengers without requiring them to check in at the ticket counter.
Specifically, 11 of the 14 air carriers we interviewed reported that
individuals on the TSA Cleared List still must approach the ticket
counter at check in.[Footnote 77] Consequently, 6 of the 14 air
carriers that we interviewed reported developing alternative clearance
processes to decrease the number of potentially matched individuals who
are required to check in at the ticket counter. These 6 carriers
explained that their internally developed clearance processes operate
by using additional data sources, such as passenger information
collected in frequent flier databases, to resolve potential matches
prior to check in. For example, if an air carrier collected date of
birth within its frequent flier database, its internal clearance system
would compare the date of birth of a potentially matched passenger who
had entered a frequent flier number when making a reservation with the
date of birth of the respective individual on the No Fly List or the
Selectee List.[Footnote 78]
After Passenger Check in: Implementing the Notification and Record-
Keeping Procedures Specified in TSA's No Fly and Selectee Security
Directives:
For match determinations made at the time of passenger check in, TSA's
No Fly and Selectee list security directives require that air carriers
follow certain notification and record-keeping procedures. With regard
to notification procedures:
* If the air carrier identifies a passenger as a potential match to the
No Fly List, the air carrier must contact both the applicable federal
security director and the appropriate law enforcement officer. Then, if
the law enforcement officer confirms that the passenger is a match, the
air carrier is to contact the local Federal Bureau of Investigation
(FBI) field office and TSA's Office of Intelligence.
* If the air carrier identifies a passenger as a potential match to the
Selectee List, the air carrier must mark the passenger's boarding pass
to indicate to checkpoint screeners that the passenger should be
subject to enhanced checkpoint screening. Also, the air carrier must
notify TSA's Office of Intelligence that the passenger has been matched
with the Selectee List.
With regard to record-keeping procedures, TSA's security directives
require that air carriers maintain a record of (1) all passengers
cleared using the TSA Cleared List, (2) all flights that had
potentially matched passengers who were determined by local law
enforcement not to be a match to the No Fly List, and (3) all
passengers identified as matches with the Selectee List.
Generally, the 14 air carriers told us that they followed the
notification and record-keeping requirements specified in TSA's
security directives, but reported having different procedures in place
to implement these requirements. For example, upon identifying a
potential match to the No Fly List, 5 air carriers reported requiring
their ticket agents to notify their respective air carrier's ground
security coordinator, who would then make the necessary calls to the
applicable TSA federal security director and to local law enforcement.
Three other air carriers reported requiring that ticket agents contact
security staff at a centralized call center, and these staff would then
make the necessary notifications.[Footnote 79] In addition, some of the
carriers reported using some slight deviations from the stated
requirements. For example, rather than notifying the local FBI field
office and TSA's Office of Intelligence of a match only after a local
law enforcement officer has confirmed the match, 8 air carriers
reported contacting TSA's Office of Intelligence for every passenger
whose information matched the No Fly List, regardless of the local law
enforcement officer's input. [Footnote 80]
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Cathleen A. Berrick, (202) 512-3404 or berrickc@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Danny Burton and Christine
Fossett (Assistant Directors) and Mona Blake and Mike Bollinger
(Analysts-in-Charge) managed this assignment.
Suzanne Heimbach, Matt Mohning, Justin Monroe, Alison Sands, and Susan
Woodward made significant contributions to the work.
David Alexander, Michele Fejfar, and Rich Hung assisted with design,
methodology, and data analysis.
Tom Lombardi and David Plocher provided legal support.
Richard Ascarate, Ryan Consaul, Kevin Copping, Kristen Jensen, Lara
Kaskie, Maria Soriano, William D. Updegraff, and Margaret Vo provided
assistance in report preparation.
[End of section]
Footnotes:
[1] Watch-list matching is one of two TSA-mandated prescreening
processes conducted by air carriers. The other mandated prescreening
activity is the Computer Assisted Passenger Prescreening System,
discussed later this report, which does not involve matching passenger
information against the No Fly and Selectee lists. These lists contain
applicable records from the Terrorist Screening Center's consolidated
database of known or appropriately suspected terrorists. See GAO,
Terrorist Watch List Screening: Recommendations to Promote a
Comprehensive and Coordinated Approach to Terrorist-Related Screening,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-253T] (Washington,
D.C.: Nov. 8, 2007).
[2] The number of domestic air carriers has varied over time, for
example, from 95 in 2005 to about 70 in 2007. For the purposes of this
report, domestic air carriers are those with operations based in the
United States that maintain full security programs in accordance with
49 C.F.R. part 1544. Foreign air carriers--air carriers with operations
based outside the United States--must also comply with U.S. security
regulations, including applicable requirements for watch-list matching,
when operating flights to or from the United States in accordance with
49 C.F.R. part 1546. Both domestic and foreign air carriers may conduct
international flights to and from the United States; however, these
operations are outside the scope of this report.
[3] See GAO, Terrorist Watch List Screening: Opportunities Exist to
Enhance Management Oversight, Reduce Vulnerabilities in Agency
Screening Processes, and Expand Use of the List, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-110] (Washington, D.C.: Oct.
11, 2007). We reported that TSA's Office of Intelligence documented
various incidents (for the period January 1, 2005, through June 3,
2007) in which air carriers--both domestic and foreign--allowed
individuals on the No Fly List to board international flights traveling
to or from the United States. Several of these incidents involved
flights of domestic air carriers. We asked TSA's Office of Intelligence
to identify any additional incidents in which a No Fly listed
individual flew on a domestic air carrier for the period June 4, 2007,
through December 31, 2007, and TSA identified no additional incidents
occurring within this time period.
[4] This issue of false negatives is addressed later in this report.
[5] See, e.g., 49 C.F.R. § 1544.305. Although generally issued in
response to an immediate or imminent threat, security directives may be
effective for an indefinite duration if TSA determines that a
continuing need for such measures exists. In some cases, aviation-
related measures implemented through a security directive have been
discontinued, amended, or incorporated into air carrier security
programs.
[6] See 49 U.S.C. 44903(j)(2)(C).
[7] GAO, Aviation Security: Computer-Assisted Passenger Prescreening
System Faces Significant Implementation Challenges, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-04-385] (Washington, D.C.: Feb.
13, 2004); Aviation Security: Management Challenges Remain for the
Transportation Security Administration's Secure Flight Program,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington,
D.C.: June 14, 2006); and Aviation Security: Transportation Security
Administration Has Strengthened Planning to Guide Investments in Key
Aviation Security Programs, but More Work Remains, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-456T] (Washington, D.C.: Feb.
28, 2008).
[8] GAO, Transportation Security: Efforts to Strengthen Aviation and
Surface Transportation Security Continue to Progress, but More Work
Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-651T]
(Washington, D.C.: Apr. 15, 2008).
[9] See Pub. L. No. 110-161, Div. E, § 513, 121 Stat. 1844, 2072-73
(2007).
[10] GAO, Aviation Security: Management Challenges Remain for the
Transportation Security Administration's Secure Flight Program,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington,
D.C.: June 14, 2006).
[11] We are conducting this review in response to requests from the
House of Representatives (Committee on Transportation and
Infrastructure, Committee on Homeland Security, and Committee on
Oversight and Government Reform). These requesters asked that we review
the current passenger prescreening system in conjunction with our
ongoing work related to TSA's progress with Secure Flight. In addition,
we are reporting on this issue to the U.S. Senate requesters and the
mandate committees associated with our Secure Flight work.
[12] Pursuant to Homeland Security Presidential Directive 6, dated
September 16, 2003, the Terrorist Screening Center--an entity that has
been operational since December 2003 under the administration of the
FBI--was established to develop and maintain the U.S. government's
consolidated terrorist screening database (the watch list) and to
provide for the use of watch-list records during security-related
screening processes.
[13] All 14 air carriers we interviewed operate under full security
programs in accordance with 49 C.F.R. part 1544 and conduct watch-list
matching in accordance with the No Fly and Selectee list security
directives issued by TSA.
[14] The Department of Transportation groups U.S.-based air carriers
according to their operating revenue. In the 2005 groupings, each of
the "major" air carriers had over $1 billion in operating revenue.
[15] Of these six, the Department of Transportation's 2005 revenue
groupings identified three as "national" air carriers, with each having
over $100 million to $1 billion in operating revenue, and one as a
"regional" air carrier, with $100 million or less in operating revenue.
The other two air carriers were not included in the department's
revenue groupings, given the small scale of operations, but were
identified by the department as air carriers that provide commuter
service. Major air carriers have over $1 billion in operating revenue.
[16] The No Fly and Selectee list security directives also address the
screening of air carrier employees against the No Fly and Selectee
lists, but our scope was confined to the passenger-specific
prescreening requirements in the security directives.
[17] Special emphasis assessments and special emphasis inspections are
nonroutine activities undertaken at the direction of TSA headquarters.
According to TSA, a special emphasis assessment addresses a
vulnerability that generally is not tied to a regulation, while a
special emphasis inspection is tied to a regulatory requirement.
[18] In September 2008, TSA provided us the results of a special
emphasis assessment (conducted during May 2008) of seven air carriers'
compliance with new requirements in the No Fly List security directive,
which was revised in April 2008 to specify a baseline capability for
conducting watch-list matching. This special emphasis assessment is
discussed later in this report.
[19] In September 2008, TSA informed us that the revised Selectee List
security directive was still in the agency's internal clearance process
but did not provide us a targeted issuance date.
[20] PARIS is the acronym for the Performance and Results Information
System, which is TSA's inspections database. This database assists TSA
management by providing factual and analytical information on the
compliance of TSA-regulated entities. There are approximately 1,700
PARIS prompts, which serve as guidelines for TSA inspectors.
[21] According to TSA data, these 1,145 watch-list-related inspections
(36 plus 1,109) covered 60 domestic air carriers, and most of the air
carriers were inspected multiple times.
[22] TSA reported that the January 2008 special emphasis inspection
covered 52 domestic air carriers and 31 foreign air carriers.
[23] In accordance with 49 U.S.C. § 114(h), TSA adopted policies and
procedures for ensuring that air carriers use information from
government agencies to identify individuals on passenger lists who may
be a threat to civil aviation or national security and, if such an
individual is identified, notify appropriate law enforcement agencies,
prevent the individual from boarding an aircraft, or take other
appropriate action with respect to that individual.
[24] Pub. L. No. 107-71, § 136, 115 Stat. 597, 637 (2001) (codified at
49 U.S.C. § 44903(j)(2)(A)) (requiring use of the Computer Assisted
Passenger Prescreening System or any successor system).
[25] For the purposes of this report, we address policies and
procedures applicable to air carriers regulated under 49 C.F.R. part
1544 (U.S.-flagged air carriers), which we refer to as domestic air
carriers. For these air carriers, we limit our discussion to the watch-
list matching TSA requires to secure the aviation sector for domestic
flights--air carrier operations between two points within the United
States or its territories. TSA requirements also address the
international operations of domestic air carriers, and the operations
of foreign-flagged air carriers flying to and from destinations within
the United States and its territories in accordance with 49 U.S.C. part
1546; however, these requirements are outside the scope of our review.
[26] The most recent version of the No Fly List Procedures security
directive is SD 1544-01-20F, dated April 9, 2008, and the most recent
version of the Selectee List Procedures security directive is SD 1544-
01-21F, dated March 8, 2007.
[27] On June 10, 2008, the Department of Justice provided us comments
on a draft of the restricted version of this report (GAO-08-453SU) and
noted that the Principals Committee, which is a senior interagency
forum under the Homeland Security Council, had approved additional
criteria that the Terrorist Screening Center would begin implementing
on June 23, 2008. The Homeland Security Council was established to
ensure coordination of all homeland-security-related activities among
executive departments and agencies and promote the effective
development and implementation of all homeland security policies. See
The White House, Homeland Security Presidential Directive/HSPD-1,
Organization and Operation of the Homeland Security Council
(Washington, D.C.: Oct. 29, 2001).
[28] Each watch-list record, however, does not necessarily indicate a
separate individual on the list. Some listed individuals have multiple
records attributed to them due to the inclusion of known aliases and
name variations.
[29] The lists may also be provided via password-protected e-mail.
[30] TSA initiated this effort in response to the Aviation and
Transportation Security Act, which requires that TSA ensure that a
system is used to evaluate all passengers before they board an aircraft
and ensure that selected individuals and their carry-on and checked
baggage are adequately screened. See Pub. L. No. 107-71, § 136, 115
Stat. at 637 (codified at 49 U.S.C. § 44903(j)(2)(A)).
[31] The National Commission on Terrorist Attacks Upon the United
States, The 9/11 Commission Report - Final Report of the National
Commission on Terrorist Attacks Upon the United States (Washington,
D.C.: 2004), p. 393.
[32] Pub. L. No. 108-458, § 4012(a)(1), 118 Stat. 3638, 3714-17 (2004)
(codified at 49 U.S.C. § 44903(j)(2)(C) (2004)). A separate provision
enacted at section 4012(a)(2) addressed the predeparture screening of
international passengers, with the Secretary of Homeland Security
giving this responsibility to U.S. Customs and Border Protection. See
49 U.S.C. § 44909(c)(6).
[33] With regard to redress protections, DHS must have a process
whereby aviation passengers determined to pose a threat to aviation
security by Secure Flight may appeal that determination and correct
erroneous information contained within the prescreening system. See
GAO, Aviation Security: Significant Management Challenges May Adversely
Affect Implementation of the Transportation Security Administration's
Secure Flight Program, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-06-374T] (Washington, D.C.: Feb. 9, 2006).
[34] See 72 Fed. Reg. 48,356 (Aug. 23, 2007). Requirements described in
the notice of proposed rulemaking are subject to revisions based on
various considerations, including input that TSA received during the
public comment period. As of the date of this report's issuance, DHS
had not issued a final Secure Flight rule.
[35] GAO, Aviation Security: Transportation Security Administration Has
Strengthened Planning to Guide Investments in Key Aviations Security
Programs, but More Work Remains, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-08-465T] (Washington, D.C.: Feb. 28, 2008).
[36] In September 2008, TSA informed us that the revised Selectee List
security directive was still in the agency's internal clearance process
but did not provide us a targeted issuance date.
[37] When making determinations on matches, air carriers must use the
TSA Cleared List, which is composed of names and other personal-
identifying information on individuals whom the Department of Homeland
Security has reviewed and determined are not individuals on the No Fly
or Selectee lists.
[38] Specifically, we reviewed and discussed the No Fly and Selectee
list security directives and identified within each the key
requirements pertaining to domestic flights. Although the same
requirements generally apply to the international flights of both
domestic and foreign air carriers, such operations fall outside the
scope of our review. For more information on how we identified
requirements for watch-list matching, see appendix I.
[39] TSA's revised No Fly List Procedures security directive (SD 1544-
01-20F) is dated April 9, 2008. Also, in April 2008, TSA reported that
the current Selectee List Procedures security directive (SD 1544-01-
21F) would be similarly revised. In September 2008, TSA informed us
that the revised Selectee List security directive was still in the
agency's internal clearance process but did not provide us a targeted
issuance date.
[40] We did not independently verify the air carriers' approaches to
watch-list matching. Unless noted otherwise, our summary of the air
carriers' approaches is based on system capabilities reported to us in
14 separate interviews with the respective air carriers. Appendix II
provides more detail on the 14 air carriers' reported approaches to
watch-list matching.
[41] Some of these flights involved passengers who flew from one
domestic location to another domestic location, where they boarded an
international flight. TSA learned that the individual on the No Fly
List flew domestically after U.S. Customs and Border Protection
identified the individual on the international leg.
[42] GAO, Terrorist Watch List Screening: Opportunities Exist to
Enhance Management Oversight, Reduce Vulnerabilities in Agency
Screening Processes, and Expand the Use of the List, GAO-08-110
(Washington, D.C.: Oct. 11, 2007).
[43] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-110].
[44] GAO, Aviation Security: Management Challenges Remain for the
Transportation Security Administration's Secure Flight Program,
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington,
D.C.: June 14, 2006).
[45] Upon completing a reassessment of the Secure Flight program in
February 2007, TSA produced this document to identify decisions made
about Secure Flight's capabilities during the reassessment. See TSA,
Secure Flight Program Baseline (Washington, D.C.: March 2007), p. 5.
[46] As mentioned previously, in September 2008, TSA informed us that
the revised Selectee List security directive was still in the agency's
internal clearance process but did not provide us a targeted issuance
date.
[47] TSA reported that the January 2008 special emphasis inspection
covered 52 domestic air carriers and 31 foreign air carriers.
[48] According to TSA officials, the agency had planned to conduct
tests of all 81 domestic air carriers that were subject to the No Fly
List Procedures security directive at that time. However, the officials
explained that due to limited resources, initial testing covered 63 air
carriers (encompassing operations at 354 airports), and the retesting
covered 36 air carriers (encompassing operations at 290 airports).
[49] As noted earlier, we concluded that these inspection data were
sufficiently reliable for the purposes of this report, but we have
concerns about the potential for error based on TSA's process for
querying its inspection database (we discuss these concerns in more
detail in app. I).
[50] Regarding the air carriers that did not receive a watch-list-
related inspection during fiscal year 2007, TSA does not require
inspectors to inspect each air carrier every year in terms of watch-
list-related requirements. However, a senior TSA official in the
compliance area who supervises inspectors stated that annually
inspecting every air carrier is a goal, at least for principal security
inspectors.
[51] We did not evaluate the basis for the inspectors' assessment
decisions regarding compliance with requirements. Although TSA's
security directives require comparisons of passenger and employee names
to the No Fly and Selectee lists, our review was confined to
requirements related to passengers only.
[52] To report their findings in TSA's automated database, inspectors
select one of four options from a computer-generated list: not
inspected, not applicable, not in compliance, and in compliance. If the
inspectors wish to add narrative to describe their findings, they can
do so in a data field reserved for comments.
[53] In appendix I, see table 3.
[54] TSA, National Inspection Manual, 2007. Inspections for all
regulated areas (not just watch-list-related inspections) generally
incorporate all of four methods--testing, document review, interviews,
and surveillance.
[55] TSA, Regulatory Activities Plan for Transportation Security
Inspectors Fiscal Year 2008.
[56] We briefed the TSA Administrator and other senior officials on the
results of our work in November 2007.
[57] As noted previously, each watch-list record does not necessarily
indicate a separate individual on the list. Some listed individuals
have multiple records attributed to them due to the inclusion of known
aliases and name variations.
[58] GAO, Aviation Security: Pending Implementation of Secure Flight,
TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify
Passengers on the No Fly and Selectee Lists, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-453SU] (Washington, D.C.: July
10, 2008).
[59] These directives apply to domestic air carriers--that is, U.S. air
carriers that maintain security programs in accordance with 49 C.F.R.
part 1544. The directives govern watch-list matching for flights
operating between two points within the United States or its
territories. Although outside the scope of our review, the directives
also apply to domestic air carriers' international operations. At the
start of our review, we based our analysis on the No Fly List
Procedures (1544-01-20D) security directive and the Selectee List
Procedures (1544-01-21E) security directive, both dated July 8, 2004.
Over the course of our review, TSA first issued revised security
directives in 2007 and has undertaken to revise them again in April
2008. The 2007 revisions of the No Fly and Selectee list security
directives (SD 1544-01-20E and SD1544-01-21F, respectively) clarified
certain elements of the directives but resulted in no substantive
changes in the requirements. Generally, in this report, we focus on the
changes in requirements resulting from revisions undertaken in April
2008 (SD 1544-01-20F and anticipated SD 1544-01-21G (Selectee List),
respectively).
[60] We based our understanding of TSA's planned capabilities for
Secure Flight on our February 2006 testimony before the Senate
Committee on Commerce, Science, and Transportation, our most recent,
comprehensive testimony on the program when we initiated our work in
July 2006. See GAO, Aviation Security: Significant Management
Challenges May Adversely Affect Implementation of the Transportation
Security Administration's Secure Flight Program, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-06-374T] (Washington, D.C.: Feb.
9, 2006).
[61] Although addressed in the security directives, other requirements
that we excluded from our scope involved, for example, procedures
involving the screening of employees and procedures related to the
international operations of domestic air carriers. We did not consider
requirements for domestic air carriers' international flights as part
of our review because at the time we were planning our review, TSA
intended for Secure Flight to take over the watch-list-matching
function for only domestic flights. U.S. Customs and Border Protection
was expected to conduct the watch-list-matching function for flights
arriving from or departing to locations outside the United States, not
Secure Flight. However, in February 2008 we reported in testimony that,
as agreed to by the respective agencies, TSA will also take over the
matching of international passengers against the No Fly and Selectee
lists from U.S. Customs and Border Protection. GAO, Aviation Security:
Transportation Security Administration Has Strengthened Planning to
Guide Investments in Key Aviation Security Programs, but More Work
Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-465T]
(Washington, D.C.: Feb. 28, 2008).
[62] One objective of the Federal Identity Match Search Engine
Performance Standards Working Group is to provide guidance to improve
the effectiveness of the automated search engines that federal agencies
use for conducting identity matching. The group began meeting in
December 2005. It included representatives from the departments of
Homeland Security, State, and Defense; FBI; the intelligence community;
and the National Institute of Standards and Technology.
[63] Specifically, we reviewed the Secure Flight notice of proposed
rulemaking (72 Fed. Reg. 48,356 (Aug. 23, 2007)) and final concept of
operations for Secure Flight (dated Mar. 9, 2007). We also reviewed our
most recent reports and testimonies on the program.
[64] Specifically, the data reflect the number of domestic passengers
who boarded (enplaned) at a flight's point of origin in calendar year
2005. The data include only revenue passengers, or passengers from whom
the air carrier received payment. As such, the data exclude passengers
using frequent flier vouchers, infants, air carrier employees, etc.
[65] The earliest case was dated December 3, 2003; the most recent was
dated August 24, 2007. Because some domestic air carriers that are
subject to security directives fly internationally, 7 of the 32 cases
involved flights arriving from or departing to international locations.
Although we excluded such flights from our review of watch-list-
matching requirements, as mentioned previously, we retained these 7
cases in our analysis of regulatory violations. We did so because (1)
the requirements for air carriers to perform watch-list matching for
flights involving an international location are, for the most part, the
same as those for air carrier operations between two points within the
United States or its territories, and (2) in August 2007, TSA announced
that Secure Flight would eventually assume watch-list matching for
passengers on flights arriving from or departing to locations outside
the United States.
[66] Federal security directors are responsible for leading and
coordinating TSA security activities at airports across the nation.
[67] The Performance and Results Information System (PARIS) is an
inspections database that assists TSA management by providing factual
and analytical information on the compliance of TSA-regulated entities.
[68] As mentioned previously, the watch-list-matching requirements
relevant to the objectives of our review are shown in table 1, which is
presented earlier in this report.
[69] TSA provided us with data for 12 inspection guidelines. These 12
are the 11 guidelines shown in table 3--plus the following guideline,
which was replaced in March 2007 with guideline 4 in table 3: "All
passenger names are compared to the most current No Fly and Selectee
Lists in accordance with the procedures outlined in Security Directive
1544-01-20 series (No Fly) and Security Directive 1544-01-21 series
(Selectee)." Because these two guidelines were used for the same
purpose but at different times during fiscal year 2007, we combined the
data associated with each one and treated them as one inspection
guideline for the purposes of this report.
[70] Our calculations were based only on the 12 inspection guidelines
relevant to our review.
[71] To identify these requirements, we reviewed the No Fly List
Procedures and Selectee List Procedures security directives (series SD
1544-01-20 and SD 1544-01-21, respectively). This report discusses only
the requirements within the two security directives pertaining to
domestic flights (defined as flights occurring between points within
the United States and its territories), though these same requirements
generally apply to the international flights of both domestic and
foreign air carriers. For more information on how we identified
requirements for watch-list matching, see appendix I.
[72] For information on our methodology for selecting the 14 air
carriers and conducting the interviews, see appendix I.
[73] The implementation methods described in this appendix are based on
descriptions obtained from the 14 air carriers. We did not undertake
audits of the air carriers' processes to confirm that the processes
functioned as described in the interviews. Specifically, we asked air
carriers questions on methods for securing the most recent No Fly and
Selectee lists, executing comparisons within required time frames,
determining valid matches, and implementing required notification and
reporting procedures.
[74] The one air carrier in our review without an automated system
reported requiring all passengers, regardless of whether they were a
potential match, to check in at the ticket counter. To identify those
passengers who should submit additional information for further
comparison against the No Fly and Selectee lists at check-in, this air
carrier reported having its employee in charge of watch-list matching
make a written notation next to the name of all identified potential
matches on a printed list of passengers with reservations.
[75] In addition, to check potentially matched passenger information
against the No Fly and Selectee lists, three air carriers reported that
they had developed kiosks with capabilities to read electronic date of
birth information from certain forms of identification that are machine
readable.
[76] After this point, the passenger generally experiences no further
inconvenience due to watch-list matching. However, the passenger may be
selected for enhanced checkpoint screening as a result of the Computer
Assisted Passenger Prescreening System (CAPPS)--an electronic
application that selects individuals for enhanced screening at the
passenger checkpoint based on certain travel characteristics identified
by TSA as indicating potential risk.
[77] These individuals are required to check in at the ticket counter
because the air carrier must confirm that the passenger is the cleared
individual by comparing the passenger's legal identifying documentation
with the TSA Cleared List.
[78] Air carriers with frequent flier programs generally have the
capability to collect a frequent flier number within the PNR;
therefore, unlike date of birth information, frequent flier numbers are
available to air carriers prior to a passenger's arrival at check-in
and can be used to assist in the confirmation of a passenger's identity
because of the presence of date of birth information in the passenger's
frequent flier account.
[79] Another air carrier reported requiring the ticket agent to make
these notifications; the other five air carriers we interviewed did not
discuss this aspect of the watch-list-matching process.
[80] Two air carriers reported that (per the security directive
requirement) they waited for local law enforcement officer confirmation
before calling the FBI field office or TSA's Office of Intelligence.
One air carrier reported that it could not answer the question; that
is, having never identified an individual as a name and date of birth
match to the No Fly List, the air carrier could not say what its
actions would be. During our interviews, three air carriers did not
discuss this aspect of the watch-list-matching process.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
