Aviation Security
Secure Flight Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed Gao ID: GAO-05-356 March 28, 2005Among its efforts to strengthen aviation security, the Transportation Security Administration (TSA) is developing a new passenger prescreening system--known as Secure Flight. As required by Congress, TSA is planning to assume, through Secure Flight, the prescreening function currently performed by the air carriers. This report assesses the (1) status of Secure Flight's development and implementation, (2) factors that could influence the effectiveness of Secure Flight, (3) processes used to oversee and manage the Secure Flight program, and (4) efforts taken to minimize the impacts on passengers and protect passenger rights. In conducting this assessment, we addressed the 10 specific areas of congressional interest related to Secure Flight outlined in Public Law 108-334.
TSA is making progress in addressing each of the key areas of congressional interest related to the development and implementation of Secure Flight, including developing and testing the system. However, TSA has not yet completed these efforts or fully addressed these areas, due largely to the current stage of the system's development. For example, while TSA has drafted a concept of operations and system requirements, it has not finalized these key documents or completed test activities that will need to be accomplished before Secure Flight becomes operational. Until requirements are defined, operating policies are finalized, and testing is completed--scheduled for later in the system's development--we cannot determine whether Secure Flight will fully address these areas of interest. TSA also initiated a number of actions designed to improve the ability of Secure Flight to identify passengers who should undergo additional security scrutiny, in place of the prescreening currently conducted by air carriers. Specifically, TSA officials stated that recently completed initial testing identified improvements over the current prescreening system, and TSA plans to use intelligence analysts to increase the accuracy of data matches. However, the effectiveness of Secure Flight in identifying passengers who should undergo additional security scrutiny has not been fully determined. For example, TSA has not resolved how passenger data will be transmitted from air carriers to TSA to support Secure Flight operations. Further, the ability of Secure Flight to make accurate matches between passenger data and data contained in the terrorist screening database is dependent on the quality of the data used, which has not been determined. TSA has also strengthened the oversight and management of Secure Flight, and has established relationships with key program stakeholders. However, air carriers expressed concerns regarding the uncertainty of system requirements, and the impact these requirements may have on the airline industry in terms of system modifications and costs. Additionally, TSA has taken steps to minimize potential impacts on passengers and to protect passenger rights during Secure Flight testing. However, TSA has not yet clearly defined the privacy impacts of the operational system or all of the actions TSA plans to take to mitigate potential impacts.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-356, Aviation Security: Secure Flight Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed
This is the accessible text file for GAO report number GAO-05-356
entitled 'Aviation Security: Secure Flight Development and Testing
Under Way, but Risks Should Be Managed as System Is Further Developed'
which was released on March 28, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
March 2005:
Aviation Security:
Secure Flight Development and Testing Under Way, but Risks Should Be
Managed as System Is Further Developed:
GAO-05-356:
GAO Highlights:
Highlights of GAO-05-356, a report to congressional committees
Why GAO Did This Study:
Among its efforts to strengthen aviation security, the Transportation
Security Administration (TSA) is developing a new passenger
prescreening system”known as Secure Flight. As required by Congress,
TSA is planning to assume, through Secure Flight, the prescreening
function currently performed by the air carriers. This report assesses
the (1) status of Secure Flight‘s development and implementation, (2)
factors that could influence the effectiveness of Secure Flight, (3)
processes used to oversee and manage the Secure Flight program, and (4)
efforts taken to minimize the impacts on passengers and protect
passenger rights. In conducting this assessment, we addressed the
10 specific areas of congressional interest related to Secure Flight
outlined in Public Law 108-334.
What GAO Found:
TSA is making progress in addressing each of the key areas of
congressional interest related to the development and implementation of
Secure Flight, including developing and testing the system. However,
TSA has not yet completed these efforts or fully addressed these areas,
due largely to the current stage of the system‘s development. For
example, while TSA has drafted a concept of operations and system
requirements, it has not finalized these key documents or completed
test activities that will need to be accomplished before Secure Flight
becomes operational. Until requirements are defined, operating policies
are finalized, and testing is completed”scheduled for later in the
system‘s development”we cannot determine whether Secure Flight will
fully address these areas of interest.
TSA also initiated a number of actions designed to improve the ability
of Secure Flight to identify passengers who should undergo additional
security scrutiny, in place of the prescreening currently conducted by
air carriers. Specifically, TSA officials stated that recently
completed initial testing identified improvements over the current
prescreening system, and TSA plans to use intelligence analysts to
increase the accuracy of data matches. However, the effectiveness of
Secure Flight in identifying passengers who should undergo additional
security scrutiny has not been fully determined. For example, TSA has
not resolved how passenger data will be transmitted from air carriers
to TSA to support Secure Flight operations. Further, the ability of
Secure Flight to make accurate matches between passenger data and data
contained in the terrorist screening database is dependent on the
quality of the data used, which has not been determined.
TSA has also strengthened the oversight and management of Secure
Flight, and has established relationships with key program
stakeholders. However, air carriers expressed concerns regarding the
uncertainty of system requirements, and the impact these requirements
may have on the airline industry in terms of system modifications and
costs. Additionally, TSA has taken steps to minimize potential impacts
on passengers and to protect passenger rights during Secure Flight
testing. However, TSA has not yet clearly defined the privacy impacts
of the operational system or all of the actions TSA plans to take to
mitigate potential impacts.
Secure Flight Passenger Prescreening Process:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO recommends that the Department of Homeland Security (DHS) direct
TSA to take several actions to mange risks associated with Secure
Flight‘s development, including (1) finalizing requirements and test
plans, privacy and redress requirements, and program cost estimates;
and (2) establishing plans to achieve connectivity to obtain data, and
performance goals and measures. DHS generally concurred with GAO‘s
findings and recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-05-356.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cathleen A. Berrick (202)-
512-3404 or berrickc@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Development and Testing of Secure Flight Are Under Way, but Key
Activities Have Not Yet Been Completed:
TSA Is Taking Steps to Improve the Ability of Secure Flight to Identify
Passengers Who Should Undergo Additional Security Scrutiny, but System
Effectiveness Has Not Been Determined:
DHS and TSA Have Taken Actions to Strengthen Their Oversight and
Management of Secure Flight, but Key Issues Will Need to Be Resolved as
the Program Is Further Developed:
TSA Has Taken Steps to Minimize Impacts on Passengers and Protect
Passenger Rights, but Its Operational Plans Must Be More Fully Defined
before Protections and Impacts Can Be Accurately Assessed:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Homeland Security:
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
GAO Related Products:
Tables:
Table 1: Summary of TSA's Status in Addressing Ten Areas of
Congressional Interest Included in Public Law 108-334 as of March 15,
2005:
Table 2: System Capabilities Planned for CAPPS II:
Table 3: Key Capabilities for Passenger Prescreening Programs:
Table 4: TSA's Schedule for Final Phases of Secure Flight Testing:
Table 5: Cross-references of Legislatively Mandated Issues to Be
Reviewed by GAO with the Sections in this Report:
Figures:
Figure 1: Planned Operations of Secure Flight:
Figure 2: TSA Projected Key Milestones for the Development and
Implementation of Secure Flight, as of March 2005:
Figure 3: Slippage in Key Secure Flight Milestones between September
2004 and February 2005:
Figure 4: TSA's Completed, Current, and Future Planned Testing and
Operations for Secure Flight:
Abbreviations:
CAPPS I: Computer-Assisted Passenger Prescreening System I:
CAPPS II: Computer-Assisted Passenger Prescreening System II:
CBP: U.S. Customs and Border Protection:
DHS: Department of Homeland Security:
OMB: Office of Management and Budget:
PNR: Passenger name record:
TSA: Transportation Security Administration:
TSC: Terrorist Screening Center:
United States Government Accountability Office:
Washington, DC 20548:
March 28, 2005:
Congressional Committees:
Strengthening the security of commercial aviation has been a goal--and
a challenge--for many years, but since the September 11, 2001,
terrorist attacks, it has become a much more critical issue. The
attacks demonstrated that the consequences of inadequate security can
be more severe and tragic than previously imagined. Moreover, the
attacks showed that terrorists are targeting commercial aviation within
the nation's borders, and that measures taken to provide security were
not always effective. Consequently, since that time, the federal
government has initiated a number of efforts designed to strengthen the
security of virtually all aspects of commercial aviation.
Efforts to strengthen aviation security cover many areas, including
improved controls over screening passengers and baggage, and securing
restricted airport areas and airport perimeters. A recent initiative to
strengthen security is in the area of passenger prescreening. The
prescreening of passengers--that is, identifying passengers that pose a
security risk before they reach the passenger screening checkpoint--can
enable officials to focus security efforts on those passengers
representing the greatest potential threat. Since the late 1990s,
passenger prescreening has been conducted using the Computer-Assisted
Passenger Prescreening System (CAPPS I)--in which data related to a
passenger's reservation and travel itinerary are compared against
characteristics used to select passengers who require additional
security scrutiny, known as CAPPS I rules--and through the matching of
passenger names to terrorist watch lists. However, following the events
of September 11, it became clear that the capabilities of the existing
prescreening system to identify possible terrorists needed improvement.
Consequently, in November 2001, Congress passed the Aviation and
Transportation Security Act, which established the Transportation
Security Administration (TSA) and directed that it assume most of the
responsibilities for civil aviation security.[Footnote 1] In accordance
with the act's requirement that a computer-assisted passenger
prescreening system be used to evaluate all passengers, TSA
subsequently began an effort to develop a new prescreening system known
as CAPPS II that, unlike the current system that operates as part of
each airline's reservation system, would be operated by TSA. Further,
in July 2004, the National Commission on Terrorists Attacks upon the
United States, also known as the 9/11 Commission, reported that the
current passenger prescreening system needed improvements, and that the
watch lists used by the air carriers did not include all terrorists or
terrorism suspects because of concerns about sharing intelligence
information with private firms and foreign countries. The commission
recommended that passenger screening be performed by the federal
government, and make use of the larger consolidated watch list database
maintained by the government.[Footnote 2]
Because of a variety of delays and challenges, in August 2004, the
Department of Homeland Security (DHS) cancelled the development of
CAPPS II. In its place, TSA announced that it would develop a new
prescreening program, called Secure Flight, that would respond to the
commission's recommendation by taking over the responsibility--from air
carriers--for prescreening passengers, using the larger consolidated
watch list database not currently available to air carriers. In
developing Secure Flight, TSA plans to incorporate some but not all of
the functionality planned for the CAPPS II program. Specifically,
Secure Flight is being developed to compare passenger information
against data from the consolidated watch list database. TSA is also
considering incorporating CAPPS I rules processing as part of Secure
Flight, and may include the use of commercial data (e.g., personally
identifiable information that either identifies an individual or is
directly attributed to an individual, such as name, address, and phone
number) if the data can be shown, through testing, to add to the
security benefits of Secure Flight.
Public Law 108-334, enacted in October 2004, mandated that we assess
and report on 10 aspects of the development and implementation of
Secure Flight.[Footnote 3] This report satisfies the requirements of
that mandate. Specifically, this report addresses the following
questions: (1) What is the status of Secure Flight's development and
implementation? (2) What factors could influence the effectiveness of
Secure Flight? (3) What procedures have been put in place to oversee
and manage the Secure Flight program, including ensuring stakeholder
coordination? And (4) What efforts are being taken to minimize the
impacts on passengers and protect passenger rights? In answering these
questions, we addressed the 10 specific areas of congressional interest
that we were mandated to review based on the current status of Secure
Flight's development. These areas address the establishment of a
redress process, assessment of the accuracy of databases and the
effectiveness of Secure Flight, system stress testing, program
oversight, operational safeguards, security measures, oversight
policies governing the use and operation of the system, system privacy
protections, system modifications to accommodate states with unique air
transportation needs, and life-cycle cost estimates and expenditure
plans. (See app. I, table 5, for a description of the 10 areas
identified in Public Law 108-334 and the sections of the report in
which they are addressed.) Since some of the information addressing the
congressional areas of interest is considered Sensitive Security
Information, we are also issuing a separate letter containing this
information.[Footnote 4]
To address these questions, we reviewed available Secure Flight program
documentation to include system requirements, test plans, and privacy
notices. We also interviewed officials from DHS, TSA, U.S. Customs and
Border Protection (CBP), and the Terrorist Screening Center
(TSC)[Footnote 5] to discuss the status of the program's development as
of March 2005, as well as its anticipated operations. Since TSA
developed Secure Flight from a modified version of the CAPPS II
program, and will incorporate program criteria from CAPPS I, we also
reviewed relevant CAPPS II and CAPPS I program documentation. Further,
we questioned officials from selected air carriers and interviewed
personnel from several trade organizations and privacy advocacy
organizations regarding issues related to Secure Flight's development
and implementation. We conducted our work from April 2004 until March
2005 in accordance with generally accepted government auditing
standards. A detailed discussion of our scope and methodology is
contained in appendix I.
Results in Brief:
Overall, TSA is making progress in addressing key areas of
congressional interest related to the development and testing, system
effectiveness, program management and oversight, and privacy
protections for the Secure Flight program, as outlined in Public Law
108-334. Table 1 provides a summary of TSA's status in addressing each
of the ten areas of congressional interest. However, TSA has not yet
completed these efforts or fully addressed these areas, due largely to
the current stage of the program's development. Specifically, initial
tests have only recently been completed, and key policy decisions--
including what data will be collected and how it will be transmitted--
have not yet been made. Until requirements are fully defined, operating
policies are finalized, and testing is completed--scheduled for later
in the system's development--we cannot determine whether TSA will fully
address these areas of interest.
Table 1: Summary of TSA's Status in Addressing Ten Areas of
Congressional Interest Included in Public Law 108-334 as of March 15,
2005:
Areas of congressional interest (short title and page number in report
that further describes status): Stress test system and demonstrate
efficacy and accuracy (page 25);
TSA status in addressing area of congressional interest: Under way[A].
Areas of congressional interest (short title and page number in report
that further describes status): Assess accuracy of databases (page 27);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Modifications with respect to
intrastate travel to accommodate states with unique air transportation
needs (page 34, also see GAO-05-445SU);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Establish internal oversight board
(page 39);
TSA status in addressing area of congressional interest: Addressed[B].
Areas of congressional interest (short title and page number in report
that further describes status): Establish effective oversight of system
use and operation (page 43);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Install operational safeguards to
protect system from abuse (page 48);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Install security measures to protect
system from unauthorized access (page 48);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Life-cycle costsand expenditure plans
(page 50)c[C];
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Address all privacy concerns (page 54);
TSA status in addressing area of congressional interest: Under way.
Areas of congressional interest (short title and page number in report
that further describes status): Create redress process for passengers
to correct erroneous information (page 56);
TSA status in addressing area of congressional interest: Under way.
Source: GAO analysis.
[A] Under way indicates that TSA provided evidence that it has begun to
address this issue.
[B] Addressed indicates that TSA provided evidence that it has
addressed this issue.
[C] TSA officials stated that they plan to develop life-cycle cost
estimates after system requirements have been defined, and that they
recently finalized an expenditure plan.
[End of table]
TSA is making progress in the development and testing of Secure Flight
and is attempting to build in more rigorous processes than those used
for CAPPS II. Specifically, TSA has drafted a number of key documents
to assist in providing program oversight, including a draft concept of
operations, a draft requirements document, and a draft project
schedule. However, TSA has not yet finalized these documents. Further,
although TSA uses a working milestone chart to coordinate its many
activities, key milestones for the Secure Flight program have slipped.
For example, the date when Secure Flight is expected to achieve initial
operational capability with two air carriers slipped by about 4 months.
TSA is also completing initial Secure Flight testing to determine data
needs and system functions, which are basic to defining how Secure
Flight will operate. However, key system testing including stress
testing--to verify that the entire system will function as intended in
an operational environment--has not been completed. Further, although
TSA expects to complete stress testing prior to initial operational
deployment, scheduled for August 2005, it has not yet designed the
procedures it will use to conduct these tests. Until TSA finalizes key
program documents and completes additional system testing, it is
uncertain whether Secure Flight will perform as intended, and whether
it will be ready for initial operational deployment by August 2005.
TSA has begun, or has plans to initiate, a number of actions designed
to improve the ability of Secure Flight to identify passengers who
should undergo additional security scrutiny, in place of prescreening
currently conducted by air carriers. Specifically, TSA recently
completed initial testing to identify those elements that will be used
to match air carrier passenger data to data contained in the TSC's
terrorist screening database, and the effectiveness of these data in
making accurate matches. According to TSA officials, initial test
results showed that the Secure Flight system was effective in matching
PNR data with data contained in the terrorist screening database, and
that data matching can be improved by adding additional information to
PNR data, such as date of birth. However, because this testing has only
recently been completed and test results have not been fully documented
and analyzed, we were unable to independently assess these results. TSA
also plans to use intelligence analysts to help resolve discrepancies
in the matching of passenger data to data contained in the terrorist
screening database. In addition, TSA recently modified the CAPPS I
rules, which are currently being implemented and may also be used in
Secure Flight, to facilitate more targeted screening of individuals.
Although TSA is taking these actions, the effectiveness of Secure
Flight in identifying passengers who should undergo additional security
scrutiny has not been fully determined, and it can be affected by data
quality and other factors. For example, TSA has not resolved how
passenger data will be transmitted from air carriers to TSA to support
Secure Flight operations. Further, the ability of Secure Flight to make
accurate matches between passenger data and data contained in the
terrorist screening database is dependent on the type and quality of
the data. Although the TSC and TSA have taken, or plan to take, a
number of actions to improve the quality of the data in the terrorist
screening database, the accuracy of this data has not been fully
determined. Another factor that could impact the effectiveness of
Secure Flight in identifying known or suspected terrorists is the
system's ability to identify passengers who assume the identity of
another individual by committing identity theft.
DHS and TSA have also taken steps to strengthen their oversight and
management of Secure Flight, including coordinating with key
stakeholders. However, a number of important issues will need to be
resolved as program requirements are finalized and system testing is
completed, and before Secure Flight becomes operational. DHS and TSA
have provided oversight through a number of bodies designed to manage
Secure Flight's development and implementation. TSA also reported
strengthening its oversight of Secure Flight contractors through
various methods, including increasing the number of TSA staff with
contract oversight responsibilities. TSA officials also reached out to
key external stakeholders, such as air carriers, whom they identified
as integral to the successful implementation and operations of Secure
Flight. These efforts should help DHS and TSA in managing its
development and implementation efforts. Although DHS and TSA have taken
these actions, however, TSA has not yet finalized oversight policies
governing the use and operation of Secure Flight, or completed
performance measures to measure program results. Further, although TSA
has reached out to key external stakeholders who will be integral to
Secure Flight operations, officials from these organizations expressed
concerns regarding the uncertainty of Secure Flight system and data
requirements, and the impact these requirements may have on the airline
industry in terms of system modifications and costs. Data requirements
and associated impacts on air carriers will need to be resolved before
TSA can begin its initial operations with two air carriers in August
2005. TSA also has not finalized a security risk assessment and
security plan, due largely to the early stage of the system's
development. In addition, TSA did not develop life-cycle cost estimates
and only recently completed an expenditure plan. Life-cycle cost
estimates and expenditure plans are critical components of sound
program management for the development of any major investment. Without
fully developed plans addressing Secure Flight operations, security,
and costs, individuals responsible for overseeing the program may not
have the information needed to manage program risks and allocate
resources.
Additionally, TSA has recognized that Secure Flight has the inherent
potential to adversely affect the privacy rights of the traveling
public because of the use of passenger data, and has begun to take
steps to minimize potential impacts on passengers and to protect
passenger rights during the testing phase of Secure Flight. However,
TSA has not yet clearly defined the privacy impacts of Secure Flight in
an operational environment, or all of the actions TSA plans to take to
mitigate potential impacts. TSA also drafted a redress process to
provide passengers who believe they were inappropriately delayed from
boarding their scheduled flights because of Secure Flight a means by
which to appeal these decisions and possibly correct erroneous data
found in the terrorist screening database or in commercial databases,
should TSA decide to use commercially available data. However, TSA has
not yet clearly defined how it plans to implement its redress process
for Secure Flight, such as how errors, if identified, will be
corrected, particularly if commercial databases are used. In addition,
although DHS and TSA have taken steps to address international privacy
concerns in developing Secure Flight, such as limiting Secure Flight to
prescreening only domestic passengers, issues remain, particularly with
regard to the European Union. Specifically, TSA has acknowledged that
the use of passenger data that originates in reservations made in a
European Union country may create concerns under that country's privacy
laws. Until TSA fully defines its operational plans for Secure Flight-
-which officials stated they plan to do later in the system's
development--and addresses international privacy concerns, it will
remain difficult to determine whether the planned system will offer
reasonable privacy protections to passengers who are subject to
prescreening or mitigate potential impacts on passengers' privacy.
To help manage risks associated with Secure Flight's continued
development and implementation, and to assist TSA in developing a
framework from which to support its efforts in addressing congressional
areas of interest outlined in Public Law 108-334, we are making a
number of recommendations to the Secretary of the Department of
Homeland Security. These recommendations include finalizing
requirements and test plans, developing a plan for transmitting data
from and to air carriers to support Secure Flight operations,
developing performance goals and measures and life-cycle costs, and
finalizing policies and issuing associated documentation detailing
privacy protections and a system of redress.
We provided a draft of this report to DHS for its review and comment.
DHS, in its written comments, generally agreed with our findings and
recommendations, and identified some actions it has initiated to
implement the recommendations. For example, DHS stated that TSA plans
to complete the Secure Flight concept of operations by March 2005, and
system requirements by April 2005. DHS also noted that TSA is currently
finalizing a redress process for passengers who feel they have been
unfairly or incorrectly singled out for additional screening.
DHS also provided technical comments related to the program's
development, testing, and implementation. These comments were
incorporated as appropriate. A copy of DHS's comments is included in
appendix II.
Background:
The Transportation Security Administration is responsible for securing
all modes of transportation while facilitating commerce and ensuring
the freedom of movement for the traveling public. Passenger
prescreening is one program among many that TSA uses to secure the
aviation sector. The process of prescreening passengers--that is,
determining whether airline passengers pose a security risk before they
reach the passenger screening checkpoint--is used to focus security
efforts on those passengers representing the greatest potential threat.
Currently, U.S. air carriers conduct passenger prescreening using the
Computer-Assisted Passenger Prescreening System, known as CAPPS I, and
by comparing passenger names against government-supplied terrorist
watch lists.
Current Passenger Prescreening:
Passenger prescreening is used to identify passengers who may pose a
higher risk to aviation security than other passengers and therefore
should receive additional and more thorough security scrutiny. The
current prescreening process consists of two components. First, after a
passenger makes a reservation, the air carrier checks the passenger's
reservation information contained in the air carrier's passenger name
record (PNR)[Footnote 6] against a set of established system rules,
referred to as the CAPPS I rules.[Footnote 7] Second, the air carrier
checks the passenger's name against government-supplied watch lists
that contain the names of individuals who, for certain reasons, are
either not allowed to fly (the no-fly list) or pose a higher than
normal risk and therefore require additional security attention (the
selectee list). Passengers on the no-fly list are denied boarding
passes and are not permitted to fly unless cleared by law enforcement
officers. Passengers who are selected by the CAPPS I rules or who are
on the selectee list are issued boarding passes, and they and their
baggage undergo additional security measures. Approximately 99 percent
of all passengers on domestic flights are screened under the air
carrier-operated, automated CAPPS I system.[Footnote 8]
CAPPS II:
Following the events of September 11, and in accordance with the
requirement set forth in the Aviation and Transportation Security Act
that a computer-assisted passenger prescreening system be used to
evaluate all passengers before they board an aircraft,[Footnote 9] TSA
established the Office of National Risk Assessment to develop and
maintain a capability to prescreen passengers in an effort to protect
U.S. transportation systems and the public against potential
terrorists. In March 2003, this office began developing the second-
generation computer-assisted passenger prescreening system, known as
CAPPS II, to provide improvements over the current prescreening
process, and to screen all passengers flying into, out of, and within
the United States. Under the CAPPS II program, the responsibility and
financial costs of passenger prescreening were to be transferred from
the air carriers to the government. In addition, CAPPS II was to
perform different analyses and access more diverse data, including data
from government and commercial databases, to classify passengers
according to their level of risk (i.e., acceptable risk, unknown risk,
or unacceptable risk), which would in turn be used to determine the
level of security screening each passenger would receive. Table 2 lists
the specific capabilities that TSA planned to incorporate into CAPPS
II, which the agency believed were needed to strengthen passenger
prescreening.[Footnote 10]
Table 2: System Capabilities Planned for CAPPS II:
Capability: Watch list matching;
Description: Comparison of data contained in the passenger's
reservation (PNR) with information contained in government watch lists
(selectee and no-fly lists) to identify potential threats to aviation
security and other individuals of interest to the counterterrorism
community.
Capability: CAPPS I rules application;
Description: Matching information in the PNR to CAPPS I rules to
identify individuals who should be subject to additional security
screening.
Capability: Identity authentication;
Description: Checking PNR data against commercial databases to assist
in confirming the passenger's identity.
Capability: Criminal checks;
Description: Matching PNR data against lists of international fugitives
and government "wanted lists" to identify known criminals.
Capability: Intelligence-based search for unknown terrorists;
Description: Using algorithms developed through intelligence modeling
to identify previously unknown terrorists by searching for patterns in
an individual's travel or transaction history that are indicative of
terrorist activities.
Capability: Use of opt-in lists;
Description: Maintaining a list of individuals, who have been
previously cleared under credentialing programs, such as registering
passengers in advance of making reservations, to minimize the volume of
passengers that must be prescreened.
Capability: Use of alert lists;
Description: Providing the capability to create a temporary watch list
based on information extracted from current intelligence reports, such
as blocks of stolen passports.
Source: TSA.
[End of table]
In February 2004, we reported--in response to a mandate in the fiscal
year 2004 Department of Homeland Security Appropriation Act[Footnote
11]--that TSA had not yet developed critical elements associated with
sound project planning for CAPPS II, including a plan for the specific
functionality to be delivered and the costs expected to be incurred
throughout the system's development.[Footnote 12] We also reported that
TSA had not fully addressed seven of eight issues identified by
Congress as key areas of interest related to the development and
implementation of CAPPS II, such as privacy protection, passenger
redress, and system security. Following our evaluation and
congressional oversight hearings, DHS initiated an internal review of
the CAPPS II program.
Further, in July 2004, the National Commission on Terrorists Attacks
upon the United States, commonly known as the 9/11 Commission, reported
that the current air carrier-operated passenger prescreening system--
CAPPS I and watch list matching--needed improvements, and that the
watch lists used by the air carriers did not include all terrorists or
terrorism suspects because of concerns about the government sharing
intelligence information with private firms and foreign countries. The
commission recommended that passenger prescreening be performed by the
federal government and make use of the larger consolidated watch list
database maintained by the government.[Footnote 13] Taking into
consideration the commission's recommendations and the results of DHS's
internal review of CAPPS II, among other factors, TSA cancelled the
development of CAPPS II in August 2004.
Secure Flight:
Shortly after the CAPPS II program was cancelled, TSA announced that it
planned to develop a new passenger prescreening program called Secure
Flight. TSA plans to operate Secure Flight on the Transportation
Vetting Platform--the development of which began under CAPPS II and
includes the software for watch list matching and CAPPS I rules
analysis.[Footnote 14] According to TSA, Secure Flight will leverage
the system development efforts already accomplished for CAPPS II, but
will have several fundamental differences. Specifically, TSA is
designing Secure Flight to incorporate only some of the capabilities
planned for CAPPS II such as the core capabilities of watch list
matching and CAPPS I rules application.[Footnote 15] Secure Flight will
also only prescreen passengers flying domestically within the United
States, rather than passengers flying into and out of the United
States. Table 3 provides a summary of the capabilities planned for
CAPPS II, as compared with the capabilities currently provided by the
current passenger prescreening program and those planned for the Secure
Flight program. As shown in table 3, TSA does not plan to add
additional features beyond the current passenger prescreening program,
with the exception of matching PNR data against an expanded terrorist
watch list, which will be provided by the TSC. TSA is also exploring
the feasibility of using commercial data as part of Secure Flight if
the data are shown, through testing, to increase the effectiveness of
the watch list matching feature. TSA does not currently plan for Secure
Flight to include checking for criminals, performing intelligence-based
searches, or using alert lists.[Footnote 16] TSA has not yet determined
whether Secure Flight will assume the application of CAPPS I rules from
the air carriers, or if an opt-in list capability will be used as part
of Secure Flight.[Footnote 17]
Table 3: Key Capabilities for Passenger Prescreening Programs:
Capability: Watch list matching;
Capability included in program: Current prescreening program; CAPPS II;
Secure Flight[A].
Capability: CAPPS I rules application;
Capability included in program: Current prescreening program; CAPPS II;
Secure Flight: To be determined[B].
Capability: Identity authentication;
Capability included in program: CAPPS II; Secure Flight: To be
determined[C].
Capability: Criminal checks;
Capability included in program: CAPPS II.
Capability: Intelligence-based search for unknown terrorists;
Capability included in program: CAPPS II.
Capability: Use of opt-in lists;
Capability included in program: CAPPS II; Secure Flight: To be
determined[D].
Capability: Use of alert lists;
Capability included in program: CAPPS II.
Source: GAO analysis of TSA information.
[A] Secure Flight will use an expanded watch list that includes more
information than the current no-fly and selectee lists used by the air
carriers.
[B] TSA has not yet determined whether air carriers will retain
responsibility for applying the CAPPS I rules or whether this function
will be preformed by TSA.
[C] TSA plans to make a decision on the use of commercial data for
Secure Flight based on the results of current testing.
[D] TSA plans to examine whether Secure Flight will use an opt-in list,
which could include those passengers participating in TSA's Registered
Traveler program.
[End of table]
Secure Flight is currently undergoing development and testing, and
policy decisions regarding the operations of the program have not been
finalized.[Footnote 18] However, TSA officials have described how they
anticipate Secure Flight to operate, as illustrated in figure 1. When a
passenger makes flight arrangements, the air carrier or reservation
company will complete the reservation by entering PNR data in its
reservation system, as is done currently. Once the reservation is
completed, the PNR will be electronically stored by the air carriers.
Approximately 72 hours prior to the flight, the PNR will be sent to
Secure Flight through a network connection provided by DHS's CBP.
Reservations that are made less than 72 hours prior to flight time will
be sent immediately to TSA. Upon receipt of the PNR, TSA plans to
process the PNR data through the Transportation Vetting Platform.
During this process, Secure Flight will determine if the data contained
in the PNR match the data in the TSC terrorist screening database and
potentially analyze the passenger's PNR data against the CAPPS I rules,
should TSA decide to assume this responsibility from the air carriers.
As noted earlier, TSA has not yet determined whether CAPPS I rules
processing will be performed by TSA or by the air carriers. In order to
match PNR data to information contained in the terrorist screening
database, TSC plans to provide TSA with a subset of the database for
use in Secure Flight, and provide updates as they occur. All
individuals listed in the TSC data subset are to be classified as
either selectees (will be required to undergo secondary screening
before being permitted to board an aircraft) or no-flys (will be denied
boarding unless they are cleared by law enforcement personnel). When
Secure Flight completes its analysis, each passenger will be assigned
one of three screening categories: normal screening required (no match
against the terrorist screening database or CAPPS I rules), selectee (a
match against the selectee list or the CAPPS I rules, or random
selection), or no-fly (a match against the no-fly list). The results
will be stored within the Secure Flight system until 24 hours prior to
departure, at which time they will be returned to the air carriers.
Figure 1: Planned Operations of Secure Flight:
[See PDF for image]
[End of figure]
As shown in figure 1, when the passenger checks in for the flight at
the airport, the passenger will receive a level of screening based on
his or her designated category. A "normal screening" passenger will be
provided a boarding pass and allowed to proceed to the screening
checkpoint in the normal manner. A "selectee" passenger will receive a
boarding pass but will undergo additional security scrutiny at the
screening checkpoint. A "no-fly" passenger will not be issued a
boarding pass. Instead, appropriate law enforcement agencies will be
notified. Law enforcement officials will determine whether the
individual will be allowed to proceed through the screening checkpoint
or if other actions are warranted, such as additional questioning of
the passenger or taking the passenger into custody. TSA expects that
all information specific to a PNR record will be purged from the Secure
Flight temporary storage database 72 hours after completion of the
itinerary, unless a redress action is initiated by the passenger. TSA
plans to use the redress process to provide passengers who believe they
were inappropriately delayed from boarding their scheduled flights
because of Secure Flight a means by which to appeal these decisions.
After the completion of testing, TSA plans to make policy decisions
regarding the scope and operation of Secure Flight, including the
required PNR data to be obtained from air carriers and whether Secure
Flight will use commercial data to enhance the watch list matching
capability. TSA expects to begin initial operations of Secure Flight
with two U.S. air carriers in August 2005 and systematically bring
other U.S. air carriers online with Secure Flight in 2006. TSA
estimates that Secure Flight will prescreen about 2 million domestic
passengers per day when fully operational with all domestic air
carriers. For fiscal year 2005, TSA was allocated $35 million for the
development of Secure Flight. The President's fiscal year 2006 budget
request includes approximately $81 million for Secure Flight
development and implementation.
To consolidate and strengthen TSA's screening capability, in November
2004, DHS combined the Office of National Risk Assessment--which
developed CAPPS II--with the Credentialing Program Office to become the
Office of Transportation Vetting and Credentialing.[Footnote 19] By
merging these two offices, TSA expects to help provide assurance that
Secure Flight and the various credentialing programs within DHS and
TSA, which operate on the Transportation Vetting Platform, will be
executed effectively. In addition, in an attempt to achieve greater
synergy and avoid duplication of effort, DHS has proposed in its fiscal
year 2006 budget request to create an Office of Screening Coordination
and Operations within DHS's Border and Transportation Security
Directorate. The purpose of this office will be to coordinate a
comprehensive approach to several ongoing terrorist-related screening
initiatives--in immigration, law enforcement, intelligence,
counterintelligence, and protection of the border, transportation
systems, and critical infrastructure. If implemented, this office would
absorb Secure Flight as well as additional DHS and TSA screening
programs, including programs operating on the Transportation Vetting
Platform.
Development and Testing of Secure Flight Are Under Way, but Key
Activities Have Not Yet Been Completed:
TSA is making progress in the development and testing of Secure Flight
and is attempting to build in more rigorous processes than those used
for CAPPS II. To accomplish these efforts, TSA has developed a draft
concept of operations, a draft systems requirement document, and a
draft project schedule to guide its activities. However, TSA has not
yet finalized these documents. Further, although TSA is taking actions
to more effectively manage the Secure Flight system's development, key
milestones have slipped, including the date when Secure Flight is
expected to begin initial operations with two air carriers, by about 4
months. TSA has acknowledged that meeting its Secure Flight schedule
constitutes an area of risk.
Currently, TSA is completing testing to determine Secure Flight's data
needs and system functions, which are basic to defining how Secure
Flight will operate, and plans to complete important system testing
activities such as end-to-end performance and stress testing the entire
system.[Footnote 20] According to TSA officials, TSA plans to finalize
its concept of operations and system requirements prior to its final
phase of testing the entire system, which is scheduled to begin in
April 2005. Until TSA finalizes these documents and completes
additional system testing, it is uncertain how well Secure Flight will
perform or whether it will be ready for operational deployment by
August 2005.
TSA Recently Developed a Comprehensive Schedule, but Key System
Documentation and Development Activities Have Not Yet Been Completed:
TSA is continuing the development of the centralized platform
originally developed under CAPPS II--known as the Transportation
Vetting Platform--and the Secure Flight application to conduct its
prescreening activities. In continuing its development activities, TSA
has developed a draft concept of operations, a draft system
requirements document, and a project schedule to guide its efforts.
However, these documents have not yet been finalized. These documents
will need to be finalized in order to guide the system's development
and to proceed with the final phases of testing. The concept of
operations identifies to the eventual users of the system how the
system will operate, while a detailed set of requirements agreed on by
the government and the contractor helps ensure that Secure Flight is
built with the desired functionality.
TSA completed a draft concept of operations in February 2005. This
document provides a high-level perspective of how the system will
operate and includes the roles and responsibilities of key staff and
organizations. It also provides information necessary to begin
finalizing other documents, such as system requirements. However, the
concept of operations also identifies that many key decisions regarding
Secure Flight operations have not yet been made. For example, the
responsibilities between TSA's Office of Transportation Vetting and
Credentialing, which is responsible for developing and implementing
Secure Flight, and CBP, which TSA expects will provide the connectivity
and data transport services to and from the airlines for Secure Flight,
have not yet been determined. Further, TSA has not defined how the air
carriers or airline reservations systems will interface with CBP. TSA
acknowledges that not being able to obtain personally identifiable
passenger data found in PNRs from the air carriers because of costs to
the industry and lack of funding is an area of risk. TSA also
recognized that it has to make these and other policy decisions before
finalizing the concept of operations. However, TSA does not plan to
finalize these documents until after completing the testing that is
currently being conducted to determine Secure Flight's data needs and
functions. According to TSA's schedule, the final concept of operations
and the definition of requirements are expected to be completed in
March 2005 and April 2005, respectively. The sooner these key documents
are completed, the greater the chance TSA has of developing a system
that meets its needs. With Secure Flight currently scheduled to
prescreen its first passenger in August 2005, the lack of these key
documents in final form increases the risk that TSA will develop a
system that will not function as intended or meet TSA's needs.
In addition to the concept of operations and the system requirements
documents, TSA uses a working milestone chart and a draft project
schedule to guide its system development and testing activities. In
February 2004, we reported that CAPPS II development was behind
schedule and critical plans were incomplete. Specifically, TSA was
behind schedule in testing and developing initial increments of the
system, and had not yet established a complete plan to identify
specific system functionality that would be delivered. We reported that
TSA increased the risk of CAPPS II not providing expected functionality
and of its deployment being delayed. TSA officials recognized that they
had not fully developed CAPPS II with the thorough processes needed to
properly develop a system. As a result, TSA officials stated that they
are now attempting to build greater rigor into the Secure Flight
development approach. During the transition from CAPPS II to Secure
Flight, TSA modified its acquisition strategy and plan, obtained new
contractors to develop and test Secure Flight, used another contractor
to help develop key system documents and schedules, and hired more
government personnel with knowledge and experience in project
management. These steps have helped improve TSA's approach for the
development of the Secure Flight system. For example, after announcing
the start of Secure Flight in August 2004, TSA developed an initial
working milestone chart in September 2004, and a more detailed draft
integrated project schedule with milestones for developing, testing,
and securing the system in November 2004. These documents provide
information needed for program oversight officials, managers, and
stakeholders to understand the projected and revised time frames for
carrying out key activities. Figure 2 identifies TSA's projected key
program milestones as of March 2005.
Figure 2: TSA Projected Key Milestones for the Development and
Implementation of Secure Flight, as of March 2005:
[See PDF for image]
[End of figure]
Although TSA developed working milestones, TSA has revised its working
milestone chart several times, as figure 3 illustrates. During the 5-
month period between September 2004, when Secure Flight began, and
February 2005, when the project plan was most recently revised, TSA
delayed key milestones by up to 5 months. For example, TSA delayed the
date Secure Flight is ready to begin prescreening passengers during
initial operations, using two air carriers, from April 2005 to August
2005--a 4-month delay. According to TSA officials, they delayed initial
operations and other key milestones since the Secure Flight program
began because of a number of factors. For example, TSA officials stated
they received more than 500 comments on the Secure Flight privacy
notices, which caused delays in meeting key milestones. TSA officials
identified that not meeting the Secure Flight schedule is a key risk
that they plan to mitigate by assessing the program's progress against
information technology program management standards and implementing
tools to facilitate program execution, monitoring, and documentation.
Figure 3: Slippage in Key Secure Flight Milestones between September
2004 and February 2005:
[See PDF for image]
[End of figure]
TSA Is Conducting Initial Testing, but Key System Testing Remains:
TSA acknowledges the importance of testing the Secure Flight system to
refine system requirements and help ensure desired functionality is
achieved. TSA conducted some testing under the CAPPS II program that
will benefit Secure Flight, and is currently completing additional
testing to determine the information that will be needed in the
passenger record to match PNR data against the TSC terrorist screening
database and the CAPPS I rules, and plans to fully test the entire
system before it becomes operational. TSA plans to conduct this system
testing after key decisions are made about Secure Flight's functions,
such as what passenger data will be used, which will be based in part
on the results of current testing. Figure 4 summarizes TSA's completed,
current, and future testing and operations for the Secure Flight system.
Figure 4: TSA's Completed, Current, and Future Planned Testing and
Operations for Secure Flight:
[See PDF for image]
[End of figure]
The testing phase of a system development project is used to help
ensure that system functions meet their specified requirements.
According to leading information technology organizations, to be
effective, practices for testing software--such as that to be used in
Secure Flight--should be planned and conducted in a structured and
disciplined approach. Typically, this involves testing increasingly
larger increments of a system until the complete system and all of its
functionality are tested and accepted, and resolving critical problems
before moving to the next phase of testing. It also involves stress
testing and fully demonstrating the effectiveness and accuracy of the
system. TSA's recently drafted Test and Evaluation Master Plan provides
a high-level description of Secure Flight's overall test program and
identifies TSA's plans to conduct the required tests. TSA also prepared
detailed test plans for its current testing and will need to develop
additional plans before beginning its future system tests, scheduled to
begin in April 2005.
TSA Completed Initial Testing on CAPPS II System That Will Support
Secure Flight Testing:
Since April 2004, TSA has completed several tests on the CAPPS II and
Secure Flight systems. In March and April 2004, TSA tested several
components of the CAPPS II system including matching names against a
basic watch list and applying the CAPPS I rules. To conduct these
tests, TSA used simulated passenger data based on personal information
volunteered by 32 government and contractor personnel who had
originally worked on the CAPPS II program. When CAPPS II ended, several
features had not yet been tested, including system effectiveness,
security, privacy controls, system availability, backup and recovery,
and system monitoring.
In November 2004, during the transition from CAPPS II to Secure Flight,
TSA conducted several tests to verify that the system features brought
forward from CAPPS II functioned as intended after modifications had
been made for Secure Flight.[Footnote 21] TSA used the same simulated
passenger test data for these tests that it had used in April 2004. At
the conclusion of these tests, according to TSA officials, they found
that the watch list matching and CAPPS I rules application worked
sufficiently well enough to move forward with the current testing phase
of Secure Flight. However, our analysis shows that TSA tested only 28
percent of the system's requirements. According to TSA officials, they
only tested the system requirements that were necessary to support
initial performance testing. Officials further stated that they plan to
test all Secure Flight requirements as part of the final phase of
system testing beginning in April 2005.
TSA Currently Conducting Tests to Further Define Secure Flight Data
Needs and Functionality:
TSA is currently testing Secure Flight to determine (1) what data will
be needed in the PNR for the system to most effectively match PNR data
with data contained in the terrorist screening database and (2) whether
commercial data (personal data, such as name, address, and phone
number, maintained by private companies) can enhance the ability of
Secure Flight to match PNR data with data contained in the terrorist
screening database. To accomplish these tests--referred to as the PNR
tests and commercial data concept tests, respectively--TSA obtained
historical PNRs from domestic air carriers for passengers who flew
flight segments beginning and completed during the month of June
2004.[Footnote 22] TSA officials expect the results of these PNR and
commercial data tests to allow them to make informed policy decisions
regarding what passenger data will be required for Secure Flight
operations. According to TSA officials, after these tests are
completed, TSA plans to use the test results to help finalize the
concept of operations and system requirements. For example, according
to TSA officials, these tests could show that TSA may need air carriers
to collect date of birth information, which is currently not collected
by air carriers when taking reservations and could therefore delay
system deployment, or TSA may need to pay for commercial data, which
could increase system operating costs.
PNR testing: TSA recently completed testing that compares the various
combinations of passenger-provided information contained in air carrier
reservation systems,[Footnote 23] known as PNR data, against data
contained in the terrorist screening database, in order to identify
individuals known or reasonably suspected to be engaged in terrorism.
TSA developed test cases to help determine how effective Secure Flight
is in identifying individuals who were incorrectly identified as being
listed in the terrorist screening database (referred to as false
positives), or individuals not identified as being on a terrorist watch
list when in fact they should have been identified (referred to as
false negatives). Preliminary test results of matching data in the
terrorist screening database against various combinations of PNR data
showed that watch list matching is possible;
however, there are challenges in obtaining the data in a format that
the system can use. Further, although TSA attempted to test the
application of CAPPS I rules, the data provided by the air carriers
were insufficient to test the CAPPS I rules as part of the Secure
Flight program since not all of the data air carriers' require to run
CAPPS I are contained in PNRs. We discuss these points in further
detail later in this report.
Commercial data concept testing: TSA is currently conducting a concept
test,[Footnote 24] using commercial data to enhance or augment the June
2004 historical PNR data, to determine if the inclusion of additional
information in the PNR can improve the matching of passenger-provided
information against the terrorist screening database by reducing false
positives and false negatives. The commercial data concept test is also
intended to determine if the accuracy of passenger-provided data can be
verified using commercial data. To determine the effectiveness of using
commercial data, TSA developed initial measures for commercial data
concept testing, such as the overall percentage of passenger-provided
records from which identity can be verified using commercial data, and
plans to refine the measures throughout the testing process.[Footnote
25] TSA awarded a contract to conduct commercial data concept testing
in February 2005, and expects to obtain the test results in April 2005.
When these tests are completed, DHS and TSA plan to make policy
decisions regarding the data elements that should be included in the
PNR and whether commercial data will be used in support of the Secure
Flight program. These critical decisions could lead to changes in
system requirements.
Area of Congressional Interest:
TSA Plans to Conduct Stress Testing as Part of Final System Testing:
Stress Testing:
Beginning in June 2005, TSA plans to conduct a series of tests
consisting of increasingly larger increments of the system's
functionality until the complete system is tested. These tests are
designed to demonstrate the efficiency and accuracy of the entire
system, including 100 percent of the requirements. This testing will
include external interfaces for two-way data exchange between the air
carriers and TSA, and also for obtaining data from the TSC. These tests
will also include stress testing. Secure Flight has a stringent
performance requirement to process 2.5 million transactions per day,
with a peak load of 180,000 transactions within 10 minutes. During the
PNR testing, TSA conducted limited stress tests of the system by
running 1.8 million matching requests within 24 hours. TSA did not test
the number of matches against its more stringent requirement of
completing 180,000 matches within 10 minutes. Further, these results
are based on testing that did not involve the entire system, including
connectivity to air carriers and the possible application of CAPPS I
rules. Although TSA conducted the limited stress testing, it is
planning to conduct system stress tests that are designed to help
ensure that Secure Flight can operate efficiently, accurately, and
during peak load, and will use test results to determine system
readiness to operate live with two carriers by August 2005. Table 4
identifies TSA's planned milestones for its final phases of system
testing.
Table 4: TSA's Schedule for Final Phases of Secure Flight Testing:
Testing activity: Unit testing;
Purpose: To verify that the smallest defined module of the system works
as intended before integrating with other modules;
Begin: April 20, 2005;
End: May 31, 2005.
Testing activity: Integration testing;
Purpose: To verify that units of the system, when combined, work
together as intended;
Begin: June 1, 2005;
End: June 9, 2005.
Testing activity: System testing;
Purpose: To verify that the complete system (all the units combined)
satisfies specific requirements such as functionality, performance, and
security;
Begin: June 9, 2005;
End: June 23, 2005.
Testing activity: End-to-end testing;
Purpose: To verify that the entire system, including any external
systems with which it interfaces, functions as intended in an
operational environment;
Begin: June 23, 2005;
End: July 15, 2005.
[End of table]
Source: GAO analysis of TSA data.
Although TSA has developed this testing schedule and has described its
overall strategy for conducting these tests, it has not yet developed
the detailed test plans needed for unit, integration, system, and end-
to-end testing, which are scheduled to begin in April 2005. TSA
officials stated that they have identified a time frame during end-to-
end testing when they plan to conduct performance and complete system
stress testing. However, officials stated that the specific test plans
cannot be finalized until TSA makes key decisions regarding the final
operational and functional requirements for Secure Flight. Until TSA
develops detailed and complete test plans and fully executes these
plans, it is unknown how well Secure Flight will perform and whether it
will be ready to be operational with two air carriers in August 2005.
TSA Is Taking Steps to Improve the Ability of Secure Flight to Identify
Passengers Who Should Undergo Additional Security Scrutiny, but System
Effectiveness Has Not Been Determined:
Area of Congressional Interest: Accuracy of Databases and Effectiveness
of Secure Flight:
TSA has begun, or has plans to initiate, a number of actions designed
to improve the ability of Secure Flight to identify passengers who
should undergo additional security scrutiny, relative to the
prescreening currently conducted by the air carriers. These actions are
in response to the 9/11 Commission's recommendation that the government
improve passenger prescreening by taking over, from the air carriers,
responsibility for prescreening passengers using an expanded set of
terrorist watch lists currently not available to air carriers. TSA
efforts to strengthen passenger prescreening include conducting initial
testing, prior to the further development and implementation of Secure
Flight, to identify the most effective combination of data elements in
PNR and the terrorist screening database to be matched. TSA also plans
to use intelligence analysts to help resolve discrepancies in the
matching of PNR data to data contained in the terrorist screening
database, and recently modified the CAPPS I rules to facilitate more
targeted screening of individuals.
Although TSA is taking these actions, the effectiveness of Secure
Flight in identifying passengers who should undergo additional security
scrutiny has not yet been determined, and can be affected by data
quality and other factors. Specifically, TSA officials reported that
recently completed testing identified an improvement in Secure Flight's
ability to match PNR data to data contained in the terrorist screening
database over watch list matching conducted by the air carriers.
However, key issues regarding how these data will be obtained and
transmitted have not yet been resolved. Further, as is the case with
the current airline-operated process of matching passenger names
against no-fly and selectee lists--which are extracted from the
terrorist screening database and provided by TSA--the ability of Secure
Flight to make accurate matches between PNR data and data contained in
the terrorist screening database is dependent on the type and quality
of data contained in the database as well as in PNRs. While TSC and TSA
have taken, or plan to take, a number of actions to improve the quality
of the data in the terrorist screening database, the accuracy of the
database has not been determined. The effectiveness of data matches
will also be dependent on the accuracy of commercial data used to
augment the matching, should TSA decide to use commercial data for
Secure Flight. However, the accuracy of commercial data is undetermined
because there are no industry standards for processes or requirements
to ensure accuracy. Further, although TSA recently modified CAPPS I
rules to result in more targeted screening, TSA has been unable to
determine the impact of these changes on the screening process, and may
not be able to obtain all of the information needed to apply the rules
from PNR data. Another factor that could impact the effectiveness of
Secure Flight in identifying known or suspected terrorists is the
system's ability to identify passengers who assume the identity of
another individual, known as identity theft.
Initial Secure Flight Test Results Show Improvements over Current
Passenger Prescreening, but Key Issues Regarding How Data Will Be
Obtained and Transmitted Have Not Yet Been Resolved:
TSA recently completed testing intended to help identify those data
elements in both PNR data and the terrorist screening database that
will be needed to make the most accurate matches, and to identify error
rates that occur with the various combinations of data elements being
matched. Specifically, TSA matched different combinations of data
elements from both PNR data and data contained in the terrorist
screening database, such as last name only, full name only, or full
name and date of birth. TSA is in the process of analyzing the results
of these tests to determine which data elements would be most effective
for successful matching once Secure Flight becomes operational. TSA
also identified estimated error rates in matching PNR data with data
contained in the terrorist screening database under the various
combinations of data matched. In the context of Secure Flight, errors
occur if an individual is incorrectly identified as being on a
terrorist watch list (referred to as a false positive) or if an
individual is not identified as being on a terrorist watch list when in
fact he or she should have been identified (referred to as a false
negative). According to TSA, these test results will be used to help
determine whether additional or different combinations of data are
needed to help reduce error rates. TSA will also use this data to
determine whether identified error rates are acceptable and whether
additional work will be required to reduce these rates.
Although initial PNR testing was only recently completed, and test
results have not been fully documented and analyzed, TSA officials
stated that these results show that Secure Flight will be more
effective in matching PNR data with data contained in the terrorist
screening database than matches currently conducted by the air
carriers. Specifically, TSA officials believe that the results showed
that Secure Flight will be capable of detecting names that are exact
matches as well as minor variations in names with information in the
terrorist screening database. TSA officials further stated that test
results indicate that adding date of birth to PNR data may further
reduce the number of false positives. However, according to TSA
officials, the affect of adding date of birth on false negative rates
was less clear. Because this testing has only recently been completed
and test results have not been fully compiled and analyzed by TSA, we
were unable to independently assess these results. Specifically, we did
not independently assess whether the results showed an improved
capability over the current air carrier process, or the basis from
which this measurement was made. TSA officials stated that they would
continue to review the recently completed test results before making
decisions regarding the data to be used in Secure Flight.
Although TSA believes, based on initial test results, that Secure
Flight can effectively match PNR data with data contained in the
terrorist screening database, key issues regarding how these data will
be obtained and transmitted have not yet been resolved. Specifically,
TSA officials have not yet determined what data elements they will
require to be collected in PNR data and what data elements will be
needed from the terrorist screening database to support Secure Flight
operations. Based on test results, TSA officials stated that requiring
airlines to collect full name and date of birth in PNR data will
ultimately increase the effectiveness of data matches. However, air
carriers are not currently required to collect full name and date of
birth information in PNR data. Requiring air carriers to collect this
information could require significant changes to their reservation
systems and could take time to implement. TSA plans to identify
required data elements that must be collected in PNRs in April 2005.
TSA also plans to identify data requirements from the terrorist
screening database, through a memorandum of understanding with the TSC,
expected to be finalized in May 2005.
Further, although TSA officials stated that CBP will provide
connectivity between the air carriers and Secure Flight, TSA has not
yet developed a plan identifying how connections will be made between
air carrier reservation systems and TSA to support Secure Flight
prescreening. Currently, international air carriers have a one-way
connection through the existing infrastructure that supports the
Advanced Passenger Information System, which allows them to send data
to CBP, but does not allow air carriers to receive data.[Footnote 26]
According to TSA officials, they are working with CBP to resolve how
air carriers could both send and receive data, as air carriers would
have to receive information from Secure Flight, after data matches have
occurred, to identify whether passengers will require additional
security attention. TSA will also need to resolve how data will be
transmitted between smaller airports and carriers that fly only
domestically and therefore do not currently have an established
connection through CBP. TSA officials stated that CBP's current
communications infrastructure would need minor enhancements in order to
support Secure Flight's initial operating capability with two air
carriers. However, officials from CBP stated that it is unclear whether
the current communications infrastructure used by the Advanced
Passenger Information System can handle the high volume of data that
would be required to be transmitted to support Secure Flight once it is
fully operational. According to TSA officials, they plan to resolve
these and additional issues with CBP during Secure Flight's initial
operations with two air carriers.
TSA identified the ability of the airline industry to provide TSA with
the PNR data needed to support Secure Flight operations as a key
program risk because of potential costs to the industry of changes to
their reservation and other systems that may be required. TSA also
noted that establishing a connection between the air carriers and TSA
to transmit data is a risk, and that potential requirements for
additional PNR data could result in boarding delays. TSA plans to
mitigate these risks by supporting the development of a funding
strategy to reduce and defray expenses to air carriers and other
transportation industries. However, TSA has not described how it plans
to do this. TSA also plans to coordinate the development of operating
policies and procedures with officials from CBP, TSC, select airline
industry officials, and industry technical working groups.
Efforts Are Being Taken to Improve the Quality of Data That Will
Support Secure Flight Operations, but the Accuracy of These Data Has
Not Been Determined:
In order to identify individuals known or suspected to be engaged in
terrorism, Secure Flight plans to compare PNR data with information
contained in the terrorist screening database, a database that is
government-owned and controlled by the TSC. The TSC is responsible for
maintaining the accuracy of the information contained in the terrorist
screening database.[Footnote 27] Although a senior TSC official stated
that the TSC considers the data in the terrorist screening database to
be accurate, the official stated that the underlying accuracy of the
data has not been fully determined, and that the TSC does not know with
certainty whether errors in the database may exist, such as incorrect
name or date of birth. According to TSC officials, the underlying
accuracy of the data is dependent upon a number of factors outside the
control of the TSC, such as the process used by nominating agencies to
assess the information and the reliability of sources.
While the complete accuracy of data contained in this database can
never be certain--given the varying quality of intelligence information
gathered, and changes in this information over time--TSC has
established processes to help ensure the quality of these data. For
example, in order to add an entry to the database, an agency must go
through a nomination process in which representatives from the
nominating agency review available information and make a determination
whether the person should be included in the database.[Footnote 28]
Another quality control mechanism to improve the accuracy of data,
according to the TSC official, involves the process of removing records
from the database. The TSC has the sole authority to remove records
from the database. Each time a record within the database is searched,
TSC is to reexamine the record to ensure that the information can be
substantiated. If the information cannot be substantiated, TSC can
remove the record from the database. According to the TSC official,
approximately 4,800 records have been removed from the database as of
December 16, 2004.[Footnote 29]
In order to match PNR data to information contained in the terrorist
screening database, TSC plans to provide TSA with daily copies of a
subset of the database for use in Secure Flight. All individuals listed
in the data subset are to be designated as either selectees (will be
required to undergo secondary screening before being permitted to board
an aircraft) or as no-flys (will be denied boarding unless they are
cleared by law enforcement personnel). TSA officials stated they would
not receive the entire terrorist screening database because certain
portions of the database do not contain basic elements required for
Secure Flight matching (e.g., full name). TSA officials further stated
that they do not plan to assess the accuracy of the data provided by
TSC prior to matching PNR data against data contained in the database
because assessing the accuracy of the data is the responsibility of TSC
and the nominating agencies. That is, officials stated that they will
not attempt to determine whether individuals listed in the database are
inappropriately identified as being associated with terrorism, and will
not attempt to determine if specific data contained in the database are
accurate, such as name spelling, date of birth, or passport number.
However, TSA officials stated that as a nominating agency for the
terrorist screening database, TSA works with TSC to increase the
quality of the entries nominated by TSA. TSA officials also noted that
accuracy of the data provided by TSC is also not assessed under the
current prescreening program operated by the air carriers.
TSA is also considering using commercial data to validate PNR data by
comparing these records against information contained in commercial
databases, or to augment incomplete passenger records, as PNR data are
matched against data in the terrorist screening database.[Footnote 30]
However, the accuracy of commercial data is uncertain, which could
limit the effectiveness of these data in helping to make accurate
matches of PNR data to data contained in the terrorist screening
database for Secure Flight. As we reported in February 2004, commercial
data providers use varied measures and criteria to assess accuracy, and
there are no industry standards for processes or requirements to ensure
accuracy. We also reported that even databases determined to have an
acceptable level of accuracy will still contain errors.[Footnote 31] As
part of commercial data testing that TSA began in February 2005, TSA
plans to review methods for assessing the types and quality of data
available from commercial sources, as well as the relative accuracy of
commercial data products.[Footnote 32] However, TSA has not yet decided
how the accuracy of these data will be determined, or what an
acceptable level of accuracy would be in terms of Secure Flight. If the
data in commercial databases are determined to have an unacceptable
level of accuracy to support Secure Flight operations, the usefulness
of commercial data in augmenting data contained in PNRs may be limited.
Although TSA does not plan to assess the accuracy of data contained in
the terrorist screening database, and recognizes that the accuracy of
commercial data is uncertain, TSA expects to improve the accuracy of
data used to support Secure Flight operations, over time, through the
development of a redress process to provide passengers, who believe
they were inappropriately delayed from boarding their scheduled flights
because of Secure Flight, a means by which to appeal these decisions.
Specifically, TSA expects that the redress process will help identify
inaccurate data contained in the terrorist screening database or
commercial databases, should TSA decide to use them, which in turn
could potentially be corrected. Under the proposed Secure Flight
redress process, TSA officials stated that TSC has agreed in concept to
investigate--if passengers seek redress because they believe they were
inappropriately targeted for additional security scrutiny by Secure
Flight--the reason a person was listed in the database, including
consulting with the originating agency and removing a person from the
database if appropriate. However, TSA has not determined how this
process is likely to work in practice, or worked out the agreements
needed with TSC on how the data will be corrected. TSA's ability to
correct data in commercial databases is also questionable. The Secure
Flight draft redress policy indicates that TSA will be responsible for
identifying errors in commercial databases, should TSA decide to use
them for Secure Flight, and will work with commercial data aggregators
(who maintain the commercial databases) to correct errors, should those
errors result in passengers being incorrectly selected for additional
screening. However, it could be difficult to correct errors found in
commercial databases because data aggregators purchase their data from
other sources and may not be obligated to correct the data. Moreover,
data aggregators may not be permitted to share the source of their
data. In order to be most effective, errors would need to be corrected
at the source. Without information on how these processes will be
implemented, it is too early to determine whether they will be
effective in improving the quality of data matches. TSA plans for a
Secure Flight redress process are discussed in greater detail later in
this report.
TSA plans to use intelligence analysts during the actual matching of
PNR data to data contained in the terrorist screening database to
increase the accuracy of data matches. Specifically, TSA plans to have
intelligence analysts staffed within TSA to identify false positives--
passengers inappropriately matched against data contained in the
terrorist screening database--as PNR data are matched against data in
the terrorist screening database, and resolve mistakes to the extent
possible before inconveniencing passengers. One of the goals of Secure
Flight testing is to determine the number of TSA intelligence analysts
that will be required to clear misidentified passengers. However, TSA
has not yet determined how the TSA intelligence analysts will consult
with TSC to obtain the information necessary to increase the accuracy
of data matches. Accordingly, the effectiveness of using intelligence
analysts to clear misidentified passengers during Secure Flight
operations is unclear.[Footnote 33]
Changes to CAPPS I Rules May Result in More Targeted Security
Screening, but Potential Benefits to Secure Flight Are Not Yet Known:
Area of Congressional Interest: Modifications with Respect to
Intrastate Travel to Accommodate States with Unique Air Transportation
Needs:
TSA recently modified the passenger screening criteria currently used
by the CAPPS I system, known as the CAPPS I rules, to facilitate more
targeted screening of individuals and to reduce the number of
passengers selected for additional security scrutiny--termed
selectees.[Footnote 34] As described earlier, passenger prescreening
will encompass the matching of PNR data to data contained in the
terrorist screening database and the application of CAPPS I rules. TSA
has attempted to conduct testing to determine the impact of CAPPS I
rules changes on estimated selectee rates for Secure Flight. However,
since air carriers' PNRs do not contain all of the data required to run
CAPPS I, the data provided by the air carriers were insufficient to
enable TSA to determine the impact of these changes on selectee rates.
Further, TSA has not yet determined whether it will assume the CAPPS I
rules application as part of the Secure Flight program or whether air
carriers will continue to apply CAPPS I rules. Should TSA decide to
incorporate the application of CAPPS I rules into Secure Flight, it
will need to resolve how the system will obtain the necessary data from
the air carriers, since some of the data needed for the operation of
CAPPS I are not currently contained in PNRs.
Currently, air carriers prescreen passengers using CAPPS I, which
identifies selectees by comparing passenger information found in the
PNR and other air carrier passenger data systems with a set of
characteristics, known as CAPPS I rules. CAPPS I is not specifically
intended to identify individuals known or suspected to be associated
with terrorism. However, TSA considers CAPPS I to be an effective risk
management tool by helping to identify the relatively small number of
passengers whose PNR data correlates closely with the behaviors of
terrorists.
TSA officials stated that recent changes in the airline industry have
produced disproportionably high selectee rates for certain air carriers
as a result of certain CAPPS I rules. To address this issue, TSA
officials stated that the agency's Aviation Operations group conducted
an analysis of the CAPPS I rules. As a result of this effort, TSA
officials reported that they have changed certain CAPPS I rules, which
they believe will reduce overall selectee rates. Although changes to
these CAPPS I rules were not specifically intended to respond to
concerns of any particular state or air carrier with regard to selectee
rates, TSA officials stated that the changes should reduce the overall
CAPPS I selectee rate thereby addressing some of the concerns of states
with unique air transportation needs and high selectee rates.
Although TSA does not have estimates for the selectee rates for any
particular state, TSA has estimated the variability of selectee rates
for different types of air carriers. While TSA estimates the overall
selectee rate for air carriers is 15 percent, more detailed TSA
estimates of selectee rates, such as rates for specific air carriers,
and potential affects of CAPPS I rules changes are Sensitive Security
Information and have been removed from this report. Accordingly, we are
issuing a separate letter summarizing this information in more
detail.[Footnote 35]
TSA officials expected that Secure Flight testing would allow TSA to
more accurately identify the effect of CAPPS I rule changes on the
selectee rate, to determine whether these changes will result in more
targeted and effective security screening and reduce selectee rates.
Specifically, TSA had planned to identify actual selectee rates by
comparing the June 2004 historical PNR data it obtained for testing
against the CAPPS I rules that were in effect during that month. Using
that selectee rate as a baseline, TSA planned to determine the selectee
rate using the modified CAPPS I rules to measure any changes. However,
TSA could not determine the effect of the CAPPS I rule changes on
selectee rates because PNR data that TSA obtained from the air carriers
for testing did not contain all of the information needed to run CAPPS
I rules, since some of the information needed was contained in other
air carrier databases.[Footnote 36] Without these data, the effect of
the CAPPS I rule changes in conducting more targeted screening cannot
be determined. Further, TSA has not yet determined whether it will
assume the CAPPS I rules application as part of the Secure Flight
program or whether air carriers will continue to apply CAPPS I rules.
Should TSA decide to incorporate the application of CAPPS I rules into
Secure Flight, it will need to resolve how the system will obtain the
necessary data from the air carriers, since not all of the data needed
are currently contained in PNRs.
False Identifying Information and Identity Theft Could Affect the
Security Benefits of Secure Flight:
Another factor that could affect how well Secure Flight identifies
known or suspected terrorists is the system's ability to identify
passengers who falsify their identifying information or who commit
identity theft. Falsifying identifying information involves passengers
attempting to hide their true identities by submitting fictitious
identifying information, such as false addresses, when purchasing
tickets. Identity theft would involve a passenger "stealing" another
person's identifying information, such as name and date of birth, and
then using that identifying information to create fraudulent documents
associated with the identity (such as a driver's license containing the
stolen identifiers with the thief's picture).[Footnote 37] As our
previous work has shown, identity theft is growing in this
country.[Footnote 38]
TSA officials recognize that checking passenger information contained
in PNRs against information contained in the terrorist screening
database, which will be the basis of Secure Flight operations, will not
identify those using a stolen identity. TSA officials further stated
that Secure Flight is not intended to address identity theft, but
rather is designed to take over the responsibility, from air carriers,
of matching passenger data against terrorist watch lists. The current
prescreening process of matching passenger names against no-fly and
selectee lists also does not address identity theft.
Although TSA acknowledged that Secure Flight cannot fully address the
creation of false identifying information or identity theft, officials
stated that the use of commercial data may help identify situations in
which a passenger submits fictitious information such as a false
address. TSA officials are examining whether the use of commercial data
could detect these instances because the data being provided by the
passenger would either not be validated or would be inconsistent with
the information maintained by the commercial data provider. However,
whether the use of commercial data will assist Secure Flight in
identifying fictitious information cannot be determined until
commercial data testing is complete. Further, using commercial data
would likely not be able to detect instances of identity theft
involving stolen identifying information of an individual. TSA is
conducting tests, using commercial data, to determine the extent to
which commercial data can address fictitious identities as well as
mitigate false positives and false negatives in the matching of
passenger PNR data to data contained in the terrorist screening
database. Based on the results of these tests, TSA plans to decide
whether to incorporate the use of commercial data as part of Secure
Flight.
TSA officials further stated that passenger information will continue
to be compared against CAPPS I rules, whether by the air carriers or by
TSA. While CAPPS I rules are not designed to address the creation of
false identifying information or identity theft, TSA believes the
application of CAPPS I rules--which are not dependent upon passenger
identity--can provide an additional security layer. In addition, the
CAPPS I process randomly identifies some airline passengers as
selectees--passengers who were not initially selected based on CAPPS I
rules--to ensure that no passenger is guaranteed selectee-free status.
TSA officials further stated that Secure Flight is just one layer in a
series of systems designed to strengthen aviation security, and that
passengers who were able to thwart Secure Flight by committing identity
theft would still need to go through normal checkpoint screening and
other standard security procedures.
TSA officials recognized that Secure Flight would best address identity
theft by implementing some type of biometric technology. As noted in
our previous work, the seven leading biometric technologies are facial
recognition, fingerprint recognition, hand geometry, iris recognition,
retina recognition, signature recognition, and speaker
recognition.[Footnote 39] According to TSA officials, incorporating
biometrics into the Secure Flight program is not currently envisioned.
However, TSA plans to expand the Registered Traveler program, which
uses biometrics to verify passenger identity. Although TSA has not
determined how Secure Flight and Registered Traveler will be
integrated, if at all, TSA officials stated that expanding the
Registered Traveler program could help alleviate the problem of
identity theft with respect to Secure Flight since passengers must
verify their identity with a biometric captured during program
enrollment and assessed every time they fly. The Registered Traveler
program is currently operating in the pilot phase at five airports.
According to TSA officials, approximately 10,000 people are
participating in the Registered Traveler pilot program.
DHS and TSA Have Taken Actions to Strengthen Their Oversight and
Management of Secure Flight, but Key Issues Will Need to Be Resolved as
the Program Is Further Developed:
DHS and TSA have taken a number of actions designed to strengthen their
oversight and management of Secure Flight. These efforts include
providing oversight through a number of boards and working groups
designed to manage the program's development and implementation. TSA
also strengthened its oversight of Secure Flight contractors through
various methods, including increasing the number of TSA staff with
contract oversight responsibilities and recently finalizing an
acquisition plan for Secure Flight and the Transportation Vetting
Platform. TSA officials further engaged in outreach to key external
stakeholders, to include air carriers, who they identified as integral
to the successful implementation and operations of the Secure Flight
program. These efforts should help DHS and TSA in managing their
development and implementation efforts and help ensure, as the
development of Secure Flight progresses, that key risks are identified
and managed.
Although DHS and TSA have taken action to strengthen their oversight
and management of Secure Flight, key issues will need to be resolved as
program requirements are finalized, system testing is completed, and
Secure Flight becomes operational. For example, TSA has not yet
developed oversight policies governing the use and operation of the
system, or finalized performance measures to measure program results.
Further, although TSA is working with key external stakeholders who
will be integral to Secure Flight operations, officials from some of
these organizations expressed concerns to us regarding the uncertainty
of Secure Flight system and data requirements, and the impact these
requirements may have on the airline industry. TSA also has not
finalized a security risk assessment and security plan, nor has it
developed life-cycle cost estimates and only recently finalized an
expenditure plan. TSA has recognized the importance of these plans and
estimates to the successful implementation of Secure Flight, and
because of uncertainties regarding program requirements--such as the
possible use of commercial data--TSA identified system security and
life-cycle costs as key program risks. Because plans addressing program
operations, security, and costs are not fully developed, and key issues
affecting the program--such as data requirements and connectivity to
air carriers--have not been resolved, it will be important for
established and planned oversight and management bodies to ensure that
key program risks are appropriately managed.
DHS Oversight Board and External Advisory Committee Are in Place to
Oversee the Development and Implementation of Secure Flight:
Area of Congressional Interest: Internal Oversight Board for Secure
Flight:
Oversight mechanisms operate through a number of boards and working
groups within DHS and TSA to oversee the development and implementation
of Secure Flight. Each of these groups has a distinct role, ranging
from overseeing the program at the executive level to providing TSA
with comments on actions and processes related to information
technology and privacy protection issues. These varying levels of
oversight can help provide assurance that Secure Flight development and
implementation issues are considered throughout the program's
development. However, as development continues and Secure Flight
becomes operational, it will be important that a consistent and
continuing level of oversight be provided to monitor the program's
progress and manage risks as system requirements and operations are
refined, and that issues identified by these oversight bodies are fully
addressed, given the state of Secure Flight's development.
Acquisition Oversight Is Provided by DHS:
DHS established an Investment Review Board to provide executive-level
review of department and agency acquisition activities. The Investment
Review Board consists of senior DHS executives and is chaired by the
Deputy Secretary. The board is tasked with reviewing all capital assets
with contracts exceeding $50 million, and all information technology
programs with expected life-cycle costs in excess of $200
million.[Footnote 40] The board's purpose in reviewing programs meeting
these thresholds during key phases of program development is to help
ensure that programs meet mission needs at expected levels of cost and
risk.[Footnote 41]
To date, the DHS Investment Review Board has reviewed the
Transportation Vetting Platform[Footnote 42] --from which Secure Flight
will operate--and Secure Flight one time, on January 27, 2005. As a
result of this review, the board withheld approval for the
Transportation Vetting Platform and Secure Flight to proceed into the
production and deployment phase until three issues were addressed.
These issues included requiring that a formal acquisition plan be
developed and approved for the platform by February 22, 2005;
developing a plan for integrating and coordinating the platform with
other DHS "people screening" programs; and resubmitting a revised
acquisition program baseline (cost, schedule, and performance
parameters). In response to these requirements, TSA officials stated
that they have revised the acquisition plan and the acquisition program
baseline, and participated in a cross-agency working group to develop a
plan for coordinating "people screening" programs within DHS. In doing
so, TSA officials stated they have met all the requirements of the DHS
Investment Review Board. However, TSA has not yet received approval
from the DHS Investment Review Board to proceed. The DHS Investment
Review Board further noted that additional concerns remained regarding
system privacy protections and data security, and because of the
platform's and Secure Flight's aggressive schedule, the risks of not
meeting cost, schedule, and performance goals remained. The DHS
Investment Review Board plans to meet again to review the
Transportation Vetting Platform and Secure Flight when commercial data
testing is complete, or no later than the spring of 2005. However, as
we previously reported, DHS officials stated that the Investment Review
Board was having difficulty reviewing all of the critical departmental
programs in a timely manner.[Footnote 43] Considering the risks
identified by the Investment Review Board, it will be important that it
continue to review the development and implementation of the
Transportation Vetting Platform and Secure Flight as these programs
move forward.
External Advisory Committee Designed to Provide Advice and Assistance
for Secure Flight:
In addition to the DHS Investment Review Board, the Aviation Security
Advisory Committee established a Secure Flight working group to provide
TSA with advice and assistance related to the development and
implementation of the program. The advisory committee, now within DHS,
is a standing committee created in 1989 in the wake of the explosion of
Pan Am Flight 103 over Lockerbie, Scotland. The advisory committee is
composed of federal and private sector organizations and was created to
provide advice on a variety of aviation security issues. The Secure
Flight Working Group, within the advisory committee, was formed in
September 2004 to provide the committee with comments on actions,
procedures, and processes related to the initial testing phase of
Secure Flight. The working group is chaired by the TSA Privacy Officer
and includes representatives from privacy advocacy groups, academia,
and information technology firms. The primary focus of the working
group is on privacy and information technology issues. Among other
things, the working group is designed to review the initial testing
phase of Secure Flight to provide advice on whether information used by
the program is adequately protected and secure, as well as review
Secure Flight redress and appeals procedures regarding their
timeliness, sufficiency, and ease of use. According to TSA officials,
the working group has met four times. Following the completion of
initial Secure Flight testing, scheduled for April 2005, the working
group plans to incorporate its findings into a report to be presented
to the advisory committee for its review and approval and to transmit
the report to TSA. A TSA official stated the agency is considering
continuing the working group beyond the Secure Flight initial testing
phase.
TSA Has Taken Steps to Strengthen Contractor Oversight and Acquisition
Planning, but Risks Remain:
Recognizing problems in providing contractor oversight during the
development of CAPPS II, TSA has reported strengthening its oversight
of Secure Flight contractors and acquisition planning. According to TSA
officials, the successful development and implementation of Secure
Flight is heavily dependent on contractor performance and TSA's
acquisition strategy. TSA's strategy involves reliance on contractors
to provide many of the developmental and testing services for Secure
Flight, while TSA's role is primarily to manage the program by
providing program support, oversight of contractor activities, and
technical expertise. TSA currently has two contractors dedicated to
Secure Flight testing--one for testing PNR data matching against the
TSC terrorist screening database, and one for testing the use of
commercial data. TSA also oversees other contractors dedicated to the
development and testing of the Transportation Vetting Platform.
According to TSA officials, governmental oversight of the CAPPS II
program was limited. Specifically, TSA acknowledged that the program
office responsible for developing CAPPS II was understaffed in terms of
government employees and relied heavily on contractors to work under
limited TSA oversight. As a result, TSA officials stated they did not
always have assurance that the contractor was meeting its expected
goals. Our previous work assessing TSA's overall acquisition management
capability found similar problems across the agency. In May 2004, we
reported that TSA had not developed an acquisition capability that
facilitated the successful management and execution of acquisition
activities.[Footnote 44] We also found that TSA's acquisition policies
and procedures had not been effectively communicated across the agency.
Since our review, TSA has taken steps intended to strengthen its
contract management and oversight efforts. TSA officials stated that
their contract oversight capability has been maturing in recent months,
and that the agency now uses improved tracking mechanisms to monitor
contractor schedule and cost information. TSA officials further stated
that since program managers lacked adequate staff to gather and
evaluate information needed for effective oversight, the agency uses
several support contractors to assist with these tasks.
In addition to the agency's overall efforts to improve contract
management, TSA officials also reported taking steps to strengthen
contractor oversight for Secure Flight. For example, the Secure Flight
program is using one of TSA's support contractors to help track the
progress of the contractors developing Secure Flight in the areas of
cost, schedule, and performance. Program officials stated they meet
with the support contractor on a weekly basis and obtain frequent
reports on the Secure Flight contractors' performance. TSA officials
also stated they have increased the number of TSA staff with oversight
responsibilities for Secure Flight contracts. Since TSA is relying on a
support contractor to provide direct oversight over other contractors
developing and testing Secure Flight, it will be important that TSA
maintain strong oversight.
TSA also recently developed an acquisition plan that presents the
acquisition strategy for the Secure Flight and the Transportation
Vetting Platform. Acquisition plans, which set forth the overall
strategy for managing a system's acquisition, are intended to help
ensure that the government meets its needs in a timely manner and at a
reasonable cost. Organizations within TSA are expected to use
acquisition planning as an opportunity to evaluate and review the
entire acquisition process so that sound judgments and decision making
can help facilitate program success. Although best practices show that
acquisition planning should begin as soon as the agency need is
identified, with reviews and updates as needed, TSA has only recently
finalized the acquisition plan for Secure Flight and the Transportation
Vetting Platform. TSA officials cited the organizational changes within
the Secure Flight program office as slowing their progress in
developing the plan.
Although TSA has taken steps to strengthen contract oversight and
acquisition planning, TSA has identified contract management as a key
risk facing the development and implementation of Secure Flight. To
mitigate this risk, TSA plans to develop communication mechanisms among
DHS acquisitions officials, Secure Flight contractors, and Secure
Flight program management officials. However, TSA has not yet defined
what these mechanisms are or how they are intended to work. TSA also
intends to use its acquisition plan to identify strategies for
improving contract management. Since the successful development and
implementation of Secure Flight is heavily dependent on contractor
performance and TSA's acquisition strategy, maintaining contractor
oversight and monitoring and updating its acquisition strategy can help
TSA ensure that intended results from contracts are achieved as Secure
Flight moves forward.
TSA Plans to Develop Oversight Policies and Performance Measures after
System Testing:
Area of Congressional Interest: Oversight of System Use and Operation:
TSA has not yet finalized oversight policies governing the use and
operation of Secure Flight or developed performance measures to assess
program performance once Secure Flight becomes operational. TSA plans
to use Secure Flight's initial testing results to make decisions
regarding system data requirements, including the effectiveness of
various combinations of PNR data in system operations, and whether the
use of commercial data would improve Secure Flight's ability to
correctly match PNR data with data contained in the terrorist screening
database. TSA officials stated that they plan to use these test results
to finalize the Secure Flight concept of operations, which will detail
how Secure Flight will operate and interface with other systems. Until
this concept of operations is finalized, oversight policies governing
the use and operation of the system will not be known. TSA expects to
finalize the concept of operations by March 2005.
TSA has also not yet established performance goals or measures to gauge
the success of the Secure Flight program once it is operational.
Performance goals and measures are intended to provide Congress and
agency management with information to be able to systematically assess
a program's strengths, weaknesses, and performance, and then identify
appropriate remedies. The Government Performance and Results Act
requires that agencies establish performance goals and performance
measures in order to report on program results.[Footnote 45] As defined
by the act, a performance goal is the target level of performance--
either output or outcome--expressed as a tangible, measurable
objective, against which actual achievement will be compared. Until
Secure Flight testing is complete and key policy decisions are made,
such as what data elements will be required in the PNR and whether
commercial data will be used, TSA will not be able to finalize
performance goals and measures for Secure Flight in an operational
environment. However, without performance goals and measures, it will
be difficult to determine whether Secure Flight is meeting its
objectives. TSA officials stated that while they recognize the need for
performance goals and measures for Secure Flight once it is
operational, they have not yet identified how or when they will be
developed. Until operating policies and performance goals and measures
are developed, it is unknown whether needed controls will be put in
place to guide and monitor Secure Flight operations.
Although TSA has not developed policies or performance measures for an
operational system, it has developed measures for PNR testing and
commercial data testing, to identify information on what data
combinations are most useful in prescreening passengers and to
determine the utility of using commercial data to support Secure Flight
operations. For example, TSA developed initial measures for commercial
data testing that it plans to refine throughout system testing, should
TSA decide to use commercial data. These measures are designed to help
determine the effectiveness of using commercial data, and to guide DHS
and TSA policy decisions regarding whether the data should be used for
the Secure Flight program. Although these measures, and measures
developed for PNR testing, were not designed to identify impacts on
aviation security in an operational environment, they should help
provide TSA a means by which to make informed policy decisions
regarding system requirements prior to finalizing its concept of
operations.
TSA Has Engaged in Outreach with Key External Stakeholders, but
Concerns Exist over Potential Impacts of Secure Flight Operational
Requirements:
TSA officials have engaged in outreach with key external stakeholders
whom they identified as integral to the successful implementation and
operations of Secure Flight. However, officials from many of these
organizations, primarily air carriers and privacy groups, expressed
concerns regarding the uncertainty of Secure Flight system and data
requirements, and the impact these requirements may have on the airline
industry and traveling public. Officials from a majority of air
carriers and privacy groups who answered our questions regarding the
implementation of Secure Flight, and who provided comments on the
amount of TSA coordination, were generally satisfied with the level of
outreach provided related to Secure Flight. However, officials from a
majority of the air carriers who provided written comments expressed
concern regarding the potential for costly and time-consuming changes
that may be required of their reservation systems because of additional
data requirements, and the uncertainties surrounding Secure Flight's
ability to establish a link for the transfer of data between the air
carriers and TSA.[Footnote 46] Privacy group officials also expressed
concerns regarding the integrity of data contained in the terrorist
screening database, and the potential lack of a redress process for
Secure Flight that would allow a system of recourse for passengers who
were misidentified during system screening. TSA officials stated that
they will not be able to finalize system requirements until after the
completion of initial Secure Flight testing. However, officials
identified potential adjustments to reservation systems, and the
establishment of a connection with air carriers, as program risks, and
are in the process of developing risk mitigation strategies.
TSA Has Discussed Secure Flight Development Activities with Key
External Stakeholders:
TSA has established relationships with numerous stakeholders--outside
of the federal government--that will be involved with, or affected by,
the Secure Flight program. These stakeholders include, but are not
limited to, air carriers; global reservation management companies;
aviation associations; and civil liberties, privacy, and policy
advocacy groups. TSA stated that the success of Secure Flight is
dependent on building trusted relationships with these stakeholders in
order to leverage needed cooperation between the public and the private
sector. For instance, TSA officials indicated that the ability of
Secure Flight to receive passenger PNR data from air carriers is
critical to the operation of the system and that in order to support
Secure Flight requirements, the airline industry may need to change its
data collection requirements for passengers when reservations are made.
TSA also recognized that the protection of passengers' identifiable
information is essential for Secure Flight to be successful, since the
government will be obtaining, from air carriers, these data in order to
conduct Secure Flight prescreening.
TSA focused its outreach efforts on air carriers and privacy groups in
an attempt to mitigate their concerns about Secure Flight and resolve
issues regarding the implementation and operations of the system.
According to TSA officials, they generally held two teleconferences a
week with officials from air carriers and privacy groups.[Footnote 47]
TSA officials stated that they selected these air carriers and privacy
groups based on each group's ability to inform the development of
Secure Flight. In addition, TSA provided air carriers with a dedicated
e-mail address to provide them a means by which to ask questions about,
and provide comments on, Secure Flight. TSA committed to responding to
all questions and comments within 3 days. During our review of TSA
outreach efforts, officials from a majority of privacy groups that we
interviewed, and air carriers who provided written comments on TSA's
level of outreach, stated that they were generally satisfied or pleased
with TSA's level of contact with them related to Secure Flight. In
addition, officials from 4 large air carriers stated that TSA's
outreach effort had improved from what it had been during the
development of CAPPS II. Officials from all three of the privacy groups
we interviewed also stated that TSA's outreach effort was a positive
change compared with the outreach provided during the development of
CAPPS II.
Air Carriers and Privacy Groups Are Concerned about System Connectivity
and Data Accuracy and Protections:
Although air carriers were generally satisfied with the level of
outreach provided by TSA, officials from 13 of the 14 air carriers who
answered questions on Secure Flight's implementation expressed concerns
about modifications that may be required of their reservation systems
and the lack of detailed information from TSA regarding Secure Flight
system requirements. Specifically, officials stated that they were
concerned about "unknown requirements" and the possibility of being
required to collect additional PNR data elements, such as date of
birth, when taking passenger flight reservations. According to these
officials, requiring the collection of additional PNR data from
passengers each time a reservation is made, such as date of birth,
would require that all reservation systems--including travel agency
systems, Internet engines, self-service kiosks at airports, airport
check-in counters, departure systems, and PNR storage databases--be
modified, which could place a significant strain on the industry. In
addition, officials from 6 of the 14 air carriers expressed various
concerns related to customer inconvenience, including concerns about
the collection of additional information at the check-in or departure
gate, potentially resulting in congested airports and delayed
departures and possibly creating an increased workload for airline
personnel. Officials further stated that passengers could face delays
by having to provide additional data when making reservations or during
the check-in process at the airport. Officials were unable to provide
estimates of potential costs of system changes or expected delays since
TSA has not yet defined what data elements Secure Flight will require
to conduct passenger prescreening. However, some officials--although
uncertain of what the Secure Flight system requirements will be--
estimated that it may require anywhere from 8 weeks to over 1 year to
make required changes to their reservation systems, depending on data
requirements.
Air carrier officials also expressed concern that TSA has not yet
developed a plan identifying how connections will be made between air
carrier reservation systems and TSA to support Secure Flight
prescreening. Officials from 11 of the 14 air carriers who provided
written comments expressed various concerns regarding connectivity,
including Secure Flight's ability to provide a two-way real-time
exchange of data to allow for the almost instantaneous prescreening of
passengers. Officials further stated that the maximum load capacity of
systems that may be used to transfer data between the air carriers and
TSA, such as the Advanced Passenger Information System, may not be
sufficient to handle the large amount of data that will need to be
regularly transferred. Air carrier officials also expressed concern
that the programming effort needed to establish a two-way connection
between their reservation systems and the Advanced Passenger
Information System, enabling carriers to both send and receive data
almost instantly, would be costly and time-consuming. As we noted
earlier, TSA will need to resolve these and additional issues with TSC,
which will provide data from the terrorist screening database, and CBP,
to receive PNR data, before these connections can be determined.
Although air carrier officials identified concerns related to unknown
system requirements, some officials stated that they believed Secure
Flight will provide improvements over the current prescreening process,
and may provide additional benefits to air carriers and passengers.
Specifically, officials from 5 of the 14 air carriers stated that they
expect to realize benefits, such as eliminating the air carriers'
responsibilities for operating CAPPS I and watch list matching and
transitioning the prescreening responsibility to the government. In
addition, officials from 2 of the 5 air carriers stated that Secure
Flight may result in a more consistent application of procedures. Three
officials further stated that transferring the prescreening
responsibility to the federal government will eliminate the need for
air carriers to maintain terrorist watch list data and to manually
process customers, which should result in a reduced workload and
operational savings to the air carriers. Officials further stated that
Secure Flight may minimize unnecessary delays for passengers who may
have been falsely matched against the selectee and no-fly lists, which
would have required them to undergo additional security screening.
Privacy group officials we contacted also expressed concern regarding
the potential impact of Secure Flight requirements once they are
defined, primarily the integrity of data contained in the terrorist
screening database and the lack of a Secure Flight redress policy.
Although officials from all three privacy groups we contacted
recognized that the quality of data contained in the terrorist
screening database was outside the control of TSA, they stressed the
importance of having established processes for adding individuals to,
and removing individuals from, the database to help ensure the accuracy
of the data. One official stated that inaccurate data in the terrorist
screening database could lead to an increase in the number of
individuals being misidentified as positive matches against a terrorist
watch list. Officials from all three groups also expressed concern over
the lack of a finalized redress process, which would provide passengers
who were misidentified as positive matches against data in the
terrorist screening database a means by which to correct erroneous
information. According to one official, a redress process should
incorporate access to information, the ability to challenge a decision,
and the identification of the information's source in order to correct
the information if necessary. As noted earlier, TSA is in the process
of addressing these concerns by establishing a memorandum of
understanding with TSC to help ensure the accuracy of data contained in
the terrorist screening database, and it is developing a redress policy.
TSA Has Initiated Information System Security Activities but Cannot
Complete All Key Actions until Secure Flight Is Further Developed:
Area of Congressional Interest: Operational Safeguards and Security
Measures:
TSA is planning to implement an information systems security management
program for Secure Flight, but key elements of this program have not
yet been completed, due in part to the status of Secure Flight's
development. Although TSA has taken steps to initiate a security risk
assessment and a security plan, other steps, such as certification and
accreditation, cannot occur until the system has been developed and
tested.
The Federal Information Security Management Act,[Footnote 48] Office of
Management and Budget (OMB) guidance,[Footnote 49] and industry best
practices describe critical elements of a comprehensive information
system security management program. These elements include conducting a
security risk assessment and developing a system security plan,
obtaining a security certification, and having an agency official
accredit the security of the system. Together, these elements can help
provide a strong security framework for protecting information and
assets. A comprehensive information system security management program
can, among other benefits, help ensure that information systems contain
safeguards to reduce opportunities for abuse and have substantial
security measures in place to protect against unauthorized access by
hackers or other intruders.
In part because Secure Flight has not yet been fully defined or
developed, TSA has not yet completed a security risk assessment and a
security plan. Risk assessments are essential steps in determining what
controls are required and what level of resources should be expended on
controls, while security plans provide an overview of the security
requirements of the system, describe established controls for meeting
those requirements, and delineate responsibilities and expected
behaviors for all individuals who access the system. TSA has drafted a
risk assessment for Secure Flight and the Transportation Vetting
Platform. TSA also developed a draft security plan that references the
high-level system controls needed for security, including management,
operational, and technical controls. However, greater detail regarding
the specific steps to be taken to secure the system will be needed
before the plan can be finalized. For example, the security plan should
include details about security controls associated not only with the
Secure Flight program but also its many interfaces and networks that
are to provide connectivity to the carriers. TSA estimates that it will
complete the risk assessment and security plan by April 2005.
Furthermore, since Secure Flight requirements have not been fully
defined and the system is still undergoing development and testing, TSA
is unable to certify and accredit the system as secure. Certifying and
accrediting a system as secure requires that the appropriate officials
have the necessary information to make a credible risk-based decision
regarding whether to put the system into operation. This process is
typically completed after the system is fully developed. Identifying
and assessing information security risks and developing system security
plans are two critical activities that directly support security
accreditation. TSA estimates that it will obtain system certification
and accreditation by July 2005.
Although TSA plans to implement a security management program for
Secure Flight, TSA officials acknowledged that information security is
a key risk area. To mitigate a possible risk of not certifying and
accrediting the Secure Flight system on schedule, TSA officials stated
that the Office of Transportation Vetting and Credentialing would apply
resources to these security issues--within a minimum of 4 months prior
to the planned operational date--to provide time to meet the
certification and accreditation requirements. TSA initially projected
that Secure Flight would be certified and accredited by January 2005
based upon key development and testing milestones. However, these
milestones have since slipped to July 2005 to align with system
readiness.
TSA acknowledged that completion of the security risk assessment,
system security plan, and certification and accreditation process is
critical to ensuring the security of Secure Flight. DHS Management
Directive 4300 requires that these be completed before the system can
become fully operational. TSA has developed a schedule to accomplish
these activities. Failure to complete the comprehensive risk assessment
and security plan on schedule, however, could result in an increased
risk that the system certification and accreditation may be delayed.
Life-Cycle Cost Estimates Have Not Been Developed and An Expenditure
Plan Was Recently Finalized:
Area of Congressional Interest: Life-Cycle Cost Estimates and
Expenditure Plans:
TSA's life-cycle cost estimates have not been developed, in part
because key decisions regarding how Secure Flight will operate, and the
data it will use, have not yet been made. TSA also recently finalized
an expenditure plan detailing plans for future program expenditures.
Life-cycle cost estimates and expenditure plans are critical components
of sound program management for the development of any major
investment. Developing life-cycle cost estimates also reflects Office
of Management and Budget guidance and can be important in making
realistic decisions about developing a system.[Footnote 50] Expenditure
plans, which generally identify near-term spending, are designed to
provide lawmakers and other officials overseeing a program's
development with a sufficient understanding of the system acquisition
to permit effective oversight, and to allow for informed decision
making about the use of appropriated funds.
TSA officials stated that they have not yet developed reliable life-
cycle cost estimates for the Secure Flight program because of the
uncertainties surrounding Secure Flight's requirements, such as whether
commercial data will be used. Life-cycle costs represent the overall
estimated cost for a particular investment alternative over a period of
time corresponding to the life of the investment, including initial
direct and indirect costs plus any periodic or continuing costs of
operation and maintenance. According to TSA officials, life-cycle cost
estimates cannot be accurately developed until after initial testing
has taken place and policy decisions have been made regarding Secure
Flight requirements. For example, TSA officials stated that the
estimated cost to operate Secure Flight can more accurately be made
after TSA has decided whether to use commercial data to verify a
person's identity as part of the program. According to TSA officials,
the use of commercial data could greatly increase the annual cost to
operate the Secure Flight program. TSA has also not determined the cost
associated with obtaining system connectivity, such as developing an
interface between CBP and air carriers in order to transmit data.
Because of these uncertain program requirements, TSA considers life-
cycle costs to be a key risk facing Secure Flight.
While TSA believes it cannot provide reliable cost estimates at this
point in the development of Secure Flight, TSA should be able to
develop initial estimates of life-cycle cost ranges for Secure Flight,
using certain assumptions about the program's components. Life-cycle
cost estimates can include a cost range based on certain factors. For
example, the high-end estimate would assume the most expensive
operating cost possible for the system (if all components being
considered were incorporated), and the low-end estimate would assume
the least expensive operating cost (if all components being considered
were not incorporated). However, TSA officials stated that they will
not develop life-cycle costs until after testing is complete and policy
decisions have been made regarding program requirements. Officials
could not identify a date when they expect these estimates to be
developed.
Moreover, estimating life-cycle costs is an important oversight
procedure for a program. A reliable life-cycle cost estimate can be
important in making realistic decisions about developing a system, and
can alert an agency to growing cost problems and the need for
mitigating actions. Accordingly, reliable life-cycle cost estimates
should be developed as early in the program's development as possible.
Failure to develop reliable life-cycle cost estimates could increase
the risk that a program may be underfunded and subject to cost
overruns, which could result in a program being reduced in scope or
additional funding being requested and appropriated to ensure the
program meets its objectives. Conversely, overestimating life-cycle
costs creates the risk that a program will be deemed unaffordable. As
TSA moves forward with the development and implementation of Secure
Flight, it will be important for TSA to follow guidance issued by the
OMB in developing life-cycle cost estimates.
TSA recently finalized its Secure Flight expenditure plan, which TSA
refers to as a spend plan, for its fiscal year 2005
appropriation.[Footnote 51] According to TSA officials, this plan
includes planned expenses for each month in fiscal 2005 for each major
program, project, or activity, such as government personnel-related
costs; communications, including information technology; and other
contractual services. Because TSA had only recently finalized the
expenditure plan, it was not available for our review. However, our
experience in working with Congress and other agencies in developing
and implementing expenditure plans shows that these plans need to
disclose a sufficient level and scope of information for oversight
officials to understand what system capabilities and benefits are to be
delivered, by when, and at what cost, and what progress is being made
against the commitments that were made in prior expenditure
plans.[Footnote 52] Further, expenditure plans should disclose how the
program will be managed to provide reasonable assurance that system
capability, benefit, schedule, and cost commitments will be met. TSA's
expenditure plan should include this level of detail in order to
provide the Congress with the information needed for effective
oversight.
TSA Has Taken Steps to Minimize Impacts on Passengers and Protect
Passenger Rights, but Its Operational Plans Must Be More Fully Defined
before Protections and Impacts Can Be Accurately Assessed:
The data-matching functionality planned for Secure Flight, which TSA is
in the process of testing, involves accessing and manipulating personal
information about travelers and thus has the inherent potential to
adversely affect their privacy or impact their rights. Aware of this
potential, TSA has begun to take steps to minimize potential impacts
and protect passenger rights. However, TSA has not yet clearly defined
the privacy impacts of the planned system or the full actions it plans
to take to mitigate them. For example, although TSA developed
documentation identifying potential privacy impacts for Secure Flight
data processing tests, it has not yet assessed the potential impact on
passenger privacy of the system in an operational environment, because
of the early stage of Secure Flight's development. TSA has also drafted
a redress process to provide passengers who believe they were
inappropriately delayed from boarding their scheduled flights because
of Secure Flight a means by which to appeal these decisions. However,
TSA has not yet clearly defined how it plans to implement this process.
According to TSA, the draft Secure Flight redress process is similar to
the current process for addressing passenger complaints about the watch
list screening process, but differs in that it will provide individuals
who believe they have been inappropriately selected for secondary
screening the opportunity to seek redress. Further, in order to provide
redress with respect to the terrorist screening database, agreements
must be reached with other key stakeholders. These agreements have not
yet been reached, adding to the uncertainty about how the operational
system may affect passengers and whether the redress process will be an
improvement over what is currently in place. In addition, although DHS
and TSA have taken steps to address international privacy concerns in
developing Secure Flight, such as limiting Secure Flight to
prescreening only domestic passengers, issues remain, particularly with
regard to the European Union. Until TSA fully defines its operational
plans for the Secure Flight system--which officials stated they plan to
do later in the system's development--it will remain difficult to
determine whether the planned system will offer reasonable privacy
protection to passengers who are subject to prescreening or mitigate
potential impacts on passengers' privacy.
Privacy Protections and Impacts Cannot Yet Be Assessed:
Area of Congressional Interest: Privacy Concerns:
The Privacy Act--the primary legislation that regulates the
government's use of personal information[Footnote 53] --requires that
agencies maintain only such information about an individual as is
relevant and necessary to accomplish a purpose of the agency.[Footnote
54] However, it is difficult to determine whether Secure Flight will
meet this requirement because TSA has not determined what personal
information will be maintained in the system. TSA officials stated that
the purpose of recently completed Secure Flight testing was to
determine what information from PNRs was relevant and necessary to
support Secure Flight operations. TSA officials further stated that
during testing, they planned to determine whether additional data
elements, such as date of birth, would be necessary to match PNR data
against data in the terrorist screening database. Until TSA determines
which data elements will be required for Secure Flight operations,
based on the results of these tests, whether TSA is collecting only
relevant and necessary personal information cannot be determined.
The Privacy Act also requires agencies to publicly release specific
information regarding the handling of privacy-related information in
systems that contain such information. On September 21, 2004, TSA
released privacy notices for the Secure Flight data processing test.
These notices included a privacy impact assessment, system of records
notice, proposed information collection request, and a proposed order
to airlines to provide PNR data.[Footnote 55] In the system of records
notice, TSA claimed several exemptions from Privacy Act requirements
for the test.[Footnote 56] However, to date, TSA has not published a
rule explaining the reasons for these exemptions, as required by the
Privacy Act.[Footnote 57] TSA officials stated that they subsequently
decided not to claim Privacy Act exemptions and, therefore, did not
need to issue a rule. According to TSA officials, they made their
decision based on TSA's confidence in its ability to control access to
the information pursuant to other legal authority. On March 14, 2005,
TSA officials stated that they intend to issue a revised system of
records notice reflecting their decision not to claim Privacy Act
exemptions. Further, they stated that an additional set of privacy
notices would be issued once the data processing test was complete and
results had been analyzed, and that they intended to issue a Privacy
Act exemption rule for the operational phase of the program that would
implement any exemptions claimed and explain the agency's basis for
claiming such exemptions. TSA officials stated that they plan to issue
a draft rule and privacy notices for Office of Management and Budget
review in May 2005, and a final rule and privacy package in June 2005.
A determination of whether Secure Flight will be in compliance with the
Privacy Act cannot be made until such notices are issued.
Privacy is also a consideration within the broader context of Fair
Information Practices--a set of internationally recognized privacy
principles that underlie the Privacy Act.[Footnote 58] As with the
Privacy Act, given the stage of Secure Flight's development, it cannot
yet be determined whether Secure Flight will adhere to the Fair
Information Practices. For example, one of the Fair Information
Practices is data quality: Personal information should be relevant to
the purpose for which it is collected and be accurate, complete and
current as needed for that purpose. However, as we have noted,
potential concerns exist regarding reliance on the terrorist screening
database that is outside the scope of TSA's control, and regarding how
passengers will be able to access and correct erroneous information. In
addition, although TSA required that airlines provide all information
from designated PNRs for its data processing test, TSA will need to
make an explicit determination about what data elements from the PNR or
other data it plans to collect in order for the operational system to
comply with the "relevant and necessary" standard. Whether TSA will
collect only relevant and necessary personal information cannot be
assessed until this determination is made. For example, TSA officials
acknowledged that they still have to reach agreements with TSC
regarding the information TSA plans to receive from TSC, including data
quality requirements and the correction of erroneous information
contained in the terrorist screening database, and they stated that
they are in the process of negotiating this agreement. Further, TSA's
plans to test the use of commercial data include consideration of the
possible use of such data to augment airline-provided PNR data.
According to TSA officials, they plan to define the final redress
process in April 2005 and issue a final privacy rule and notices in
June 2005.
A Redress Process Is Being Developed, but Key Stakeholder Roles and
Responsibilities Have Not Yet Been Defined:
Area of Congressional Interest: Redress Process:
A robust redress process is key to protecting passenger rights because
it establishes a system of due process whereby aviation passengers who
believe they have been inappropriately delayed from boarding their
scheduled flights by TSA may appeal such decisions and correct any
erroneous underlying information contained in the Secure Flight system.
A robust redress system would address the Privacy Act's requirement
that individuals be able to access and correct their personal
information. It is also fundamental to the Fair Information Practice
known as individual participation--the ability of individuals to know
about the collection of personal information, to access that
information, to request correction, and to challenge the denial of
those rights.
Under the current passenger prescreening system, air carriers compare
passenger information against no-fly and selectee lists provided by
TSA. The comparison of passenger information against the no-fly and
selectee lists can result in passengers being unnecessarily delayed or
denied boarding should they have a name that is the same as, or similar
to, that of a person on a watch list. To address this issue within the
current system, TSA developed a clearance procedure whereby passengers
who experience delays may submit a passenger identity verification form
to TSA for a determination about whether the passenger is to be placed
on a "cleared" list. If upon review, TSA determines that the
passenger's identity is distinct from the person on a watch list, TSA
is to notify the airlines and notify the passenger that, in the future,
the clearance procedure will aid in expediting the person's check-in
process. However, the effectiveness of the current redress process is
uncertain. For example, TSA officials stated that the process currently
in place does not provide redress for those who are included on a watch
list but who believe such inclusion is inappropriate.[Footnote 59]
According to TSA officials, the redress process envisioned for Secure
Flight will be based on the current process, with two major extensions.
First, individuals who believe they have been inappropriately included
in the terrorist screening database are to have the opportunity to seek
redress. While exact implementation details remain undetermined, TSA
officials said they plan on establishing an agreement with TSC to
review the reasons for an individual being in the terrorist screening
database should that individual seek redress. According to this
concept, TSC would assess the reason a person is listed in the
database, including consulting with the originating agency, and would
remove a person from the database if appropriate. Second, the Secure
Flight redress process is to include an appeals process--a feature also
not available under the redress process. According to TSA officials,
although the criteria to be used for handling redress cases is under
development, the Secure Flight redress process would allow passengers
to file a first-level appeal with the TSA Privacy Officer or the
Director of Civil Rights if discrimination is alleged, and, if
necessary, a second-level appeal with the DHS Privacy Officer.
Like the current redress process, the proposed Secure Flight redress
process would be initiated by a passenger registering a compliant with
TSA. After receiving a completed passenger identity verification form
from the complainant, TSA is to investigate the cause behind the
screening decision. If the cause is a name similarity (false positive)
or an exact match with the terrorist screening database, TSA is to
refer the case to TSC for further investigation--not a feature of the
current redress process. While TSA and TSC have not reached an
agreement related to Secure Flight, the system's draft redress process
states that TSC will review screening decisions, including verification
of any match, review of intelligence information, and consultation with
originating intelligence agencies. The resolution of these reviews,
including responsibilities for adjudication of different views and
information, remains to be determined. Additionally, it remains unclear
whether the appeals process will provide passengers with the ability to
appeal determinations made by the TSC.
Ensuring that the proposed redress process for Secure Flight is robust
will be challenging for TSA for two significant reasons. First, much of
the information underlying decisions to add individuals to the TSC
terrorist screening database is likely to be classified, and as such,
it will not be accessible to passengers, who will inevitably face
substantial restrictions on their ability to know what information is
being associated with them, as is the case with the current process.
Second, TSA does not control the content of the terrorist screening
database that it intends to use as the primary input in making
screening decisions, and will have to reach a detailed agreement with
the TSC outlining a process for correcting erroneous information in the
terrorist screening database. Until TSA and TSC reach an agreement, it
will remain difficult to determine whether redress under Secure Flight
will be an improvement over the process currently used or if it will
provide passengers with a reasonable opportunity to challenge and
correct erroneous information contained in the system.
In addition, although still in draft, TSA's concept for redress focuses
on individuals inconvenienced by the system--persons "singled out too
frequently." The draft redress process documentation does not address a
means for passengers who are inappropriately denied boarding to seek
redress. A robust redress process should not only alleviate the
annoyance of repeated additional screening, but should also provide
redress to those who are wrongfully denied boarding. TSA will need to
fully define how to handle redress for those denied boarding as it
develops the redress process for Secure Flight.
At the time of our review, TSA had not yet decided whether Secure
Flight would use commercial data to assist in reducing false positives,
identifying false negatives, and verifying the validity of the
identities presented by passengers. However, should TSA decide to
proceed with the use of commercial data, it will need to address
several concerns. First, since TSA does not control the content of
commercial databases, it will need to reach specific agreements with
commercial data aggregators on a process for correcting erroneous
information. We previously reported that under CAPPS II, TSA proposed
that it would be the responsibility of passengers to contact the owners
of commercial databases directly in order to correct inaccurate
information.[Footnote 60] However, correcting such erroneous
information may be difficult because commercial data providers, which
aggregate data from other sources, may have no obligation to correct
the data they maintain. Further, the exact source of commercial data
used in any given screening decision might not be disclosed to the
passenger, because of licensing agreements. Should TSA proceed with
using commercial identity verification, it will need to address these
concerns and reach specific agreements with commercial data aggregators
similar to the agreement it will need to reach with TSC.
Secure Flight Design Reduces Some International Privacy Concerns, but
Issues Remain:
As noted in our February 2004 report on CAPPS II, obtaining
international cooperation to obtain passenger data to prescreen
international passengers for CAPPS II was a significant
challenge.[Footnote 61] In order to provide prescreening of passengers
on international flights in addition to domestic flights, CAPPS II
needed data on passengers from foreign countries, flying on foreign
airlines, or purchasing tickets through foreign sources. However, the
European Union, in particular, raised concerns about its citizens' data
being used by CAPPS II, asserting that using such data is not in
compliance with its privacy directive. At the end of 2003, DHS and
European Union officials finalized an agreement regarding the transfer
of data for use by CBP that would permit TSA to use European Union
passenger data for testing CAPPS II. The agreement, however, did not
permit TSA to use these data for CAPPS II operations. According to
European Union officials, they were prepared to discuss the use of
these data in a second, later round of negotiations when U.S.
governmental processes were complete and congressional concerns about
privacy protections were addressed.
TSA officials stated they have been sensitive to European Union privacy
concerns in developing Secure Flight and have taken steps to address
these concerns. Specifically, TSA officials stated that Secure Flight
will only screen passengers on domestic flights. Passengers on
international flights will continue to be screened by CBP. TSA also
agreed that the agreement to permit the use of European Union data for
CAPPS II testing does not apply to Secure Flight. Further, in its order
requiring airlines to provide historical PNR data for Secure Flight
testing, TSA allowed air carriers to exclude from the June 2004 PNR
submission any European Union flight segments. According to TSA
officials, this provision was designed to help the air carriers avoid
any potential liability that could arise from providing European Union
passenger data for Secure Flight testing, while making clear that TSA
has statutory authority to prescreen European Union citizens on U.S.
domestic flights.[Footnote 62] Nonetheless, TSA has acknowledged that
the use of passenger data that originates in reservations made in a
European Union country may create concerns under that country's privacy
laws. For example, European Union privacy laws cover personal
information originating in the European Union. Thus, even a wholly
domestic U.S. flight could involve European Union data if the passenger
purchased the ticket in the European Union. Further, because TSA and
CBP have not finalized plans for how CBP will transmit airline
passenger data (PNRs) to TSA for Secure Flight, it has not been decided
whether CBP or TSA will filter out international passenger data before
the PNRs are inputted into Secure Flight. If TSA performs this
filtering of international passenger data, additional questions may be
raised about TSA handling personal data of individuals from the
European Union and other countries. According to TSA officials, they
are working toward both a political and a technical solution to these
issues. DHS and TSA officials further stated that they briefed European
Union officials of plans for Secure Flight and would continue regular
discussions to keep them apprised of Secure Flight development.
According to TSA officials, there is no indication of significant
concerns with Secure Flight from any other nations.
Conclusions:
TSA is making progress in addressing key areas of congressional
interest related to the development and testing, system effectiveness,
program management and oversight, and privacy protections for the
Secure Flight program, as outlined in Public Law 108-334. Specifically,
TSA is in various stages of addressing each of the 10 areas of interest
outlined in the law, including establishing a framework for a redress
process; beginning testing to measure the effectiveness of system data
matches; and using oversight boards to oversee the development of
Secure Flight. However, TSA has not yet completed these efforts or
fully addressed these areas, due largely to the current stage of the
system's development. Specifically, initial system testing has only
recently been completed, and key policy decisions--including what data
will be collected and how they will be transmitted--have not yet been
made. Until requirements are defined and testing is completed, and
operating policies are finalized--scheduled for later in the system's
development--we cannot determine whether Secure Flight, in an
operational environment, will fully address these areas of interest.
As development and testing of Secure Flight continue, and program
policy decisions are made, TSA will need to manage key program risks in
order to help ensure the system meets its intended objectives as it
becomes operational. A key program risk is related to requirements
definition and system testing. TSA has made progress in recently
completing initial testing for Secure Flight. However, TSA has not
finalized its system requirements or concept of operations, or
developed detailed test plans for critical system testing. Until TSA
finalizes these documents and completes additional system testing, it
is uncertain how well Secure Flight will perform, or whether it will be
ready for operational deployment in August 2005. It will be important
for TSA to effectively manage the system changes that are likely to
result from the final testing phases with sound management discipline
and rigor.
Another key program risk is the ability of TSA to establish
connectivity between air carrier reservation systems and TSA to allow
for the transmission of data to support Secure Flight operations. TSA
officials have not yet developed a plan identifying how connections
will be made between air carrier reservation systems and TSA to support
Secure Flight prescreening. The majority of air carrier officials we
interviewed expressed various concerns regarding connectivity,
including Secure Flight's ability to provide a two-way real-time
exchange of data to allow for the almost instantaneous prescreening of
passengers. Further, officials from TSA and CBP stated that it was
uncertain whether CBP's existing systems--which will support the
transfer of data--will be able to handle the large amount of data that
will need to be regularly transferred. The effectiveness of Secure
Flight in obtaining the data it needs to make accurate matches against
the terrorist screening database, and to transmit the results of data
matches to air carriers in a timely manner, is directly affected by the
system's ability to send and receive data. Moreover, key decisions on
how connectivity will be established could affect the cost, schedule,
and performance of Secure Flight.
Ensuring that impacts on passengers are minimized, and passenger rights
are protected, is also critical to the success of Secure Flight.
Concerns over privacy protections related to Secure Flight's
predecessor, CAPPS II, led--in part--to an internal departmental review
of the program and its ultimate cancellation. TSA has begun to take
steps to minimize potential impacts on passengers and to protect
passenger rights during the initial testing phase of Secure Flight,
including releasing privacy notices for Secure Flight data processing
tests. However, TSA has not yet clearly defined privacy impacts of
Secure Flight in an operational environment, or the full actions it
plans to take to mitigate potential impacts, due in part to the current
stage of the system's development. For example, TSA does not plan to
determine whether additional data elements will be necessary to match
passenger data to data contained in the terrorist screening database
until further testing is completed. Until TSA determines which data
elements will be required, based on the results of testing, it is
unclear whether TSA will collect only relevant and necessary personal
information for Secure Flight. Further, although TSA developed a
conceptual description of its planned redress process for Secure
Flight, key elements of this process are still being determined,
including agreements with key stakeholders, such as TSC. Ensuring that
a robust redress process is developed for Secure Flight will be
challenging, since much of the information underlying decisions to add
individuals to the terrorist screening database is likely to be
classified, and may not be easily accessed and corrected.
Additionally, TSA has not yet developed performance goals and measures
to gauge the effectiveness of the Secure Flight program, once it
becomes operational. Performance goals and measures are intended to
provide Congress and agency management the ability to systematically
assess a program's strengths, weaknesses, and performance, and then
identify appropriate remedies. Performance goals and measures can
assist TSA in determining whether Secure Flight, once operational,
achieves its intended results. TSA also has not developed life-cycle
cost estimates and only recently finalized an expenditure plan, which
are key steps in providing those with oversight responsibilities with
information needed to make informed decisions. Life-cycle cost
estimates should be developed as early in a program's development as
possible. Failure to develop reliable estimates can increase the risk
that a program may be underfunded and subject to cost overruns, or will
not be affordable. Further, expenditure plans should be developed to
include a sufficient level of detail to identify what system
capabilities will be delivered, by when, and at what cost. In addition
to providing system development and contractor oversight, TSA will need
to develop and finalize these estimates and plans to help ensure sound
program management and oversight.
Recommendations for Executive Action:
To help manage risks associated with Secure Flight's continued
development and implementation, and to assist the Transportation
Security Administration in developing a framework from which to support
its efforts in addressing congressional areas of interest outlined in
Public Law 108-334, we recommend that the Secretary of the Department
of Homeland Security direct the Assistant Secretary, Transportation
Security Administration, to take the following six actions:
* Finalize the system requirements document and the concept of
operations, and develop detailed test plans to help ensure that all
Secure Flight system functionality is properly tested and evaluated.
These system documents should address all system functionality and
include system stress test requirements.
* Develop a plan for establishing connectivity among the air carriers,
U.S. Customs and Border Protection, and the Transportation Security
Administration to help ensure the secure, effective, and timely
transmission of data for use in Secure Flight operations.
* Develop reliable life-cycle cost estimates and expenditure plans for
Secure Flight--in accordance with guidance issued by the Office of
Management and Budget--to provide program managers and oversight
officials with information needed to make informed decisions regarding
program development and resource allocations.
* Develop results-oriented performance goals and measures to evaluate
the effectiveness of Secure Flight in achieving intended results in an
operational environment--as outlined in the Government Performance and
Results Act--including measures to assess associated impacts on
aviation security.
* Prior to achieving initial operational capability, finalize policies
and issue associated documentation specifying how the Secure Flight
program will protect personal privacy, including addressing how the
program will comply with the requirements of the Privacy Act of 1974
and related legislation.
* Prior to achieving initial operational capability, finalize policies
and procedures detailing the Secure Flight passenger redress process,
including defining the appeal rights of passengers and their ability to
access and correct personal data.
Agency Comments and Our Evaluation:
We provided a draft copy of this report to DHS for its review and
comment. On March 22, 2005, we received written comments on the draft
report, which are reproduced in full in appendix II. DHS generally
agreed with the report and recommendations, and described some actions
it has initiated to address the recommendations. DHS further stated
that initial system testing demonstrated that needed functionality is
in place to support program implementation. DHS also provided technical
comments related to the program's development, testing, and
implementation. These comments were incorporated as appropriate.
Regarding actions DHS reported taking to address the recommendations,
DHS stated that TSA plans to complete the Secure Flight concept of
operations by March 2005, and system requirements by April 2005. DHS
also noted that formal arrangements between CBP and TSA and for two-way
connectivity with air carriers are in progress. DHS also acknowledged
that while they plan to prepare life-cycle costs and a comprehensive
set of critical performance measures for Secure Flight, these efforts
will be accomplished during the later stages of the system's
development. DHS further stated that TSA will issue for public comment
a new privacy package as it implements Secure Flight, and is finalizing
a redress process for passengers who feel they have been unfairly or
incorrectly singled out for additional screening.
DHS also highlighted several key TSA achievements, including issuing a
privacy package for Secure Flight testing, awarding a contract for
testing, developing an acquisition plan, and working jointly with the
TSC and CBP to prepare a draft concept of operations. DHS further
expressed concern that the report did not appropriately characterize
the status of the system's development and testing. Specifically, DHS
stated that recently completed functionality testing confirmed TSA's
key hypotheses about Secure Flight's data matching capabilities, and
demonstrated that the needed functionality exists to support the
implementation of Secure Flight. We recognized that TSA recently
reported completing testing of key data matching functions, and that it
believes this testing confirmed its hypotheses and demonstrated some
functionality. However, because this testing was only recently
completed and test results have not been fully documented and analyzed,
we were unable to independently assess these results. In addition, TSA
did not test all of the functions planned for Secure Flight, such as
the connectivity needed to obtain and match data from the air carriers
with data in the terrorist screening database. The testing of this
function and other key functions is scheduled to occur during the final
phases of testing. In fact, TSA plans to begin a full range of unit,
integration, system, stress testing, and end-to-end testing in April
2005. Thus, while we acknowledge that TSA completed important initial
testing of system functionality, critical system testing has not yet
been conducted. These tests are needed to determine whether Secure
Flight will provide the desired functionality and operate as intended
in an operational environment.
In addition, DHS highlighted that TSA had issued a comprehensive
privacy package for Secure Flight testing and, in response to our
recommendation that TSA finalize how it will comply with the Privacy
Act, DHS stated that TSA is currently in compliance with the Privacy
Act. However, as discussed in the report, the Privacy Act requires TSA
to publish a rule explaining the reasons for the exemptions it claimed
in its system of records notice, issued in September 2004. To date, TSA
has not published such a rule. In a discussion with us on March 14,
2005, TSA officials stated they no longer wish to claim an exemption
from the Privacy Act and that they intend to issue a revised system of
records notice that would serve to notify the public of this change.
TSA has not yet published a revised notice, and DHS official comments
to a draft of this report do not refer to plans for a revised notice.
Until TSA either publishes the rule required by the Privacy Act or
issues a revised system of records notice, it will not be fully
compliant with the Privacy Act with regard to the test phase of the
program. Further, as identified in the report, TSA will have to comply
with the Privacy Act for Secure Flight beyond the testing phase once
the system becomes operational.
We are sending copies of this report to the Secretary of the Department
of Homeland Security, the Administrator of the Transportation Security
Administration, and the Assistant Administrator of the Office of
Transportation Vetting and Credentialing. Copies of this report will be
made available to others on request. In addition, the report will be
available at no charge on GAO's Web site at http://www.gao.gov.
If you have any questions about this report, please contact Cathleen
Berrick at (202) 512-3404, or berrickc@gao.gov, or Christine Fossett,
Assistant Director, at (202) 512-2956, or fossettc@gao.gov. Questions
concerning system development and testing or security should be
directed to David Powner at (202) 512-9286, or pownerd@gao.gov. Major
contributors to this report are listed in appendix III.
Signed by:
Cathleen A. Berrick:
Director, Homeland Security and Justice Issues:
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
List of Congressional Committees:
The Honorable Thad Cochran:
Chairman:
The Honorable Robert C. Byrd:
Ranking Minority Member:
Committee on Appropriations:
United States Senate:
The Honorable Ted Stevens:
Chairman:
The Honorable Daniel K. Inouye:
Ranking Minority Member:
Committee on Commerce, Science, and Transportation:
United States Senate:
The Honorable Jerry Lewis:
Chairman:
The Honorable David R. Obey:
Ranking Minority Member:
Committee on Appropriations:
House of Representatives:
The Honorable Don Young:
Chairman:
The Honorable James L. Oberstar:
Ranking Minority Member:
Committee on Transportation and Infrastructure:
House of Representatives:
The Honorable Tom Davis:
Chairman:
Committee on Government Reform:
House of Representatives:
The Honorable Adam H. Putnam:
House of Representatives:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To assess efforts by the Transportation Security Administration (TSA)
to develop and implement Secure Flight as mandated by Public Law 108-
334, enacted in October 2004,[Footnote 63] we addressed the following
four questions: (1) What is the status of Secure Flight's development
and implementation? (2) What factors could influence the effectiveness
of Secure Flight? (3) What procedures have been put in place to oversee
and manage the Secure Flight program, including ensuring stakeholder
coordination? And (4) What efforts are being taken to minimize the
impacts on passengers and protect passenger rights? In addressing these
four questions, we also addressed the 10 specific issues that we were
mandated to review under Public Law 108-334. Since some of the
information addressing the congressional areas of interest is
considered Sensitive Security Information, we are also issuing a
separate letter containing this information.
To determine the status of Secure Flight's development and
implementation, we interviewed officials from the TSA's Office of
Transportation Vetting and Credentialing--the Office of National Risk
Assessment prior to November 2005--which is responsible for developing
and implementing Secure Flight, and the Office of Aviation Operations.
We also reviewed program documentation including Secure Flight system
requirements, a draft concept of operations, test plans, a project
schedule, and a working milestone chart. We also reviewed a summary of
TSA's preliminary Secure Flight test results. In addition, we traced
existing test results to Secure Flight system requirements to determine
the completeness of Secure Flight testing. We interviewed testing
officials to discuss test activities and results and plans for future
testing. We also obtained information on requirements and testing of
the computer-assisted passenger prescreening system (CAPPS II) and
obtained additional information regarding the differences and
similarities between the current computer-assisted passenger
prescreening system (CAPPS I), CAPPS II, and Secure Flight. We reviewed
relevant legislation as it pertained to Secure Flight. Further, in
determining the status of Secure Flight's development and
implementation, we addressed the mandated issue identified in Public
Law 108-334 related to TSA's efforts to stress test all search tools in
Secure Flight and demonstrate that the system can make accurate
predictive assessments of passengers who might constitute a threat to
aviation.
To address our second objective, related to factors that could
influence the effectiveness of Secure Flight, we interviewed officials
from TSA's Office of Transportation Vetting and Credentialing and TSA's
Office of Aviation Operations. We also interviewed officials from the
U.S. Customs and Border Protection and the Terrorist Screening Center,
which are key stakeholders for Secure Flight. We reviewed program
documentation, including Secure Flight system requirements, a draft
concept of operations, test plans, and test results, as available. We
interviewed TSA officials regarding their recently completed tests
designed to identify the most effective combination of data elements in
air carriers' passenger name records (PNR) and the terrorist screening
database to be matched. We discussed the testing and analysis conducted
and reviewed a summary of the initial test results, because the test
data and final reports were not yet available for our review. We also
discussed issues relating to the commercial data test with TSA
officials. We interviewed officials associated with the Terrorist
Screening Center, which is responsible for the development and
maintenance of the terrorist screening database, regarding their
process for placing names on and removing names from the database and
the methods used to ensure the accuracy of the database. However, we
did not independently verify the procedures used. We also reviewed
recent changes to the CAPPS I rules and interviewed TSA officials to
determine modifications that have been made to the system to
accommodate intrastate transportation in states with unique needs. In
addition, we interviewed TSA officials and reviewed documents regarding
the ability of Secure Flight to identify passengers who assume the
identity of another individual, known as identity theft. In determining
what factors could influence the effectiveness of Secure Flight, we
addressed the mandated issues identified in Public Law 108-334 related
to TSA's efforts (1) to ensure that the underlying error rate of the
databases that will be used will not result in a large number of false
positives, and (2) to modify Secure Flight with respect to intrastate
transportation to accommodate states with unique needs and passengers
who might otherwise regularly trigger selectee status.
To address our third objective, regarding determining the processes and
procedures in place to oversee and manage the Secure Flight program,
including stakeholder coordination, we interviewed officials from the
Office of Transportation Vetting and Credentialing and other TSA and
DHS officials with Secure Flight oversight and management
responsibilities. We reviewed documentation on internal and external
oversight mechanisms, including documents submitted to DHS's Investment
Review Board and the board's decision, the draft business case for the
Transportation Vetting Platform, and documents related to the Aviation
Security Advisory Committee working group focusing on Secure Flight. We
also reviewed documentation on program management--contract and
security management, performance measures, oversight policies on the
use and operation of the system, and life-cycle costs and expenditure
plans. In addition, to assess TSA's coordination with government
stakeholders, we interviewed officials from the Terrorist Screening
Center, U.S. Customs and Border Protection, and TSA's Office of
Aviation Operations regarding coordination with TSA, and memorandums of
understanding regarding services to be provided for Secure Flight
during its testing phases and when fully operational. To assess TSA's
external coordination, we interviewed officials from 4 large air
carriers and 3 major privacy groups to discuss TSA's outreach efforts
to the airline industry and to provide industry stakeholders with an
opportunity to communicate perspectives about Secure Flight. We
selected these air carriers and privacy groups due to their ongoing
involvement with TSA during the CAPPS II project and the Secure Flight
project. In addition, we had formal interviews with officials from two
air carrier associations and these officials agreed subsequently to
disseminate written questions regarding Secure Flight to their member
air carriers. Officials from 14 air carriers emailed written responses
to our questions regarding the development and implementation of Secure
Flight. These 14 air carriers and their regional affiliates accounted
for 91 percent of all domestic enplanements during the 1-year period
from October 2003 until September 2004. Because we selected non-
probability samples of air carriers and privacy groups, the results of
the interviews with air carrier and privacy group officials and the
written responses provided by air carrier officials cannot be
generalized to the airline industry or all privacy groups. In assessing
TSA's efforts to provide program oversight and management and to
coordinate with stakeholders, we addressed the specific mandated issues
identified in Public Law 108-334 related to (1) the establishment of an
internal oversight board to monitor the manner in which Secure Flight
is being developed; (2) the incorporation of operational safeguards to
reduce opportunities for abuse; (3) the establishment of security
measures to protect Secure Flight from unauthorized users; (4) the
adoption of policies establishing effective oversight of the use and
operation of the system; and (5) the existence of appropriate life-
cycle cost estimates and expenditure and program plans.
To examine the efforts being taken to minimize the impacts of Secure
Flight on passengers and protect passenger rights, we assessed TSA's
efforts to address Privacy Act requirements[Footnote 64] and Fair
Information Practices,[Footnote 65] as well as TSA's plans for
developing a system of redress for passengers identified for additional
screening or denied boarding based on Secure Flight. We analyzed TSA's
documentation on privacy issues, such as the draft redress process, and
interviewed agency officials with privacy-related responsibilities,
including TSA's Privacy Officer. We also reviewed data on TSA's current
redress process. We also interviewed officials from several privacy
advocacy organizations to gain insight into privacy concerns regarding
Secure Flight. In addition, we assessed TSA's efforts to address
international privacy concerns regarding Secure Flight, which were a
key concern during the development of CAPPS II. In determining the
efforts being taken to minimize the impacts on passengers and protect
passenger rights, we addressed the specific mandated issues identified
in Public Law 108-334 related to (1) the assurance that there are no
specific privacy concerns with the technological architecture of the
system, and (2) TSA having a system in place whereby passengers
determined to pose a threat may appeal such decision and correct
erroneous information contained in Secure Flight.
As described above, in answering these four questions, we addressed the
10 specific issues we were mandated to review by Public Law 108-
334.[Footnote 66] Table 4 describes the 10 issues and provides a cross-
reference to the sections in this report that address each issue. TSA
has not made key decisions concerning Secure Flight's implementation
and operations and, therefore, documents describing many of these
issues, such as final security plans, privacy impact assessments, and a
redress process, have not been developed or finalized. As a result,
since Secure Flight is currently undergoing development and testing,
and the system is not yet operational, we assessed the 10 areas we were
mandated to review based on the current stage of the system's
development. We conducted our work from April 2004 until March 2005 in
accordance with generally accepted government auditing standards.
Table 5: Cross-references of Legislatively Mandated Issues to Be
Reviewed by GAO with the Sections in this Report:
Legislative mandated issue (number and short title): 1. Redress
process;
Description of mandated issue: A system of due process exists whereby
aviation passengers determined to pose a threat are either delayed or
prohibited from boarding their scheduled flights by the TSA may appeal
such decisions and correct erroneous information contained in CAPPS II
or Secure Flight or other follow-on/successor programs;
Report sections/questions: 4. Privacy and redress.
Legislative mandated issue (number and short title): 2. Accuracy of
databases and effectiveness of Secure Flight;
Description of mandated issue: The underlying error rate of the
government and private databases that will be used to both establish
identity and assign a risk level to a passenger will not produce a
large number of false positives that will result in a significant
number of passengers being treated mistakenly or security resources
being diverted;
Report sections/questions: 2. Factors affecting effectiveness.
Legislative mandated issue (number and short title): 3. Stress testing;
Description of mandated issue: TSA has stress-tested and demonstrated
the efficacy and accuracy of all search tools in CAPPS II or Secure
Flight or other follow-on/successor programs and has demonstrated that
CAPPS II or Secure Flight or other follow-on/successor programs can
make an accurate predictive assessment of those passengers who may
constitute a threat to aviation;
Report sections/questions: 1. Status of development and implementation.
Legislative mandated issue (number and short title): 4. Internal
oversight;
Description of mandated issue: The Secretary of Homeland Security has
established an internal oversight board to monitor the manner in which
CAPPS II or Secure Flight or other follow-on/successor programs are
being developed and prepared;
Report sections/questions: 3. Processes for oversight and management.
Legislative mandated issue (number and short title): 5. Operational
safeguards;
Description of mandated issue: TSA has built in sufficient operational
safeguards to reduce the opportunities for abuse;
Report sections/questions: 3. Processes for oversight and management.
Legislative mandated issue (number and short title): 6. Security
measures;
Description of mandated issue: Substantial security measures are in
place to protect CAPPS II or Secure Flight or other follow-on/
successor programs from unauthorized access by hackers or other
intruders;
Report sections/questions: 3. Processes for oversight and management.
Legislative mandated issue (number and short title): 7. Oversight of
system use and operation;
Description of mandated issue: TSA has adopted policies establishing
effective oversight of the use and operation of the system;
Report sections/questions: 3. Processes for oversight and management.
Legislative mandated issue (number and short title): 8. Privacy
concerns;
Description of mandated issue: There are no specific privacy concerns
with the technological architecture of the system;
Report sections/questions: 4. Privacy and redress.
Legislative mandated issue (number and short title): 9. Modifications
with respect to intrastate travel to accommodate states with unique air
transportation needs;
Description of mandated issue: TSA has, in accordance with the
requirements of section 44903 (j)(2)(B) of title 49, United States
Code, modified CAPPS II or Secure Flight or other follow-on/successor
programs with respect to intrastate transportation to accommodate
states with unique air transportation needs and passengers who might
otherwise regularly trigger primary selectee status;
Report sections/questions: 2. Factors affecting effectiveness.
Legislative mandated issue (number and short title): 10. Life-cycle
cost estimates and expenditure plans;
Description of mandated issue: Appropriate life-cycle cost estimates,
and expenditure and program plans exist;
Report sections/questions: 3. Processes for oversight and management.
Source: GAO.
[End of table]
[End of section]
Appendix II: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
March 22, 2005:
Ms. Cathleen Berrick:
Director, Homeland Security & Justice Issues:
U.S. Government Accountability Office:
441 G Street, N.W.:
Washington, D.C. 20548:
Dear Ms. Berrick:
Thank you for the opportunity to comment on GAO's draft report
entitled, "Secure Flight Development and Testing Under Way, but Risks
Should Be Managed as System is Further Developed" GAO-05-356 received
March 17, 2005. TSA generally concurs with the recommendations in this
report. We appreciate the opportunity to provide formal written
comments and for the work of GAO over the past year.
The GAO report is being issued on Secure Flight in the eighth month of
a fourteen month planning, development, testing and implementation
cycle. The GAO review occurred during the first phases of development
and testing. (The program is scheduled to be implemented in August
2005.) TSA provided extensive access to early drafts of all major
program documents and testing results available at the time of the
audit to support GAO in its reporting. In addition, TSA met with GAO on
a regular basis to provide updates and status briefs during planning
and development.
As GAO noted during this development and testing phase, the Secure
Flight team has increased its management, oversight, and delivery
capability during its first eight months. We are very pleased with the
progress on Secure Flight, with key achievements including:
* Issuance of a comprehensive privacy package for Secure Flight
testing, including a Privacy Impact Assessment (PIA), System of Records
Notice (SORN), and Paperwork Reduction Act Notice (PRA).
* Issuance of an Order for June 2004 passenger name records (PNR); all
66 U.S. air carriers complied with the Order, providing more than 15
million PNRs to TSA.
* Award of a contract for Secure Flight Watch List and CAPPS I testing,
and successful completion of comprehensive tests and drafting of
multiple comprehensive reports of results;
* Award of a contract for Commercial Data Testing;
* Departmental approval of an Acquisition Plan that will support Secure
Flight implementation;
www.dhs.gov:
* Joint work on Concepts of Operations with our key partners the
Terrorist Screening Center (TSC) and U.S. Customs and Border Protection
(CBP) for program implementation.
TSA generally concurs with the report, but is concerned that the report
states that system development of Secure Flight is not advanced
because, in part, "initial system testing has only recently been
completed." This statement seems to carry a negative connotation when
none should be implied. The Secure Flight hardware and IT
infrastructure are largely in place and functionality testing is on
schedule and was completed in mid-February (as the audit was
concluding). This testing not only confirmed all of TSA's key
hypotheses, but also demonstrated functionality that supports program
implementation. For example, our assessment that having passengers'
full name and date of birth greatly improves watch list matching
capabilities was confirmed. In addition, our technology platform
demonstrated the capability to screen the required 1.8 million
passengers per day.
TSA's Responses to GAO Recommendations:
GAO Recommendation: Finalize the system requirements document and the
concept of operations, and develop detailed test plans-establishing
measures of performance to be tested-to help ensure that all Secure
Flight system functionality is properly tested and evaluated. These
system documents should address all system functionality and include
system stress test requirements.
TSA Concurs/Work Already in Progress: The Secure Flight Concept of
Operations has been drafted as a joint plan across key government
elements including the Terrorist Screening Center (TSC) and U.S.
Customs and Border Protection (CBP). It is in review with these
organizations and is on schedule for completion in March 2005. The
Secure Flight System Requirements are dependent upon the watch list and
commercial data testing which is commencing in late March. The System
requirements will be revised based upon final test results and are on
schedule for completion in April 2005.
GAO Recommendation Develop a plan for establishing connectivity among
the air carriers, U.S. Customs and Border Protection, and the
Transportation Security Administration to help ensure the secure,
effective, and timely transmission of data for use in Secure Flight
operations.
TSA Concurs/Work Already in Progress: TSA has been working closely with
CBP since August 2004 to establish two-way connectivity to U.S. air
carriers for Secure Flight. Preliminary agreement has been reached
between the senior leadership of both agencies concerning the roles in
the process. We agree on preliminary architecture, design and cost
estimates for this connectivity. Formal agreements between the agencies
are on track for completion in April 2005.
GAO Recommendation: Develop reliable life-cycle cost estimates and
expenditure plans for Secure Flight-in accordance with guidance issued
by the Office of Management and Budget-to provide program managers and
oversight officials with information needed to make informed decisions
regarding program development and resource allocations.
TSA Concurs/Work Already in Progress: In accordance with OMB
requirements, Secure Flight is delivering a 10-year life cycle cost
estimate in the 3rD quarter of FY05 as part of the required resource
allocation planning process. As required by the DHS Investment Review
Board (IRB) process, Secure Flight will also develop and deliver a 20-
year life cycle cost estimate by 3rD Quarter FY05. These projections
will be updated as TSA moves to program implementation and key cost
parameters are established.
As TSA moves forward in the testing and development of the Secure
Flight program, we are concurrently developing an appropriate
regulation and its required benefit/cost analysis. TSA, working with
its industry partners, will re-evaluate the benefits and costs of the
regulation as new requirements are validated during testing. As testing
is still ongoing, questions surrounding specific new or expanded data
requirements are not yet resolved. Accordingly, it is difficult to
calculate the final costs associated with the Federal Government's
operation of these functions. However, investments already made in
platform infrastructure from initial passenger pre-screening program
efforts are being leveraged for the Secure Flight program.
GAO Recommendation: Develop results-oriented performance goals and
measures to evaluate the effectiveness of Secure Flight in achieving
intended results in an operational environment-as outlined in the
Government Performance and Results Act-including measures to assess
associated impacts on aviation security.
TSA Concurs/Work Already in Progress: In accordance with OMB and the
DHS investment review process, Secure Flight is developing a
comprehensive set of critical performance measures to assess
implementation and operation of Secure Flight. These measures will be
refined and augmented during finalization of Secure Flight capability
and prior to initial passenger screening in August 2005.
GAO Recommendation: Prior to achieving initial operational capability,
finalize policies and issue associated documentation specifying how the
Secure Flight program will protect personal privacy, including
addressing how the program will comply with the requirements of the
Privacy Act of 1974 and related legislation.
TSA Concurs/ In Compliance: TSA is currently in compliance with the
Privacy Act. TSA's handling of personal information during the test
phase has been in compliance with its obligations to limit disclosure,
secure data, and provide notice on the uses of the data. In addition,
TSA established handling procedures, including a chain of custody
arrangement for the receipt, transfer and storage of the personal data
it received. TSA issued a comprehensive privacy package in September
2004, published in the Federal Register. This package included:
* A Privacy Impact Assessment (PIA) that explains how PNR data would be
used and protected by TSA:
* A System of Records Notice (SORN) that explains TSA's statutory
authority to collect passenger information and conduct the test:
* A Paperwork Reduction Act Notice (PRA) that included the Order to air
carriers and provided TSA with the authority to collect data:
TSA sought and received more than 500 comments from the public on these
documents, and incorporated requested changes where appropriate. These
documents provide disclosure to the public and establish transparency
for the public.
As TSA moves to implement Secure Flight, the agency will issue for
public comment a new PIA and SORN for the program's operational phase
and an Interim Final Rule (IFR) to implement the program. TSA also will
seek comment from the public on this document. Compliance with the
Privacy Act will continue to be a priority.
GAO Recommendation: Prior to achieving initial operational capability,
finalize policies and procedures detailing the Secure Flight passenger
redress process, including defining the appeal rights of passengers and
their ability to access and correct personal data.
TSA Concurs/Work Already in Progress: TSA is currently finalizing a
redress process for addressing any situation where passengers believe
they have been unfairly or incorrectly singled out for additional
screening. An appeals process will be included to allow for review by
TSA leadership, DHS leadership, and/or the respective TSA and DHS
Offices of Civil Rights, if discrimination is alleged.
For further information from TSA on this report and Secure Flight,
please contact TSA public affairs at (571) 227-2829.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Acting Director:
Departmental GAO/OIG Liaison Office:
[End of section]
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Cathleen A. Berrick (202) 512-3404;
David A. Powner (202) 512-9286;
Christine Fossett (202) 512-2956:
Staff Acknowledgments:
In addition to the above, J. Michael Bollinger, Grace Coleman, John de
Ferrari, R. Denton Herring, Adam Hoffman, David Hooper, Linda Koontz,
Thomas Lombardi, Michele Mackin, Colleen Phillips, Jamie Pressman,
David Plocher, John R. Schulze, Karl Seifert, Adam Vodraska, and Eric
Winter made key contributions to this report.
[End of section]
GAO Related Products:
Aviation Security: Systematic Planning Needed to Optimize the
Development of Checked Baggage Screening Systems. GAO-05-365.
Washington, D.C.: March 15, 2005.
Aviation Security: Measures for Testing the Impact of Using Commercial
Data for the Secure Flight Program. GAO-05-324. Washington, D.C.:
February 23, 2005.
Transportation Security: Systematic Planning Needed to Optimize
Resources. GAO-05-357T. Washington, D.C.: February 15, 2005.
Department of Homeland Security: Formidable Information and Technology
Management Challenge Requires Institutional Approach. GAO-04-702.
Washington, D.C.: August 27, 2004:
Aviation Security: Improvement Still Needed in Federal Aviation
Security Efforts. GAO-04-592T. Washington, D.C.: March 30, 2004.
Aviation Security: Challenges Delay Implementation of Computer-
Assisted Passenger Prescreening System.GAO-04-504T. Washington, D.C.:
March 17, 2004.
Aviation Security: Computer-Assisted Passenger Prescreening System
Faces Significant Implementation Challenges. GAO-04-385. Washington,
D.C.: February 13, 2004.
Information Technology: OMB and Department of Homeland Security
Investment Reviews. GAO-04-323. Washington, D.C.: February 10, 2004:
Aviation Security: Efforts to Measure Effectiveness and Strengthen
Security Programs. GAO-04-285T. Washington, D.C.: November 20, 2003.
Aviation Security: Efforts to Measure Effectiveness and Address
Challenges. GAO-04-232T. Washington, D.C.: November 5, 2003.
Aviation Security: Progress Since September 11, 2001, and the
Challenges Ahead. GAO-03-1150T Washington, D.C.: September 9, 2003.
Transportation Security: Federal Action Needed to Enhance Security
Efforts. GAO-03-1154T. Washington, D.C.: September 9, 2003. , September
9, 2003:
Transportation Security: Post-September 11th Initiatives and Long-term
Challenges. GAO-03-616T. Washington, D.C.: April 1, 2003.
Aviation Security: Registered Traveler Program Policy and
Implementation Issues. GAO-03-253. Washington, D.C.: November 22, 2002.
FOOTNOTES
[1] Aviation and Transportation Security Act, Pub. L. No. 107-71, 115
Stat. 597 (2001).
[2] The 9/11 Commission, The 9/11 Commission Report: Final Report of
the National Commission on Terrorist Attacks upon the United States,
July 2004.
[3] Department of Homeland Security Appropriations Act, 2005, Pub. L.
No. 108-334, § 522, 118 Stat. 1298, 1319-20 (2004).
[4] GAO, Aviation Security: TSA Modifications to Rules for Prescreening
Passengers, GAO-05-445SU (Washington, D.C.: Mar. 28, 2005).
[5] TSC was established in accordance with Homeland Security
Presidential Directive/HSPD-6 to consolidate the government's approach
to terrorism screening, including the use of terrorist information for
screening purposes. TSC is an interagency effort involving DHS,
Department of Justice, Department of State, and intelligence community
representatives, and is administered by the Federal Bureau of
Investigation. TSC maintains the terrorist screening database, which
consolidates information from terrorist watch lists to provide
government screeners with a unified set of antiterrorist information.
[6] The PNR contains data related to a passenger's reservation and
travel itinerary and is contained in an air carrier's reservation
system. Such data can include the passenger's name, phone number,
number of bags, seat number, and form of payment, among other
information.
[7] CAPPS I rules are characteristics that are used to select
passengers who require additional security scrutiny.
[8] The remaining 1 percent of passengers are manually screened by air
carriers who do not have an automated system.
[9] Pub. L. No. 107-71, § 136, 115 Stat. 597, 637 (2001).
[10] TSA planned to incorporate eight capabilities into the CAPPS II
program. We have only listed seven of these capabilities, because one
is Sensitive Security Information.
[11] The Department of Homeland Security Appropriations Act, 2004, Pub.
L. No. 108-90, § 519, 117 Stat. 1137, 1155-56 (2003), mandated that GAO
review eight areas related to the development and implementation of
CAPPS II, including system development and security, privacy, redress,
and oversight.
[12] GAO, Aviation Security: Computer-Assisted Passenger Prescreening
System Faces Significant Implementation Challenges, GAO-04-385
(Washington, D.C.: Feb. 12, 2004).
[13] The 9/11 Commission Report.
[14] TSA plans to use this centralized vetting capability to identify
terrorist threats in support of various DHS and TSA programs. Further,
TSA plans to use the platform to ensure that persons working at
sensitive locations; serving in trusted positions with respect to the
transportation infrastructure; or traveling as cockpit and cabin crew
into, within, and out of the United States are properly screened
depending on their activity within the transportation system. In
addition to supporting the Secure Flight and Crew Vetting programs, TSA
expects to leverage the platform with other applications such as TSA
Screeners and Screener applicants, commercial truck drivers with
Hazardous Materials Endorsements, aviation workers with access to
secure areas of the airports, alien flight school candidates, and
applicants for TSA's domestic Registered Traveler program.
[15] TSA planned to incorporate eight capabilities into the CAPPS II
program. We have only listed seven of these capabilities, since one is
Sensitive Security Information.
[16] While TSA does not plan to include criminal checks within Secure
Flight, it does plan to incorporate this capability into the platform,
where it may be used by other vetting applications, such as Crew
Vetting.
[17] An opt-in list could include passengers participating in TSA's
Registered Traveler program, which is currently operating in the pilot
phase at five airports. Under this program, frequent travelers at
select airports are able to volunteer for the program. Volunteers are
asked to submit information, including biometrics, necessary for TSA to
determine eligibility. The biometric information, such as fingerprints,
is used for identity verification purposes and, in conjunction with a
security assessment, allows passengers at the pilot airport locations
to go through an expedited security screening process. The results of
the five-airport pilot program will determine future applications of
the Registered Traveler concept at other airports.
[18] The Intelligence Reform and Terrorism Prevention Act of 2004, Pub.
L. No. 108-458, § 4012, 118 Stat. 3638, 3714-19, requires that TSA
begin to assume responsibility for the passenger prescreening function
within 180 days after the completion of testing.
[19] The Credentialing Program Office was responsible for worker-
screening programs, including aviation workers, alien flight students,
and the Registered Traveler Program.
[20] End-to-end testing is conducted to verify that the entire system,
including any external systems with which it interfaces, functions as
intended in an operational environment. Stress testing refers to
measuring a system's performance and availability in times of
particularly heavy (i.e., peak) load.
[21] As described earlier in this report, the scope of Secure Flight is
more limited than CAPPS II. Therefore, several features of the CAPPS II
system were deactivated, such as the identity authentication process
and alert list capability.
[22] To obtain data for Secure Flight testing, TSA issued an order in
November 2004 requiring domestic airlines to provide passenger records
for the month of June 2004. Sixty-six air carriers, representing 99.8
percent of the total enplanements, provided more than 15 million PNRs.
[23] These reservation systems contain detailed information about an
individual's travel on a particular flight, including information
provided by the passenger when making a flight reservation. Such
information can include (1) passenger name; (2) reservation date; (3)
travel agency or agent; (4) travel itinerary information; (5) form of
payment; (6) flight number; and (7) seating location.
[24] The purpose of the concept test is limited to identifying the
utility of using commercial data in improving the effectiveness of
comparing passenger information against the terrorist watch list in a
test environment.
[25] In February 2005, we issued a report assessing TSA's measures for
commercial data testing. GAO, Aviation Security: Measures for Testing
the Impact of Using Commercial Data for the Secure Flight Program, GAO-
05-324 (Washington, D.C.: Feb. 23, 2005). We also have an ongoing
follow-up review examining the Secure Flight commercial data testing
process and will report to Congress on our findings.
[26] The Advanced Passenger Information System, maintained by CBP, is
an automated system used to prescreen passengers and crew members prior
to their arrival in or departure from the United States.
[27] According to TSC officials, the TSC is dedicated to maintaining
"the most thorough, accurate, and current information possible" about
individuals in its database in accordance with the Memorandum of
Understanding on the Integration and Use of Screening Information to
Protect Against Terrorism, dated September 16, 2003.
[28] Domestic terrorist nominations come through the Federal Bureau of
Investigations. International terrorist nominations come through the
National Counter Terrorism Center, which was formerly the Terrorist
Threat Integration Center.
[29] GAO has an ongoing review examining the reliability and accuracy
of the TSC terrorist screening database.
[30] Commercial data are maintained by private companies and can
include personally identifiable information that either identifies an
individual or is directly attributed to an individual, such as name,
address, and phone number.
[31] GAO-04-385.
[32] TSA expects commercial data testing to be completed by April 2005.
[33] According to TSA, it currently uses intelligence analysts to
perform similar functions for a variety of other programs.
[34] CAPPS I rules are Sensitive Security Information.
[35] GAO-05-445SU.
[36] According to TSA, one air carrier provided sufficient data for TSA
to test the application of CAPPS I rules. TSA reported that the results
of that test indicated a potential reduction in the number of
selectees. However, because this testing has only recently been
completed, we were unable to independently assess the results.
[37] This is sometimes referred to as identity fraud.
[38] GAO, Identity Theft: Prevalence and Cost Appear to Be Growing, GAO-
02-363 (Washington, D.C.: Mar.1, 2002).
[39] GAO, Technology Assessment: Using Biometrics for Border Security,
GAO-03-174 (Washington, D.C.: Nov. 15, 2002).
[40] DHS is currently revising their policy governing the thresholds
for review by the DHS Investment Review Board.
[41] TSA programs are reviewed by the TSA Investment Review Board prior
to review by the DHS Investment Review Board.
[42] The Transportation Vetting Platform is intended to provide
screening services for a number of DHS programs, such as Secure Flight
and Crew Vetting.
[43] GAO-04-385.
[44] GAO, Transportation Security Administration: High-Level Attention
Needed to Strengthen Acquisition Function, GAO-04-544 (Washington,
D.C.: May 28, 2004).
[45] Government Performance and Results Act of 1993, Pub. L. No. 103-
62, 107 Stat. 285.
[46] We interviewed officials from four air carriers and two aviation
associations to assess TSA's outreach efforts to the airline industry
and to provide industry stakeholders with an opportunity to communicate
perspectives about Secure Flight. In addition to conducting interviews,
we asked officials from air carriers to provide written responses to
questions about the Secure Flight program.
[47] TSA did not identify how many air carriers or privacy groups it
met with to discuss Secure Flight.
[48] Federal Information Security Management Act of 2002, Pub. L. No.
107-347, §§ 301-305, 116 Stat. 2946, 2946-61.
[49] OMB, Management of Federal Information Resources, Office of
Management and Budget Circular A-130.
[50] OMB, Planning, Budgeting, Acquisition, and Management of Capital
Assets, Office of Management and Budget Circular A-11, Part 7 (July
2002).
[51] TSA uses the term expenditure statement to refer to its record of
funds that have been spent.
[52] GAO, Information Technology: Homeland Security Needs to Improve
Entry Exit System Expenditure Planning, GAO-03-563 (Washington, D.C.:
June 9, 2003).
[53] Privacy Act of 1974, Pub. L. No. 93-579, 88 Stat. 1896 (codified
as amended at 5 U.S.C. § 552a).
[54] See 5 U.S.C. § 552a(e)(1).
[55] The E-Government Act of 2002, Pub. L. No. 107-347, 116 Stat. 2899,
requires agencies to conduct a privacy impact assessment before
developing systems that collect, maintain, or disseminate information
in an identifiable form. Further, the Privacy Act requires that an
agency publish a system of records notice in the Federal Register upon
establishment or revision of the existence and character of any system
of records. The system of records notice is to include information such
as the name and location of the system, and "routine uses" of the
records contained in the system. Under the Paperwork Reduction Act of
1995, Pub. L. 104-13, 109 Stat. 163, agencies must submit to the Office
of Management and Budget for approval an information collection
request, which in this case was the proposed order to the airlines to
provide passenger name records.
[56] Portions of the system of records being tested were claimed to be
exempt from 5 U.S.C. § 552a(c)(3),(d), (e)(1), (e)(4)(G) and (H), and
(f) pursuant to 5 U.S.C. § 552a(k)(1) and (k)(2).
[57] See 5 U.S.C. § 552a(k). According to OMB guidance, "upon
determining that a system is to be exempted under this section, the
agency head is required to publish that determination as a rule under
the Administrative Procedure Act, subject to public comment." 40 Fed.
Reg. 28,948, 28,972 (July 9, 1975).
[58] For purposes of this review, we used the eight Fair Information
Practices proposed in 1980 by the Organization for Economic Cooperation
and Development that were endorsed by the U.S. Department of Commerce
in 1981. These practices are collection limitation, purpose
specification, use limitation, data quality, security safeguards,
openness, individual participation, and accountability.
[59] TSA officials stated that under the current process, they reviewed
the reasons three or four individuals were included on the watch list.
However, the current redress process does not contain formal provisions
for this review.
[60] GAO-04-385.
[61] GAO-04-385.
[62] TSA did not require the air carriers to exclude these segments
because of concerns over the cost and time constraints imposed on the
air carriers in providing the data. Because not all air carriers were
able to separate passenger data from European Union flight segments,
TSA officials stated that they excluded these segments when designing
their tests.
[63] Department of Homeland Security Appropriations Act, 2005, Pub. L.
No. 108-334, § 522, 118 Stat. 1298, 1319-20 (2004).
[64] Privacy Act of 1974, Pub. L. No. 93-579, 88 Stat. 1896 (codified
as amended at 5 U.S.C. § 552a).
[65] For purposes of this review, we used the eight Fair Information
Practices proposed in 1980 by the Organization for Economic Cooperation
and Development and that were endorsed by the U.S. Department of
Commerce in 1981. These practices are collection limitation, purpose
specification, use limitation, data quality, security safeguards,
openness, individual participation, and accountability.
[66] The Department of Homeland Security Appropriations Act, 2005,
mandated that the GAO report to the Committees on Appropriations of the
Senate and the House of Representatives on ten issues related to the
development and implementation of Secure Flight, including system
development and security, privacy, redress, oversight and other issues
listed in table 4.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
