Homeland Security
Recommendations to Improve Management of Key Border Security Program Need to Be Implemented Gao ID: GAO-06-296 February 14, 2006The Department of Homeland Security (DHS) has established a program--the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals entering and exiting the United States. US-VISIT uses these identifiers (digital fingerscans and photographs) to screen persons against watch lists and to verify that a visitor is the person who was issued a visa or other travel document. Visitors are also to confirm their departure by having their visas or passports scanned and undergoing fingerscanning at selected air and sea ports of entry (POE). GAO has made many recommendations to improve the program, all of which DHS has agreed to implement. GAO was asked to report on DHS's progress in responding to 18 of these recommendations.
The current status of DHS's implementation of the 18 recommendations is mixed, but progress in critical areas has been slow. DHS has implemented 2 of the recommendations: it defined program staff positions, roles, and responsibilities, and it hired an independent verification and validation contractor. It has also taken steps to implement the other recommendations, partially completing 11 and beginning to implement another 5. In September 2003, GAO reported that the program had not assessed the costs and benefits of Increment 1 (which provides entry capabilities to air and sea POEs) and recommended that the program determine whether proposed increments will produce mission value commensurate with cost. In the latest cost-benefit analysis, dated June 23, 2005, the program identified potential costs and benefits for three alternatives for an air and sea exit solution. However, the analysis does not meet key Office of Management and Budget criteria; for example, it does not include a complete uncertainty analysis, which helps to provide decision makers with perspective on the potential variability of the cost and benefit estimates should circumstances change. GAO reported in May 2004 and February 2005 that system testing was not based on well-defined test plans and recommended that before testing begins, the program develop and approve test plans meeting certain criteria. However, although the latest test plan did cover many required areas (such as the tests to be performed), it did not adequately trace between test cases and the requirements to be verified by testing. Without complete and traceable test plans, the risk is increased that the deployed system will not perform as intended. In May 2004, GAO reported that the program had not assessed its workforce and facility needs for Increment 2B (which extends entry capabilities to the 50 busiest land POEs) and recommended that it do so. Since then, the program evaluated the processing times to issue and process entry/exit forms at 3 of the 50 busiest POEs and concluded that the results showed that no additional staff and only minor facilities modifications were required. However, the scope of the evaluation was limited. Since then, DHS has deployed and implemented Increment 2B capabilities to these 50 POEs, making the collection of predeployment baseline data for these sites impractical. Nonetheless, other alternatives, such as surveying site officials about the increment's impacts, have yet to be explored. Until they are, the program may not be able to accurately project resource needs or make any needed modifications to achieve its goals of minimizing US-VISIT's impact on POE operations, which was the impetus for GAO's recommendation. DHS attributed the pace of progress to competing demands on time and resources. The longer that US-VISIT takes to implement the recommendations, the greater the risk that the program will not meet its stated goals on time and within budget.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-296, Homeland Security: Recommendations to Improve Management of Key Border Security Program Need to Be Implemented
This is the accessible text file for GAO report number GAO-06-296
entitled 'Homeland Security: Recommendations to Improve Management of
Key Border Security Program Need to Be Implemented' which was released
on February 14, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
February 2006:
Homeland Security:
Recommendations to Improve Management of Key Border Security Program
Need to Be Implemented:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-296]:
GAO Highlights:
Highlights of GAO-06-296, a report to congressional requesters:
Why GAO Did This Study:
The Department of Homeland Security (DHS) has established a program”the
U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)”to
collect, maintain, and share information, including biometric
identifiers, on selected foreign nationals entering and exiting the
United States. US-VISIT uses these identifiers (digital fingerscans and
photographs) to screen persons against watch lists and to verify that a
visitor is the person who was issued a visa or other travel document.
Visitors are also to confirm their departure by having their visas or
passports scanned and undergoing fingerscanning at selected air and sea
ports of entry (POE). GAO has made many recommendations to improve the
program, all of which DHS has agreed to implement. GAO was asked to
report on DHS‘s progress in responding to 18 of these recommendations.
What GAO Found:
The current status of DHS‘s implementation of the 18 recommendations is
mixed, but progress in critical areas has been slow. DHS has
implemented 2 of the recommendations: it defined program staff
positions, roles, and responsibilities, and it hired an independent
verification and validation contractor. It has also taken steps to
implement the other recommendations, partially completing 11 and
beginning to implement another 5. ? In September 2003, GAO reported
that the program had not assessed the costs and benefits of Increment 1
(which provides entry capabilities to air and sea POEs) and recommended
that the program determine whether proposed increments will produce
mission value commensurate with cost. In the latest cost-benefit
analysis, dated June 23, 2005, the program identified potential costs
and benefits for three alternatives for an air and sea exit solution.
However, the analysis does not meet key Office of Management and Budget
criteria; for example, it does not include a complete uncertainty
analysis, which helps to provide decision makers with perspective on
the potential variability of the cost and benefit estimates should
circumstances change. ? GAO reported in May 2004 and February 2005 that
system testing was not based on well-defined test plans and recommended
that before testing begins, the program develop and approve test plans
meeting certain criteria. However, although the latest test plan did
cover many required areas (such as the tests to be performed), it did
not adequately trace between test cases and the requirements to be
verified by testing. Without complete and traceable test plans, the
risk is increased that the deployed system will not perform as
intended. ? In May 2004, GAO reported that the program had not assessed
its workforce and facility needs for Increment 2B (which extends entry
capabilities to the 50 busiest land POEs) and recommended that it do
so. Since then, the program evaluated the processing times to issue and
process entry/exit forms at 3 of the 50 busiest POEs and concluded that
the results showed that no additional staff and only minor facilities
modifications were required. However, the scope of the evaluation was
limited. Since then, DHS has deployed and implemented Increment 2B
capabilities to these 50 POEs, making the collection of predeployment
baseline data for these sites impractical. Nonetheless, other
alternatives, such as surveying site officials about the increment‘s
impacts, have yet to be explored. Until they are, the program may not
be able to accurately project resource needs or make any needed
modifications to achieve its goals of minimizing US-VISIT‘s impact on
POE operations, which was the impetus for GAO‘s recommendation. DHS
attributed the pace of progress to competing demands on time and
resources. The longer that US-VISIT takes to implement the
recommendations, the greater the risk that the program will not meet
its stated goals on time and within budget.
What GAO Recommends:
GAO is closing its existing recommendation related to DHS‘s assessment
of Increment 2B and recommending that DHS explore alternative means to
fully assess the impact of US-VISIT entry capabilities on land POEs. In
its comments on a draft of this report, DHS stated that it agreed with
many areas of the report and disagreed with others. It also concurred
with the need to quickly implement GAO‘s open recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-06-296.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
The Status of DHS's Implementation of Our Recommendations Is Mixed:
Conclusions:
Recommendation for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Appendix II: Comments from the Department of Homeland Security:
Appendix III: Description of US-VISIT Processes:
Pre-entry Process:
Entry Process:
Status Management Process:
Exit Process:
Analysis Process:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: US-VISIT Satisfaction of OMB Economic Analysis Criteria:
Table 2: Reduction in Reported Processing Times for Increment 2B Pilot
and Full Deployment:
Table 3: Satisfaction of SEI's 13 Cost-Estimating Criteria:
Figures:
Figure 1: US-VISIT Program Office Structure:
Figure 2: DHS's Progress toward Implementing GAO's 18 Recommendations:
Figure 3: Summary of Program Office Structure, Functions, and Filled
and Vacant Positions:
Figure 4: US-VISIT Process Overview:
Abbreviations:
ACE: Automated Commercial Environment:
ADIS: Arrival Departure Information System:
AIDMS: Automated Identification Management System:
APIS: Advance Passenger Information System:
APMO: Acquisition and Program Management Office:
CBA: cost-benefit analysis:
CBP: Customs and Border Protection:
CLAIMS 3: Computer Linked Application Information Management System:
CMMI: Capability Maturity Model-Integration:
DHS: Department of Homeland Security:
ICE: Immigration and Customs Enforcement:
IDENT: Automated Biometric Identification System:
IV&V: independent verification and validation:
NIST: National Institute of Standards and Technology:
NSEERS: National Security Entry Exit Registration System:
OMB: Office of Management and Budget:
OPM: Office of Personnel Management:
POE: port of entry:
RF: radio frequency:
SEI: Software Engineering Institute:
SEVIS: Student Exchange Visitor Information System:
TECS: Treasury Enforcement Communications Systems:
US-VISIT: U.S. Visitor and Immigrant Status Indicator Technology:
Letter February 14, 2006:
Congressional Requesters:
The U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)
is a multibillion-dollar program of the Department of Homeland Security
(DHS) that is intended to record the entry into and exit from the
United States of selected individuals, verify their identity, and
confirm their compliance with the terms of their admission into and
stay in the United States. The goals of the program are to (1) enhance
the security of our citizens and visitors, (2) facilitate legitimate
travel and trade, (3) ensure the integrity of the U.S. immigration
system, and (4) protect the privacy of our visitors.
Since fiscal year 2002, DHS has been legislatively directed to submit
annual expenditure plans for the program, and we have been directed to
review these plans and issue reports. These reports have, among other
things, identified risks that face the department in delivering
promised program capabilities and benefits on time and within
cost.[Footnote 1] For example, we reported that the program office did
not have the human capital and acquisition process discipline needed to
effectively manage the program. Because of the number and severity of
program management challenges that we identified, we concluded that the
program was risky.
To address program risks, our reports have included 18 recommendations
in such areas as system acquisition process controls, economic
justification, human capital management, cost estimating, and test
management, all of which DHS has agreed to implement.[Footnote 2]
Because of your continued interest in ensuring that DHS is taking the
necessary actions to successfully implement US-VISIT, you asked us to
determine the progress being made in implementing these
recommendations. To achieve this objective, we analyzed program plans,
reports, and system documentation relative to the intent of each of our
recommendations, and we interviewed appropriate DHS and program
officials. (Further details on our objective, scope, and methodology
are provided in app. I.) Our work was performed from August 2005
through December 2005 in accordance with generally accepted government
auditing standards.
Results in Brief:
The current status of DHS's implementation of the 18 recommendations is
mixed, but progress in critical areas has been slow. DHS has
implemented 2 of the recommendations: it defined program staff
positions, roles, and responsibilities, and it hired an independent
verification and validation contractor. It has also taken steps to
implement the other recommendations, partially completing 11 and
beginning to implement another 5. However, although considerable time
has passed since the recommendations were made, key actions have not
yet been taken in such critical areas as (1) assessing security risks
and planning for cost-effective controls to address the risks, (2)
determining--before US-VISIT increments are deployed--whether each
increment will produce mission value commensurate with cost and risk,
and (3) ensuring that each increment is adequately tested. Of the 11
recommendations that are partially implemented, 7 are about 2 years
old, and 4 are about 10 to 19 months old. Of the 5 that are in
progress, 3 are about 10 months old.[Footnote 3] According to the
Program Director, the pace of progress is attributable to competing
demands on time and resources. The longer that US-VISIT takes to
implement the recommendations, the greater the risk that the program
will not meet its stated goals on time and within budget.
DHS provided written comments on a draft of this report. In its
comments, the department stated that it agreed with many areas of the
report and that our recommendations had made US-VISIT a stronger
program. Further, the department stated that while it disagreed with
certain areas of the report, it nevertheless concurred with the need to
implement our open recommendations with all due speed and diligence.
One area of disagreement was regarding the program's ability to
thoroughly assess the impact of US-VISIT entry capabilities on the 50
busiest land port of entry (POE) facilities and staffing levels, an
assessment that we called for in our recommendation. In particular, DHS
stated that since US-VISIT was operational at these POEs, the
collection of predeployment baseline performance data was no longer
practical. In light of these comments, we are making a new
recommendation to the Secretary of DHS that recognizes these facts and
circumstances and that replaces the open recommendation discussed in
this report. This recommendation provides for the department to explore
alternative means of assessing the impact of US-VISIT entry
capabilities on land POE facilities and staffing levels. All of DHS's
comments, along with our responses, are discussed in detail in the
Agency Comments and Our Evaluation section of this report. The comments
are also reprinted in their entirety in appendix II.
Background:
US-VISIT is a governmentwide program intended to enhance the security
of U.S. citizens and visitors, facilitate legitimate travel and trade,
ensure the integrity of the U.S. immigration system, and protect the
privacy of our visitors. Its scope includes the pre-entry, entry,
status, and exit of hundreds of millions of foreign national travelers
who enter and leave the United States at over 300 air, sea, and land
POEs, and the provision of new analytical capabilities across the
overall process.
To achieve its goals, US-VISIT uses biometric information (digital
fingerscans and photographs) to verify identity.[Footnote 4] In many
cases, the US-VISIT process begins overseas at U.S. consular offices,
which collect biometric information from applicants for visas and check
this information against a database of known criminals and suspected
terrorists. When a visitor arrives at a POE, the biometric information
is used to verify that the visitor is the person who was issued the
visa. In addition, at certain sites, visitors are required to confirm
their departure by undergoing US-VISIT exit procedures--that is, having
their visas or passports scanned and undergoing fingerscanning. The
exit confirmation is added to the visitor's travel records to
demonstrate compliance with the terms of admission to the United
States. (App. III provides a detailed description of the pre-entry,
entry, status, exit, and analysis processes.)
Key US-VISIT functions include:
* collecting, maintaining, and sharing information on certain foreign
nationals who enter and exit the United States;
* identifying foreign nationals who (1) have overstayed or violated the
terms of their admission; (2) may be eligible to receive, extend, or
adjust their immigration status; or (3) should be apprehended or
detained by law enforcement officials;
* detecting fraudulent travel documents, verifying traveler identity,
and determining traveler admissibility through the use of biometrics;
and:
* facilitating information sharing and coordination within the
immigration and border management community.
In July 2003, DHS established a program office with responsibility for
managing the acquisition, deployment, operation, and sustainment of the
US-VISIT system and its associated supporting people (e.g., Customs and
Border Protection (CBP) officers), processes (e.g., entry/exit policies
and procedures), and facilities (e.g., inspection booths and lanes), in
coordination with its stakeholders (CBP and the Department of State).
As of October 2005, about $1.4 billion has been appropriated for the
program, and, according to program officials, about $962 million has
been obligated.
Acquisition and Implementation Strategy: A Brief Description:
DHS plans to deliver US-VISIT capability in four increments, with
Increments 1 through 3 being interim, or temporary, solutions that
fulfill legislative mandates to deploy an entry/exit system, and
Increment 4 being the implementation of a long-term vision that is to
incorporate improved business processes, new technology, and
information sharing to create an integrated border management system
for the future. In Increments 1 through 3, the program is building
interfaces among existing ("legacy") systems; enhancing the
capabilities of these systems; and deploying these capabilities to air,
sea, and land POEs. These increments are to be largely acquired and
implemented through existing system contracts and task orders.
In May 2004, DHS awarded an indefinite-delivery/indefinite-
quantity[Footnote 5] prime contract to Accenture and its partners.
According to the contract, the prime contractor will help support the
integration and consolidation of processes, functionality, and data,
and it will develop a strategy to build on the technology and
capabilities already available to produce the strategic solution, while
also assisting the program office in leveraging existing systems and
contractors in deploying the interim solutions.
US-VISIT Is Being Implemented in Four Increments:
Increment 1 concentrates on establishing capabilities at air and sea
POEs. It is divided into two parts--1 and 1B.
* Increment 1 (air and sea entry) includes the electronic capture and
matching of biographic and biometric information (two digital index
fingerscans and a digital photograph) for selected foreign nationals,
including those from visa waiver countries.[Footnote 6] Increment 1 was
deployed on January 5, 2004, for individuals requiring a nonimmigrant
visa to enter the United States, through the modification of pre-
existing systems.[Footnote 7] These modifications accommodated the
collection and maintenance of additional data fields and established
interfaces required to share data among DHS systems in support of entry
processing at 115 airports and 14 seaports.
* Increment 1B (air and sea exit) involves the testing of exit devices
to collect biometric exit data for select foreign nationals at 11
airports and seaports. Three exit alternatives were pilot tested:
* Kiosk--A self-service device (which includes a touch-screen
interface, document scanner, finger scanner, digital camera, and
receipt printer) that captures a digital photograph and fingerprint and
prints out an encoded receipt.
* Mobile device--A hand-held device that is operated by a workstation
attendant;[Footnote 8] it includes a document scanner, finger scanner,
digital camera, and receipt printer and is used to capture a digital
photograph and fingerprint.
* Validator--A hand-held device that is used to capture a digital
photograph and fingerprint, which are then matched to the photograph
and fingerprint captured via the kiosk and encoded in the receipt.
Increment 2 focuses primarily on extending US-VISIT to land POEs. It is
divided into three parts--2A, 2B, and 2C.
* Increment 2A (air, sea, and land) includes the capability to
biometrically compare and authenticate valid machine-readable visas and
other travel and entry documents issued by State and DHS to foreign
nationals at all POEs. Increment 2A was deployed on October 23, 2005,
according to program officials. It also includes the deployment by
October 26, 2006, of technology to read biometrically enabled passports
from visa waiver countries.
* Increment 2B (land entry) redesigns the Increment 1 entry solution
and expands it to the 50 busiest land POEs. The process for issuing
Form I-94[Footnote 9] was redesigned to enable the electronic capture
of biographic, biometric (unless the traveler is exempt), and related
travel documentation for arriving travelers. This increment was
deployed to the busiest 50 U.S. land border POEs as of December 29,
2004. Before Increment 2B, all information on the Form I-94s was
handwritten. The redesigned systems electronically capture the
biographic data included in the travel document. In some cases, the
form is completed by CBP officers, who enter the data electronically
and then print the form.
* Increment 2C is to provide the capability to automatically,
passively, and remotely record the entry and exit of covered
individuals using radio frequency (RF) technology tags at primary
inspection and exit lanes.[Footnote 10] An RF tag that includes a
unique ID number is to be embedded in each Form I-94, thus associating
a unique number with a record in the US-VISIT system for the person
holding that Form I-94. In August 2005, the program office deployed the
technology to five border crossings (three POEs) to verify the
feasibility of using passive RF technology to record traveler entries
and exits via a unique ID number embedded in the CBP Form I-94. The
results of this demonstration are to be reported in February 2006.
Increment 3 extended Increment 2B (land entry) capabilities to 104 land
POEs; this increment was essentially completed as of December 19,
2005.[Footnote 11]
Increment 4 is the strategic US-VISIT program capability, which program
officials stated will likely consist of a further series of incremental
releases or mission capability enhancements that will support business
outcomes. The program reports that it has worked with its prime
contractor and partners to develop this overall vision for the
immigration and border management enterprise.
Increments 1 through 3 include the interfacing and integration of
existing systems and, with Increment 2C, the creation of a new system,
the Automated Identification Management System (AIDMS). The three main
existing systems are as follows:
* The Arrival Departure Information System (ADIS) stores:
* noncitizen traveler arrival and departure data received from air and
sea carrier manifests,
* arrival data captured by CBP officers at air and sea POEs,
* Form I-94 issuance data captured by CBP officers at Increment 2B land
POEs,
* departure information captured at US-VISIT biometric departure pilot
(air and sea) locations,
* pedestrian arrival information and pedestrian and vehicle departure
information captured at Increment 2C POE locations, and:
* status update information provided by the Student and Exchange
Visitor Information System (SEVIS) and the Computer Linked Application
Information Management System (CLAIMS 3) (described below).
ADIS provides record matching, query, and reporting functions.
* The passenger processing component of the Treasury Enforcement
Communications System (TECS) includes two systems: Advance Passenger
Information System (APIS), a system that captures arrival and departure
manifest information provided by air and sea carriers, and the
Interagency Border Inspection System, a system that maintains lookout
data and interfaces with other agencies' databases. CBP officers use
these data as part of the admission process. The results of the
admission decision are recorded in TECS and ADIS.
* The Automated Biometric Identification System (IDENT) collects and
stores biometric data on foreign visitors.
US-VISIT also exchanges biographic information with other DHS systems,
including SEVIS and CLAIMS 3. These two systems contain information on
foreign students and foreign nationals who request benefits, such as a
change of status or extension of stay.
Some of the systems previously described, such as IDENT and the new
AIDMS, are managed by the program office, while some systems are
managed by other organizational entities within DHS. For example, TECS
is managed by CBP, SEVIS is managed by Immigration and Customs
Enforcement, CLAIMS 3 is under United States Citizenship and
Immigration Services, and ADIS is jointly managed by CBP and US-VISIT.
US-VISIT also interfaces with other, non-DHS systems for relevant
purposes, including watch list updates and checks to determine whether
a visa applicant has previously applied for a visa or currently has a
valid U.S. visa. In particular, US-VISIT receives biographic and
biometric information from State's Consular Consolidated Database as
part of the visa application process, and returns fingerscan
information and watch list changes.
Program Management Roles and Responsibilities:
The US-VISIT program office structure includes nine component offices.
Each of the program offices includes a director and subordinate
organizational units, as established by the director. The
responsibilities for each office are stated below. Figure 1 shows the
program office structure, including its nine offices.
Figure 1: US-VISIT Program Office Structure:
[See PDF for image]
[End of figure]
The roles and responsibilities for each of the nine offices include the
following:
* Chief Strategist is responsible for developing and maintaining the
strategic vision, strategic documentation, transition plan, and
business case.
* Budget and Financial Management is responsible for establishing the
program's costs estimates; analysis; and expenditure management
policies, processes, and procedures that are required to implement and
support the program by ensuring proper fiscal planning and execution of
the budget and expenditures.
* Mission Operations Management is responsible for developing business
and operational requirements based on strategic direction provided by
the Office of the Chief Strategist.
* Outreach Management is responsible for enhancing awareness of US-
VISIT requirements among foreign nationals, key domestic audiences, and
internal stakeholders by coordinating outreach to media, third parties,
key influencers, Members of Congress, and the traveling public.
* Information Technology Management is responsible for developing
technical requirements based on strategic direction provided by the
Office of the Chief Strategist and business requirements developed by
the Office of Mission Operations Management.
* Implementation Management is responsible for developing accurate,
measurable schedules and cost estimates for the delivery of mission
systems and capabilities.
* Acquisition and Program Management is responsible for establishing
and managing the execution of program acquisition and management
policies, plans, processes, and procedures.
* Administration and Training is responsible for developing and
administering a human capital plan that includes recruiting, hiring,
training, and retaining a diverse workforce with the competencies
necessary to accomplish the mission.
* Facilities and Engineering Management is responsible for establishing
facilities and environmental policies, procedures, processes, and
guidance required to implement and support the program office.
Our Prior Work Has Resulted in Several Recommendations:
In response to legislative mandate, we have issued four reports on
DHS's annual expenditure plans for US-VISIT.[Footnote 12] Our reports
have, among other things, assessed whether the plans satisfied the
legislative conditions and provided observations on the plans and DHS's
program management. As a result of our assessments, we made 24
recommendations aimed at improving both plans and program management,
all of which DHS has agreed to implement. Of these 24 recommendations,
18 address risks stemming from program management.[Footnote 13]
The Status of DHS's Implementation of Our Recommendations Is Mixed:
The current status of DHS's implementation of our 18 recommendations on
program risks is mixed, but progress in critical areas has been slow.
For example, over 2 years have passed, and the program office has yet
to develop a security plan consistent with federal guidance or to
economically justify its investment in system increments. According to
the Program Director, the pace of progress is attributable to competing
demands on time and resources.
DHS agreed to implement all 18 recommendations. Of these 18, DHS has
completely implemented 2, has partially implemented 11, and is in the
process of implementing another 5. Of the 11 that are partially
implemented, 7 are about 2 years old, and 4 are about 10 to 19 months
old. Of the 5 that are in progress, 3 are about 10 months old.
These 18 recommendations are aimed at strengthening the program's
management effectiveness. The longer that the program takes to
implement the recommendations, the greater the risk that the program
will not meet its goals on time and within budget.
Figure 2 provides an overview of the extent to which each
recommendation has been implemented.The figure is followed by sections
providing details on each recommendation and our assessment of its
implementation status.
Figure 2: DHS's Progress toward Implementing GAO's 18 Recommendations:
[See PDF for image]
[A] A recommendation is completely implemented when documentation
demonstrated that it had been fully addressed.
[B] A recommendation is partially implemented when documentation
indicated that actions were under way to implement it.
[C] A recommendation is in progress when documentation indicated that
actions had been initiated to implement it.
[D] Carnegie Mellon University Software Engineering Institute, Software
Acquisition Capability Maturity Model, Version 1.03 (March 2002).
[E] Automated Commercial Environment is a new trade processing system
planned to support the movement of legitimate imports and exports and
to strengthen border security.
[End of figure]
Development and Implementation of a Security Plan and Performance of a
Privacy Impact Assessment Are Partially Complete:
In June 2003,[Footnote 14] we reported that the Immigration and
Naturalization Service[Footnote 15] had not developed a security plan
and performed a privacy impact assessment for the entry exit program
(as US-VISIT was then known). A security plan and privacy impact
assessment are important to understanding system requirements and
ensuring that the proper safeguards are in place to protect system data
and resources. System acquisition best practices and federal guidance
advocate understanding and defining security and privacy requirements
both early and continuously in a system's life cycle, and effectively
planning for their satisfaction. Accordingly, we recommended that DHS
do the following:
Develop and begin implementing a system security plan, and perform a
privacy impact assessment and use the results of the analysis in near-
term and subsequent system acquisition decision making.
Security Plan:
Since we made the system security plan recommendation about 2˝ years
ago, its implementation has been slow. For example, we reported in
September 2003 and again in May 2004 that the program office had not
developed a security plan. In February 2005, we reported that the
program office had developed a security plan, dated September 2004, and
that this plan was generally consistent with federal guidance.[Footnote
16] That is, the plan provided an overview of system security
requirements, described the controls in place or planned for meeting
those requirements, referred to the applicable documents that prescribe
the roles and responsibilities for managing the US-VISIT component
systems, and addressed security awareness and training. However, the
program office had not conducted a risk assessment or included in the
plan when an assessment would be completed. According to guidance from
the Office of Management and Budget (OMB), the security plan should
describe the methodology that is used to identify system threats and
vulnerabilities and to assess risks, and it should include the date the
risk assessment was completed.
According to program officials, they completed a programwide risk
assessment in December 2005, but have yet to provide a copy of the
assessment to us. Therefore, we cannot confirm that the assessment has
been done, and done properly. The absence of a risk assessment and a
security plan that reflects this assessment is a significant program
weakness. Risk assessments are critical to establishing effective
security controls because they provide the basis for establishing
appropriate policies and selecting cost-effective controls to implement
these policies. Without such an assessment, US-VISIT does not have
adequate assurance that it knows the risks associated with the program
and thus whether it has implemented effective controls to address them.
Notwithstanding these limitations in the security plan, the program
office has begun to implement aspects of its September 2004 security
plan. For example, the Information Systems Security Manager told us
that a security awareness program is established and key personnel have
attended security training.
Privacy Impact Assessment:
Since June 2003, US-VISIT has also developed and periodically updated a
privacy impact assessment. An initial impact assessment was issued in
January 2004, and a revised assessment was issued in September
2004.[Footnote 17] A more recent assessment, dated July 2005, reflects
changes related to Increments 1B and 2C. Each of these assessments is
generally consistent with OMB guidance.[Footnote 18] That is, each of
the assessments addressed most OMB requirements, including the impact
that the system will have on individual privacy, the privacy
consequences of collecting the information, and alternatives considered
to collect and handle information. The most recent impact assessment,
for example, states that three alternatives were considered for
Increment 1B--the kiosk, the mobile device, and the validator (a
combination of the two)--and discusses proposals to mitigate the
privacy risks of all three, such as by limiting the duration of data
retention on the exit devices and using encryption.
However, OMB guidance also requires that privacy impact assessments
developed for systems under development address privacy in relevant
system documentation, including statements of need, functional
requirements documents, and cost-benefit analyses. As we reported about
previous privacy impact assessments, privacy is only partially
addressed in system documentation. For example, the Increment 1B cost-
benefit analysis assesses the privacy risk associated with each exit
alternative, and the Increment 2C business requirements state that all
solutions are to be compliant with privacy laws and regulations and
adhere to US-VISIT privacy policy. However, we did not find privacy in
the Increment 1B business requirements or the Increment 2C functional
requirements. Program officials, including the US-VISIT Privacy
Officer, acknowledged that privacy is not included in the system
documentation, but stated that privacy is considered in the development
of the documentation and that the privacy office reviews key system
documentation at relevant times during the system development life
cycle. Nevertheless, we did not find evidence of privacy being
addressed in the system documentation, and program officials
acknowledged that it was not included.
Until the program performs a risk assessment and fully implements a
security plan that reflects this assessment, it cannot adequately
ensure that US-VISIT is cost-effectively safeguarding assets and data.
Moreover, without reflecting privacy in system documentation, it cannot
adequately ensure that privacy needs are being fully addressed.
Development and Implementation of Key Acquisition Controls Are
Partially Complete:
We reported in September 2003[Footnote 19] that the program office had
not defined key acquisition management controls to support the
acquisition of US-VISIT, and therefore its efforts to acquire, deploy,
operate, and maintain system capabilities were at risk of not
satisfying system requirements and of not meeting benefit expectations
on time and within budget.
The Capability Maturity Model-Integration® (CMMI) developed by Carnegie
Mellon University's Software Engineering Institute (SEI) explicitly
defines process management controls that are recognized hallmarks of
successful organizations and that, if implemented effectively, can
greatly increase the chances of successfully acquiring software-
intensive systems.[Footnote 20] SEI's CMMI model uses capability levels
to assess process maturity.[Footnote 21] Because establishing the basic
acquisition process capabilities, according to SEI, can take on average
about 19 months, we recognized the importance of starting early to
build effective acquisition management capabilities by recommending
that DHS do the following:
Develop and implement a plan for satisfying key acquisition management
controls, including acquisition planning, solicitation, requirements
management, program management, contract tracking and oversight,
evaluation, and transition to support, and implement the controls in
accordance with SEI guidance.
The program office has recently taken foundational steps to establish
key acquisition management controls. For example, it has developed a
process improvement plan, dated May 16, 2005 (about 20 months after our
recommendation), to define and implement these controls. As part of its
improvement program, the program office is implementing a governance
structure for overseeing improvement activities, consisting of three
groups: a Management Steering Group, an Enterprise Process Group, and
Process Action Teams. Specific roles for each of these groups are
described below.
* The Management Steering Group is to provide policy and procedural
guidance and to oversee the entire improvement program. The steering
group is chaired by the US-VISIT Director, with the Deputy Director and
the functional office directors serving as core members.
* The Enterprise Process Group is to provide planning, management, and
operational guidance in day-to-day process improvement activities. The
group is chaired by the process improvement leader and is composed of
individuals from each functional office.
* Process Action Teams are to provide specific process documentation
and to provide implementation support and training services. These
teams are to be active as long as a particular process improvement
initiative is under way. To date, the program office has chartered five
process teams--configuration management, cost analysis, process
development, communications, and policy.
In addition, the program office has recently completed a self-
assessment of its acquisition process maturity, and it plans to use the
assessment results to establish a baseline of its acquisition process
maturity for improvement. According to program officials, the
assessment included 13 key process areas that are generally consistent
with the process areas cited in our recommendation. The program has
ranked these 13 process areas according to their priority, and, for
initial implementation, it plans to focus on the following 6:[Footnote
22]
* Configuration management. Establishing and maintaining the integrity
of the products throughout their life cycle.
* Process and product quality assurance. Taking actions to provide
management with objective insight into the quality of products and
processes.
* Project monitoring and control. Tracking the project's progress so
that appropriate corrective actions can be taken when performance
deviates significantly from plans.
* Project planning. Establishing and maintaining plans for work
activities.
* Requirements management. Managing the requirements and ensuring a
common understanding of the requirements between the customer and the
product developers.
* Risk management. Identifying potential problems before they occur so
that they can be mitigated to minimize any adverse impact.
The improvement plan is currently being updated to reflect the results
of the baseline assessment and to include a detailed work breakdown
structure, process prioritization, and resource estimates. According to
the Director, Acquisition and Program Management Office (APMO), the
goal is to conduct a formal SEI appraisal to assess the capability
level of some or all of the six processes by October 2006.
Notwithstanding the recent steps to begin addressing our
recommendation, much work remains to fully implement key acquisition
management controls. Moreover, effectively implementing these controls
takes considerable time. Therefore, it is important that these
improvement efforts stay on track. Until these processes are
effectively implemented, US-VISIT will be at risk of not delivering
promised capabilities on time and within budget.
Determination and Disclosure of Whether Increments Produce Mission
Value Commensurate with Costs and Risks Are Partially Complete:
In September 2003, we reported that the program had not assessed the
costs and benefits of Increment 1, which is extremely important because
the decision to invest in any capability should be based on reliable
analyses of return on investment. Further, according to OMB guidance,
individual increments of major systems are to be individually supported
by analyses of benefits, cost, and risk.[Footnote 23] Without reliable
analyses, an organization cannot adequately know that a proposed
investment is a prudent and justified use of limited resources.
Accordingly, we recommended that DHS do the following:
Determine whether proposed US-VISIT increments will produce mission
value commensurate with cost and risks and disclose to the Congress
planned actions.
As we reported in September 2003 and again in February 2005,[Footnote
24] the program office did not justify its planned investment in
Increments 1 and 2B, respectively, based on expected return on
investment. Since then, the program has developed a cost-benefit
analysis for Increment 1B.
OMB has issued guidance concerning the analysis needed to justify
investments.[Footnote 25] According to this guidance, such analyses
should meet certain criteria to be considered reasonable. These
criteria include, among other things, comparing alternatives on the
basis of net present value and conducting uncertainty analyses of costs
and benefits. DHS has also issued guidance on such economic analyses
that is consistent with that of OMB.[Footnote 26]
The latest cost-benefit analysis for Increment 1B (dated June 23, 2005)
identifies potential costs and benefits for three exit solutions at air
and sea POEs and provides a general rationale for the viability of the
three alternatives described. This latest analysis meets four of eight
OMB economic analysis criteria. However, it does not, for example,
include a complete uncertainty analysis (i.e., both a sensitivity
analysis and a Monte Carlo simulation[Footnote 27]) for the three exit
alternatives evaluated. That is, the cost-benefit analysis does include
a Monte Carlo simulation, but it does not include a sensitivity
analysis for the three alternatives. An analysis of uncertainty is
important because it provides decision makers with a perspective on the
potential variability of the cost and benefit estimates should the
facts, circumstances, and assumptions change.
Table 1 summarizes our analysis of the extent to which US-VISIT's June
23, 2005, cost-benefit analysis for Increment 1B satisfies eight OMB
criteria.
Table 1: US-VISIT Satisfaction of OMB Economic Analysis Criteria:
Criterion: 1. The cost-benefit analysis clearly explained why the
investment was needed;
Explanation: The analysis should clearly explain the reason why the
investment is needed, that is, why the status quo is unacceptable;
Criterion met? Yes;
GAO analysis: The analysis identifies the need for the investment and
identifies eight key business objectives of the Increment 1B exit
solution.
Criterion: 2. At least two alternatives to the status quo were
considered;
Explanation: At least two meaningful alternatives to the status quo
should be examined to help ensure that the alternative chosen was not
preselected;
Criterion met? Yes;
GAO analysis: The analysis considers three alternatives for the
Increment 1B exit solution: kiosk, mobile, and validator.
Criterion: 3. The general rationale for the cost-benefit analysis,
including each alternative, was discussed;
Explanation: The general rationale for the inclusion of each
alternative considered should be discussed to enable reviewers of the
analysis to gain an understanding of the context for the selection of
one alternative over the others;
Criterion met? Yes;
GAO analysis: The assessment includes the rationale for the judgment
that the three exit alternatives were viable options.
Criterion: 4. The quality of the cost estimate for each alternative was
reasonable;
Explanation: The quality of the cost estimate for each alternative
should be complete and reasonable for a net present value to be
accurate;
Criterion met? No;
GAO analysis: The cost estimates are not complete or reliably derived.
(See later section of this report for detailed analysis.)
Criterion: 5. The quality of the benefits to be realized from each
alternative was reasonable;
Explanation: The quality of the benefit estimate for each alternative
should be complete and reasonable for a net present value to be
calculable and accurate. According to OMB Circular A-94,[A] year-by-
year estimates should be reported to promote independent analysis and
review of those estimates;
Criterion met? No;
GAO analysis: Year-by-year benefit estimates were not reported.
Criterion: 6. Alternatives were compared on the basis of net present
value;
Explanation: The net present value should be calculated because it
consistently allows for the selection of the alternative with the
greatest benefit net of cost;
Criterion met? Yes;
GAO analysis: Net present values were calculated for the three
alternatives. However, the preferred alternative could not be selected
on this basis, in part because the estimated net present value for all
alternatives was negative. OMB guidance presumes that at least one will
be positive, and that the selected alternative will have the greatest
total benefit net of total cost. The alternative with the more
favorable cost-benefit was identified on the basis of its lower labor
intensity (resulting in lower operating and maintenance costs) and
lower risk that personally identifiable information would be
compromised.
Criterion: 7. The proper discount rate for calculating each
alternative's net present value should be used;
Explanation: OMB Circular A-94 provides specific guidance on the choice
of discount rate for evaluating projects whose benefits and costs will
be distributed over time;
Criterion met? No;
GAO analysis: The analysis does not explicitly state the numerical
value of the discount rate used for computing the alternatives' net
present values.
Criterion: 8. A complete uncertainty analysis of cost and benefit was
included;
Explanation: Estimates of costs and benefits are typically uncertain
because of imprecision in both underlying data and modeling
assumptions. Because such uncertainty is basic to virtually any cost-
benefit analysis, its effects should be analyzed and reported. OMB
guidance recommends both Monte Carlo simulation and sensitivity
analysis as uncertainty analysis techniques;
Criterion met? No;
GAO analysis: Although the cost-benefit analysis did include Monte
Carlo simulation results for the three exit alternatives, no
sensitivity analysis was conducted for those alternatives. Instead, the
cost- benefit analysis reports sensitivity analysis results for the
five deployment scenarios.
Source: GAO.
[A] OMB's Circular A-94 is the general guidance for conducting cost-
benefit analyses for the federal government.
[End of table]
It is important that the program adhere to relevant guidance in
developing its incremental cost-benefit analyses. If this is not done,
the reliability of the analyses is diminished, and an adequate basis
for prudent investment decision making does not exist. Moreover, if the
mission value of a proposed investment is not commensurate with costs,
it is vital that this information be fully disclosed to DHS and
congressional decision makers. The underlying intent of our
recommendation is that this information be available to inform such
decisions.
Definition of the Operational Context for US-VISIT Is in Progress:
In September 2003, we reported that key aspects of the larger homeland
security environment in which US-VISIT would need to operate had not
been defined. For example, we stated that certain policy and standards
decisions had not been made (e.g., whether official travel documents
will be required for all persons who enter and exit the country,
including U.S. and Canadian citizens, and how many fingerprints are to
be collected). In the absence of this operational context, program
officials were making assumptions and decisions that, if they proved
inconsistent with subsequent policy or standards decisions, would
require US-VISIT rework. To minimize the impact of these changes, we
recommended that DHS do the following:
Clarify the operational context in which US-VISIT is to operate.
After about 27 months, defining this operational context remains a work
in progress. According to the Chief Strategist, an immigration and
border management strategic plan was drafted in March 2005 that shows
how US-VISIT is aligned with DHS's organizational mission and defines
an overall vision for immigration and border management. This official
stated that this vision provides for an immigration and border
management enterprise that unifies multiple internal departmental and
other external stakeholders with common objectives, strategies,
processes, and infrastructures.
Since the plan was drafted, DHS has reported that other relevant
initiatives have been undertaken, such as the Security and Prosperity
Partnership of North America and the Secure Border Initiative. The
Security and Prosperity Partnership is to, among other things,
establish a common approach to securing the countries of North America-
-the United States, Canada, and Mexico--by, for example, implementing a
border facilitation strategy to build capacity and improve the
legitimate flow of people and cargo at our shared borders. The Secure
Border Initiative is to implement a comprehensive approach to securing
our borders and reducing illegal immigration. According to the Chief
Strategist, while portions of the strategic plan are being incorporated
into these initiatives, these initiatives and their relationship with
US-VISIT are still being defined. We have yet to receive the US-VISIT
strategic plan because, according to program officials, it had not yet
been approved by DHS management.
Until US-VISIT's operational context is fully defined, DHS is
increasing its risk of defining, establishing, and implementing a
program that is duplicative of other programs and not interoperable
with them. This in turn will require rework to address these areas.
While this issue was significant 27 months ago, when we made the
recommendation, it is still more significant now.
Provision of Program Office Resources Is Partially Complete:
We reported in September 2003 that the program had not fully staffed
its program office. Our prior experience with major acquisitions like
US-VISIT shows that to be successful, they need, among other things, to
have adequate resources. Accordingly, we recommended that DHS do the
following:
Ensure that human capital and financial resources are provided to
establish a fully functional and effective program office.
About 2 years later, US-VISIT had filled 102 of its 115 planned
government positions and all of its planned 117 contractor positions.
For the remaining 13 government positions, 5 positions had been
selected (pending completion of security clearances), and recruitment
action was in process for filling the remaining 8 vacancies. According
to the Office of Administration and Training Manager, funding is
available to complete the hiring of all 115 government employees.
Notwithstanding this progress, in February 2005, US-VISIT completed a
workforce analysis and requested additional positions based on the
results. According to program officials, a revised analysis was
submitted in the summer of 2005, but the request has not yet been
approved. Figure 3 shows the program office organization structure and
functions and how many of the 115 positions needed have been filled.
Figure 3: Summary of Program Office Structure, Functions, and Filled
and Vacant Positions:
[See PDF for image]
[End of figure]
Securing necessary resources will be a continuing challenge and an
essential ingredient to the program's ability to acquire, deploy,
operate, and maintain system capabilities on time and within budget.
Definition of Program Office Roles and Responsibilities Has Been
Completed:
We reported in September 2003 that the program had not defined specific
roles and responsibilities for its staff. Our prior experience and
leading practices show that for major acquisitions like US-VISIT to be
successful, program staff need, among other things, to understand what
they are to do, how they relate to each other, and how they fit in
their organization. Accordingly, we recommended that DHS do the
following:
Define program office positions, roles, and responsibilities.
The program office has developed charters for its nine component
offices that include roles and responsibilities for each. For example,
the Acquisition and Program Management Office is responsible, among
other things, for establishing acquisition and program management
policies; coordinating development of configuration management plans
and project schedules, including the integrated milestone schedule; and
developing policies and procedures for guidance and oversight of
systems development and implementation activities. The program has also
defined a set of core competencies (knowledge, skills, and abilities)
for each position. For example, it has defined critical competencies
for program and management analysts that include, among others,
flexibility, interpersonal skills, organizational awareness, oral
communication, problem solving, and teamwork.
These efforts to define position, roles, and responsibilities should
help in managing the program effectively.
Development and Implementation of a Human Capital Strategy Are
Partially Complete:
As previously stated, we reported in September 2003 that US-VISIT had
not fully staffed its program office or defined roles and
responsibilities for its program staff. We observed that prior research
and evaluations of organizations showed that effective human capital
management can help agencies establish and maintain the workforce they
need to accomplish their missions. Accordingly, we recommended that DHS
do the following:
Develop and implement a human capital strategy for the program office
that provides for staffing positions with individuals who have the
appropriate knowledge, skills, and abilities.
In February 2005, we reported that the program office, in conjunction
with the Office of Personnel Management (OPM), developed a draft human
capital plan that employed widely accepted human capital planning tools
and principles. The draft plan included, for example, an action plan
that identified activities, proposed completion dates, and the office
(OPM or the program office) responsible for the action. We also
reported that the program office had completed some of the activities,
such as designating a liaison responsible for ensuring alignment
between departmental and program human capital policies.
Since then, the program office has finalized the human capital plan and
completed more activities. For example, program officials told us that
they have:
* analyzed the program office's workforce to determine diversity
trends, retirement and attrition rates, and mission-critical and
leadership competency gaps;
* updated the program's core competency requirements to ensure
alignment between the program's human capital and business needs;
* developed an orientation program for new employees; and:
* administered competency assessments to incoming employees.
Program officials also told us that they have plans to complete other
activities, such as:
* developing a staffing forecast to inform succession planning;
* analyzing workforce data to maintain strategic focus on preserving
the skills, knowledge, and leadership abilities required for the US-
VISIT program's success; and:
* developing organizational leadership competency models for the
program's senior executive, managerial, and supervisory levels.
In addition, the officials said that several activities in the plan
have not been completed, such as assessing the extent of any current
employees' competency gaps and developing a competency-based listing of
training courses. These officials said that the reason these activities
have not been completed is that they are related to the department's
new human capital initiative, MAXHR, which is to provide greater
flexibility and accountability in the way employees are paid,
developed, evaluated, afforded due process, and represented by labor
organizations. MAXHRis to include the development of departmentwide
competencies. Because of this, the officials told us that it could
potentially impact the program's ongoing competency-related activities.
As a result, these officials said that they are coordinating these
activities closely with the department as it develops and implements
this new initiative, which is currently being reviewed by the DHS
Deputy Secretary for approval.
Until US-VISIT fully implements a comprehensive human capital strategy,
it will continue to risk not having staff with the right skills and
abilities to successfully execute the program.
Defining Performance Standards for US-VISIT Increments Is Partially
Complete:
We reported in September 2003 that the operational performance of
initial system increments was largely dependent on the performance of
existing systems that were to be interfaced to create these increments.
For example, we said that the performance of an increment will be
constrained by the availability and downtime of the existing systems
that it includes. Accordingly, we recommended that DHS do the
following:
Define performance standards for each increment that are measurable and
reflect the limitations imposed by relying on existing systems.
In February 2005 (17 months later), we reported that several technical
performance standards for Increments 1 and 2B had been defined, but
that it was not clear that these standards reflected the limitations
imposed by the reliance on existing systems. Since then, for the
Increment 2C Proof of Concept (Phase 1), the program office has defined
certain other performance standards. For example, the functional
requirements document for Increment 2C (Phase 1) defines several
technical performance standards, including reliability, recoverability,
and availability. For each, the document states that the performance
standard is largely dependent on those of Increment 2B. More
specifically, the document states that Phase 1 system availability is
largely dependent upon the individual and collective availability of
the current systems. The document also states that the Increment 2C
components shall have an aggregated availability greater than or equal
to 97.5 percent. However, the document does not contain sufficient
information to determine whether these performance standards actually
reflect the limitations imposed by reliance on existing systems.
To further develop performance standards, the program office has
prepared a Performance Engineering Plan, dated March 31, 2005, that
links US-VISIT performance engineering activities to its System
Development Life Cycle. Further, the plan (1) provides a framework to
be used to align its business, application, and infrastructure
performance goals and measures; (2) describes an approach to translate
business goals into operational measures, and then to quantitative
metrics; and (3) identifies system performance measurement areas
(effectiveness, efficiency, reliability, and availability). According
to program officials, they intend to establish a group to develop
action plans for implementing the engineering plan, but did not have a
time frame for doing so.
Without defining performance standards that reflect the limitations of
the existing systems upon which US-VISIT relies, the program lacks the
ability to identify and effectively address performance shortfalls.
Development and Implementation of a Risk Management Plan Are Partially
Complete:
In September 2003, we reported that US-VISIT was a risky undertaking
because of several factors inherent to the program, such as its large
scope and complexity, as well as because of various program management
weaknesses. We concluded that these risks, if not effectively managed,
would likely cause program cost, schedule, and performance problems.
Risk management is a continuous, forward-looking process that is
intended either to prevent such problems from occurring or to minimize
their impact if they occur by proactively identifying risks,
implementing risk mitigation strategies, and measuring and disclosing
progress in doing so. Because of the importance of effectively managing
program risks, we recommended that DHS do the following:
Develop and implement a risk management plan and ensure that all high
risks and their status are reported regularly to the executive body.
About 2 years later, the program office has developed and has begun
implementing a risk management plan. The plan, which was approved in
September 2005, includes, among other things, a process for
identifying, analyzing, handling, and monitoring risk. It also defines
the governance structure to be used in overseeing and managing the
process. The program also maintains a risk database, which includes,
among other things, a description of the risk, its priority (e.g.,
high, medium, or low), and its mitigation strategy. According to
program officials, the database is currently available to program
management and staff.
The program has also begun implementing its risk management plan. For
example, it has established a Risk Review Board, Risk Review Council,
and Risk Owners to govern its risk activities. The roles and
responsibilities are described below.
* The Risk Review Board directs all risk governance within the program
and provides the mechanism to escalate/transfer the consideration of
risks to program governing boards and to organizations external to the
program.
* The Risk Review Council oversees and manages program-related risks
that are significant, controversial, or cross-project or that may
require escalation to the Risk Review Board.
* Risk Owners analyze, handle, and monitor risks.
However, full implementation of the risk management plan has yet to
occur. As part of its CMMI process maturity baseline self-assessment
(previously discussed), the program office found that the risk
management process detailed in its plan was not being consistently
applied across the program. In response, according to program
officials, they have developed risk management training and began
conducting training sessions in November 2005. These officials also
stated that the Risk Review Board, where risks are reviewed with
program executives, has been meeting monthly since September 2005.
With respect to regular risk reports to program executives, the plan
includes thresholds for escalating risks within the risk governance
structure and to DHS governance entities. For example, risks are to be
elevated to the Risk Review Board when the cost of the project exceeds
more than 5 percent of the project baseline cost, the schedule slippage
exceeds more than 5 percent of the baseline schedule, major areas of
scope are affected, or quality reduction requires approval. However,
program officials stated that these thresholds are not currently being
applied. They further stated that although the plan allows for
escalation of risks to officials outside the program office, doing so
is at the discretion of the Program Director; in addition, according to
these officials, although high risks are not routinely escalated
outside the program, selected high risks have been disclosed to the
Assistant Secretary for Policy in weekly program status reports. As of
December 5, 2005, the Program Director proposed submitting monthly
reports of high-priority risks and issues through the Assistant
Secretary for Policy to the Deputy Secretary.
Until US-VISIT fully implements its risk management plan and process,
it cannot be assured that all program risks are being identified and
managed in order to effectively mitigate any negative impact on the
program's ability to deliver promised capabilities on time and within
budget.
Development of Test Plans Is Partially Complete:
We reported in May 2004, and again in February 2005, that system
testing was not based on well-defined test plans, and thus the quality
of testing being performed was at risk.[Footnote 28] The purpose of
system testing is to identify and correct system defects (i.e., unmet
system functional, performance, and interface requirements) and thereby
obtain reasonable assurance that the system performs as specified
before it is deployed and operationally used. To be effective, testing
activities should be planned and implemented in a structured and
disciplined fashion. Among other things, this includes developing
effective test plans to guide the testing activities and ensuring that
test plans are developed and approved before test execution. According
to relevant systems development guidance, an effective test plan (1)
specifies the test environment; (2) describes each test to be
performed, including test controls, inputs, and expected outputs; (3)
defines the test procedures to be followed in conducting the tests; and
(4) provides traceability between the test cases and the requirements
to be verified by the testing. Because these criteria were not being
met, we recommended that DHS do the following:
Develop and approve test plans before testing begins that (1) specify
the test environment; (2) describe each test to be performed, including
test controls, inputs, and expected outputs; (3) define the test
procedures to be followed in conducting the tests; and (4) provide
traceability between test cases and the requirements to be verified by
the testing.
About 19 months later, the quality of the system test plans, and thus
system testing, is still problematic. To the program's credit, the test
plans for the Increment 2C Proof of Concept (Phase 1), dated June 28,
2005, satisfied part of our recommendation. Specifically, the test plan
for this increment was approved on June 30, 2005, and, according to
program officials, testing began on July 5, 2005. Further, the test
plan described, for example, the scope, complexity, and completeness of
the test environment, and it described the tests to be performed,
including a high-level description of controls, inputs, and outputs,
and it identified test procedures to be performed.
However, the test plan did not adequately trace between test cases and
the requirements to be verified by testing. For example, 300 of the 438
functional requirements, or about 70 percent of the requirements that
we analyzed, did not have specific references to test cases.
In addition, we identified traceability inconsistencies, including the
following:
* One requirement was mapped to over 50 test cases, but none of the 50
cases referenced the requirement.
* One requirement was mapped to a group of test cases in the
traceability matrix, but several of the test cases to which the
requirement was mapped did not reference the requirement, and several
test cases referenced the requirement and were not included in the
traceability matrix.
* One requirement was mapped to all but one of the test cases within a
particular group of test cases, but that test case did refer to the
requirement.
Time and resources were identified as the reasons that test plans have
not been complete. Specifically, program officials stated that
milestones do not permit existing testing/quality personnel the time
required to adequately review testing documents.[Footnote 29] According
to these officials, even when the start of testing activities is
delayed because, for example, requirements definition or product
development takes longer than anticipated, testing milestones are not
extended.
Without complete test plans, the program does not have adequate
assurance that the system is being fully tested, and thus unnecessarily
assumes the risk that system defects will not be detected and addressed
before the system is deployed. This means that the system may not
perform as intended when deployed, and defects will not be addressed
until late in the systems development cycle, when they are more
difficult and time-consuming to fix. As we previously reported, this
has happened: postdeployment system interface problems surfaced for
Increment 1, and manual work-arounds had to be implemented after the
system was deployed.
Assessment of the Impact of Increment 2B on Workforce Levels and
Facilities Is Partially Complete:
We reported in May 2004 that the program had not assessed its workforce
and facility needs for Increment 2B. Because of this, we questioned the
validity of the program's workforce and facility assumptions used to
develop its workforce and facility plans, noting that the program
lacked a basis for determining whether its assumptions and thus its
plans were adequate. Accordingly, we recommended that DHS do the
following:
Assess the full impact of Increment 2B on land POE workforce levels and
facilities, including performing appropriate modeling exercises.
Seven months later, the program office evaluated Increment 2B
operational performance. The purpose of the evaluation was to determine
the effectiveness of Increment 2B performance at the 50 busiest land
POEs. To assist in the evaluation, the program office established a
baseline for comparing the average Form I-94 or Form I-94W[Footnote 30]
issuance processing times at 3 of the 50 POEs where processing times
were to be evaluated.[Footnote 31] The program office then conducted
two evaluations of the processing times at the 3 POEs following
Increment 2B deployment. The first was in December 2004, after
Increment 2B was deployed to these sites as a pilot, and the second was
in February 2005, after Increment 2B was deployed to all 50 POEs. The
evaluation results showed that the average processing times decreased
for all 3 sites. Table 2 compares the results of the two evaluations
and the baseline.
Table 2: Reduction in Reported Processing Times for Increment 2B Pilot
and Full Deployment:
Pilot site: Douglas, Arizona;
Baseline (October 2004): 4 minutes, 16 seconds;
Pilot: Decrease in time from baseline (December 2004): -47 seconds;
Full deployment: Change in time from pilot (February 2005): - 17
seconds.
Pilot site: Laredo, Texas;
Baseline (October 2004): 12 minutes, 10 seconds;
Pilot: Decrease in time from baseline (December 2004): -9 minutes, 37
seconds;
Full deployment: Change in time from pilot (February 2005): -15
seconds.
Pilot site: Port Huron, Michigan;
Baseline (October 2004): 11 minutes, 42 seconds;
Pilot: Decrease in time from baseline (December 2004): -1 minutes, 51
seconds;
Full deployment: Change in time from pilot (February 2005): +7 seconds.
Source: GAO analysis of DHS data.
[End of table]
According to program officials, these evaluations supported the
workforce and facilities planning assumption that no additional staff
were required to support deployment of Increment 2B, and that minimal
modifications to interior workspace were required to accommodate
biometric capture devices and printers and to install electrical
circuits. These officials stated that modifications to existing officer
training and interior space were the only changes needed.
However, the scope of the evaluation was too limited to satisfy the
evaluation's stated purpose or our recommendation for assessing the
full impact of Increment 2B. Specifically, program officials stated
that the evaluation focused on the time to process Form I-94s and not
on operational effectiveness, including workforce impacts and traveler
waiting time. Second, the 3 sites were selected, according to program
officials, on the basis of a number of factors, including whether the
sites already had sufficient staff to support the pilot. Selecting
sites on the basis of this factor could affect the results and
presupposes that not all POEs have the staff needed to support
Increment 2B. Third, evaluation conditions were not always held
constant. For example, fewer workstations were used to process
travelers in establishing the baseline processing times at 2 of the
POEs--Port Huron (9 versus 14) and Douglas (4 versus 6)--than were used
during the pilot evaluations.
Moreover, CBP officials from 1 POE, which was not an evaluation site,
told us that US-VISIT has actually lengthened processing times. (San
Ysidro processes the highest volume of travelers of all land POEs.)
While these officials did not provide specific data to support this
statement, it nevertheless raises questions about the potential impact
of Increment 2B on the 47 sites that were not evaluated.
It is important that the impact of Increment 2B on workforce and
facilities be fully assessed. Since we made our recommendation,
Increment 2B deployment and operational facts and circumstances have
materially changed, making the implementation of our recommendation
using predeployment baseline data for the other 47 sites impractical.
Nevertheless, other alternatives, such as surveying officials at these
sites to better understand the increment's impact on workforce levels
and facilities, have yet to be explored. Until they are, the program
may not be able to accurately project resource needs or make required
modifications to achieve its goals of minimizing US-VISIT's impact on
POE processing times.
Implementation of Configuration Management Practices Is in Progress:
We reported in May 2004 that US-VISIT had not established effective
configuration management practices. Configuration management
establishes and maintains the integrity of system components and items
(e.g., hardware, software, and documentation). A key ingredient is a
change control board to evaluate and approve proposed configuration
changes. Accordingly, we concluded that the program did not have
adequate assurance that approved system changes were actually made, and
that changes made to the component systems (for non-US-VISIT purposes)
did not interfere with US-VISIT functionality. Accordingly, we
recommended that DHS do the following:
Implement effective configuration management practices, including
establishing a US-VISIT change control board to manage and oversee
system changes.
After 19 months, US-VISIT has begun implementing configuration
management practices. To its credit, the program recently issued a
configuration management policy (September 2005) and prepared a draft
configuration management plan (August 2005). The policy contains
guiding principles, direction, and expectations for planning and
performing configuration management, and includes activities,
authorities, and responsibilities. The draft plan describes the
configuration management governance structure, including organizational
entities and their responsibilities, the processes and procedures to be
applied, and how controls are to be applied to products. The governance
structure includes the Executive Configuration Control Board and the
Configuration Management Impact Review Team. According to its charter,
the configuration control board is responsible for determining the
status of requested configuration changes and resolving any conflicts
related to those changes for US-VISIT-managed systems (i.e., not for US-
VISIT component systems managed by other DHS organizations). The Impact
Review Team, which reports to the board, is responsible for reviewing
requests for system changes and submitting a recommendation to the
appropriate change review authority (i.e., either the US-VISIT control
board or the control board in the DHS organization that manages the
component system). According to program officials, for US-VISIT-
managed systems, the review authority is the Executive Configuration
Control Board. For other systems, such as TECS (which CBP manages), the
US-VISIT review team may submit a recommendation to the appropriate
control board (in this case, the CBP Control Board).
The APMO director stated that the planned configuration management
program is intended to complement rather than replace the configuration
management programs for the legacy systems. That is, change requests
approved by the US-VISIT Executive Configuration Control Board that
require changes to a legacy system will be coordinated with the board
having responsibility for that system. This means, however, that
changes to component systems (e.g., IDENT, ADIS, and TECS) that are
initiated and approved by another DHS organization, and that could
affect US-VISIT performance, are not subject to US-VISIT configuration
management processes and are not also being examined and approved by
the US-VISIT control board. This lack of US-VISIT control was the
impetus for our recommendation.
Although US-VISIT has recently taken steps to begin addressing our
recommendation, the program still does not adequately control changes
to the component systems upon which US-VISIT performance depends. Until
programwide configuration management practices are implemented, the
program does not have an effective means for ensuring that approved
system changes are actually made and that changes made to the component
systems for non-US-VISIT purposes do not compromise US-VISIT
functionality and performance.
Efforts to Ensure the Independence of the Verification and Validation
Contractor Are Complete:
We reported in May 2004 that the program office's independent
verification and validation (IV&V) contractor was not independent of
the products and processes that it was verifying and validating. The
purpose of IV&V is to provide management with objective insight into
the program's processes and associated work products. Its use is a
recognized best practice for large and complex system development and
acquisition projects like US-VISIT. To be effective, the verification
and validation function is to be performed by an entity that is
independent of the processes and products that are being reviewed.
Accordingly, we recommended that DHS do the following:
Ensure the independence of the IV&V contractor.
In July 2005, the program office issued a new contract for IV&V
services. To ensure the contactor's independence, the program office
(1) required that IV&V contract bidders be independent of the
development and integration contractors; (2) reviewed each of the
bidder's affiliations with the prime contract; (3) included provisions
in the contract that prohibit the contractor from soliciting,
proposing, or being awarded work (other than IV&V services) for the
program; (4) required all contractor personnel to certify that they do
not have any conflicts of interest; and (5) ensured that the
contractor's management plan (Oct. 17, 2005) describes how the
contractor will ensure technical, managerial, and financial
independence.
Such steps, if effectively enforced, should adequately ensure that
verification and validation activities are performed in an objective
manner and, thus, should provide valuable assistance to program
managers and decision makers.
Development of a Plan to Address Open Recommendations Is Partially
Complete:
We reported in May 2004 that US-VISIT's overall progress on
implementing our recommendations had been slow, and considerable work
remained to fully address them. As we also noted, given that most of
our recommendations focused on fundamental limitations in US-VISIT's
ability to manage the program, it was important to implement the
recommendations quickly and completely. Accordingly, we recommended
that DHS do the following:
Develop a plan, including explicit tasks and milestones, for
implementing all of our open recommendations and periodically report to
the DHS Secretary and Under Secretary on progress in implementing this
plan; and report this progress, including reasons for delays, in all
future expenditure plans.
About 19 months after our recommendation, the program assigned
responsibility to specific individuals for preparing a plan, including
specific actions and milestones, to address each recommendation. In
addition, it developed a report that identifies the responsible person
for each recommendation and summarizes progress made in implementing
each. The program office provided this report for the first time to the
DHS Deputy Secretary on October 3, 2005, and plans to forward
subsequent reports every 6 months.
However, the report's description of progress on 4 recommendations is
inconsistent with our assessment, as discussed below:
* First, the report states that the program completed a privacy impact
assessment that is in full compliance with OMB guidance. As previously
discussed, an assessment has been developed, but OMB guidance requires
that these assessments for systems under development (such as Increment
2C) address privacy in the system's documentation. Increment 2C systems
documentation does not address privacy and therefore is not fully
compliant with OMB guidance.
* Second, the report states that a human capital strategy has been
completed. However, as previously discussed, several of the activities
in the human capital plan have yet to be implemented. For example, the
program has not developed a staffing forecast to inform succession
planning.
* Third, the report states that the impact of Increment 2B on land POE
workforce levels and facilities has been fully assessed. However, as we
previously stated, the scope of the evaluations was not sufficient to
satisfy our recommendation. For example, program officials stated that
the evaluation focused on the time to process Form I-94s and not on
operational effectiveness, including workforce impacts and traveler
waiting time. Moreover, officials at the largest land POE told us that
the effect of Increment 2B was the opposite of that reported in the
pilot results.
* Fourth, the report states that the program has partially completed
implementing configuration management practices. However, as previously
discussed, the program office has yet to implement practices or
establish a configuration control board with authority over all changes
affecting US-VISIT functionality and performance, including those made
to component systems for non-US-VISIT purposes, which was the intent of
our recommendation.
In addition, the report does not specifically describe progress against
11 of our other recommendations, so that we could not determine whether
the program's assessment is consistent with ours (described in this
report). For example, we recommended that the program reassess plans
for deploying an exit capability to ensure that the scope of the exit
pilot provides for adequate evaluation of alternative solutions. The
report states that the program office has completed exit testing and
has forwarded the exit evaluation report to the Deputy Secretary for a
decision. However, it does not state whether the program office had
expanded the scope or time frames of the pilot.
Fully understanding and disclosing progress against our recommendations
are essential to building the capability needed to effectively manage
the program, and to ensuring that key decision makers have the
information needed to make well-informed choices among competing
investment options.
Establishment of Effective Cost-Estimating Practices Is in Progress:
We reported in February 2005 that US-VISIT had not followed effective
practices to develop cost estimates for its system increments, and thus
the reliability of its cost estimates was questionable.[Footnote 32]
Such cost-estimating practices are embedded in the 13 criteria in SEI's
checklist for determining the reliability of cost estimates.[Footnote
33] Of these 13 criteria, we reported in February 2005 that the
program's cost estimate met 2, partially met 6, and did not meet 5.
Accordingly, we recommended that DHS do the following:
Follow effective practices for estimating the costs of future
increments.
The latest US-VISIT-related cost estimate is for Increment 1B. This
estimate is in the June 2005 cost-benefit analysis for Increment 1B and
establishes the costs associated with three exit solutions for air and
sea POEs. As was the case for the estimate described in our February
2005 report, this latest estimate also did not meet all 13 criteria,
meeting 3 and partially meeting another 5.[Footnote 34] For example,
these estimates did not include a detailed work breakdown structure and
omitted important cost elements, such as system testing. A work
breakdown structure serves to organize and define the work to be
performed, so that associated costs can be identified and estimated.
Thus, it provides a reliable basis for ensuring that the estimates
include all relevant costs. In addition, the uncertainties associated
with the Increment 1B cost estimate were not identified. An uncertainty
analysis provides the basis for adjusting these estimates to reflect
unknown facts and circumstances that could affect costs and identifies
the risk associated with the cost estimate. Table 3 summarizes our
analysis of the extent to which US-VISIT's Increment 1B cost estimates
satisfy SEI's 13 criteria.
Table 3: Satisfaction of SEI's 13 Cost-Estimating Criteria:
Criterion: 1. The objectives of the program are stated in writing;
Explanation: The objectives of the program should be clearly and
concisely stated for the cost estimator to use;
Criterion met[A]? Yes;
GAO analysis: The objectives of the program were clearly stated.
Specifically, the objectives are to provide a more complete traveler
history and to capture travelers' biometric and biographic data.
Criterion: 2. The life cycle to which the estimate applies is clearly
defined;
Explanation: The life cycle should be clearly defined to ensure that
the full cost of the program is captured--that is, all direct and
indirect costs for planning, procurement, operations and maintenance,
and disposal;
Criterion met[A]? Partially;
GAO analysis: The life cycle was not clearly defined to ensure that the
full cost of the program was included. For example, the analysis did
not include evidence that software maintenance costs were included in
the cost estimate.
Criterion: 3. The task has been appropriately sized;
Explanation: An appropriate sizing metric should be used in the
development of the estimate, such as the amount of software to be
developed and the amount of software to be revised;
Criterion met[A]? No;
GAO analysis: The program office provided no evidence to demonstrate
that an appropriate sizing mechanism was used, and program officials
stated that they had not collected these data.
Criterion: 4. The estimated cost and schedule are consistent with
demonstrated accomplishments on other projects;
Explanation: Estimates should be validated by being related back to
demonstrated and documented performance on completed projects;
Criterion met[A]? Partially;
GAO analysis: Officials stated that pilot data were used to develop the
estimate. They stated they extrapolated pilot data to estimate costs
for all Increment 1B sites; however, they further stated that there
were no previous projects with which to compare the results to see if
they were consistent.
Criterion: 5. A written summary of parameter values and their
rationales accompanies the estimate;
Explanation: If a parametric equation was used to generate the
estimate, the parameters that feed the equation should be provided,
along with an explanation of why they were chosen;
Criterion met[A]? Partially;
GAO analysis: High-level cost categories, such as labor, information
technology, facilities, and other costs, were identified, but detailed
parameters used to develop the estimate, such as number of software
lines of code, which would be relevant to software maintenance costs,
were not provided in the analysis.
Criterion: 6. Assumptions have been identified and explained;
Explanation: Assumptions regarding issues such as schedule, quantity,
technology, development processes, manufacturing techniques, software
language, etc., should be understood and documented;
Criterion met[A]? Yes;
GAO analysis: General cost assumptions are identified and explained, as
well as assumptions for workforce, information technology, training,
and facilities.
Criterion: 7. A structured process, such as a template or format, has
been used to ensure that key factors have not been overlooked;
Explanation: A work breakdown structure or similar structure that
organizes, defines, and graphically displays the individual work units
to be performed should be used. The structure should be revised over
time as more information becomes known about the work to be performed;
Criterion met[A]? Partially;
GAO analysis: The analysis included four high-level cost categories
(labor, facilities, operations and maintenance, and information
technology), but it did not include a detailed work breakdown structure
and omitted important cost elements, such as system testing.
Criterion: 8. Uncertainties in parameter values have been identified
and quantified;
Explanation: For all major cost drivers, an uncertainty analysis should
be performed to recognize and reflect the risk associated with the cost
estimate;
Criterion met[A]? Partially;
GAO analysis: A risk analysis was performed, but this analysis did not
identify detailed parameter values.
Criterion: 9. If a dictated schedule has been imposed, an estimate of
the normal schedule has been compared to the additional expenditures
required to meet the dictated schedule;
Explanation: Managers should be informed of all potential cost savings
associated with alternative schedules;
Criterion met[A]? N/A;
GAO analysis: Program officials stated that the Increment 1B schedule
was not dictated.
Criterion: 10. If more than one cost model or estimating approach has
been used, any differences in results have been analyzed and explained;
Explanation: The primary methodology or cost model results should be
compared with any secondary methodology (e.g., cross checks) to ensure
consistency;
Criterion met[A]? No;
GAO analysis: No evidence of a secondary cost model was included in the
analysis, and program officials stated that they did not use a second
model.
Criterion: 11. Estimators independent of the performing organization
concurred with the reasonableness of the parameter values and
estimating methodology;
Explanation: The purpose of an independent estimate is to determine the
reasonableness of the parameter values based on an unbiased
perspective. This approach usually results in a more accurate estimate
because it allows for better insight into program risks;
Criterion met[A]? No;
GAO analysis: Program officials stated that the estimate was not
independently reviewed.
Criterion: 12. Estimates are current;
Explanation: Estimates are updated whenever changes to requirements
affect cost or schedule, constraints, and resources, or when priorities
change;
Criterion met[A]? Yes;
GAO analysis: Estimates reflected current conditions.
Criterion: 13. The results of the estimate have been integrated with
project planning and tracking;
Explanation: Plans are reviewed and updated whenever estimates change,
and estimates used for project planning are also used as baselines for
project tracking;
Criterion met[A]? No;
GAO analysis: Program officials stated that the results of the estimate
have not been incorporated with project planning.
Source: GAO.
[A] We assessed each of the criteria as satisfied (US-VISIT provided
substantiating evidence for the criterion), partially satisfied (US-
VISIT provided partial evidence, including testimonial evidence, for
the criterion), or not satisfied (no evidence was found for the
criterion).
[End of table]
Program officials stated that they recognize the importance of
developing reliable cost estimates and have initiated actions to more
reliably estimate the costs of future increments. For example, as part
of its process improvement program, the program has chartered a cost-
analysis process action team, which is to develop, document, and
implement a cost-analysis policy, process, and plan for the program.
Program officials also stated that they have hired additional
contracting staff with cost-estimating experience.
Strengthening the program's cost-estimating capability is extremely
important. The absence of reliable cost estimates, among other things,
prevents the development of reliable economic justification for program
decisions and impedes effective performance measurement.
Reassessment of Plans for Deploying the Exit Capability Is Partially
Complete:
In February 2005, we reported that US-VISIT had not adequately planned
for evaluating the Increment 1B exit alternative because its exit pilot
evaluation's scope and timeline were compressed. Accordingly, we
recommended that DHS do the following:
Reassess plans for deploying an exit capability to ensure that the
scope of the exit pilot provides for adequate evaluation of alternative
solutions and better ensures that the exit solution selected is in the
best interest of the program.
Over the last 10 months, the program office has taken actions to expand
the scope and time frames of the pilot. For example, it extended the
pilot from 5 to 11 POEs--9 airports and 2 seaports.[Footnote 35] It
also extended the time frame for data collection and evaluation to
April 2005, which is about 7 months beyond the date for which all exit
pilot evaluation tasks were to be completed. Further, according to
program officials, they achieved the target sample sizes necessary to
have a 95 percent confidence level.
Notwithstanding the expanded scope of the pilot, questions remain about
whether the exit alternatives have been evaluated sufficiently to
permit selection of the best exit solution for national deployment. For
example, each of the three exit alternatives was evaluated against
three criteria, including compliance with the US-VISIT exit process
(i.e., foreign travelers providing information as they exit the United
States).[Footnote 36] However, across the three alternatives, the
average compliance with this process was only 24 percent, which raises
questions as to the effectiveness of the three alternatives.[Footnote
37] The evaluation report cites several reasons for the low compliance
rate, including that compliance during the pilot was voluntary. The
report further concludes that national deployment of the exit solution
will not have the desired compliance rate unless the exit process
incorporates an enforcement mechanism, such as not allowing persons to
reenter the United States if they do not comply with the exit process.
Although an enforcement mechanism might indeed improve compliance,
program officials stated that no formal evaluation has been conducted
of enforcement mechanisms or their effect on compliance. The program
director stated that he agrees that additional evaluation is needed to
assess the impact of implementing potential enforcement mechanisms and
plans to do so.
Until the program office adequately evaluates the exit alternatives and
knows whether the alternative to be selected will be effective, the
program office will not be in a position to select the exit solution
that is in the best interest of the program. This is very important
because without an effective exit capability, the benefits and the
mission value of US-VISIT are greatly diminished.
Development and Implementation of Capacity Management Processes Are in
Progress:
We reported in February 2005 that the overall capacity of the system
was not being effectively managed. At that time, US-VISIT, which
comprises several legacy systems, was relying on the capacity
management activities of these systems. It was not focused on the
capacity requirements and performance of the collective systems that
make up US-VISIT. This approach increases the risk that the system may
not be properly designed and configured for efficient performance, and
that it has insufficient processing and storage capacity for current,
future, and unpredictable workload requirements. Accordingly, we
recommended that DHS do the following:
Develop and implement processes for managing the capacity of the US-
VISIT system.
According to program officials, they have initiated efforts to develop
a capacity management process, including a high-level description of
the necessary steps, such as identifying tools needed to implement the
process. However, a plan, including specific tasks and milestones for
developing and implementing capacity management processes, has not yet
been developed.
Until the program office develops a programwide capacity management
program, it increases the risk that US-VISIT may not be able to
adequately support program mission needs.
Identification of ACE and US-VISIT Relationships and Dependencies Is in
Progress:
We reported in February 2005 that the program office recognized that US-
VISIT and the Automated Commercial Environment (ACE)[Footnote 38] have
related missions and operational environments. In addition, US- VISIT
and ACE could potentially develop, deploy, and use common information
technology infrastructures and services. We also reported that managing
this relationship has not been a priority. Accordingly, we recommended
that DHS do the following:
Make understanding the relationships and dependencies between the US-
VISIT and ACE programs a priority matter, and report periodically to
the Under Secretary on progress in doing so.
US-VISIT and ACE managers met in February 2004, to identify potential
areas for collaboration between the two programs and to clarify how the
programs could best support the DHS mission and provide officers with
the information and tools they need. According to program officials,
they have established a US-VISIT/ACE integrated project team to, among
other things, ensure that the two programs are programmatically and
technically aligned. The team has discussed potential areas of focus
and agreed to three areas: RF technology, program control, and data
governance. However, it does not have an approved charter, and it has
not developed explicit plans or milestone dates for identifying the
dependencies and relationships between the two programs. Program
officials stated that the team has met three times and plans to meet on
a quarterly basis going forward.
It is important that the relationships and dependencies between these
two programs be managed effectively. The longer it takes for the
programs to understand and exploit their relationships, the more rework
will be needed at a later date to do so.
Conclusions:
Over the last 3 years, we have made recommendations aimed at correcting
fundamental limitations in US-VISIT's program management ability and
thereby better ensuring the delivery of mission capability and value on
time and commensurate with costs. While progress on the implementation
of the recommendations is mixed, progress in critical areas has been
slow. As with any program, introducing and institutionalizing the
program management and accountability discipline at which our
recommendations are aimed require investing time and resources while
continuing to meet other program demands. In making such investment
choices, it is important to remember that institutionalizing such
program discipline in the near term will produce long-term payback in a
program's ability to meet these other demands. Accordingly, the longer
that US-VISIT takes to implement our recommendations, the greater the
risk that the program will not meet its stated goals and commitments.
Our open recommendations are all aimed at strengthening US-VISIT
program management and improving DHS's ability to make informed US-
VISIT investment decisions. With the exception of one, these
recommendations are still relevant and applicable. Since we made our
recommendation, facts and circumstances surrounding Increment 2B
deployment and operational status have materially changed, making the
collection of Increment 2B predeployment impractical. Nevertheless, the
need remains to better understand the impact of US-VISIT entry
capabilities on all land POEs. Until this understanding exists, the
department will be challenged in its ability to accurately estimate and
provide facilities and staff resource needs.
Recommendation for Executive Action:
To recognize both the need to fully assess the impact of US-VISIT entry
capabilities on staffing levels and facilities at land POEs, as well as
the current operational status of Increment 2B, we are closing our
existing recommendation related to assessing the impact of Increment
2B. We recommend that the DHS Secretary direct the US-VISIT Program
Director to explore alternative means of obtaining an understanding of
the full impact of US-VISIT at all land POEs, including its impact on
workforce levels and facilities; these alternatives should include
surveying the sites that were not part of the previous assessment.
Agency Comments and Our Evaluation:
In its written comments on a draft of this report, signed by the
Director, Departmental GAO/OIG Liaison Office, and reprinted in
appendix II, DHS stated that it agreed with many areas of the report
and that our recommendations had made US-VISIT a stronger program.
Further, the department stated that while it disagreed with certain
areas of the report, it nevertheless concurred with the need to
implement our open recommendations with all due speed and diligence.
DHS commented specifically on 11 of the 18 recommendations discussed in
the report. The recommendations, the department's comments, and our
responses follow:
1. Recommendation: Develop and begin implementing a system security
plan, and perform a privacy impact assessment and use the results of
the analysis in near-term and subsequent system acquisition decision
making.
DHS stated that this recommendation has been fully implemented. In
support, it said that it has completed a US-VISIT security plan that is
consistent with National Institute of Standards and Technology (NIST)
guidance, and that it provided the plan to us in September 2004. It
also stated that the security risk assessment aspect of this
recommendation was established in February 2005, 20 months after we
made the recommendation, and thus the age of the recommendation should
be shown as 10 months rather that the 30 months cited in the report.
The department also commented that there is no US-VISIT system, but
rather a US-VISIT program with capabilities delivered by existing
interconnected systems. According to the department, these component
systems have been certified and accredited, consistent with NIST
guidance, and as part of their certification and accreditation,
security plans and risk assessments, as well as risk mitigation
strategies, have been developed for each system. The department stated
that it provided us with these system-level risk assessments, as well
as system-specific action plans and milestones for implementing the
mitigation strategies. In addition, the department noted that it
completed a programwide risk assessment in December 2005 that
specifically addresses information security issues that might not be
captured in the system-specific documentation used to certify and
accredit each system. In light of its system-specific certification and
accreditation efforts, existing system-level risk assessments, and the
program-level risk management process (see response 4 for discussion of
the risk management process), DHS commented that it is inaccurate to
state that US-VISIT officials are not in a position to know program
risks, and the recommendation should be closed.
While we agree that we received a copy of the US-VISIT security plan,
dated September 2004, we do not agree that the plan satisfied all
relevant federal guidance and that DHS has fully implemented our
recommendation. In particular, it has not provided us with evidence
that a programwide risk assessment has been done and that a security
plan reflective of such an assessment exists. According to relevant
guidance,[Footnote 39] a security plan should describe, among other
things, the methodology that is to be used to identify system threats
and vulnerabilities and to assess risks, and it should include the date
the risk assessment was completed because the assessment is a necessary
driver of the security controls described in the plan. As we reported
in February 2005 and state in this report, the US-VISIT security plan
did not include this information; further, although DHS stated in its
comments that it completed this risk assessment in December 2005, this
statement is contradicted by a statement elsewhere in its comments that
it is still in the process of doing the assessment. In addition to this
contradiction, DHS's comments did not include any evidence to
demonstrate that it has developed a complete risk assessment, such as a
copy of the assessment.
With regard to the age of the recommendation, we do not agree with
DHS's position that we established a new finding regarding the lack of
a programwide risk assessment in our February 2005 report. Rather, as
part of our analysis of actions to implement our prior recommendation
to develop a security plan, which is to include information about the
related security risk assessment, we observed that the plan did not
indicate a date for completing a risk assessment in accordance with
federal guidelines. Therefore, our position that about 30 months had
passed from the time of our initial recommendation (June 2003) is
accurate.
With regard to the individual system-level risk assessments, we agree
that we have received them. However, we do not agree that we have
received the action plans and milestones cited in the comments.
Regardless, we do not believe that system-level assessments are a
sufficient substitute for a programwide assessment. Accordingly, our
recommendation focused on the need for an integrated US-VISIT system
risk assessment as part of security planning. While the system-level
plans and risk assessments are relevant and useful, they neither
individually nor collectively address the threats and vulnerabilities
imposed as a result of these systems' integration. By stating in its
comments its commitment to having a programwide risk assessment that
identifies and proposes mitigations for security risks that arise as a
result of the interface and integration of the legacy systems, DHS is
agreeing with our position. Moreover, without evidence that the program
has completely assessed its risks, we continue to find no basis for how
program officials would know the full range and degree of US-VISIT
security risks. Our position in this regard has been reinforced by a
recent DHS Inspector General report that identified a number of US-
VISIT security risks.[Footnote 40]
To further support its position that this recommendation has been fully
implemented, DHS also commented that it has completed numerous privacy
impact assessments and continues to update them to reflect system
changes. In particular, it said that it updated the privacy impact
assessment in December 2005 to reflect all increments and that it
considers the assessment to be part of US-VISIT system documentation.
It further commented that we appear to be unaware of privacy staff
activities to review system documents and perform privacy risk
assessments throughout the system life cycle. Nevertheless, the
department acknowledged that its privacy work was not always noted
within US-VISIT system documentation. Accordingly, DHS stated that it
plans to appropriately reference all privacy requirements and privacy
risk assessments in the program's system documentation in the future.
We agree that US-VISIT has developed and updated its privacy impact
assessment and would note that our report states this fact. We do not
agree, however, with the comment that we are not aware that the privacy
staff review system documents and perform privacy risk assessments. In
fact, it is because we were aware of these facts that we were careful
to ensure that they were reflected in our report. The point that we are
making is that privacy is not addressed in all relevant systems
documentation, which DHS acknowledged in its comments. With regard to
this point of agreement, we support the department's stated plans to
reference all privacy requirements and any privacy risk assessments in
all relevant system documentation in the future.
2. Recommendation: Develop and implement a plan for satisfying key
acquisition management controls, including acquisition planning,
solicitation, requirements management, program management, contract
tracking and oversight, evaluation, and transition to support, and
implement the controls in accordance with SEI guidance.
DHS commented that the report should reflect that US-VISIT had
initially adopted Carnegie Mellon University's Software Engineering
Institute (SEI) Software Acquisition Capability Maturity Model®to guide
its software-related process improvement efforts and that, in December
2004, it transitioned to SEI's Capability Maturity Model-Integration
(CMMI®). As a result, it said that the program's process improvement
strategy and plans, process development, and process appraisals are now
aligned to the most applicable CMMI process areas.
We agree that US-VISIT has transitioned to CMMI. We state in our report
that US-VISIT has done so and that the key process areas it is
addressing in its process improvement strategy and plan are consistent
with those cited in our recommendation. We do not believe that this
transition materially affects our recommendation, however, because even
though the names of the key processes in these two models may in some
cases differ, the processes and respective practices are fundamentally
consistent.
3. Recommendation: Clarify the operational context in which US-VISIT is
to operate.
Consistent with our report, DHS commented that the operational context
in which US-VISIT operates is in progress, meaning that it has yet to
be fully established. For example, it said that the mission of DHS, and
therefore the scope of US-VISIT activities to meet the mission, is
continually expanding. Further, it acknowledged that more certainty in
the operational context is desirable. In mitigation of the risks
associated with not having a more stable operational context, DHS made
several statements. For example, it said that the principal role of US-
VISIT is to integrate information and immigration and border management
systems across DHS and the State Department, and to facilitate agencies
working toward a common environment that will eliminate redundancies.
It also said that elements of its draft immigration and border
management strategic plan are being used in current US-VISIT
operations. In addition, the department said that mechanisms to
mitigate the risks that we cited have been developed and are being
implemented.
We support DHS's acknowledgment of the importance of having a well-
defined operational context within which to define and implement US-
VISIT and related border security programs. However, we do not believe
that DHS's comments provided any evidence showing that sufficient steps
and activities to mitigate the associated risks have been taken or are
planned.
4. Recommendation: Determine whether proposed US-VISIT increments will
produce mission value commensurate with cost and risks and disclose to
the Congress planned actions.
DHS commented that its cost-benefit analysis (CBA) for Increment 1B
conforms to relevant federal guidance, and noted that our expectations
as to the scope and level of detail of analysis that should be included
in the CBA document are inconsistent with its understanding of OMB
Circular A-94 and DHS's CBA workbook,[Footnote 41] which were used to
guide the development of the CBA analysis. As an example, the
department took exception with our statement that year-by-year benefit
estimates were not reported by noting that the net present value was
based on an estimate of annual benefits and costs, and that net present
value could not be estimated without a year-by-year benefit analysis.
The department further commented that a comprehensive uncertainty
analysis was conducted because it completed a risk analysis, which is
more comprehensive, rigorous, and appropriate than conducting a
sensitivity analysis. In this regard, it added that the results of the
risk analysis provided an indication of Increment 1B's worthiness in
light of existing uncertainty, rather than information on a specific
CBA variable or another. The department further noted that it had
provided some of these supporting analyses to us.
DHS also stated that any investment that has a 5-year life cycle and is
considered interim in nature will face considerable challenge in
providing economic benefits commensurate with cost.
We do not agree that the CBA fully conforms to relevant federal
guidance. As our report states, for example, the analysis does not
explicitly state the numerical value of the discount rate used for
calculating each alternative's net present value, and hence does not
conform to OMB guidance. In addition, the cost estimates used in the
analysis were not complete and reliably derived. In deriving the
estimate, for example, the department did not clearly define the
project's life cycle to ensure that key factors were not overlooked and
that the full cost of the program was included. (See response 10 below
for more information on this point.) Last, while we agree that a year-
by-year benefit analysis is a necessary component of a net present
value determination, OMB nevertheless requires that the year-by-year
benefit estimates be reported in the analysis to promote independent
review of the estimates.
Also, we do not agree that DHS performed a complete uncertainty
analysis. According to OMB and DHS guidance, a complete uncertainty
analysis should include both a risk analysis and a sensitivity
analysis. However, the latter was not done. Thus, our point is not, as
DHS comments suggest, that US-VISIT should have performed a sensitivity
analysis instead of a risk analysis, but rather, that both types of
analyses are necessary to completely examine investment uncertainty.
5. Recommendation: Develop and implement a risk management plan and
ensure that all high risks and their status are reported regularly to
the executive body.
DHS commented that US-VISIT began the development and implementation of
its risk management plan in 2004 immediately after we made our
recommendation. It further commented that, as part of a CMMI maturity
internal appraisal that it completed in July 2005, it found that the
risk management process had not been consistently applied across the
program. To address this, the department cited actions that it has
taken to fully implement risk management, such as approving the risk
management plan in September 2005; defining a risk governance
structure; establishing and maintaining a risk database; and developing
risk management training and providing this training to program
personnel and contractors beginning in November 2005.
We support the recent actions that the program cited as having been
taken to strengthen risk management. However, the actions cited do not
demonstrate that the risk management process is being consistently
applied. Until US-VISIT fully implements its risk management plan and
process, it cannot be assured that all program risks are being
identified and managed in order to effectively mitigate any negative
impact on the program's ability to deliver promised capabilities on
time and within budget.
6. Recommendation: Develop and approve test plans before testing begins
that (1) specify the test environment; (2) describe each test to be
performed, including test controls, inputs, and expected outputs; (3)
define the test procedures to be followed in conducting the tests; and
(4) provide traceability between test cases and the requirements to be
verified by the testing.
DHS stated that our report does not accurately reflect the status of
the Increment 2C Phase 1 testing. In particular, it said that the
issues associated with the traceability of requirements to test cases
were minor and that the extent of the discrepancies is far less than
what our report presents. It further stated that the discrepancies in
our report are based on old traceability documentation and do not
reflect revised documentation provided to us on November 9, 2005.
We agree that DHS provided us with revised traceability matrixes after
we had shared with them our analysis of the test plans and traceability
matrixes, dated June 28, 2005, and June 27, 2005, respectively.
However, the revised documentation referenced in DHS's comments was
provided in November 2005, about 4 months after testing began. This
means that the test plans and traceability matrixes available at the
time of testing--which are what we reviewed because they governed the
scope and nature of actual testing performed--did not adequately trace
between test cases and the requirements to be verified. Specifically,
300 of the 438 Increment 2C requirements, or about 70 percent, did not
have specific references to test cases.
7. Recommendation: Implement effective configuration management
practices, including establishing a US-VISIT change control board to
manage and oversee system changes.
DHS commented that a US-VISIT representative attends all configuration
control board meetings for all applicable legacy component systems, and
that any proposed change request from a legacy component control board
that could affect US-VISIT functionality is brought to the attention of
the US-VISIT Executive Configuration Control Board for consideration.
We do not question these statements. However, we do not believe that
they demonstrate that US-VISIT has adequate control over system changes
that could affect the program. That is, they do not ensure that changes
to the component systems that are initiated and approved by another DHS
organization and that could affect US-VISIT performance are subject to
US-VISIT configuration management and approval processes. US-VISIT
could establish explicit and enforceable control over changes to the
legacy systems through such mechanisms as defined and enforced
memorandums of understanding among the affected DHS organizations. It
was the lack of such control that prompted our recommendation.
8. Recommendation: Assess the full impact of Increment 2B on land POE
workforce levels and facilities, including performing appropriate
modeling exercises.
The department stated that, given the imperative to meet the
legislatively mandated time frames, the scope of Increment 2B was
limited to only one part of POE operations--incorporating the
collection of a biometric into the previously manual Form I-94 issuance
process. It also stated that wait times are affected by various
factors, including traffic volume, staffing levels, and availability of
officers. Therefore, DHS focused the Increment 2B evaluation on just
the change to this process.
The department further commented that given the events since the
evaluation--namely, Increment 2B full operations--it is not practical
to collect and model baseline data for the 47 sites that were not part
of the initial evaluation.
Regarding the 3 pilot sites included in the assessment, the department
stated that the sites were selected based on criteria developed from
input from US-VISIT, as well as CBP operational constraints. The
department further commented that the 3 sites provided a reasonable mix
of travelers and they did not have other constraints that directly
impacted the collection of performance data specific to Form I-94
issuance. DHS also stated that the I-94 processing times vary by POE,
and therefore they are not easily generalized from one port to another.
Further, the department commented that the number of workstations and
officers available to operate those workstations to process applicants
for a Form I-94 do not impact the time it takes to issue a Form I-94.
We agree that the scope of the Increment 2B evaluation was limited to
the I-94 issuance process, and that it did not address the increment's
impact on the POEs' ability to meet other performance parameters. Our
point is that the limited nature of the evaluation does not satisfy
either the intent of our recommendation or DHS's own stated purpose for
the evaluation, which was to determine the effectiveness of Increment
2B performance at the 50 busiest land POEs. We also agree that the I-94
processing times vary by POE and cannot be easily generalized. It is
for this reason, among others, that we questioned whether the 3 sites
selected for the assessment were sufficiently representative to satisfy
both our recommendation and the evaluation's stated purpose.
In addition, while we also agree that collecting pre-Increment 2B
baseline data is not practical at this time, the fact remains that the
operational impact of Increment 2B on workforce levels and facilities
has not been adequately assessed, as evidenced by officials at 1 large
POE telling us that processing times have increased and DHS's
recognition that each POE is somewhat different. In light of these new
facts and circumstances, we are closing our existing recommendation and
making a new recommendation to recognize the need for DHS to explore
alternative means to assess the impact of US-VISIT entry capabilities
at land POEs. This new recommendation will be shown as an open
recommendation, and the original recommendation will be closed.
9. Recommendation: Develop a plan, including explicit tasks and
milestones, for implementing all of our open recommendations and
periodically report to the DHS Secretary and Under Secretary on
progress in implementing this plan; and report this progress, including
reasons for delays, in all future expenditure plans.
DHS stated that it is untrue that 19 months had elapsed from the time
we made this recommendation to the time that it assigned
responsibilities to program officials for addressing each of our
recommendations. In support, it commented that it issued its first plan
to address our recommendations on August 18, 2003, and subsequent
reports have been issued periodically that update progress in doing so.
We agree that DHS has assigned responsibilities to specific individuals
for addressing each recommendation. However, we have yet to be provided
any evidence to support its statement that it issued the first report
addressing our recommendations on August 18, 2003. Similarly, we have
not received evidence showing that it has prepared a plan, including
specific actions and milestones, for implementing all of our open
recommendations, which is a focus of this recommendation. We would also
observe that we made this recommendation in May 2004, and at that time
the department stated that it agreed with the recommendation but did
not indicate that it had taken any steps to address it, such as
commenting that a report was issued on August 18, 2003.
10. Recommendation: Follow effective practices for estimating the costs
of future increments.
DHS either tacitly or explicitly agreed with our findings relative to
its satisfaction of 8 of the 13 cost-estimating criteria presented in
table 4 (now table 3) of our draft report. For example, it agreed that
it did not clearly define the life cycle to which the cost estimate
applies. It also agreed that it did not include a work breakdown
structure, noting that it used the available project implementation
schedule as a proxy for the activities related to the deployment of the
exit alternatives.
Regarding our five findings concerning its satisfaction of cost-
estimating with which DHS disagreed, the department's primary area of
disagreement was with the intended purpose of the Increment 1B CBA that
used the cost estimate, which it said in its comments was to inform
decision makers about the relative worthiness of each of the three exit
alternatives considered for deployment. Hence, DHS stated that the
purpose of the CBA was to analyze only the costs associated with
deploying an operational solution, not to analyze the costs and
benefits of both developing and deploying alternative solutions. DHS
further stated that the CBA thus includes only those costs to be
incurred in deploying a selected alternative, and it does not include
costs already incurred in developing system alternatives (i.e., sunk
costs). It further commented that DHS guidance states that sunk costs
are not relevant to the current investment analysis because "only
current decisions can affect the future consequences of investment
alternatives."
DHS also disagreed that the cost estimate in the CBA should have
included nonrecurring development costs, and commented that it did
appropriately size the task described in the cost estimates for each
alternative exit solution, noting that sizing metrics related to
software development were not relevant to deployment of the
alternatives because development activities had already occurred and
therefore are sunk costs. The department added that those sizing
metrics that are relevant to the cost estimate are discussed in the
CBA, as are the cost estimating parameters (i.e., those associated with
deployment and not those associated with development and testing).
In addition, DHS disagreed that DHS's cost estimate excluded important
cost categories, such as system testing, and stated that the estimate
addresses labor, facilities, operations and maintenance, information
technology, travel, and training costs. Once again, DHS emphasized that
since the focus of the CBA was on operational deployment and not system
design and development, system testing costs were not included because
they were not considered relevant. DHS also reiterated its early point
that the uncertainty analysis that it conducted was comprehensive.
We agree that actual sunk costs should not be included in a CBA cost
estimate. However, we disagree that the cost categories that DHS cited
as not relevant are only costs that are associated with predeployment
activities. Testing, for example, is an activity that is normally
performed before, during, and following deployment, and thus the
associated costs would be relevant to the stated purpose of the
Increment 1B CBA. However, a testing cost category was missing from the
CBA cost estimate, as was a cost category for software maintenance.
Regarding DHS's statement that it conducted a complete uncertainty
analysis, we reiterate our previous point that a complete uncertainty
analysis should include both a risk analysis and a sensitivity
analysis, and the CBA did not include the latter.
11. Recommendation: Reassess plans for deploying an exit capability to
ensure that the scope of the exit pilot provides for adequate
evaluation of alternative solutions and better ensures that the exit
solution selected is in the best interest of the program.
Concerning the questions we raised about the adequacy of the exit
pilots in light of the 24 percent compliance rate, DHS commented that
we failed to consider the compliance rate of the previous exit pilot
program, the National Security Entry Exit Registration System (NSEERS),
which, according to DHS, had a 75 percent compliance rate. DHS added
that NSEERS achieved this compliance rate with a very limited number of
exit locations, and therefore, any of the three US-VISIT exit
alternatives would have at least a 75 percent compliance rate once
national deployment was completed.
Further, the department commented that Immigration and Customs
Enforcement (ICE) had recently conducted enforcement operations at the
Denver International Airport, and that the compliance rate during these
operations increased from 30 percent to over 90 percent. It then
concluded that the combined results of the exit pilot evaluation, the
NSEERS pilot, and the ICE enforcement activities at the Denver
International Airport lead it to believe that the US-VISIT exit
alternatives have been adequately evaluated.
We do not agree with this conclusion because it is based on unsupported
assumptions. Specifically, DHS did not provide any evidence to support
its claim that that US-VISIT would achieve a comparable compliance rate
to the NSEERS program. Moreover, even if DHS could achieve a 75 percent
compliance rate for US-VISIT exit,that still means that 25 percent of
eligible persons would not be complying with the US-VISIT exit process.
Further, DHS did not provide any information about the recent
enforcement actions conducted by ICE, nor did it provide any evidence
that this is a practical and viable option for the US-VISIT exit
solution. While we agree that enforcement actions may indeed increase
the exit compliance rate, DHS has not yet assessed the impact of such a
solution on the US-VISIT exit process. Further, the US-VISIT program
director acknowledged the need to evaluate the impact of implementing
potential enforcement actions on US-VISIT exit and planned to do so.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of the Senate and House Appropriations Committees, as
well as to the Chairmen and Ranking Minority Members of other Senate
and House committees that have authorization and oversight
responsibilities for homeland security. We are also sending copies to
the Secretary of Homeland Security, Secretary of State, and the
Director of OMB. Copies of this report will also be available at no
charge on our Web site at [Hyperlink, http://www.gao.gov].
Should you or your offices have any questions on matters discussed in
this report, please contact me at (202) 512-3439 or at [Hyperlink,
hiter@gao.gov]. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. Key contributors to this report are listed in appendix IV.
Signed by:
Randolph C. Hite:
Director, Information Technology Architecture and Systems Issues:
List of Requesters:
The Honorable Peter T. King:
Chairman:
The Honorable Bennie G. Thompson:
Ranking Minority Member:
Committee on Homeland Security:
House of Representatives:
The Honorable Bob Filner:
House of Representatives:
The Honorable Raul M. Grijalva:
House of Representatives:
The Honorable Ruben Hinojosa:
House of Representatives:
The Honorable Solomon Ortiz:
House of Representatives:
The Honorable Silvestre Reyes:
House of Representatives:
[End of section]
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Our objective was to determine the progress of the Department of
Homeland Security (DHS) in implementing 18 of our recommendations
pertaining to the U.S. Visitor and Immigrant Status Indicator
Technology (US-VISIT) program. To accomplish this objective, we
reviewed and analyzed US-VISIT's most recent status reports on the
implementation of our open recommendations and related key documents,
augmented as appropriate by interviews with program officials. More
specifically, we analyzed relevant systems acquisition documentation,
including the program's process improvement plan, risk management plan,
and configuration management plan. We also analyzed the US-VISIT
security plan, privacy impact assessment, cost-benefit analysis, cost
estimates, test plans, human capital plans, and related evaluations and
assessments. In performing our analyses, we compared available
documentation and program officials' statements with relevant federal
guidance and associated best practices.[Footnote 42] A more detailed
description of our scope and methodology relative to the cost-benefit
analysis, cost estimates, and test plans follows:
* Our analysis of the cost-benefit analysis focused on Increment 1B
because this was the latest cost-benefit analysis and cost estimate
prepared. In doing this analysis, we compared the US-VISIT cost-benefit
analysis to eight criteria in Office of Management and Budget (OMB)
guidance.[Footnote 43]
* Our analysis of the cost estimate also focused on Increment 1B for
the same reason previously cited. In doing this analysis, we compared
the estimate to 13 criteria from the Software Engineering
Institute[Footnote 44] that we have previously reported to be the
minimum set of actions needed to develop a reliable cost estimate. We
then determined whether the criteria were satisfied, partially
satisfied, or not satisfied using the definitions given below.
* Our analysis of the test plans focused on Increment 2C because it is
the most recently tested increment. This analysis included determining
the extent to which the test plans for this increment met 4 key
criteria that we have previously reported as essential to effective
test plans. In doing this analysis, we examined Increment 2C systems
documentation, including business and functional requirements and
traceability matrixes. We also independently traced 58 business
requirements and 438 functional requirements to the test cases in the
test plan. Further, we independently traced all test cases to the
requirements to determine consistency.
In performing our work, we used the following categories and
definitions in deciding the extent to which each recommendation had
been implemented. Specifically, we considered a recommendation:
* completely implemented when documentation demonstrated that it had
been fully addressed,
* partially implemented when documentation indicated that actions were
under way to implement it, and:
* in progress when documentation indicated that action had been
initiated to implement it.
These categories and definitions are consistent with those used in our
prior US-VISIT reports.
In determining the amount of time it has taken to implement actions on
our recommendations, we calculated the time from the date the report
was issued through December 2005.
We conducted our audit work at the US-VISIT program office in Rosslyn,
Virginia, from August 2005 through December 2005, in accordance with
generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
January 13, 2006:
Randolph C. Hite:
Director, Information Technology Architecture and Systems Issues:
U.S. Government Accountability Office:
Washington, D.C. 20548:
Dear Mr. Hite:
Thank you for the opportunity to review the draft report, Homeland
Security: Recommendations to Improve Management of Key Border Security
Program Need to Be Implemented (GAO-06-296). As with prior reports that
your office has issued regarding US-VISIT, there are many areas with
which we agree, and the recommendations have made US-VISIT a stronger
program. However, as with those past reports, the Department of
Homeland Security (DHS) has certain areas of disagreement. They appear
in our comments, which begin on page 2 of this letter.
All of the issues covered by this report need to be viewed in the
larger framework of one simple fact: US-VISIT is working as Congress
intended.
Thanks to the hard work and dedication of the US-VISIT team, all three
congressionally mandated phases of implementation were completed ahead
of schedule and under budget. US-VISIT is now in place at our nation's
airports, seaports, and land border ports of entry. As you know, this
program has a significant effect on our national security, economic
prosperity, and international relationships around the world. Through
biometric authentication, US-VISIT makes entering the U.S. easier for
legitimate tourists, students, and business travelers, while making it
more difficult to illegally enter and stay in our country.
US-VISIT-working in partnership with stakeholders within DHS, the
federal government, the private sector, and other countries-has
exceeded the goals set by Congress and DHS for this program. In the
final report of the 9/11 Commission, which issued grades to U.S.
government responses to the recommendations outlined in its 2004
report, the 9/11 Commission awarded a "B" to "Biometric entry-exit
screening system," one of the highest grades achieved by any government
agency. The Commission recognized US-VISIT's successful screening
operations at our ports of entry, and found that the program has
collaborated well with Interpol.
In the two and a half years since its inception, US-VISIT has processed
more than 45 million visitors at ports of entry, linking together
systems from DHS and the Departments of State and Justice. In FY 2005,
US-VISIT was successfully deployed at the 154 land border ports of
entry (POEs), with the majority of ports reporting improved process
times. US-VISIT also worked closely with the Department of State to
implement the same capability at its 211 visa issuing posts around the
world. US-VISIT has now intercepted nearly 1,000 prior or suspected
criminals and immigration violators-including murderers, rapists,
pedophiles, and drug traffickers-from entering the country, and enabled
the Department of State to identify criminals and immigration violators
who applied for visas. During this same period, DHS has provided 14,700
matches against the biometric watchlist to the Department of State
through its BioVisa program, which is fully integrated with US-VISIT.
Use of biometrics has allowed the United States to deprive potential
terrorists of one of the tools they use to threaten our nation and
other countries around the world: the ability to cross our borders
using fraudulent documents and violate our immigration laws without
detection.
Even with US-VISIT's increased security checks, travelers have not been
inconvenienced; in fact, wait times at land border ports of entry have
actually gone slightly down, and surveys from travelers show that the
vast majority do not object to US-VISIT's biometric procedures. By
working closely with federal, state, and local governments; conducting
a thorough, concentrated, and continuing global outreach campaign; and
through a commitment to respect for the privacy of those who would be
enrolled in the system, US-VISIT has gained worldwide acceptance. US-
VISIT's success inspired the European Union to adopt the inclusion of
fingerprints into its biometric passports; and the government of Japan
has indicated that it will model its own biometric border management
system after US-VISIT.
The GAO draft report is organized by discussion of progress on the
implementation of prior open recommendations. US-VISIT comments on
GAO's assessments are also provided by recommendation:
Recommendation:
Develop and begin implementing a system security plan, and perform a
privacy impact assessment and use the results of the analysis in near-
term and subsequent system acquisition decision.
Response:
While US-VISIT has completed a security plan and is in the process of
completing a risk assessment, the relationship of these documents to
system security must be clearly understood. As the GAO report details,
US-VISIT is being implemented incrementally. Increments 1 through 3
fulfilled legislative mandates through the introduction of interfaces
and enhancements to existing "legacy" systems. As such, there is no US-
VISIT system, but rather a US-VISIT program with capabilities delivered
by these interconnected systems. Consistent with both National
Institute of Standards (KIST) guidance and the DHS inventory, these
systems have undergone extensive security evaluation leading to the
certification and accreditation of each component system. The
accreditation status of these systems is shown below:
[See PDF for image]
[End of table]
As an integral part of certification and accreditation, security plans
and risk assessments are developed for each system. Additionally, risk
mitigations are proposed and tracked in a DHS tool for each system. To
posit that US-VISIT does not understand system requirements or did not
ensure that "proper safeguards are in place to protect system data and
resources" fails to acknowledge the extensive security procedures in
place at the system level.
As stated in the draft report, US-VISIT was preparing an enterprise-
wide risk assessment. This document was completed in December 2005, and
it identifies and proposes mitigations for security risks that arise
from the complex interplay of the interconnected systems cited above.
This document specifically addresses information security issues that
might not be captured in the system-level documentation prepared for
legacy system certification and accreditation. It also complements the
security strategy document under development that supersedes the
existing US-VISIT security plan.
GAO properly notes that program management-as opposed to system
security management-is the mechanism to address programmatic risks. US-
VISIT coordinates issues derived from security reviews with a Risk
Review Board to ensure that security issues are elevated when they
impact overall program risk.
In regard to the performance of privacy impact assessments, as GAO has
noted, US-VISIT has completed numerous Privacy Impact Assessments
(PIAs) and continues to update them to reflect changes in US-VISIT
systems. The US-VISIT PIA is regarded throughout the privacy community
as a model document. However, GAO appears to be unaware that the
privacy program staff fully participates in US-VISIT integrated project
teams and has effectively integrated privacy activities into the system
development lifecycle by reviewing all system documents and performing
privacy risk assessments for both specific issues as well as for
overall increment planning and implementation. In this manner, US-VISIT
believes that it has implemented the GAO recommendation to fully
address privacy issues in the relevant system documentation, but
understands that the privacy work completed was not always noted within
each individual system document. To ensure that GAO has full visibility
into the privacy work completed by US-VISIT in the future, all relevant
system documents will be annotated to specifically reference the
privacy requirements and reference any privacy risk assessments that
were completed.
There are specific areas of the draft report's assessment of progress
on this recommendation that need clarification:
In the Executive Summary on page 17, first bullet, Security Plan:
The US-VISIT Security Plan provided to GAO was composed in accordance
with DHS requirements and NIST SP 800-18. The security plan devotes an
entire section (section 4.1) to Risk Assessment and Management. In
February 2005, GAO established another finding to develop a program-
wide risk assessment, which was completed at the end of calendar year
2005. This finding was only open for less than 10 months, not "about
30" as it appears in the chart.. In addition to the program-wide risk
assessment, US-VISIT certifies and accredits all of its systems in
accordance with DHS policies and NIST 800-37 guidance. Systems that
operate to achieve the US-VISIT mission have individual system-level
risk assessments completed, evaluated, and updated throughout the
lifecycle to ensure that risk is known and managed by US-VISIT program
officials. These risk assessments have been provided to GAO. Plans of
Actions and Milestones (POA&Ms) exist for each US-VISIT system-also
provided to GAO-that establish an implementation schedule for
mitigation strategies to reduce the overall risk to the systems. In
addition to the system-level risk assessments and POA&Ms, risks
determined to be significant to US-VISIT are elevated to the US-VISIT
Risk Management Team. Based on all of the certification and
accreditation efforts, existing system security risk assessments, and
the program level risk management process, it is inaccurate to state
that US-VISIT officials "are not in a position to know the risks
associated with their program."
In regard to Table 1, the length of time that GAO asserts that this
recommendation has been open is inaccurate. The initial recommendation
was to complete a US-VISIT Program Security Plan. The Security Plan was
written in accordance with the format proscribed by NIST SP 800-18. It
was delivered in September 2004, which should have closed the
recommendation. A second follow-on recommendation from GAO to complete
a program-level security risk assessment was issued in February 2005.
US-VISIT is in the process of finalizing this document.
In regard to the Privacy Impact Assessment, page 18:
US-VISIT has completed numerous Privacy Impact Assessments (PIAs) and
continues to update them to reflect changes in US-VISIT systems. The
July 2005 PIA was found to be consistent with federal guidance, as
stated in the draft report. That PIA was updated in December 2005 based
on the same guidelines. Numerous privacy risk assessments are also
conducted to ensure that privacy is thoroughly accounted for throughout
the entire US-VISIT program. The PIA has been updated to reflect all
increments, and is considered to be part of system documentation. In
addition, privacy is built into the US-VISIT lifecycle and is
considered throughout the development of a system. GAO reports that
privacy is not included in functional requirements documentation. A
"functional privacy requirement" falls under the security controls and
requirements which are included in both business and functional
requirements documents. Security documentation specifically reflects
that "Privacy Act Information" is processed by the systems comprising
US-VISIT. A FIPS 199 Security Categorization was performed for each
system to determine that adequate security controls are in place or
planned to protect this Privacy Act information. System Security Plans
outline the specific controls in place to protect the data.
Recommendation:
Develop and implement a plan for satisfying key acquisition management
controls, including acquisition planning, solicitation, requirements
development and management, project management, contract tracking and
oversight, evaluation, and transition to support, and implement the
controls in accordance with the Software Engineering Institute's (SEI)
guidance.
Response:
In regard to the discussion of the Capability Maturity Model-Integrated
(CMMI):
The draft report should reflect that, initially, US-VISIT adopted
Carnegie Mellon University's Software Engineering Institute (SEI)
Software Acquisition Capability Maturity Model® (SA-CMM(R) to guide its
management process implementation. US-VISIT transitioned from the SA-
CMM to the Capability Maturity Model-Integration (CMMI®) in December
2004 based on recommendations from the SEI, MITRE, and the newly hired
US-VISIT Process Improvement Lead. The CMMI® is a more robust model and
is now the "best practice" standard in use at hundreds of commercial
and government organizations. Additionally, SEI expects to retire the
SA-CMM® very soon. SEI developed a guidance document-the CMMIO-
Acquisition Module-to assist acquisition organizations such as US-VISIT
in applying the CMMI®. As a result, the US-VISIT process improvement
strategy and plans, process development, and appraisals are now
realigned to the selected CMMI® process areas most applicable to US-
VISIT.
Recommendation:
Clarify the operational context in which US-VISIT is to operate.
Response:
As noted in the draft report, "..an immigration and border management
strategic plan was drafted in March 2005 that shows how US-VISIT is
aligned with DHS' organizational mission and defines an overall vision
for immigration and border management." GAO further noted that, "Since
the plan was drafted, DHS has reported that other relevant initiatives
have been undertaken, such as the Security and Prosperity Partnership
of North America and the Secure Border Initiative." And the draft
report concluded that, "Until US-VISIT's operational context is fully
defined, DHS is increasing its risk of defining, establishing, and
implementing a program that is duplicative of other programs and not
interoperable with them."
The mission of DHS is continually expanding and, as a result, the scope
of US-VISIT's activities in providing for capabilities to meet that
mission is constantly evolving. US-VISIT agrees that the operational
context in which it operates is, in a sense, "in progress" in that it
continues to evolve in compliance with new legislative, administrative,
and Departmental mandates and priorities. However, the principal role
of US-VISIT is to integrate information and make interoperable
immigration and border management systems across the Departments of
Homeland Security and State and, as such, US-VISIT will be an enabler
of other programs. A significant part of US-VISIT's role is to
establish an environment that will ensure agencies work toward a common
environment that will eliminate redundancies. The immigration and
border' management strategic plan, as well as the first MCE derived
from that plan, are being used in current operations. Elements of this
plan are being incorporated into the planning and operational context
for the projects noted by GAO as having potential for redundancy.
Although US-VISIT concurs that more certainty would be desirable,
mechanisms to mitigate the risk noted by GAO have been developed and
are being implemented.
Recommendation:
Determine whether proposed US-VISIT increments will produce mission
value commensurate with cost and risks and disclose to the Congress
planned actions.
Response:
US-VISIT disagrees with the assertion in the draft report that it did
not perform a complete uncertainty analysis for the three alternatives.
A comprehensive uncertainty analysis was conducted throughout the
study. The Risk Analysis Process, summarized in Appendix F, is a state-
of-the-art process to account for uncertainty surrounding key benefit
and cost assumptions used in the analysis. Chapter 6 of the cost
benefit analysis (CBA) explicitly shows the assumptions used in the
analysis, expressed in the form of ranges built around the major
variables. These assumptions are based on observations of historical
trends, pilot study results, and expert opinion solicited during risk
analysis sessions that were organized with the participation of various
stakeholders. Therefore, the process incorporates both objective and
subjective perspectives. The results of the risk analysis are
subsequently portrayed as probabilistic distributions in Chapter 7.
This approach is comprehensive, more rigorous, and more appropriate for
this study than sensitivity analysis. Sensitivity analysis
theoretically provides insight into which factors in the decision are
most important. Risk analysis, on the hand, allows for the simultaneous
variation of key assumptions within their assigned boundaries-a better
reflection of reality-rather than varying one variable at a time. The
risk analysis outcome is more appropriate for this study as the results
must provide the decision maker with an indication of the project's
worthiness given the existing uncertainty, rather than how the outcome
is sensitive to one specific variable or another.
US-VISIT was guided by, and adhered to, OMB Circular A-94 and the DHS
CBA handbook, Capital Planning and Investment Control: Department
ofHomeland Security Cost Benefit Analysis (CBA) Work Book May 2003, in
developing the Increment 1B CBA. US-VISIT's disagreement fundamentally
concerns expectations as to the scope and level of detail of analysis
that should be included with the formal CBA document. The auditors
apparently believe that all detail should be included within the formal
CBA document. US-VISIT instead chose to communicate the substance of
its analysis in the formal CBA, believing the results of the final
analyses were the more relevant input for DHS decision-makers. US-
VISIT's reading of Circular A-94 and the DHS CBA Work Book does not
lead to the conclusion that these documents require the level of detail
GAO desires. US-VISIT provided GAO with some of the detailed analyses
supporting the Increment 1 B CBA, and is prepared to provide other
detailed analyses for GAO review.
US-VISIT also takes exception to GAO's assertions in Table 2: US-VISIT
Satisfaction of OMB Economic Analysis Criteria. For Criterion 5, "The
quality of the benefits to be realized from each alternative was
reasonable," GAO concludes that the criterion was not met based upon
its analysis that "Year-by-year benefit estimates were not reported."
It is important to note that the net present value (NPV) estimate was
based upon an estimation of the stream of benefits and costs annually.
The NPV cannot be estimated without a year-by-year benefit analysis.
The detailed annual analysis GAO desires was performed and is available
for review. Again, the content of the formal CBA was focused on meeting
the information needs of DHS executives, with detailed supporting
analyses available upon request. For Criterion 8, "a complete
uncertainty analysis of cost and benefit was included," GAO concludes
that the criterion was not met based upon its analysis that "Although
the cost-benefit analysis did include Monte Carlo simulation results
for the three exit alternatives, no sensitivity analysis was conducted
for those alternatives. Instead, the cost-benefit analysis reports
sensitivity analysis results for the five deployment scenarios." US-
VISIT disagrees with the assertion that it did not perform a complete
uncertainty analysis for the three alternatives. A comprehensive
uncertainty analysis was conducted.
The draft report also states, "It is important that the program adhere
to relevant guidance in developing its incremental cost-benefit
analyses. If this is not done, the reliability of the analyses is
diminished, and an adequate basis for the prudent investment decision-
making does not exist. Moreover, if the mission value of a proposed
investment is not commensurate with costs, it is vital that this
information be fully disclosed to DHS and congressional decision
makers. The underlying intent of our recommendation is that this
information be available to inform such decisions." US-VISIT believes
that the Increment 1B CBA does conform to relevant guidance and that
the heart of the disagreement with GAO involves a difference in
interpretation as to the amount of detail necessary for inclusion
within the formal CBA, as opposed to having supporting detailed
analyses available upon request. Further, the NPV of each Increment 1B
alternative was clearly communicated in the executive summary of the
CBA in order to provide decision makers with the primary measure of
each alternative's relative worthiness. As these NPVs indicate, any
investment with a five-year lifecycle and considered interim in nature
will face a considerable challenge in providing economic benefits
commensurate with cost. To quote the CBA, "The full economic benefit of
this exit solution is not realized during the initial five years of
operation, but is harvested over an adequate life cycle of the
investment."
Recommendation:
Develop and implement a risk management plan and ensure that all high
risks and their status are reported regularly to the executive body.
Response:
In analyzing US-VISIT's efforts at managing risk, it is important to
consider that US-VISIT began the development and implementation of its
risk management plan in 2004 immediately after GAO made its initial
recommendation. As part of its CMMI process maturity baseline internal
appraisal completed in July 2005, US-VISIT found that the risk
management process detailed in its plan was not consistently applied
across the program. In response, positive steps have since been taken.
The Risk Management Plan was approved in September 2005 and includes,
among other things, a process for planning, identifying, analyzing,
handling, and monitoring risk. It also defines the governance structure
to be used in overseeing and managing the process. US-VISIT also
maintains a risk management database, which includes among other things
a description of the risk, its priority (high, medium, or low) and
impact, and its mitigation strategy. The database is currently
available to program management and staff.
US-VISIT established a Risk Review Board, Risk Review Council, and Risk
Owner to govern its risk activities. The roles and responsibilities are
described below.
* The Risk Review Board directs all risk governance within the program
and provides the mechanism to escalate/transfer the consideration of
risks to program governing boards and to organizations external to the
program.
* The Risk Review Council oversees and manages risks that are
significant, controversial, or cross-project, or that may require
escalation to the Risk Review Board.
* Risk Owners analyze, handle, and monitor risks.
Risk management training has been developed and training sessions for
US-VISIT personnel and contractors began in November 2005. The Risk
Review Board, chartered in September 2004, reviews risks with US-VISIT
executives and has been meeting periodically since January 2005.
Recommendation:
Develop and approve test plans before testing begins that (1) specify
the test environment; (2) describe each test to be performed, including
test controls, inputs, and expected outcomes; (3) define the test
procedures to be followed in conducting the tests; and (4) provide
Taceability between test cases and the requirements to be verified by
the testing.
Response:
While there were minor issues with the traceability of requirements to
test cases, the extent of the discrepancies is far less than presented
by the draft report. The data cited in the report is consistent with
GAO's initial findings as reported in its document, Topics for
Discussion and Request for Documentation Regarding Testing of US-VISIT
Increment 2C Proof of Concept Phase I, received on October 12, 2005, by
US-VISIT. However, the findings do not accurately reflect the status of
Increment 2C Phase 1 testing.
In the October 12, 2005, document, GAO requested the updated version of
the Requirements Traceability Matrix (RTM) to "..show proof that the
test cases were actually executed and the outcome(s) achieved." GAO
also requested the updated RTM to resolve requirements and test case
mapping issues identified in the GAO report. US-VISIT System Assurance
provided the current versions of the US-VISIT Increment 2C RTM along
with current versions of the US-VISIT Increment 2C Test Plan on
November 9, 2005, to GAO. Documents provided that day included:
* US-VISIT Increment 2C Requirements Traceability Matrix:
* US-VISIT Increment 2C Proof of Concept IV&V Test Cases:
* US-VISIT Increment 2C Proof of Concept IV&V Test Cases Appendix A - H
* US-VISIT System Engineering Plan:
* US-VISIT Task Order 4 Option Year 1:
These documents resolved the issues that GAO identified with earlier
versions of the documents, namely test case traceability to
requirements and testing results.
Recommendation:
Implement effective configuration practices, including establishing a
US-VISIT change control board to manage and oversee system changes.
Response:
The draft report states that "..changes to component systems that are
initiated and approved by another DHS organization and that could
affect US-VISIT performance are not subject to US-VISIT configuration
management processes and are not also being examined and approved by
the US-VISIT control board. This lack of US-VISIT control was the
impetus for our' recommendation." A representative from US-VISIT's
Office of Mission Operations or Office of Information Technology
attends all CCB meetings for applicable legacy component systems. Any
proposed change request from a legacy component CCB that could affect
US-VISIT functionality is brought by the US-VISIT representative to the
US-VISIT ECCB for consideration.
Recommendation:
Assess the full impact of Increment 2B on land POE workforce levels and
facilities, including performing appropriate modeling exercises.
Response:
The draft report asserts that the scope of US-VISIT's evaluation of the
impact of Increment 2B was too limited. Given the imperative to meet
the December 31, 2004,' legislative mandate, US-VISIT's Increment 2B
was limited by time, funding, and resources, and as such the
performance evaluation had to focus on representative sites. Three
pilot sites were identified by Customs and Border Protection (CBP), and
the selection criteria were based upon input from US-VISIT as well as
CBP's own operational constraints. The three locations offered by CBP
provided a reasonable mix of travelers and did not have other
constraints that would directly impact the collection of performance
data specific to the Form 1-94 issuance.
Wait times are a complex function of CBP operations, receipt of
intelligence, traffic volume, staffing levels, availability of Officers
to staff lanes/booths, weather, seasonal changes to traffic, holidays,
and local events. Since Increment 2B incorporated the collection of a
biometric into the previously manual process of Form I-94 issuance,
which is only one process in CBP border operations, measurements were
taken that specifically addressed the delta introduced by Increment 2B.
[In addition, on page 38, Table 3, concerning the reduction in reported
processing times, has an incorrect heading for the last column: it
should read "(February 2005)," not "(February 2004)."]
Going back to assess the full impact of Increment 2B would require
baseline data collection that represents operational performance prior
to the Increment 2B deployment. This is not practicable in the
production environment that exists at the 47 ports that were not
evaluated. The alternative approach is to model the baseline
performance using historical data from the three ports evaluated and
possibly supplement this data with data from previous studies. However,
it is very likely that the modeling approach used to reconstruct the
baseline performance will be subject to question. The detailed step-by-
step processing times are site specific and not easily generalized from
one port to another. As a result, any baseline estimates prepared ex
post will not be as accurate as the actual results reported from the
three ports. Lacking an acceptable baseline, any conclusions developed
from such a follow-up study on the remaining 47 ports could be refuted.
The reference in the draft report to the number of workstations
(baseline versus evaluation) is confusing. The number of workstations
available to process applicants for a Form 1-94 and/or the number of
Officers available to operate those workstations are often utilized to
address the number of applicants (or volume). Such resources do not
impact the time it takes to issue a Form 1-94 to an individual;
consequently, the time it takes to issue a Form 1-94 is the only true
valid measure.
The draft report also describes the San Ysidro port of entry (POE) as
the busiest land POE. This is not entirely accurate; while San Ysidro
is the largest POE by volume of travelers, the three bridges combined
for Laredo make it the busiest port that issues Form I-94s. In 2003,
San Ysidro issued approximately 409,683 I-94s; the combined bridges at
Laredo issued 432,892 Form I-94s.
Recommendation:
Develop a plan, including explicit tasks and milestones, for
implementing all our open recommendations and periodically report to
the DHS Secretary and Under Secretary on progress in implementing this
plan; and report this progress, including reasons for delays, in all
future expenditure plans.
Response:
GAO's assertion that 19 months elapsed from the issuance of this
recommendation until US-VISIT assigned responsibilities to specific
individuals for addressing each recommendation is untrue. In fact, the
first such plan for addressing GAO recommendations was issued on August
18, 2003-less than a month after former DHS Secretary Ridge officially
created the US-VISIT program office. Subsequent reports, issued
periodically and updated with progress on implementation, have included
all additional recommendations as they appeared in all GAO reports
affecting US-VISIT.
Recommendation:
Follow effective practices for estimating the costs of future
increments.
Response:
US-VISIT disagrees with GAO's evaluation in Table 4 of the Increment 1B
cost benefit analysis against the 13 SEI criteria for satisfaction of
cost estimating.
For Criterion 2, the lifecycle to which the estimate applies is clearly
defined. GAO concludes that the criterion was partially met based upon
its analysis that "The lifecycle was not clearly defined to ensure that
the full cost of the program was included. For example, the analysis
did not include evidence that nonrecurring development costs were
included in the cost estimate." US-VISIT does agree that it did not
clearly identify the lifecycle to which the estimate applies. The crux
of the disagreement is once again related to the purpose of the CBA
document, which is to inform DHS decision makers as to the relative
worthiness of each of the three exit alternatives considered for
deployment as part of Increment 1 B. The analysis supports the decision
related to the deployment of an operational solution for the project.
It does not analyze conceptual alternatives early in the investment
lifecycle that would necessitate the inclusion of planning, analysis,
design, and development activities in the cost estimates for each
alternative, as these activities had already occurred and therefore had
no bearing on the decision to deploy. The general cost assumptions
listed in Chapter 6 of the CBA include the following lifecycle
assumption: "Cost estimates represent only the incremental cost
associated with acquiring and maintaining the interim exit solution to
be delivered to 76 airports and 12 seaports as part of Increment 1 B."
Within the context of that overall lifecycle assumption, the following
information technology cost assumption is stated in the CBA: "IT
systems development, integration, and security costs [are] assumed to
be sunk historical costs incurred prior to full deployment of exit
alternatives and therefore not included in cost estimates." In other
words, the analysis includes only those acquisition costs that will be
incurred as a result of the decision on which exit alternative to
deploy, and does not include sunk costs for the plan, analyze, design,
build, and test stages that have already been incurred and do not
impact the deployment decision informed by this analysis. Per the DHS
CBA Work Book, pages 33-34, "Sunk costs are not relevant to the current
investment analysis because only current decisions can affect; the
future consequences of investment alternatives. The IPT will not
include sunk costs in any CBA calculations."
For Criterion 3, "The task has been appropriately sized," GAO concludes
that the criterion was not met based upon its analysis that "An
appropriate sizing metric should be used in the development of the
estimate, such as the amount of software to be developed and the amount
of software to be revised. The program office provided no evidence that
an appropriate sizing mechanism was used, and program officials stated
that they had not collected these data." US-VISIT believes that it
appropriately sized the task described in the cost estimates for the
Increment 1 B Exit CBA alternatives. As stated above, the alternatives
considered in the analysis represent operational deployment
alternatives, not conceptual program initiation phase alternatives.
Therefore, activities related to the plan, analyze, design, build, and
test stages were not considered relevant to the scope of the estimates
and were not included. Sizing metrics related to software development
were not applicable to the deployment phase because these activities
had already occurred and were therefore considered sunk costs not to be
included in the CBA calculations. Sizing metrics relevant to the
deployment phase were used in the cost estimates and were derived based
upon the actual costs of deployment experienced during the exit pilot.
By determining the average cost of deployment for sample airports and a
seaport based upon size and relative activity, and extrapolating those
sample deployment cost estimates across their respective operational
environments, a total cost of deployment was calculated. The deployment
cost estimate sizing technique described above is clearly communicated
in the CBA in the general cost assumptions in Chapter 6.
For Criterion 5, "A written summary of parameter values and their
rationales accompanies the estimate," GAO concludes that the criterion
was partially met based upon its analysis that "If a parametric
equation was used to generate the estimate, the parameters that feed
the equation should be provided along with an explanation of why they
were chosen. High-level cost categories, such as labor, information
technology, facilities, and other costs were identified, but detailed
parameters used to develop the estimate, such as number of software
lines of code, were not provided in the analysis." US-VISIT did provide
the detailed parameters used to develop the cost estimates for the
Increment 1 B Exit CBA alternatives. As stated above, the alternatives
considered in the analysis represent operational deployment
alternatives, not conceptual program initiation phase alternatives.
Therefore activities related to the plan, analyze, design, develop, and
test stages were not considered relevant to the scope of the estimates
and were not included. Parameters related to software development, such
as the number of software lines of code, were not applicable to the
deployment phase because these activities had already occurred and were
therefore considered sunk costs not to be included in the CBA
calculations. Cost estimating parameters relevant to the deployment
phase were used in the cost estimates and were derived from actual
costs of deployment experienced during the exit pilot. By determining
the average cost of deployment for sample airports and a seaport based
upon size and relative activity, and extrapolating those sample
deployment cost estimates across their respective operational
environments, a total cost of deployment was calculated. The deployment
cost estimating parameters described above are clearly communicated in
the CBA in the general cost assumptions in Chapter 6.
For Criterion 7, "A structured process, such as a template or format,
has been used to ensure that key factors have not been overlooked," GAO
concluded that the criterion was partially met based upon its analysis
that "The analysis included four high-level cost categories (labor,
facilities, operations and maintenance, and information technology),
but did not include a detailed work breakdown structure and omitted
important cost elements, such as system testing and training." US-VISIT
agrees that the estimate was not derived using a work breakdown
structure, although it did use the available project implementation
schedule as a proxy for the activities related to the deployment of the
Increment 1B exit criterion. However, US-VISIT disagrees with GAO's
assertion that the cost categories did not include important cost
elements such as system testing and training. The analysis examined the
costs of labor, facilities, operations and maintenance, information
technology, travel, and training as stated in Chapter 6 of the CBA. In
addition, as stated above, the alternatives considered in the analysis
represent operational deployment alternatives, not conceptual program
initiation phase alternatives. Therefore, activities related to the
plan, analyze, design, build, and test stages were not considered
relevant to the scope of the estimates and were not included. Costs
related to systems development and testing were not applicable to the
deployment phase because these activities had already occurred and were
therefore considered sunk costs not to be included in the CBA
calculations.
For Criterion 8, "Uncertainties in parameter values have been
identified and quantified," GAO concludes that the criterion was
partially met based upon its analysis that "A sensitivity and risk
analysis was performed, but this analysis did not identify detailed
parameter values." As stated previously, US-VISIT did conduct a
comprehensive uncertainty analysis.
Recommendation:
Reassess plans for deploying an exit capability to ensure that the
scope of the exit pilot provides for adequate evaluation of alternative
solutions and better ensures that the exit solution selected is in the
best interest of the program.
Response:
The draft report states that "..questions remain about whether the exit
alternatives have been evaluated sufficiently to permit selection of
the best exit solution for national deployment." The draft report
raises questions about the effectiveness of the three alternatives
since the average compliance rate was only 24 percent for the three
alternatives.
The GAO analysis fails to take into account the compliance rate of the
previous pilot program for exit, the National' Security Entry Exit
Registration System (NSEERS). Since its inception, the NSEERS
compliance rate is 75 percent. NSEERS has very limited exit locations-
typically not in the departure areas of airports-for aliens to
biometrically check out. Therefore, any of the three alternatives
tested would have at least a minimum 75 percent compliance rate once
the national deployment was completed. This information was not in the
evaluation report but was presented in the US-VISIT memorandum to the
Deputy Secretary with the subject, Direction for the US-VISIT Air/Sea
Exit Program.
GAO also states that the effect of the enforcement mechanism to improve
compliance is unknown and that additional evaluation is warranted.
However, within the past two months, Immigration and Customs
Enforcement (ICE) has conducted enforcement operations at the Denver
International Airport. As a result of these enforcement efforts, the
compliance rate at Denver International Airport has increased from 30
percent to over 90 percent. The combined results of the US-VISIT exit
evaluation, the NSEERS pilot, and the ICE enforcement activities at
Denver International Airport lead US-VISIT to believe that the exit
alternatives have been adequately evaluated.
While we may disagree with some of GAO's assessment of the amount of
progress on the open recommendations addressed in the draft report, we
nevertheless concur in the need for their implementation with all due
speed and diligence. However, in perspective, the discussion of these
recommendations does not alter the overall assessment of the Department
and many others--that US-VISIT's continuing success is making a
valuable contribution to the enhanced security of the United States.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Director, Departmental GAO/IG Liaison Office:
[End of section]
Appendix III: Description of US-VISIT Processes:
US-VISIT involves complex processes governing the stages of a
traveler's visit to the United States (pre-entry, entry, status, and
exit) and analysis of hundreds of millions of foreign national
travelers at over 300 air, sea, and land ports of entry (POE). A
simplified depiction of these processes is shown in figure 4.
Figure 4: US-VISIT Process Overview:
[See PDF for image]
[End of figure]
Pre-entry Process:
Pre-entry processing begins with initial petitions for visas, grants of
visa status, or the issuance of travel documentation. When a foreign
national applies for a visa at a U.S. consulate, biographic and
biometric data are collected and shared with border management
agencies. The biometric data are transmitted from the Department of
State to DHS, where the prints are run against the Automated Biometric
Identification System (IDENT) database[Footnote 45] to verify identity
and to run a check against the biometric watch list. The results of the
biometric check are transmitted back to State. A "hit" response
prevents State's system from printing a visa for the applicant until
the information is reviewed and cleared by a consular officer.
Pre-entry also includes transmission by commercial air and sea carriers
of crew and passenger manifests to appropriate immigration officers
before these carriers arrive in the United States.[Footnote 46] These
manifests are transmitted through the Advanced Passenger Information
System (APIS). The APIS lists are run against the biographic lookout
system to identify those arrivals for whom biometric data are
available. In addition, POEs review the APIS list in order to identify
foreign nationals who need to be scrutinized more closely.
Entry Process:
When a foreign national arrives at a POE's primary (air and sea) or
secondary (land) inspection booth, the inspector, using a document
reader, scans the machine-readable travel documents. APIS returns any
existing records on the foreign national to the US-VISIT workstation
screen, including manifest data matches and biographic lookout hits.
When a match is found in the manifest data, the foreign national's name
is highlighted and outlined on the manifest data portion of the screen.
Biographic information, such as name and date of birth, is displayed on
the bottom half of the computer screen, along with a photograph
obtained from State's Consular Consolidated Database.[Footnote 47] The
inspector at the booth scans the foreign national's fingerprints (left
and right index fingers) and takes a digital photograph. This
information is forwarded to the IDENT database, where it is checked
against stored fingerprints in the IDENT lookout database. If the
foreign national's fingerprints are already in IDENT, the system
performs a match (a comparison of the fingerprint taken during the
primary inspection to the one on file) to confirm that the person
submitting the fingerprints is the person on file. If no prints are
currently in IDENT, the foreign national is enrolled in US-VISIT (i.e.,
biographic and biometric data are entered into IDENT).
During this process, the inspector also questions the foreign national
about the purpose of his or her travel and length of stay. The
inspector adds the class of admission and duration of stay information
into the Treasury Enforcement Communications Systems,[Footnote 48] and
stamps the "admit until" date on the Form I-94.[Footnote 49] If the
foreign national is ultimately determined to be inadmissible, the
person is detained, lookouts are posted in the databases, and
appropriate actions are taken.
Status Management Process:
The status management process manages the foreign national's temporary
presence in the United States, including the adjudication of benefits
applications and investigations into possible violations of immigration
regulations.
As part of this process, commercial air and sea carriers transmit
departure manifests electronically for each departing passenger. These
manifests are transmitted through APIS and shared with the Arrival
Departure Information System (ADIS).[Footnote 50] ADIS matches entry
and exit manifest data (i.e., each record showing a foreign national
entering the United States is matched with a record showing the foreign
national exiting the United States). ADIS also receives status
information from the Computer Linked Application Information Management
System[Footnote 51] and the Student Exchange Visitor Information
System[Footnote 52] on foreign nationals.
Exit Process:
The exit process includes the carriers' submission of electronic
manifest data to APIS. This biographic information is transmitted to
ADIS, where it is matched against entry information. At the 11 POEs
where the exit solution is being implemented, the departure is
processed by one of three exit methods. Within each port, one or more
of the exit methods may be used. The three methods are as follows:
* Kiosk: At the kiosk, the traveler, guided by a workstation attendant
if needed, scans the machine-readable travel documents, provides
electronic fingerprints, and has a digital photograph taken. A receipt
is printed to provide documentation of compliance with the exit process
and to assist in compliance on the traveler's next attempted entry to
the country. After the receipt prints, the traveler proceeds to his or
her departure gate. At the conclusion of the transaction, the collected
information is transmitted to IDENT.
* Mobile device: At the departure gate, and just before the traveler
boards the departure craft, either a workstation attendant or law
enforcement officer scans the machine-readable travel documents, scans
the traveler's fingerprints (right and left index fingers), and takes a
digital photograph. A receipt is printed to provide documentation of
compliance with the exit process and to assist in compliance on the
traveler's next attempted entry to the country. The device wirelessly
transmits the captured data in real time to IDENT via the
Transportation Security Administration's Data Operations Center.
If the device is being operated by a workstation attendant, he or she
provides a printed receipt to the traveler, and the traveler then
boards the departure craft. If the mobile device is being operated by a
law enforcement officer, the captured biographic and biometric
information is checked in near real time against watch lists. Any
potential match is returned to the device and displayed visually for
the officer. If no match is found, the traveler is allowed to board the
departure craft.
* Validator: Using a kiosk, the traveler, guided by a workstation
attendant if needed, scans the machine-readable travel documents,
provides electronic fingerprints, and has a digital photograph taken.
As with the kiosk, a receipt is printed to provide documentation of
compliance with the exit process and to assist in compliance on the
traveler's next attempted entry to the country. However, this receipt
has biometrics (i.e., the traveler's fingerprints and photograph)
embedded on the receipt. At the conclusion of the transaction, the
collected information is transmitted to IDENT.
The traveler presents his or her receipt to the attendant or law
enforcement officer at the gate or departure area, who scans the
receipt using a mobile device. The traveler's identity is verified
against the biometric data embedded on the receipt. Once the traveler's
identity is verified, he or she is allowed to board the departure
craft. The captured data are not transmitted in real time back to
IDENT. Instead, the data are periodically uploaded through the kiosk to
IDENT.
Analysis Process:
An analysis capability is to provide for the continuous screening
against watch lists of individuals enrolled in US-VISIT for appropriate
reporting and action. As more entry and exit information becomes
available, it is to be used for analysis of traffic volume and patterns
as well as for risk assessments. The analysis is also to be used to
support resource and staffing projections across POEs, strategic
planning for integrated border management analysis performed by the
intelligence community, and determination of travel use levels and
expedited traveler programs.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Randolph C. Hite, (202) 512-3439 or [Hyperlink, hiter@gao.gov]:
Staff Acknowledgments:
In addition to the contact named above, the following people made key
contributions to this report: Deborah Davis, Assistant Director; Hal
Brumm; Tonia Brown; Joanna Chan; Barbara Collier; Neil Doherty;
Jennifer Echard; James Houtz; Scott Pettis; Karen Richey; and Karl
Seifert.
(310606):
FOOTNOTES
[1] Our previous reports regarding US-VISIT's expenditure plans, which
include recommendations, were published in GAO, Homeland Security: Some
Progress Made, but Many Challenges Remain on U.S. Visitor and Immigrant
Status Indicator Technology Program, GAO-05-202 (Washington, D.C.: Feb.
23, 2005); Homeland Security: First Phase of Visitor and Immigration
Status Program Operating, but Improvements Needed, GAO-04-586
(Washington, D.C.: May 11, 2004); Homeland Security: Risks Facing Key
Border and Transportation Security Program Need to Be Addressed, GAO-
03-1083 (Washington, D.C.: Sept. 19, 2003); and Information Technology:
Homeland Security Needs to Improve Entry Exit System Expenditure
Planning, GAO-03-563 (Washington, D.C.: June 9, 2003).
[2] Our reports included 24 recommendations, of which 6 related
specifically to the contents of the expenditure plan. Those 6 are not
included in the scope of this report, but they will be included in the
scope of our fiscal year 2006 expenditure plan review.
[3] We considered a recommendation (1) completely implemented when
documentation demonstrated that it had been fully addressed, (2)
partially implemented when documentation indicated that actions were
under way to implement it, and (3) in progress when documentation
indicated that actions had been initiated to implement it.
[4] Biometric comparison is a means of identifying a person by
biological features unique to that individual.
[5] An indefinite-delivery/indefinite-quantity contract provides for an
indefinite quantity, within stated limits, of supplies or services
during a fixed period of time. The government schedules deliveries or
performance by placing orders with the contractor.
[6] The Visa Waiver Program permits foreign nationals from designated
countries to apply for admission to the United States for a maximum of
90 days as nonimmigrant visitors for business or pleasure.
[7] On September 30, 2004, US-VISIT expanded biometric entry procedures
to include individuals from visa waiver countries applying for
admission.
[8] Workstation attendants assist travelers in using the kiosk.
[9] Form I-94s are used to record a foreign national's entry into the
United States. The form has two parts--arrival and departure--and each
part contains a unique number for the purposes of recording and
matching the arrival and departure records of nonimmigrants.
[10] RF technology relies on proximity cards and card readers. RF
devices read the information contained on the card when the card is
passed near the device and can also be used to verify the identity of
the cardholder.
[11] At one POE, these capabilities were deployed by December 19, 2005,
but were not fully operational until January 7, 2006, because of a
telephone company strike that prevented the installation of a T-1 line.
[12] GAO-05-202, GAO-04-586, GAO-03-1083, and GAO-03-563.
[13] As previously mentioned, the remaining 6 recommendations related
specifically to the contents of the expenditure plans and are not
reported on in this report; their status will be included in the scope
of our fiscal year 2006 expenditure plan review.
[14] GAO-03-563.
[15] In March 2003, the Immigration and Naturalization Service was
subsumed within DHS, and, in April 2003, the entry exit program became
known as US-VISIT.
[16] OMB, Security of Federal Automated Information Resources, Circular
A-130, Revised (Transmittal Memorandum No. 4), Appendix III
(Washington, D.C.: Nov. 28, 2000); and National Institute of Standards
and Technology, Guide for Developing Security Plans for Information
Technology Systems, Special Publication 800-18 (December 1998).
[17] The initial assessment was updated in September 2004 to reflect
the inclusion of Visa Waiver Program travelers in US-VISIT, the
expansion of US-VISIT to the 50 busiest land border POEs (Increment
2B), and changes in the business processes used by DHS to share
information with federal law enforcement agencies. The assessment was
again updated in June 2005 to include the live test to read
biometrically enabled travel documents (Increment 2A).
[18] OMB, Guidance for Implementing the Privacy Provisions of the E-
Government Act of 2002, OMB M-03-22 (Sept. 26, 2003).
[19] GAO-03-1083.
[20] Carnegie Mellon University Software Engineering Institute,
Capability Maturity Model Integration, Systems Engineering Integrated
Product and Process Development, Continuous Representation, version 1.1
(March 2002).
[21] When we made our original recommendation, we referred to an
earlier SEI model, the Software Acquisition Capability Maturity Model.
However, SEI is transitioning to an integrated model, and the program
office is using the CMMI model for its improvement program.
[22] The 7 remaining process areas are supplier agreement management,
measurement and analysis, solicitation and contract monitoring,
transition to operations and support, organizational training,
organizational process focus, and organizational process definition.
[23] OMB, Planning, Budgeting, Acquisition and Management of Capital
Assets, Circular A-11, Part 7 (Washington, D.C.: June 21, 2005).
[24] GAO-05-202 and GAO-03-1083.
[25] OMB, Guidelines and Discount Rates for Benefits-Cost Analysis of
Federal Programs, Circular A-94 (Washington, D.C.: Oct. 29, 1992).
[26] Department of Homeland Security, Capital Planning and Investment
Control: Cost-Benefit Analysis Workbook (Washington, D.C.: May 2003).
[27] Uncertainty analyses generally include both a sensitivity analysis
and a Monte Carlo simulation. A sensitivity analysis is a quantitative
assessment of the effect that a change in an assumption--the numerical
value of a single parameter (such as unit labor cost)--will have on net
present value. A Monte Carlo simulation allows all of the model's
parameters to vary simultaneously according to their associated
probability distribution. The result is a set of estimated
probabilities of achieving alternative outcomes (costs, benefits,
and/or net benefits), given the uncertainty in the underlying
parameters.
[28] GAO-05-202 and GAO-04-586.
[29] The Systems Assurance Manager stated that she has only two staff,
including herself, for ensuring testing quality of the US-VISIT
composite system.
[30] Form I-94W is used for foreign nationals from visa waiver
countries.
[31] The sites were Douglas, Arizona; Port Huron, Michigan; and Laredo,
Texas.
[32] GAO-05-202.
[33] Carnegie Mellon University Software Engineering Institute, A
Manager's Checklist for Validating Software Cost and Schedule
Estimates, CMU/SEI-95-SR-004 (January 1995).
[34] One criterion--when a dictated schedule is imposed, an estimate of
the normal schedule is compared to the additional expenditures required
to meet the dictated schedule--was not applicable because a schedule
was not imposed.
[35] The initial plan was to expand the pilot to 15 sites, but 4 of the
sites were not fully operational in time to be evaluated. According to
the Pilot Evaluation Report, this was largely due to the lengthy
security clearance process for workstation attendants, who assist
travelers in using one of the exit devices.
[36] The other two evaluation criteria were conduciveness to travel and
cost.
[37] Compliance rate for kiosk was 23 percent; for the mobile device,
36 percent; and for the validator, 26 percent.
[38] ACE is a new trade processing system planned to support the
movement of legitimate imports and exports and strengthen border
security.
[39] OMB, Security of Federal Automated Information Resources, Circular
A-130, Revised (Transmittal Memorandum No. 4), Appendix III
(Washington, D.C.: Nov. 28, 2000); and National Institute of Standards
and Technology, Guide for Developing Security Plans for Information
Technology Systems, Special Publication 800-18 (December 1998).
[40] Department of Homeland Security, US-VISIT System Security
Management Needs Strengthening (Redacted), Office of Inspector General,
OIG-06-16 (Washington, D.C.: December 2005).
[41] OMB, Guidelines and Discount Rates for Benefits-Cost Analysis of
Federal Programs, Circular A-94 (Washington, D.C.: Oct. 29, 1992); and
Department of Homeland Security, Capital Planning and Investment
Control: Cost-Benefit Analysis Workbook (Washington, D.C.: May 2003).
[42] See, for example, OMB, Guidance for Implementing the Privacy
Provisions of the E-Government Act of 2002, OMB M-03-22 (Sept. 26,
2003); and Planning, Budgeting, Acquisition and Management of Capital
Assets, Circular A-11, Part 7 (Washington, D.C.: June 21, 2005).
[43] OMB, Planning, Budgeting, Acquisition and Management of Capital
Assets, Circular A-11, Part 7 (Washington, D.C.: June 21, 2005) and
Guidelines and Discount Rates for Benefits-Cost Analysis of Federal
Programs, Circular A-94 (Washington, D.C.: Oct. 29, 1992).
[44] Carnegie Mellon University Software Engineering Institute, A
Manager's Checklist for Validating Software Cost and Schedule
Estimates, CMU/SEI-95-SR-004 (January 1995).
[45] IDENT collects and stores biometric data about foreign nationals,
including Federal Bureau of Investigation information on all known and
suspected terrorists, selected wanted persons (foreign-born, unknown
place of birth, previously arrested by DHS), and previous criminal
histories for high-risk countries; DHS Immigration and Customs
Enforcement information on deported felons and sexual registrants; and
DHS information on previous criminal histories and previous IDENT
enrollments. Information from the FBI includes fingerprints from the
Integrated Automated Fingerprint Identification System.
[46] Enhanced Border Security and Visa Entry Reform Act of 2002, Pub.
L. No. 107-173 (May 14, 2002).
[47] The Consular Consolidated Database is a system that includes
information on whether a visa applicant has previously applied for a
visa or currently has a valid visa.
[48] Treasury Enforcement Communications Systems maintains lookout data
and interfaces with other agencies' databases; it is currently used by
inspectors at POEs to verify traveler information and update traveler
data.
[49] The Form I-94 is used to track the arrival and departure of
nonimmigrants. It is divided into two parts. The first part is an
arrival portion, which includes, for example, the nonimmigrant's name,
date of birth, and passport number. The second part is a departure
portion, which includes the name, date of birth, and country of
citizenship.
[50] ADIS is a database that stores traveler arrival and departure data
and that provides query and reporting functions.
[51] The Computer Linked Application Information Management System is a
system that contains information on foreign nationals who request
benefits, such as change of status or extension of stay.
[52] The Student Exchange Visitor Information System is a system that
contains information on foreign students.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
