Homeland Security
Visitor and Immigrant Status Program Operating, but Management Improvements Are Still Needed Gao ID: GAO-06-318T January 25, 2006The Department of Homeland Security (DHS) has established a program--the U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals who enter and exit the United States. US-VISIT uses these biometric identifiers (digital fingerscans and photographs) to screen persons against watch lists and to verify that a visitor is the person who was issued a visa or other travel document. Visitors are also to confirm their departure by having their visas or passports scanned and undergoing fingerscanning at selected air and sea ports of entry. GAO was asked to testify on (1) the status of US-VISIT and (2) DHS progress in implementing recommendations that GAO made as part of its prior reviews of US-VISIT annual expenditure plans. The testimony is based on GAO's prior reports as well as ongoing work for the House Committee on Homeland Security. GAO's recommendations are directed at helping the department improve its capabilities to deliver US-VISIT capability and benefit expectations on time and within budget. According to DHS, the recommendations have made US-VISIT a stronger program.
The US-VISIT program has met a number of demanding requirements that were mandated in legislation. A pre-entry screening capability is in place in overseas visa issuance offices, and an entry identification capability is operating at 115 airports, 14 seaports, and 154 land ports of entry. This has been accomplished during a period of DHS-wide change, and has resulted in preventing criminal aliens from entering the country and potentially deterring others from even attempting to do so. Nevertheless, DHS has more to do to implement GAO recommendations aimed at better ensuring that US-VISIT is maximizing its potential for success and holding itself accountable for results. DHS has taken steps to address those GAO recommendations intended to ensure that US-VISIT as defined is the "right thing." For example, it is clarifying the strategic context within which US-VISIT is to operate, having drafted a strategic plan to show how US-VISIT is aligned with DHS's mission goals and operations and to provide an overall vision for immigration and border management. However, the plan has yet to be approved, causing its integration with other departmentwide border security initiatives to remain unclear. In addition, the department has analyzed the program's costs, benefits, and risks, but its analyses do not yet demonstrate that the program is producing or will produce mission value commensurate with expected costs and risks. In particular, the department's return-on-investment analyses for exit options do not demonstrate that these solutions will be cost-effective. DHS has also taken steps to address those GAO recommendations aimed at ensuring that the program is executed in the "right way." The department has made good progress in establishing the program's human capital capabilities, which should help ensure that it has sufficient staff with the necessary skills and abilities. This is particularly important in light of the program's more limited progress in establishing capabilities in certain program management process areas, such as test management. For example, a test plan used in a recent system acceptance test did not adequately trace between test cases and the requirements to be verified by testing. Incomplete test plans reduce assurance that systems will perform as intended once they are deployed. DHS also has begun addressing GAO's recommendations to establish accountability for program performance and results, but more needs to be done. For example, DHS's expenditure plans have not described progress against commitments made in previous plans. Unless performance against commitments is measured and disclosed, the ability to manage and oversee the program will suffer. The longer the program proceeds without fully addressing GAO's recommendations, the greater the risk that it will not deliver promised capabilities and benefits on time and within budget.
GAO-06-318T, Homeland Security: Visitor and Immigrant Status Program Operating, but Management Improvements Are Still Needed
This is the accessible text file for GAO report number GAO-06-318T
entitled 'Homeland Security: Visitor and Immigrant Status Program
Operating, but Management Improvements Are Still Needed' which was
released on January 25, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Testimony:
Before the Subcommittee on Homeland Security, Committee on
Appropriations, U.S. Senate:
For Release on Delivery:
Expected at 10 a.m. EST January 25, 2006:
Homeland Security:
Visitor and Immigrant Status Program Operating, but Management
Improvements Are Still Needed:
Statement of Randolph C. Hite,
Director:
Information Technology Architecture and Systems Issues:
GAO-06-318T:
GAO Highlights:
Highlights of GAO-06-318T, a testimony before the Subcommittee on
Homeland Security, Committee on Appropriations, U.S. Senate:
Why GAO Did This Study:
The Department of Homeland Security (DHS) has established a program”the
U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT)”to
collect, maintain, and share information, including biometric
identifiers, on selected foreign nationals who enter and exit the
United States. US-VISIT uses these biometric identifiers (digital
fingerscans and photographs) to screen persons against watch lists and
to verify that a visitor is the person who was issued a visa or other
travel document. Visitors are also to confirm their departure by having
their visas or passports scanned and undergoing fingerscanning at
selected air and sea ports of entry.
GAO was asked to testify on (1) the status of US-VISIT and (2) DHS
progress in implementing recommendations that GAO made as part of its
prior reviews of US-VISIT annual expenditure plans. The testimony is
based on GAO‘s prior reports as well as ongoing work for the House
Committee on Homeland Security. GAO‘s recommendations are directed at
helping the department improve its capabilities to deliver US-VISIT
capability and benefit expectations on time and within budget.
According to DHS, the recommendations have made US-VISIT a stronger
program.
What GAO Found:
The US-VISIT program has met a number of demanding requirements that
were mandated in legislation. A pre-entry screening capability is in
place in overseas visa issuance offices, and an entry identification
capability is operating at 115 airports, 14 seaports, and 154 land
ports of entry. This has been accomplished during a period of DHS-wide
change, and has resulted in preventing criminal aliens from entering
the country and potentially deterring others from even attempting to do
so.
Nevertheless, DHS has more to do to implement GAO recommendations aimed
at better ensuring that US-VISIT is maximizing its potential for
success and holding itself accountable for results.
* DHS has taken steps to address those GAO recommendations intended to
ensure that US-VISIT as defined is the ’right thing.“ For example, it
is clarifying the strategic context within which US-VISIT is to
operate, having drafted a strategic plan to show how US-VISIT is
aligned with DHS‘s mission goals and operations and to provide an
overall vision for immigration and border management. However, the plan
has yet to be approved, causing its integration with other
departmentwide border security initiatives to remain unclear. In
addition, the department has analyzed the program‘s costs, benefits,
and risks, but its analyses do not yet demonstrate that the program is
producing or will produce mission value commensurate with expected
costs and risks. In particular, the department‘s return-on-investment
analyses for exit options do not demonstrate that these solutions will
be cost-effective.
* DHS has also taken steps to address those GAO recommendations aimed
at ensuring that the program is executed in the ’right way.“ The
department has made good progress in establishing the program‘s human
capital capabilities, which should help ensure that it has sufficient
staff with the necessary skills and abilities. This is particularly
important in light of the program‘s more limited progress in
establishing capabilities in certain program management process areas,
such as test management. For example, a test plan used in a recent
system acceptance test did not adequately trace between test cases and
the requirements to be verified by testing. Incomplete test plans
reduce assurance that systems will perform as intended once they are
deployed.
* DHS also has begun addressing GAO‘s recommendations to establish
accountability for program performance and results, but more needs to
be done. For example, DHS‘s expenditure plans have not described
progress against commitments made in previous plans. Unless performance
against commitments is measured and disclosed, the ability to manage
and oversee the program will suffer.
The longer the program proceeds without fully addressing GAO‘s
recommendations, the greater the risk that it will not deliver promised
capabilities and benefits on time and within budget.
www.gao.gov/cgi-bin/getrpt?GAO-06-318T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
We appreciate the opportunity to participate in the Subcommittee's
hearing on US-VISIT (the United States Visitor and Immigrant Status
Indicator Technology), a multibillion-dollar program of the Department
of Homeland Security (DHS) that is intended to achieve a daunting set
of goals: to enhance the security of our citizens and visitors and
ensure the integrity of the U.S. immigration system, and at the same
time to facilitate legitimate trade and travel and protect privacy. To
achieve these goals, US-VISIT is to record the entry into and exit from
the United States of selected travelers, verify their identity, and
determine their compliance with the terms of their admission and stay.
Since fiscal year 2002, the House and Senate Appropriations Committees
have provided valuable oversight and direction to DHS on US-VISIT by
legislatively directing it to submit annual expenditure plans for
committee approval. This legislation also directed us to review these
plans. Our reviews have produced four reports that, among other things,
described DHS progress against legislatively mandated milestones and
identified fundamental challenges that the department faced in
delivering promised program capabilities and benefits on time and
within cost.[Footnote 1] For example, we reported in September 2003
that the program office did not have the human capital and acquisition
process discipline needed to effectively manage the program. In light
of the challenges that we identified, we concluded that the program
carries an appreciable level of risk, meaning that it must be managed
effectively if it is to be successful.
Managing US-VISIT effectively requires high levels of capability and
expertise. Fundamentally, it entails being able to respond
affirmatively to two basic questions. First, are we doing the right
thing? To be sure that a program is doing the right thing, it needs to
be justified by sufficient fact-based and verifiable analysis to show
that the program as defined will properly fit within the larger
homeland security operational and technological environments and that
it will produce mission value commensurate with expected costs and
risks. The second question is, are we doing it the right way? To be
done the right way, a program needs to be executed in a rigorous and
disciplined manner, which means that it needs to employ the necessary
mix of people, processes, and tools to reasonably ensure that promised
program capabilities and expected mission value are delivered on time
and within budget. Beyond these two questions, effective program
management also means that the program is held accountable for results,
which involves measuring and disclosing performance relative to
explicitly defined program goals, outcomes, and commitments.
Over the last 4 years, our reports have provided recommendations to DHS
to ensure that these questions are answered and used as the basis for
informed decision making about US-VISIT. They have also provided
recommendations to promote DHS accountability for the program. These
recommendations have been aimed at helping the department to ensure
that this program fulfills expectations: in other words, that the
program is doing the right thing in the right way, and that it is
holding itself accountable for doing so. According to DHS, the
recommendations have made US-VISIT a stronger program. Further, they
concur with the need to implement them with due speed and diligence.
My statement will describe the status of US-VISIT and where the
department now stands in implementing these recommendations and thus in
addressing the challenges that it faces. It is based on our
aforementioned reports to the Appropriations Committees and our ongoing
work for the House Committee on Homeland Security. All work on which
this testimony is based was performed in accordance with generally
accepted government auditing standards.
Results in Brief:
To its credit, the US-VISIT program has met a number of legislatively
mandated requirements. A pre-entry screening capability is in place in
visa issuance offices, and an entry identification capability is
available at 115 airports, 14 seaports, and in the secondary inspection
areas[Footnote 2] of 154 land ports of entry. This has been
accomplished despite the considerable departmental change occurring
around the program, and according to DHS, it has prevented criminal
aliens from entering the United States, besides probably deterring
other criminals and terrorists from attempting to enter through these
ports.
Our recommendations over the last 4 years have been aimed at helping
DHS meet its US-VISIT obligations by ensuring that it is doing the
right thing in the right way, and that the department holds itself
accountable for results. To address these recommendations, DHS has
taken a number of steps. To help ensure that is doing the right thing,
the department is in the process of clarifying the strategic context in
which US-VISIT is to operate; it has analyzed the program's costs,
benefits, and risks; and it has begun analyzing program impacts and
options that will provide a basis for future program increments.
However, the program's fit within the department's operational and
technology context remains unclear, and DHS has yet to demonstrate that
early program increments are producing or will produce mission value
commensurate with expected costs and risks. In particular, the
department's return on investment analyses for exit solutions do not
demonstrate that investment options will be cost-effective.
On our recommendations aimed at ensuring that the program is executed
in the right way, DHS has made mixed progress. For example, the
department has made good progress in establishing the program's human
capital capabilities, which is important, because progress in
establishing program management process controls, such as test
management, has not been as good. For example, a test plan used in a
recent system acceptance test did not adequately trace between test
cases and the requirements to be verified by testing. As we have
previously reported, incomplete test plans reduce assurance that
systems will perform as intended once they are deployed. Our experience
in reviewing large, complex programs like US-VISIT has shown that such
process management weaknesses typically result in programs falling
short of expectations.
With regard to our recommendations for establishing accountability for
program results by measuring and disclosing performance relative to
program goals, outcomes, requirements, and commitments, more also
remains to be done. For example, DHS has yet to define performance
standards that reflect limitations of the existing systems that make up
US-VISIT. Also, its expenditure plans have not described progress
against commitments made in previous plans. Unless performance against
requirements and commitments is measured and disclosed, the ability to
manage and oversee the program will suffer.
Background:
US-VISIT is a governmentwide program intended to enhance the security
of U.S. citizens and visitors, facilitate legitimate travel and trade,
ensure the integrity of the U.S. immigration system, and protect the
privacy of our visitors. The scope of the program includes the pre-
entry, entry, status, and exit of hundreds of millions of foreign
national travelers who enter and leave the United States at over 300
air, sea, and land ports of entry, as well as analytical capabilities
spanning this overall process.
To achieve its goals, US-VISIT uses biometric information (digital
fingerscans and photographs) to verify identity and screen persons
against watch lists.[Footnote 3] In many cases, the US-VISIT process
begins overseas, at U.S. consular offices, which collect biometric
information from applicants for visas, and check this information
against a database of known criminals and suspected terrorists. When a
visitor arrives at a port of entry, the biometric information is used
to verify that the visitor is the person who was issued the visa or
other travel documents. Ultimately, visitors are to confirm their
departure by having their visas or passports scanned and undergoing
fingerscanning. (Currently, at a few pilot sites, departing visitors
are asked to undergo these exit procedures.) The exit confirmation is
added to the visitor's travel records to demonstrate compliance with
the terms of admission to the United States.
Other key US-VISIT functions include:
* collecting, maintaining, and sharing information on certain foreign
nationals who enter and exit the United States;
* identifying foreign nationals who (1) have overstayed or violated the
terms of their admission; (2) may be eligible to receive, extend, or
adjust their immigration status; or (3) should be apprehended or
detained by law enforcement officials;
* detecting fraudulent travel documents, verifying traveler identity,
and determining traveler admissibility through the use of biometrics;
and:
* facilitating information sharing and coordination within the
immigration and border management community.
In July 2003, DHS established a program office with responsibility for
managing the acquisition, deployment, operation, and sustainment of the
US-VISIT system and its associated supporting people (e.g., Customs and
Border Protection officers), processes (e.g., entry/exit policies and
procedures), and facilities (e.g., inspection booths and lanes).
As of October 2005, about $1.4 billion has been appropriated for the
program, and according to program officials, about $962 million has
been obligated to acquire, develop, deploy, operate, and maintain US-
VISIT entry capabilities, and to test and evaluate exit capability
options.
Acquisition and Implementation Strategy:
DHS plans to deliver US-VISIT capability in four increments, with
Increments 1 through 3 being interim, or temporary, solutions that
fulfill legislative mandates to deploy an entry/exit system, and
Increment 4 being the implementation of a long-term vision that is to
incorporate improved business processes, new technology, and
information sharing to create an integrated border management system
for the future. In Increments 1 through 3, the program is building
interfaces among existing ("legacy") systems, enhancing the
capabilities of these systems, and deploying these capabilities to air,
sea, and land ports of entry. These first three increments are to be
largely acquired and implemented through existing system contracts and
task orders.
In May 2004, DHS awarded an indefinite-delivery/indefinite-
quantity[Footnote 4] prime contract to Accenture and its partners.
According to the contract, the prime contractor will help support the
integration and consolidation of processes, functionality, and data,
and it will develop a strategy to build on the technology and
capabilities already available to produce the strategic solution, while
also assisting the program office in leveraging existing systems and
contractors in deploying the interim solutions.
US-VISIT Is Being Implemented in Four Increments:
Increment 1 concentrates on establishing capabilities at air and sea
ports of entry. It is divided into two parts--1A and 1B.
* Increment 1A (air and sea entry) includes the electronic capture and
matching of biographic and biometric information (two digital index
fingerscans and a digital photograph) for selected foreign nationals,
including those from visa waiver countries.[Footnote 5] Increment 1A
was deployed on January 5, 2004, through the modification of pre-
existing systems.[Footnote 6] These modifications accommodated the
collection and maintenance of additional data fields and established
interfaces required to share data among DHS systems in support of entry
processing at 115 airports and 14 seaports.
* Increment 1B (air and sea exit) involves the testing of exit devices
to collect biometric exit data for select foreign nationals. Three exit
alternatives were pilot tested at 11 air and sea ports of entry. These
alternatives are as follows.
* Kiosk--A self-service device (including a touch screen interface,
document scanner, finger scanner, digital camera, and receipt printer)
that captures a digital photograph and fingerprint and prints out an
encoded receipt.
* Mobile device--A hand-held device that is operated by a workstation
attendant and includes a document scanner, finger scanner, digital
camera, and receipt printer to capture a digital photograph and
fingerprint.
* Validator--A hand-held device that is used to capture a digital
photograph and fingerprint, which are then matched to the photograph
and fingerprint captured via the kiosk and encoded in the receipt.
Increment 2 focuses primarily on extending US-VISIT to land ports of
entry. It is divided into three parts--2A, 2B, and 2C.
* Increment 2A (air, sea, and land entry) includes the capability to
biometrically compare and authenticate valid machine-readable visas and
other travel and entry documents at all ports of entry. Increment 2A
was deployed on October 23, 2005, according to program officials. It
also includes the deployment by October 26, 2006, of the capability to
read biometrically enabled passports from visa waiver countries.
* Increment 2B (land entry) redesigned the Increment 1 entry solution
and expanded it to the 50 busiest land ports of entry. The process for
issuing entry/exit forms[Footnote 7] was redesigned to enable the
electronic capture of biographic, biometric (unless the traveler is
exempt), and related travel documentation for arriving travelers. This
increment was deployed to the busiest 50 U.S. land border ports of
entry on December 29, 2004. Before Increment 2B, all information on the
entry/exit forms was hand written. The redesigned process provides for
electronically capturing the biographic data on the entry/exit form. In
some cases, Customs and Border Protection (CBP) officers enter the data
electronically and then print the completed form.
* Increment 2C (land entry and exit) is to provide the capability to
automatically, passively, and remotely record the entry and exit of
covered individuals using radio frequency (RF) technology tags at
primary inspection and exit lanes.[Footnote 8] This tag includes a
unique ID number that is to be embedded in each entry/exit form, thus
associating a unique number with a US-VISIT record for the person
holding that form. One of DHS's goals in using this technology is to
improve the ability to collect entry and exit information. In August
2005, the program office deployed the technology to three land ports of
entry to verify the feasibility of using passive RF technology to
record traveler entries and exits from the number embedded in the
entry/exit form. The results of this demonstration are to be reported
in February 2006.
Increment 3 extended Increment 2B (land entry) capabilities to 104 land
ports of entry; this increment was essentially completed as of December
19, 2005.[Footnote 9]
Increment 4 is the strategic US-VISIT program capability, which program
officials stated will likely consist of a further series of incremental
releases or mission capability enhancements that will support business
outcomes. The program reports that it has worked with its prime
contractor and partners to develop this overall vision for the
immigration and border management enterprise.
All increments before Increment 4 depend on the interfacing and
integration of existing systems,[Footnote 10] including the following:
* The Arrival and Departure Information System (ADIS) stores:
* noncitizen traveler arrival and departure data received from air and
sea carrier manifests,
* arrival data captured by CBP officers at air and sea ports of entry,
* I-94 issuance data captured by CBP officers at Increment 2B land
ports of entry,
* departure information captured at US-VISIT biometric departure pilot
(air and sea) locations,
* pedestrian arrival information and pedestrian and vehicle departure
information captured at Increment 2C port of entry locations, and:
* status update information provided by SEVIS and CLAIMS 3 (described
below).
* ADIS provides record matching, query, and reporting functions.
* The passenger processing component of the Treasury Enforcement
Communications System (TECS) includes two systems: Advance Passenger
Information System (APIS), a system that captures arrival and departure
manifest information provided by air and sea carriers, and the
Interagency Border Inspection System, a system that maintains lookout
data and interfaces with other agencies' databases. CBP officers use
these data as part of the admission process. The results of the
admission decision are recorded in TECS and ADIS.
* The Automated Biometric Identification System (IDENT) collects and
stores biometric data about foreign visitors.
* The Student and Exchange Visitor Information System (SEVIS) and the
Computer Linked Application Information Management System (CLAIMS 3)
contain information on foreign students and foreign nationals who
request benefits, such as change of status or extension of stay.
Some of these systems, such as IDENT, are managed by the program
office, while some systems are managed by other organizational entities
within DHS. For example, TECS is managed by CBP, SEVIS is managed by
Immigration and Customs Enforcement, CLAIMS 3 is under United States
Citizenship and Immigration Services, and ADIS is jointly managed by
CBP and US-VISIT.
US-VISIT also interfaces with other, non-DHS systems for relevant
purposes, including watch list updates and checks to determine whether
a visa applicant has previously applied for a visa or currently has a
valid U.S. visa. In particular, US-VISIT receives biographic and
biometric information from the Department of State's Consular
Consolidated Database as part of the visa application process, and
returns fingerscan information and watch list changes.
US-VISIT Capability Is Operating at Ports of Entry:
Over the last 3 years, US-VISIT program officials and supporting
contractor staff have worked to meet challenging legislative time
frames, as well as a DHS-imposed requirement to use biometric
identifiers. Under law, for example, DHS was to create an electronic
entry and exit system to screen and monitor the stay of foreign
nationals who enter and leave the United States and implement the
system at (1) air and sea ports of entry by December 31, 2003, (2) the
50 highest-volume land ports of entry by December 31, 2004, and (3) the
remaining ports of entry by December 31, 2005.[Footnote 11] It was also
to provide the means to collect arrival/departure data from
biometrically enabled and machine-readable travel documents at all
ports of entry.[Footnote 12]
To the program office's credit, it has largely met its obligations
relative to an entry capability. For example, on January 5, 2004, it
deployed and began operating most aspects of its planned entry
capability at 115 airports and 14 seaports, and added the remaining
aspects in February 2005. During 2004, it also deployed and began
operating this entry capability in the secondary inspection areas of
the 50 highest volume land ports of entry. As of December 19, 2005, it
had deployed and begun operating its entry capability at all but 1 of
the remaining 104 land ports of entry.[Footnote 13] The program has
also been working to define feasible and cost-effective exit solutions,
including technology feasibility testing at 3 land ports of entry and
operational performance evaluations at 11 air and sea ports of entry.
Moreover, the development and deployment of this entry capability has
occurred during a period of considerable organizational change,
starting with the creation of DHS from 23 separate agencies in early
2003, followed by the establishment of a US-VISIT program office
shortly thereafter--which was only about 5 months before it had to meet
its first legislative milestone. Compounding these program challenges
was the fact that the systems that were to be used in building and
deploying an entry capability were managed and operated by a number of
the separate agencies that had been merged to form the new department,
each of which was governed by different policies, procedures, and
standards.
As a result of the program's efforts to deploy and operate an entry
capability, DHS reports that it has been able to apprehend and prevent
the entry of hundreds of criminal aliens: as of March 2005, DHS
reported that more than 450 people with records of criminal or
immigration violations have been prevented from entering. For example,
its biometric screening prevented the reentry of a convicted felon,
previously deported, who was attempting to enter under an alias;
standard biographic record checks using only names and birth dates
would have likely cleared the individual.
Another potential consequence, although difficult to demonstrate, is
the deterrent effect of having an operational entry capability.
Although deterrence is not an expressly stated goal of the program,
officials have cited it as a potential byproduct of having a publicized
capability at the border to screen entry on the basis of identity
verification and matching against watch lists of known and suspected
terrorists. Accordingly, the deterrent potential of the knowledge that
unwanted entry may be thwarted and the perpetrators caught is arguably
a layer of security that should not be overlooked.
DHS Has Yet to Demonstrate that US-VISIT as Defined Is the Right
Solution:
A prerequisite for prudent investment in programs is having reasonable
assurance that a proposed course of action is the right thing to do,
meaning that it properly fits within the larger context of an agency's
strategic plans and related operational and technology environments,
and that the program will produce benefits in excess of costs over its
useful life. We have made recommendations to DHS aimed at ensuring that
this is in fact the case for US-VISIT, and the department has taken
steps intended to address our recommendations. These steps, however,
have yet to produce sufficient analytical information to demonstrate
that US-VISIT as defined is the right solution. Without this knowledge,
investment in the program cannot be fully justified.
Operational and Technological Context Are Still Being Defined:
Agency programs need to properly fit within a common strategic context
or frame of reference governing key aspects of program operations--
e.g., what functions are to be performed by whom, when and where they
are to be performed, what information is to be used to perform them,
and what rules and standards will govern the application of technology
to support them. Without a clear operational context for US-VISIT, the
risk is increased that the program will not interoperate with related
programs and thus not cost-effectively meet mission needs.
In September 2003 we reported that DHS had not defined key aspects of
the larger homeland security environment in which US-VISIT would need
to operate. For example, certain policy and standards decisions had not
been made, such as whether official travel documents would be required
for all persons who enter and exit the country--including U.S. and
Canadian citizens--and how many fingerprints would be collected.
Nonetheless, program officials were making assumptions and decisions at
that time that, if they turned out to be inconsistent with subsequent
policy or standards decisions, would require US-VISIT rework. To
minimize the impact of these changes, we recommended that DHS clarify
the context in which US-VISIT is to operate.
About 28 months later, defining this operational context remains a work
in progress. For example, the program's relationships and dependencies
with other closely allied initiatives and programs are still unclear.
According to the US-VISIT Chief Strategist, an immigration and border
management strategic plan was drafted in March 2005 that shows how US-
VISIT is aligned with DHS's organizational mission and that defines an
overall vision for immigration and border management. According to this
official, the vision provides for an immigration and border management
enterprise that unifies multiple internal departmental and other
external stakeholders with common objectives, strategies, processes,
and infrastructures. As of December 2005, however, we were told that
this strategic plan has not been approved.
In addition, since the plan was drafted, DHS has reported that other
relevant initiatives have been undertaken. For example:
* The DHS Security and Prosperity Partnership of North America is to,
among other things, establish a common approach to securing the
countries of North America--the United States, Canada, and Mexico--by,
for example, implementing a border facilitation strategy to build
capacity and improve the legitimate flow of people and cargo at our
shared borders.
* The DHS Secure Border Initiative is to implement a comprehensive
approach to securing our borders and combating illegal immigration.
According to the Chief Strategist, portions of the strategic plan are
being incorporated into these initiatives, but these initiatives and
their relationships with US-VISIT are still being defined.
Similarly, the mission and operational environment of US-VISIT are
related to those of another major DHS program--the Automated Commercial
Environment (ACE), which is a new trade processing system that is
planned to support the movement of legitimate imports and exports and
to strengthen border security. In addition, both US-VISIT and ACE could
potentially use common IT infrastructures and services. As we reported
in February 2005, the program office recognized these similarities, but
managing the relationship between the two programs had not been a
priority matter. Accordingly, we recommended that DHS give priority to
understanding the relationships and dependencies between the US-VISIT
and ACE programs.
Since our recommendation, the US-VISIT and ACE managers have formed an
integrated project team to, among other things, ensure that the two
programs are programmatically and technically aligned. Program
officials stated that the team has met three times since April 2005 and
plans to meet on a quarterly basis going forward. The team has
discussed potential areas of focus and agreed to three areas: RF
technology, program control, and data governance. However, it does not
have an approved charter, and it has not developed explicit plans or
milestone dates for identifying the dependencies and relationships
between the two programs.
It is important that DHS define the operational context for US-VISIT,
as well as its relationships and dependencies with closely allied
initiatives and such programs as ACE. The more time it takes to settle
these issues, the more likely that extensive and expensive rework will
be needed at a later date.
Return on Investment Has Yet to be Determined:
Prudent investment also requires that an agency have reasonable
assurance that a proposed program will produce mission value
commensurate with expected costs and risks. Thus far, DHS has yet to
develop an adequate basis for knowing that this is the case for its
early US-VISIT increments. Without this knowledge, it cannot adequately
ensure that these increments are justified.
Assessments of costs and benefits are extremely important, because the
decision to invest in any capability should be based on reliable
analyses of return on investment. According to OMB guidance, individual
increments of major systems are to be individually supported by
analyses of benefits, cost, and risk.[Footnote 14] In addition, OMB
guidance on the analysis needed to justify investments states that such
analysis should meet certain criteria to be considered
reasonable.[Footnote 15] These criteria include, among other things,
comparing alternatives on the basis of net present value and conducting
uncertainty analyses of costs and benefits. (DHS has also issued
guidance on such economic analyses, which is consistent with that of
OMB.[Footnote 16]) Without reliable analyses, an organization cannot be
reasonably assured that a proposed investment is a prudent and
justified use of resources.
In September 2003, we reported that the program had not assessed the
costs and benefits of Increment 1. Accordingly, we recommended that DHS
perform such assessments for future increments.[Footnote 17] In
February 2005, we reported that although the program office had
developed a cost-benefit analysis for Increment 2B (which provides the
capability for electronic collection of traveler information at land
ports of entry),[Footnote 18] it had again not justified the
investment, because its treatment of both benefits and costs was
unclear and insufficient.[Footnote 19] Further, we reported that the
cost estimates on which the cost-benefit analysis was based were of
questionable reliability, because effective cost-estimating practices
were not followed. Accordingly, we recommended that DHS follow certain
specified practices for estimating the costs of future
increments.[Footnote 20]
Since our February 2005 report, the program has developed a cost-
benefit analysis for Increment 1B (which is to provide exit
capabilities at air and sea ports of entry). The latest version of this
analysis, dated June 23, 2005, identifies potential costs and benefits
for three exit solutions at air and sea ports of entry and provides a
general rationale for the viability of the three alternatives
described.[Footnote 21] This latest analysis meets some but not all the
OMB criteria for economic analyses. For example, it explains why the
investment was needed, and it shows that at least two alternatives to
the status quo were considered. However, it does not include, for
example, a complete uncertainty analysis for the three exit
alternatives evaluated. That is, it does not include a sensitivity
analysis for the three alternatives, which is a major part of an
uncertainly analysis.[Footnote 22] (A sensitivity analysis is a
quantitative assessment of the effect that a change in a given
assumption--such as unit labor cost--will have on net present value.) A
complete analysis of uncertainty is important because it provides
decision makers with a perspective on the potential variability of the
cost and benefit estimates should the facts, circumstances, and
assumptions change.
In addition, the quality of a cost-benefit analysis is dependent on the
quality of the cost assessments on which it is based. However, the cost
estimate associated with the June 2005 cost-benefit analysis for the
three exit solutions (Increment 1B) did not meet key criteria for
reliable cost estimating. For example, it did not include a detailed
work breakdown structure. A work breakdown structure serves to organize
and define the work to be performed, so that associated costs can be
identified and estimated. Thus, it provides a reliable basis for
ensuring that the estimates include all relevant costs.
Program officials stated that they recognize the importance of
developing reliable cost estimates and have initiated actions to more
reliably estimate the costs of future increments. For example, the
program has chartered a cost analysis process action team, which is to
develop, document, and implement a cost analysis policy, process, and
plan for the program. Program officials also stated that they have
hired additional contracting staff with cost-estimating experience.
Strengthening the program's cost-estimating capability is extremely
important. The absence of reliable cost estimates impedes, among other
things, both the development of reliable economic justification for
program decisions and the effective measurement of performance.
Analysis of Program Impacts and Options Is Being Performed:
Program decisions and planning depend on adequate analyses and
assessments of program impacts and options. The department has begun to
develop such analyses, but some of these, such as its analyses of the
operational impact of Increment 2B and of the options for its exit
capability, do not yet provide an adequate basis for investment and
deployment decisions.
We reported in May 2004 that the program had not assessed its workforce
and facility needs for Increment 2B (which provides the capability for
electronic collection of traveler information at land ports of entry).
Because of this, we questioned the validity of the program's
assumptions and plans concerning workforce and facilities, since the
program lacked a basis for determining whether its assumptions were
correct and thus whether its plans were adequate. Accordingly, we
recommended that DHS assess the full impact of Increment 2B on
workforce levels and facilities at land ports of entry, including
performing appropriate modeling exercises.
Seven months later, the program office evaluated Increment 2B
operational performance, with the stated purpose of determining the
effectiveness of Increment 2B performance at the 50 busiest land ports
of entry. For this evaluation, the program office established a
baseline for comparing the average times to issue and process
entry/exit forms at 3 of these 50 ports of entry. The program office
then conducted two evaluations of the processing times at the three
ports, first after Increment 2B was deployed as a pilot, and next 3
months later, after it was deployed to all 50 ports of entry. The
evaluation results showed that the average processing times decreased
for all three sites. Program officials concluded that these results
supported their workforce and facilities planning assumptions that no
additional staff was required to support deployment of Increment 2B and
that minimal modifications were required at the facilities.[Footnote
23]
However, the scope of the evaluations is not sufficient to satisfy the
evaluations' stated purpose or our recommendation for assessing the
full impact of 2B. For example, the selection of the three sites,
according to program officials, was based on a number of factors,
including whether the sites already had sufficient staff to support the
pilot. Selecting sites based on this factor could affect the results,
and it presupposes that not all ports of entry have the staff needed to
support 2B. In addition, evaluation conditions were not always held
constant: specifically, fewer workstations were used to process
travelers in establishing the baseline processing times at two of the
ports of entry than were used during the pilot evaluations.
Moreover, CBP officials from a land port of entry that was not an
evaluation site (San Ysidro) told us that US-VISIT deployment has not
reduced but actually lengthened processing times. (San Ysidro processes
the highest volume of travelers of all land ports of entry.) Although
these officials did not provide specific data to support their
statement, their perception nevertheless raises questions about the
potential impact of Increment 2B on the 47 sites that were not
evaluated.
Similarly, in February 2005, we reported that US-VISIT had not
adequately planned for evaluating the alternatives for Increment 1B
(which provides exit capabilities at air and sea ports of entry)
because the scope and timeline of its exit pilot evaluation were
compressed. Accordingly, we recommended that DHS reassess plans for
deploying an exit capability to ensure that the scope of the exit pilot
provides for adequate evaluation of alternative solutions.
Over the last 11 months, the program office has taken actions to expand
the scope and time frames of the pilot. For example, it increased the
number of ports of entry in the pilot from 5 to 11, and it also
extended the time frame by about 7 months. Further, according to
program officials, they were able to achieve the target sample sizes
necessary to have a 95 percent confidence level in their results.
Nevertheless, questions remain about whether the exit alternatives have
been adequately evaluated to permit selection of the best exit solution
for national deployment. For example, one of the criteria against which
the alternatives were evaluated was the rate of traveler compliance
with US-VISIT exit policies (that is, foreign travelers providing
information as they exit the United States).[Footnote 24] However,
across the three alternatives, the average compliance with these
policies was only 24 percent, which raises questions as to their
effectiveness.[Footnote 25] The evaluation report cites several reasons
for the low compliance rate, including that compliance during the pilot
was voluntary. The report further concludes that national deployment of
the exit solution will not meet the desired compliance rate unless the
exit process incorporates an enforcement mechanism, such as not
allowing persons to reenter the United States if they do not comply
with the exit process. Although an enforcement mechanism might indeed
improve compliance, program officials stated that no formal evaluation
has been conducted of enforcement mechanisms or their possible effect
on compliance. The program director agreed that additional evaluation
is needed to assess the impact of implementing potential enforcement
mechanisms and plans to do such evaluation.
DHS Is Still Establishing Needed Program Management Capabilities:
Establishing effective program management capabilities is important to
ensure that an organization is going about delivering a program in the
right way. Accordingly, we have made recommendations to establish
specific people and process management capabilities. While DHS is
making progress in implementing many of our recommendations in this
area, this progress has often been slow.
One area in which DHS has made good progress is in implementing our
recommendations to establish the human capital capabilities necessary
to manage US-VISIT. In September 2003, we reported that the US-VISIT
program had not fully staffed or adequately funded its program office
or defined specific roles and responsibilities for program office
staff. Our prior experience with major acquisitions like US-VISIT shows
that to be successful, they need, among other things, to have adequate
resources, and program staff need to understand what they are to do,
how they relate to each other, and how they fit in their organization.
In addition, prior research and evaluations of organizations show that
effective human capital management can help agencies establish and
maintain the workforce they need to accomplish their missions.
Accordingly, we recommended that DHS ensure that human capital and
financial resources are provided to establish a fully functional and
effective program office, and that the department define program office
positions, roles, and responsibilities. We also recommended that DHS
develop and implement a human capital strategy for the program office
that provides for staffing positions with individuals who have the
appropriate knowledge, skills, and abilities.
DHS has implemented our recommendation that it define program office
positions, roles, and responsibilities, and it has partially completed
our two other people-related recommendations. It has filled most of its
planned government positions and is on the way to filling the rest, and
it has filled all of its planned contractor positions. However, the
program completed a workforce analysis in February 2005 and requested
additional positions based on the results. Securing these necessary
resources will be a continuing challenge.
In addition, as we reported in February 2005, the program office,
working with the Office of Personnel Management, developed a draft
human capital plan that employed widely accepted human capital planning
tools and principles (for example, it included an action plan that
identified activities, their proposed completion dates, and the office
responsible for the action). In addition, the program office had
completed some of the activities in the plan. Since then, the program
office has finalized the human capital plan, completed more activities,
and formulated plans to complete others (for example, according to the
program office, it has completed an analysis of its workforce to
determine diversity trends, retirement and attrition rates, and mission-
critical and leadership competency gaps, and it has plans to complete
an analysis of workforce data to maintain strategic focus on preserving
the skills, knowledge, and leadership abilities required for the US-
VISIT program's success).
Program officials also said that the reason they have not completed
several activities in the plan is that these activities are related to
the department's new human capital initiative, MAXHR.[Footnote 26]
Because this initiative is to include the development of departmentwide
competencies, program officials told us that it could potentially
affect ongoing program activities related to competencies. As a result,
these officials said that they are coordinating these activities
closely with the department as it develops and implements this new
initiative, which is currently being reviewed by the DHS Deputy
Secretary.
DHS's progress in implementing our human capital recommendations should
help ensure that it has sufficient staff with the right skills and
abilities to successfully execute the program. Having such staff has
been and will be particularly important in light of the program's more
limited progress to date in establishing program management process
capabilities.
DHS's progress in establishing effective processes governing how
program managers and staff are to perform their respective roles and
responsibilities has generally been slow. In our experience, weak
process management controls typically result in programs falling short
of expectations. From September 2003, we have made numerous
recommendations aimed at enabling the program to strengthen its process
controls in such areas as acquisition management, test management, risk
management,[Footnote 27] configuration management,[Footnote 28]
capacity management,[Footnote 29] security, privacy, and independent
verification and validation (IV&V).[Footnote 30] DHS has not yet
completed the implementation of any of our recommendations in these
areas, with one exception. It has ensured that the program office's
IV&V contractor was independent of the products and processes that it
was verifying and validating, as we recommended. In July 2005, the
program office issued a new contract for IV&V services after following
steps to ensure the contractor's independence (for example, IV&V
contract bidders were to be independent of the development and
integration contractors and are prohibited from soliciting, proposing,
or being awarded work for the program other than IV&V services). If
effectively implemented, these steps should adequately ensure that
verification and validation activities are performed in an objective
manner, and thus should provide valuable assistance to program managers
and decision makers.
In the other management areas, DHS has partially completed or has only
begun to address our recommendations, and more remains to be done. For
example, DHS has not completed the development and implementation of
key acquisition controls. We reported in September 2[Footnote 31]003
that the program office had not defined key acquisition management
controls to support the acquisition of US-VISIT, increasing the risk
that the program would not satisfy system requirements or meet benefit
expectations on time and within budget. Accordingly, we recommended
that DHS develop and implement a plan for satisfying key acquisition
management controls in accordance with best practi[Footnote 32]ces.:
The program office has recently taken steps to lay the foundation for
establishing key acquisition management controls. For example, it has
developed a process improvement plan to define and implement these
controls that includes a governance structure for overseeing
improvement activities. In addition, the program office has recently
completed a self-assessment of its acquisition process maturity, and it
plans to use the assessment results to establish a baseline of its
acquisition process maturity as a benchmark for improvement. According
to program officials, the assessment included key process areas that
are generally consistent with the process areas cited in our
recommendation. The program has ranked these process areas and plans to
focus on those with highest priority. (Some of these high-priority
process areas are also areas in which we have made recommendations,
such as configuration management and risk management.)
The improvement plan is currently being updated to reflect the results
of the baseline assessment and to include a work breakdown structure,
process prioritization, and resource estimates. According to a program
official, the goal is to conduct a formal appraisal to assess the
capability level of some or all of the high-priority process areas by
October 2006.
These recent steps provide a foundation for progress, but fully and
effectively implementing key acquisition management controls takes
considerable time, and DHS is still in the early stages of the process.
Therefore, it is important that these improvement efforts stay on
track. Until these controls are effectively implemented, US-VISIT will
be at risk of not delivering promised capabilities on time and within
budget.
Another management area of high importance to a complex program like US-
VISIT is test management. The purpose of system testing is to identify
and correct system defects before the system is deployed. To be
effective, testing activities should be planned and implemented in a
structured and disciplined fashion. Among other things, this includes
developing effective test plans to guide the testing activities and
ensuring that test plans are developed and approved before test
execution.
In this area also, DHS's progress responding to our recommendation has
been limited. We reported in May 2004, and again in February 2005, that
system testing was not based on well-defined test plans, and thus the
quality of testing being performed was at risk. Because DHS test plans
were not sufficiently well-defined to be effective, we recommended that
before testing begins, DHS develop and approve test plans that meet the
criteria that relevant systems development guidance prescribes for
effective test plans: namely, that they (1) specify the test
environment; (2) describe each test to be performed, including test
controls, inputs, and expected outputs; (3) define the test procedures
to be followed in conducting the tests; and (4) provide traceability
between the test cases and the requirements to be verified by the
testing.
About 20 months later, the quality of the system test plans, and thus
system testing, is still a challenge. To the program's credit, the test
plans for the Proof of Concept for Increment 2C, dated June 28, 2005
(which introduces RF technology to automatically record the entry and
exit of covered individuals), satisfied part of our recommendation.
Specifically, the test plan for this increment was approved on June 30,
2005, before testing began (according to program officials, it began on
July 5, 2005). Further, the test plan described, for example, the
scope, complexity, and completeness of the test environment; it
described the tests to be performed, including a high-level description
of controls, inputs, and outputs; and it identified the test procedures
to be performed.
However, the test plan did not adequately trace between test cases and
the requirements to be verified by testing. For example, about 70
percent of the requirements that we analyzed did not have specific
references to test cases. Further, we identified traceability
inconsistencies, such as one requirement that was mapped to over 50
test cases, even though none of the 50 cases referenced the
requirement.
Time and resource constraints were identified as the reasons that test
plans have not been complete. Specifically, program officials stated
that milestones do not permit existing testing/quality personnel the
time required to adequately review testing documents.[Footnote 33]
According to these officials, even when the start of testing activities
is delayed because, for example, requirements definition or product
development takes longer than anticipated, testing milestones are not
extended.
Without complete test plans, the program does not have adequate
assurance that the system is being fully tested, and thus unnecessarily
assumes the risk of system defects not being detected and addressed
before the system is deployed. This means that the system may not
perform as intended when deployed, and defects will not be addressed
until late in the systems development cycle, when they are more
difficult and time-consuming to fix. This has in fact happened already:
postdeployment system interface problems surfaced for Increment 1, and
manual work-arounds had to be implemented after the system was
deployed.
Until process management weaknesses such as these are addressed, the
program will continue to be overly dependent on the exceptional
performance of individuals to produce results. Such dependence
increases the risk of the US-VISIT program falling short of
expectations.
DHS Has Yet to Fully Establish Program Accountability Mechanisms:
To better ensure that US-VISIT and DHS meet expectations, we made
recommendations related to measuring and disclosing progress against
program commitments. Thus far, such performance and accountability
mechanisms have yet to be fully established. Measurements of the
operational performance of the system are necessary to ensure that the
system adequately supports mission operations, and measurements of
program progress and outcomes are important for demonstrating that the
program is on track and is producing results. Without such
measurements, program performance and accountability can suffer.
As we reported in September 2003, the operational performance of
initial system increments was largely dependent on the performance of
existing systems that were to be interfaced to create these increments.
For example, we said that the performance of an increment would be
constrained by the availability and downtime of the existing systems,
some of which had known problems in these areas. Accordingly, we
recommended that DHS define performance standards for each increment
that are measurable and that reflect the limitations imposed by this
reliance on existing systems. In February 2005, we reported that
several technical performance standards for increments 1 and 2B had
been defined, but that it was not clear that these standards reflected
the limitations imposed by the reliance on existing systems. Since
then, the program office has defined certain other technical
performance standards for the next increment (Increment 2C, Phase 1),
including standards for availability. Consistent with what we reported,
the functional requirements document states that these performance
standards are largely dependent upon those of the current systems, and
for system availability, it sets an aggregated availability standard
for Increment 2C components. However, the document does not contain
sufficient information for a determination of whether these performance
standards actually reflect the limitations imposed by reliance on
existing systems. Unless the program defines performance standards that
do this, it will be unable to identify and effectively address
performance shortfalls.
Similarly, as we observed in June 2003, to permit meaningful program
oversight, it is important that expenditure plans describe how well DHS
is progressing against the commitments made in prior expenditure plans.
The expenditure plan for fiscal year 2005 (the fourth US-VISIT
expenditure plan) does not describe progress against commitments made
in the previous plans. For example, according to the fiscal year 2004
plan, US-VISIT was to analyze, field test, and begin deploying
alternative approaches for capturing biometrics during the exit
process. However, according to the fiscal year 2005 plan, US-VISIT was
to expand its exit pilot sites during the summer and fall of 2004, and
it would not deploy the exit solution until fiscal year 2005. The plan
does not explain the reason for this change from its previous
commitment nor its potential impact. Nor does it describe the status of
the exit pilot testing or deployment, such as whether the program has
met its target schedule or whether the schedule has slipped.
Additionally, the fiscal year 2004 plan stated that $45 million in
fiscal year 2004 was to be used for exit activities. However, in the
fiscal year 2005 plan, the figure for exit activities was $73 million
in fiscal year 2004 funds. The plan does not highlight this difference
or address the reason for the change in amounts. Also, although the
fiscal year 2005 expenditure plan includes benefits stated in the
fiscal year 2004 plan, it does not describe progress in addressing
those benefits, even though in the earlier plan, US-VISIT stated that
it was developing metrics for measuring the projected benefits,
including baselines by which progress could be assessed. The fiscal
year 2005 plan again states that performance measures are under
development.
Figure 1 provides our analysis of the commitments made in the fiscal
year 2003 and 2004 plans, compared with progress reported and planned
in February 2005.
Figure 1: Time Line Comparing Commitments Made in the US-VISIT Fiscal
Year 2003 and 2004 Plans with Commitments and Reported Progress in the
Fiscal Year 2005 Plan:
[See PDF for image]
[End of figure]
The deployment of an exit capability, an important aspect of the
program that was to result from the exit pilots shown in the figure,
further illustrates missed commitments that need to be reflected in the
next expenditure plan. In the fiscal year 2005 expenditure plan, the
program committed to deploying an exit capability to air and sea ports
of entry by September 30, 2005. Although US-VISIT has completed its
evaluation of exit solutions at 11 pilot sites (9 airports and 2
seaports), no decision has yet been made on when an exit capability
will be deployed. According to program officials, deployment to further
sites would take at least 6 months from the time of the decision. This
means that the program office will not meet its commitment.
Another accountability mechanism that we recommended in May 2004 is for
the program to develop a plan, including explicit tasks and milestones,
for implementing all our open recommendations, and report on progress,
including reasons for delays, both to department leadership (the DHS
Secretary and Under Secretary) in periodic reports and to the Congress
in all future expenditure plans. The department has taken action to
address this recommendation, but the initial report does not disclose
enough information for a complete assessment of progress. The program
office did assign responsibility to specific individuals for preparing
the implementation plan, and it developed a report identifying the
person responsible for each recommendation and summarizing progress.
This report was provided for the first time to the DHS Deputy Secretary
on October 3, 2005, and the program office plans to forward subsequent
reports every 6 months. However, some of the report's progress
descriptions are inconsistent with our assessment. For example, the
report states that the impact of Increment 2B on workforce levels and
facilities at land ports of entry has been fully assessed. However, as
mentioned earlier, evaluation conditions were not always held constant-
-that is, fewer workstations were used to process travelers in
establishing the baseline processing times at two of the ports of entry
than were used during the pilot evaluations.
In addition, the report does not specifically describe progress against
most of our recommendations. For example, we recommended that the
program reassess plans for deploying an exit capability to ensure that
the scope of the exit pilot provides for adequate evaluation of
alternative solutions. With regard to the exit evaluation, the report
states that the program office has completed exit testing and has
forwarded the exit evaluation report to the Deputy Secretary for a
decision. However, it does not state whether the program office had
expanded the scope or time frames of the pilot.
In closing, I would emphasize that the program has met many of the
demanding requirements in law for deployment of an entry-exit system,
owing, in large part, to the hard work and dedication of the program
office and its contractors, as well as the close oversight and
direction of the House and Senate Appropriations Committees.
Nevertheless, core capabilities, such as exit, have yet to be
established and implemented, and fundamental questions about the
program's fit within the larger homeland security context and its
return on investment remain unanswered. Moreover, the program is
overdue in establishing the means to effectively manage the delivery of
future capabilities. The longer the program proceeds without these, the
greater the risk that the program will not meet its commitments.
Measuring and disclosing the extent to which these commitments are
being met are also essential to holding the department accountable, and
thus are an integral aspect of effective program management. Our
recommendations provide a comprehensive framework for addressing each
of these important areas and thus ensuring that the program as defined
is the right solution, that delivery of this solution is being managed
in the right way, and that accountability for both is in place. We look
forward to continuing to work constructively with the program to better
ensure the program's success.
Mr. Chairman, this concludes my statement. I would be happy to answer
any questions that you or members of the committee may have at this
time.
Contact and Acknowledgement:
If you should have any questions about this testimony, please contact
Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Other major
contributors to this testimony included Tonia Brown, Barbara Collier,
Deborah Davis, James Houtz, Scott Pettis, and Dan Wexler.
FOOTNOTES
[1] GAO, Information Technology: Homeland Security Needs to Improve
Entry Exit System Expenditure Planning, GAO-03-563 (Washington, D.C.:
June 9, 2003); Homeland Security: Risks Facing Key Border and
Transportation Security Program Need to Be Addressed, GAO-03-1083
(Washington, D.C.: Sept. 19, 2003); Homeland Security: First Phase of
Visitor and Immigration Status Program Operating, but Improvements
Needed, GAO-04-586 (Washington, D.C.: May 11, 2004); and Homeland
Security: Some Progress Made, but Many Challenges Remain on U.S.
Visitor and Immigrant Status Indicator Technology Program, GAO-05-202
(Washington, D.C.: Feb. 23, 2005).
[2] Secondary inspection is used for more detailed inspections that may
include checking more databases, conducting more intensive interviews,
or both.
[3] Biometric comparison is a means of identifying a person by
biological features unique to that individual.
[4] An indefinite-delivery/indefinite-quantity contract provides for an
indefinite quantity, within stated limits, of supplies or services
during a fixed period of time. The government schedules deliveries or
performance by placing orders with the contractor.
[5] The Visa Waiver Program permits foreign nationals from designated
countries to apply for admission to the United States for a maximum of
90 days as nonimmigrant visitors for business or pleasure.
[6] Foreign nationals from visa waiver countries were included as of
September 30, 2004.
[7] Entry/exit forms (Form I-94, entry/exit form, and Form I-94W,
entry/exit for foreign nationals from visa waiver countries) are used
to record a foreign national's entry into the United States. Each form
has two parts--arrival and departure--and each part contains a unique
number for the purposes of recording and matching arrival and departure
records.
[8] RF technology relies on proximity cards and card readers. RF
devices read the information contained on the card when the card is
passed near the device and can also be used to verify the identity of
the cardholder.
[9] At one port of entry, these capabilities were deployed by December
19, but were not fully operational until January 7, 2006, because of a
telephone company strike that prevented the installation of a T-1 line.
[10] In addition, Increment 2C (RF technology) will include the
creation of a new system, the Automated Identification Management
System.
[11] 8 USC. 1365a; 6 USC. 251 (transferred Immigration and
Naturalization Service functions to DHS); 8 USC. 1732(b).
[12] 8 USC 1732(b); 6 USC 251.
[13] One port of entry was not fully operational until January 7, 2006,
because of a telephone company strike that prevented the installation
of a T-1 line.
[14] OMB, Planning, Budgeting, Acquisition and Management of Capital
Assets, Circular A-11, Part 7 (Washington, D.C.: June 21, 2005).
[15] OMB, Guidelines and Discount Rates for Benefits-Cost Analysis of
Federal Programs Circular A-94 (Washington, D.C.: Oct. 29, 1992).
[16] Department of Homeland Security, Capital Planning and Investment
Control: Cost-Benefit Analysis Workbook (Washington, D.C.: May 2003).
[17] GAO, Homeland Security: Risks Facing Key Border and Transportation
Security Program Need to Be Addressed, GAO-03-1083 (Washington, D.C.:
Sept. 19, 2003).
[18] GAO, Homeland Security: Some Progress Made, but Many Challenges
Remain on U.S. Visitor and Immigrant Status Indicator Technology
Program, GAO-05-202 (Washington, D.C.: Feb. 23, 2005).
[19] For example, the cost-benefit analysis identified two categories
of quantifiable benefits, but gave no quantitative or monetary
estimates for those benefits. Instead, the analysis addressed two
categories of benefits said to be nonquantifiable (strategic alignment
benefits, such as the improvement of national security and the
promotion of legitimate trade and travel, and operational performance
benefits, such as improvement of traveler identification and validation
of traveler documentation), but it did not explain why those benefits
could not be quantified.
[20] Such cost-estimating practices are provided in a checklist for
determining the reliability of cost estimates that was developed by
Carnegie Mellon University Software Engineering Institute: A Manager's
Checklist for Validating Software and Schedule Estimates, CMU/SEI-95-
SR-004 (January 1995).
[21] As described in the background section, these alternatives are a
mobile device, a kiosk, and a validator.
[22] The other major component of an uncertainty analysis is a Monte
Carlo simulation. A Monte Carlo simulation allows all a model's
parameters to vary simultaneously according to their associated
probability distribution. The result is a set of estimated
probabilities of achieving alternative outcomes (costs, benefits,
and/or net benefits), given the uncertainty in the underlying
parameters.
[23] Specifically, they said minimal modifications to interior
workspace were required to accommodate biometric capture devices and
printers and to install electrical circuits. These officials stated
that modifications to existing officer training and interior space were
the only changes needed.
[24] The other two evaluation criteria were cost and conduciveness to
travel.
[25] Compliance rates were 23 percent for the kiosk, 36 percent for the
mobile device, and 26 percent for the validator.
[26] This initiative is to provide greater flexibility and
accountability in the way employees are paid, developed, evaluated,
afforded due process, and represented by labor organizations.
[27] Risk management is a process for identifying potential problems
before they occur so that they can be mitigated to minimize any adverse
impact.
[28] Configuration management is a process for establishing and
maintaining the integrity of the products throughout their life cycle.
[29] Capacity management is intended to ensure that systems are
properly designed and configured for efficient performance and have
sufficient processing and storage capacity for current, future, and
unpredictable workload requirements.
[30] The purpose of IV&V is to provide management with objective
insight into the program's processes and associated work products. Its
use is a recognized best practice for large and complex system
development and acquisition projects like US-VISIT.
[31] GAO, Homeland Security: Risks Facing Key Border and Transportation
Security Program Need to Be Addressed, GAO-03-1083 (Washington, D.C.:
Sept. 19, 2003).
[32] Specifically, we recommended that DHS follow guidance from
Carnegie Mellon University's Software Engineering Institute (SEI),
which has developed the Software Acquisition Capability Maturity Model
(SA-CMM®). This model explicitly defines process management controls
that are recognized hallmarks of successful organizations and that, if
implemented effectively, can greatly increase the chances of
successfully acquiring software-intensive systems. The SA-CMM uses
maturity levels to assess process maturity. See Carnegie Mellon
Software Engineering Institute, Software Acquisition Capability
Maturity Model, version 1.03 (March 2002). Since we made our
recommendation, however, SEI has begun transitioning to an integrated
model and for its improvement program, the program office is using this
integrated model: SEI, Capability Maturity Model Integrated, Systems
Engineering Integrated Product and Process Development, Continuous
Representation, version 1.1 (March 2002).
[33] The Systems Assurance Manager stated that she has only two staff,
including herself, for ensuring testing quality of the US-VISIT
composite system.
