Homeland Security
First Phase of Visitor and Immigration Status Program Operating, but Improvements Needed Gao ID: GAO-04-586 May 11, 2004The Department of Homeland Security (DHS) has established a program--the United States Visitor and Immigrant Status Indicator Technology (US-VISIT)--to collect, maintain, and share information, including biometric identifiers, on selected foreign nationals who travel to the United States. By congressional mandate, DHS is to develop and submit for approval an expenditure plan for US-VISIT that satisfies certain conditions, including being reviewed by GAO. Among other things, GAO was asked to determine whether the plan satisfied these conditions, and to provide observations on the plan and DHS's program management.
DHS's fiscal year 2004 US-VISIT expenditure plan and related documentation at least partially satisfies all conditions imposed by the Congress, including meeting the capital planning and investment control review requirements of the Office of Management and Budget (OMB). DHS developed a draft risk management plan and a process to implement and manage risks. However, DHS does not have a current life cycle cost estimate or a cost/benefit analysis for US-VISIT. The US-VISIT program merges four components into one integrated whole to carry out its mission. GAO also developed a number of observations about the expenditure plan and DHS's management of the program. These generally recognize accomplishments to date and address the need for rigorous and disciplined program practices. US-VISIT largely met its commitments for implementing an initial operating capability, known as Increment 1, in early January 2004, including the deployment of entry capability to 115 air and 14 sea ports of entry. However, DHS has not employed rigorous, disciplined management controls typically associated with successful programs, such as test management, and its plans for implementing other controls, such as independent verification and validation, may not prove effective. More specifically, testing of the initial phase of the implemented system was not well managed and was completed after the system became operational. In addition, multiple test plans were developed during testing, and only the final test plan, completed after testing, included all required content, such as describing tests to be performed. Such controls, while significant for the initial phases of US-VISIT, are even more critical for the later phases, as the size and complexity of the program will only increase. Finally, DHS's plans for future US-VISIT resource needs at the land ports of entry, such as staff and facilities, are based on questionable assumptions, making future resource needs uncertain.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-586, Homeland Security: First Phase of Visitor and Immigration Status Program Operating, but Improvements Needed
This is the accessible text file for GAO report number GAO-04-586
entitled 'Homeland Security: First Phase of Visitor and Immigration
Status Program Operating, but Improvements Needed' which was released
on May 11, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
May 2004:
HOMELAND SECURITY:
First Phase of Visitor and Immigration Status Program Operating, but
Improvements Needed:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-586]:
GAO Highlights:
Highlights of GAO-04-586, a report to the Subcommittees on Homeland
Security, Senate and House Committees on Appropriations
Why GAO Did This Study:
The Department of Homeland Security (DHS) has established a program”the
United States Visitor and Immigrant Status Indicator Technology (US-
VISIT)”to collect, maintain, and share information, including biometric
identifiers, on selected foreign nationals who travel to the United
States. By congressional mandate, DHS is to develop and submit for
approval an expenditure plan for US-VISIT that satisfies certain
conditions, including being reviewed by GAO. Among other things, GAO
was asked to determine whether the plan satisfied these conditions, and
to provide observations on the plan and DHS‘s program management.
What GAO Found:
DHS‘s fiscal year 2004 US-VISIT expenditure plan and related
documentation at least partially satisfies all conditions imposed by
the Congress, including meeting the capital planning and investment
control review requirements of the Office of Management and Budget
(OMB). For example, DHS developed a draft risk management plan and a
process to implement and manage risks. However, DHS does not have a
current life cycle cost estimate or a cost/benefit analysis for US-
VISIT. The US-VISIT program merges four components into one integrated
whole to carry out its mission (see figure).
US-VISIT Integrates People, Process, Technology, and Facilities:
[See PDF for image]
[End of figure]
GAO also developed a number of observations about the expenditure plan
and DHS‘s management of the program. These generally recognize
accomplishments to date and address the need for rigorous and
disciplined program practices. For example, US-VISIT largely met its
commitments for implementing an initial operating capability, known as
Increment 1, in early January 2004, including the deployment of entry
capability to 115 air and 14 sea ports of entry. However, DHS has not
employed rigorous, disciplined management controls typically associated
with successful programs, such as test management, and its plans for
implementing other controls, such as independent verification and
validation, may not prove effective. More specifically, testing of the
initial phase of the implemented system was not well managed and was
completed after the system became operational. In addition, multiple
test plans were developed during testing, and only the final test plan,
completed after testing, included all required content, such as
describing tests to be performed. Such controls, while significant for
the initial phases of US-VISIT, are even more critical for the later
phases, as the size and complexity of the program will only increase.
Finally, DHS‘s plans for future US-VISIT resource needs at the land
ports of entry, such as staff and facilities, are based on questionable
assumptions, making future resource needs uncertain.
What GAO Recommends:
To better ensure that the US-VISIT program is worthy of investment, GAO
is reiterating its previous recommendations aimed at establishing
effective program management capabilities. Additionally, GAO is making
several new recommendations designed to encourage stronger management
of the initial phases of the US-VISIT program, including implementing
effective test management practices and assessing the full impact of
future US-VISIT deployment on land port of entry workforce levels and
facilities. DHS agreed with all of GAO‘s recommendations and most of
its observations.
www.gao.gov/cgi-bin/getrpt?GAO-04-586.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Contents:
Letter:
Compliance with Legislative Conditions:
Status of Open Recommendations:
Observations on the Expenditure Plan:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Briefing to the Staffs of the Subcommittees on Homeland
Security, Senate and House Committees on Appropriations:
Appendix II: Comments from the Department of Homeland Security:
GAO Comments:
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Staff Acknowledgments:
Abbreviations:
ADIS: Arrival Departure Information System:
APIS: Advance Passenger Information System:
CBP: U.S. Customs and Border Protection:
CCD: Consular Consolidated Database:
CIO: Chief Information Officer:
CIS: U.S. Citizenship and Immigration Services:
CLAIMS 3: Computer Linked Application Information Management System 3:
DHS: Department of Homeland Security:
FFRDC: Federally Funded Research and Development Center:
IBIS: Interagency Border Inspection System:
ICE: U.S. Immigration and Customs Enforcement:
IDENT: Automated Biometric Identification System:
INS: Immigration and Naturalization Service:
IRB: Investment Review Board:
IV&V: independent verification and validation:
OMB: Office of Management and Budget:
POE: port of entry:
RF: radio frequency:
RFP: request for proposal:
SA-CMM: Software Acquisition Capability Maturity Model:
SAT: system acceptance test:
SEI: Software Engineering Institute:
SER: security evaluation report:
SEVIS: Student Exchange Visitor Information System:
US-VISIT: U.S. Visitor and Immigrant Status Indicator Technology:
Letter May 11, 2004:
The Honorable Thad Cochran:
Chairman:
The Honorable Robert C. Byrd:
Ranking Minority Member:
Subcommittee on Homeland Security:
Committee on Appropriations:
United States Senate:
The Honorable Harold Rogers:
Chairman:
The Honorable Martin Olav Sabo:
Ranking Minority Member:
Subcommittee on Homeland Security:
Committee on Appropriations:
House of Representatives:
Pursuant to the Department of Homeland Security Appropriations Act,
2004,[Footnote 1] the Department of Homeland Security (DHS) submitted
to the Congress in January 2004 its fiscal year 2004 expenditure plan
for the United States Visitor and Immigrant Status Indicator Technology
(US-VISIT) program. US-VISIT is a governmentwide program to collect,
maintain, and share information on foreign nationals.[Footnote 2] The
program's goals are to enhance national security, facilitate legitimate
trade and travel, contribute to the integrity of the U.S. immigration
system, and adhere to U.S. privacy laws and policies. On January 5,
2004, DHS began operating the first stage of its planned US-VISIT
operational capability, known as Increment 1, at 115 air and 14 sea
ports of entry (POE).
As required by the appropriations act, we reviewed US-VISIT's fiscal
year 2004 expenditure plan. Our objectives were to (1) determine
whether the expenditure plan satisfies the legislative conditions
specified in the act,[Footnote 3] (2) determine the status of our
US-VISIT open recommendations,[Footnote 4] and (3) provide any other
observations about the expenditure plan and DHS's management of
US-VISIT.
On March 2, 2004, we provided your offices with a written briefing
detailing the results of our review. This report summarizes and
transmits this briefing; the full briefing, including our scope and
methodology, is reprinted as appendix I. The purpose of this report is
to provide the published briefing slides to you and to officially
transmit our recommendations to the Secretary of Homeland Security.
Compliance with Legislative Conditions:
DHS satisfied or partially satisfied each of the applicable legislative
conditions specified in the act. In particular, the plan, including
related program documentation and program officials' statements,
satisfied or provided for satisfying all key aspects of (1) compliance
with the DHS enterprise architecture;[Footnote 5] (2) federal
acquisition rules, requirements, guidelines, and systems acquisition
management practices; and (3) review and approval by DHS and the Office
of Management and Budget (OMB). Additionally, the plan, including
program documentation and program officials' statements, satisfied or
provided for satisfying many, but not all, key aspects of OMB's capital
planning and investment review requirements. For example, DHS fulfilled
the OMB requirement that it justify and describe its acquisition
strategy. However, DHS does not have current life cycle costs or a
current cost/benefit analysis for US-VISIT.
Status of Open Recommendations:
DHS has implemented one, and either partially implemented or has
initiated action to implement most of the remaining recommendations
contained in our reports on the fiscal year 2002 and fiscal year 2003
expenditure plans. Each recommendation, along with its current status,
is summarized below:
* Develop a system security plan and privacy impact assessment.
The department has partially implemented this recommendation. As to the
first part of this recommendation, the program office does not have a
system security plan for US-VISIT. However, the US-VISIT Chief
Information Officer (CIO) accredited Increment 1 based upon security
certifications[Footnote 6] for each of Increment 1's component systems
and a review of each component's security-related documentation.
Second, although the program office has conducted a privacy impact
assessment for Increment 1, the assessment does not satisfy all aspects
of OMB guidance for conducting an assessment. For example, the
assessment does not discuss alternatives to the methods of information
collection, and the system documentation does not address privacy
issues.
* Develop and implement a plan for satisfying key acquisition
management controls, including acquisition planning, solicitation,
requirements management, program management, contract tracking and
oversight, evaluation, and transition to support, and implement the
controls in accordance with the Software Engineering Institute's (SEI)
guidance.[Footnote 7]
The department plans to implement this recommendation. The US-VISIT
program office has assigned responsibility for implementing the
recommended controls. However, it has not yet developed explicit plans
or time frames for defining and implementing them.
* Ensure that future expenditure plans are provided to the department's
House and Senate Appropriations Subcommittees in advance of US-VISIT
funds being obligated.
With respect to the fiscal year 2004 expenditure plan, DHS implemented
this recommendation by providing the plan to the Senate and House
subcommittees on January 27, 2004. According to the program director,
as of February 2004 no funds had been obligated to US-VISIT.
* Ensure that future expenditure plans fully disclose US-VISIT
capabilities, schedule, cost, and benefits.
The department has partially implemented this recommendation.
Specifically, the plan describes high-level capabilities, high-level
schedule estimates, categories of expenditures by increment, and
general benefits. However, the plan does not describe planned
capabilities by increment and provides only general information on how
money will be spent in each increment. Moreover, the plan does not
identify all expected benefits in tangible, measurable, and meaningful
terms, nor does it associate any benefits with increments.
* Establish and charter an executive body composed of senior-level
representatives from DHS and each US-VISIT stakeholder organization to
guide and direct the program.
The department has implemented this recommendation by establishing a
three-entity governance structure. The entities are (1) the Homeland
Security Council, (2) the DHS Investment Review Board, and (3) the US-
VISIT Federal Stakeholders Advisory Board. The purpose of the Homeland
Security Council is to ensure the coordination of all homeland
security-related activities among executive departments and agencies,
and the Investment Review Board is expected to monitor US-VISIT's
achievement of cost, schedule, and performance goals. The advisory
board is chartered to provide recommendations for overseeing program
management and performance activities, including providing advice on
the overarching US-VISIT vision; recommending changes to the vision and
strategic direction; and providing a communications link for aligning
strategic direction, priorities, and resources with stakeholder
operations.
* Ensure that human capital and financial resources are provided to
establish a fully functional and effective program office.
The department is in the process of implementing this recommendation.
DHS has determined that US-VISIT will require 115 government personnel
and has filled 41 of these, including 12 key management positions.
However, 74 positions have yet to be filled, and all filled positions
are staffed by detailees from other organizational units within the
department.
* Clarify the operational context in which US-VISIT is to operate.
The department is in the process of implementing this recommendation.
DHS released Version 1 of its enterprise architecture in October
2003,[Footnote 8] and it plans to issue Version 2 in September 2004.
* Determine whether proposed US-VISIT increments will produce mission
value commensurate with cost and risks.
The department plans to implement this recommendation. The fiscal year
2004 expenditure plan identifies high-level benefits to be delivered,
but the benefits are not associated with specific increments.
Additionally, the plan does not identify the total cost of Increment 2.
Program officials expected to finalize a cost-benefit analysis this
past March and a US-VISIT life cycle cost estimate this past April.
* Define program office positions, roles, and responsibilities.
The department is in the process of implementing this recommendation.
Program officials are currently working with the Office of Personnel
Management to define program position descriptions, including roles and
responsibilities. The program office has partially completed defining
the competencies for all 12 key management areas. These competencies
are to be used in defining the position descriptions.
* Develop and implement a human capital strategy for the program
office.
The department plans to implement this recommendation in conjunction
with DHS's ongoing workforce planning, but stated that they have yet to
develop a human capital strategy. According to these officials, DHS's
departmental workforce plan is scheduled for completion during fiscal
year 2004.
* Develop a risk management plan and report all high risks areas and
their status to the program's governing body on a regular basis.
The department has partially implemented this recommendation. The
program has completed a draft risk management plan, and is currently
defining risk management processes. The program is creating a risk
management team to operate in lieu of formal processes until these are
completed, and also maintains a risk-tracking database that is used to
manage risks.
* Define performance standards for each program increment that are
measurable and reflect the limitations imposed by relying on existing
systems.
The department is in the process of implementing this recommendation.
The program office has defined limited performance standards, but not
all standards are being defined in a way that reflects the performance
limitations of existing systems.
Observations on the Expenditure Plan:
Our observations recognize accomplishments to date and address the need
for rigorous and disciplined program management practices relating to
system testing, independent verification and validation, and system
change control. An overview of specific observations follows:
* Increment 1 commitments were largely met. An initial operating
capability for entry (including biographic and biometric data
collection) was deployed to 115 air and 14 sea ports of entry on
January 5, 2004, with additional capabilities deployed on February 11,
2004. Exit capability (including biometric capture) was deployed to one
air and one sea port of entry.
* Increment 1 testing was not managed effectively and was completed
after the system became operational. The Increment 1 system acceptance
test plan[Footnote 9] was developed largely during and after test
execution. The department developed multiple plans, and only the final
plan, which was done after testing was completed, included all required
content, such as tests to be performed and test procedures. None of the
test plan versions, including the final version, were concurred with by
the system owner or approved by the IT project manager, as required. By
not having a complete test plan before testing began, the US-VISIT
program office unnecessarily increased the risk that the testing
performed would not adequately address Increment 1 requirements and
failed to have adequate assurance that the system was being fully
tested. Further, by not fully testing Increment 1 before the system
became operational, the program office assumed the risk of introducing
errors into the deployed system. In fact, post-deployment problems
surfaced with the Student and Exchange Visitor Information System
(SEVIS) interface as a result of this approach, and manual work-arounds
had to be implemented.
* The independent verification and validation contractor's roles may be
in conflict.[Footnote 10] The US-VISIT program plans to use its
contractor to review some of the processes and products that the
contractor may be responsible for defining or executing. Depending on
the products and processes in question, this approach potentially
impedes the contractor's independence, and thus its effectiveness.
* A program-level change control board has not been
established.[Footnote 11] Changes related to Increment 1 were
controlled primarily through daily coordination meetings (i.e., oral
discussions) among representatives from Increment 1 component systems
teams and program officials, and the various boards already in place
for the component systems. Without a structured and disciplined
approach to change control, program officials do not have adequate
assurance that changes made to the component systems for non-US-VISIT
purposes do not interfere with US-VISIT functionality.
* The fiscal year 2004 expenditure plan does not disclose management
reserve funding.[Footnote 12] Program officials, including the program
director, stated that reserve funding is embedded within the
expenditure plan's various areas of proposed spending. However, the
plan does not specifically disclose these embedded reserve amounts. By
not creating, earmarking, and disclosing a specific management reserve
fund in the plan, DHS is limiting its flexibility in addressing
unexpected problems that could arise in the program's various areas of
proposed spending, and it is limiting the ability of the Congress to
exercise effective oversight of this funding.
* Plans for future US-VISIT increments do not call for additional staff
or facilities at land ports of entry. However, these plans are based on
various assumptions that potential policy changes could invalidate.
These changes could significantly increase the number of foreign
nationals who would require processing through US-VISIT. Additionally,
the Data Management Improvement Act Task Force's 2003 Second Annual
Report to Congress[Footnote 13] has noted that existing land port of
entry facilities do not adequately support even the current entry and
exit processes. Thus, future US-VISIT staffing and facility needs are
uncertain.
Conclusions:
The fiscal year 2004 US-VISIT expenditure plan (with related program
office documentation and representations) at least partially satisfies
the legislative conditions imposed by the Congress. Further, steps are
planned, under way, or completed to address most of our open
recommendations. However, overall progress on all of our
recommendations has been slow, and considerable work remains to fully
address them. The majority of these recommendations are aimed at
correcting fundamental limitations in the program office's ability to
manage US-VISIT in a way that reasonably ensures the delivery of
mission value commensurate with costs and provides for the delivery of
promised capabilities on time and within budget. Given this background,
it is important for DHS to implement the recommendations quickly and
completely through active planning and continuous monitoring and
reporting. Until this occurs, the program will continue to be at high
risk of not meeting expectations.
To the US-VISIT program office's credit, the first phase of the program
has been deployed and is operating, and the commitments that DHS made
regarding this initial operating capability were largely met. However,
this was not accomplished in a manner that warrants repeating. In
particular, the program office did not employ the kind of rigorous and
disciplined management controls that are typically associated with
successful programs, such as effective test management and
configuration management practices. Moreover, the second phase of US-
VISIT is already under way, and these controls are still not
established. These controls, while significant for the initial phases
of US-VISIT, are even more critical for the later phases, because the
size and complexity of the program will only increase, and the later
that problems are found, the harder and more costly they are to fix.
Also important at this juncture in the program's life are the still
open questions surrounding whether the initial phases of US-VISIT will
return value to the nation commensurate with their costs. Such
questions warrant answers sooner rather than later, because of the
program's size, complexity, cost, and mission significance. It is
imperative that DHS move swiftly to address the US-VISIT program
management weaknesses that we previously identified, by implementing
our remaining open recommendations. It is equally essential that the
department quickly corrects the additional weaknesses that we have
identified. Doing less will only increase the risk associated with US-
VISIT.
Recommendations for Executive Action:
To better ensure that the US-VISIT program is worthy of investment and
is managed effectively, we are reiterating our prior recommendations,
and we further recommend that the Secretary of Homeland Security direct
the Under Secretary for Border and Transportation Security to ensure
that the US-VISIT program director takes the following actions:
* Develop and approve complete test plans before testing begins. These
plans, at a minimum, should (1) specify the test environment, including
test equipment, software, material, and necessary training;
(2) describe each test to be performed, including test controls,
inputs, and expected outputs; (3) define the test procedures to be
followed in conducting the tests; and (4) provide traceability between
test cases and the requirements to be verified by the testing.
* Establish processes for ensuring the independence of the IV&V
contractor.
* Implement effective configuration management practices, including
establishing a US-VISIT change control board to manage and oversee
system changes.
* Identify and disclose to the Appropriations Committees management
reserve funding embedded in the fiscal year 2004 expenditure plan.
* Ensure that all future US-VISIT expenditure plans identify and
disclose management reserve funding.
* Assess the full impact of a key future US-VISIT increment on land
port of entry workforce levels and facilities, including performing
appropriate modeling exercises.
To ensure that our recommendations addressing fundamental program
management weaknesses are addressed quickly and completely, we further
recommend that the Secretary direct the Under Secretary to have the
program director develop a plan, including explicit tasks and
milestones, for implementing all of our open recommendations, including
those provided in this report. We further recommend that this plan
provide for periodic reporting to the Secretary and Under Secretary on
progress in implementing this plan. Lastly, we recommend that the
Secretary report this progress, including reasons for delays, in all
future US-VISIT expenditure plans.
Agency Comments and Our Evaluation:
In written comments on a draft of this report signed by the US-VISIT
Director (reprinted in app. II, along with our responses), DHS agreed
with our recommendations and most of our observations. It also stated
that it appreciated the guidance that the report provided and described
actions that it is taking or plans to take in response to our
recommendations.
However, DHS stated that it did not fully agree with all of our
findings, specifically offering comments on our characterization of the
status of one open recommendation and two observations. First, it did
not agree with our position that it had not developed a security plan
and completed a privacy impact assessment. According to DHS, it has
completed both. We acknowledge DHS's activity on both of these issues,
but disagree that completion of an adequate security plan and privacy
impact assessment has occurred. As we state in the report, the
department's security plan for US-VISIT, titled Security and Privacy:
Requirements & Guidelines Version 1.0, is a draft document, and it does
not include information consistent with relevant guidance for a
security plan, such as a risk assessment methodology and specific
controls for meeting security requirements.[Footnote 14] Moreover, much
of the document discusses guidelines for developing a security plan,
rather than specific contents of a plan. Also, as we state in the
report, the Privacy Impact Assessment was published but is not complete
because it does not satisfy important parts of OMB guidance governing
the content of these assessments, such as discussing alternatives to
the designed methods of information collection and handling.
Second, DHS stated that it did not fully agree with our observation
that the Increment 1 system test plan was developed largely during and
after testing, citing several steps that it took as part of Increment 1
requirements definition, test preparation, and test execution. However,
none of the steps cited address our observations that DHS did not have
a system acceptance test plan developed, approved, and available in
time to use as the basis for conducting system acceptance testing and
that only the version of the test plan modified on January 16, 2004
(after testing was completed) contained all of the required test plan
content. Moreover, DHS's comments acknowledge that the four versions of
its Increment 1 test plan were developed during the course of test
execution, and that the test schedule did not permit sufficient time
for all stakeholders to review, and thus approve, the plans.
Third, DHS commented on the roles and responsibilities of its various
support contractors, and stated that we cited the wrong operative
documentation governing the role of its independent verification and
validation contractor. While we do not question the information
provided in DHS's comments concerning contractor roles, we would add
that its comments omitted certain roles and responsibilities contained
in the statement of work for one of its contractors. This omitted
information is important because it is the basis for our observation
that the program office planned to task the same contractor that was
responsible for program management activities with performing
independent verification and validation activities. Under these
circumstances, the contractor could not be independent. In addition, we
disagree with DHS's comment that we cited the wrong operative
documentation, and note that the document DHS said we should have used
relates to a different support contractor than the one tasked with both
performing program activities and performing independent verification
and validation activities.
The department also provided additional technical comments, which we
have incorporated as appropriate into the report.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of other Senate and House committees and subcommittees
that have authorization and oversight responsibilities for homeland
security. We are also sending copies to the Secretary of State and the
Director of OMB. Copies of this report will also be available at no
charge on our Web site at [Hyperlink, http://www.gao.gov].
Should you or your offices have any questions on matters discussed in
this report, please contact me at (202) 512-3439 or at [Hyperlink,
hiter@gao.gov]. Another contact and key contributors to this report are
listed in appendix III.
Signed by:
Randolph C. Hite,
Director, Information Technology Architecture and Systems Issues:
[End of section]
Appendixes:
Appendix I: Briefing to the Staffs of the Subcommittees on Homeland
Security, Senate and House Committees on Appropriations:
[See PDF for image]
[End of figure]
[End of section]
Appendix II: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security
Washington, DC 20528:
27 April 2004:
Randolph C. Hite:
Director, Information Technology Architecture And Systems Issues:
U.S. General Accounting Office
Washington, DC 20548:
Dear Mr. Hite:
Thank you for the opportunity to review the draft report, Homeland
Security: First Phase of Visitor and Immigration Status Program
Operating, but Improvements Needed (GAO-04-586). The Department of
Homeland Security largely agrees with GAO on the majority of the
findings. However, there are some findings with which we cannot agree,
and we have provided appropriate comments in the enclosure. You will
also note that we have concurred with, and addressed, the new
recommendations generated by this review.
As you know, US-VISIT represents the greatest advancement in border
technology in three decades. The Department of Homeland Security
established US-VISIT to achieve the following goals:
* Enhance the safety of our citizens and visitors;
* Facilitate legitimate travel and trade;
* Ensure the integrity of our immigration system; and:
* Protect the privacy of travelers to the United States.
The first increment of US-VISIT was deployed on time and within budget,
and has exceeded the mandate established by Congress as it includes
biometrics ahead of schedule. On January 5, 2004, US-VISIT entry
procedures were operational at 115 airports and 14 seaports and by the
end of this year US-VISIT will be in operation at our 50 busiest land
ports of entry. In addition, we began pilot testing biometric exit
procedures at one airport and one seaport and will be expanding to
additional pilot locations later this summer.
As of April 20, 2004, more than three million foreign visitors have
been processed through the US-VISIT entry procedures - without any
increase in wait times. On average, US-VISIT procedures take less than
15 seconds during the inspection process.
US-VISIT has already matched over 300 persons against criminal
databases and prevented more than 100 known or suspected criminals from
entering the country. Over 200 were matched while applying for a visa
at a State Department post overseas.
Through the US-VISIT biometric process, the Departments of Homeland
Security and State have identified many individuals who are the
subjects of lookout records. These included rapists, drug traffickers,
convicted criminals, and those who have committed immigration offenses
or visa fraud.
US-VISIT is critical to our national security as well as our economic
security, and its implementation is already making a significant
contribution to the efforts of the Department to provide a safer and
more secure America. We recognize that we have a long way still to go.
We will build upon the initial framework and solid foundation to ensure
that we continue to meet our goals of enhancing the security of our
citizens and visitors while facilitating travel for the millions of
visitors we welcome each year.
For all the successes of US-VISIT, the Department realizes, and your
report supports the fact, that we need to improve the management of the
program. We have already established a great deal of the foundation for
meeting future challenges and will continue to improve the necessary
disciplines for excellent program management. We realize that much
needs to be done, and we appreciate the guidance that reports such as
this provide.
Sincerely,
Signed by:
James A. Williams:
Enclosure:
Enclosure: Proposed Changes, Clarifications, and Responses to
Recommendations for Draft Report GAO-04-586:
Letter to Sen. Cochran and Rep. Rogers:
Page 3, Status of Open Recommendations:
1. Develop a system security plan and privacy impact assessment.
The US-VISIT program does have an existing security plan. In addition,
as GAO notes in the explanation of this action item, US-VISIT did
complete a Privacy Impact Assessment for Increment 1. As US-VISIT
proceeds with future increments, these documents will be updated to
reflect changes in the program.
Pages. 3 - 6, Status of Open Recommendations 2 through 12:
With respect to recommendations 2 through 12, we recognize GAO'
acknowledges that US-VISIT has implemented, partially implemented, or
plans to implement them. While we could offer minor clarifications to
the status of these issues, we agree in general with the
recommendations and therefore provide no further comment.
Page 6, Observations on the Expenditure Plan:
A management reserve fund has been identified in the amount of $33
million in fiscal year 2004. However, this was not specifically
detailed in the FY 2004 Expenditure Plan. While we concur with the
concept for such a reserve, our concern lies with any potential
restrictions and/or new approval processes that may accompany such a
set-aside.
Page 10 - Recommendations for Executive Action:
1. Develop and approve complete test plans before testing begins. These
plans, at a minimum, should (1) specify the test environment, including
test equipment, software, material, and necessary training; (2)
describe each test to be performed, including test controls, inputs,
and expected outputs; (3) define the test procedures to be followed in
conducting the tests; and (4) provide traceability between test cases
and the requirements to be verified by the testing.
We concur. Complete test plans will be developed and approved before
future testing begins. Corrective action completed.
2. Establish processes for ensuring the independence of the IV & V
contractor.
We concur. US-VISIT is aggressively researching IV&V resources that
will be utilized to independently evaluate any future development work
to be performed by the US-VISIT prime integrator and future increments.
Corrective action completed.
3. Implement effective configuration management practices, including
establishing a US-VISIT change control board to manage and oversee
system changes.
We concur. Effective configuration management practices for US-VISIT
will be implemented. Corrective action in progress.
4. Identify and disclose management reserve funding embedded in the
fiscal year 2004 expenditure plan to the Appropriations Committees.
We concur. The FY 2004 Expenditure Plan has been revised to identify a
$33 million management reserve, separate from incremental spending
Corrective action completed.
5. Ensure that all future US-VISIT expenditure plans identify and
disclose management reserve funding.
We concur. All future expenditure plans will identify and disclose
management reserve funding. Corrective action completed.
6. Assess the full impact of a key future US-VISIT increment [2B] on
land port of entry workforce levels and facilities, including
performing appropriate modeling exercises.
We concur. A full reassessment of the impact of Increment 2B will be
performed with the new prime contractor, pending award of the contract
in May 2004. Corrective action in progress.
Slides:
Slide 58:
The listing of membership for the US-VISIT Advisory Board needs
correction. The "Associate Director of Operations, Customs and
Immigration Services" needs to be changed to "...Citizenship and
Immigration Services." In addition, the "Assistant Commissioner, Office
of Field Operations, Customs and Border Protection" needs to be added.
Slide 70. Observation 2: The system test (SAT) plan was developed
largely during and a ter testing (and Recommendations. Slide 103).
US-VISIT does not fully concur with the observation that the systems
test plan was developed largely during and after testing. A
comprehensive test strategy outlining the work pattern to be following
for independent end-to-end testing was developed in a structured and
disciplined fashion and was approved by the US-VISIT Chief Information
Officer in May 2003. This document outlined the environment and
interfaces to be tested, as well as assumptions and constraints.
Coordination between the US-VISIT IV&V contractor and the component
development teams (CPB/ICE/TSA/CIS) took place from July through
September 2003 to ensure that Use Cases were documented from the US-
VISIT Functional Requirements Document and that technical requirements
regarding the environment were resolved prior to the commencement
of testing in September 2003. These Use Cases were the basis for the
development of the Draft Test Plan that was delivered on September 19,
2003. Furthermore, since US-VISIT Increment 1 leveraged established
systems, test cases were available in previous test plans and were
established in the test cases repository of Test Director (the software
toolset/application utilized by the independent testers). Additional
versions of the Test Plan were developed throughout the Systems
Assurance Testing period due to corrections or inclusion of clarifying
data provided by the component development teams. Throughout this
iterative process the overarching Use Cases were never modified. US-
VISIT does agree with GAO's observation that the compressed timeline
did not allow ample time for all US-VISIT stakeholders to review the
draft Test Plan, although daily status reports were provided as a basis
for validating that all Use Cases were fully tested, as documented in
the Test Analysis Report.
Slide 90-91:
The US-VISIT program office was established in July 2003 and acquired
two contractors, PEC (Program Office Support) and the MITRE Corporation
(FFRDC), to initially help with the implementation of the program
office (PO), acquisition of a prime contractor, and establishment of
SA-CMM compliant processes and procedures to guide and manage the US-
VISIT program acquisition.
During the initiation phase, PEC is responsible for helping the PO with
the establishment of plans, processes, and procedures for program
planning and program/project management and control. Once these
processes are established, PEC will assist in executing these
processes, under PO direction. MITRE is responsible for assisting with
strategic planning for the program and PO. MITRE is also responsible
for assisting the PO in the acquisition and source selection of the
prime contractor, and for working with PEC to ensure that the program
planning, management, and control processes being developed are SA-CMM
compliant and that an effective process improvement program is being
put in place.
As the program moves to the execution phase, PEC will continue to
provide program management planning and process execution support.
MITRE will focus on providing oversight of the prime contractor and PO
support contractor to ensure that:
SA-CMM compliant processes are being followed:
The plans, designs, and products being developed by the prime
contractor address the program requirements, conform to the DHS
enterprise architecture, and are cost-effective for the government:
The program risks are being identified and managed:
The peformance of the program (US-VISIT mission goals and program
management controls) is being measured and validated:
Slide 90. Observation S: Independent verification and validation (IV&V)
contractor's roles may be conflicting.
The US-VISIT program office endorses the concept of Independent
Validation and Verification (IV&V) as a mechanism to provide an
independent review of system processes and work products. Furthermore,
US-VISIT recognizes the need for the IV&V to be independent of the
processes and products that are being developed. US-VISIT utilized an
existing IV&V vehicle for Increment 1 that was available through the
Bureau of Immigration and Customs Enforcement (ICE) and identified by
DHS as a center of excellence. Unit testing was performed by component
system owners and their respective application development contractors
under distinctly separate task orders, while end-to-end, security, and
performance testing was completed by SAIC. The technology IV&V work
completed under this contract vehicle was provided by SAIC under Task
Order 02-SM/I-IRM-417, dated September 25, 2003. GAO incorrectly cited
the July 18, 2003, statement of work for other general program and
project management support. The scope of the September 25, 2003, task
order specifically addressed the provision for technical governance,
systems assurance standards and direction, as well as independent end-
to-end testing.
Slide 92, Observation 6: Program-level change control board has not
been established (and Recommendations, Slide 103).
The US-VISIT program office endorses a structured and disciplined
approach to change control and is actively building a process to
establish and maintain the integrity of work products with its
stakeholders. While the principles of software configuration management
were followed based on the ICE Enterprise Systems Assurance Plan (i.e.,
the establishment of a Functional Baseline [FB] and Allocated Baseline
[AB], versioned naming conventions for software, and recording all
documentation to an Enterprise Library) a formal Change Control Board
was not established prior to the implementation of Increment 1. It is
the intention of the US-VISIT Program Office to institute a CM process
that will define policy for any modifications or System Change Requests
for any future releases of software.
The following are GAO's comments on the Department of Homeland
Security's letter dated April 27, 2004.
GAO Comments:
1. We do not agree that the US-VISIT program has a security plan. In
response to our request for the US-VISIT security plan, DHS provided a
draft document entitled Security and Privacy: Requirements & Guidelines
Version 1.0. However, as we state in the report, this document does not
include information consistent with relevant guidance for a security
plan.[Footnote 15] For example, this guidance states that a system
security plan should (1) provide an overview of the system security
requirements, (2) include a description of the controls in place or
planned for meeting the requirements, (3) delineate roles and
responsibilities of all individuals who have access to the system,
(4) describe the risk assessment methodology to be used, and
(5) address security awareness and training. The document provided by
DHS addressed two of these requirements--security requirements and
training and awareness. As we state in the report, the document does
not (1) describe specific controls to satisfy the security
requirements, (2) describe the risk assessment methodology, and
(3) identify roles and responsibilities of individuals with system
access. Further, much of the document discusses guidelines for
developing a security plan, rather than providing the specific content
expected of a plan.
2. Although DHS has completed a Privacy Impact Assessment for Increment
1, the assessment is not consistent with the Office of Management and
Budget guidance.[Footnote 16] This guidance says that a Privacy Impact
Assessment should, among other things, (1) identify appropriate
measures for mitigating identified risks, (2) discuss the rationale for
the final design or business process choice, (3) discuss alternatives
to the designed information collection and handling, and (4) address
whether privacy is provided for in system development and
documentation. While the Privacy Impact Assessment for US-VISIT
Increment 1 discusses mitigation strategies for identified risks and
briefly discusses the rationale for design choices, it does not discuss
alternatives to the designed information collection and handling.
Further, Increment 1 system documentation does not address privacy.
3. DHS's comments did not include a copy of its revised fiscal year
2004 expenditure plan because, according to an agency official, OMB has
not yet approved the revised plan for release, and thus we cannot
substantiate its comments concerning either the amount or the
disclosure of management reserve funding. Further, we are not aware of
any unduly burdensome restrictions and/or approval processes for using
such a reserve. We have modified our report to reflect DHS's statement
that it supports establishing a management reserve and the status of
revisions to its expenditure plan.
4. We have modified the report as appropriate to reflect these comments
and subsequent oral comments concerning the membership of the US-VISIT
Advisory Board.
5. We do not believe that DHS's comments provide any evidence to
counter our observation that the system acceptance test plan was
developed largely during and after testing. In general, these comments
concern the Increment 1 test strategy, test contractor and component
system development team coordination, Increment 1 use cases, and pre-
existing component system test cases, none of which are related to our
point about the completeness of the four versions of the test plan.
More specifically, our observation does not address whether or not an
Increment 1 test strategy was developed and approved, although we would
note that the version of the strategy that the program office provided
to us was incomplete, was undated, and did not indicate any level of
approval. Further, our observation does not address whether some
unspecified level of coordination occurred between the test contractor
and the component system development teams; it does not concern the
development, modification, and use of Increment 1 "overarching" use
cases, although we acknowledge that such use cases are important in
developing test cases; and it does not address the pre-existence of
component system test cases and their residence in a test case
repository, although we note that when we previously asked for
additional information on this repository, none was provided.
Rather, our observation concerns whether a sufficiently defined US-
VISIT Increment 1 system acceptance test plan was developed, approved,
and available in time to be used as the basis for conducting system
acceptance testing. As we state in the report, to be sufficient such a
plan should, among other things, define the full complement of test
cases, including inputs and outputs, and the procedures for executing
these test cases. Moreover, these test cases should be traceable to
system requirements. However, as we state in our report, this content
was added to the Increment 1 test plan during the course of testing,
and only the version of the test plan modified January 16, 2004,
contained all of this content. Moreover, DHS's comments recognize that
these test plan versions were developed during the course of test
execution and that the test schedule did not permit sufficient time for
all stakeholders to review the versions.
6. We do not disagree with DHS's comments describing the roles and
responsibilities of its program office support contractor and its
Federally Funded Research and Development Center (FFRDC) contractor.
However, DHS's description of the FFRDC contractor's roles and
responsibilities do not cover all of the taskings envisioned for this
contractor. Specifically, DHS's comments state that the FFRDC
contractor is to execute such program and project management activities
as strategic planning, contractor source selection, acquisition
management, risk management, and performance management. These roles
and responsibilities are consistent with the FFRDC contractor's
statement of work that was provided by DHS. However, DHS's comments
omit other roles and responsibilities specified in this statement of
work. In particular, the comments do not cite that this contractor is
also to conduct audits and evaluations in the form of independent
verification and validation activities. It is this audit and evaluation
role, particularly the independence element, which is the basis for our
concern and observation. As we note above and state in the report, US-
VISIT program plans and the contractor's statement of work provide for
using the same contractor both to perform program and project
management activities, including creation of related products, and to
assess those activities and products. Under these circumstances, the
contractor could not be sufficiently independent to effectively
discharge the audit and evaluation tasks.
7. We do not agree with DHS's comment that we cited the wrong operative
documentation pertaining to US-VISIT independent verification and
validation plans. As discussed in our comment No. 6, the statement of
work that we cite in the report relates to DHS plans to use the FFRDC
contractor to both perform program and project management activities
and develop related products and to audit and evaluate those activities
and products. The testing contractor and testing activities discussed
in DHS comments are separate and distinct from our observation about
DHS plans for using the FFRDC contractor. Accordingly, our report does
not make any observation regarding the independence of the testing
contractor.
8. We agree that US-VISIT lacks a change control board and support
DHS's stated commitment to establish a structured and disciplined
change control process that would include such a board.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Deborah Davis, (202) 512-6261:
Staff Acknowledgments:
In addition to the individual named above, Barbara Collier, Gary
Delaney, Neil Doherty, Tamra Goldstein, David Hinchman, Thomas
Keightley, John Mortin, Debra Picozzi, Karl Seifert, and Jessica
Waselkow made key contributions to this report.
(310277):
FOOTNOTES
[1] Pub. L. 108-90 (Oct. 1, 2003).
[2] The US-VISIT program has a large number of government stakeholders,
including the Departments of State, Transportation, Commerce, Justice,
and the General Services Administration. State will play a significant
role in creating a coordinated and interlocking network of border
security by gathering biographic and biometric data during the
application process for visas, grants of visa status, and the issuance
of travel documentation. DHS inspectors will use this information at
ports of entry to verify the identity of the foreign national.
[3] The legislative conditions are that the plan (1) meet the capital
planning and investment control review requirements established by the
Office of Management and Budget (OMB), including those in OMB Circular
A-11, part 3 (capital investment and control requirements are now found
in part 7, rather than part 3); (2) comply with DHS's enterprise
architecture; (3) comply with the acquisition rules, requirements,
guidelines, and systems acquisition management practices of the federal
government; (4) be reviewed and approved by DHS and OMB; and (5) be
reviewed by GAO.
[4] Our previous recommendations regarding US-VISIT's expenditure plans
were published in U.S. General Accounting Office, Information
Technology: Homeland Security Needs to Improve Entry Exit System
Expenditure Planning, GAO-03-563 (Washington, D.C.: June 9, 2003) and
Homeland Security: Risks Facing Key Border and Transportation Security
Program Need to Be Addressed, GAO-03-1083 (Washington, D.C.: Sept. 19,
2003).
[5] Enterprise architectures are blueprints, or models, simplifying the
complexity of how agencies operate today, how they want to operate in
the future, and how they will get there.
[6] Accreditation is the authorization and approval granted to a system
to process sensitive data in an operational environment; this is made
on the basis of a compliance certification by designated technical
personnel of the extent to which design and implementation of the
system meet defined technical requirements for achieving data security.
Certification is the evaluation of the extent to which a system meets a
set of security requirements.
[7] Carnegie Mellon University Software Engineering Institute, Software
Acquisition Capability Maturity Model", Version 1.03 (March 2002)
defines acquisition process management controls for planning, managing,
and controlling software-intensive system acquisitions.
[8] Department of Homeland Security Enterprise Architecture Compendium
Version 1.0 and Transitional Strategy.
[9] The purpose of system acceptance testing is to verify that the
complete system satisfies functional, performance, and security
requirements and is acceptable to end users.
[10] The purpose of independent verification and validation (IV&V) is
to provide an independent review of system processes and products. To
be effective, the IV&V function must be performed by an entity that is
independent of the processes and products that are being reviewed.
[11] The purpose of configuration management is to establish and
maintain the integrity of work products (e.g., hardware, software, and
documentation). A key ingredient to effectively controlling
configuration change is the functioning of a change control board.
[12] The creation and use of a management reserve fund to earmark
resources for addressing the many uncertainties that are inherent in
large-scale systems acquisition programs is an established practice and
a prudent management approach.
[13] Data Management Improvement Act Task Force, Second Annual Report
to Congress (Washington, D.C., December 2003).
[14] Office of Management and Budget Circular Number A-130, Revised
(Transmittal Memorandum No. 4), Appendix III, "Security of Federal
Automated Information Resources" (Nov. 28, 2000) and National Institute
of Standards and Technology, Guide for Developing Security Plans for
Information Systems, NIST Special Publication 800-18 (December 1998).
[15] Office of Management and Budget Circular Number A-130, Revised
(Transmittal Memorandum No. 4), Appendix III, "Security of Federal
Automated Information Resources" (Nov. 28, 2000) and National Institute
of Standards and Technology, Guide for Developing Security Plans for
Information Systems, NIST Special Publication 800-18 (December 1998).
[16] OMB Guidance for Implementing the Privacy Provisions of the E-
Government Act of 2002, OMB M-03-22 (Sept. 26, 2003).
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
