Combating Nuclear Smuggling
Inadequate Communication and Oversight Hampered DHS Efforts to Develop an Advanced Radiography System to Detect Nuclear Materials
Gao ID: GAO-10-1041T September 15, 2010
The Department of Homeland Security's (DHS) Domestic Nuclear Detection Office (DNDO) is charged with developing and acquiring equipment to detect nuclear and radiological materials to support federal efforts to combat nuclear smuggling. Also within DHS, Customs and Border Protection (CBP) has the lead for operating systems to detect nuclear and radiological materials entering the country at U.S. ports of entry. In 2005, DNDO began working on the cargo advanced automated radiography system (CAARS) intending that it be used by CBP to detect certain nuclear materials in vehicles and containers at U.S. ports of entry. However, in 2007 DNDO decided to cancel the acquisition phase of the program and convert it to a research and development program. GAO was asked to examine events that led to DNDO's decision to cancel the acquisition phase of the program and provide lessons learned from DNDO's experience. This statement is based on prior GAO reports from March 2006 through July 2010 and ongoing work reviewing DHS efforts to develop radiography technology. For ongoing work, GAO reviewed CAARS planning documents and interviewed DHS, DNDO, and CBP officials. GAO provided a draft of the information in this testimony to DHS and component agencies, which provided technical comments and which were incorporated as appropriate.
From the start of the CAARS program in 2005 until DNDO cancelled the acquisition phase of the program in December 2007, DNDO pursued the acquisition and deployment of CAARS machines without fully understanding that they would not fit within existing primary inspection lanes at CBP ports of entry. This occurred because during the first year or more of the program DNDO and CBP had few discussions about operating requirements at ports of entry. When CBP and DNDO officials met, shortly before DNDO's decision to cancel the acquisition phase of the program, CBP officials said they made it clear to DNDO that they did not want the CAARS machines because they would not fit in primary inspections lanes and would slow down the flow of commerce through these lanes and cause significant delays. Also, the CAARS program was among numerous DHS acquisition programs about which GAO reported in 2008 that appropriate oversight was lacking. Further, the development of the CAARS algorithms (software)--a key part of the machine needed to identify shielded nuclear materials automatically--did not mature at a rapid enough pace to warrant acquisition and deployment. Also, the description of the progress of the CAARS program used to support funding requests in DNDO's budget justifications was misleading because it did not reflect the actual status of the program. For example, the fiscal years 2010 and 2011 DHS budget justifications both cited that an ongoing CAARS testing campaign would lead to a cost-benefit analysis. However, DNDO officials told GAO that when they cancelled the acquisition part of the program in 2007, they also decided not to conduct any associated cost benefit analysis. During recent discussions with DNDO officials, they agreed that the language in the budget justifications lacked clarity, and they have no plans to prepare a cost benefit analysis. Based on GAO's review of the CAARS program and its prior reports on DHS development and acquisition efforts, GAO identified lessons learned for DHS to consider in its continuing efforts to develop the next generation of radiography imaging technology. For example, GAO previously reported that agencies can enhance coordination by agreeing on roles and responsibilities. In this regard, a draft memorandum of agreement among DHS agencies that intends to clarify roles and responsibilities in developing technologies and help ensure effective coordination has not been finalized. Completing this memorandum could give DHS reasonable assurance that problems associated with the CAARS program do not recur. In discussions with senior officials from DHS, DNDO, CBP and S&T, they all agreed with the need for the memorandum and said that they intend to work toward finalizing the draft memorandum of agreement. Other lessons GAO identified include (1) engage in a robust departmental oversight review process (2) separate the research and development functions from acquisition functions (3) determine the technology readiness levels before moving forward to acquisition, and (4) rigorously test devices using actual agency operational tactics before making decisions on acquisition.
GAO-10-1041T, Combating Nuclear Smuggling: Inadequate Communication and Oversight Hampered DHS Efforts to Develop an Advanced Radiography System to Detect Nuclear Materials
This is the accessible text file for GAO report number GAO-10-1041T
entitled 'Combating Nuclear Smuggling: Inadequate Communication and
Oversight Hampered DHS Efforts to Develop an Advanced Radiography
System to Detect Nuclear Materials' which was released on September
15, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Statement for the Record:
To the Committee on Homeland Security and Governmental Affairs,
U.S. Senate:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 10:00 a.m. EDT:
Wednesday, September 15, 2010:
Combating Nuclear Smuggling:
Inadequate Communication and Oversight Hampered DHS Efforts to Develop
an Advanced Radiography System to Detect Nuclear Materials:
Statement for the Record by Gene Aloise, Director, Natural Resources
and Environment:
and:
Stephen L. Caldwell, Director, Homeland Security and Justice:
GAO-10-1041T:
GAO Highlights:
Highlights of GAO-10-1041T, a statement for the record to the
Committee on Homeland Security and Governmental Affairs, U.S. Senate.
Why GAO Did This Study:
The Department of Homeland Security‘s (DHS) Domestic Nuclear Detection
Office (DNDO) is charged with developing and acquiring equipment to
detect nuclear and radiological materials to support federal efforts
to combat nuclear smuggling. Also within DHS, Customs and Border
Protection (CBP) has the lead for operating systems to detect nuclear
and radiological materials entering the country at U.S. ports of
entry. In 2005, DNDO began working on the cargo advanced automated
radiography system (CAARS) intending that it be used by CBP to detect
certain nuclear materials in vehicles and containers at U.S. ports of
entry. However, in 2007 DNDO decided to cancel the acquisition phase
of the program and convert it to a research and development program.
GAO was asked to examine events that led to DNDO‘s decision to cancel
the acquisition phase of the program and provide lessons learned from
DNDO‘s experience. This statement is based on prior GAO reports from
March 2006 through July 2010 and ongoing work reviewing DHS efforts to
develop radiography technology. For ongoing work, GAO reviewed CAARS
planning documents and interviewed DHS, DNDO, and CBP officials. GAO
provided a draft of the information in this testimony to DHS and
component agencies, which provided technical comments and which were
incorporated as appropriate.
What GAO Found:
From the start of the CAARS program in 2005 until DNDO canceled the
acquisition phase of the program in December 2007, DNDO pursued the
acquisition and deployment of CAARS machines without fully
understanding that they would not fit within existing primary
inspection lanes at CBP ports of entry. This occurred because during
the first year or more of the program DNDO and CBP had few discussions
about operating requirements at ports of entry. When CBP and DNDO
officials met, shortly before DNDO‘s decision to cancel the
acquisition phase of the program, CBP officials said they made it
clear to DNDO that they did not want the CAARS machines because they
would not fit in primary inspections lanes and would slow down the
flow of commerce through these lanes and cause significant delays.
Also, the CAARS program was among numerous DHS acquisition programs
about which GAO reported in 2008 that appropriate oversight was
lacking. Further, the development of the CAARS algorithms (software)”-
a key part of the machine needed to identify shielded nuclear
materials automatically”-did not mature at a rapid enough pace to
warrant acquisition and deployment. Also, the description of the
progress of the CAARS program used to support funding requests in
DNDO‘s budget justifications was misleading because it did not reflect
the actual status of the program. For example, the fiscal years 2010
and 2011 DHS budget justifications both cited that an ongoing CAARS
testing campaign would lead to a cost-benefit analysis. However, DNDO
officials told GAO that when they canceled the acquisition part of the
program in 2007, they also decided not to conduct any associated cost
benefit analysis. During recent discussions with DNDO officials, they
agreed that the language in the budget justifications lacked clarity,
and they have no plans to prepare a cost benefit analysis.
Based on GAO‘s review of the CAARS program and its prior reports on
DHS development and acquisition efforts, GAO identified lessons
learned for DHS to consider in its continuing efforts to develop the
next generation of radiography imaging technology. For example, GAO
previously reported that agencies can enhance coordination by agreeing
on roles and responsibilities. In this regard, a draft memorandum of
agreement among DHS agencies that intends to clarify roles and
responsibilities in developing technologies and help ensure effective
coordination has not been finalized. Completing this memorandum could
give DHS reasonable assurance that problems associated with the CAARS
program do not recur. In discussions with senior officials from DHS,
DNDO, CBP and S&T, they all agreed with the need for the memorandum
and said that they intend to work toward finalizing the draft
memorandum of agreement. Other lessons GAO identified include (1)
engage in a robust departmental oversight review process (2) separate
the research and development functions from acquisition functions (3)
determine the technology readiness levels before moving forward to
acquisition, and (4) rigorously test devices using actual agency
operational tactics before making decisions on acquisition.
View [hyperlink, http://www.gao.gov/products/GAO-10-1041T] or key
components. For more information, contact Gene Aloise at 202-512-3841
or aloisee@gao.gov.
[End of section]
Mr. Chairman and Members of the Committee:
We appreciate the opportunity to provide a statement for the record on
efforts of the Department of Homeland Security's (DHS) Domestic
Nuclear Detection Office (DNDO) to develop the cargo advanced
automated radiography system (CAARS) to strengthen DHS's ability to
prevent the smuggling of nuclear materials into the United States.
Preventing terrorists from using radiological or nuclear materials to
carry out an attack in the United States is a top national priority.
As we reported in January 2009, a terrorist could try to smuggle
nuclear materials into the United States in a variety of ways,
including hiding them in a car, train, private aircraft or small
vessel; sending them through the mail; carrying them in personal
luggage through an airport; walking them across the border; or
concealing them in maritime cargo containers in the global supply
chain.[Footnote 1] Maritime cargo containers are of particular concern
because they can be filled overseas at many different locations and
are transported through complex logistics networks before reaching
U.S. ports. As a result, terrorists could try to take advantage of
such vulnerabilities by placing nuclear materials into a container for
shipment to the United States. U.S. government officials believe that
the likelihood of terrorists smuggling nuclear materials into the
United States in cargo containers is relatively low, but criminals
have long exploited containers for other illegal purposes, such as
smuggling weapons, people and illicit substances.
As we testified before this committee in June, DHS has made
significant progress over the past several years in both deploying
radiation detection equipment and developing procedures to scan cargo
and conveyances entering the United States through fixed land and sea
ports of entry for nuclear and radiological materials. [Footnote 2]
Moreover, DHS reports that while it scans nearly 100 percent of the
cargo and conveyances entering the United States through land borders
and major seaports, it has made less progress scanning for radiation
in other pathways into the United States such as general aviation and
small maritime craft.
DNDO is charged with developing, acquiring, and deploying equipment to
detect nuclear and radiological materials in order to support the
efforts of DHS and other federal agencies, such as the departments of
Energy and State, in combating nuclear smuggling. DNDO is also charged
with enhancing and coordinating federal, state, and local efforts to
prevent radiological and nuclear attacks. In doing this, DNDO is
required to work with other federal agencies to develop a global
nuclear detection architecture.[Footnote 3] To date, DHS has spent
nearly $4 billion on various aspects of the architecture but has not
developed a strategic plan to guide its efforts to develop and
implement this architecture as we recommended in 2008.[Footnote 4]
DNDO agreed with this recommendation but has not developed such a plan.
Also within DHS, Customs and Border Protection (CBP) has the lead for
deploying, operating, and maintaining systems to detect nuclear and
radiological materials entering the country through land borders,
seaports, and other ports of entry. CBP also has a broad mission to
detect more traditional contraband, such as drugs and guns; to prevent
the inflow of inadmissible aliens; and to conduct its operations in a
way that does not impede the flow of commerce. To detect nuclear
materials, CBP, in coordination with DNDO, has deployed over 1,400
radiation portal monitors (RPM) at U.S. ports of entry. Most of the
RPMs are installed in primary inspection lanes through which nearly
all traffic and shipping containers must pass. These monitors alarm
when they detect radiation coming from a package, vehicle, or shipping
container. CBP then conducts further inspections at its secondary
inspection locations to identify the cause of the alarm and whether it
is a reason for concern.
While these RPMs are sensitive and have been effective at detecting
radiation, they also have limitations. In particular, in May 2009 we
reported that RPMs are capable of detecting certain nuclear materials
only when these materials are unshielded or lightly shielded.[Footnote
5] In contrast, advanced radiography can be used to detect dense
material that may be consistent with the presence of certain nuclear
materials in vehicles and cargo containers, and CBP already uses
radiography to more closely investigate the contents of a vehicle or
cargo container that has been selected for secondary inspection at a
U.S. port of entry. However, according to CBP officials, only a small
percentage of vehicles or cargo containers are subjected to secondary
inspections.
DNDO began working on the CAARS program in 2005 with the intention
that through advanced radiography and improved algorithms (software),
CAARS would be used by CBP to automatically detect and identify highly
shielded nuclear material in vehicles and cargo containers in both
primary and secondary inspection lanes at U.S. ports of entry. Thus,
through CAARS, DNDO expected that CBP would be able to detect more
heavily shielded nuclear material in nearly all vehicles and cargo
containers going through primary inspection lanes, and therefore,
close a gap in the nuclear detection architecture. In September 2006,
DNDO awarded a contract for the CAARS program to research, develop,
acquire, and deploy advanced radiography imaging technology designed
to detect highly shielded nuclear material being smuggled through U.S.
ports of entry. At that time, DNDO expected the program could cost as
much as $1.5 billion. However, in December 2007, DNDO made what it
called a "course correction," by canceling the program's acquisition
and deployment plans and significantly reducing its scope. The CAARS
program then became a research and development program designed to
demonstrate the potential capability of the technology. As a result of
this change, DNDO no longer expected to deploy CAARS machines but
instead opted to demonstrate the maturity and promise of CAARS
technology. As part of the CAARS demonstration project, DNDO, with the
assistance of scientists from the Department of Energy's Lawrence
Livermore National Laboratory (LLNL), began testing the capabilities
of CAARS' prototypes in the fall of 2009 and completed testing in
March 2010. According to DNDO officials, DNDO plans to report on the
results of the tests and lay out a way forward regarding the future
application of CAARS radiography imaging by the end of September 2010.
Overall, from the inception of the program in 2005 until today, DNDO
officials reported that the agency has spent about $113 million on the
CAARS program.
Since the capabilities of radiography systems are an important part of
cargo security, you asked us to examine the history of the CAARS
program. Accordingly, this statement discusses events that led to the
course correction in the CAARS program, and provides potential lessons
learned from DNDO's experience with the CAARS program. This statement
is based on prior reports and testimonies we issued from March 2006
through July 2010.[Footnote 6] Detailed information on our scope and
methodology for our prior work can be found in these reports. This
statement is also based on results from our ongoing work in response
to your request to review radiography systems within DHS.
As part of our ongoing review of radiography systems, specifically
CAARS, from March 2010 to September 2010 we analyzed key CAARS
planning documents developed in the early stages of program
development, such as the acquisition plan, program baseline,
performance specifications, and requests for proposals and reviewed
subsequent CAARS documents that reflect DNDO's decision to scale back
the program and removal of the acquisition phase from CAARS contracts.
We conducted interviews with former and current CAARS program managers
and other key officials within DNDO, CBP, the DHS Science & Technology
Directorate (S&T), and the DHS Office of Policy. We also met with
representatives from contractors that were developing CAARS and
consulted with subject matter experts from LLNL involved in testing
those contractors' CAARS prototypes. We provided a draft of the
information in this testimony to DHS and component agencies, which
provided technical comments and which we incorporated as appropriate.
We conducted this work in accordance with generally accepted
government auditing standards. Those standards require that we plan
and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on
our audit objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
DNDO Planned for the Acquisition and Deployment of CAARS without Fully
Understanding that It Could Not Feasibly Operate in a U.S. Port
Environment:
From the start of the CAARS program in 2005 until the course
correction in December 2007, DNDO planned the acquisition and
deployment of CAARS machines without understanding that they would not
fit within existing primary inspection lanes at CBP ports of entry.
This occurred because during the first year or more of the program
DNDO and CBP had few discussions about operating requirements for
primary inspection lanes at ports of entry. In addition, the CAARS
program was among numerous acquisition programs about which we
previously reported that appropriate DHS oversight was lacking.
Furthermore, the development of the CAARS algorithms--a key part of
the machine needed to identify shielded nuclear materials
automatically--did not mature at a rapid enough pace to warrant
acquisition and deployment. Moreover, the description of the progress
of the CAARS program used to support funding requests in DNDO's budget
justifications for fiscal years 2009 through 2011 was misleading
because it did not reflect the actual status of the program.
Inadequate Communication:
From the inception of the CAARS program until the decision in December
2007 to cancel acquisition of the program, DNDO and CBP had few, if
any, in-depth discussions about CBP's requirements to be able to use
radiography in primary inspection lanes. According to DNDO officials,
they requested information from CBP on its user requirements for the
CAARS system, but CBP was slow to respond to these requests. DNDO
continued with its plans to develop CAARS machines because, according
to DNDO officials, at the time it was thought that a solution was
urgently needed to be able to detect shielded nuclear materials in
primary inspection lanes. In discussing this with senior CBP
officials, they said that DNDO officials did not attempt to meet with
them during the beginning of the CAARS program. When CBP and DNDO
officials met, shortly before the course correction, CBP officials
said they made it clear to DNDO that they did not want the CAARS
machines because they would not fit in primary inspections lanes and
would slow down the flow of commerce through these lanes and cause
significant delays.[Footnote 7]
In our view, had CBP and DNDO officials met early in the development
of the program to discuss CBP's needs and operational requirements, as
stated in DHS's acquisition policy at the time, it is unlikely that
DNDO would have found reason to move forward with its plan to develop
and acquire CAARS technology. Nonetheless, in September 2006, DNDO
awarded contracts to three CAARS vendors. In December 2007, DNDO
decided to cancel the acquisition of CAARS and limit any further work
to a research and development effort. In recent joint discussions with
CBP and DNDO officials, they acknowledged that communication between
the two agencies could have been improved during the early part of the
CAARS program. They said they communicate much more routinely now and
that, in their view, it would be unlikely that the communication
problems associated with the CAARS program would reoccur.
Inadequate Oversight:
DNDO did not follow DHS acquisition protocols for the CAARS program.
Specifically, in 2008, we reported that CAARS was among numerous major
DHS acquisition programs that did not have a mission needs statement--
a required DHS acquisition document that formally acknowledges that
the need for an acquisition is justified and supported.[Footnote 8]
DHS policy also called for programmatic reviews at key decision points
and required certain analytical documents. However, CAARS did not
undergo annual department level reviews as called for nor did DNDO
program officials obtain or prepare basic analytical documents. For
example, one of these documents, a concept of operations (CONOPS), was
intended to demonstrate how CBP would use CAARS machines in primary
inspection areas at the ports. However, as a result of inadequate
communication and collaboration between CBP and DNDO discussed
earlier, no CONOPS was developed during the early phase of the CAARS
program. Ultimately, according to DNDO officials, once DNDO made the
decision to cancel the acquisition portion of CAARS in December 2007,
a CONOPS was no longer required.
Immature Technology:
According to DNDO officials, at the time of the inception of the CAARS
program, there was a widespread view within DNDO that something had to
be done to provide CBP with the capability to detect highly shielded
nuclear material in primary inspection lanes. DNDO officials
acknowledged that the agency decided to move forward with the CAARS
program despite the fact that automatic detection, a key feature of
CAARS, depended on the rapid development of algorithms that were
technologically immature. The algorithms are critical because they
provide the capability for CAARS to automatically detect highly
shielded nuclear material in primary inspection areas without the need
for extensive operator review and interpretation of an image--two
factors that could adversely affect CBP's ability to avoid delays to
the flow of commerce along with its overall effectiveness in detecting
highly shielded nuclear material. Although algorithms supporting the
CAARS technology were technologically immature, DNDO created an
aggressive production and deployment schedule that was to begin in
August 2008, the end of DNDO's planned 2-year development period for
the CAARS program. At the time it decided on this production
milestone, DNDO officials said it was likely that the algorithms would
be developed in time to meet the start of planned production. However,
the technology did not develop as expected and contributed to DNDO's
decision to cancel the acquisition phase of CAARS.
Basis of CAARS Funding Requests:
For fiscal year 2009 through fiscal year 2011, DHS justified annual
budget requests to Congress by citing significant plans and
accomplishments of the CAARS program, including that CAARS technology
development and deployment was feasible, even though DNDO had made the
decision in December 2007 to cancel the acquisition of CAARS. For
example, in its fiscal year 2009 budget justification, DHS stated that
a preliminary DNDO/CBP CAARS production and deployment program had
been successfully developed and that CAARS machines would be developed
that would detect both contraband and shielded nuclear material with
little or no impact on CBP operations. The fiscal years 2010 and 2011
DHS budget justifications both cited that an ongoing testing campaign
would lead to a cost benefit analysis, followed by rapid development
of a prototype that would lead to a pilot deployment at a CBP point of
entry. Furthermore, the fiscal year 2010 budget justification stated
that while the CAARS technology was less mature than originally
estimated, successful development was still feasible. However, DHS's
description and assessment of the CAARS program in its budget
justification did not reflect the actual progress of the program.
Specifically, DNDO officials told us that when they made their course
correction and canceled the acquisition part of the program in 2007,
they also decided not to conduct a cost benefit analysis because such
analyses are generally needed to justify going forward with
acquisitions. In addition, DNDO completed CAARS testing in March 2010;
however, as of today, the final test results for two of the three
CAARS machines are not yet available. Currently, no CAARS machines
have been deployed. CAARS machines from various vendors have either
been disassembled or sit idle without being tested in a port
environment, and CBP is considering whether to allow DNDO to collect
operational data in a port environment. During recent discussions with
DNDO officials, they agreed that the language in the budget
justifications lacked clarity and stated that they are not planning to
complete a cost benefit analysis since such analyses are generally
associated with acquisition programs.
CAARS Offers Lessons Learned Regarding the Importance of Developing
Requirements, Coordinating with Users, and Managing Acquisitions:
Based on our review of the CAARS program and our reports on DNDO
efforts to develop an advanced RPM called the advanced spectroscopic
portal (ASP),[Footnote 9] we have identified lessons learned for DHS
to consider in its continuing efforts to develop the next generation
of radiography imaging technology.
Enhance Interagency Collaboration and Coordination:
Despite the importance of coordinating crosscutting program efforts,
we have reported that weak coordination of those efforts has been a
long-standing problem in the federal government and has proven to be
difficult to resolve.[Footnote 10] We have also reported that agencies
can enhance and sustain their collaborative efforts. One way we
reported that agencies can enhance coordination is to agree on roles
and responsibilities and establish mutually reinforcing or joint
strategies.[Footnote 11] As discussed, DNDO did not coordinate and
collaborate with CBP early in the development of the CAARS program to
identify CBP's needs and requirements. According to DHS budget
documents, in fiscal year 2011, the responsibility for research and
development of advanced radiography will shift from DNDO to S&T.
Leading up to this transition, there is confusion related to roles and
responsibilities among DNDO, S&T, and CBP. For example, DNDO officials
said they have requested permission from CBP to collect operational
data in a port environment on an enhanced radiography machine.
However, CBP officials stated that they had already purchased,
operationally tested, and deployed 11 of these machines in secondary
inspection areas. We recently discussed this issue at a joint meeting
with DNDO and CBP officials. CBP and DNDO officials agreed that there
was confusion over this issue, and both agencies agreed with the need
to collect operational data on this enhanced radiography machine, and
CBP has begun making arrangements to do so.
Also, S&T officials said that they are about to contract out for
radiography imaging technology for CBP that will improve imaging
capabilities. DNDO officials told us that S&T's efforts will include
development of radiography capabilities to detect shielded nuclear
material, while S&T officials told us that this is not an area of
their focus. As DHS transitions its research and development of
radiography, DHS officials said that a draft memorandum of agreement
intended to clarify roles and responsibilities for cooperation and
coordination among DNDO, CBP, and S&T has not been finalized.
Completing the memorandum of agreement to clarify roles and
responsibilities before proceeding with the research, development, and
deployment of radiography technology could give DHS reasonable
assurance that problems resulting from a lack of clearly defined roles
and responsibilities in the CAARS program do not recur. In discussions
with senior officials from DHS, DNDO, CBP and S&T, they all agreed
with the need for the memorandum and said that they intend to work
toward finalizing the draft memorandum of agreement.
Engage in a Robust Oversight Review Process:
DNDO officials said that they were aware of the DHS draft management
directive in 2006 that was intended to guide management and oversight
of acquisition programs like CAARS but did not follow it. DHS policy
officials acknowledged that at the time CAARS was in its early stages,
DHS was continuing the process of organizing and unifying its many
disparate components and there was not strong oversight over its major
programs, including CAARS. Policy officials told us the oversight
review process is more robust today. However, we reported in June 2010
that DHS acquisitions need further improvement and sustained
management attention. [Footnote 12] For example, while DHS' current
management directive includes more detailed guidance than the previous
2006 management directive for programs to use in preparing key
documentation to support component and departmental decision making,
it is not applied consistently and most major programs have not been
reviewed.
Separate Research and Development from Acquisition Functions:
DNDO was simultaneously engaged in a research and development phase
while planning for an acquisition phase of the CAARS program. In this
regard, we have previously reported that separating technology
development from product development and acquisition is a best
practice that can help reduce costs and deliver a product on time and
within budget because separation of the technology development phase
from production in particular helps to ensure that (1) a sound
business case is made for the product, (2) product design is stable,
and (3) production processes are mature and the design is reliable.
[Footnote 13]
Determine the Technology Readiness Levels Before Acquisition:
At the time that the CAARS program was in its early stages, DHS and
DNDO did not have clearly defined ways to define and communicate the
maturity of technology leading to acquisition. We have previously
reported on the need for a disciplined and knowledge-based approach of
assessing technology maturity, such as using technology readiness
levels.[Footnote 14] In that report, we recommended that technologies
need to reach a high readiness level before an agency should make a
commitment to production. DNDO officials acknowledged that CAARS
algorithm's readiness level was not high enough to warrant entering
into the acquisition phase.
Rigorously Test Devices Using Actual Agency Operational Tactics:
As we testified in June 2009 on DNDO's testing of ASPs, a primary
lesson to be learned regarding testing is that the push to replace
existing equipment with the new portal monitors led to an ASP testing
program that lacked the necessary rigor.[Footnote 15] We reported that
testing programs designed to validate a product's performance against
increasing standards for different stages in product development are a
best practice for acquisition strategies for new technologies and if
properly implemented, would provide rigor to DHS's testing of other
advanced technologies.
GAO Contacts and Staff Acknowledgment:
For further information about this statement, please contact Gene
Aloise at (202) 512-3841 or aloisee@gao.gov; or Stephen L. Caldwell at
202-512-9610 or caldwells@gao.gov. Dr. Timothy Persons (Chief
Scientist), Ned Woodward (Assistant Director), Mike Harmond, Jonathan
Kucskar, Linda Miller, Ron Salo, Kiki Theodoropoulos, and Franklyn Yao
also made key contributions to this testimony.
[End of section]
Appendix I: Related GAO Products:
Maritime Security: DHS Progress and Challenges in Key Areas of Port
Security. [hyperlink, http://www.gao.gov/products/GAO-10-940T].
Washington, D.C.: July 21, 2010.
Combating Nuclear Smuggling: DHS Has Made Some Progress but Not Yet
Completed a Strategic Plan for Its Global Nuclear Detection Efforts or
Closed Identified Gaps, [hyperlink,
http://www.gao.gov/products/GAO-10-883T], Washington D.C.: June 30,
2010.
Supply Chain Security: Feasibility and Cost-Benefit Analysis Would
Assist DHS and Congress in Assessing and Implementing the Requirement
to Scan 100 Percent of U.S.-Bound Containers. [hyperlink,
http://www.gao.gov/products/GAO-10-12]. Washington, D.C.: Oct. 30,
2009.
Combating Nuclear Smuggling: Lessons Learned from DHS Testing of
Advanced Radiation Detection Portal Monitors, [hyperlink,
http://www.gao.gov/products/GAO-09-804T], Washington, D.C.: June 25,
2009.
Combating Nuclear Smuggling: DHS Improved Testing of Advanced
Radiation Detection Portal Monitors, but Preliminary Results Show
Limits of the New Technology. [hyperlink,
http://www.gao.gov/products/GAO-09-655]. Washington D.C.: May 21, 2009.
Nuclear Detection: Domestic Nuclear Detection Office Should Improve
Planning to Better Address Gaps and Vulnerabilities. [hyperlink,
http://www.gao.gov/products/GAO-09-257]. Washington, D.C.: Jan. 29,
2009,
Combating Nuclear Smuggling: DHS's Program to Procure and Deploy
Advanced Radiation Detection Portal Monitors is Likely to Exceed the
Department's Previous Cost Estimates. [hyperlink,
http://www.gao.gov/products/GAO-08-1108R]. Washington, D.C.: Sept. 22,
2008.
Supply Chain Security: CBP Works with International Entities to
Promote Global Customs Security Standards and Initiatives, but
Challenges Remain. [hyperlink,
http://www.gao.gov/products/GAO-08-538]. Washington, D.C.: Aug. 15,
2008:
Maritime Security: National Strategy and Supporting Plans Were
Generally Well-Developed and Are Being Implemented. [hyperlink,
http://www.gao.gov/products/GAO-08-672]. Washington, D.C.: June 20,
2008.
Supply Chain Security: Challenges to Scanning 100 Percent of U.S.-
Bound Cargo Containers. [hyperlink,
http://www.gao.gov/products/GAO-08-533T]. Washington, D.C.: June 12,
2008.
Supply Chain Security: U.S. Customs and Border Protection Has Enhanced
Its Partnership with Import Trade Sectors, but Challenges Remain in
Verifying Security Practices. [hyperlink,
http://www.gao.gov/products/GAO-08-240]. Washington, D.C.: Apr. 25,
2008.
Supply Chain Security: Examination of High-Risk Cargo at Foreign
Seaports Have Increased, but Improved Data Collection and Performance
Measures Are Needed. [hyperlink,
http://www.gao.gov/products/GAO-08-187]. Washington, D.C.: Jan. 25,
2008.
Department of Homeland Security: Billions Invested in Major Programs
Lack Appropriate Oversight. [hyperlink,
http://www.gao.gov/products/GAO-09-29]. Washington, D.C.: Nov. 18,
2008).
Maritime Security: The SAFE Port Act: Status and Implementation One
Year Later. [hyperlink, http://www.gao.gov/products/GAO-08-126T].
Washington, D.C.: Oct. 30, 2007.
Combating Nuclear Smuggling: Additional Actions Needed to Ensure
Adequate Testing of Next Generation Radiation Detection Equipment.
[hyperlink, http://www.gao.gov/products/GAO-07-1247T]. Washington,
D.C.: Sept. 18, 2007.
Department of Homeland Security: Progress Report on Implementation of
Mission and Management Functions. [hyperlink,
http://www.gao.gov/products/GAO-07-454]. Washington, D.C.: Aug. 17,
2007.
International Trade: Persistent Weaknesses in the In-Bond Cargo System
Impede Customs and Border Protection's Ability to Address Revenue,
Trade, and Security Concerns. [hyperlink,
http://www.gao.gov/products/GAO-07-561]. Washington, D.C.: Apr. 17,
2007.
Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy the
Next Generation of Radiation Detection Equipment is Not Supported by
Its Cost-Benefit Analysis. [hyperlink,
http://www.gao.gov/products/GAO-07-581T]. Washington, D.C.: Mar. 14,
2007.
Combating Nuclear Smuggling: DNDO Has Not Yet Collected Most of the
National Laboratories' Test Results on Radiation Portal Monitors in
Support of DNDO's Testing and Development Program. [hyperlink,
http://www.gao.gov/products/GAO-07-347R]. Washington, D.C.: Mar. 9,
2007.
Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support
the Purchase of New Radiation Detection Portal Monitors was Not Based
on Available Performance Data and Did not Fully Evaluate All the
Monitors' Costs and Benefits. [hyperlink,
http://www.gao.gov/products/GAO-07-133R]. Washington, D.C.: Oct. 17,
2006.
Cargo Container Inspections: Preliminary Observations on the Status of
Efforts to Improve the Automated Targeting System. [hyperlink,
http://www.gao.gov/products/GAO-06-591T]. Washington, D.C.: Mar. 30,
2006.
Combating Nuclear Smuggling: Challenges Facing U.S. Efforts to Deploy
Radiation Detection Equipment in Other Countries and in the United
States. [hyperlink, http://www.gao.gov/products/GAO-06-558T].
Washington, D.C.: Mar. 28, 2006.
Combating Nuclear Smuggling: DHS Has Made Progress Deploying Radiation
Detection Equipment at U.S. Ports-of-Entry, but Concerns Remain.
[hyperlink, http://www.gao.gov/products/GAO-06-389]. Washington, D.C.:
Mar. 22, 2006.
[End of section]
Footnotes:
[1] GAO, Nuclear Detection: Domestic Nuclear Detection Office Should
Improve Planning to Better Address Gaps and Vulnerabilities,
[hyperlink, http://www.gao.gov/products/GAO-09-257] (Washington, D.C.:
Jan. 29, 2009).
[2] GAO, Combating Nuclear Smuggling: DHS Has Made Some Progress but
Not Yet Completed a Strategic Plan for Its Global Nuclear Detection
Efforts or Closed Identified Gaps, [hyperlink,
http://www.gao.gov/products/GAO-10-883T] (Washington, D.C.: Jun. 30,
2010).
[3] National Security Presidential Directive-43 (also known as
Homeland Security Presidential Directive-14), sec. 2(d).
[4] GAO, Nuclear Detection: Preliminary Observations on the Domestic
Nuclear Detection Office's Efforts to Develop a Global Nuclear
Detection Architecture, [hyperlink,
http://www.gao.gov/products/GAO-08-999T] (Washington, D.C.: July 16,
2008).
[5] GAO, Combating Nuclear Smuggling: DHS Improved Testing of Advanced
Radiation Detection Portal Monitors, but Preliminary Results Show
Limits of the New Technology, [hyperlink,
http://www.gao.gov/products/GAO-09-655] (Washington, D.C.: May 21,
2009).
[6] See Appendix I for a list of related GAO products.
[7] DNDO officials later acknowledged that they proceeded in
developing CAARS with the CBP specifications for using radiography in
secondary inspection areas--not knowing that these same specifications
were not suitable for primary inspection lanes.
[8] GAO, Department of Homeland Security: Billions Invested in Major
Programs Lack Appropriate Oversight, [hyperlink,
http://www.gao.gov/products/GAO-09-29] (Washington, D.C.: Nov. 18,
2008).
[9] GAO, Combating Nuclear Smuggling: Additional Actions Needed to
Ensure Adequate Testing of Next Generation Radiation Detection
Equipment, [hyperlink, http://www.gao.gov/products/GAO-07-1247T]
(Washington, D.C.: Sept. 18, 2007), and Combating Nuclear Smuggling:
Lessons Learned from DHS Testing of Advanced Radiation Detection
Portal Monitors, [hyperlink, http://www.gao.gov/products/GAO-09-804T]
(Washington, D.C.: June 25, 2009)
[10] GAO, Managing for Results: Barriers to Interagency Coordination,
[hyperlink, http://www.gao.gov/products/GAO/GGD-00-106] (Washington,
D.C.: Mar. 29, 2000).
[11] GAO, Results-Oriented Government: Practices that Can Help Enhance
and Sustain Collaboration among Federal Agencies, [hyperlink,
http://www.gao.gov/products/GAO-06-15] (Washington, D.C.: Oct. 21,
2005).
[12] GAO, Department of Homeland Security: Assessments of Selected
Complex Acquisitions, [hyperlink,
http://www.gao.gov/products/GAO-10-588SP] (Washington, D.C.: June 30,
2010).
[13] See GAO, Defense Acquisitions: Assessments of Selected Weapon
Programs, [hyperlink, http://www.gao.gov/products/GAO-07-406SP]
(Washington, D.C.: Mar. 30, 2007) and Best Practices: Better
Management of Technology Development Can Improve Weapon System
Outcomes, [hyperlink, http://www.gao.gov/products/GAO/NSIAD-99-162]
(Washington, D.C.: July 30, 1999).
[14] [hyperlink, http://www.gao.gov/products/GAO/NSIAD-99-162].
[15] GAO, Combating Nuclear Smuggling: Lessons Learned from DHS
Testing of Advanced Radiation Detection Portal Monitors, [hyperlink,
http://www.gao.gov/products/GAO-09-804T] (Washington, D.C.: June 25,
2009).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: