Combating Nuclear Smuggling
DHS's Decision to Procure and Deploy the Next Generation of Radiation Detection Equipment Is Not Supported by Its Cost-Benefit Analysis Gao ID: GAO-07-581T March 14, 2007The Department of Homeland Security (DHS) is responsible for addressing the threat of nuclear smuggling. Radiation detection portal monitors are key elements in our national defenses against such threats. DHS has sponsored R&D and testing activities to develop a "next generation" portal monitor, known as the advanced spectroscopic portal monitor. However, each one costs 6 times more than a current portal monitor. In March 2006, we recommended that DHS conduct a cost-benefit analysis to determine whether the new portal monitors are worth the additional cost. In June 2006, DHS issued its analysis. In October 2006, we issued our report that assessed the DHS study. GAO's statement, based on our October 2006 report, addresses whether DHS's cost-benefit analysis provides an adequate basis for its decision to purchase and deploy the next generation portal monitors.
DHS's cost-benefit analysis does not provide a sound analytical basis for its decision to purchase and deploy the new portal monitor technology. Our review of the analysis determined that it had the following problems. Regarding the performance of the portal monitors: instead of using the results of its own portal monitor tests conducted in 2005, DHS assumed that the new portal monitor technology would correctly detect and identify highly enriched uranium (HEU) 95 percent of the time--a performance level that far exceeds the new technology's current capabilities. To determine the performance of the current generation of portal monitors in detecting HEU, DHS used data from limited tests carried out in 2004 that test officials concluded were unreliable for such purposes. DHS's analysis of the new technology portal monitors was incomplete because the analysis focused on identifying HEU, but did not fully consider how well the new portal monitor technology could correctly detect or identify other dangerous radiological or nuclear materials. Regarding cost estimates: in comparing the costs of the new and current technologies, the procurement costs of the current generation portal monitors were highly inflated because DHS assumed a unit cost of about $131,000. However, the contract price at the time of the analysis was about $55,000. According to officials who manage the contract, it was to expire and they expected portal monitor prices to increase, but not nearly as much as DHS assumed. DHS stated that the primary benefit of deploying the new portal monitors is reducing unnecessary secondary inspections. However, DHS's analysis does not fully estimate today's baseline costs for secondary inspections, which makes it impossible to determine whether the use of the new portal monitors as currently planned, will result in significant cost savings for these inspections. The new portal monitor contract price has exceeded DHS's total cost estimate by about $200 million. The cost-benefit analysis shows the total cost for deploying both current and new portal monitors to be about $1 billion. However, in July 2006, DHS announced that it had awarded contracts to develop and purchase up to $1.2 billion worth of the new portal monitors over 5 years. DHS's cost-benefit analysis omitted many factors that could affect the cost of new portal monitors, such as understating the life-cycle costs for operating and maintaining the equipment over time. For these reasons, DHS's cost-benefit analysis does not meet the intent of our March 2006 report recommendation to fully assess the costs and benefits before purchasing any new equipment.
GAO-07-581T, Combating Nuclear Smuggling: DHS's Decision to Procure and Deploy the Next Generation of Radiation Detection Equipment Is Not Supported by Its Cost-Benefit Analysis
This is the accessible text file for GAO report number GAO-07-581T
entitled 'Combating Nuclear Smuggling: DHS's Decision to Procure and
Deploy the Next Generation of Radiation Detection Equipment Is Not
Supported by Its Cost-Benefit Analysis' which was released on March 14,
2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Emerging Threats, Cybersecurity, and Science
and Technology; Committee on Homeland Security; House of
Representatives:
United States Government Accountability Office:
GAO:
For Release on Delivery Expected at 2:00 p.m. EST:
Wednesday, March 14, 2007:
Combating Nuclear Smuggling:
DHS's Decision to Procure and Deploy the Next Generation of Radiation
Detection Equipment Is Not Supported by Its Cost-Benefit Analysis:
Statement of Gene Aloise, Director:
Natural Resources and Environment:
GAO-07-581T:
GAO Highlights:
Highlights of GAO-07-581T, a testimony before the Subcommittee on
Emerging Threats, Cybersecurity, and Science and Technology; Committee
on Homeland Security; House of Representatives
Why GAO Did This Study:
The Department of Homeland Security (DHS) is responsible for addressing
the threat of nuclear smuggling. Radiation detection portal monitors
are key elements in our national defenses against such threats. DHS has
sponsored R&D and testing activities to develop a ’next generation“
portal monitor, known as the advanced spectroscopic portal monitor.
However, each one costs 6 times more than a current portal monitor. In
March 2006, we recommended that DHS conduct a cost-benefit analysis to
determine whether the new portal monitors are worth the additional
cost. In June 2006, DHS issued its analysis. In October 2006, we issued
our report that assessed the DHS study.
GAO‘s statement, based on our October 2006 report, addresses whether
DHS‘s cost-benefit analysis provides an adequate basis for its decision
to purchase and deploy the next generation portal monitors.
What GAO Found:
DHS‘s cost-benefit analysis does not provide a sound analytical basis
for its decision to purchase and deploy the new portal monitor
technology. Our review of the analysis determined that it had the
following problems:
Regarding the performance of the portal monitors:
* Instead of using the results of its own portal monitor tests
conducted in 2005, DHS assumed that the new portal monitor technology
would correctly detect and identify highly enriched uranium (HEU) 95
percent of the time”a performance level that far exceeds the new
technology‘s current capabilities.
* To determine the performance of the current generation of portal
monitors in detecting HEU, DHS used data from limited tests carried out
in 2004 that test officials concluded were unreliable for such
purposes.
* DHS‘s analysis of the new technology portal monitors was incomplete
because the analysis focused on identifying HEU, but did not fully
consider how well the new portal monitor technology could correctly
detect or identify other dangerous radiological or nuclear materials.
Regarding cost estimates:
* In comparing the costs of the new and current technologies, the
procurement costs of the current generation portal monitors were highly
inflated because DHS assumed a unit cost of about $131,000. However,
the contract price at the time of the analysis was about $55,000.
According to officials who manage the contract, it was to expire and
they expected portal monitor prices to increase, but not nearly as much
as DHS assumed.
* DHS stated that the primary benefit of deploying the new portal
monitors is reducing unnecessary secondary inspections. However, DHS‘s
analysis does not fully estimate today‘s baseline costs for secondary
inspections, which makes it impossible to determine whether the use of
the new portal monitors as currently planned, will result in
significant cost savings for these inspections.
* The new portal monitor contract price has exceeded DHS‘s total cost
estimate by about $200 million. The cost-benefit analysis shows the
total cost for deploying both current and new portal monitors to be
about $1 billion. However, in July 2006, DHS announced that it had
awarded contracts to develop and purchase up to $1.2 billion worth of
the new portal monitors over 5 years.
* DHS‘s cost-benefit analysis omitted many factors that could affect
the cost of new portal monitors, such as understating the life-cycle
costs for operating and maintaining the equipment over time.
For these reasons, DHS‘s cost-benefit analysis does not meet the intent
of our March 2006 report recommendation to fully assess the costs and
benefits before purchasing any new equipment.
What GAO Recommends:
GAO‘s October 2006 report included two recommendations designed to
improve the quality and comprehensiveness of the Department‘s analysis.
DHS neither agreed nor disagreed with our recommendations, but
continued to support its analysis as a solid basis for buying and
deploying the new generation of radiation portal monitors
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-581T].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Gene Aloise at (202) 512-
3841 or aloisee@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
I am pleased to appear here today to discuss our assessment of the
Department of Homeland Security's (DHS) May 2006 cost-benefit analysis
used to support the purchase and deployment of next generation
radiation portal monitors.[Footnote 1] This is an important decision
because, if procured, these new portal monitors will be considerably
more expensive than the portal monitors in use today. Combating nuclear
smuggling is one of our nation's key national security objectives and
the deployment of radiation detection equipment including portal
monitors at U.S. ports of entry, including border crossings and
domestic seaports, is an integral part of this system. DHS, through its
Domestic Nuclear Detection Office (DNDO), is responsible for acquiring
and supporting the deployment of radiation detection equipment,
including portal monitors, within the United States. The Pacific
Northwest National Laboratory (PNNL), one of the Department of Energy's
(DOE) national laboratories, is under contract to manage the deployment
of radiation detection equipment for DHS.[Footnote 2] U.S. Customs and
Border Protection (CBP) is responsible for screening cargo as it enters
the nation at our borders, which includes operating radiation detection
equipment to interdict dangerous nuclear and radiological materials.
The radiation portal monitors in use today can detect the presence of
radiation, but they cannot distinguish between types of radiological
material. For example, they cannot tell the difference between harmless
products that emit radiation, such as ceramic tile, and dangerous
materials, such as highly enriched uranium (HEU), that could be used to
construct a nuclear weapon. Generally, CBP's standard procedures
require incoming cargo to pass through one of these radiation portal
monitors to screen for the presence of radiation. This "primary
inspection" serves to alert CBP officers when a radioactive threat
might be present. If there is a potential threat, CBP procedures
require a "secondary inspection." To confirm the presence of radiation,
this secondary inspection usually includes a second screening by a
radiation portal monitor as well as a screening by CBP officers using
radioactive isotope identification devices. These handheld devices are
used to differentiate between types of radioactive material to
determine if the radiation being detected is dangerous. Both the
radiation portal monitors and handheld devices are limited in their
abilities to detect and identify nuclear material.
DHS would like to improve the capabilities of its portal monitors so
that they can perform the dual roles of detecting radiation and
identifying radiological materials. In this regard, DHS has sponsored
research, development, and testing activities designed to create the
next generation of portal monitors capable of performing both
functions. These new, advanced portals are known as advanced
spectroscopic portals (ASPs). In July 2006, DHS awarded contracts to
three vendors for developing the advanced spectroscopic portals'
capabilities. These awards were based mainly on performance tests
conducted at DHS's Nevada Test Site in 2005, where ten competing
advanced spectroscopic vendors' monitors were evaluated. At the same
time, three current technology portal monitors were also tested.
To ensure that DHS's substantial investment in radiation detection
technology yields the greatest possible level of detection capability
at the lowest possible cost, in a March 2006 GAO report,[Footnote 3] we
recommended that once the costs and capabilities of ASPs were well
understood, and before any of the new equipment was purchased for
deployment, the Secretary of DHS work with the Director of DNDO to
analyze the costs and benefits of deploying ASPs. Further, we
recommended that this analysis focus on determining whether any
additional detection capability provided by the ASPs was worth the
considerable additional costs. In response to our recommendation, DNDO
issued its cost-benefit analysis in May 2006, and an updated, revised
version in June 2006. According to senior agency officials, DNDO
believes that the basic conclusions of its cost-benefit analysis show
that the new advanced spectroscopic portal monitors are a sound
investment for the U.S. government.
Mr. Chairman, my remarks today focus on the cost-benefit analysis DNDO
used in support of its decision to purchase new ASP portal monitors.
Specifically, I will discuss whether DNDO's June 2006 cost-benefit
analysis provides an adequate basis for the substantial investment that
acquiring and deploying ASPs will necessitate.
My testimony is based upon our October 2006 report that evaluated
DNDO's cost-benefit analysis.[Footnote 4] The work for our report was
done in accordance with generally accepted government auditing
standards.
In summary, DNDO's 2006 cost-benefit analysis does not provide a sound
analytical basis for its decision to purchase and deploy the new
advanced spectroscopic portal monitor technology.
Regarding the performance of the portal monitors:
* Instead of using the results of its performance tests conducted in
2005, DNDO's analysis simply assumed that ASPs could detect highly
enriched uranium 95 percent of the time, a performance level far
exceeding the capabilities of the new technology's current demonstrated
capabilities. The 2005 test results showed that the best of the three
winning vendor monitors could only identify masked HEU[Footnote 5]
about 50 percent of the time.
* To determine the current generation of portal monitors' performance
in detecting HEU, DNDO used data from limited tests carried out in 2004
that test officials concluded was unreliable for such purposes. In
their written report, test officials explicitly stated that the data
were not indicative of how well current technology portal monitors
might perform in the field particularly for detecting HEU.
* DNDO's analysis of the new technology portal monitors' performance
was deficient because it focused on detecting and identifying HEU, but
did not fully consider other dangerous radiological or nuclear
materials. DNDO should have assessed the ASPs' abilities to detect
several realistic threat materials.
Regarding cost estimates:
* In comparing the costs of the new and current technologies, the
procurement costs of the current generation portal monitors were highly
inflated because DNDO assumed a unit cost of about $131,000, while the
contract price at the time of the analysis was about $55,000. According
to officials who manage the contract, it was to expire and while they
expected portal monitor prices to increase, they did not believe the
cost would be as much as the price used in DNDO's analysis.
* DNDO stated that the primary benefit of deploying the new portal
monitors is reducing unnecessary secondary inspections. However, DNDO's
analysis did not fully estimate today's baseline costs for secondary
inspections, which makes it impossible to determine whether the use of
the new portal monitors as currently planned will result in significant
cost savings for these inspections.
* The new portal monitor contract price has exceeded DNDO's total cost
estimate by about $200 million. The cost-benefit analysis shows the
total cost for deploying both current and new portal monitors to be
about $1 billion. However, in July 2006, DHS announced that it had
awarded contracts to develop and purchase up to $1.2 billion worth of
the new portal monitors over 5 years.
* DNDO's cost-benefit analysis omitted many factors that could affect
the cost of new portal monitors, such as understating the life-cycle
costs for operating and maintaining the equipment over time.
Background:
In general, DHS is responsible for providing radiation detection
capabilities at U.S. ports of entry. Until April 2005, CBP managed this
program. However, on April 15, 2005, the President directed the
establishment of DNDO within DHS. DNDO's duties include acquiring and
supporting the deployment of radiation detection equipment, including
portal monitors. CBP continues its traditional screening function at
ports of entry to interdict dangerous nuclear and radiological
materials through the use of radiation detection equipment. The SAFE
Port Act of 2006 formally authorizes DNDO's creation and
operation.[Footnote 6] PNNL manages the deployment of radiation
detection equipment for DHS.
DHS's program to deploy radiation detection equipment at U.S. ports of
entry has two goals. The first is to use this equipment to screen all
cargo, vehicles, and individuals coming into the United States. The
second is to screen this traffic without delaying its movement into the
nation. DHS's current plans call for completing the deployment of
radiation portal monitors at U.S. ports of entry by September 2013. The
current technology portal monitors, known as plastic scintillators or
PVTs, cost about $55,000 per unit, while the advanced spectroscopic
portal monitors, known as ASPs, will cost around $377,000 per
unit.[Footnote 7]
In July 2006, DHS announced that it had awarded contracts to three ASP
vendors to further develop and purchase $1.2 billion worth of ASPs over
5 years. Congress, however, provided that none of DNDO's appropriated
funds for systems acquisition could be obligated for full procurement
of ASPs until the Secretary of DHS certifies through a report to the
Committees on Appropriations for the Senate and House of
Representatives that ASPs would result in a significant increase in
operational effectiveness.[Footnote 8]
DNDO Ignored Its Own Performance Test Results and Instead Relied on the
Potential Performance of New Portal Monitors and Unreliable Estimates
of Current Equipment Performance:
DHS is developing new portal monitors, known as ASPs that, in addition
to detecting nuclear or radiological material, can also identify the
type of material. In 2005, DNDO conducted side-by-side testing at the
Nevada Test Site (NTS)[Footnote 9] on 10 ASP systems and 3 PVT systems
developed by private sector companies, including the PVT systems
currently deployed. DHS requested that the National Institutes of
Standards and Technology (NIST) provide assistance by conducting an
independent analysis of data acquired during the last phase of
developmental testing of ASPs to help DHS determine the performance of
ASP portal monitors being proposed by private sector companies. NIST
compared the 10 ASP systems, and in June 2006 submitted a report to DHS
on the results of that testing.[Footnote 10]
Performance tests of ASPs showed that they did not meet DNDO's main
performance assumption in the cost-benefit analysis of correctly
identifying HEU 95 percent of the time it passes through portal
monitors. The 95 percent performance assumption included ASPs' ability
to both detect bare, or unmasked, HEU in a container and HEU masked in
a container with a more benign radiological material.[Footnote 11]
Based on NIST's assessment of the performance data, the ASP prototypes
(manufactured by the three companies that won DNDO's recent ASP
procurement contract) tested at NTS identified bare HEU only 70 to 88
percent of time. Performance tests also showed that ASPs' ability to
identify masked HEU fell far short of meeting the 95 percent goal
established for the cost-benefit analysis. According to DNDO,
identifying masked HEU is the most difficult case to address. DOE
officials told us that benign radiological materials could be used to
hide the presence of HEU. NIST reported that the best ASP prototype
DNDO tested in Nevada during 2005, and which won a procurement
contract, was able to correctly identify masked HEU and depleted
uranium (DU) only 53 percent of the time. Similarly, the ASP prototypes
submitted by the other two companies that won DNDO ASP procurement
contracts were able to identify masked HEU and DU only 45 percent and
17 percent of the time.
Despite these results, DNDO did not use the information from these
tests in its cost-benefit analysis. Instead, DNDO officials told us
that since the new portal monitors cannot meet the 95 percent detection
goal, they relied on the assumption that they will reach that level of
performance sometime in the future. DNDO officials asserted that the
ASPs' current performance levels would improve, but they provided no
additional information as to how the 95 percent goal will be achieved
or an estimate of when the technology would attain this level of
performance.
Moreover, DNDO's cost-benefit analysis only considered the benefits of
ASPs' ability to detect and identify HEU and did not consider the ASPs'
ability to detect and identify other nuclear and radiological
materials. The ability of an ASP to identify specific nuclear or
radiological materials depends on whether the ASP contains software
that is specific to those materials. In our view, a complete cost-
benefit analysis would include an assessment of ASPs' ability to detect
and identify a variety of nuclear and radiological material, not just
HEU. By excluding radiological and nuclear materials other than HEU,
DNDO's analysis did not consider the number of secondary inspections
that would be related to these materials and hence it likely
underestimated the costs of ASP use. Further, DNDO told us the
assumptions for the ability of ASP systems to detect and identify HEU
95 percent of the time came from the ASP performance specifications.
However, we examined the performance specifications and found no
specific requirement for detecting or identifying HEU with a 95 percent
probability.[Footnote 12] While there is a requirement in the
performance specification for the identification for HEU and other
special nuclear material, we found no associated probability of success
in performing this function.
DNDO's cost-benefit analysis also may not accurately reflect the
capabilities of PVTs to detect nuclear or radiological material. DNDO
officials acknowledged that DNDO tested the performance of PVTs along
with the ASPs in 2005, but did not use the results of these tests in
its cost-benefit analysis. According to these officials and NIST staff
who assisted in the testing, the PVT performance data were unusable
because the PVTs' background settings were not set properly.
Consequently, DNDO officials told us the analysis was based on the
performance of PVT monitors that PNNL tested during 2004 in New York.
However, the results from these field tests are not definitive because,
as PNNL noted in its final report, the tests did not use HEU and,
therefore, the results from the tests did not indicate how well PVT
portal monitors would be able to detect HEU in the field.[Footnote 13]
Moreover, the PVT portal monitors that PNNL used had only one radiation
detection panel as opposed to the four-panel PVT monitors that DHS
currently deploys at U.S. ports of entry. An expert at a national
laboratory told us that larger surface areas are more likely to detect
radiological or nuclear material. DNDO also stated that due to the
nature of the testing at the Nevada Test Site, the tests did not
provide the data needed for the cost-benefit analysis. According to
DNDO officials, this data would come from analysis of the performance
of fielded systems at U.S. ports-of-entry where the probability to
detect threats could be compared to false alarm rates.
DNDO's director stated twice in testimony before the House Homeland
Security Committee, Subcommittee on Prevention of Nuclear and
Biological Attack--once on June 21, 2005, and again on May 26, 2006--
that the ASP and PVT portals would be evaluated against one another in
"extensive high-fidelity" tests. In our view, the results of such
testing are critical to any decision by DNDO to employ new technology,
such as ASPs, that might help protect the nation from nuclear
smuggling. According to DNDO officials, new tests now underway at the
DHS Nevada Test Site are comparing the performance of ASPs and PVTs
side-by-side.
DNDO's Cost-Benefit Analysis was Incomplete and Used Inflated Cost
Estimates for PVT Equipment:
DNDO officials told us they did not follow the DHS guidelines for
performing cost-benefit analyses in conducting their own cost-benefit
analysis.[Footnote 14] These guidelines stipulate, among other things,
that such studies should address all of the major costs and benefits
that could have a material effect on DHS programs. However, DNDO's
analysis omitted many factors that could affect the cost of new
radiation portal monitors. For example, DNDO officials told us that
there are over 12 different types of ASP monitors, yet they only
estimated the cost of cargo portal monitors that would be used at land
border crossings. In reality, DNDO and CBP plan to deploy different
types of ASPs that would have varying costs, such as portal monitors at
seaports which would have higher costs. Additionally, DNDO did not
capture all the costs related to developing the different types of ASP
monitors. In our view, developing realistic cost estimates should not
be sacrificed in favor of simplicity.
DNDO also underestimated the life-cycle costs for operations and
maintenance for both PVT and ASP equipment over time. DNDO's analysis
assumed a 5-year life-cycle for both PVT and ASP equipment. However,
DNDO officials told us that a 10-year life-cycle was a more reasonable
expectation for PVT and ASP equipment. The analysis assumes that the
annual maintenance costs for PVT and ASP monitors will each equal 10
percent of their respective procurement costs. This means that
maintenance costs for PVTs would be about $5,500 per year per unit
based on a $55,000 purchase price and ASP maintenance costs would be
about $38,000 per year per unit based on a $377,000 purchase price.
Given the much higher maintenance costs for ASPs and the doubling of
the life-cycle to 10 years, the long-term implications for these cost
differences would be magnified. Consequently, DNDO's analysis has not
accounted for about $181 million in potential maintenance costs for
ASPs alone.
Furthermore, DNDO did not assess the likelihood that radiation
detection equipment would either misidentify or fail to detect nuclear
and radiological materials. Rather, DNDO's cost-benefit analysis
focuses on the ability of ASPs to reduce false alarms--alarms that
indicate nuclear or radiological material is in a container when, in
fact, the material is actually non-threatening, such as ceramic tile.
Reducing false alarms would reduce the number of secondary inspections
of non-threatening nuclear and radiological materials and therefore the
costs of those inspections. However, as required by DHS's guide to
performing cost-benefit analyses, DNDO's analysis did not include all
costs. In particular, the analysis did not include the potentially much
bigger cost of "false negatives." False negatives are instances in
which a container possesses a threatening nuclear or radiological
material, but the portal monitor either misidentifies the material as
non-threatening or does not detect the material at all, thus allowing
the material to enter the country. During the 2005 Nevada tests, the
incidence of false negatives among the three vendors who received
contracts ranged from about 45 percent to slightly more that 80
percent. This raises concerns because, as explained to us by a
scientist at a national laboratory, at this level of performance, ASPs
could conceivably misidentify HEU as a benign nuclear or radiological
material or not detect it at all, particularly if the HEU is placed
side by side with a non-threatening material such as kitty litter.
In recent testimonies before Congress, DNDO's Director has cited the
primary benefit of deploying ASP monitors as reducing unnecessary
secondary inspections.[Footnote 15] DNDO's cost-benefit analysis
focused on measuring the benefits of ASP's ability to reduce false
alarms--alarms that indicate nuclear or radiological material is
present when, in fact, it is not or such material is actually non-
threatening. Reducing false alarms would reduce the number of secondary
inspections of non-threatening nuclear and radiological materials and
therefore the costs of those inspections. Even on this point, however,
DNDO's analysis was flawed. For example, it did not estimate the costs
of secondary inspections as they are carried out today. DNDO's analysis
needs these baseline costs to compare alternatives because without
them, it is impossible to determine whether the use of ASPs, as
planned, will result in cost savings for secondary inspections. While
we agree that facilitating commerce at U.S. ports of entry by reducing
unnecessary secondary inspections is an important goal, we believe that
the primary rationale for deploying portal monitors should be to
protect the nation from nuclear and/or radiological attack. We found
that DNDO's analysis did not even attempt to measure the level--or
value--of security afforded by portal monitors.
In addition, the ASP contract award has exceeded DNDO's estimate for
total cost by about $200 million. The cost-benefit analysis shows the
total cost for deploying PVT and ASP monitors to be about $1 billion,
which covers all costs related to acquisition, design, maintenance, and
physical inspection over 5 years (for both PVT and ASP). However, in
July 2006, DHS announced that it had awarded contracts to develop and
purchase up to $1.2 billion worth of ASP portal monitors over 5 years.
Furthermore, the cost-benefit analysis underestimates total deployment
costs and does not account for other major costs, such as physical
inspections of cargo containers, an additional procurement of 442 new
PVT monitors, installation and integration, and maintenance.
Finally, DNDO overstated the purchase price of PVT monitors. Although
DHS is currently paying an average of about $55,000 per monitor, DNDO's
cost-benefit analysis assumed the PVT would cost $130,959--the highest
published manufacturers' price for the government.[Footnote 16]
According to DNDO's Director, DNDO chose the highest published price
because the current contract for portal monitors at that time was to
expire, and the portal monitors will probably cost more in the future.
However, the information DNDO provided us does not explain why DNDO
assumes that the future price will be more than double what DHS was
currently paying, as assumed in DNDO's analysis. PNNL officials told us
that the future price will almost certainly be lower than the price
used in DNDO's analysis.
In conclusion, DNDO's approach to the cost-benefit analysis omitted
many factors that could affect the cost of new radiation portal
monitors. For these reasons, DHS's cost-benefit analysis does not meet
the intent of our March 2006 report recommendation to fully assess the
costs and benefits before purchasing any new equipment.
Mr. Chairman, this concludes my prepared statement. I would be happy to
respond to any questions that you or other members of the Subcommittee
may have.
GAO Contact and Staff Acknowledgements:
For further information about this testimony, please contact me, Gene
Aloise, at (202) 512-3841 or at aloisee@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this statement. John Delicath, Jim Shafer, and Eugene
Wisnoski made key contributions to this statement.
FOOTNOTES
[1] GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to
Support the Purchase of New Radiation Detection Portal Monitors Was Not
Based on Available Performance Data and Did Not Fully Evaluate All the
Monitors' Costs and Benefits, GAO-07-133R (Washington, D.C.: Oct. 17,
2006). GAO, Combating Nuclear Smuggling: DHS Has Made Progress
Deploying Radiation Detection Equipment at U.S. Ports of Entry, but
Concerns Remain, GAO-06-389 (Washington, D.C.: Mar. 22, 2006).
[2] DOE manages the largest laboratory system of its kind in the world.
The mission of DOE's 22 laboratories has evolved. Originally created to
design and build atomic weapons, these laboratories have since expanded
to conduct research in many disciplines - from high-energy physics to
advanced computing.
[3] GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying
Radiation Detection Equipment at U.S. Ports of Entry, but Concerns
Remain, GAO-06-389 (Washington, D.C.: Mar. 22, 2006).
[4] GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to
Support the Purchase of New Radiation Detection Portal Monitors Was Not
Based on Available Performance Data and Did Not Fully Evaluate All the
Monitors' Costs and Benefits, GAO-07-133R (Washington, D.C.: Oct. 17,
2006).
[5] "Masking" is an attempt to hide dangerous nuclear or radiological
material by placing it with benign radiological sources.
[6] Pub. L. No. 109-347, tit. V, 120 Stat. 1884, 1932 (2006).
[7] Prices include only equipment purchase. Installation costs are
extra.
[8] Department of Homeland Security Appropriations Act for Fiscal Year
2007, Pub. L. No. 109-295, tit. IV, 120 Stat. 1355, 1376 (2006).
[9] DHS and DOE are collaborating in building a new Radiological and
Nuclear Countermeasures Test and Evaluation Complex at the Nevada Test
Site to support the development, testing, acquisition, and deployment
of radiation detection equipment. The facility is expected to become
fully operational in early 2007. Currently, an interim facility at NTS
is being used to test radiation detection equipment.
[10] NIST did not evaluate the PVTs or compare their performance to the
performance of the ASPs.
[11] The ability to detect masked HEU is based on DOE guidance on
performing the evaluation.
[12] The performance specifications contain a requirement for
detecting, not identifying, californium-252 with a 95 percent
probability. Californium-252 has similar radiological properties to
HEU. In addition, the performance specifications contain a requirement
for detecting, but not identifying, other radiological materials such
as cobalt-57, cobalt-60, barium-133, cesium-137, and americium-241.
[13] Pacific Northwest National Laboratory, A Sensitivity Comparison of
NaI and PVT Portal Monitors at a Land-Border Port-of-Entry, p. iii,
November 2004. For Official Use Only.
[14] DHS, Capital Planning and Investment Control, Cost-Benefit
Analysis (CBA) Guidebook 2006, Version 2.0, February 2006. Traditional
rules of performing cost-benefit analyses include assessing the full
life-cycle costs for operation and maintenance, and determining the
level of confidence in cost data.
[15] Enlisting Foreign Cooperation in U.S. Efforts to Prevent Nuclear
Smuggling: Hearing Before the House Committee on Homeland Security,
Subcommittee on Prevention of Nuclear and Biological Attack, 109th
Cong. (May 25, 2006)(statement of Mr. Vayl S. Oxford, Director, DNDO);
Detecting Smuggled Nuclear Weapons, Hearing Before the Senate Judiciary
Committee, Subcommittee on Terrorism, Technology, and Homeland
Security, 109th Cong. (July 27, 2006)(statement of Mr. Vayl S. Oxford,
Director, DNDO).
[16] DNDO, Cost Benefit Analysis for Next Generation Passive Radiation
Detection of Cargo an the Nation's Border Crossings, May 30, 2006.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
