Chemical Demilitarization
Actions Needed to Improve the Reliability of the Army's Cost Comparison Analysis for Treatment and Disposal Options for Newport's VX Hydrolysate Gao ID: GAO-07-240R January 26, 2007The U.S. stockpile of 1,269 tons of VX nerve agent stored at the Newport Chemical Depot (Newport), Indiana, is one of nine stockpiles that the Department of Defense (DOD) must destroy in response to congressional direction initially provided in 1985. In addition, the stockpile must be destroyed to comply with the requirements of the Chemical Weapons Convention, which the United States became a party to in 1997. The stockpile at Newport is the first U.S. stockpile containing VX that will be destroyed by using neutralization--a process that mixes hot water and sodium hydroxide (a caustic chemical) with VX to change the chemical composition to a less toxic form. The resulting by-product is a liquid wastewater commonly referred to as hydrolysate that consists mostly of water but also has a caustic component and organic salts that need further treatment to meet Chemical Weapons Convention requirements and to meet federal and state environmental requirements for disposal. The Army, DOD's designated executive agent, began neutralizing Newport's VX stockpile on-site in May 2005 and, as of December 1, 2006, reports neutralizing about 34 percent of the stockpile. None of the generated hydrolysate--expected to be about 2 million gallons when the neutralization process is completed--has been treated. The hydrolysate is being stored on-site until a post-treatment plan can be implemented. The Army has been evaluating options for treating the hydrolysate since the mid-1990s. The John Warner National Defense Authorization Act for Fiscal Year 2007 mandated that GAO review the Army's Cost-Benefit Analysis of Off-Site Versus On-Site Treatment and Disposal of Newport Caustic Hydrolysate. Specifically, GAO (1) assessed the reasonableness of the Army's rationale to eliminate five of the eight technologies for treating Newport's hydrolysate; (2) determined what other options the Army considered, such as incineration; and (3) evaluated the adequacy of the cost comparison analysis presented for the three remaining technologies considered as alternatives to the Army's proposed plan. To meet the December 1, 2006, due date, GAO briefed or offered to brief your offices prior to that time. This report provides details of our findings and our conclusions and recommendations. GAO will also issue a separate letter on its assessment of the Army's cost-benefit analysis once DOD has completed its sensitivity review of the data in that letter.
The rationale that the Army used to eliminate five of the eight technologies for treating Newport hydrolysate appears reasonable. Based on our review of the supporting post-treatment estimate report and key National Research Council (NRC) reports referenced by the Army, there was evidence of significant difficulties associated with the five eliminated options that would make them less promising than the three others that were evaluated against the proposed DuPont option in the Army's cost comparisons. In addition to evaluating the eight alternatives discussed in its 2006 cost-benefit report, the Army previously evaluated off-site and on-site technical solutions for treating the hydrolysate, including incineration. The Army's evaluations concluded for various reasons that these alternatives would also be difficult to implement or not viable at this time. However, 5 of the 7 firms would use either of two methods: incineration or deep-well injection. Army officials believe that these two methods would garner higher levels of public concern than other methods. Moreover, while other on-site technologies were evaluated, those that were considered to be the most promising technologies in the 2001 and 2002 reports are very similar to the technologies identified in the Army's 2006 cost-benefit report. The use of any of the Army's four incinerators at its stockpile sites, while potentially technically feasible, has not been evaluated because it also has the potential for high levels of public concern, but could be evaluated if other options are no longer available. Based on our assessment of supporting documentation and analyses, we determined that the underlying cost estimates used in the Army's report were not reliable, and the impact of this on the Army's report finding that the DuPont plan had "significant cost savings" over the three considered alternatives is uncertain. Using OMB criteria and criteria approved by the cost estimating community, we determined that the estimates were unreliable because of (1) the quantity and magnitude of errors, (2) quality control weaknesses, (3) questionable or inadequate supporting source data and documentation, and (4) the undetermined sensitivity of key assumptions. Neither the Army nor the contractor has a system in place to perform cross-checks of the costs, underlying assumptions, or the technical parameters that went into the estimates. Moreover, we have determined that the results from the Army's programmatic risk analysis are unreliable because they were generated from the previously discussed unreliable cost estimates and because the Army attributed no risk to potential permitting, legal, or other challenges to the DuPont plan. Overall, we could not determine the cumulative effect of these problems on the outcome or results of the Army's analysis, in large part because we did not have confidence in much of the supporting data because of the problems that we have noted. Nevertheless, without reliable underlying cost estimates, the Army, the Congress, and the public cannot have confidence that the most cost-effective solution has been selected.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-240R, Chemical Demilitarization: Actions Needed to Improve the Reliability of the Army's Cost Comparison Analysis for Treatment and Disposal Options for Newport's VX Hydrolysate
This is the accessible text file for GAO report number GAO-07-240R
entitled 'Chemical Demilitarization: Actions Needed to Improve the
Reliability of the Army's Cost Comparison Analysis for Treatment and
Disposal Options for Newport's VX Hydrolysate' which was released on
January 26, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
January 26, 2007:
Congressional Committees:
Subject: Chemical Demilitarization: Actions Needed to Improve the
Reliability of the Army's Cost Comparison Analysis for Treatment and
Disposal Options for Newport's VX Hydrolysate:
The U.S. stockpile of 1,269 tons of VX nerve agent[Footnote 1] stored
at the Newport Chemical Depot (Newport), Indiana, is one of nine
stockpiles that the Department of Defense (DOD) must destroy in
response to congressional direction initially provided in 1985. In
addition, the stockpile must be destroyed to comply with the
requirements of the Chemical Weapons Convention,[Footnote 2] which the
United States became a party to in 1997. The stockpile at Newport is
the first U.S. stockpile containing VX that will be destroyed by using
neutralization--a process that mixes hot water and sodium hydroxide (a
caustic chemical) with VX to change the chemical composition to a less
toxic form. The resulting by-product is a liquid wastewater commonly
referred to as hydrolysate that consists mostly of water but also has a
caustic component and organic salts that need further treatment to meet
Chemical Weapons Convention requirements and to meet federal and state
environmental requirements for disposal. The Army, DOD's designated
executive agent, began neutralizing Newport's VX stockpile on-site in
May 2005 and, as of December 1, 2006, reports neutralizing about 34
percent of the stockpile.
None of the generated hydrolysate--expected to be about 2 million
gallons when the neutralization process is completed--has been treated.
The hydrolysate is being stored on-site until a post-treatment plan can
be implemented. The Army has been evaluating options for treating the
hydrolysate since the mid-1990s. Through these evaluations, on-site
supercritical water oxidation (SCWO) was initially selected as the
preferred option in 1999, but the preferred option was subsequently
changed in 2002 to using an off-site commercial treatment facility
because of concerns about the continued storage of the stockpile after
the September 11, 2001, terrorist attacks and numerous technical
challenges identified during one-tenth scale engineering testing. The
Army's plan for the treatment and disposal of the hydrolysate was to
transport it from Newport to an off-site commercial treatment and
disposal facility--the DuPont Secure Environmental Treatment Facility
(DuPont) in Deepwater, New Jersey--which would use a pretreatment
process that would include various chemical processes, including
oxidation followed by chemical precipitation to further break down the
hydrolysate. The remaining liquid effluent would be treated in the
facility's biodegradation-based waste treatment plant. This plan has
generated concerns about its safety and cost. However, on January 5,
2007, DuPont announced that it will not participate in the treatment of
Newport's hydrolysate, citing a "lengthy and arduous" approval process.
Army officials stated that the Army will explore all available on-site
and off-site options to treat Newport's hydrolysate.
The House Committee on Armed Services Report on the National Defense
Authorization Act for Fiscal Year 2006, H.R. Rep. No. 109-89, directed
the Secretary of the Army not to proceed with any action to transport
or relocate hydrolysate from Newport until health and environmental
concerns raised by the Environmental Protection Agency and the Centers
for Disease Control and Prevention were addressed in a manner that
would not result in substantial ecological or human heath risk. The
Centers for Disease Control and Prevention issued its report[Footnote
3] in July 2006. The report concluded, in part, that the Army/DuPont
proposal sufficiently addressed critical issues related to human
toxicity, transportation, and treatment of Newport's hydrolysate. The
committee report also required that the Secretary of the Army certify
to the congressional defense committees that sending the hydrolysate
off-site for treatment would result in significant cost and schedule
savings compared to on-site disposal of the hydrolysate before
transport. The report further required that the Secretary of the Army
conduct and provide the congressional defense committees a detailed
cost-benefit analysis to include an analysis comparing the proposed off-
site treatment option with eight on-site options, which are discussed
in detail in enclosure I.
In response to the latter requirement, the Army published its cost-
benefit report[Footnote 4] in April 2006, which concluded that only
chemical oxidation, SCWO, and wet-air oxidation technologies were
feasible for treating Newport's hydrolysate. In the cost-effectiveness
analysis contained in the report, the Army determined that the cost of
off-site treatment of the hydrolysate at DuPont would be from $146
million (without program risk) to $347 million (including program risk)
less expensive than the on-site options. The Army also concluded that
the off-site treatment option would allow the disposal of the
hydrolysate to be accomplished in the shortest amount of time and would
minimize the amount of time that Newport's hydrolysate must be stored
at Newport.
The John Warner National Defense Authorization Act for Fiscal Year
2007[Footnote 5] mandated that we review the Army's Cost-Benefit
Analysis of Off-Site Versus On-Site Treatment and Disposal of Newport
Caustic Hydrolysate. Specifically, we (1) assessed the reasonableness
of the Army's rationale to eliminate five of the eight technologies for
treating Newport's hydrolysate; (2) determined what other options the
Army considered, such as incineration; and (3) evaluated the adequacy
of the cost comparison analysis presented for the three remaining
technologies considered as alternatives to the Army's proposed plan. To
meet the December 1, 2006, due date, we briefed or offered to brief
your offices prior to that time. This report provides details of our
findings and our conclusions and recommendations. We will also issue a
separate letter on our assessment of the Army's cost-benefit analysis
once DOD has completed its sensitivity review of the data in that
letter.
To meet our objectives, we reviewed documentation and interviewed
officials in the Office of the Under Secretary of Defense (Acquisition,
Technology, and Logistics); Office of the Deputy Assistant Secretary of
the Army (Elimination of Chemical Weapons); U.S. Army Chemical
Materials Agency (CMA), Aberdeen, Maryland; the Newport Chemical Depot,
Newport, Indiana; DuPont's Secure Environmental Treatment Facility,
Deepwater, New Jersey; Parsons Infrastructure and Technology Group,
Inc. (Parsons); and Shaw Environmental Group, Inc. (Shaw)--the
contractor that assisted the Army in performing the analysis for the
Army's report. To assess the Army's methods of evaluating various
options for treating hydrolysate, we relied on a review of CMA's
Newport On-Site Hydrolysate Post Treatment Estimate report (post-
treatment estimate report), which was prepared by Shaw,[Footnote 6]
National Research Council (NRC) reports, and other referenced documents
and supporting documentation. We did not conduct an independent
evaluation of these technologies. To assess the adequacy of the Army's
cost comparison analysis, we compared the Army's methods and approaches
with the guidance contained in Office of Management and Budget (OMB)
and DOD instructions and Cost Estimating Standards and Practices
developed by the Society of Cost Estimating and Analysis, and assessed
the reliability of the cost estimates.[Footnote 7] We reviewed and
evaluated the cost analyses the Army used in preparing its cost-benefit
report and interviewed Army and contractor officials regarding the data
and assumptions they used in preparing their analyses. To determine the
accuracy of underlying data, we independently calculated values based
on provided assumptions to compare against values contained in
supporting spreadsheets. We also compared values from the supporting
spreadsheets to summary data provided in the supporting post-treatment
estimate report prepared for CMA by Shaw. Also, we made use of
information that we obtained during our ongoing review of DOD's
Chemical Demilitarization Program to assess the reasonableness of
certain assumptions. We conducted our review from August 2006 through
November 2006 in accordance with generally accepted government auditing
standards.
Results in Brief:
The rationale that the Army used to eliminate five of the eight
technologies for treating Newport hydrolysate appears reasonable. Based
on our review of the supporting post-treatment estimate report and key
NRC reports referenced by the Army, there was evidence of significant
difficulties associated with the five eliminated options that would
make them less promising than the three others that were evaluated
against the proposed DuPont option in the Army's cost comparisons. For
example, the Army report's finding that one alternative technology
would not be well suited to treat large volumes of wastes with high
concentrations of water, such as Newport's hydrolysate, was consistent
with determinations made in the post-treatment estimate report and was
supported by findings in a 2001 NRC report. Also, for another
alternative, the Army determined that the vendor with the rights to the
technology was no longer in business and further development of this
alternative would call for a company to acquire the rights and start
development work, creating a large cost and schedule risk.
In addition to evaluating the eight alternatives discussed in its 2006
cost-benefit report, the Army previously evaluated off-site and on-site
technical solutions for treating the hydrolysate, including
incineration. The Army's evaluations concluded for various reasons that
these alternatives would also be difficult to implement or not viable
at this time. For example, of the more than 100 commercial disposal
firms that were initially considered as candidates to treat Newport's
hydrolysate off-site in 2002, only 7 firms (not including DuPont) that
have been determined to be qualified by the Army's contractor have
provided price information through either the Army's 2002 request for
proposals or a subsequent market survey. However, 5 of the 7 firms
would use either of two methods: incineration or deep-well injection.
Army officials believe that these two methods would garner higher
levels of public concern than other methods. Moreover, while other on-
site technologies were evaluated, those that were considered to be the
most promising technologies in the 2001 and 2002 reports are very
similar to the technologies identified in the Army's 2006 cost-benefit
report. The use of any of the Army's four incinerators at its stockpile
sites, while potentially technically feasible, has not been evaluated
because it also has the potential for high levels of public concern,
but could be evaluated if other options are no longer available.
Based on our assessment of supporting documentation and analyses, we
determined that the underlying cost estimates used in the Army's report
were not reliable, and the impact of this on the Army's report finding
that the DuPont plan had "significant cost savings" over the three
considered alternatives is uncertain. Using OMB criteria and criteria
approved by the cost estimating community,[Footnote 8] we determined
that the estimates were unreliable because of (1) the quantity and
magnitude of errors, (2) quality control weaknesses, (3) questionable
or inadequate supporting source data and documentation, and (4) the
undetermined sensitivity of key assumptions. Neither the Army nor the
contractor has a system in place to perform cross-checks of the costs,
underlying assumptions, or the technical parameters that went into the
estimates. Moreover, we have determined that the results from the
Army's programmatic risk analysis are unreliable because they were
generated from the previously discussed unreliable cost estimates and
because the Army attributed no risk to potential permitting, legal, or
other challenges to the DuPont plan. It was unclear to us whether the
programmatic risks of other alternatives were understated or
overstated. Overall, we could not determine the cumulative effect of
these problems on the outcome or results of the Army's analysis, in
large part because we did not have confidence in much of the supporting
data because of the problems that we have noted. Nevertheless, without
reliable underlying cost estimates, the Army, the Congress, and the
public cannot have confidence that the most cost-effective solution has
been selected. We are making recommendations that the Army conduct its
cost-benefit analysis again using best practices so that its data and
conclusions are comprehensive, traceable, accurate, and credible;
correct any technical and mathematical errors in the cost estimate;
establish quality control and independent review processes that check
data sources, calculations, and assumptions; and perform a sensitivity
analysis of key assumptions.
In written comments on a draft of this report, DOD concurred with our
findings and recommendations, and stated that the Army will be
preparing a new cost-benefit analysis based on a revised cost estimate,
which will be independently reviewed and verified. DOD comments are
discussed in more detail at the end of this correspondence and are
reproduced in full in enclosure II. DOD also provided technical
comments, which have been incorporated where appropriate.
Background:
In 1994, the Army, in response to continued public and congressional
concerns about the use of incineration as a destruction method,
established the Alternative Technologies and Approaches Project under
the Program Manager for Chemical Demilitarization, which became CMA in
2003. The project was tasked with investigating alternatives to
incineration for the stockpiles that were located at Aberdeen Proving
Ground (Aberdeen), Maryland, and Newport. These two sites were unique
in that they consisted solely of chemical agent stored in bulk
containers, without explosives or other munitions components.
In February 1999, the Army announced that it would establish a pilot
program to destroy Newport's stockpile of VX. The treatment and
disposal method chosen to be pilot tested was neutralization followed
by SCWO. In making its decision, the Army considered evaluations made
by the NRC and independent Army reviews of alternatives to incineration
and for treating hydrolysate. Another key factor in the Army's decision
was a recommendation made by the Indiana Citizens Advisory Commission
that the Army consider technologies other than incineration.
In 2002, in response to heightened concerns about the storage of
chemical weapons after the terrorist attacks of September 11, 2001, the
Army adopted "an accelerated approach" at both its Newport and Aberdeen
stockpile locations in order to eliminate the stockpiles faster. A key
change resulting from this accelerated approach is that the Army would
not treat the hydrolysate on-site, but would transport it off-site to
commercial facilities that had the necessary environmental permits to
treat and dispose of the hydrolysate. In the case of Newport, Parsons,
the government's site contractor, awarded a contract to Perma-Fix of
Dayton, Inc., Dayton, Ohio, through a 2002 request for proposals, to
demonstrate that it could treat the hydrolysate. However, this plan
generated considerable public concern, and the contract was
subsequently terminated when a discharge permit could not be obtained.
The Army and its contractor at Newport then began pursuing efforts to
transport the hydrolysate to DuPont, which also responded to the 2002
request for proposals.
Methodology Used to Develop the Army's Cost-Benefit Analysis:
In response to the committee report,[Footnote 9] the Army tasked Shaw
with developing technical schedule and cost information comparing the
eight on-site technologies cited in the committee report to the Army's
proposed off-site transportation of the hydrolysate to DuPont. A post-
treatment estimate report was prepared to document the methodology,
assumptions, and findings used in the analysis. In conducting its
analysis, each of the eight technologies was initially evaluated to
determine its applicability to process Newport hydrolysate. The Army
determined that the eight cited treatment and disposal methods were the
eight methods evaluated by the NRC in an assessment it prepared for the
Project Manager for the Non-Stockpile Chemical Materiel
Project[Footnote 10] in 2001. The analysis used the evaluation criteria
in the NRC report to assess benefits and risks of each method, updating
information when necessary to reflect events subsequent to the report's
publishing. The post-treatment estimate report also noted that the Army
has been monitoring for years the development of emerging technologies
for the potential application to chemical demilitarization. Schedules
and cost estimates were only developed for technologies that were
determined to be applicable to the volume and characteristics of the
Newport hydrolysate.
Army officials stated that they selected Shaw because of its experience
with various aspects of the DOD's Chemical Demilitarization Program and
its familiarity with treatment methods, which provides it with the
expertise to make reasoned judgments about the treatment methods
contained in the Army's cost-benefit report. Shaw has supported CMA and
its predecessor for more than 15 years. For example, Shaw supported the
Non-Stockpile Chemical Materiel Project by participating in the
identification, evaluation, and testing of methods for treating waste
streams from the neutralization of recovered chemical materials and
binary chemical agents. According to the Army, Shaw evaluated over 140
technologies, including all eight cited in the committee report.
Moreover, Shaw was responsible for a program that monitors the
development of new technologies for potential application to CMA's
mission. In addition, Shaw participated in the study and testing of
various candidate processes for post-treatment of Newport's
hydrolysate, including SCWO.
The Rationale Used for Eliminating Technologies from Further
Consideration Appears Reasonable:
Although the Army's cost-benefit report did not provide specific
details on the process it used to make its determination to eliminate
five of the eight congressionally specified treatment technologies from
further consideration, the Army's rationale for eliminating these
methods appears reasonable based on our review of the findings in the
post-treatment estimate report prepared by Shaw. The report's
evaluations were supported by past NRC reports, past program studies,
and experience that Shaw has gained through its work with the Non-
Stockpile Chemical Materiel Project and its role in evaluating emerging
technologies.
Army Eliminated Five Treatment Methods It Determined to Be Unsuitable
for Newport Hydrolysate:
The Army's cost-benefit report stated that based on a technical review
conducted by Shaw's professional engineers with extensive experience in
these treatment methods, five treatment methods specifically referenced
in H.R. Rep. No. 109-89 were not viable because the methods were not
well suited for the known properties and volumes of Newport
hydrolysate. The five methods eliminated from further study were (1)
electrochemical oxidation, (2) solvated-electron technology, (3) gas-
phase chemical reduction, (4) plasma-arc technology, and (5) stand-
alone biodegradation. Table 1 lists the five eliminated treatment
methods and the factors the Army report cited as leading to their
elimination.
Table 1: Rationale for the Elimination of Five Technologies from the
Army's Cost-Benefit Report for Hydrolysate Treatment:
Technology: Electrochemical oxidation;
Factors leading to elimination:
* Not appropriate for aqueous wastes;
* Concern about scale-up issues and risks;
* Generates large volumes of waste streams needing additional
treatment.
Technology: Solvated-electron technology;
Factors leading to elimination:
* Not appropriate for aqueous wastes;
* Generates hydrogen;
* Uses difficult-to-handle reagents.
Technology: Gas-phase chemical reduction;
Factors leading to elimination:
* Company no longer exists;
* Generates high volumes of gaseous waste;
* Hydrogen reagent considered a safety risk.
Technology: Plasma-arc technology;
Factors leading to elimination:
* Not appropriate for large quantities of aqueous wastes;
* Considered similar to incineration;
* Limited experience with both hazardous and aqueous solutions.
Technology: Stand-alone biodegradation;
Factors leading to elimination:
* Primary reaction products in Newport caustic hydrolysate are not
amenable to direct treatment by biodegradation;
* Not efficient for on-site waste volumes; cannot obtain economies of
scale available at commercial large-scale treatment, storage, and
disposal facilities.
Source: U.S. Army.
[End of table]
Army's Eliminations Appear Reasonable Based on Our Review of Supporting
Documents:
Our review of the Army's rationale for dismissing five of the eight
alternative technologies found that the findings contained in the
Army's report appear to be reasonable and are supported by
documentation from the post-treatment estimate report prepared for CMA
by Shaw and NRC reports. The evaluation of the benefits and risks of
each technology was largely based on criteria developed by the NRC in
its 2001 report.[Footnote 11] The NRC cited four areas as "top
priority" criteria: relative process safety (low risk), technical
effectiveness, permit status, and pollution prevention. Additionally,
another six categories were designated "important" criteria:
robustness, cost, practical operability, continuity, space efficiency,
and materials efficiency. When necessary, the findings of the NRC
neutralent waste report were updated within these criteria based on
more recent technological developments, experience Shaw has gained
working with the Non-Stockpile Chemical Materiel Project, and Shaw's
role in evaluating emerging technologies. Our review of the post-
treatment estimate report and key NRC reports referenced by the Army
provided evidence of likely significant difficulties associated with
the five eliminated options that would make them less promising than
the three options that were evaluated against the proposed DuPont
option in the Army's cost comparisons. Evidence used by the Army to
support its rationale for eliminating each treatment method is
discussed below.
Electrochemical Oxidation:
* The Army report's finding that electrochemical oxidation would
generate large volumes of waste is supported by findings in the NRC
neutralent wastes report. In its report, the NRC noted that
electrochemical oxidation generates large amounts of gaseous effluents,
particularly chlorine gas, which needs to be scrubbed. The report also
noted that those effluents would be corrosive and could cause operating
problems. Shaw's evaluation determined that the amount of water present
in Newport's hydrolysate would be too large for the electrochemical
oxidation technology to process, requiring either the hydrolysate to be
concentrated or the electrochemical system to be redesigned. The NRC
neutralent wastes report also noted that treating large quantities of
water would be an issue for this treatment method.
* The finding that an electrochemical oxidation system would require a
significant scale-up is also supported by findings in the NRC
neutralent wastes report. The report also stated that the one existing
CerOx facility (the type of electrochemical oxidation proposed for use
by the Army) could process only one 35-gallon barrel at a time. In
Shaw's evaluation, it noted concerns about whether the manufacturer
could easily scale up production for the 200 electrochemical cells
necessary to operate at Newport, as their systems were only accustomed
to handling laboratory-scale amounts of waste.
Solvated-Electron Technology:
* The Army report's finding that the solvated-electron technology is
not appropriate for Newport hydrolysate is supported by the NRC
neutralent wastes report findings as well. The NRC determined that the
solvated- electron technology process' efficiency is poor when treating
aqueous waste streams, and its advantages may be outweighed by the
difficulty of handling its reagents, which are toxic and have been
known to cause fires. The NRC also was concerned that the solvated-
electron process was less mature than some of the other treatment
technologies. Shaw determined that solvated-electron technology was
evaluated by the Assembled Chemical Weapons Alternatives[Footnote 12]
program but that it could not successfully complete demonstration
testing.
Gas-Phase Chemical Reduction:
* The Army report's finding that the gas-phase chemical reduction
technology is not appropriate for Newport hydrolysate is supported by
Shaw's findings, which noted that the vendor who had the rights to this
technology, ELI Eco Logic, went out of business in 2004. Shaw noted
that further development of gas-phase chemical reduction as an option
would "require finding a company to acquire the rights and start
development work" and that process "would be a large risk to cost and
schedule."
* The NRC's neutralent wastes report notes that gas-phase chemical
reduction is a complex process that requires the management of hot
hydrogen gas, which presents unique safety concerns. The NRC report
cited the need to manage gases both in the reactor and as effluents and
the potential for the buildup of carbon soot. The report also stated
that no commercial-scale reactor of this type has received a permit to
operate in the United States, which could lead to delays.
Plasma-Arc Technology:
* The NRC neutralent wastes report notes concerns about the prospects
for plasma-arc technology to get a permit since it has not operated in
the United States and regulators may consider it to be incineration.
Also, the NRC noted that tests of the technology conducted by the
Army's Assembled Chemical Weapons Alternatives program on VX
hydrolysate generated products of environmental concern. The NRC report
also stated that this process is less efficient with wastes that
contain large amounts of water.
* The post-treatment estimate report states that regulatory hurdles
would need to be overcome and then technology development would need to
be accomplished.
Stand-alone Biodegradation:
* The NRC has repeatedly noted its concerns related to the ability of
stand-alone biodegradation to treat hydrolysate from VX. For example,
in its 1996 Review and Evaluation of Alternative Chemical Disposal
Technologies[Footnote 13], its 2000 Integrated Design of Alternative
Technologies for Bulk-Only Chemical Agent Disposal Facilities[Footnote
14], and its neutralent wastes reports, the NRC noted that the primary
reaction products of VX hydrolysate are not readily amenable to direct
treatment by biodegradation, since they cannot be easily broken down by
the microorganisms used in this process. Additionally, Shaw cited an
Assembled Chemical Weapons Assessment evaluation that found
biodegradation to be "inadequate for complete destruction" of VX
hydrolysate.
* The Army report's finding that biodegradation could not achieve the
economies of scale needed to make biodegradation efficient is supported
by the findings of the NRC. In its 1996 study, the NRC found that
because hydrolysate cannot serve as the primary substrate for the
microorganisms in this process, substantial quantities of co-substrate
need to be added to co-feed the process, making it inefficient.
* Chemical oxidation, followed by chemical precipitation in conjunction
with biodegradation, is the process proposed for use in both the off-
site DuPont option and the on-site chemical oxidation option.
The Army Has Evaluated Other Technical Solutions for Treating Newport's
Hydrolysate:
Although the Army did not discuss them in its cost-benefit report, it
has evaluated other technical solutions for treating Newport's
hydrolysate since its 1999 decision to use SCWO. These solutions
include both off-site commercial treatment facilities and treatment
technologies that would be used on-site. In general, the evaluations of
off-site options have determined that there are only a few commercial
treatment facilities that are qualified and interested in treating
Newport's hydrolysate, but addressing public comments and concerns
could be challenging. Evaluations of on-site options have determined
that the most promising options are similar to those included in the
Army's 2006 report, but concerns were raised about development costs
and operational risks. Other solutions, such as using one of the Army's
four incineration facilities, may be technically feasible but not
viable at this time.
Army's Evaluation of Off-Site Commercial Treatment Facilities Found Few
That Are Potentially Qualified and Interested:
While there may be numerous facilities that could treat the Newport
hydrolysate, only a small number have actually responded to requests
for proposals. The Army began evaluating commercial treatment
facilities that use various treatment methods, including those using
incineration, biodegradation, and deep-well injection options
subsequent to a 2001 NRC report that expressed concerns about the
reliability of SCWO reactors during engineering tests. An earlier NRC
report in 2000 recommended that the Army evaluate the potential off-
site treatment of Newport hydrolysate both for potential costs and
schedule benefits as well as a contingency in the case of start-up
problems implementing SCWO.
The Army adopted an accelerated approach in 2002 that changed the
planned treatment method for hydrolysate from on-site to an off-site
commercial treatment facility. The Army's contractor at Newport--
Parsons--conducted industry surveys to identify facilities that could
transport, treat, and dispose of Newport's hydrolysate. Parsons
conducted a nationwide survey and identified over 100 commercial
treatment and disposal facilities in the United States as capable of
handling hazardous waste. However, after considering the facilities'
technology, environmental permits, safety and environmental records,
and outreach initiatives, Parsons determined that only 45 of the
facilities should be considered to determine their qualifications to
treat and dispose of the Newport hydrolysate. These 45 facilities were
sent a qualification survey by Parsons; however, only 14 firms
completed and returned the survey. After a review of the responses to
determine if they met minimum specified requirements, Parsons provided
requests for proposals to 10 of these facilities. Ultimately, 4
facilities responded: 2 incineration-based facilities and 2
biodegradation treatment-based facilities (Perma-Fix and DuPont). None
of the facilities that use deep-well injection responded to Parson's
request. The range of proposed prices varied significantly with a
fourfold difference in price from the least expensive to the most
expensive of the four facilities.
According to the Army, during the evaluation process for the four
proposals, the two commercial incineration facilities were eliminated:
one withdrew its proposal and the other was deemed to be too high of a
risk because of concerns about public opposition. Parsons, in its
capacity as the government's contractor, awarded Perma-Fix a contract
in December 2002 to demonstrate its ability to successfully treat the
hydrolysate. However, before any hydrolysate could be shipped, Parsons
terminated the contract for convenience of the government. This
cancellation was caused by the determination that an environmental
permit would not be issued to Perma-Fix by the local county government.
As part of this continuing procurement in 2005, Parsons conducted a
market survey to establish an updated range of hazardous waste
treatment and disposal prices at the commercial treatment facilities.
The 10 commercial facilities that were surveyed were the same
facilities that were provided proposal requests in 2002. In this
instance, 7 of the 10 commercial facilities provided price data for
processing generic hazardous waste material.[Footnote 15] Of the 7
facilities that provided price data, 2 use incineration, 2 use deep-
well injection, 2 use biodegradation, and 1 uses both biodegradation
and deep-well injection. Program officials stated that the prices for
these firms represented a broad range of pricing for hazardous waste
treatment based on a range of treatment technologies, locations, and
marketplace factors, such as financial risks and regulatory and
environmental liabilities. The price for treating and disposing of any
waste depends on the facilities' capabilities, regulatory restrictions,
and permit requirements.
Past Evaluations of On-Site Technologies Yielded Similar Results
regarding the Most Promising Technologies:
Although the Army selected SCWO as its planned on-site treatment and
disposal method for Newport's hydrolysate in 1999, the Army and its
contractors conducted several more evaluations in 2001 and 2002 to
consider other on-site approaches because of Army and NRC concerns
about the reliability of SCWO. These evaluations included technologies
other than those that were considered in the Army's recent cost-benefit
report, but the technologies deemed most promising were similar to the
technologies that survived elimination in the Army's 2006 report.
Earlier study findings included the following.
* A 2001 Parsons report identified and assessed 8 potential on-site
technologies that were capable of processing the Newport hydrolysate.
Initially, more than 100 technologies were identified in literature and
database searches. After screening based on several criteria, including
process efficiency, technology maturity, and the extent that they were
considered low pressure or temperature, 8 technologies met the
criteria: two types of SCWO, electrochemical oxidation, wet-air
oxidation, two types of chemical oxidation, ozone (with and without
peroxide), and bleach treatment with biodegradation. Two additional
technologies were added--gas phase chemical reduction and plasma arc--
although they did not meet the criteria for being low-temperature
processes. This report found that the two types of SCWO, wet-air
oxidation, and chemical oxidation, were the most promising technologies
for further consideration; these are the same three technologies that
were identified in the Army's cost-benefit report.
* A 2002 Parsons report prepared for the Army compared various on-site
and off-site disposal options for consideration as a potential backup
plan for treating Newport hydrolysate. In its evaluation, the
contractor determined that the two on-site treatment options--SCWO and
the pretreatment/biological treatment option--would rate more favorably
for public acceptance, but not as favorably for cost and schedule,
primarily because of development and testing costs and operational
risks. The evaluation also determined that pretreating the hydrolysate
on-site before transporting it off-site would offer no advantage. The
evaluation concluded that off-site options had more favorable ratings
because of advantages in cost, schedule, and environmental compliance,
but would likely be at higher risk for lack of public acceptance.
Figure 1 compares the 2002 evaluation of various on-site and off-site
treatment options by cost, schedule, public acceptance, and
environmental compliance.
Figure 1: Comparison of 2002 Evaluation of the On-site, Off-site, and
Combination Treatment Options for Newport's Hydrolysate:
[See PDF for Image]
Source: U.S. Army.
[End of Figure]
Using Existing Army Incineration Sites Is Not Considered a Viable
Option at This Time:
CMA officials acknowledged that using one of the four operating
chemical agent disposal facility incinerators to process the
hydrolysate is considered a technically feasible option; however, CMA
has not formally assessed all its advantages and disadvantages because
CMA officials do not believe incineration to be viable at this time.
These officials told us that from a technical standpoint, incineration
could be used to dispose of Newport's hydrolysate, and as discussed
above, the Army has considered commercial facilities that use
incineration. However, it is not an ideal solution for treating
hydrolysate, which is primarily water (85 percent), thus leading to
greater energy consumption. These officials also stated that because of
the opposition to incineration of hydrolysate, both locally in Indiana
and nationally, the Army has committed to pursuing nonincineration
options first. However, should there be no permitted commercial
treatment facility reasonably available, the Army would once again
evaluate the viability/acceptability of using incineration for
disposing of Newport's hydrolysate, including evaluating the potential
legal and regulatory barriers.
Army's Cost Estimates and Programmatic Risk Analysis Were Not Reliable,
and Impact on Results Is Uncertain:
The Army's report found that the DuPont plan was significantly more
cost-effective than the three considered on-site alternatives, but
based on our assessment of supporting documentation and analyses, we
determined that the underlying cost estimates used in the Army's report
were not adequate or reliable, making the cost-effectiveness
determination among options uncertain. Using OMB criteria and criteria
approved by the cost estimating community to assess the methodology,
key assumptions, and data used to develop cost estimates in the Army's
cost-benefit report, we determined that the estimates were unreliable
for reasons related to (1) the quantity and magnitude of errors, (2)
quality control weaknesses, (3) questionable or inadequate supporting
source data and documentation, and (4) the undetermined sensitivity of
key assumptions. Further, neither the Army nor the contractor had a
system of cross-checking in place to verify computations or to
substantiate the basis for some assumptions. Moreover, we determined
that the results from the Army's programmatic risk analysis are
unreliable because they were generated from the previously discussed
unreliable cost estimates and because the Army attributed no risk to
potential permitting, legal, or other challenges to the DuPont plan. It
was unclear to us whether the programmatic risks of other alternatives
were understated or overstated. Overall, we could not determine the
cumulative effect of these problems on the outcome or results of the
Army's analysis, in large part because we did not have confidence in
much of the supporting data because of the stated problems and the
limited time available to further test these data.
Cost Estimating Community Has Best Practices Criteria for Reliability:
Guidance provided in OMB Circular A-94 and best practices established
by professional cost analysts, such as those identified by the Society
of Cost Estimating and Analysis, have identified characteristics of a
high-quality, reliable cost estimate. These characteristics include the
following.
* Comprehensive. The estimate should be at a level of detail
appropriate to ensure that cost elements are neither omitted nor double
counted. All cost-influencing ground rules and assumptions are detailed
in the documentation of the cost estimate.
* Traceable. The estimate is thoroughly documented, including source
data and significance, clearly detailed calculations and results, and
explanations for why a particular method or reference was chosen. Data
can be traced back to the source documentation.
* Accurate. The estimate should be unbiased, not overly conservative or
overly optimistic, and based on an assessment of most likely costs.
Few, if any, mathematical mistakes are present and are minor in nature.
* Credible. Any limitations of the analysis because of uncertainty or
biases surrounding data or assumptions should be discussed. Major
assumptions should be varied and other outcomes recomputed to determine
how sensitive outcomes are to changes in the assumptions. In addition,
the results of an estimate should be cross-checked with an independent
cost estimate and a level of risk associated with the estimate should
be identified.
Engineering Buildup Approach Was Used to Develop the Technical Cost
Estimate:
In developing the cost comparisons that were cited in the post-
treatment estimate report, an engineering buildup approach was used,
which can be an appropriate methodology for construction projects. The
approach was based on conceptual design data for manpower estimates,
facility sizing, construction, equipment costs, and throughput
estimates for the on-site options. Cost estimates for categories, such
as utilities, processing materials, and storage costs, for the
accumulating hydrolysate were also developed. For the DuPont off-site
option, DuPont's past estimate was updated based on an assessment of
the impacts of program changes since the estimate was originally
provided in 2002. For all options, costs were grouped by the following
major categories: (1) project services; (2) engineering, design, and
permitting; (3) process equipment and systems; (4) facilities
construction; (5) systemization; (6) operations and pilot test; (7)
hydrolysate storage; and (8) closure. A contingency factor was added to
each estimate to account for estimating, commercial, and technical
risks.
Cost Estimates Were Unreliable Because of Inadequate Supporting
Documentation:
and Numerous Computational Errors:
The technical cost analysis that the Army used in estimating the costs
of the proposed off-site option and the three on-site options contained
in its cost-benefit report did not follow all applicable guidance from
OMB and best practices for a cost-effectiveness analysis. Specifically,
the quality of some of the underlying estimates in the technical report
was affected because the supporting analysis was not comprehensive nor
traceable in that data sources were frequently not provided throughout
the analysis nor was it accurate because of the numerous computational
errors. Neither the Army nor the contractor performed independent cross-
checks of the costs or the technical parameters that went into the
estimates.
Our analysis revealed numerous instances where the data were not
comprehensive or traceable. We determined that the documentation
provided was not detailed enough to provide an accurate assessment of
the quality of each alternative's cost estimate. For example, neither
the technical report nor the supporting documentation referenced the
source for numerous data inputs, such as the sources for equipment
installation labor hour per unit parameters or chemical reagent unit
costs. Additionally, the basis of estimate documented for some data
inputs were found to be inadequate for assessing estimate credibility,
such as staffing estimates, labor rates, and other direct costs.
Our analysis also revealed many computation errors that affect the
accuracy of the cost estimates, including an incorrect rate being
applied to all labor categories for 5 years, leading to costs for one
category of one option being overstated by about $34 million and
another option's solid waste disposal costs being understated by
approximately $3.5 million. CMA officials acknowledged that there was
no system in place to independently verify the accuracy of the data. In
total, the errors affected all options and led to both over-and
underestimating of costs. However, to the extent that we could correct
identified inaccuracies, our recalculations just for computation errors
did not result in a significant variance from the Army's analysis. The
estimated costs would fall in the same order that the Army had
originally computed, although the net difference in costs between the
DuPont option and each on-site option was reduced. Table 2 shows the
relative comparison of the corrected costs versus the reported cost
estimates.
Table 2: Comparison of Reported and Corrected Cost Estimates for the
Three On-Site Options Relative to DuPont's Cost Estimates (without
Programmatic Risk):
Treatment option: Chemical oxidation;
Reported cost estimates relative to DuPont's cost estimate:
$145,900,000;
Corrected cost estimates relative to DuPont's corrected cost estimate:
$130,625,000;
Change in relative costs of reported and corrected cost estimates:
($15,275,000).
Treatment option: Wet-air oxidation;
Reported cost estimates relative to DuPont's cost estimate:
$148,900,000;
Corrected cost estimates relative to DuPont's corrected cost estimate:
$133,525,000;
Change in relative costs of reported and corrected cost estimates:
($15,375,000).
Treatment option: Supercritical water oxidation;
Reported cost estimates relative to DuPont's cost estimate:
$200,500,000;
Corrected cost estimates relative to DuPont's corrected cost estimate:
$178,125,000;
Change in relative costs of reported and corrected cost estimates:
($22,375,000).
Source: GAO analysis of U.S. Army data.
Note: In the Army's cost-benefit report, to protect proprietary
information generated by DuPont, the estimated costs for implementation
of each on-site option was reported relative to the estimated cost for
the DuPont option (considered the base cost) without disclosing the
actual value of the DuPont cost estimate.
[End of table]
Our estimate corrects only for obvious mathematical and spreadsheet
errors and does not account for unsubstantiated input parameters or
parameters used in the spreadsheets that are in conflict with the
documentation. For example, there were differences between consumption
levels used in the spreadsheet model versus the level documented in the
post-treatment estimate report. These discrepancies could potentially
translate to an underestimate of nearly $15 million. This discrepancy
and others like it are not reflected in our estimate because it is not
obvious which consumption level is correct. To resolve discrepancies
like these, an independent technical assessment would need to be
conducted to verify the validity of inputs and assumptions used to
prepare the estimates.
Uncertainty of Cost Estimate Is Not Adequately Addressed:
The Army's cost analysis does not sufficiently address the uncertainty
of its cost estimates, which affects the credibility of its
conclusions. First, the technical report does note that the estimate is
a rough order of magnitude estimate for cost and schedule that can be
used to provide a basis for evaluating probable life cycle costs. CMA
officials stated that the Association for the Advancement of Cost
Engineering International's Cost Engineers' Notebook was used to
develop the order of magnitude estimate. According to the technical
report, the cost estimates are based on conceptual design data, and
because of the breadth of technological alternatives considered,
relative unique processes, and lack of processing data for Newport
hydrolysate, the technical cost estimate should be considered in the
range of plus 30 percent and minus 15 percent. However, neither the
Army's cost- benefit report nor the supporting post-treatment estimate
report provided estimates that reflect these ranges of outcomes. For
example, applying the worst case to DuPont (plus 30 percent) and the
best case (minus 15 percent) to one or more of the on-site options
could significantly reduce the cost difference, although DuPont would
still be more cost-effective. Second, the Army did not perform a
sensitivity analysis to assess how variations in certain key
assumptions could affect its cost estimates although there can be
imprecision in both underlying data and modeling assumptions. For
example, the cost estimates were based on the DuPont off-site option
having about 2.5 times greater throughput capacity than each of the on-
site treatment technologies. This assumption leads to a greater
disparity between the on-site and off-site operation costs since the
operation period, and its associated costs, would be longer for the on-
site options. However, it is possible that the actual throughput could
increase or decrease based on design and operational considerations.
Because such uncertainty is basic to many analyses, its effects should
be analyzed and reported. There was no analysis done to determine the
effects of varying this assumption on the cost estimate. Third, the
cost estimates also do not address the uncertainty associated with
impacts that environmental permitting activities, actions of public or
government agencies, or public opinion could have on program execution.
This uncertainty is particularly relevant since these impacts have
greatly affected the program in the past, delaying the Army's proposed
plan for over 2 years.
Army's Programmatic Risk Analysis Was Not Reliable Because It Used
Unreliable Technical Cost Estimates and Understated Risks:
We determined that the Army's programmatic risk analysis that added
additional costs to each option is also unreliable because the analysis
was generated from the previously discussed unreliable technical cost
estimates and because the Army attributed no risk to potential
permitting, legal, or other challenges to the DuPont plan. The
programmatic risk analysis is used to account for unknown risks that
could affect the cost or schedule of given options. This analysis uses
a statistical distribution model that typically extends the schedule
durations by a scaled amount based on the level of risk (none, low,
medium, and high) assigned to each of three phases: design/
construction, operations, and closure.
Based on this programmatic risk analysis, the Army added additional
costs to each option. The proposed DuPont plan had the least additional
costs added during the programmatic analysis, while the net additional
cost (total additional costs minus additional costs for DuPont) for the
three options ranged from $84 million to $146 million.
Conducting a programmatic risk analysis is an acceptable method for
applying unknown risk; however, it depends heavily on the judgment used
when assigning risks. For example, Army officials assigned no
programmatic risk to DuPont's design/construction phase because the
process would use commercially available facilities and personnel.
Another reason cited for not assigning risk was that the technical cost
estimates contained sufficient known risk for the limited scope and
design that would be needed at DuPont. However, this risk determination
does not take into account the potential permitting, legal, or other
challenges that may arise, which could delay construction, transport of
the hydrolysate to New Jersey, or start of operations. Based on the
history of delays associated with implementing the proposed plan, it
would be prudent to account for these risks. It was unclear to us
whether the programmatic risks of other alternatives were understated
or overstated.
Conclusions:
The Army has been pursuing the off-site treatment of Newport's
hydrolysate at a commercial treatment, storage, and disposal facility
since it adopted an accelerated disposal approach in 2002. One of the
reasons that has been frequently cited for adopting this approach is
that it would provide substantial cost savings over designing,
constructing, and operating an on-site treatment and disposal method.
However, from the time that the accelerated approach was adopted, the
Army faced resistance on many fronts because of skepticism concerning
advantages attributable to the off-site treatment option. If the Army
is to be successful in garnering support for its plan, then it is
imperative that the Army use a transparent process to develop cost
estimates that are comprehensive, traceable, accurate, and credible.
Without reliable underlying cost estimates, the Army, the Congress, and
the public cannot have confidence that the most cost-effective solution
for the treatment and disposal of Newport's hydrolysate has been
selected.
Recommendations for Executive Action:
To ensure confidence in the reliability of the underlying cost
estimates for the Army's decision to send hydrolysate from the Newport
Chemical Depot, Indiana, off-site for treatment, which indicate
significant cost and schedule savings compared to on-site disposal of
the hydrolysate, we recommend that the Secretary of Defense direct the
Secretary of the Army to take the following four actions.
* Conduct the Army's cost-benefit analysis again using best practices
so that its data and conclusions are comprehensive, traceable,
accurate, and credible.
* Correct any technical and mathematical errors in the cost estimate.
* Establish quality control and independent review processes that check
data sources, calculations, and assumptions.
* Perform a sensitivity analysis of key assumptions, including, at a
minimum,
(1) variations in the throughput rates for various options; (2) the
technological uncertainty of options; and (3) for off-site treatment
and disposal options, the risks associated with potential permitting,
legal, and other challenges.
Agency Comments:
In written comments on a draft of this report, DOD concurred with our
recommendations and stated that it fully supports the use of best
practices for the development and preparation of cost estimates. DOD
stated that the Army will be preparing a new cost-benefit analysis
based on a revised cost estimate, which will be independently reviewed
and verified, and will contain an analysis of assumptions. DOD
estimated that the revised cost estimate will be available by the third
quarter of fiscal year 2007 and the new cost-benefit analysis will be
available by the fourth quarter of fiscal year 2007. DOD's comments are
reproduced in full in enclosure II. DOD also provided us with technical
comments, which have been incorporated where appropriate. Finally, we
adjusted our fourth recommendation in the draft report in light of
DuPont's January 5, 2007, announcement that it was no longer interested
in being considered as a potential treatment site.
We are sending copies of this report to other interested congressional
parties. We also are sending copies to the Secretary of Defense; the
Secretary of the Army; and the Director, Office of Management and
Budget. We will make copies available to others upon request. In
addition, the report will be available at no charge on GAO's Web site
at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-5431 or by e-mail at dagostinod@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made key contributions to this report are listed in enclosure III.
Signed by:
Davi M. D'Agostino:
Director:
Defense Capabilities and Management:
Enclosures - 3:
List of Congressional Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Minority Member:
Committee on Armed Services:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan Hunter:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:
The Honorable Daniel Inouye:
Chairman:
The Honorable Ted Stevens:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of Section]
Enclosure I: Description of Eight Congressionally Specified Disposal
and Treatment Options for Newport's Hydrolysate:
* Chemical oxidation. Organic wastes are mixed with an oxidizing agent
(such as hydrogen peroxide) and water at low temperatures and low
pressure. This process breaks down the organic components of the waste
into either benign compounds or compounds that can be more easily
treated by other means.
* Electrochemical oxidation. A similar chemical process to chemical
oxidation, electrochemical oxidation uses a metallic element as an
oxidizing agent within an electrochemical cell. This also is a low-
temperature, low-pressure process.
* Biodegradation. This process uses microorganisms to destroy certain
organic compounds in dilute aqueous (water) solutions. This low-
temperature, low-pressure process is often used to treat sewage. Some
organic compounds can be readily broken down by biotreatment, while the
structure of other compounds makes them highly resistant.
* Solvated-electron technology. This process involves the reaction of
organic waste materials with solutions of metallic sodium in anhydrous
liquid ammonia. In contrast to most of the other technologies
considered by the Department of Defense's Chemical Demilitarization
Program, this is a reduction rather than an oxidation reaction.
Solvated-electron technology is a low-temperature, low-pressure
process.
* Wet-air oxidation. This process oxidizes organic compounds in water
using dissolved oxygen and air. It operates at relatively higher
temperatures and pressures than chemical oxidation. Wet-air oxidation
is commercially in use worldwide to treat industrial wastes.
* Supercritical water oxidation. This process destroys organic
compounds through oxidation by introducing air to water that has been
superheated beyond its critical point (374°C). This is a high-
temperature, high- pressure process.
* Gas-phase chemical reduction. This process uses hydrogen and steam at
high temperatures to break down organic compounds into more easily
treated chemicals. Like solvated-electron technology, gas-phase
chemical reduction is a reduction reaction. This is a high-temperature
but low-pressure process.
* Plasma-arc technology. This process uses electrical discharges
through gases to produce intense radiant energy and high-temperatures
to break down organic compounds in a containment chamber. This is an
extremely high-temperature but low-pressure process.
[End of Section]
Enclosure II: Comments from the Department of Defense:
Nuclear And Chemical And Biological Defense Programs:
Assistant To The Secretary Of Defense:
3050 Defense Pentagon:
Washington, DC 20301-3050:
Jan 4 2007:
Ms. Davi M. D'Agostino:
Director, Defense Capabilities and Management:
United States Government Accountability Office 441 G Street, N. W.
Washington, DC 20548:
Dear Ms. D'Agostino:
This is the Department of Defense (DoD) response to the GAO draft
report, GAO-07-240R, `Review of the Cost Comparison of Off-Site Versus
On-Site and Treatment and Disposal of Hydrolysate at the Newport
Chemical Depot, Indiana,' dated December 6, 2006 (GAO Code 350933).
The DoD concurs with the draft report's recommendations. The Department
fully supports the use of best practices for the development and
preparation of cost estimates. As such, the Army will be preparing a
new cost-benefit analysis with a revised cost estimate, which will be
independently reviewed and verified.
The Department appreciates the opportunity to provide comments on the
draft report. These comments, including technical comments, are
enclosed. For further questions concerning this report, please contact
Barbara Burgess, Senior Program Analyst for the Chemical
Demilitarization Program, (703) 588-1983, extension 113.
Sincerely,
Signed by:
Jean D. Reed:
Special Assistant:
Chemical and Biological Defense and Chemical Demilitarization:
Enclosures:
As stated:
cc: Deputy Assistant Secretary of the Army (Elimination of Chemical
Weapons):
GAO Draft Report - Dated December 6, 2006 GAO Code 350933/GAO-07-240R:
"Review of the Cost Comparison of Off-Site Versus On-Site Treatment and
Disposal of Hydrolysate at the Newport Chemical Depot, Indiana"
Department Of Defense Comments To The Recommendations:
Recommendation 1: The GAO recommended that the Secretary of Defense
direct the Secretary of the Army to conduct the Army's cost-benefit
analysis again using best practices so that its data and conclusions
are accurate, replicable, traceable, verifiable, comprehensive, and
credible (p. 18/GAO Draft Report).
DOD Response: Concur. The new cost-benefit analysis, based on a revised
cost estimates, will be conducted utilizing best practices to include
the Association for the Advancement of Cost Engineering methods. The
new cost-benefit analysis will be available during the fourth quarter
of Fiscal Year 2007 (FY07).
Recommendation 2: The GAO recommended that the Secretary of Defense
direct the Secretary of the Army to correct any technical and
mathematical errors in the cost estimate (p. 18/GAO Draft Report).
DOD Response: Concur. The cost estimates will be revised for use in the
new cost-benefit analysis. The revised cost estimates will be reviewed
independently for methodology and application. The revised cost
estimate will be available during the third quarter of FY07.
Recommendation 3: The GAO recommended that the Secretary of Defense
direct the Secretary of the Army to establish quality control and
independent review processes that check date sources, calculations, and
assumptions (p. 18/GAO Draft Report).
DOD Response: Concur. The Army is directing the U.S. Army Chemical
Materials Agency, which prepared the initial analysis report, to
conduct an independent review of the revised cost estimates. The
revised cost estimate will be available during the third quarter of
FY07.
Recommendation 4: The GAO recommended that the Secretary of Defense
direct the Secretary of the Army to perform a sensitivity analysis of
key assumptions, including, at a minimum, (1) variations in the
throughput rates for various options; (2) the technological uncertainty
options; and (3) for the DuPont option, the risks associated with
potential permitting, legal, and other challenges (p. 18/GAO Draft
Report).
DOD Response: Concur. The independent review of cost data, estimates,
and analysis will include an analysis of the assumptions. The revised
cost estimate will be during the third quarter of FY07.
[End of Section]
Enclosure III:
GAO Contact and Staff Acknowledgments:
GAO Contact:
Davi M. D'Agostino, (202) 512-5431 or dagostinod@gao.gov:
Acknowledgments:
In addition to the contact named above, Mark A. Pross, Assistant
Director; Bonita Anderson; Rodell Anderson; Susan Ditto; Jennifer
Echard; Neil Feldman; James Lawson; Brian Octeau; Charles Perdue; and
Karen Richey made key contributions to this report.
(350933):
FOOTNOTES
[1] VX is a rapid-acting, lethal nerve agent that affects the nervous
system by interfering with the signals sent from the brain to vital
organs. Nerve agents are the most toxic and rapidly acting of known
chemical warfare agents.
[2] The Chemical Weapons Convention prohibits the use of chemical
weapons and specifies deadlines for signatories, of which the United
States is one, to destroy unitary stockpiles. The final deadline to
destroy existing stockpiles is April 29, 2012.
[3] Department of Health and Human Services, Centers for Disease
Control and Prevention, Review of the Revised Plan for Off-Site
Treatment of Newport's Chemical Agent Disposal Facility's Caustic VX
Hydrolysate at DuPont Secure Environmental Treatment Facility in
Deepwater, New Jersey (Atlanta. Ga.: July 2006).
[4] U.S. Army, Project Manager for Alternative Technologies and
Approaches, Cost-Benefit Analysis of Off-Site Versus On-Site Treatment
and Disposal of Newport Caustic Hydrolysate (Aberdeen Proving Ground,
Edgewood Area, Md.: April 2006).
[5] Pub. L. No. 109-364, § 922 (2006).
[6] Shaw Environmental, Inc., Newport Chemical Disposal Facility On-
Site Hydrolysate Post Treatment Estimate (Edgewood, Maryland: April
2006).
[7] The Department of the Army Cost Analysis Manual contains criteria
and a checklist that are similar to the OMB and best practices
guidance. See Department of the Army, U.S. Army Cost and Economic
Analysis Center, Cost Analysis Manual (Washington, D.C.: May 2002).
[8] The Society of Cost Estimating and Analysis (SCEA) is an
organization dedicated to improving cost estimating and analysis and to
furthering the effectiveness and efficiency of cost estimating and
analysis. The characteristics of a high-quality estimate are contained
in SCEA's Cost Programmed Review of Fundamentals © 2003 SCEA.
[9] H.R. Rep. No. 109-89 (2005).
[10] Non-stockpile chemical materiel are items not included as part of
the nation's unitary chemical stockpile, including buried munitions and
binary munitions.
[11] National Research Council, Review and Evaluation of the Army Non-
Stockpile Chemical Materiel Disposal Program: Disposal of Neutralent
Wastes (Washington, D.C.: 2001).
[12] The Congress established the Assembled Chemical Weapons Assessment
program in 1996 to identify and demonstrate at least two alternative
technologies to baseline incineration. Omnibus Consolidated
Appropriations Act, 1997, Pub. L. No. 104-208 (1996). In 2002, the
program was assigned responsibility for full-scale pilot testing of
neutralization technologies to destroy the chemical weapons stockpiles
at the Pueblo Chemical Depot in Colorado and Blue Grass Army Depot in
Kentucky. Department of Defense Appropriations Act, 2003, Pub. L. No.
107-248 (2002). In 2003, the program's name was changed to Assembled
Chemical Weapons Alternatives.
[13] National Research Council, Review and Evaluation of Alternative
Chemical Disposal Technologies (Washington, D.C.: 1996).
[14] National Research Council, Integrated Design of Alternative
Technologies for Bulk-Only Chemical Agent Disposal Facilities
(Washington, D.C.: 2000).
[15] Price data for DuPont were not included as part of Parsons' 2005
market research pricing for hazardous waste treatment.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
