Defense Transportation
DOD Has Adequately Addressed Congressional Concerns Regarding the Cost of Implementing the New Personal Property Program Initiatives Gao ID: GAO-05-715R June 9, 2005Military personnel and their families can expect to relocate many times during a servicemember's career. As the moving industry's single largest customer, the Department of Defense (DOD) spends more than $1.7 billion annually for its personal property program, which provides household goods transportation and storage services for military personnel and their families when they relocate. The program manages more than 600,000 personal property shipments each year. For more than 10 years, DOD has been pursuing various initiatives for improving the quality of its personal property program. In June 2002, the U.S. Transportation Command completed an extensive study that compared the features of the current personal property program with three pilot programs that tested alternative approaches for improving the current program. In November 2002, DOD issued a report to Congress that included three recommended program improvement initiatives resulting from this study and estimated that an additional 13 percent increase over current program costs would be required to implement two of these initiatives. In April 2003, we reported on the pilot program evaluation and stated that the recommendations contained in DOD's November 2002 report offered solutions to long-standing problems in the personal property program and should be implemented within budget constraints. However, we raised concerns about whether the two recommendations related to the claims and contracting processes could be implemented within the projected 13 percent cost estimate. Our concerns were partially based on DOD not adequately substantiating the expected economies of scale that supported its assumption of a 5 percent reduction in average prices under the pilot program when a full-scale program is implemented. As a result, we recommended that DOD quantify the risk associated with the cost estimate before DOD's recommended initiatives are implemented. In its May 2004 report on the National Defense Authorization Act for Fiscal Year 2005, the House Armed Services Committee directed the Secretary of Defense to reevaluate DOD's proposed cost estimate, quantify the risk or likelihood of achieving its goals within the 13 percent projected cost increase, and develop a range of possible cost increases associated with the risk. The committee also directed GAO to review and report on whether DOD adequately performed these tasks.
The methodology used by DOD to address congressional concerns--reevaluate its cost estimate, quantify the risk associated with achieving the 13 percent cost increase, and establish a range of possible cost increases--was reasonable for conducting such assessments. In its reevaluation of the cost estimate, DOD validated the current mix of household goods shipments to ensure that the mix used in the original assessment was appropriate. DOD also provided additional support for its assessment of the anticipated savings that would result from expanding the pilot to all of DOD. Furthermore, the results of DOD's risk-based simulation analysis, which included establishing ranges of possible cost increases, indicated that about 80 percent of the time, DOD could implement the recommended initiatives to improve the current program with an increase of 15 percent or less above current program costs.
GAO-05-715R, Defense Transportation: DOD Has Adequately Addressed Congressional Concerns Regarding the Cost of Implementing the New Personal Property Program Initiatives
This is the accessible text file for GAO report number GAO-05-715R
entitled 'Defense Transportation: DOD Has Adequately Addressed
Congressional Concerns Regarding the Cost of Implementing the New
Personal Property Program Initiatives' which was released on June 10,
2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
Washington, DC 20548:
June 9, 2005:
The Honorable John Warner:
Chairman:
The Honorable Carl Levin:
Ranking Minority Member:
Committee on Armed Services: United States Senate:
The Honorable Duncan L. Hunter:
Chairman:
The Honorable Ike Skelton:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:
Subject: Defense Transportation: DOD Has Adequately Addressed
Congressional Concerns Regarding the Cost of Implementing the New
Personal Property Program Initiatives:
Military personnel and their families can expect to relocate many times
during a servicemember's career. As the moving industry's single
largest customer, the Department of Defense (DOD) spends more than $1.7
billion annually for its personal property program, which provides
household goods transportation and storage services for military
personnel and their families when they relocate. The program manages
more than 600,000 personal property shipments each year.
For more than 10 years, DOD has been pursuing various initiatives for
improving the quality of its personal property program. In June 2002,
the U.S. Transportation Command completed an extensive study that
compared the features of the current personal property program with
three pilot programs that tested alternative approaches for improving
the current program. In November 2002, DOD issued a report to Congress
that included three recommended program improvement initiatives
resulting from this study and estimated that an additional 13 percent
increase over current program costs would be required to implement two
of these initiatives.[Footnote 1]
In April 2003, we reported on the pilot program evaluation and stated
that the recommendations contained in DOD's November 2002 report
offered solutions to long-standing problems in the personal property
program and should be implemented within budget constraints.[Footnote
2] However, we raised concerns about whether the two recommendations
related to the claims and contracting processes could be implemented
within the projected 13 percent cost estimate. Our concerns were
partially based on DOD not adequately substantiating the expected
economies of scale that supported its assumption of a 5 percent
reduction in average prices under the pilot program when a full-scale
program is implemented. As a result, we recommended that DOD quantify
the risk associated with the cost estimate before DOD's recommended
initiatives are implemented.
In its May 2004 report on the National Defense Authorization Act for
Fiscal Year 2005,[Footnote 3] the House Armed Services Committee
directed the Secretary of Defense to reevaluate DOD's proposed cost
estimate, quantify the risk or likelihood of achieving its goals within
the 13 percent projected cost increase, and develop a range of possible
cost increases associated with the risk. The committee also directed
GAO to review and report on whether DOD adequately performed these
tasks.
To conduct our work, we reviewed prior DOD and GAO reports on the
personal property program and interviewed DOD officials and officials
from DOD's contractor who were involved in the current cost
reevaluation study. We assessed the methodologies used by DOD to
reevaluate the cost estimate and quantify the risk associated with
implementing the proposed initiatives within the cost estimate,
including establishing ranges of possible cost increases. We also
assessed the reasonableness of the methodologies used by DOD to perform
these tasks. We performed our review from November 2004 through May
2005 in accordance with generally accepted government auditing
standards. The scope and methodology we used in our review are
described in further detail in enclosure I.
Results in Brief:
The methodology used by DOD to address congressional concerns---
reevaluate its cost estimate, quantify the risk associated with
achieving the 13 percent cost increase, and establish a range of
possible cost increases--was reasonable for conducting such
assessments. In its reevaluation of the cost estimate, DOD validated
the current mix of household goods shipments to ensure that the mix
used in the original assessment was appropriate. DOD also provided
additional support for its assessment of the anticipated savings that
would result from expanding the pilot to all of DOD. Furthermore, the
results of DOD's risk-based simulation analysis, which included
establishing ranges of possible cost increases, indicated that about 80
percent of the time, DOD could implement the recommended initiatives to
improve the current program with an increase of 15 percent or less
above current program costs.
DOD concurred with the content of this letter.
Background:
DOD's personal property program is managed centrally by the Military
Surface Deployment and Distribution Command, formerly known as the
Military Traffic Management Command.[Footnote 4] DOD has experienced
long-standing problems with its current personal property program,
including excessive loss of or damage to property, high claims costs
incurred by the government, and poor quality of service from moving
companies. Moreover, the program's data management system does not
provide reliable information on the status of individual shipments or
on the types of shipments and their costs.
In its November 12, 2002, report to Congress, DOD made three
recommendations aimed at improving its current personal property
program.[Footnote 5] The three recommendations were to (1) reengineer
the liability and claims process by adopting commercial practices of
minimum valuation, simplifying the filing of claims, and providing
direct settlement with the carrier; (2) change the acquisition process
to implement performance-based service contracts; and (3) implement
information technology improvements, which could interface functions
across such areas as personnel, transportation, financial, and claims.
DOD reported that the estimated cost of implementing its information
technology improvements recommendation would be from $4 million to $6
million, and estimated that the cost of implementing the claims process
and performance-based service contract recommendations would require an
additional 13 percent over the current personal property program's
costs.
In our April 2003 report, we stated that the three recommendations
contained in DOD's report were supported by the Transportation
Command's evaluation of the pilot programs' findings and offered
solutions to the long-standing problems that had plagued the current
program for many years.[Footnote 6] Our review showed that the
soundness of the methodologies DOD used to develop cost estimates for
implementing the three recommendations varied. We found that the
estimates DOD reported to Congress might understate the total initial
cost for implementing the information technology improvements
recommendation and contained a questionable adjustment for costs
associated with the claims and contracting process recommendations.
In its original cost assessment for the recommendations related to the
claims and contracting processes, DOD made three adjustments to the
average costs for the pilot programs: (1) reducing the average weight
of shipments, (2) reducing costs to adjust for a mix of small and large
businesses, and (3) reducing the pilot programs' costs to reflect
anticipated savings based on "economies of scale." In our April 2003
report, we agreed that the first two adjustments were reasonable;
however, we questioned the extent to which the third adjustment could
be achieved. Part of our concern was based on DOD not adequately
substantiating the expected economies of scale that supported its
assumption of a 5 percent reduction in the average prices under the
pilot programs when a full-scale program is implemented. Since DOD had
not quantified the risk associated with its projected implementation
cost estimate of a 13 percent increase over the current program's cost,
the military services and Congress lacked information needed to develop
and review future budget requests for the program. Therefore, we
questioned the extent to which the recommendations could be implemented
within DOD's projected cost estimate.
As a result of our findings, we recommended that DOD take the following
actions to improve its personal property program: (1) initiate actions
that will implement the recommendations contained in DOD's report to
Congress within budget constraints, (2) quantify the risk associated
with achieving implementation of the recommended initiatives within the
projected 13 percent cost estimate, (3) monitor costs during the
implementation phase to ensure that the proposed changes are being
achieved within an acceptable and a predefined range, and (4) assess
the personal property program after the recommendations have been
implemented to determine whether anticipated improvements are being
achieved at a reasonable cost.
DOD partially concurred with our second recommendation---to provide the
military services and Congress with additional information to quantify
the risk associated with achieving implementation of the recommended
initiatives within the projected 13 percent cost increase. DOD believed
it could incorporate the two recommendations into a new program within
its proposed 13 percent cost estimate due to the conservative approach
it took in developing the estimate. However, in May 2004, the House
Armed Services Committee, in its report on the National Defense
Authorization Act for Fiscal Year 2005, directed DOD to reevaluate its
proposed cost estimate, quantify the risk or likelihood of achieving
its goals within the 13 percent cost projection, and develop a range of
possible cost increases associated with that risk. DOD issued a report
addressing these congressional concerns on March 29, 2005.[Footnote 7]
DOD Adequately Addressed Congressional Concerns:
In addressing congressional concerns, DOD used a reasonable
methodological approach to adequately reevaluate its cost estimate and
quantify the risk associated with implementing the proposed personal
property program initiatives within the 13 percent cost increase,
including establishing a range of potential cost increases. DOD's
analysis included three key components: (1) an assessment of any
changes in the distribution of shipments among the services and the mix
of continental and overseas household goods shipments, (2) new support
for its assessment of anticipated savings through "full-program
efficiencies" rather than "economies of scale," and (3) a simulation
based on differing estimates of "full-program efficiencies" to
determine the likelihood that DOD could implement its initiatives
within its cost estimate. We believe that each of these components was
necessary to reassess the reasonableness of DOD's previous cost
estimate.
DOD first validated the current mix of household goods shipments to
ensure that the mix used in the original assessment was appropriate.
DOD's reevaluation assessed changes, if any, in the distribution of
shipments among the services and the mix of continental and overseas
U.S. household goods shipments. Because the original cost estimate was
based on household goods shipments picked up and delivered during the
last half of fiscal year 2001, DOD needed to determine if the mix of
household goods shipments had changed. Using data from its management
information system, the Transportation Operational Personal Property
Standard System, DOD found some minor differences in shipments among
the services from the baseline year (fiscal year 2001) to fiscal year
2004, but concluded that the differences were insignificant and would
have no effect on the calculations developed based upon the fiscal year
2001 data. This was a reasonable methodological approach to validate
the mix of shipments.
DOD then used a reasonable methodology to provide additional support
for its assessment of the anticipated savings that would result from
expanding the pilot to all of DOD. In its reassessment of the cost
estimate, DOD reported that the anticipated savings could be better
described as "full-program efficiencies" instead of "economies of
scale." For example, DOD's reevaluation explained that prospective
participants in the pilot programs faced limitations inherent in the
pilot programs in a competitive market because they could not
anticipate the longevity of the pilot program nor accurately predict
the volume of potential shipments. Potential participants in the pilot
programs also faced additional uncertainties that included not knowing
their daily or monthly volume for this new program, having to project
rates into option years without economic price adjustment, and facing
possible penalties for withdrawing from the program. Consequently, the
rates submitted by prospective participants reflected these
uncertainties and other risk factors. It could reasonably be expected
that as experience under the new program is gained and a steady-state
program is reached, these risks would be reduced and would be reflected
in lower rates. We believe that with this detailed description of the
full-program efficiencies, DOD has provided reasonable additional
support for the anticipated savings of the program, thereby reaffirming
the 5 percent efficiency gains it previously reported.
Finally, DOD used a risk-based simulation approach to provide a range
of possible cost increases within statistical confidence intervals
around the cost estimate, which is a reasonable methodology. This
simulation quantified the risk associated with implementing the
proposed initiatives within the 13 percent cost estimate. The results
of this risk analysis indicated that about 80 percent of the time, DOD
could implement the recommended initiatives to improve the current
program within an increase of 15 percent or less above current program
costs. All simulations used by DOD involved 10,000 iterations of six
different cases of variation in the full-program efficiencies estimate
to assess how different levels of full-program efficiencies affected
the cost estimates.[Footnote 8] Table 1 summarizes DOD's simulation
results for each scenario. As the table shows, only case two would
result in 80 percent of all estimates being 12.5 percent or less--an
amount lower than the original cost estimate. The remaining five
simulations of the random effect of the full-program efficiencies
contain the original 13.4 percent estimate within 90 percent confidence
intervals.
Table 1: Simulation Results Showing the Effect of Varying Program
Efficiency Distributions on Achieving the Projected Cost Estimate:
Case: 1;
Program efficiency estimate: Low value: 5%;
Program efficiency estimate: High value: 5%;
Program efficiency estimate: Most frequent value: 5%;
Program efficiency estimate: 80th percentile: 14.4%;
90% confidence interval for cost estimate: Low value: 11%;
90% confidence interval for cost estimate: High value: 15%.
Case: 2;
Program efficiency estimate: Low value: 0%;
Program efficiency estimate: High value: 20%;
Program efficiency estimate: Most frequent value: 5%;
Program efficiency estimate: 80th percentile: 12.5%;
90% confidence interval for cost estimate: Low value: 4%;
90% confidence interval for cost estimate: High value: 15%.
Case: 3;
Program efficiency estimate: Low value: 0%;
Program efficiency estimate: High value: 15%;
Program efficiency estimate: Most frequent value: 5%;
Program efficiency estimate: 80th percentile: 13.8%;
90% confidence interval for cost estimate: Low value: 7%;
90% confidence interval for cost estimate: High value: 16%.
Case: 4;
Program efficiency estimate: Low value: 0%;
Program efficiency estimate: High value: 20%;
Program efficiency estimate: Most frequent value: 2.5%;
Program efficiency estimate: 80th percentile: 13.7%;
90% confidence interval for cost estimate: Low value: 4%;
90% confidence interval for cost estimate: High value: 16%.
Case: 5;
Program efficiency estimate: Low value: 0%;
Program efficiency estimate: High value: 15%;
Program efficiency estimate: Most frequent value: 2.5%;
Program efficiency estimate: 80th percentile: 14.9%;
90% confidence interval for cost estimate: Low value: 8%;
90% confidence interval for cost estimate: High value: 17%.
Case: 6;
Program efficiency estimate: Low value: 0%;
Program efficiency estimate: High value: 10%;
Program efficiency estimate: Most frequent value: 5%;
Program efficiency estimate: 80th percentile: 15.1%;
90% confidence interval for cost estimate: Low value: 10%;
90% confidence interval for cost estimate: High value: 17%.
Source: GAO analysis of DOD data.
[End of table]
We reviewed the results of the simulation to assess its rigor and how
well the simulation reflects the household goods shipment costs. Based
on our review, a briefing of the methodology described in the report,
and our knowledge of the analytical approaches for conducting such an
assessment, we believe that DOD used a reasonable methodology that is
consistent with professional standards to quantify the risk associated
with implementing the program improvement initiatives within its cost
estimate, because DOD ran a simulation that is a reasonable approach
for assessing risk, using a commercially available program. See
enclosure II for a detailed summary of the results of DOD's risk
assessment.
Agency Comments:
DOD concurred with the content of this letter. DOD's letter is included
in enclosure III.
We are sending copies of this report to the appropriate congressional
committees; the Secretary of Defense; the Commander, U.S.
Transportation Command; and the Director, Office of Management and
Budget. We will also make copies available to others upon request. In
addition, the report will be made available at no charge on the GAO Web
site at http://www.gao.gov.
Please contact me at (202) 512-8365 or solisw@gao.gov if you or your
staff have any questions concerning this report. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report were
Jacqueline S. McColl, Arthur L. James, Jr., Charles W. Perdue, Karen N.
Harms, Renee S. Brown, and Ann Borseth.
Signed by:
William M. Solis:
Director, Defense Capabilities and Management:
Enclosures:
[End of section]
Enclosure I: Scope and Methodology:
To assess the methods used by the Department of Defense (DOD) to
reevaluate the cost estimate and quantify the risk associated with
implementing the program improvement initiatives within the stated cost
projection, including establishing a range of possible cost increases,
we reviewed the cost projection methodology used by DOD in its report
on the reevaluation of the projected cost estimate. We also met with
DOD officials and its contractor officials to discuss the methodology
used to perform the cost reevaluation and risk assessment.
To assess the reasonableness of the methodology used by DOD to perform
these tasks, we compared DOD's reevaluation methodology to analytical
approaches applicable for this type of evaluation, assessed adjustments
made in the evaluation strategy to address issues that could affect the
validity of the results, and reviewed the evaluation techniques used to
analyze data. We reviewed the results of the simulation DOD used to
conduct its risk assessment to assess its rigor and how well the
simulation reflects the household goods shipment costs. We also
reviewed the contents of the DOD contractor's briefings on the planned
approach and evaluation techniques used to reevaluate the projected 13
percent cost estimate.
We did not assess whether the anticipated benefits to be derived from
implementing the three recommendations would warrant the additional
costs that DOD estimates will be required to fund these improvements.
Furthermore, we did not independently test the reliability of data DOD
extracted from its data system to develop costs.
During this review of DOD's evaluation efforts, we met with officials
and obtained documents from the Office of the Assistant Deputy Under
Secretary of Defense (Transportation Policy), Washington, D.C.; the
U.S. Transportation Command, Scott Air Force Base, Illinois; the
Surface Deployment and Distribution Command, Alexandria, Virginia; and
LMI (Surface Deployment and Distribution Command contractor), McLean,
Virginia. In addition to these agency meetings and documents, we drew
upon information contained in a previous GAO report resulting from our
prior review of this program.
Our work for this review was performed from November 2004 through May
2005 in accordance with generally accepted government auditing
standards.
[End of section]
Enclosure II: Summary of DOD's Risk Assessment:
DOD's methodology to quantify the risk associated with the cost
estimate included using a commercially available program to run a Monte
Carlo simulation,[Footnote 9] which is a reasonable methodological
approach for assessing risk. The simulation contained six cases of
variation in the full-program efficiencies estimate to assess how these
full-program efficiencies affected the cost estimates. Case one was the
baseline case involving the original 13.4 percent increase resulting
from an assumption of a fixed 5 percent reduction for full-program
efficiencies. This case represents the original estimates provided by
DOD in its November 2002 report to Congress. The remaining five cases
randomly varied the assumption for the full-program efficiencies over
five different ranges, from a very compact range between zero and 10
percent to a very diverse range from zero to 20 percent.
Two cases in particular illustrated the effects of the simulation. Case
six, the very compact case, allowed the frequency of the full-program
efficiencies estimate to increase from zero smoothly to 5 percent and
decline smoothly to 10 percent. The result of this simulation was very
similar to the base case (case one, in which the original estimate of
full-program efficiencies was held equal to 5 percent). The results
indicate that 90 percent of the time the cost estimate would fall from
10 to 17 percent, with 80 percent of all estimates less than 15.1
percent.
The most "generous" case, case two, was based upon the assumption of
the frequency for full-program efficiencies increasing smoothly to 5
percent and declining smoothly to 20 percent. This case is most
generous because it assumes that large cost savings (up to 20 percent)
are achievable even though the most likely cost savings would be 5
percent. The results indicate that 90 percent of the time the cost
estimate would fall from 4 to 15 percent, with 80 percent of all
estimates less than 12.5 percent. This was the one case where the
original estimate was outside the 80th percentile and thus a lower cost
estimate was determined to be most likely. However, the case was most
generous in its assumption of the frequency with which "large" full-
program efficiencies would occur and could reasonably be expected to
indicate a smaller increase in estimated costs over the baseline
program.
The other three cases, between the two extremes, varied the frequency
of full-program efficiencies over different ranges and the simulation
results contained the original 13.4 percent estimate within the 90
percent confidence intervals. Based on our review of DOD's analysis, we
believe that DOD used a reasonable approach that was consistent with
professional standards to quantify the risk associated with
implementing the program improvement initiatives within its cost
estimate.
[End of section]
Enclosure III: Comments from the Department of Defense:
DEPUTY UNDER SECRETARY OF DEFENSE FOR LOGISTICS AND MATERIEL READINESS:
3500 DEFENSE PENTAGON:
WASHINGTON, DC 20301-3500:
June 2. 2005:
Mr. William M. Solis:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
Washington, D.C. 20548:
Dear Mr. Solis:
This is the Department of Defense (DoD) response to the Government
Accountability Office Draft Report, GAO-05-715R, "Defense
Transportation: DoD Has Adequately Addressed Congressional Concerns
Regarding the Cost of Implementing the New Personal Property Program
Initiatives," dated May 20, 2005 (GAO Code 350619).
The Department has reviewed the draft GAO report and concurs without
comment. We appreciate the opportunity to review the draft report.
Sincerely,
Signed by:
Bradley Berkson:
Acting:
[End of section]
Related GAO Products:
Defense Transportation: Monitoring Costs and Benefits Needed While
Implementing a New Program for Moving Household Goods. GAO-03-367.
Washington, D.C.: April 18, 2003.
Defense Transportation: Final Evaluation Plan Is Needed to Assess
Alternatives to the Current Personal Property Program. GAO/NSIAD-00-
217R. Washington, D.C.: September 27, 2000.
Defense Transportation: The Army's Hunter Pilot Project Is Inconclusive
but Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.: June
23, 1999.
Defense Transportation: Plan Needed for Evaluating the Navy Personal
Property Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.
Defense Transportation: Efforts to Improve DOD's Personal Property
Program. GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.
Defense Transportation: The Army's Hunter Pilot Project to Outsource
Relocation Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.
Defense Transportation: Reengineering the DOD Personal Property
Program. GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.
FOOTNOTES
[1] Department of Defense, U.S. Transportation Command Personal
Property Pilot Programs Evaluation Report (Washington, D.C.: November
2002).
[2] GAO, Defense Transportation: Monitoring Costs and Benefits Needed
While Implementing a New Program for Moving Household Goods, GAO-03-367
(Washington, D.C.: Apr. 18, 2003).
[3] H.R. Rep. No. 108-491, at 298 (2004).
[4] The Military Surface Deployment and Distribution Command, a
component of the U.S. Transportation Command, is the DOD executive
agent responsible for managing the relocation process for
servicemembers and their families.
[5] Department of Defense, U.S. Transportation Command Personal
Property Pilot Programs Evaluation Report.
[6] GAO-03-367.
[7] Department of Defense, Reevaluation of Cost Estimate for DOD
Families First Program (Washington, D.C.: March 2005).
[8] In its report, DOD noted that the advantage of the probability-
based simulation technique used to conduct its risk assessment is that
it can account for multiple input costs that have variability in risk
in relationship to how they affect the overall cost estimate.
[9] A Monte Carlo simulation is a method of evaluating hypotheses by
developing a computer model of a process, defining the parameters of
the process to reflect the "real world" situation, calculating multiple
results of varying parameters through some range of values, and
evaluating the distribution of results obtained from these samples. The
Monte Carlo method assigns a value to the model's parameters from a
sequence of random numbers, runs the model multiple times through
randomly differing parameter values, captures the outcome of each
iteration of the model, and assesses the distribution of outcomes using
standard statistical methods.
