Defense Budget
Independent Review Is Needed to Ensure DOD's Use of Cost Estimating Tool for Contingency Operations Follows Best Practices Gao ID: GAO-08-982 September 15, 2008Since the September 2001 terrorist attacks, Congress has provided about $800 billion as of July 2008 to the Department of Defense (DOD) for military operations in support of the Global War on Terrorism (GWOT). GWOT budget requests have grown in scope and the amount requested has increased every year. DOD uses various processes and the Contingency Operations Support Tool (COST) to estimate costs for these operations and to develop budget requests. GAO assessed (1) how DOD uses COST and other processes to develop GWOT budget requests and (2) what actions DOD has taken to ensure COST adheres to best practices for cost estimation. GAO interviewed DOD officials and others to determine how the services develop GWOT budget requests using COST and other processes. GAO also used its Cost Assessment Guide as criteria for best practices for cost estimation.
The services use COST as part of their process to develop a GWOT budget request. While the Army relies more on the estimate resulting from COST, the other services adjust the results of COST to reflect estimates they generate outside of COST, based on historical obligation data and other information. DOD's financial management regulation and other guidance require components to use COST to develop an estimate for the deployment and sustainment of military personnel and equipment for ongoing operations in support of GWOT. While all services use COST to develop an initial estimate, Air Force, Marine Corps, and Navy budget officials alter the results of the tool to match information provided by lower level commands and historical obligation data that they believe are more accurate than the COST-generated estimate. These officials stated that the tool routinely overestimates some costs and therefore most changes made are decreases in the amount estimated by COST. These officials believe that the requirement to use COST to develop a GWOT budget request is a duplicative process to their preferred method of using historical obligation data and other information better suited to their specific service. For example, they stated that COST better represents the needs of Army ground forces and the tool has not been refined to be as effective for estimating needs for their service's mission. These officials also mentioned that COST is better suited for developing estimates for smaller-scale contingency operations than for the lengthy deployments and sustainment phases associated with a large campaign such as GWOT. To develop estimates for items that are outside the scope of COST, such as procurement and certain contracts, the military services rely primarily on needs assessments developed by commanders and historical obligation data. DOD has taken steps to improve the performance and reliability of COST; however, COST could benefit from an independent review of the tool's adherence to best practices for high-quality cost estimation as described in GAO's Cost Assessment Guide. COST has been refined many times and cost factors are routinely updated in an effort to use the most current information available to develop an estimate. DOD officials stated they are confident in the tool's ability to provide reasonable estimates because COST is frequently updated. However, COST has not been assessed against best practices for cost estimation to determine whether COST can provide high-quality estimates that are well documented, comprehensive, accurate, and credible. While GAO did not undertake a full assessment of COST against best practices, it determined that some features of the tool meet best practices while other features would benefit from further review. For example, the tool adheres to several best practices for a comprehensive and accurate cost estimate, such as frequent updates to the structure of COST and the data the tool uses to generate estimates. However, COST relies on GWOT obligation data that GAO has identified as being of questionable reliability. A thorough, independent review of COST against best practices could provide decision makers with information about whether the tool creates cost estimates for GWOT expenses that are well documented, comprehensive, accurate, and credible.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-982, Defense Budget: Independent Review Is Needed to Ensure DOD's Use of Cost Estimating Tool for Contingency Operations Follows Best Practices
This is the accessible text file for GAO report number GAO-08- entitled
'Defense Budget: Independent Review Is Needed to Ensure DOD‘s Use of
Cost Estimating Tool for Contingency Operations Follows Best Practices'
which was released on September 15, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
September 2008:
Defense Budget:
Independent Review Is Needed to Ensure DOD's Use of Cost Estimating
Tool for Contingency Operations Follows Best Practices:
Defense Budget:
GAO-08-982:
GAO Highlights:
Highlights of GAO-08-982, a report to congressional committees.
Why GAO Did This Study:
Since the September 2001 terrorist attacks, Congress has provided about
$800 billion as of July 2008 to the Department of Defense (DOD) for
military operations in support of the Global War on Terrorism (GWOT).
GWOT budget requests have grown in scope and the amount requested has
increased every year. DOD uses various processes and the Contingency
Operations Support Tool (COST) to estimate costs for these operations
and to develop budget requests.
GAO assessed (1) how DOD uses COST and other processes to develop GWOT
budget requests and (2) what actions DOD has taken to ensure COST
adheres to best practices for cost estimation.
GAO interviewed DOD officials and others to determine how the services
develop GWOT budget requests using COST and other processes. GAO also
used its Cost Assessment Guide as criteria for best practices for cost
estimation.
GAO is recommending that DOD (1) arrange for an independent review of
COST to ensure that the model adheres to best practices and (2)
consider options for refining COST to better meet the needs of the
services. DOD agreed with both of GAO‘s recommendations.
What GAO Found:
The services use COST as part of their process to develop a GWOT budget
request. While the Army relies more on the estimate resulting from
COST, the other services adjust the results of COST to reflect
estimates they generate outside of COST, based on historical obligation
data and other information. DOD‘s financial management regulation and
other guidance require components to use COST to develop an estimate
for the deployment and sustainment of military personnel and equipment
for ongoing operations in support of GWOT. While all services use COST
to develop an initial estimate, Air Force, Marine Corps, and Navy
budget officials alter the results of the tool to match information
provided by lower level commands and historical obligation data that
they believe are more accurate than the COST-generated estimate. These
officials stated that the tool routinely overestimates some costs and
therefore most changes made are decreases in the amount estimated by
COST. These officials believe that the requirement to use COST to
develop a GWOT budget request is a duplicative process to their
preferred method of using historical obligation data and other
information better suited to their specific service. For example, they
stated that COST better represents the needs of Army ground forces and
the tool has not been refined to be as effective for estimating needs
for their service‘s mission. These officials also mentioned that COST
is better suited for developing estimates for smaller-scale contingency
operations than for the lengthy deployments and sustainment phases
associated with a large campaign such as GWOT. To develop estimates for
items that are outside the scope of COST, such as procurement and
certain contracts, the military services rely primarily on needs
assessments developed by commanders and historical obligation data.
DOD has taken steps to improve the performance and reliability of COST;
however, COST could benefit from an independent review of the tool‘s
adherence to best practices for high-quality cost estimation as
described in GAO‘s Cost Assessment Guide. COST has been refined many
times and cost factors are routinely updated in an effort to use the
most current information available to develop an estimate. DOD
officials stated they are confident in the tool‘s ability to provide
reasonable estimates because COST is frequently updated. However, COST
has not been assessed against best practices for cost estimation to
determine whether COST can provide high-quality estimates that are well
documented, comprehensive, accurate, and credible. While GAO did not
undertake a full assessment of COST against best practices, it
determined that some features of the tool meet best practices while
other features would benefit from further review. For example, the tool
adheres to several best practices for a comprehensive and accurate cost
estimate, such as frequent updates to the structure of COST and the
data the tool uses to generate estimates. However, COST relies on GWOT
obligation data that GAO has identified as being of questionable
reliability. A thorough, independent review of COST against best
practices could provide decision makers with information about whether
the tool creates cost estimates for GWOT expenses that are well
documented, comprehensive, accurate, and credible.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-982]. For more
information, contact Sharon L. Pickup, (202) 512-9619, pickups@gao.gov.
[End of section]
GAO Highlights:
Highlights of GAO-08-982, a report to congressional committees.
Why GAO Did This Study:
Since the September 2001 terrorist attacks, Congress has provided about
$800 billion as of July 2008 to the Department of Defense (DOD) for
military operations in support of the Global War on Terrorism (GWOT).
GWOT budget requests have grown in scope and the amount requested has
increased every year. DOD uses various processes and the Contingency
Operations Support Tool (COST) to estimate costs for these operations
and to develop budget requests.
GAO assessed (1) how DOD uses COST and other processes to develop GWOT
budget requests and (2) what actions DOD has taken to ensure COST
adheres to best practices for cost estimation.
GAO interviewed DOD officials and others to determine how the services
develop GWOT budget requests using COST and other processes. GAO also
used its Cost Assessment Guide as criteria for best practices for cost
estimation.
GAO is recommending that DOD (1) arrange for an independent review of
COST to ensure that the model adheres to best practices and (2)
consider options for refining COST to better meet the needs of the
services. DOD agreed with both of GAO‘s recommendations.
What GAO Found:
The services use COST as part of their process to develop a GWOT budget
request. While the Army relies more on the estimate resulting from
COST, the other services adjust the results of COST to reflect
estimates they generate outside of COST, based on historical obligation
data and other information. DOD‘s financial management regulation and
other guidance require components to use COST to develop an estimate
for the deployment and sustainment of military personnel and equipment
for ongoing operations in support of GWOT. While all services use COST
to develop an initial estimate, Air Force, Marine Corps, and Navy
budget officials alter the results of the tool to match information
provided by lower level commands and historical obligation data that
they believe are more accurate than the COST-generated estimate. These
officials stated that the tool routinely overestimates some costs and
therefore most changes made are decreases in the amount estimated by
COST. These officials believe that the requirement to use COST to
develop a GWOT budget request is a duplicative process to their
preferred method of using historical obligation data and other
information better suited to their specific service. For example, they
stated that COST better represents the needs of Army ground forces and
the tool has not been refined to be as effective for estimating needs
for their service‘s mission. These officials also mentioned that COST
is better suited for developing estimates for smaller-scale contingency
operations than for the lengthy deployments and sustainment phases
associated with a large campaign such as GWOT. To develop estimates for
items that are outside the scope of COST, such as procurement and
certain contracts, the military services rely primarily on needs
assessments developed by commanders and historical obligation data.
DOD has taken steps to improve the performance and reliability of COST;
however, COST could benefit from an independent review of the tool‘s
adherence to best practices for high-quality cost estimation as
described in GAO‘s Cost Assessment Guide. COST has been refined many
times and cost factors are routinely updated in an effort to use the
most current information available to develop an estimate. DOD
officials stated they are confident in the tool‘s ability to provide
reasonable estimates because COST is frequently updated. However, COST
has not been assessed against best practices for cost estimation to
determine whether COST can provide high-quality estimates that are well
documented, comprehensive, accurate, and credible. While GAO did not
undertake a full assessment of COST against best practices, it
determined that some features of the tool meet best practices while
other features would benefit from further review. For example, the tool
adheres to several best practices for a comprehensive and accurate cost
estimate, such as frequent updates to the structure of COST and the
data the tool uses to generate estimates. However, COST relies on GWOT
obligation data that GAO has identified as being of questionable
reliability. A thorough, independent review of COST against best
practices could provide decision makers with information about whether
the tool creates cost estimates for GWOT expenses that are well
documented, comprehensive, accurate, and credible.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-982]. For more
information, contact Sharon L. Pickup, (202) 512-9619, pickups@gao.gov.
[End of section]
Contents:
Letter:
Results In Brief:
Background:
Services Vary in Their Reliance on COST to Develop GWOT Budget
Estimates:
COST Is Frequently Updated and Refined, but Could Benefit from a Review
of Adherence to Best Practices:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: The 12 Steps of High-Quality Cost Estimating Mapped to
Best Practice Criteria:
Appendix III: An Example of DOD's GWOT Budget Estimate Process Using
the Sustainment and Redeployment of the Troop Surge:
Appendix IV: Comments from the Department of Defense:
Appendix V: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Four Characteristics of a High-Quality, Reliable Cost
Estimate:
Table 2: Primary Sources of Data within COST from DOD Components:
Table 3: Example of DOD Estimation for Operations in Support of GWOT:
Table 4: Non-COST estimates for Army Operation and Maintenance Costs:
United States Government Accountability Office:
Washington, DC 20548:
September 15, 2008:
Congressional Committees:
Between the terrorist attacks of September 11, 2001, and July 2008,
Congress has provided about $800 billion to the Department of Defense
(DOD) for domestic and overseas military operations in support of the
Global War on Terrorism (GWOT).[Footnote 1] Over the years, DOD has
used different approaches to request GWOT funding, including yearly
emergency supplemental budget requests, amendments to prior
supplemental requests, and requests included as part of the President's
annual budget submission. GWOT budget requests have increased from $14
billion for fiscal year 2002 to $189 billion for the total fiscal year
2008 request. The DOD components responsible for carrying out
activities in support of GWOT use various processes to develop budget
requests for GWOT. DOD's financial management regulation for
contingency operations[Footnote 2] and other guidance require military
components to use a DOD-sponsored cost estimation tool, called the
Contingency Operations Support Tool (COST), to develop cost estimates
for the deployment and sustainment of military personnel and equipment
for GWOT. This tool was constructed 10 years ago to develop estimates
for smaller-scale contingency operations, such as peacekeeping
operations and disaster relief operations, but is now being used to
also develop a portion of cost estimates for the large-scale GWOT
operations in Iraq and Afghanistan.
While COST has the capability to develop an estimate for military
personnel and operating costs, it does not have the capability to
estimate other costs such as procurement, some reset-level[Footnote 3]
maintenance of equipment, and certain contract costs. For the first few
years of GWOT, COST-generated estimates for personnel and operating
costs ranged from around 80 to 50 percent of the total budget request
for GWOT for fiscal years 2003 to 2005, respectively. However, this
COST-generated portion of the total has continued to decrease every
year. For the fiscal year 2008 request submitted in February 2007, the
COST-generated portion of the request was about 30 percent due to a
significant increase in certain types of estimated costs, such as
procurement and reset-related maintenance, which are outside the scope
of COST. Estimates for these types of costs, or the remaining 70
percent of the fiscal year 2008 request, are developed using other
models and formulas, historical obligation data, and requests submitted
by commanders. Each component submits its estimate for its GWOT budget
request, including the COST-related portion for military personnel and
operations needs and the portions developed by other means, to the
Office of the Under Secretary of Defense (Comptroller).[Footnote 4]
These estimates are reviewed and revised by the DOD Comptroller and the
Office of Management and Budget before they are submitted to Congress.
Because federal guidelines are limited on the processes, procedures,
and practices for ensuring credible cost estimates, we reviewed
existing best practices and developed the Cost Assessment Guide to fill
that gap.[Footnote 5] The best practices identified in our guide were
compiled from a thorough review of relevant legislation, regulations,
policy, and guidance for the criteria that most pertained to cost
estimating. Our guide defines cost estimation as the process of using
established methods and valid data to estimate the future costs of a
program based on what is known today, such as the use of COST to
develop a portion of GWOT budget requests. DOD and the Office of
Management and Budget have issued the majority of guidance and
regulations regarding cost estimating, and the best practices in our
guide were compiled from these and other sources. The characteristics
of high-quality cost estimation support a process that is well
documented, comprehensive, accurate, and credible.
Over the past several years, we have conducted a series of reviews
under the authority of the Comptroller General examining the funding
and reported obligations for operations in support of GWOT. We have
previously reported that there are reliability concerns regarding DOD's
reported GWOT obligations, and problems exist with transparency over
certain costs. We have made a series of recommendations to the
Secretary of Defense intended to improve the transparency and
reliability of DOD's reporting of GWOT obligations and to adjust GWOT
funding requests.[Footnote 6] DOD has implemented many of these prior
recommendations and continues to improve its cost reporting efforts.
Because of broad congressional interest in issues related to GWOT, we
prepared this report under the Comptroller General's authority to
conduct evaluations on his own initiative. Due to increasing budget
requests for DOD's ongoing operations in support of GWOT and to assist
members of Congress in their oversight role as they consider future
GWOT funding, we evaluated DOD's efforts to develop its GWOT budget
requests. Specifically, we assessed (1) how DOD uses COST and other
processes to develop GWOT budget requests and (2) what actions DOD has
taken to ensure COST adheres to best practices for cost estimation.
To examine how DOD uses COST and other processes to develop budget
requests for GWOT, we met with key officials from the DOD Comptroller,
the Joint Chiefs of Staff, and various service officials from the Air
Force, Army, Marine Corps, and Navy to understand their use of COST to
develop estimates for the deployment and sustainment of troops and
equipment and about other processes used to develop portions of GWOT
budget requests that are outside the scope of COST, such as procurement
and certain types of contracts. We also obtained their perspectives on
their experiences using COST and other processes to develop a cost
estimate for yearly emergency GWOT budget requests or amendments to
those requests. We interviewed representatives from the Institute for
Defense Analyses (IDA)--the contractor that developed and maintains
COST--to understand the development, function, and role of
COST.[Footnote 7] We reviewed DOD guidance and regulations as well as
COST-related documents such as briefings and the COST training manual.
We also analyzed an example of how the cost estimate for the operations
portion of an amendment to a yearly GWOT supplemental request was
developed. To assess the actions DOD has taken to ensure COST adheres
to best practices for cost estimation, we reviewed COST-related
documents, such as the manual for COST use and financial management
regulations. Our Cost Assessment Guide provides best practices for cost
estimating processes. We interviewed DOD and service officials and
representatives from IDA about steps that have been taken to improve
COST's effectiveness and functionality. Finally, we analyzed documents
and testimonial evidence from DOD officials and IDA representatives and
compared this information to the four characteristics of high-quality
cost estimation. While we did not conduct a full assessment of COST
against best practices, we identified some features of the tool that
either met best practices or appeared to not meet best practices.
We conducted this performance audit from July 2007 through September
2008 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Results In Brief:
While the military services use COST as part of their process to
develop a GWOT budget request, the Army relies more on the estimate
resulting from COST than the other services, which significantly adjust
the results of COST to reflect estimates they generate outside of COST.
These estimates are based on historical obligation data and other
information, which they believe is more representative of future GWOT
budget needs. DOD's financial management regulation and other guidance
require the components to use COST to develop an estimate for the
deployment and sustainment of military personnel and equipment for
ongoing operations in support of GWOT, and the guidance additionally
provides certain parameters and assumptions to be used in the tool.
While all the services use COST to develop an initial estimate, Air
Force, Marine Corps, and Navy budget officials stated that they alter
the results of the tool to match information provided by lower level
commands and historical obligation data that they believe are more
accurate than the COST-generated estimate. These officials report that
the required process of using COST to develop cost estimates is
duplicative to their preferred process of using historical obligation
data and other information better suited to their specific service to
develop a GWOT budget request. For example, they stated that while COST
better represents Army ground forces than the unique characteristics of
the other forces, the tool has not been refined to be as effective for
estimating certain needs for their service's mission. These officials
also mentioned that COST is better suited for developing estimates for
small-scale, short-duration contingency operations than for the lengthy
deployments and sustainment phases associated with a large campaign
such as GWOT. For the remaining types of funding needs that are outside
the scope of COST, such as procurement and certain contracts, DOD
guidance or regulations do not prescribe set processes for estimating
these types of costs. As a result, all military services rely primarily
on needs assessments developed by commanders and historical obligation
data to develop estimates for these types of costs.
DOD has taken steps to improve the performance and reliability of COST;
however, COST could benefit from an independent review of the tool's
adherence to best practices for high-quality cost estimation. For
example, COST has been refined many times over the past several years
and cost factors are routinely updated in an effort to use the most
current information available to develop an estimate. However, COST has
not been assessed against best practices for cost estimation to
determine whether COST can provide high-quality estimates that are well
documented, comprehensive, accurate, and credible. While we did not
undertake a full assessment of COST against best practices, during the
course of our review we identified some features of the tool that meet
best practices, as well as other features that would benefit from
further review. For example, the tool meets several best practices that
help create a comprehensive and accurate cost estimate, such as
frequent updates to the structure of COST and the data that COST uses
to generate estimates. However, while COST appears to largely encompass
the types of costs that are incurred to deploy and sustain Army ground
forces and their related equipment, it does not comprehensively or
accurately estimate some costs for the other services. The tool also
relies on GWOT obligation data that we have previously identified as
being of questionable reliability. DOD Comptroller officials stated
they are confident in the tool's ability to provide reasonable
estimates because COST is frequently updated. However, COST has not
been independently verified as an effective tool for the estimation of
GWOT costs. Based on our Cost Assessment Guide, we believe that a
thorough, independent review of COST against best practices could
provide decision makers with information about whether the tool creates
a cost estimate for GWOT expenses that is well documented,
comprehensive, accurate, and credible.
To ensure that the DOD budget requests for GWOT are based on a sound
cost estimation process, we recommend that the Secretary of Defense
direct the Under Secretary of Defense (Comptroller) to arrange for an
independent review of COST. Furthermore, based on the results of such a
review and taking into consideration how each service presently uses
COST, we also recommend that the Secretary of Defense direct the Under
Secretary of Defense (Comptroller) to consider options for refining
COST, determine the appropriate items or types of costs for which COST
should be used, and identify methods to be used when COST is not
appropriate. In written comments on a draft of this report, DOD agreed
with both of our recommendations. In addition, DOD provided technical
comments which we have incorporated as appropriate. DOD's comments and
our evaluation of them are discussed in detail in a later section of
this report and the department's comments are reprinted in appendix IV.
Background:
Cost estimation is a difficult process that requires both data and
judgment, and seldom, if ever, are estimates precise--the goal is to
find a "reasonable" estimate of future needs. Cost estimates are
necessary for government programs for many reasons: for example, to
support decisions about whether to fund one program over another,
develop annual budget requests, or evaluate resource requirements at
key decision points. As discussed in our Cost Assessment Guide,
developing a good cost estimate requires stable program requirements,
access to detailed documentation and historical data, well-trained and
experienced cost analysts, a risk and uncertainty analysis, and the
identification of a range of confidence levels. The guide also outlines
12 steps for a high-quality cost estimation process,[Footnote 8] which
are to:
* Define the estimate's purpose:
* Develop the estimating plan:
* Define the program:
* Determine the estimating approach:
* Identify ground rules and assumptions:
* Obtain the data:
* Develop the point estimate:
* Conduct sensitivity analysis:
* Conduct risk and uncertainty analysis:
* Document the estimate:
* Present estimate to management for approval:
* Update the estimate to reflect actual costs and changes.
It is important that cost estimators and independent organizations
validate that all cost elements are credible and can be justified by
acceptable estimating methods, adequate data, and detailed
documentation. Hence, in addition to the 12 steps of a high-quality
cost estimation process, the guide also describes four best practice
characteristics of a high-quality, reliable estimate generated by a
sound cost estimation process. Specifically, the estimate should be
well documented, comprehensive, accurate, and credible. Table 1
describes these characteristics in more detail. Adherence to these best
practices can help ensure that a cost estimation process provides a
reasonable estimate of how much it will cost to accomplish all tasks
related to a program and that the estimate is traceable, accurate, and
reflects realistic assumptions.
Table 1: Four Characteristics of a High-Quality, Reliable Cost
Estimate:
Characteristic: Well documented;
Description: Thoroughly documented, including source data and
significance, clearly detailed calculations and results, and
explanations of why particular methods and references were chosen. Data
can be traced to their source documents.
Characteristic: Comprehensive;
Description: Enough detail to ensure that cost elements are neither
omitted nor double counted. All cost- influencing ground rules and
assumptions are detailed in the estimate's documentation.
Characteristic: Accurate;
Description: Unbiased, not overly conservative or overly optimistic,
and based on an assessment of most likely costs. Few, if any,
mathematical mistakes are present and those that are present are minor.
Characteristic: Credible;
Description: Any limitations of the analysis because of uncertainty or
bias surrounding data or assumptions are discussed. Major assumptions
are varied and other outcomes are recomputed to determine how sensitive
they are to changes in the assumptions. Risk and uncertainty analysis
are performed. Estimate's results are cross-checked and an independent
cost estimate is developed to determine whether other estimation
methods produce similar results.
Source: GAO.
[End of table]
Appendix II presents the 12 steps of a high-quality cost estimation
process mapped to the four characteristics of reliable, high-quality
estimates.
Because DOD's cost estimates for military operations in Bosnia during
the 1990s were consistently well below the actual costs, DOD contracted
with IDA to develop a tool to assist in developing preliminary and
detailed cost estimates for contingency operations. By 1998, IDA had
developed the first version of COST. The tool generates a cost estimate
for a contingency operation on the basis of type of mission, duration,
operational tempo or intensity, number of personnel and equipment,
transportation needs, subsistence for personnel, and the originating
and destination site. Accordingly, the tool contains data relating to
geographic locations, military unit types, military equipment types,
management and cost factors, and adjustment factors pertaining to
climate, terrain, and operational intensity. Formulas within the tool
draw on these data and user-defined inputs, such as the number of
personnel or equipment and duration of operations, to develop cost
estimates for many types of costs associated with contingency
operations from predeployment to reconstitution, up to 250 line items
or types of costs depending on the operation in question. The tool
cannot estimate every type of cost that might be incurred for a
contingency operation. Rather, it can only estimate certain incremental
costs from the personnel, personnel support, operating, and
transportation cost categories. However, some types of costs within
those categories--such as depot-level maintenance of equipment and
certain contracts, as well as all costs in the investment cost
category, including procurement and military construction--are outside
the scope of COST's estimating capabilities.
Military components, including the services, are primary sources of
data for the tool. Table 2 illustrates the primary sources of data from
each component that uses COST to develop estimates for GWOT budget
requests.[Footnote 9] Cost factors and management factors are key types
of data that the tool uses to develop estimates. Cost factors function
as variables in the tool, and a few examples are hardship duty pay,
cost for operating and support of a ship, or average cost per ton mile
for equipment airlift. Cost factors are developed from four main
sources; DOD databases of record, service models for various statistics
such as Air Force flying hours, DOD's Cost of War reports, and other
information provided by the service budget offices. Management factors
include information such as the average metric tons per person of
materiel for deployment or redeployment of the monthly flying hours of
aircraft.
Table 2: Primary Sources of Data within COST from DOD Components:
Military component: Army;
Data source: Army Force and Organization Cost Estimating System Model.
Military component: Army;
Data source: Army Cost and Factors Handbook.
Military component: Army;
Data source: Operating and Support Management Information System (part
of DOD's Visibility and Management of Operating and Support Costs).
Military component: Air Force;
Data source: Air Force Portal.
Military component: Air Force;
Data source: Air Force Instruction 65-503.
Military component: Navy;
Data source: Navy Visibility and Management of Operating and Support
Costs.
Military component: Navy;
Data source: Navy Budget Office.
Military component: Marine Corps;
Data source: I Marine Expeditionary Force.
Military component: Navy;
Data source: II Marine Expeditionary Force.
Military component: Special Operations Command (SOCOM);
Data source: SOCOM.
Military component: All;
Data source: Defense Finance and Accounting Service.
Military component: All;
Data source: Defense Health Program.
Military component: All;
Data source: Transportation Command.
Military component: All;
Data source: Joint Operation Planning and Execution System.
Source: GAO analysis of DOD and IDA provided information.
[End of table]
Services Vary in Their Reliance on COST to Develop GWOT Budget
Estimates:
The services use COST as required as part of their process to develop a
GWOT budget request, but the degree to which each service relies on the
COST results differs. Both DOD officials and IDA representatives state
that COST is better suited for Army ground forces and was primarily
intended to develop incremental[Footnote 10] personnel and operations
cost estimates for operations with discreet phases and time frames.
They noted it was not built to estimate costs for the long-term nature
and broad scope of activities and needs related to operations such as
GWOT. Therefore, service officials report varying degrees of confidence
in the tool's functionality and accuracy for their specific service.
The Army relies on COST's results when developing estimates related to
personnel and operations, while Air Force, Navy, and Marine Corps
officials often rely more on historical obligation data and other
information than the COST-generated estimates. COST does not currently
have the capability to estimate other types of costs, such as
procurement, most equipment maintenance, and some contract costs. As a
result, all the services develop this portion of GWOT budget requests
using historical obligation data and other information.
Use of COST Is Required as Part of DOD's GWOT Budget Request Process:
DOD components are required to use COST as part of the GWOT budget
request process. The DOD financial management regulation that provides
financial policy and procedures for small-, medium-, and large-scale
campaign level military contingency operations requires that COST be
used to develop a cost estimate for the deployment of military
personnel and equipment. The regulation further states that the DOD
Comptroller will issue specific guidance providing factors and cost
criteria necessary to develop an estimate, and that the COST estimate
will address the funding requirements for operations and maintenance
and military personnel costs. The DOD Comptroller issues guidance that
directs the development of a fiscal year's GWOT budget request. This
guidance specifically directs the use of COST to calculate operations
costs related to GWOT, and also provides information guiding COST use,
such as the level and intensity of operations to be assumed. Much of
the guidance details the type and level of detail that must be provided
in supporting materials that should accompany components' estimated
GWOT budget requests.
COST does not develop estimates for items that are not attributable to
the deployment or sustainment of personnel and equipment, such as
procurement, most types of equipment maintenance, and certain major
contracted needs and services. As discussed later, the services must
use other processes to develop this portion of their GWOT budget
requests. Neither DOD Comptroller guidance nor DOD financial management
regulations prescribe the use of any particular method of developing
estimates for these categories. A 2006 memo from the Deputy Secretary
of Defense expanded allowable costs for GWOT in several categories,
especially reset-related procurement and equipment
maintenance.[Footnote 11] These types of costs accounted for about 70
percent of the total GWOT budget request for fiscal year 2008, which is
a significant increase over previous years. Due to this increase in
costs outside the tool, the COST-related portion of GWOT budget
requests fell from about 80 percent to about 30 percent of the total,
although the amount of the COST-generated estimate remains stable at
between $40 billion and $55 billion per year.
The Army Relies More on COST to Develop Personnel and Operations
Estimates, but the Other Services Rely on Their Own Estimates:
The Army uses COST as intended by relying on the tool to generate an
estimate for many personnel and operations costs for GWOT budget
requests. During the Army's development of an estimate using COST,
minor adjustments to COST's standard settings are made to better match
realities on the ground. For example, an official developing an
estimate in COST might reduce the costs for the transportation of
equipment if a unit will be using equipment already in theater instead
of taking equipment with them. DOD officials and IDA representatives
stated that COST is better suited for Army ground forces; therefore,
the Army relies on the final estimate developed by COST and submits
this information to the DOD Comptroller as part of its GWOT budget
request. About 40 to 45 percent of the Army's final GWOT budget
requests are typically for the operation and maintenance category of
appropriations, and an Army budget office official stated that the
majority of this portion is estimated by COST. Army officials further
stated that COST is an effective tool for cost estimation because it is
frequently updated with cost data the Army submits to IDA. An Army
model and database that contain cost information for personnel and
equipment are the sources for much of the Army-related data used by
COST and also are primary sources for developing the Army's base budget
requests.
The Air Force, Marine Corps, and Navy all fulfill the requirement to
develop an estimate for their respective service's personnel and
operations funding requirements for GWOT using COST; however, these
services significantly alter the COST results to match estimates they
have developed outside COST, using historical obligation data and other
information. Officials from the Air Force, Marine Corps, and Navy
budget offices reported various concerns regarding the functionality
and accuracy of COST as reasons for relying more on historical
obligation data and other information. Several budget officials from
each of these services reported that COST routinely overestimated some
costs. As a result, most of the changes they make to COST results,
based on historical obligation data, are decreases in the amount
estimated by COST. For example, a service budget official stated that
for one fiscal year's GWOT budget request, the total COST-developed
estimate was $100 million more than the estimate developed by the
service for the same types of costs using historical obligation data.
Specifically, COST overestimated transportation costs by about $275
million, while underestimating certain personnel support costs by about
$200 million, among other discrepancies. Navy officials stated that
COST often overestimated some types of transportation costs and the
results had to be manually adjusted to match historical obligation data
or other information. Similarly, Marine Corps officials reported that
most adjustments they make to the COST output result in a decrease of
the estimate. An Air Force budget official stated that COST
overestimated some transportation costs by about $1 billion in a prior
year's estimate, while other costs were not captured and therefore
underestimated. While these discrepancies were adjusted prior to
submission as a GWOT budget request, this official stated that an IDA
representative had since suggested strategies to develop a more
accurate transportation estimate in future uses of the tool.
Furthermore, a Navy official stated that COST is unable to project
certain costs associated with civilians, and hence might underestimate
the total costs due to this exclusion. For example, the official stated
that COST does not automatically estimate costs for civilian support
positions associated with an operational unit, such as a ship or ground
unit. Aside from accuracy concerns, officials from the Air Force,
Marine Corps, and Navy reported that COST and the cost breakdown
structure that forms the basis of COST's organization and resulting
estimations better represent Army ground forces than the unique
characteristics of the other forces. Navy officials stated that COST
automatically estimates food, ice, and water for all units because
deployed ground forces require these items. However, the Navy funds
these items for sailors on deployed ships through base budget funding
because these costs are incurred regardless of a ship's location. The
tool has not been refined to accurately estimate these costs;
therefore, Navy officials must manually remove these types of costs
from an estimate for GWOT funding. Several service officials stated
that, because of the limitations to COST, the required process of using
COST to develop a cost estimate was duplicative of their preferred
method of using historical obligation data and other information better
suited to their specific service to develop a GWOT budget request.
Service officials stated that COST is a useful tool for estimating
costs for small-scale and short-duration operations or for situations
for which information is unknown or new, such as the recent troop surge
or for other general rough-order-of-magnitude estimates produced early
in operation planning while options are being weighed by decision
makers. However, officials stated the tool does not perform as well for
estimating costs for the lengthy deployment and sustainment phases
associated with a large campaign such as GWOT. Furthermore, COST is not
able to estimate all costs associated with GWOT.
Military Services Use Other Methods to Determine Cost Estimates for
Procurement, Most Equipment Maintenance, and Some Contract Costs:
Because COST does not have the capability to estimate costs such as
procurement, reset-related equipment maintenance, and contracted needs
and services, service budget officials report using historical
obligation data, other models and formulas, and other information, such
as deployment information, to estimate these costs. Reset-related
procurement estimates for GWOT are devised in multiple ways. For
example, to estimate costs to procure new equipment to replace lost or
damaged equipment, officials stated that incident reports are tracked
to provide information on how many pieces of equipment are needed to
replace battle losses. For procurement to replace equipment that has
reached the end of its useful life because of GWOT's higher operating
tempo, formulas, based on historical data, provide information on the
normal extent of wear and tear for certain types of equipment, and wear
above the normal extent is attributed to GWOT.[Footnote 12] Most types
of equipment maintenance are not estimated by COST, such as
intermediate-and depot-level maintenance; therefore the services again
rely on historical obligation data to develop estimates for the GWOT-
related costs. For example, Army logistics officials track the units
that are scheduled for redeployment and develop estimates for the cost
of resetting a particular unit's equipment based on the type of brigade
the unit is part of, such as a heavy brigade combat team or Stryker
brigade combat team, and the average cost of resetting that type of
brigade unit developed from historical obligation data, adjusted for
inflation. For other items outside the scope of COST--such as the
Logistics Civil Augmentation Program costs for the Army, intelligence
needs, and contracts for other needs such as linguists--functional
experts within each service provide the service budget offices with
information to develop estimates. This information is based on contract
task orders or needs assessments developed by in-theater commanders.
For example, in-theater commanders submit requests for additional
linguists to Army intelligence officials, and the estimated cost for
these linguists is developed based on historical costs for the same
type of linguist. Army budget officials stated that contracting costs,
including the Logistics Civil Augmentation Program, linguist, and
security services, are one of the most expensive cost categories that
falls outside of COST. Officials from all services stated that, after
many years of ongoing operations in support of GWOT, they believe few
requirements are truly unknown or based on emerging needs, so therefore
they are comfortable relying on historical obligation data and other
information to develop estimates for these types of costs. For an
example of how DOD develops a GWOT budget request and the use of COST
and other methods of developing an estimate, see appendix III.
COST Is Frequently Updated and Refined, but Could Benefit from a Review
of Adherence to Best Practices:
DOD has taken steps to improve the performance and reliability of COST;
however, COST could benefit from a review of the tool's adherence to
best practices for high-quality cost estimation as outlined in our Cost
Assessment Guide. Revisions have been made to the tool to improve its
performance, and frequent updates are made to the data used by the
tool. However, a review of COST according to the best practices for
cost estimation could provide decision makers with information on the
extent to which the tool generates reliable estimates and identify
opportunities for improvement. Our guide defines high-quality cost
estimates as well documented, comprehensive, accurate, and credible.
While we did not undertake a full assessment of COST against best
practices, during the course of our review we identified features of
COST's estimation process that meet best practices and other features
that would benefit from further review. For example, COST adheres to
several best practices for a comprehensive and accurate cost estimate,
such as frequent updates to the structure of COST and the data that
COST uses to generate estimates. While COST appears to largely
encompass the types of costs that are incurred to deploy and sustain
Army ground forces and their related equipment, COST may not
comprehensively and accurately estimate costs for the other services.
COST also relies on GWOT obligation data from DOD's Supplemental and
Cost of War Execution Reports that we have identified as being of
questionable reliability, and DOD is taking steps to improve.[Footnote
13] These might be areas for which a thorough and full review of the
tool could improve the resulting estimates that are used to develop
GWOT budget requests.
COST Revised and Frequently Updated in an Effort to Improve
Effectiveness:
IDA has made changes to the structure of COST, either at the request of
the DOD Comptroller or the services, or on its own initiative, and has
refined the tool many times over the past several years to improve
functionality or performance. Recent refinements to COST included
changes that provide the ability to alter the percentage of officer and
enlisted personnel within a unit and the types of diagnostic and
summary reports that the tool can create. A 3-year development effort
culminated in June 2007 with the release of a new version of COST
supported by new software and hardware that increased functionality and
performance. This new version allows the user to simultaneously develop
estimates for multiple operations within the same contingency. The
services and others frequently submit new information for tool updates.
For example, components are asked to submit updated cost factor data
prior to the development of any request for supplemental emergency
funding. Additionally, the DOD Comptroller has asked IDA to review the
inputs, assumptions, and processes the services used to generate COST
estimates for GWOT budget requests since fiscal year 2005. The reviews
revealed issues that were consistent across the services or significant
enough to warrant attention in future use of COST. For example, a
review found that COST users estimated an excessive use of airlift for
the movement of cargo with no scheduled cargo for the return flight.
Additionally, a review identified confusion regarding the use of
different operational tempo factors and pay offsets. IDA consolidated
these and other issues identified in the review of fiscal years 2007
and 2008 into a lessons learned briefing and checklist to assist the
services as they use COST in the development of future GWOT budget
requests.
COST Could be Assessed for Adherence to Cost Estimation Best Practices:
While DOD has taken steps to revise and update COST to improve
effectiveness, COST has not been assessed according to best practices
for cost estimation that define reliable, high-quality cost estimates.
A review of regulations, guidance, and best practices for cost
estimation and best practices established by professional cost
analysts, and compiled in our Cost Assessment Guide, identified four
characteristics of high-quality, reliable cost estimates.[Footnote 14]
As shown in table 1, cost estimates should be well documented,
comprehensive, accurate, and credible. DOD Comptroller officials stated
they are confident in the tool's ability to provide reasonable
estimates because COST is frequently updated. However, neither DOD, nor
any other entity, has assessed COST and its resulting cost estimates
against these best practices. While we did not perform a full
assessment of COST against best practices, during the course of our
work we identified some features of COST that meet best practices for
cost estimation and other areas that could benefit from further review.
* Well Documented: A well documented cost estimate is based on data
that have been gathered from actual historical costs and technical
experts, analyzed for cost drivers, and collected from primary sources.
These best practices appear to be met by COST. Additionally, any
adjustments made to COST's standard settings are flagged and must be
accompanied by an explanatory note that details why information was
changed, and these situations are reviewed by DOD Comptroller
officials. Furthermore, best practices also require that previous cost
factors and data are stored after updates so that an estimation process
is repeatable and can be later verified. The newest version of COST
does have this capability and IDA maintains records of cost factors and
other data. However, best practices also require that data used in a
model should be traced back to the source documentation and any
normalization steps should be documented. The services are responsible
for ensuring data are reliable and neither IDA nor DOD, including the
services, traces all data back to the source documents. Additionally,
IDA officials stated that unusually high or low data might be removed
and no record of these actions would be kept, and best practices
require that these sorts of steps should be documented. Further review
could reveal if these or other areas might need more work to ensure the
estimate is well documented.
* Comprehensive: Estimates for personnel and operations costs developed
by COST appear to meet several, but not all, of the criteria for
comprehensive cost estimates. For example, the cost breakdown
structure, which defines the cost elements within COST and forms the
foundation of formulas within the tool, has more than three levels of
detail, the structure is updated as changes occur, and each element is
defined in a cost breakdown structure dictionary included in the
financial management regulation for contingency operations. These steps
are all considered best practices for a comprehensive cost estimate.
However, our analysis of the cost breakdown structure in the financial
management regulation revealed that there are some errors and
ambiguities in the structure's definitions that might allow for double
counting of costs.[Footnote 15] Furthermore, while COST appears to
largely encompass the types of costs that are incurred to deploy and
sustain Army ground forces and their related equipment, it may not
comprehensively estimate some of the types of costs incurred by the
other services. For example, COST does not develop an estimate for the
costs of certain Navy ground support units, such as intelligence, that
are not associated with a naval fleet. Moreover, COST is not used by
the Air Force to develop estimates for the cost of transporting people
in non-combat situations, such as the transport of military or civilian
personnel from the International Zone to a forward operating base, for
example. An Air Force official stated that this is because the tool
automatically assumes that any flying hour-related expenses are
operational or combat-related and these costs should instead be
attributed to transportation-related expenses. Further review could
identify cost data or formulas within the tool that could be refined to
better suit the other services or might reveal some types of costs for
which COST should not be used to develop an estimate. Additionally, the
tool does not comprehensively estimate all GWOT costs, such as
procurement and many types of equipment maintenance. The GWOT cost
estimate presented in appendix III illustrates the revisions made to
COST results and the types of costs that were estimated outside of COST
for a particular estimate that was to be comprised primarily of
military personnel and operations costs.
* Accurate: Similarly, the tool's estimates appear to meet some, but
not all, of the best practices for accuracy. For example, an accurate
estimate should be based on cost factors that reflect updates and
changes. IDA does ask the components to submit updated cost factors,
which are sometimes based on historical obligation data, prior to every
run of the tool for a fiscal year's GWOT request, but the components
are not required to update the factors. IDA reviews the cost factor
submission to ensure general consistency across years, but the services
and other components that submit data are ultimately responsible for
the data and are not required to validate the data prior to their
inclusion into the tool. According to IDA representatives, IDA does not
validate the data sources or obtain assurances that the data submitted
are reliable, because this requirement is not included in its contract
with DOD. The services do not validate all data submitted to IDA for
updates, nor do they submit updates for all factors that need updating.
Additionally, some cost factors are developed from the Cost of War
reports and other data are compared against these reports as a
validation check. These practices might benefit from review since our
previous work has raised concerns about the reliability of reported
GWOT obligation data. For example, we have reported that there is a
lack of transparency over certain obligations in the Cost of War
reports and we have identified inaccuracies in these reports.
Consequently, we were unable to ensure that DOD's reported obligations
for GWOT are complete, reliable, and accurate, and believe that they
should therefore be considered approximations. However, we acknowledge
that DOD has taken steps to address our recommendations and the
department has several initiatives underway to further improve the
reliability of GWOT obligation data. Additionally, according to service
officials, many adjustments to COST results for transportation and
other types of costs are made to decrease estimates. This raises
concerns regarding the accuracy or applicability of some data in COST.
* Credible: Many of the best practices associated with a credible cost
estimate require sensitivity analysis, risk and uncertainty analysis,
and a comparison against an independent cost estimate to be
performed.[Footnote 16] Sensitivity analysis has been performed to
understand cost drivers and this type of analysis can be performed
during estimate development as characteristics of the operation change.
However, IDA representatives stated that risk analysis is unnecessary
since the largest sources of risk stem from the changing and
unpredictable nature of warfare and policy changes that also cannot be
predicted. Additionally, uncertainly analysis can assess the impact
that the variability of certain unknown factors will have on resulting
estimates. For example, uncertainty analysis of possible fuel price
changes could result in various cost scenarios that might occur
depending on future fuel prices. Finally, cost results should be
compared to an independent cost estimate which is another best practice
for a credible cost estimation process. A Joint Staff or other official
will often develop a COST estimate to compare against an estimate
developed by a service official to ensure the appropriate assumptions
were used. A thorough assessment against best practices would reveal if
these or other areas might need more work to ensure the estimate is
sufficiently credible.
Conclusions:
Estimating needs and costs that will occur in the future is not an
exact science, and the use of cost estimation tools or historical
obligation data, along with other factors, can be reasonable means of
projecting future costs. However, due to the significant and ever-
increasing size of GWOT budget requests, every attempt should be made
to ensure that a cost estimation process is sound. COST has been used
to generate hundreds of billions of dollars in budget requests for GWOT
over the past several years and the services are required to use it.
While the tool and underlying data have been refined and updated,
COST's overall effectiveness for estimating GWOT costs has not been
assessed. Best practices criteria to measure COST's extent of
sufficient documentation, comprehensiveness, accuracy, and credibility
could provide additional information about the tool's effectiveness at
generating high-quality cost estimates and steps of the process that
could be further improved. Without a thorough review of COST according
to these best practices, decision makers within both DOD and Congress
cannot be assured that estimates generated by the tool are developed
using valid data and sound processes. Officials from across DOD and the
services report that COST performs well for predicting budget needs for
ground forces and for small-scale operations of short duration, or for
situations in which detailed information is unknown. However, in light
of service officials' concerns that COST does not perform as well for
their needs and might generate estimates that are too high in certain
areas, it is important that DOD review the applicability of COST for
all of the services or investigate ways in which to make the tool
better suit the needs of all the services.
Recommendations for Executive Action:
To ensure that DOD budget requests for GWOT are based on a sound cost
estimation process, we recommend that the Secretary of Defense direct
the Under Secretary of Defense (Comptroller) to:
* Arrange for an independent review of COST against best practices for
cost estimation.
Based on that review, and taking into consideration how each service
uses COST, we recommend that the Secretary of Defense direct the Under
Secretary of Defense (Comptroller) to:
* Consider options for refining COST, determine the appropriate items
or types of costs for which COST should be applied, and identify
methods to be used when COST is not appropriate.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD agreed with both of
our recommendations for executive action. These written comments
additionally provided examples of steps DOD has taken or plans to take
that it considers actions that address aspects of our recommendations.
Also, DOD provided us with technical comments which we incorporated in
the report where appropriate. DOD's comments are reprinted in appendix
IV.
DOD agreed with our recommendation that the DOD Comptroller arrange for
an independent review of the Contingency Operations Support Tool (COST)
against best practices for cost estimation. In its comments on this
recommendation, DOD agreed with the concept of an independent review of
the tool against best practices, and further noted that the Air Force
Studies and Analyses Agency (AFSAA) conducted a review of COST's use in
developing cost estimates for the air war over Serbia. This review
compared the tool's output against the actual reported costs compiled
by the Defense Finance and Accounting Service. DOD stated that the
recommendations for improvement identified in the review were
incorporated into the tool. While the review and the incorporation of
its recommendations into the tool are positive steps, the review did
not assess COST's use in estimating the broad scope of costs that are
associated with a large-scale campaign such as GWOT. For example, the
scope of the AFSAA review of COST was limited to the 3-month air war
over Serbia and primarily reviewed costs of the Air Force. Furthermore,
the AFSAA review was not a thorough review against best practices for
cost estimation, which requires cost estimates to be well documented,
comprehensive, accurate, and credible. Therefore, we continue to
recommend that an independent and thorough review of the tool against
best practices for cost estimation be pursued. This type of review
would include an assessment of the risk and uncertainty associated with
the inputs to the tool and the accuracy of the underlying equations and
data the tool relies on to estimate costs. DOD additionally stated that
(1) COST's factors, processes, and algorithms are updated as needed and
the tool is updated to reflect changes to congressionally determined
factors, such as military pay rates, and (2) COST relies on many of the
same service-specific cost factors that are used during the development
of baseline budgets. While we acknowledge in this report that factors
and other aspects of the model are regularly updated, these actions are
a check for accuracy by the users of the data and our recommendation
specifically calls for an independent review of COST. Finally, DOD
stated that, due to our recommendation, the DOD Comptroller will issue
guidance that will be incorporated into DOD's financial management
regulation that includes a process for updating COST to ensure it
reflects the most current budgetary assumptions and a process for
evaluating the functionality of the model to determine if adjustments
are needed. As this revision of the financial management regulation has
not been finalized, we did not assess this planned action. However,
this positive step, once completed, should be taken into account as
part of an independent and thorough review of COST against best
practices for cost estimation.
DOD agreed with our recommendation that, based on an independent
review, it should consider options for refining COST, determine the
appropriate items or types of costs for which COST should be applied,
and identify methods to be used when COST is not appropriate. In its
comments, DOD stated that it uses every opportunity to refine and
improve the COST model, such as the multi-level review of COST results
and assumptions during the development of GWOT budget requests.
Additionally, changes to COST are made based on training sessions and
feedback from COST users. Finally, DOD stated that an extensive review
process is in place for each non-COST line item of GWOT budget
requests. While we did not assess the process DOD uses to review non-
COST line items, our report acknowledges that DOD has refined and
updated COST many times as information has changed and the needs of the
department have evolved. However, we reiterate our view that the tool
should be subject to an independent review, and COST should be further
refined based on the findings of that review, as needed.
We are sending copies of this report to other interested congressional
committees; the Secretary of Defense; the Under Secretary of Defense
(Comptroller), and the Director, Office of Management and Budget.
Copies of this report will also be made available to others upon
request. In addition, this report will be available at no charge on the
GAO Web site at [hyperlink, http://www.gao.gov].
If you have any questions regarding this report, please contact Sharon
Pickup at (202) 512-9619 or pickups@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix V.
Signed by:
Sharon L. Pickup:
Director, Defense Capabilities and Management:
List of Congressional Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Kent Conrad:
Chairman:
The Honorable Judd Gregg:
Ranking Member:
Committee on the Budget:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan L. Hunter:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable John M. Spratt, Jr.:
Chairman:
The Honorable Paul Ryan:
Ranking Member:
Committee on the Budget:
House of Representatives:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To assess how the Department of Defense (DOD) uses the Contingency
Operations Support Tool (COST) and other processes to develop Global
War on Terror (GWOT) budget requests, we reviewed and analyzed relevant
documents and interviewed key DOD and service officials and
representatives from the Institute for Defense Analyses (IDA).
Documents that we used for our review included, but were not limited
to, (1) relevant DOD directives, instructions, and memoranda related to
budgeting processes; (2) DOD financial management regulations that
provide policy and procedures for contingency operations; (3) DOD
guidance for the preparation and submission of requests for incremental
funding for GWOT; and (4) service budget office guidance for the
preparation and submission of GWOT budget requests. We obtained
testimonial evidence from officials representing the Office of the
Under Secretary of Defense (Comptroller) and the Joint Staff regarding
the processes used to develop GWOT budget requests and the role of COST
in that process. Specifically, we obtained their perspectives on COST's
effectiveness and accuracy, as well as the processes employed to
develop estimates for costs that are outside the scope of the tool's
estimating capabilities. We similarly interviewed key service officials
in the financial management or budget office responsible for developing
GWOT cost estimates for contingency operations in the Air Force, Army,
Marine Corps, and Navy to understand their experiences using COST
during the development of prior years' GWOT budget requests, including
strengths and weaknesses of the tool. We additionally interviewed
service officials that were identified as functional experts for the
types of costs that must be estimated outside of the tool, such as
procurement, reset-level equipment maintenance, and intelligence needs.
We discussed what processes they use to develop estimates for these
types of costs. We attended several briefings regarding COST presented
by IDA and attended training sessions on the tool. We interviewed IDA
representatives about the tool and reviewed numerous briefings about
COST's use in cost estimation, the structure of the tool, and the
tool's development. Finally, in order to understand how DOD developed
an actual GWOT budget request and the use of COST and other methods to
develop that estimate, we asked DOD to demonstrate how an estimate was
developed for a case study, which was the $6.3 billion estimate for
military operations that was included as part of DOD's October 2007
$42.3 billion amendment to the Fiscal Year 2008 GWOT supplemental
request for emergency funding. We chose this example as our case study
because it was a recent estimate and was assumed to be comprised
primarily of military personnel and operations costs, due to the
description of this estimate in the justification document for the
amendment to the Fiscal Year 2008 GWOT budget request.[Footnote 17] DOD
provided the initial and approved estimate for this request by service
and by appropriation category. We discussed this estimate with DOD and
service officials, including the assumptions and processes that were
used to develop this estimate, the reasons for changes between the
initial and approved estimates, and the types of costs estimated by
COST or outside of COST. We did not validate the assumptions used to
generate this estimate or the data DOD presented in this example.
To assess what actions DOD has taken to ensure COST adheres to best
practices for cost estimation, we reviewed applicable best practices
and compared DOD's efforts against those best practices we found to be
consistently associated with reliable, high-quality cost
estimation.[Footnote 18] We reviewed DOD guidance regarding
requirements for the verification, validation, and accreditation of
tools and simulations used by DOD. We reviewed documents obtained from
DOD and IDA and had discussions with DOD officials and IDA
representatives about COST; for example, we reviewed numerous briefings
about COST and the training manual that documents the tool's
specifications, development, and use in detail. We interviewed DOD
Comptroller and IDA representatives about improvements and updates that
have been made to COST and the purpose of those updates. We obtained
testimonial evidence from DOD Comptroller officials and IDA
representatives about the steps to develop COST and the processes that
surround the tool's use as part of developing a GWOT budget request,
and identified steps that appeared to meet certain criteria of
established best practices and those that appeared to warrant further
review. We did not perform a full review of COST against all best
practices, but presented examples of how the tool meets certain best
practices to provide some context to decision makers about what might
be considered strengths of the tool and we highlighted some areas that
might benefit from a full and independent review of COST. These
examples are meant to serve as illustrative detail to provide more
information to decision makers, but should not be considered the
results of a complete, thorough, and independent review of the tool.
We conducted this performance audit from July 2007 through September
2008 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: The 12 Steps of High-Quality Cost Estimating Mapped to
Best Practice Criteria:
Characteristic: Well documented: * The estimate is thoroughly
documented, including source data and significance, clearly detailed
calculations and results, and explanations for choosing a particular
method or reference;
* Data have been traced back to the source documentation;
* A technical baseline description is included;
* All steps in developing the estimate are documented, so that a cost
analyst unfamiliar with the program can recreate it quickly with the
same result;
* All data sources for how the data were normalized are documented;
* The estimating methodology and rationale used to derive each cost
breakdown structure element's cost are described in detail;
Related step: Define the estimate's purpose;
Define the program;
Identify ground rules and assumptions;
Obtain the data;
Document the estimate;
Present estimate to management.
Characteristic: Comprehensive: * The estimate's level of detail ensures
that cost elements are neither omitted nor double counted;
* All cost- influencing ground rules and assumptions are detailed;
* The cost breakdown structure is defined and each element is described
in a cost breakdown structure dictionary;
Related step: Develop the estimating plan;
Determine the estimating approach.
Characteristic: Accurate: * The estimate is unbiased, not overly
conservative or overly optimistic, and based on an assessment of most
likely costs;
* It has few, if any, mathematical mistakes;
those it has are minor;
* It has been validated for errors like double counting and omitted
costs;
* It has been compared to the independent cost estimate for
differences;
* Cross-checks have been made on cost drivers to see if results are
similar;
* The estimate is timely;
* It is updated to reflect changes in technical or program assumptions
and new phases or milestones;
* Estimates are replaced with the earned value management estimate at
completion and the independent estimate at completion from the
integrated earned value management system;
Related step: Develop the point estimate;
Update the estimate to reflect actual costs and changes.
Characteristic: Credible: * Any limitations of the analysis because of
uncertainty or biases surrounding data or assumptions are discussed;
* Major assumptions are varied and other outcomes recomputed to
determine how sensitive outcomes are to changes in the assumptions;
* Risk and uncertainty analysis is performed to determine the level of
risk associated with the estimate;
* The results are cross-checked and an independent cost estimate is
developed to determine if other estimating methods produce similar
results;
Related step: Conduct sensitivity analysis;
Conduct risk and uncertainty analysis.
Source: GAO.
[End of table]
[End of section]
Appendix III: An Example of DOD's GWOT Budget Estimate Process Using
the Sustainment and Redeployment of the Troop Surge:
To better understand how the Department of Defense (DOD) develops a
Global War on Terrorism (GWOT) budget request and the use of the
Contingency Operations Support Tool (COST) and other methods of
developing an estimate, we asked DOD to demonstrate how the services
used COST and other methods to develop the $6.3 billion budget estimate
for military operations that was included as part of DOD's October 2007
$42.3 billion amendment to the fiscal year 2008 GWOT supplemental
request for emergency funding. DOD justification materials for the
amendment describe this $6.3 billion portion as needed to support the
continued sustainment and redeployment of the five Army brigades and
two Marine Corps infantry battalions that were considered part of the
troop surge.[Footnote 19] This portion of the amendment was also
requested to support the simultaneous deployment of combat support
forces augmenting these combat forces and other costs related to the
presence of the troop surge. Table 3 presents DOD's estimate for this
$6.3 billion portion of the amendment meant to fund additional
operations, broken into the COST-generated portion and the portion that
was estimated by other means. Typically, service budget officials
develop cost estimates for GWOT, but officials stated that time
constraints for this specific estimate required that Joint Staff and
DOD Comptroller officials develop the initial estimate using both COST
and other processes outside of COST as necessary. After discussions
with service budget officials, and the September 10, 2007, testimony of
the Commander of the Multi-National Force-Iraq before Congress, the
estimates were revised. This is reflected in the approved estimate for
each service, again broken into the COST portion and the "non-COST"
portion. Each service's estimate is further broken down into estimates
for Military Personnel appropriations and Operation and Maintenance
appropriations.
Table 3: Example of DOD Estimation for Operations in Support of GWOT:
Dollars in millions.
Army Total;
Initial Estimate: COST Estimate: $4,107;
Initial Estimate: Non-COST Estimate: $775;
Initial Estimate: Total Estimate: $4,882;
Approved Estimate: COST Estimate: $3,918;
Approved Estimate: Non-COST Estimate: $1,837;
Approved Estimate: Total Estimate: $5,755.
Military Personnel;
Initial Estimate: COST Estimate: 731;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 731;
Approved Estimate: COST Estimate: 705;
Approved Estimate: Non-COST Estimate: (961);
Approved Estimate: Total Estimate: (256).
Operation & Maintenance (O&M);
Initial Estimate: COST Estimate: 3,376;
Initial Estimate: Non-COST Estimate: 775;
Initial Estimate: Total Estimate: 4,151;
Approved Estimate: COST Estimate: 3,213;
Approved Estimate: Non-COST Estimate: 2,798;
Approved Estimate: Total Estimate: 6,011.
Navy Total;
Initial Estimate: COST Estimate: $92;
Initial Estimate: Non-COST Estimate: $0;
Initial Estimate: Total Estimate: $92;
Approved Estimate: COST Estimate: $96;
Approved Estimate: Non-COST Estimate: $100;
Approved Estimate: Total Estimate: $196.
Military Personnel;
Initial Estimate: COST Estimate: 10;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 10;
Approved Estimate: COST Estimate: 10;
Approved Estimate: Non-COST Estimate: 30;
Approved Estimate: Total Estimate: 40.
O&M;
Initial Estimate: COST Estimate: 83;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 83;
Approved Estimate: COST Estimate: 87;
Approved Estimate: Non-COST Estimate: 69;
Approved Estimate: Total Estimate: 156.
Marine Corps Total;
Initial Estimate: COST Estimate: $25;
Initial Estimate: Non-COST Estimate: $0;
Initial Estimate: Total Estimate: $25;
Approved Estimate: COST Estimate: $146;
Approved Estimate: Non-COST Estimate: $163;
Approved Estimate: Total Estimate: $309.
Military Personnel;
Initial Estimate: COST Estimate: 6;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 6;
Approved Estimate: COST Estimate: 25;
Approved Estimate: Non-COST Estimate: 163;
Approved Estimate: Total Estimate: 188.
O&M;
Initial Estimate: COST Estimate: 19;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 19;
Approved Estimate: COST Estimate: 121;
Approved Estimate: Non-COST Estimate: 0;
Approved Estimate: Total Estimate: 121.
Air Force Total;
Initial Estimate: COST Estimate: $103;
Initial Estimate: Non-COST Estimate: $0;
Initial Estimate: Total Estimate: $103;
Approved Estimate: COST Estimate: $19;
Approved Estimate: Non-COST Estimate: $25;
Approved Estimate: Total Estimate: $44.
Military Personnel;
Initial Estimate: COST Estimate: 11;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 11;
Approved Estimate: COST Estimate: 4;
Approved Estimate: Non-COST Estimate: 0;
Approved Estimate: Total Estimate: 4.
O&M;
Initial Estimate: COST Estimate: 92;
Initial Estimate: Non-COST Estimate: 0;
Initial Estimate: Total Estimate: 92;
Approved Estimate: COST Estimate: 15;
Approved Estimate: Non-COST Estimate: 25;
Approved Estimate: Total Estimate: 40.
SOCOM;
Initial Estimate: COST Estimate: $10;
Initial Estimate: Non-COST Estimate: $0;
Initial Estimate: Total Estimate: $10;
Approved Estimate: COST Estimate: $7;
Approved Estimate: Non-COST Estimate: $0;
Approved Estimate: Total Estimate: $7.
Total Defense-Wide;
Initial Estimate: COST Estimate: $4,337;
Initial Estimate: Non-COST Estimate: $775;
Initial Estimate: Total Estimate: $5,112;
Approved Estimate: COST Estimate: $4,186;
Approved Estimate: Non-COST Estimate: $2,125;
Approved Estimate: Total Estimate: $6,311.
Source: DOD.
[End of table]
As shown in table 3, the Army's estimate of nearly $5.8 billion
comprises the majority of the $6.3 billion total estimate. COST-
generated estimates for military personnel or operation and maintenance
did not change significantly for the Army from its initial estimate to
the approved final estimate. Substantial changes were made, however, to
the portions of the estimate that are outside the scope of COST. Army
budget officials reported that the large decrease in requested military
personnel funding was due to lower mobilization levels than originally
predicted, adjusted overstrength levels, and reduced permanent change
of station and subsistence costs. The COST portion for both military
personnel and operations for the Army decreased slightly from the
initial to the approved estimate. Army budget officials reported that
the approximately $2 billion increase in operation and maintenance
costs that was derived outside of COST was due to force protection and
other equipment or services needed to support the troop surge and the
additional operations performed by surge troops. Table 4 provides more
details on these costs.
Table 4: Non-COST estimates for Army Operation and Maintenance Costs:
Operation and Maintenance Cost Estimated: Force protection;
Amount Estimated: $255.
Operation and Maintenance Cost Estimated: Theater maintenance support;
Amount Estimated: 763.
Operation and Maintenance Cost Estimated: Communications support;
Amount Estimated: 198.
Operation and Maintenance Cost Estimated: Predeployment training;
Amount Estimated: 447.
Operation and Maintenance Cost Estimated: CONUS based support;
Amount Estimated: 70.
Operation and Maintenance Cost Estimated: Detainee operations;
Amount Estimated: 251.
Operation and Maintenance Cost Estimated: Linguistics and intelligence
support;
Amount Estimated: 200.
Operation and Maintenance Cost Estimated: LOGCAP;
Amount Estimated: 242.
Operation and Maintenance Cost Estimated: Subsistence transportation;
Amount Estimated: 373.
Total Non-COST Estimate;
Amount Estimated: $2,798.
Source: Army.
Note: Total may not add due to rounding.
[End of table]
Similarly, major adjustments were made to the non-COST portions of the
Air Force, Marine Corps, and Navy estimates for either military
personnel costs, operation and maintenance costs, or both appropriation
categories, depending on the service. These changes were due to changed
assumptions regarding deployment and specialized needs. For example,
the Navy's non-COST personnel estimate was increased for costs
associated with permanent change of station and active duty for special
work needs. The Marine Corps increased its non-COST personnel request
by $163 million in anticipation of increased requirements for reserve
and Individual Ready Reserve activations to active duty. The
adjustments to the Marine Corps and Air Force COST-generated portions
were likewise substantial. The Marine Corps' increase for personnel
costs reflected in the COST-attributed portion reflects an anticipated
increase in counter-insurgency operations in Afghanistan, while the Air
Force estimate was refined to exclude a KC-10 and the Air National
Guard and Air Force Reserve personnel that would accompany the KC-10.
From this case study, it is clear that changing the assumptions
regarding the deployment of personnel and equipment can have
substantial impact on the COST-related portion of a GWOT budget
request. Also, significant changes were made to the non-COST portion of
the request, which in this example increased from $775 million to about
$2.1 billion, including the nearly $1 billion offset in Army personnel
cost estimates. The information regarding the non-COST portion of this
example reveals the significance and size of estimates generated
outside the tool, even in a situation where the majority, if not all,
of the costs would be assumed to be related to personnel and
operations.
We did not validate any of the data DOD presented in the above
discussion, or any of the assumptions or other information used by DOD
to develop this estimate.
[End of section]
Appendix IV: Comments from the Department of Defense:
Under Secretary Of Defense:
1100 Defense Pentagon:
Washington DC 20301-1100:
August 29, 2008:
Ms. Sharon Pickup:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street N.W.:
Washington, DC 20548
Dear Ms. Pickup:
Enclosed is the Department of Defense (DoD) response to the Government
Accountability Office (GAO) Draft Report GAO-08-982, Defense Budget:
"Independent Review is Needed to Ensure DOD's Use of Cost Estimating
Tool for Contingency Operations Follows Best Practices" dated July 28,
2008.
The Department concurs with comment on both of the recommendations for
executive action and has taken action to address them. Thank you for
the opportunity to provide the Department's response to GAO's
recommendations.
Sincerely,
Signed by:
for:
Tina W. Jonas:
Enclosure:
As stated:
Department of Defense Comments:
GAO CODE 351074/GAO-08-982:
"Defense Budget: Independent Review is Needed to Ensure DoD's Use of
Cost Estimating Tool for Contingency Operations Follows Best
Practices":
Department Of Defense Comments To The Recommendations:
Discussion:
* The GAO report outlined two recommendations for executive action.
* The Department of Defense concurs with comment on both items for
executive action.
* In addition. technical comments are provided in response to the draft
report.
Recommendation 1 :The Secretary of Defense direct the USD(C) to
"arrange for an independent review of the Contingency Operations
Support Tool [COST] against best practices for cost estimation."
DoD Comment To Recommendation 1: Concur with comment.
* The Department concurs with the concept of an independent review of
the COST model against best practices for cost estimation. In fact, the
Air Force Studies and Analyses Agency (AFSAA) conducted such a review
in 2004 in their report titled "Contingency Operations Support Tool
Evaluation: The Air War over Serbia." The report includes a comparison
of the model output to actual reported costs compiled by Defense
Finance and Accounting Service. Many of the issues that AFSAA
articulated within the report were incorporated by the Department
resulting in improvements in the model.
* On an annual, informal, basis, the Services' provide recommendations
for improvements to the model that would better meet the needs of each
individual Service. The factors, processes, and algorithms are updated
to reflect the most recent assumptions for each Service, as required.
Consistent with the intent of GAO's recommendation and effective
immediately, the Under Secretary of Defense (Comptroller) has issued
guidance that will be incorporated in the Financial Management
Regulation. The guidance includes a process for updating the COST model
to ensure it reflects the most current budgetary assumptions and a
process for evaluating the functionality of the model to determine if
adjustments are needed
* In addition, the COST model is updated to reflect congressionally
determined factors such as military personnel pay-rates as those
legislative changes become law. All of these procedures bolster the
continual update and refinement of the model.
* Finally each Service uses models to varying degrees to create and
review cost factors for their baseline budgets. The COST model uses
many of these same factors. The Services have procedures in place to
validate their models. These validated factors serve as the basis for
the Services' baseline budgets and are the foundation upon which
supplemental requirements are developed.
Recommendation 2: The Secretary of Defense direct the USD(C) to
"consider options for refining COST, determine the appropriate items or
types of costs for which COST should be applied, and identify methods
to be used when COST is not appropriate."
DoD Comment To Recommendation 2: Concur with comment.
* The Department uses every opportunity to refine and improve the COST
model for comprehensiveness and accuracy. For example, during a
supplemental build OSD Comptroller, the Joint Staff; Institute for
Defense Analyses and Service personnel review the model results and
critique assumptions, factors, and outputs of the COST model. The
results of the review are used to adjust the Supplemental amounts and
for initiating improvements to the COST model.
* In addition, annual training involving all COST users is conducted
before supplemental requests are developed. The feedback and exchange
between the attendees (Services, OUSD(C), the Joint Staff and on
occasion the Office of Management and Budget) has resulted in changes
to the model that made it more useful and accurate. The COST model has
proven itself to be a dynamic and adaptable tool that is used
successfully by different organizations in DoD to create estimates for
a variety of force deployments.
* As for supplemental requirements that are not calculated in the COST
model, the Department has developed an extensive review process for
each non-COST line item. Each Service must submit all non-COST model
items to OUSD(C) at the line items level of detail which are then
compared to historical execution and/or anticipated operational needs.
Only those line items that are fully justified are included in the
supplemental request.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Sharon Pickup, (202) 512-9619 or pickups@gao.gov:
Acknowledgements:
In addition to the contact named above, Ann Borseth (Assistant
Director), Grace Coleman, Susan Ditto, Linda Keefer, Lonnie McAllister
II, Lisa McMillen, Charles Perdue, Suzanne Perkins, Karen Richey, and
Karen Werner made key contributions to this report.
[End of section]
Footnotes:
[1] After the terrorist attacks of September 11, 2001, the President
announced a global war on terrorism requiring the collective
instruments of the entire federal government to counter the threat of
terrorism. Ongoing military and diplomatic operations overseas,
especially in Iraq and Afghanistan, constitute a key part of GWOT.
These operations involve a wide variety of activities, such as
combating insurgents, civil affairs, capacity building, infrastructure
reconstruction, and training military forces of other nations.
[2] DOD, Financial Management Regulation, 7000.14-R, vol. 12, ch. 23
(September 2007).
[3] A January 2007 DOD memorandum defines reset as actions to restore
units to a desired level of combat capability, including maintenance
and supply activities that restore and enhance equipment that was
destroyed, damaged, stressed, or worn out beyond economic repair due to
combat operations by repairing, rebuilding, or procuring replacement
equipment.
[4] For the purpose of this report, we refer to the Office of the Under
Secretary of Defense (Comptroller) as the DOD Comptroller.
[5] GAO, Cost Assessment Guide: Best Practices for Estimating and
Managing Program Costs, GAO-07-1134SP (Washington, D.C.: July 2007).
[6] GAO, Global War on Terrorism: DOD Needs to Take Action to Encourage
Fiscal Discipline and Optimize the Use of Tools Intended to Improve
GWOT Cost Reporting, GAO-08-68 (Washington, D.C.: Nov. 6, 2007).
[7] The Institute for Defense Analyses is a federally funded research
and development center that developed and maintains COST under a
contract with DOD.
[8] The 12 steps of the cost estimating process are iterative and can
sometimes be accomplished in varying order or concurrently.
Additionally, the analysis, presentation, and updating of the estimate
can lead to repeating previous steps.
[9] For this review, we did not assess the reliability of any of these
data sources nor did we assess the appropriateness of these data
sources for the purposes of GWOT cost estimation.
[10] The term "incremental costs" means those directly attributable
costs that would not have been incurred if it were not for the
operation.
[11] Deputy Secretary of Defense Memorandum Ground Rules and Process
for Fiscal Year 2007 Spring Supplemental (Oct. 25, 2006).
[12] During the course of this review, we did not validate or assess
the appropriateness or effectiveness of the processes the services use
to estimate reset-related funding needs. We have reported that the Army
and Marine Corps cannot be assured that their reset strategies will
sustain equipment availability for units deployed in or preparing to
deploy to Iraq and Afghanistan while meeting ongoing operational
requirements and also that the Army and Marine Corps did not report
detailed reset expenditures within the procurement accounts in a way
that confirms that funds appropriated for reset were obligated and
expended for reset. See GAO, Defense Logistics: Army and Marine Corps
Cannot Be Assured That Equipment Reset Strategies Will Sustain
Equipment Availability While Meeting Ongoing Operational Requirements,
GAO-07-814 (Washington, D.C.: Sept. 19, 2007).
[13] See GAO, Global War on Terrorism: DOD Needs to Improve the
Reliability of Cost Data and Provide Additional Guidance to Control
Costs, GAO-05-882 (Washington, D.C.: Sept. 21, 2005), Global War on
Terrorism: Fiscal Year 2006 Obligation Rates Are Within Funding Levels
and Significant Multiyear Procurement Funds Will Likely Remain
Available for Use in Fiscal Year 2007, GAO-07-76 (Washington, D.C.:
Nov. 13, 2006), and GAO-08-68.
[14] See GAO-07-1134SP and OMB, Guidelines and Discount Rates for
Benefit-Cost Analysis of Federal Programs, Circular No. A-94 Revised
(Washington, D.C.: Oct. 29, 1992).
[15] For example, two categories are defined as including costs related
to the logistics civil augmentation program (LOGCAP): both the category
for facilities and base support and a separate category for LOGCAP. DOD
stated in its technical comments regarding a draft of this report that
this issue has been reviewed and is currently under revision.
[16] Sensitivity analysis can identify key elements that drive cost and
provide information to decision makers regarding the potential for cost
increases and reasons for increases. Risk and uncertainty analysis
assess the variability in the cost estimate. Specifically, uncertainty
analysis provides perspective on the potential variability of the
estimate should facts, circumstances, and assumptions change.
[17] DOD, Fiscal Year 2008 Global War on Terror Amendment (October
2007).
[18] GAO, Cost Assessment Guide: Best Practices for Estimating and
Managing Program Costs, GAO-07-1134SP (Washington, D.C.: July 2007).
[19] In late January 2007, the President announced a "surge" strategy
in Iraq, providing for the deployment of an additional 30,000 troops to
support stability operations. He also announced the deployment of
additional personnel to Afghanistan to provide increased security
against an anticipated insurgent offensive.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
