Defense Infrastructure
Navy Can Improve the Quality of Its Cost Estimate to Homeport an Aircraft Carrier at Naval Station Mayport Gao ID: GAO-11-309 March 3, 2011The 2001 Quadrennial Defense Review called for the Navy to provide more warfighting assets more quickly to multiple locations. Subsequently, the Navy made a preliminary decision to homeport a nuclear-powered aircraft carrier at Naval Station Mayport, Florida, which was affirmed by the 2010 Quadrennial Defense Review. In House Report 111-491, accompanying a proposed bill for the Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), GAO was directed to develop an independent estimate of the total federal costs for the proposed homeporting. GAO's objectives were to (1) develop an independent estimate of the full life-cycle costs to homeport a nuclear aircraft carrier at Mayport and (2) determine to what extent the Navy's estimate meets the characteristics of a high-quality cost estimate. To do this, GAO worked with a firm experienced in preparing life-cycle cost estimates for major federal acquisitions and compared the Navy's cost estimating practices with the best practices in GAO's "Cost Estimating and Assessment Guide."
GAO's independent cost estimate suggests that the total one-time cost of homeporting a nuclear-powered aircraft carrier at Naval Station Mayport is expected to be between $258.7 million and $356.0 million, in base year 2010 dollars. The Navy's estimate of the one-time cost is $537.6 million--also in base year 2010 dollars--which is outside the upper range of GAO's estimate. Unlike GAO's estimate, the Navy did not conduct a risk and uncertainty analysis on its one-time costs; as a result, its estimate does not include a range. The largest difference between GAO's estimate of one-time costs and the Navy's estimate is the cost of constructing new facilities at Mayport. Based on the historical costs of constructing similar facilities, GAO estimates at the 65 percent confidence level that the cost for constructing the controlled industrial facility will be $70.5 million, and the cost for constructing the ship maintenance support facilities will be $45.6 million. The Navy estimates the construction costs to be much higher at $139.1 million and $157.2 million, respectively. Navy officials told GAO the difference is due to the increased cost involved in protecting the buildings from a potential storm surge associated with a Category 4 hurricane. GAO included a hurricane factor in its estimate to account for this increase, but GAO and the Navy used different estimating methods in developing the estimates for the construction costs. GAO used adjusted actual costs from similar construction projects, while the Navy used a detailed engineering estimate. For recurring costs, GAO's independent cost estimate suggests that the total is expected to be between $9.0 million and $17.6 million per year. The Navy's estimate of $15.3 million per year is within GAO's estimated range. The Navy's estimate did not fully meet any of the four characteristics-- comprehensive, accurate, well documented, and credible--for producing a high-quality cost estimate. Specifically, although the estimate included almost all of the life-cycle costs related to homeporting a nuclear aircraft carrier at Mayport, it partially met the criteria for being comprehensive because it does not fully describe the cost-influencing ground rules and assumptions. The estimate was only minimally accurate and well documented in that although many elements of the estimate are based on actual experiences from other comparable programs, it is difficult to say if the cost estimates are the most likely costs since the Navy did not conduct a risk and uncertainty analysis. Further, the estimate contains very little step-by-step description of how the estimate was developed so that a cost analyst unfamiliar with the program could independently replicate it. The Navy had to recreate several portions of the estimate in order to provide GAO with supporting documentation. Further, the Navy's estimate does not meet the GAO best practice for a credible estimate because it does not include a sensitivity analysis and was not compared by the Navy to an independent cost estimate conducted by a group outside the Navy. Without fully meeting the characteristics of a high-quality estimate, the Navy's ability to present a convincing argument of the estimate's affordability and credibly answer decision makers' and oversight groups' questions about the estimate is hampered. GAO recommends DOD take several actions to improve the quality of its Mayport cost estimate. DOD partially concurred with two recommendations and disagreed with one, generally stating that additional direction or change is not required. GAO believes the recommendations remain valid as discussed in the report.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Brian J. Lepore Team: Government Accountability Office: Defense Capabilities and Management Phone: (202) 512-4523GAO-11-309, Defense Infrastructure: Navy Can Improve the Quality of Its Cost Estimate to Homeport an Aircraft Carrier at Naval Station Mayport
This is the accessible text file for GAO report number GAO-11-309
entitled 'Defense Infrastructure: Navy Can Improve the Quality of Its
Cost Estimate to Homeport an Aircraft Carrier at Naval Station
Mayport' which was released on March 3, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
March 2011:
Defense Infrastructure:
Navy Can Improve the Quality of Its Cost Estimate to Homeport an
Aircraft Carrier at Naval Station Mayport:
GAO-11-309:
GAO Highlights:
Highlights of GAO-11-309, a report to congressional committees.
Why GAO Did This Study:
The 2001 Quadrennial Defense Review called for the Navy to provide
more warfighting assets more quickly to multiple locations.
Subsequently, the Navy made a preliminary decision to homeport a
nuclear-powered aircraft carrier at Naval Station Mayport, Florida,
which was affirmed by the 2010 Quadrennial Defense Review.
In House Report 111-491, accompanying a proposed bill for the Fiscal
Year 2011 National Defense Authorization Act (H.R. 5136), GAO was
directed to develop an independent estimate of the total federal costs
for the proposed homeporting. GAO‘s objectives were to (1) develop an
independent estimate of the full life-cycle costs to homeport a
nuclear aircraft carrier at Mayport and (2) determine to what extent
the Navy‘s estimate meets the characteristics of a high-quality cost
estimate. To do this, GAO worked with a firm experienced in preparing
life-cycle cost estimates for major federal acquisitions and compared
the Navy‘s cost estimating practices with the best practices in GAO‘s
Cost Estimating and Assessment Guide.
What GAO Found:
GAO‘s independent cost estimate suggests that the total one-time cost
of homeporting a nuclear-powered aircraft carrier at Naval Station
Mayport is expected to be between $258.7 million and $356.0 million,
in base year 2010 dollars. The Navy‘s estimate of the one-time cost is
$537.6 million-”also in base year 2010 dollars-”which is outside the
upper range of GAO‘s estimate. Unlike GAO‘s estimate, the Navy did not
conduct a risk and uncertainty analysis on its one-time costs; as a
result, its estimate does not include a range. The largest difference
between GAO‘s estimate of one-time costs and the Navy‘s estimate is
the cost of constructing new facilities at Mayport. Based on the
historical costs of constructing similar facilities, GAO estimates at
the 65 percent confidence level that the cost for constructing the
controlled industrial facility will be $70.5 million, and the cost for
constructing the ship maintenance support facilities will be $45.6
million. The Navy estimates the construction costs to be much higher
at $139.1 million and $157.2 million, respectively. Navy officials
told GAO the difference is due to the increased cost involved in
protecting the buildings from a potential storm surge associated with
a Category 4 hurricane. GAO included a hurricane factor in its
estimate to account for this increase, but GAO and the Navy used
different estimating methods in developing the estimates for the
construction costs. GAO used adjusted actual costs from similar
construction projects, while the Navy used a detailed engineering
estimate. For recurring costs, GAO‘s independent cost estimate
suggests that the total is expected to be between $9.0 million and
$17.6 million per year. The Navy‘s estimate of $15.3 million per year
is within GAO‘s estimated range.
The Navy‘s estimate did not fully meet any of the four
characteristics”-comprehensive, accurate, well documented, and
credible-”for producing a high-quality cost estimate. Specifically,
although the estimate included almost all of the life-cycle costs
related to homeporting a nuclear aircraft carrier at Mayport, it
partially met the criteria for being comprehensive because it does not
fully describe the cost-influencing ground rules and assumptions. The
estimate was only minimally accurate and well documented in that
although many elements of the estimate are based on actual experiences
from other comparable programs, it is difficult to say if the cost
estimates are the most likely costs since the Navy did not conduct a
risk and uncertainty analysis. Further, the estimate contains very
little step-by-step description of how the estimate was developed so
that a cost analyst unfamiliar with the program could independently
replicate it. The Navy had to recreate several portions of the
estimate in order to provide GAO with supporting documentation.
Further, the Navy's estimate does not meet the GAO best practice for a
credible estimate because it does not include a sensitivity analysis
and was not compared by the Navy to an independent cost estimate
conducted by a group outside the Navy. Without fully meeting the
characteristics of a high-quality estimate, the Navy‘s ability to
present a convincing argument of the estimate‘s affordability and
credibly answer decision makers‘ and oversight groups‘questions about
the estimate is hampered.
What GAO Recommends:
GAO recommends DOD take several actions to improve the quality of its
Mayport cost estimate. DOD partially concurred with two
recommendations and disagreed with one, generally stating that
additional direction or change is not required. GAO believes the
recommendations remain valid as discussed in the report.
View [hyperlink, http://www.gao.gov/products/GAO-11-309] or key
components. For more information, contact Brian Lepore at (202) 512-
4523 or leporeb@gao.gov.
[End of section]
Contents:
Letter:
Background:
GAO's Independent Cost Estimate Is Lower Than the Navy's Estimate for
Homeporting a Nuclear Aircraft Carrier at Naval Station Mayport:
Navy's Estimate Does Not Fully Meet Any of the Four Characteristics
for Producing a High-Quality Cost Estimate:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Additional Information on GAO's Independent Cost Estimate:
Appendix III: GAO's Assessment of the Navy's Cost Estimating Practices:
Appendix IV: Comments from the Department of Defense:
Appendix V: GAO Contacts and Staff Acknowledgments:
Tables:
Table 1: Twelve Steps of a High-Quality Cost Estimate and
Corresponding Characteristics:
Table 2: Comparison of GAO Estimated Range and Navy Point Estimates of
One-Time Costs (in Base Year 2010 dollars):
Table 3: Comparison of GAO's 65 Percent Confidence Level Estimates and
Navy's Point Estimates of One Time Costs (in Base Year 2010 Dollars):
Table 4: Comparison of GAO Estimated Range and Navy Point Estimates of
Recurring Costs (in Base Year 2010 Dollars):
Table 5: Comparison of GAO's 65 Percent Confidence Level Estimates and
Navy's Point Estimates of Recurring Costs (in Base Year 2010 Dollars):
Figure:
Figure 1: Location and Timeline for Completion of Planned
Infrastructure and Facility Improvements at Naval Station Mayport:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
March 3, 2011:
Congressional Committees:
Until the U.S.S. John F. Kennedy--a conventional aircraft carrier--was
retired in 2007, the Navy had homeported at least one aircraft carrier
at Naval Station Mayport, Florida[Footnote 1] for many decades. Prior
to the U.S.S. John F. Kennedy's retirement, the 2001 Quadrennial
Defense Review called for the Navy to provide more warfighting assets
more quickly to multiple locations and to meet this requirement, the
Navy made a preliminary decision to homeport additional surface ships
at Mayport. The Navy subsequently prepared an environmental impact
statement to evaluate a broad range of strategic homeport and
dispersal options for Atlantic Fleet surface ships in Mayport and on
January 14, 2009, issued its decision to pursue an option that would
include homeporting a nuclear-powered aircraft carrier at Mayport. The
Navy's decision was reviewed as part of the Department of Defense's
(DOD) 2010 Quadrennial Defense Review, which in its report supported
the Navy's decision to homeport a nuclear-powered aircraft carrier in
Mayport, indicating that homeporting an East Coast carrier in Mayport
would contribute to mitigating the risk of a terrorist attack,
accident, or natural disaster.[Footnote 2] Additionally, officials in
DOD's Office of Cost Assessment and Program Evaluation, which was part
of the review's working group assessing the Navy's Mayport decision,
reviewed the Navy's presentations and supporting documentation, but
did not develop an independent cost estimate for comparison.
In House Report 111-491, accompanying a proposed bill for the Fiscal
Year 2011 National Defense Authorization Act (H.R. 5136), the House
Committee on Armed Services directed GAO to submit a report no later
than February 15, 2011,[Footnote 3] that contains an independent
estimate of the total direct and indirect costs[Footnote 4] to be
incurred by the federal government for the proposed homeporting at
Naval Station Mayport.[Footnote 5] In response to this report, our
objectives were to (1) develop an independent estimate of the full
life-cycle costs associated with the Navy's plan to establish a second
East Coast homeport for a nuclear-powered aircraft carrier at Naval
Station Mayport and determine to what extent it differs from the
Navy's estimate and (2) determine to what extent the Navy's estimate
meets the characteristics of a high-quality cost estimate.
To develop the independent cost estimate, we worked with an outside
cost estimating consulting company with expertise in preparing
estimates of the life-cycle costs of major federal acquisitions. We
followed the 12 steps outlined in the GAO Cost Estimating and
Assessment Guide,[Footnote 6] which have been identified as best
practices by GAO and experts in the cost estimating community, to
develop a risk-adjusted range of cost estimates. We compared our risk-
adjusted range of estimates, at an 80 percent confidence interval, to
the Navy's point estimates to determine whether the Navy's estimates
were within our risk-adjusted range.[Footnote 7] For a direct element-
by-element comparison between our estimate and the Navy's, we also
compared our estimates at the 65 percent confidence level with the
Navy's point estimates.[Footnote 8]Additionally, we reviewed the
Navy's cost estimate and interviewed Navy officials with knowledge of
the methodology and key underlying assumptions used in developing the
Navy's estimate.[Footnote 9] We reviewed the Navy's environmental
impact study used in evaluating options for homeporting surface ships
at Mayport. We also obtained background information about nuclear-
powered aircraft carrier maintenance, visited locations where nuclear-
powered aircraft carrier maintenance is performed, and spoke with Navy
officials knowledgeable about the personnel and facilities
requirements for nuclear-powered aircraft carrier maintenance. From
this information we developed a list of individual cost elements and
used standard cost estimating methodologies to estimate the cost of
each element.[Footnote 10] We also conducted sensitivity and risk and
uncertainty analyses on our estimate.[Footnote 11] We compared our
estimate with the Navy's and interviewed Navy officials to determine
the reasons for any differences. Through document reviews and
interviews with agency officials knowledgeable about the Navy's cost
data and the systems that produced them and the internal controls used
to maintain the integrity of the data, we determined the data were
sufficiently reliable for our purposes.
To determine to what extent the Navy's estimate has the
characteristics of a high-quality cost estimate, we compared the
Navy's cost estimating practices against those outlined in the GAO
Cost Estimating and Assessment Guide. We analyzed the extent to which
the Navy's cost estimate was (1) comprehensive, (2) accurate, (3) well
documented, and (4) credible--the four key characteristics of high-
quality cost estimates and assigned each characteristic a rating of
not met, minimally met, partially met, substantially met, or
met.[Footnote 12] More detailed information about our scope and
methodology is provided in appendix I.
We conducted this performance audit from July 2010 to February 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings, and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
Planned Infrastructure and Facility Improvements at Naval Station
Mayport to Support a Nuclear-Powered Aircraft Carrier:
The U.S. Navy currently maintains 11 nuclear carriers--5 homeported on
the West Coast of the United States, 5 on the East Coast of the United
States, and 1 in Yokosuka, Japan. Of the West Coast carriers, 3 are
homeported in San Diego, California; 1 in Bremerton, Washington; and 1
in Everett, Washington. All 5 East Coast carriers are homeported in
Norfolk, Virginia. As some of the most technologically advanced ships
in the world, nuclear-powered aircraft carriers require continuous and
regularly scheduled maintenance to keep them ready to meet mission
requirements.
Naval Station Mayport has served as a conventional aircraft carrier
homeport at various times since the 1950s, including homeporting two
conventional carriers at one time, but it has not served as homeport
for a nuclear-powered aircraft carrier. Consequently, to homeport a
nuclear powered aircraft carrier, several major infrastructure
upgrades are required at Mayport. These infrastructure improvements
include construction of four new buildings: (1) controlled industrial
facility, (2) ship maintenance facility, (3) maintenance support
facility, and (4) parking garage. The controlled industrial facility
would be used for the inspection, modification, and repair of
radiologically controlled equipment and components associated with
naval nuclear propulsion plants. The ship maintenance facility is
necessary to perform nonnuclear maintenance on the carrier's nuclear
propulsion plants. The maintenance support facility would provide the
needed administrative and technical staff offices to support
propulsion plant maintenance and a central area for receiving,
inspecting, shipping, and storing materials. The parking garage would
replace parking spaces lost through construction of the three new
maintenance buildings. The infrastructure upgrades also include
improvements to Wharf F, which would receive improvement to its
structural components, berthing capability, and utility and security
systems in order to make it capable of homeporting a nuclear carrier.
Additionally, traffic improvements would be made to the Massey Avenue
corridor to rebuild a boulevard-style roadway with two lanes each way
divided by a grass median.[Footnote 13] Dredging would also be
necessary to deepen the Mayport turning basin to allow access for a
fully loaded nuclear aircraft carrier.[Footnote 14] Figure 1 shows the
location and expected time line for completion of the planned
infrastructure and facility improvements at Naval Station Mayport.
Figure 1: Location and Timeline for Completion of Planned
Infrastructure and Facility Improvements at Naval Station Mayport:
[Refer to PDF for image: illustrated map and timeline]
1. Dredging, November 2010 to December 2011.
2. Traffic improvements, May 2012 to August 2013.
3. Wharf F, July 2013 to June 2015.
4. CIF, July 2015 to May 2018.
5. Parking garage, construction time line to be determined.
6. SMF/MSF, construction time line to be determined.
Depicted on the map:
Mayport turning basin:
New carrier mooring location;
Dredging area;
Wharf F;
SMF building;
CIF building;
MDF building;
Parking garage;
Traffic improvements.
CIF: Controlled industrial facility.
MSF: Maintenance support facility.
SMF: Ship maintenance facility.
Source: GAO analysis of Navy's Final Environmental Impact Study for
the Proposed Homeporting of Additional Surface Ships at Naval Station
Mayport and military construction budget documents.
[End of figure]
Navy's Estimate for Establishing a Nuclear-Powered Aircraft Carrier at
Naval Station Mayport:
The Navy's estimate of one-time and recurring costs for establishing a
nuclear-powered aircraft carrier homeport at Naval Station Mayport is
contained in two DOD information papers dated February 25, 2010, and
August 5, 2010.[Footnote 15] The February 2010 information paper was
produced by the Navy's Office of the Chief of Naval Operations in
response to a request from the Congressional Research Service and
provided only the one-time costs of the carrier homeport. Those one-
time costs were primarily drawn from a series of DD Form 1391s, which
DOD uses as budget justification material to support funding requests
for military construction. However in addition to military
construction costs, there are also one-time costs associated with
planning and design, the initial equipment outfitting for the new
maintenance facilities to be built in Mayport, and the permanent
change of station cost for the relocation of the carrier crew to
Mayport.
The August 2010 information paper was also prepared by staff working
for the Chief of Naval Operations, in response to questions from GAO
about the recurring costs for the planned homeporting. This
information paper contained estimates of the recurring costs
associated with the carrier homeport. There were nine recurring costs
listed in the information paper including: (1) permanently assigned
labor for the new maintenance facilities, (2) base operating support,
(3) facilities sustainment, (4) facilities restoration and
modernization, (5) transportation of equipment for the carrier air
wing, (6) travel and per diem for public shipyard workers traveling
from Norfolk to Mayport to work on the carrier's nuclear propulsion
systems, (7) the additional biennial maintenance dredging needed to
allow access for a fully loaded nuclear aircraft carrier, (8) the
differential between Norfolk and Mayport in basic allowance for
housing costs, and (9) the differential between Norfolk and Mayport in
utilities costs. The recurring costs do not represent the total cost
of operating a nuclear-powered aircraft carrier. Rather, they are the
subset of costs associated with nuclear carrier maintenance in which
there is a cost differential between conducting the maintenance in
Mayport and conducting the maintenance in Virginia.
On December 28, 2010, the Chief of Naval Operations transmitted a
report to the congressional defense committees that essentially
consolidated the estimates provided in the two information papers.
[Footnote 16] However, the report differed from the information papers
in its treatment of inflation for one-time costs and in the total for
the one-time permanent change of station cost. The February 2010
information paper showed the one-time military construction costs as
budgeted amounts in then-year dollars, which include the effects of
inflation, while the December 2010 report showed all one-time cost in
base year 2010 dollars, which exclude the effects of inflation and use
2010 prices throughout the life cycle. The one-time permanent change
of station cost was reduced from $26 million to $10 million in the
December 2010 report. The report did not provide any additional
description or documentation that was not contained in the two DOD
information papers.
Characteristics of a High-Quality Cost Estimate:
A reliable cost estimate is critical to the success of any program
because it can provide the basis for informed investment decision
making, realistic budget formulation and program resourcing,
meaningful progress measurement, proactive course correction when
warranted, and accountability for results. In March 2009, GAO issued a
cost estimating and assessment guide that compiled cost estimating
best practices drawn from across industry and government.
Specifically, the guide includes four characteristics of a high-
quality cost estimate by which an organization's management can assess
cost estimates in making informed decisions. A cost estimate is:
* comprehensive when it accounts for all possible costs associated
with a project, is structured in sufficient detail to ensure that
costs are neither omitted nor double-counted, and the estimating
teams' composition is commensurate with the assignment;
* well documented when supporting documentation is accompanied by a
narrative explaining the process, sources, and methods used to create
the estimate and contains the underlying data used to develop the
estimate;
* accurate when it is based on an assessment of the costs most likely
to be incurred and not overly conservative or too optimistic; and:
* credible when it has been cross-checked with an independent cost
estimate, the level of confidence associated with the point estimate
has been identified through a risk and uncertainty analysis, and a
sensitivity analysis has been conducted--that is, the project has
examined the effect of changing one assumption related to each project
activity while holding all other variables constant in order to
identify which variable most affects the cost estimate.
In addition, GAO's cost estimating and assessment guide lays out 12
key steps that, when followed, should result in high-quality cost
estimates that are comprehensive and accurate and that can be easily
and clearly traced, replicated, and updated. The guide also contains
hundreds of best practices drawn from across industry and government
for carrying out these steps. Table 1 shows these 12 key steps and how
they relate to the four characteristics of a high-quality cost
estimate.
Table 1: Twelve Steps of a High-Quality Cost Estimate and
Corresponding Characteristics:
Key cost estimating step: Define the estimate's purpose;
Characteristic: Well documented.
Key cost estimating step: Develop the estimating plan;
Characteristic: Comprehensive.
Key cost estimating step: Define the program characteristics;
Characteristic: Well documented.
Key cost estimating step: Determine the estimating structure;
Characteristic: Comprehensive.
Key cost estimating step: Identify the ground rules and assumptions;
Characteristic: Well documented.
Key cost estimating step: Obtain the data;
Characteristic: Well documented.
Key cost estimating step: Determine the point estimate and compare it
to an independent cost estimate;
Characteristic: Credible and accurate.
Key cost estimating step: Conduct a sensitivity analysis;
Characteristic: Credible.
Key cost estimating step: Conduct a risk and uncertainty analysis;
Characteristic: Credible.
Key cost estimating step: Document the estimate;
Characteristic: Well documented.
Key cost estimating step: Present the estimate to management for
approval; Characteristic: Well documented.
Key cost estimating step: Update the estimate to reflect actual costs
and changes;
Characteristic: Accurate.
Source: GAO.
[End of table]
GAO's Independent Cost Estimate Is Lower Than the Navy's Estimate for
Homeporting a Nuclear Aircraft Carrier at Naval Station Mayport:
Our independent estimate of the total cost for homeporting a nuclear-
powered aircraft carrier at Naval Station Mayport is lower than the
Navy's estimate. Specifically, the Navy's estimate of one-time costs
is outside the upper range of our estimate, while its estimate of
recurring costs falls within the upper range of our estimate.
GAO Estimates One-Time Costs to Be Between $258.7 Million and $356.0
Million, Lower Than the Navy's Estimate:
Our independent cost estimate suggests that the total one-time cost of
homeporting a nuclear-powered aircraft carrier at Naval Station
Mayport will be between $258.7 million and $356.0 million, in base
year 2010 dollars. The Navy's estimate of the one-time costs is $537.6
million, also in base year 2010 dollars, which is outside the upper
range of our estimate. Unlike our estimate, the Navy did not conduct a
risk and uncertainty analysis on its one-time costs; as a result, its
estimate does not include a range. Reporting a range of costs around a
point estimate is useful to decision makers because it conveys a level
of confidence in achieving the most likely cost.
Table 2 shows a comparison between our estimated range and the Navy's
estimate for one-time costs. Specifically, the table shows our
estimated range at an 80 percent confidence interval and whether the
Navy's estimate falls into that range. The low value of the estimate
range ($258.7 million) represents a 10 percent chance that the cost
will be that amount or less, and the high value of the estimated range
($356.0 million) represents a 90 percent chance that the cost will be
that amount or less. The last column in the table identifies whether
the Navy's estimate is within our estimated range.
Table 2: Comparison of GAO Estimated Range and Navy Point Estimates of
One-Time Costs (in Base Year 2010 dollars):
Cost element: Planning and design;
GAO's estimated low cost: $15.0 million;
GAO's estimated high cost: $22.9 million;
Navy's point estimate: $30.0 million;
Within GAO's estimated range: No.
Cost element: Dredging;
GAO's estimated low cost: $31.3 million;
GAO's estimated high cost: $33.1 million;
Navy's point estimate: $46.3 million;
Within GAO's estimated range: No.
Cost element: Construction: Parking garage;
GAO's estimated low cost: $21.9 million;
GAO's estimated high cost: $53.3 million;
Navy's point estimate: $27.8 million;
Within GAO's estimated range: Yes.
Cost element: Construction: Road improvements;
GAO's estimated low cost: $9.5 million;
GAO's estimated high cost: $24.3 million;
Navy's point estimate: $15.3 million;
Within GAO's estimated range: Yes.
Cost element: Construction: Wharf F improvements;
GAO's estimated low cost: $28.3 million;
GAO's estimated high cost: $75.3 million;
Navy's point estimate: $38.9 million;
Within GAO's estimated range: Yes.
Cost element: Construction: Controlled industrial facility;
GAO's estimated low cost: $35.2 million;
GAO's estimated high cost: $94.9 million;
Navy's point estimate: $139.1 million;
Within GAO's estimated range: No.
Cost element: Construction: Ship maintenance support facilities;
GAO's estimated low cost: $23.0 million;
GAO's estimated high cost: $59.3 million;
Navy's point estimate: $157.2 million;
Within GAO's estimated range: No.
Cost element: Initial equipment outfitting;
GAO's estimated low cost: $24.5 million;
GAO's estimated high cost: $64.4 million;
Navy's point estimate: $73.0 million;
Within GAO's estimated range: No.
Cost element: Permanent change of station for crew;
GAO's estimated low cost: $4.7 million;
GAO's estimated high cost: $6.0 million;
Navy's point estimate: $10.0 million;
Within GAO's estimated range: No.
Source: GAO analysis.
Notes: The low cost estimate is at the 10 percent confidence level and
the high cost estimate is at the 90 percent confidence level. It is
not statistically appropriate to add each of the individual confidence
intervals to come up with an overall confidence interval. In addition
to the road improvements on Naval Station Mayport, information
provided by a Department of Transportation official indicates that the
Jacksonville Transportation Authority, based on its study of traffic
improvements to the intersection approaching the main gate outside of
the naval station, has identified up to $8 million in improvements
that are unfunded at this time. The official indicated that these
improvements are potentially eligible for federal-aid funds.
[End of table]
Our initial point estimate of the one-time dredging cost based on
historical data was $44.3 million, which was $2 million less than the
Navy's estimate. However, in accordance with the best practices
recommended by the GAO Cost Estimating and Assessment Guide, we have
updated our estimate based on the $26.6 million contract awarded in
August 2010. Our estimated range now reflects the contract award price
plus the Navy's overhead and contingency amounts. Navy officials
attributed the lower than estimated contract costs to supply and
demand within the private dredging sector, which because of lower
demand has caused firms to compete at lower cost.
We also compared the Navy's point estimate to our estimate at the 65
percent confidence level. A confidence level is the chance that the
actual costs will be equal to or less than the adjusted amount. In
table 2, we reported the 10 percent confidence level estimate as our
low estimate range and the 90 percent confidence level estimate as our
high estimate range. For an element-by-element comparison of our
estimate with the Navy's point estimate, we chose the 65 percent
confidence level since we consider this confidence level to be an
appropriate budgeting range. Experts agree that program cost estimates
should be budgeted to at least the 50 percent confidence level, but
budgeting to a higher level, such as 70 percent to 80 percent, or the
mean, is now a common practice.
Table 3 shows our 65 percent confidence level estimate in comparison
to the Navy's point estimate. To facilitate comparisons against the
Navy's estimate, the one-time costs are expressed in base year 2010
dollars, which represent amounts based on 2010 prices, with the impact
of inflation removed. While useful for comparisons against the Navy's
estimate, base year 2010 dollars should not be used as the basis for
budgetary decisions. In order to support a budgetary amount, base year
2010 dollars would need to be converted into then-year dollars.
Table 3: Comparison of GAO's 65 Percent Confidence Level Estimates and
Navy's Point Estimates of One Time Costs (in Base Year 2010 Dollars):
Cost element: Planning and design;
GAO's 65 percent confidence level estimate: $19.7 million;
Navy's point estimate: $30.0 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $10.3 million.
Cost element: Dredging;
GAO's 65 percent confidence level estimate: $32.3 million;
Navy's point estimate: $46.3 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $14.0 million.
Cost element: Construction: Parking garage;
GAO's 65 percent confidence level estimate: $38.5 million;
Navy's point estimate: $27.8 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$10.7 million.
Cost element: Construction: Road improvements;
GAO's 65 percent confidence level estimate: $18.4 million;
Navy's point estimate: $15.3 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$3.1 million.
Cost element: Construction: Wharf F improvements;
GAO's 65 percent confidence level estimate: $58.2 million;
Navy's point estimate: $38.9 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$19.3 million.
Cost element: Construction: Controlled industrial facility;
GAO's 65 percent confidence level estimate: $70.5 million;
Navy's point estimate: $139.1 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $68.3 million.
Cost element: Construction: Ship maintenance support facilities;
GAO's 65 percent confidence level estimate: $45.6 million;
Navy's point estimate: $157.2 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $111.6 million.
Cost element: Initial equipment outfitting;
GAO's 65 percent confidence level estimate: $48.4 million;
Navy's point estimate: $73.0 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $24.6 million.
Cost element: Permanent change of station for crew;
GAO's 65 percent confidence level estimate: $5.5 million;
Navy's point estimate: $10.0 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $4.5 million.
Source: GAO analysis.
Notes: The low cost estimate is at the 10 percent confidence level and
the high cost estimate is at the 90 percent confidence level. It is
not statistically appropriate to add each of the individual confidence
intervals to come up with an overall confidence interval. In addition
to the road improvements on Naval Station Mayport, information
provided by a Department of Transportation official indicates that the
Jacksonville Transportation Authority, based on its study of traffic
improvements to the intersection approaching the main gate outside of
the naval station, has identified up to $8 million in improvements
that are unfunded at this time. The official indicated that these
improvements are potentially eligible for federal-aid funds.
[End of table]
Table 3 shows the largest difference between the two point estimates
is in the construction of the controlled industrial facility and the
ship maintenance support facilities, where the Navy's estimates were
$68.3 million and $111.6 million higher than our 65 percent confidence
level estimates, respectively. The Navy's estimate of these costs is
based on a detailed engineering study developed by an engineering
firm. In contrast, our estimate is based on historical costs observed
for analogous facility construction projects at Naval Air Station
North Island in San Diego in 1995. These analogous costs are adjusted
to account for differences in size, inflation, geographical
differences between the two areas, and any identified complexities or
requirements unique to the Mayport construction. These factors were
not present in the San Diego projects. One known complexity of the
Mayport construction projects is that the controlled industrial
facility must be built to withstand a storm surge from a category 4
hurricane.[Footnote 17] Our estimate includes a factor of 10 percent
of the total construction cost to account for hurricane-related
construction complexities.[Footnote 18] Other complexity factors
included environmental considerations and costs associated with
antiterrorism and force protection that were not required for the
construction at Naval Station North Island. We included both of these
factors in our estimate based on a percentage of total construction
cost. Neither GAO nor the Navy has identified any other specific
complexity factors that would account for the significant discrepancy
between the two estimates.
Additionally, other elements of the Navy's estimate were higher than
our own. Specifically, the Navy's estimate of initial equipment
outfitting was $24.6 million higher than ours and the Navy's estimate
of the permanent change of station cost for relocating the carrier
crew was $4.5 million higher than ours. Finally, the Navy's planning
and design estimate was $10.3 million higher than ours. Both we and
the Navy estimated planning and design as a percentage of the overall
construction cost. The $10.3 million difference between the Navy's
estimate and ours results from the Navy's higher estimates for
construction of the parking garage and controlled industrial facility
and ship maintenance support buildings. Appendix II includes our
complete cost estimate and explains in more detail our cost estimating
results for each element of the one-time costs.
GAO Estimates Recurring Costs to Be Between $9.0 and $17.6 Million Per
Year, and the Navy's Estimate Falls Within This Range:
Our independent cost estimate indicates that the annual recurring cost
of homeporting a nuclear-powered aircraft carrier at Naval Station
Mayport is expected to be between $9.0 million and $17.6 million each
year, and the Navy's $15.3 million per year estimate of the annual
recurring cost is within the range of our estimate. Because the Navy
did not conduct a risk and uncertainty analysis on its recurring
costs, its estimate does not include a range.
Table 4 shows a comparison between our estimated range and the Navy's
estimate for recurring costs. Specifically, the table shows our
estimate range at an 80 percent confidence interval and whether the
Navy's estimate falls into that range. The low value of the estimated
range ($9.0 million) represents a 10 percent chance that the cost will
be that amount or less, and the high value of the estimated range
($17.6 million) represents a 90 percent chance that the cost will be
that amount or less. The last column in the table identifies whether
the Navy's estimate is within our estimated range.
Table 4: Comparison of GAO Estimated Range and Navy Point Estimates of
Recurring Costs (in Base Year 2010 Dollars):
Permanently assigned labor for nuclear facilities;
GAO's estimated low cost: $0.7 million;
GAO's estimated high cost: $1.9 million;
Navy's point estimate: $1.2 million;
Within GAO's estimated range: Yes.
Base operating support;
GAO's estimated low cost: $0.7 million;
GAO's estimated high cost: $1.4 million;
Navy's point estimate: $1.0 million;
Within GAO's estimated range: Yes.
Facilities sustainment;
GAO's estimated low cost: $0.7 million;
GAO's estimated high cost: $1.2 million;
Navy's point estimate: $0.8 million;
Within GAO's estimated range: Yes.
Facilities restoration and modernization;
GAO's estimated low cost: $2.2 million;
GAO's estimated high cost: $3.3 million;
Navy's point estimate: $6.8 million;
Within GAO's estimated range: No.
Operations;
GAO's estimated low cost: $0.5 million;
GAO's estimated high cost: $1.4 million;
Navy's point estimate: $0.8 million;
Within GAO's estimated range: Yes.
Travel/per diem for public shipyard workers;
GAO's estimated low cost: $4.8 million;
GAO's estimated high cost: $12.2 million;
Navy's point estimate: $7.0 million;
Within GAO's estimated range: Yes.
Biennial maintenance dredging;
GAO's estimated low cost: $0.1 million;
GAO's estimated high cost: $0.2 million;
Navy's point estimate: $0.1 million;
Within GAO's estimated range: Yes.
Basic allowance for housing differential;
GAO's estimated low cost: -$5.5 million;
GAO's estimated high cost: -$4.4 million;
Navy's point estimate: -$3.2 million;
Within GAO's estimated range: No.
Utilities;
GAO's estimated low cost: $0.6 million;
GAO's estimated high cost: $1.8 million;
Navy's point estimate: $0.8 million;
Within GAO's estimated range: Yes.
Permanent change of station;
GAO's estimated low cost: $1.0 million;
GAO's estimated high cost: $1.3 million;
Navy's point estimate: 0;
Within GAO's estimated range: No.
Private sector travel;
GAO's estimated low cost: $0.1 million;
GAO's estimated high cost: $1.2 million;
Navy's point estimate: 0;
Within GAO's estimated range: No.
Source: GAO analysis.
Notes: The low cost estimate is at the 10 percent confidence level and
the high cost estimate is at the 90 percent confidence level. The
range and estimate for the basic allowance for housing is negative
because the housing allowance is less in the Jacksonville, Florida
metropolitan area than it is in Norfolk, Virginia. This results in a
net cost savings for the Navy for this cost element. It is not
statistically appropriate to add each of the individual confidence
intervals to come up with an overall confidence interval. The overall
65 percent confidence level that is reported in appendix II is
calculated separately from the individual elements.
[End of table]
As we did with one time costs, we also compared our 65 percent
confidence level estimates with the Navy's point estimates for a
direct element-by-element comparison between our estimate and the
Navy's, as shown in table 5.
Table 5: Comparison of GAO's 65 Percent Confidence Level Estimates and
Navy's Point Estimates of Recurring Costs (in Base Year 2010 Dollars):
Cost element: Permanently assigned labor for nuclear facilities;
GAO's 65 percent confidence level estimate: $1.4 million;
Navy's point estimate: $1.2 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.2 million.
Cost element: Base operating support;
GAO's 65 percent confidence level estimate: $1.1 million;
Navy's point estimate: $1.0 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.1 million.
Cost element: Facilities sustainment;
GAO's 65 percent confidence level estimate: $1.0 million;
Navy's point estimate: $0.8 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.2 million.
Cost element: Facilities restoration and modernization;
GAO's 65 percent confidence level estimate: $2.9;
Navy's point estimate: $6.8;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $3.9.
Cost element: Operations;
GAO's 65 percent confidence level estimate: $1.0 million;
Navy's point estimate: $0.8 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.2 million.
Cost element: Travel/per diem for public shipyard workers;
GAO's 65 percent confidence level estimate: $9.1v;
Navy's point estimate: $7.0 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$2.1 million.
Cost element: Biennial maintenance dredging;
GAO's 65 percent confidence level estimate: $0.1 million;
Navy's point estimate: $0.1 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: 0.
Cost element: Basic allowance for housing differential;
GAO's 65 percent confidence level estimate: -$4.8 million;
Navy's point estimate: -$3.2 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: $1.6 million.
Cost element: Utilities;
GAO's 65 percent confidence level estimate: $1.3 million;
Navy's point estimate: $0.8 million;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.5 million.
Cost element: Permanent change of station;
GAO's 65 percent confidence level estimate: $1.2 million;
Navy's point estimate: $0;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$1.2 million.
Cost element: Private sector travel;
GAO's 65 percent confidence level estimate: $0.6 million;
Navy's point estimate: 0;
Difference between Navy's point estimate and GAO's 65 percent
confidence level estimate: -$0.6 million.
Source: GAO analysis.
Note: The estimate for the basic allowance for housing is negative
because the housing allowance is less in the Jacksonville, Florida
metropolitan area than it is in Norfolk, Virginia. This results in a
net cost savings for the Navy for this cost element. It is not
statistically appropriate to add each of the individual confidence
intervals to come up with an overall confidence interval. The overall
65 percent confidence level that is reported in appendix II is
calculated separately from the individual elements.
[End of table]
The largest difference between the two estimates is in the cost of
facilities restoration and modernization, where the Navy's estimate is
$3.9 million higher. Since the Navy determined this cost as a
percentage of total building construction costs for their estimates,
the Navy's higher cost results from its higher estimate for
construction of the controlled industrial facility and the ship
maintenance support buildings. Some of the recurring cost elements,
such as the permanently assigned labor for nuclear facilities and base
operating support, are slightly higher in our estimate than the Navy's
estimate. For these cost elements we used the Navy's estimated costs
because we did not have access to a source of information from which
we could estimate the costs independently from the Navy. Although we
used the same estimate as the Navy, our numbers at the 65 percent
confidence level are slightly higher than the Navy's because they are
risk-adjusted, whereas the Navy's point estimates have not been
adjusted for risk. In using the Navy's calculations, we reviewed the
Navy's ground rules and assumptions to the extent possible based on
the information that the Navy provided to us.
In addition, we included two additional recurring costs in our
estimate that were not included in the Navy's estimate--the additional
travel costs that the private sector may incur if they have to bring
in workers from outside of the Mayport area and the permanent change
of station for the aircraft carrier crew during the 10.5-month docking
maintenance availabilities and the 3-year midlife refueling overhaul.
[Footnote 19] The recurring costs have been annualized over the 50-
year carrier life cycle. Consistent with the Navy's estimate, we have
not identified the recurring costs as beginning in any particular
fiscal year. Appendix II provides our complete cost estimate and
explains in greater detail our cost estimating results for each
element of recurring costs. It also contains additional information on
the differences between GAO's and the Navy's estimates.
Navy's Estimate Does Not Fully Meet Any of the Four Characteristics
for Producing a High-Quality Cost Estimate:
The Navy's estimate did not fully meet any of the four
characteristics--comprehensive, accurate, well documented, and
credible--for producing a high-quality cost estimate. Specifically,
the Navy's estimate partially met the criteria for being
comprehensive; however, the estimate was minimally accurate and well
documented and overall, did not meet the criteria for being credible.
The Navy's Estimate Is Partially Comprehensive:
The Navy's estimate partially meets the criteria for being
comprehensive. Specifically, it includes almost all of the life-cycle
costs related to homeporting a nuclear-powered aircraft carrier at
Naval Station Mayport. The Navy did not include two small recurring
costs: (1) the additional travel costs that the private sector may
incur if they have to bring in workers from outside of the Mayport
area and (2) the permanent change of station for the carrier crew
during the 10.5-month docking planned incremental availability and 39-
month midlife refueling overhaul. However, the estimate does not fully
document the cost-influencing ground rules and assumptions.
Additionally, the Navy only minimally provided a cost element
breakdown structure in its estimate, and in conducting our assessment,
we had to develop our own more robust structure to ensure that cost
elements were neither omitted nor double counted. Individual segments
of the Navy's cost estimate were completed by separate Navy teams and
then pieced together into two documents that represent the Navy's cost
estimate. According to Navy officials, there was no single person or
team in charge of ensuring that the entire Mayport homeporting
estimate was packaged into a comprehensive and well documented final
estimate.
According to GAO's guide, a cost estimate is comprehensive when it
includes both government and contractor costs of the program over its
full life cycle, from inception of the program through design,
development, deployment, and operation and maintenance to retirement
of the program. It should also completely define the program, reflect
the current schedule, and be technically reasonable. Comprehensive
cost estimates should be structured in sufficient detail to ensure
that cost elements are neither omitted nor double counted.
Specifically, the GAO guide states that the cost estimate should be
based on a product-oriented work breakdown structure that allows a
program to track cost and schedule by defined deliverables, such as
hardware or software components. Finally, where information is limited
and judgments must be made, the cost estimate should document all cost-
influencing ground rules and assumptions. Appendix III provides our
assessment of the extent to which the Navy's estimate met the best
practices for the four characteristics of a high-quality cost
estimate--comprehensive, well documented, accurate, and credible--
described in the GAO Cost Estimating and Assessment Guide.
The Navy's Estimate Is Minimally Accurate:
The Navy's estimate is minimally accurate. Some elements of the
estimate are based on a historical record of cost estimating and
actual experiences from other comparable programs. Many of the
recurring costs--such as the facilities sustainment, operations, and
the biennial dredging costs--are based on a historical record of past
costs. However, no evidence was provided by the Navy to show that a
risk and uncertainty analysis was conducted. Therefore, the confidence
level of the estimate can not be determined, and it is difficult to
say if the cost estimates are the most likely costs. In addition, the
initial cost estimate provided to us by the Navy contained calculation
errors, although the Navy identified the errors after discussions with
GAO and corrected the errors in subsequent versions of its estimate.
An estimate is considered accurate when it is based on an assessment
of the costs most likely to be incurred, according to GAO's cost
estimating guide. The results should be unbiased, that is not overly
conservative or optimistic. Additionally, the estimate should be based
on a historical record of cost estimating and actual experiences from
other comparable programs. The Navy based its estimate of construction
costs on a detailed engineering estimate produced by a private
architecture and engineering firm, but prior to GAO's review, the Navy
had not reviewed the firm's engineering estimate to verify the
accuracy and completeness of the estimate and did not check whether
the engineering estimate relied on overly conservative assumptions or
fell within the range of historical costs. Without ensuring that its
military construction estimates are accurate, the Navy may be
overestimating the costs of new construction at Mayport, which could
result in the Navy requesting more funding than is needed and limit
the amount of money available to fund other projects.
The Navy's Estimate Is Minimally Well Documented:
Although the Navy's documentation contains some discussion of the
technical information used in the estimate, the Navy's estimate is
minimally well documented. The documentation contains very little step-
by-step description of how the estimate was developed so that a cost
analyst unfamiliar with the program could understand what was done and
replicate it. Additionally, the documentation does not capture the
source data used, the reliability of the data, and how the data were
standardized. Moreover, Navy officials had to recreate several
portions of the estimate to provide us with supporting documentation.
Cost estimates, according to GAO's cost estimating and assessment
guide, are well documented when they can be easily repeated or updated
and can be traced to original sources. The documentation should
explicitly identify the primary methods, calculations, assumptions,
and sources of the data used to generate each cost element. The
estimating process should be described and an explanation provided for
why particular methods and data sets were chosen and why these choices
are reasonable. All the steps involved in developing the estimate
should be documented so that a cost analyst unfamiliar with the
program could recreate it with the same result. In addition,
documentation for the cost estimate should reflect changes in
technical or program assumptions or new program phases or milestones.
Insufficient documentation makes it difficult for outside
organizations to validate the Navy's life-cycle cost estimate for
homeporting an aircraft carrier at Naval Station Mayport. Not having
adequate documentation also impedes the Navy's ability to develop
future estimates and facilitate oversight by Congress.
The Navy's Estimate Does Not Meet GAO's Best Practices for a Credible
Estimate:
Overall, the Navy's estimate does not meet the GAO best practices'
criteria for a credible estimate, because it does not include a
sensitivity analysis and was not compared by the Navy to an
independent cost estimate conducted by a group outside the acquiring
organization. Although DOD's Office of Cost Assessment and Program
Evaluation did review the Navy's presentations and supporting
documentation in conjunction with the 2010 Quadrennial Defense Review
working group, the office only performed a general review of the
Navy's costs, not a full independent cost estimate with a sensitivity
analysis. Further, while the Navy did identify some risks, it did not
conduct a formal risk and uncertainty analysis that would identify a
level of confidence in the estimate. In addition, although some cost
elements in the Navy's estimate were derived from models that have
undergone validation, verification, and accreditation, others were
not. For example, the model that was used to develop the estimate of
the recurring travel cost for the maintenance workers who would travel
from Norfolk Naval Shipyard to work on the carrier's nuclear
propulsion systems in Mayport underwent a documented validation,
verification, and accreditation process. Similarly, the DOD's
facilities sustainment model that was used to calculate the recurring
facilities sustainment costs was also validated, verified, and
accredited. However, we did not find any evidence that any validated,
verified, or accredited model was used for calculating the costs of
transporting equipment for the carrier air wing, the cost of the
differential in the basic allowance for housing between Norfolk and
Mayport, the cost of the differential in utilities charges between
Norfolk and Mayport, or the cost of the permanent change of station
for the aircraft carrier crew. In terms of risk analysis, only the
military construction estimates contained any consideration of risk,
in the form of contingency factors.
Moreover, we found that the Navy has not assigned a single office with
the responsibility for ensuring that the overall estimate for Mayport
homeporting was packaged into a comprehensive and well documented
final estimate and that a sensitivity analysis was conducted on the
estimate to determine which cost elements, and which factors of those
cost elements, most influenced the overall cost. When an organization
does not assign a single office responsibility for assembling the
estimate into a comprehensive package, it is difficult to conduct
proper sensitivity and risk analyses. Additionally, different offices
preparing parts of the estimate may use conflicting assumptions. For
example, different offices may use different assumptions regarding the
treatment of inflation. Some costs may be expressed in base year
dollars, while others are expressed in then-year dollars. This was the
case with the Navy's original estimates which showed one-time costs in
then-year dollars and recurring costs in base year 2010 dollars. Even
if amounts are all expressed using the same type of dollars, different
contributors may use different inflation indexes, resulting in
conflicting underlying assumptions.
According to GAO's guide, cost estimates are credible when:
* major assumptions have been varied and other outcomes recomputed to
determine how sensitive outcomes are to changes in the assumptions,
* risk and uncertainty analyses have been performed to determine the
level of risk associated with the estimate, and:
* the estimate's results have been cross-checked and an independent
cost estimate has been developed to determine whether other estimating
methods produce similar results.
Sensitivity and risk analyses should be included in all cost estimates
as a best practice because all estimates have some uncertainty. A
sensitivity analysis addresses some of the estimating uncertainty by
testing discrete cases of assumptions and other factors that could
change. By examining each assumption or factor independently, while
holding all others constant, the cost estimator can evaluate the
results to discover which assumptions or factors most influence the
estimate. However, because many parameters could change at the same
time, a risk analysis should also be performed to capture the
cumulative effect of additional risks. Risk analysis adds to the
credibility of a cost estimate because it quantifies the uncertainty
and provides a level of confidence associated with the point estimate.
The results of a high-quality, reliable cost estimate should also be
cross-checked, and an independent cost estimate should be developed to
determine whether other estimating methods produce similar results. An
independent cost estimate is considered to be one of the most reliable
validation methods and is typically performed by organizations higher
in the decision-making process, or entirely outside of the office
performing the baseline cost estimate, using different estimating
techniques and, where possible, different data sources from those used
to develop the baseline cost estimate.
Performing a sensitivity analysis for the Mayport aircraft carrier
estimate would help to identify and focus on key elements with the
greatest effects on cost and understand the potential for cost growth
and the reasons for it. Additionally, because the Navy has not
conducted a risk and uncertainty analysis, it is unable to provide
Congress with a reasonably high confidence level for its Mayport
aircraft carrier cost estimate. Performing a risk and uncertainty
analysis would enable the Navy to quantify the risk and uncertainty
associated with the cost model; provide a level of confidence for its
cost estimate; and give decision makers perspective on the potential
variability of the cost estimate should facts, circumstances, and
assumptions change.
Navy officials told us that short time frames and budgetary
constraints prevented them from following the criteria for
comprehensiveness, accuracy, documentation, and credibility in
developing their estimate. For elements of the recurring costs, such
as the travel and per diem for public shipyard workers, the estimates
were produced in response to different congressional requests for
information often requiring quick turnaround--in some cases as short
as 3 days. Navy officials told us that in the case of the military
construction estimates, their intention was not to develop budget
quality estimates, but rather an estimate more suitable for initial
programming and planning purposes. Navy officials noted that this is a
typical approach for construction projects that are not scheduled to
begin for several years. According to the Navy, it would have been
cost prohibitive to develop budget quality estimates for every
proposed military construction project. Navy officials stated that the
Navy further reviews and refines the construction estimates as they
proceed through the DOD budget process. Because many of the projects
planned for Mayport would be included in future budgets, these
projects have not yet undergone this review. Consistent with prior
experience, it may not have been feasible for the Navy to follow cost
estimating best practices to the fullest extent in developing initial
estimates, especially given the short time frames often available.
However, as the Navy continues to respond to congressional interest in
the costs associated with homeporting a nuclear carrier at Mayport, it
has the opportunity to more completely incorporate cost estimating
best practices in its revised estimate, thereby improving the
estimate's quality and making it easier to defend in future budgets
and decision making.
Conclusions:
The Navy's ability to produce a comprehensive, accurate, well
documented, and credible cost estimate for homeporting a nuclear-
powered aircraft carrier at Naval Station Mayport will continue to be
hampered until it makes certain fundamental changes to the process it
uses to develop, document, and update its overall estimate of Mayport
homeporting costs. Specifically, without full documentation of the
data sources, assumptions, and calculation methods it uses, the Navy
cannot assure that its estimate can be validated or defended or any
differences between estimated and actual costs can be explained--an
important step in improving and updating the estimate. Additionally,
without detailed documentation that describes how the estimate was
derived, the Navy can neither present a convincing argument of the
estimate's affordability, nor credibly answer decision makers' and
oversight groups' questions about specific details in the estimate.
Further, without conducting sensitivity and risk and uncertainty
analyses on its cost estimate, the Navy is unable to identify and
focus on major cost drivers, analyze the potential for cost growth,
and quantify the risk and uncertainty associated with the cost
estimate. Moreover, without a comprehensive, accurate, well
documented, and credible cost estimate, Congress cannot have
reasonable confidence that it has a complete understanding and an
accurate and realistic determination of the projected costs to
evaluate and make decisions on the Navy's planned homeporting of a
nuclear-powered aircraft carrier at Mayport.
Recommendations for Executive Action:
To improve the Navy's life-cycle cost estimate for the planned
homeporting of a nuclear-powered aircraft carrier at Naval Station
Mayport, Florida, we recommend that the Secretary of Defense direct
the Secretary of the Navy to take the following three actions to
incorporate to a greater extent the best practices identified by GAO
for developing a high-quality cost estimate in future revisions of its
Mayport nuclear carrier homeporting cost estimate as part of the
annual budgetary process or in response to future congressional
requests:
1. To improve the comprehensiveness of its cost estimate, the Navy
should:
* include all potential recurring costs, and:
* clearly describe the ground rules and assumptions underlying the
estimation of each cost element;
2. To improve the quality and transparency of the Navy's estimate, the
Navy should thoroughly document the life-cycle costs associated with
homeporting a nuclear-powered aircraft carrier at Naval Station
Mayport. Specifically, documentation should:
* identify the source data used, their reliability, and how the data
were normalized,
* describe the steps used in developing the overall estimate so that
it can be clearly understood and easily replicated, and:
* describe in sufficient detail the estimating methodology and
calculations performed to derive each element's cost; and:
3. To improve the accuracy and credibility of its cost estimate, the
Navy should assign a single office with the responsibility for
assembling the overall estimate into a comprehensive and well
documented package and for performing a sensitivity and risk and
uncertainty analyses on the overall estimate to identify the:
* major cost drivers,
* extent to which estimates could vary due to changes in key cost
assumptions, and:
* level of confidence in the estimate.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD partially concurred
with two and nonconcurred with one of our three recommended actions.
DOD also provided technical comments that we have incorporated into
this report where applicable. DOD's written comments are reprinted in
appendix IV.
DOD partially concurred with our two recommendations that the
Secretary of Defense direct the Secretary of the Navy to improve the
(1) comprehensiveness of the Navy's life-cycle cost estimate for the
planned homeporting of a nuclear-powered aircraft carrier at Naval
Station Mayport by including all potential recurring costs and clearly
describing the ground rules and assumptions underlying the estimation
of each cost element and (2) quality and transparency of the Navy's
estimate by thoroughly documenting the life-cycle costs associated
with the planned homeporting. DOD stated that the elements, or cost
estimating best practices, cited in our recommendations may contribute
to improved quality in cost estimates, but they may not be ideally
suited to the myriad situations in which the Navy is required, or may
be requested, to provide cost estimates to Congress.
We recognize that it may not have been feasible for the Navy to always
follow cost estimating best practices to the fullest extent in
developing some of its initial estimates, especially when the
preparation time for some estimates may have been limited. However,
both recommendations were directed at improving future Navy cost
estimates associated with the Mayport carrier homeporting. As the Navy
continues to respond to congressional interest, we believe it has the
opportunity to more completely incorporate the cost estimating best
practices we identified in our report in its revised estimates for
Mayport in order to improve the quality of its estimates.
Additionally, DOD states that the similarity between our initial
estimate and the Navy's budget estimate for dredging costs suggests
that the Navy's programming/budgeting process leads to high-quality,
credible cost estimates. However, the Navy's overall Mayport estimate
currently includes several cost elements that have not been refined
through a programming/budgeting process. Furthermore, as described in
our report, GAO's criteria for assessing high-quality, credible
estimates involve factors that go beyond the accuracy of any
individual point estimate. The convergence between our estimate and
the Navy's estimate on a single cost element is not an indication by
itself of the quality of the Navy's overall estimate for homeporting a
nuclear carrier at Mayport. Improving the Mayport estimate's quality
will allow the Navy to more easily defend its plans in future
decisionmaking and budgets within DOD. We believe that in order to
best inform the budget decisionmaking process of the likely costs,
affordability, and scheduling of funding needed to support the Navy's
plans for Mayport, DOD should take every available opportunity to
provide Congress with the highest quality cost estimates possible. We
therefore believe these two recommendations remain valid.
DOD disagreed with our third recommendation that the Secretary of
Defense direct the Secretary of the Navy to improve the accuracy and
credibility of its cost estimate by assigning a single office with the
responsibility for assembling the overall estimate into a
comprehensive and well documented package and for performing
sensitivity and risk and uncertainty analyses on the overall estimate.
DOD stated that responsibilities for centralized supervision and
coordination of programming and budgeting within the Department of the
Navy are well-established and that Navy program/budget estimates and
exhibits are prepared and submitted in accordance with the Office of
the Secretary of Defense guidance. DOD further noted that reports to
Congress are typically assigned to a central office for preparation
and coordination and that where necessary, instructions to the offices
providing inputs can include guidance on baseline assumptions.
As we state in our report, the Navy's level of coordination used in
packaging its final cost estimate did not ensure that the offices
providing inputs adequately and completely documented the cost
estimates and did not provide for the implementation of sensitivity
and risk and uncertainty analyses. Additionally, the Navy's final cost
estimate was not consistent in its use of various baseline
assumptions, such as consistency in using either then-year or constant
dollars in the individual original estimates that made up the Navy's
final estimate for Mayport. This lack of consistency limits the Navy's
ability to appropriately combine the estimates from different offices
into a single, coherent cost estimate. High-quality cost estimating
requires good organization and a consistent approach in pulling
together disparate data for each cost element and packaging it as an
accurate and credible whole. Thus, without a single office--within the
already established programming/budgeting process--that is responsible
for assembling future revisions of its Mayport estimate, the Navy will
continue to lack assurance that its overall estimate is comprehensive,
well documented, accurate, and credible. We therefore believe this
recommendation remains valid.
We are sending copies of this report to interested congressional
committees, the Secretary of Defense, the Secretary of the Navy, and
the Director, Office of Management and Budget. The report is also
available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. Key contributors to this report are
listed in appendix V.
Signed by:
Brian J. Lepore:
Director, Defense Capabilities and Management:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Howard P. McKeon:
Chairman:
The Honorable Adam Smith:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable C.W. Bill Young:
Chairman:
The Honorable Norman D. Dicks:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To develop an independent cost estimate of the full life-cycle costs
associated with the Navy's plan to establish a second East Coast
homeport for a nuclear-powered aircraft carrier we worked with a
consultant from a cost estimating company--Technomics, Inc.--with
expertise in preparing estimates of the full life-cycle costs of major
federal programs.[Footnote 20] We followed the 12 steps for developing
a high-quality cost estimate outlined in the GAO Cost Estimating and
Assessment Guide.[Footnote 21]
Step 1: Define Estimate's Purpose:
The purpose of the estimate is to satisfy the direction provided to
GAO in House Report 111-491, accompanying a proposed bill for the
Fiscal Year 2011 National Defense Authorization Act (H.R. 5136), to
develop an independent estimate of the total direct and indirect costs
to be incurred by the federal government for the proposed homeporting
of an nuclear-powered aircraft carrier at Naval Station Mayport.
[Footnote 22] The estimate will provide the defense committees with an
independent cost estimate against which they can evaluate the Navy's
estimate and give Congress greater confidence that it has an accurate
understanding of the total cost associated with establishing a
homeport for a nuclear-powered aircraft carrier at Naval Station
Mayport.
Step 2: Develop Estimating Plan:
Upon identification of the direction in the committee report, we
identified resources and developed a design matrix that included
identification of information required and sources, and the proposed
scope and methodology for addressing the direction. Additionally, in
accordance with Generally Accepted Government Auditing Standards
(GAGAS), we conducted an entrance conference with the Navy, collected
cost data, and conducted interviews with program officials.
Step 3: Define Program Characteristics:
We collected information directly from the Navy that enabled us to
define the program's technical and programmatic baseline.
Specifically, the Navy provided us with its estimates of one-time and
recurring costs, including any supporting documentation that outlined
the underlying assumptions and ground rules supporting the estimates.
[Footnote 23] We also reviewed the Navy's environmental impact study
along with background information about nuclear-powered aircraft
carrier maintenance, visited locations where nuclear-powered aircraft
carrier maintenance is performed, and spoke with Navy officials
knowledgeable about the personnel and facilities requirements for
nuclear-powered aircraft carrier maintenance.
Step 4: Determine Estimating Structure:
We developed a cost element structure that was derived from the Navy's
cost estimate and supporting documentation. Where necessary, we added
subelements or additional elements. At the highest level of our
estimate, costs were categorized by one-time costs (1.0 elements) and
recurring costs (2.0 elements). For each cost element, the most
appropriate estimating approach was identified according to data
availability and credibility. Estimating approaches applied included
analogy, expert opinion, and parametric methods.
Step 5: Identify Ground Rules and Assumptions:
Estimating ground rules and assumptions were identified and
documented. Assumptions were minimized, relying instead on real data
when possible. In order to develop an independent estimate, we
generally used methods and data that differed from the Navy's, relying
instead on two types of estimating methods: scaled analogy and
parametric. Scaled analogy is a cost estimating method that relies on
using actual costs from analogous projects. Adjustments to cost were
made for the relative size of the analogous project as well as
appropriate adjustments for complexity factors. A parametric estimate
is a method that uses an established relationship between cost and one
or more input variables. The input variables become parameters in an
equation that is used to predict cost. It is important to note,
however, that the Navy was the only source of information for certain
cost subelements, such as the number of man-days of maintenance
required for aircraft carrier availabilities and the cost of
transporting equipment for the carrier air wing and this limited the
degree to which our cost estimate is completely independent from the
Navy's for those elements.
Step 6: Obtain Data:
Cost, technical, and programmatic data were solicited and collected
through interviews and supporting documentation. Interviews were
conducted with and supporting documentation was obtained from
officials in the Office of the Chief of Naval Operations, Naval Sea
Systems Command, Regional Maintenance Centers, as well as officials
from Norfolk Naval Shipyard, Naval Station Mayport, and Naval Air
Station North Island. Specific documents and data we collected
included Navy responses to information requests, programming models,
and construction planning documents. Additionally, we collected and
reviewed data on unrelated but analogous Navy and DOD construction
projects. In addition, we analyzed the uncertainty associated with our
estimates to help gauge their accuracy. We assessed the reliability of
all data provided by the Navy by (1) reviewing existing information
about the data and the systems that produced them, (2) interviewing
agency officials knowledgeable about the data to determine the steps
taken to ensure the accuracy and completeness of the data, and (3)
reviewing the related internal controls. Based on this assessment, we
determined the data were sufficiently reliable for our purposes.
Step 7: Develop Point Estimate[Footnote 24]
Data collected were thoroughly analyzed, resulting in the development
of several different cost estimating methodologies. These
methodologies were integrated into a comprehensive cost model using
Microsoft® Excel, resulting in a detailed point estimate.
Step 8: Conduct Sensitivity Analysis:
A comprehensive sensitivity analysis was conducted on the cost model
also using Microsoft ® Excel. Sensitivity analysis addresses the
question of which cost model inputs have the greatest impact on the
overall cost estimate. Inputs that have the greatest impact are often
referred to as cost drivers. Our sensitivity analysis identified all
of the inputs used in each cost element. Each input was changed, in
isolation, to determine the impact on the bottom line cost. Some
inputs, such as geographic adjustment factors, have an impact on
multiple elements. Other inputs only impact a specific cost element.
Step 9: Conduct Risk and Uncertainty Analysis:
Risk and uncertainty analysis was conducted in order to transform the
static point estimate into a probabilistic range based on the risk and
uncertainty inherent in the model input variables. Within the risk
analysis section of our estimate, each cost input and each cost
estimating method was assigned a low and high value based on the
relative uncertainty associated with each value. Inputs that had
little or no uncertainty (such as actual costs, or known square
footages) had a small, or possibly a zero range between the high and
low values. In contrast, variables with a high level of uncertainty
had a wider range. We then assigned a triangular probability
distribution to each element, defined as either a low value, a most
likely value (the point estimate), or a high value. Using a Monte
Carlo simulation, the risk analysis was then assigned a random value,
according to the probability distribution of each variable. Total cost
was then calculated and the process was repeated to obtain a range of
possible total cost outcomes. The simulation resulted in establishing
probability distributions about the estimated values, enabling the
identification of the confidence level of the point estimate.
Step 10: Document the Estimate:
Detailed documentation was generated that provides the cost estimate
results, as well as ground rules and assumptions, and all underlying
cost estimating methodology. The cost model was thoroughly referenced
to source documents and then traced and verified in accordance with
generally accepted government auditing standards.
Step 11: Present Estimate to Management for Approval:
The resulting cost estimates were presented to various levels of GAO
management. Additionally, GAO presented the cost estimate and
underlying methodology in detail to the Navy in January 2011.
Step 12: Update the Estimate to Reflect Actual Costs and Changes:
Upon presenting the cost estimate and underlying methodology in detail
to Navy, the Navy provided comments, questions, and some additional
data which we incorporated as appropriate in our estimate.
Within each cost element, we compared our estimate to the Navy's.
Wherever possible, the reasons for significant differences were
discussed with the Navy and explained in the report. Our estimate does
not estimate or evaluate any of the benefits of a potential homeport
move, and therefore does not address or evaluate the military
advantages or disadvantages associated with the homeporting decision.
Additionally, our estimate only includes estimated costs incurred by
the Navy and it does not include any estimate of the economic impact
to the local economy in Virginia or Florida.
To determine to what extent the estimate adheres to the
characteristics of a high-quality cost estimate, we evaluated the
Navy's life-cycle cost estimate to determine whether it met key
characteristics identified in the GAO Cost Estimating and Assessment
Guide. Our guide, which is based on extensive research of best
practices for estimating program schedules and costs, indicates that a
high-quality, valid, and reliable cost estimate should be well
documented, comprehensive, accurate, and credible. We analyzed the
cost estimating practices used by the Navy against these best
practices. Specifically, to determine whether the Navy's estimate was
well documented, we examined whether the Navy estimate's documentation
explicitly identified the primary methods, calculations, results,
rationales or assumptions, and sources of data used to generate each
cost element. To determine whether the estimate was comprehensive, we
examined whether the Navy's cost estimate included all life-cycle
costs required to homeport a nuclear aircraft carrier at Naval Station
Mayport. To determine whether the Navy's estimate was accurate, we
verified the calculations in the Navy's estimate and examined whether
the estimate accounted for all costs, including indirect costs.
Finally, to determine whether the Navy's estimate was credible, we
examined the extent to which the Navy tested key cost elements for
sensitivity or used other cost estimating techniques to cross-check
the reasonableness of their ground rules and assumptions. We also
interviewed Navy officials from the Naval Sea Systems Command and the
Office of the Chief of Staff of Naval Operations about the process
used to prepare the life-cycle cost estimates and the assumptions used
to prepare the Navy's estimate. We shared the GAO Cost Estimating and
Assessment Guide and the criteria against which we would be evaluating
the Navy's cost estimate with the Navy.
We conducted our performance audit from July 2010 through February
2011 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Additional Information on GAO's Independent Cost Estimate:
Nuclear-Powered Aircraft Carrier Homeporting Costs at Naval Station:
Mayport, Florida:
GAO's Independent Cost Estimate:
Contents:
* Purpose of the Estimate;
* Program Characteristics;
* Estimating Structure;
* Ground Rules and Assumptions;
* Methodology;
* Overview;
* Comparison of GAO's 65 Percent Confidence Level Estimates and Navy's
Point Estimates;
* Risk-adjusted Ranges;
* Individual Cost Elements.
Purpose of the Estimate:
* Satisfy the direction in House Report 111-491, accompanying a
proposed bill for the Fiscal Year 2011 National Defense Authorization
Act (H.R. 5136), to develop an independent estimate of the total
direct and indirect costs to be incurred by the federal government for
the proposed homeporting of a nuclear-powered aircraft carrier at
Naval Station Mayport.
* Provide the congressional defense committees with an independent
cost estimate against which they can evaluate the Navy's estimate and
give Congress greater confidence that it has an accurate understanding
of the total cost associated with establishing a homeport for a
nuclear-powered aircraft carrier at Naval Station Mayport, Florida.
Program Characteristics:
The controlled industrial facility and ship maintenance support
facilities in Mayport have been modeled on the facilities built at
Naval Air Station North Island, San Diego, California in the late
1990s to support the first nuclear-powered aircraft carriers to be
homeported in San Diego.
Nuclear-powered aircraft carrier maintenance is performed on a 32-
month cycle that includes two 1-month long Carrier Incremental
Availabilities and either one 6-month Planned Incremental Availability
or a 10.5-month Docking Planned Incremental Availability. There is
also one refueling complex overhaul around the carrier's midlife that
lasts about 39 months.
The Carrier Incremental Availabilities and Planned Incremental
Availabilities will be performed in Mayport, while the Docking Planned
Incremental Availabilities will be performed at Norfolk Naval Shipyard.
Estimating Structure:
The cost element structure used in this estimate was derived from the
Navy's cost estimate and associated documentation. Where necessary, we
have added subelements or additional elements. At the highest level,
costs are categorized by onetime costs (1.0 elements) and recurring
costs (2.0 elements).
Table: Estimating Structure:
1.0: One-tines costs:
1.1: Planning and Design;
1.2: Dredging;
1.3: Construction:
1.3.1: Parking;
1.3.2: Road Improvements;
1.3.3: Wharf F Improvements;
1.3.4: Controlled Industrial Facility (CIF);
1.3.5: Ship Maintenance Facility (SMF);
1.3.6: Maintenance Support Facility (MSF);
1.4: Initial Outfitting;
1.4.1: CIF Equipment;
1.4.2: SMF Equipment;
1.4.3: MSF Equipment;
1.5: Permanent Change of Station for the Crew.
2.0 Recurring Costs:
2.1: Permanently Assigned Labor for Nuclear Facilities;
2.2: Base Operating Support for New Facilities;
2.2.1: Entry Control Point Sentries;
2.2.2: Facilities Planning, Facilities Services, and Transportation;
2.3: Facilities Sustainment;
2.4: Facilities Restoration and Modernization;
2.5: Operations;
2.6: Travel/Per Diem for Government Maintenance Workers, Annualized;
2.6.1: Carrier Incremental Availabilities;
2.6.2: Planned Incremental Availabilities;
2.6.3: Voyage Repair;
2.7: Biennial Dredging;
2.8: Basic Allowance for Housing Differential;
2.9: Utilities;
2.10: Permanent Change of Station for the Crew;
2.10.1: Docking Planned Incremental Availability;
2.10.2: Refueling Complex Overhaul;
2.11: Private Sector Travel.
Source: GAO analysis.
[End of table]
Ground Rules and Assumptions:
The following general assumptions apply to all cost elements:
* In order to develop an independent estimate, we generally use
methods and data that differ from the methods and data used in the
Navy's cost estimate. However, the Navy is likely the only source of
information for certain cost subelements such as the number of man-
days of maintenance required for aircraft carriers and the cost of
transporting equipment for the carrier air wing.
* The time frame for one-time costs will be consistent with the Navy's
estimate, which spans from fiscal year 2010 to fiscal year 2019.
* The recurring costs have been annualized over the 50-year carrier
life cycle. Consistent with the Navy's estimate, we have not
identified an exact start date for the recurring costs.
* We estimated the cost differentials associated with moving a nuclear-
powered aircraft carrier to Mayport. This means that our estimate only
includes costs that differ from what the Navy would spend under the
current mode of homeporting all East Coast carriers in Norfolk,
Virginia. This assumption is consistent with the Navy's cost estimate.
* Both one-time and recurring costs will be reported in base-year 2010
dollars, which have the effects of inflation removed. In calculations
where inflation needs to be added or removed, the estimate uses the
appropriation-specific cost indexes published annually by the Naval
Center for Cost Analysis.
* We do not estimate or evaluate any potential economic impacts that
may be experienced by either the Norfolk or Mayport communities from
the planned homeporting. As directed in House Report 111491,
accompanying a proposed bill for the Fiscal Year 2011 National Defense
Authorization Act (H.R. 5136), our estimate is limited to the costs to
be incurred by the federal government in homeporting a nuclear-powered
aircraft carrier at Naval Station Mayport.
Methodology:
The data that we collected were integrated into a comprehensive cost
model that we developed using Microsoft® Excel, resulting in cost
estimates for each individual cost element, as well as overall cost
estimates.
To develop the cost estimates, we worked with an outside cost
estimating company with expertise in preparing estimates of the life-
cycle costs of major federal acquisitions.
We then conducted a sensitivity analysis to determine which of the
cost model inputs had the greatest impact on the overall cost element.
Those inputs are known as cost drivers.
Next, we conducted a risk and uncertainty analysis as a way to assess
the variability in the point estimate. Risk analysis addresses the
uncertainty that is inherent within cost inputs and cost estimating
methods. Inputs that have little or no uncertainty (such as sunk
costs, or known square footages) will have a small, or possibly a zero
range between the high and low values. In contrast, variables with a
high level of uncertainty have a wider range.
The results of the risk analysis can also be viewed as an "S-Curve",
which plots various risk-adjusted cost totals against a cumulative
confidence level. The probabilities shown on the Y axis of an S curve
represent the probability that costs will be equal to or less than the
amount shown on the X axis. For example, to obtain the risk-adjusted
cost estimate at which there is a 90 percent probability that the true
costs will be equal to or less than the risk-adjusted total, determine
the dollar value from the X axis where the S curve crosses 90 percent.
Overview:
Our independent cost estimate suggests that the one-time cost for
establishing a nuclear-powered aircraft carrier homeport at Naval
Station Mayport is expected to be between $258.7 million and $356.0
million, with a 65 percent confidence level estimate of $322.0
million. The recurring costs are expected to be between $9.0 million
and $17.6 million per year, with a 65 percent confidence level
estimate of $14.3 million.
The Navy's point estimate of $537.6 million for the one-time cost
falls outside the upper range of our estimate, but the Navy's point
estimate of $15.3 million/year for recurring costs does fall within
our estimated range. he largest difference between our estimate and
the Navy's is in the one-time construction costs for the new
controlled industrial facility and ship maintenance support facilities
in Mayport.
The parameters related to the construction of the controlled
industrial facility and the ship maintenance support facilities are
the cost drivers for the one-time costs, while the parameters related
to the travel cost are the cost drivers for the recurring costs.
Figure: Total Homeporting Cost:
[Refer to PDF: two vertical bar graphs]
One-time cost, base-year 2010:
GAO: $322.0 million;
Navy: $537.6 million.
Recurring costs, base-year 2010:
GAO: $14.3 million;
Navy: $15.3 million.
Source: GAO analysis.
[End of figure]
Figure: One-Time Cost:
[Refer to PDF: two vertical bar graphs]
GAO estimate - $322.0 million:
Planning and design: $19.7 million;
Initial dredging: $32.3 million;
Construction: $218.1 million;
Equipment: $48.4 million;
PCS: $5.5 million.
Navy estimate - $537.6 million:
Planning and design: $30.0 million;
Initial dredging: $46.3 million;
Construction: $378.3 million;
Equipment: $73.0 million;
PCS: $10.0 million.
Source: GAO analysis.
It is not statistically appropriate to add each of the individual 65
percent confidence level cost estimates to come up with an overall 65
percent confidence level. The overall 65 percent confidence level is
calculated separately from the individual elements.
[End of figure]
Figure: Construction Cost:
[Refer to PDF: two vertical bar graphs]
GAO estimate - $218.1 million:
Parking garage: $38.5 million;
Road Improvements: $18.4 million;
Wharf F Improvements: $58.2 million;
Controlled Industrial Facility: $70.5 million;
Ship maintenance support facilities: $45.6 million.
Navy estimate - $378.3 million:
Parking garage: $27.8 million;
Road Improvements: $15.3 million;
Wharf F Improvements: $38.9 million;
Controlled Industrial Facility: $139.1 million;
Ship maintenance support facilities: $157.2 .
Source: GAO analysis.
It is not statistically appropriate to add each of the individual 65
percent confidence level cost estimates to come up with an overall 65
percent confidence level. The overall 65 percent confidence level is
calculated separately from the individual elements.
[End of figure]
Figure: Recurring Costs:
[Refer to PDF: two vertical bar graphs]
GAO estimate - $14.3 million:
Permanent labor: $1.4 million;
Base operating support: $1.1 million;
Facilities sustainment: $1.0 million;
Facilities modernization: $2.9 million;
Operations: $1.0 million;
Travel: $9.1 million;
Maintenance dredging: $0.1 million;
Basic allowance for housing differential: -$4.8 million;
Utilities: $1.3 million;
Permanent change of station: $1.2 million;
Private sector travel: $0.6 million.
Navy estimate - $15.3 million:
Permanent labor: $1.2 million;
Base operating support: $1.0 million;
Facilities sustainment: $0.8 million;
Facilities modernization: $6.8 million;
Operations: $0.8 million;
Travel: $7.0 million;
Maintenance dredging: $0.1 million;
Basic allowance for housing differential: -$3.2 million;
Utilities: $0.8 million;
Permanent change of station: $0;
Private sector travel: $0.
Source GAO analysis.
It is not statistically appropriate to add each of the individual 65
percent confidence level cost estimates to come up with an overall 65
percent confidence level. The overall 65 percent confidence level is
calculated separately from the individual elements.
[End of figure]
Risk-adjusted Range: One-Time Cost:
The 65 percent confidence level estimate ($322.0 million) is plotted,
along with the 10 percent ($258.7 million) and 90 percent ($356.0
million) confidence levels, which when reported as a range represent
an 80 percent confidence interval.
A confidence interval describes a range of possible costs, based on a
specified probability level that is determined through a risk
analysis. The low and high values of the confidence interval can be
expressed as confidence levels that represent the probability that the
point estimate will be met. Our 80 percent confidence interval is the
range of costs from the 10 percent confidence level to the 90 percent-
confidence level. At the 10 percent confidence level, there is a 10
percent chance that the cost will be at or below the estimate and a 90
percent chance that the cost will be above the estimate. Likewise at
the 90 percent confidence level, there is a 90 percent chance that the
cost will be at or below the estimate and a 10 percent chance that the
final cost will be above the estimate.
Figure: Risk-adjusted Range: One-Time Cost:
[Refer to PDF for image: line graph]
$258.7 million: 10%;
$322.2 million: 65%;
$356.0 million: 90%.
Source: GAO analysis.
[End of figure]
Risk-adjusted Range: Recurring Costs:
The 65 percent confidence level estimate ($14.3 million) is plotted,
along with the 10 percent ($9.0 million) and 90 percent ($17.6
million) confidence levels, which when reported as a range represent
an 80 percent confidence interval.
Figure: Risk-adjusted Range: Recurring Costs:
[Refer to PDF for image: line graph]
$9.0 million: 10%;
$14.3 million: 65%;
$17.6 million: 90%.
Source: GAO analysis.
[End of figure]
One-Time Cost Element: Planning and Design:
* GAO risk-adjusted range - $15.0 million to $22.9 million;
* GAO 65 percent confidence level estimate - $19.7 million;
* Navy point estimate - $ 30.0 million.
The Navy's estimate is outside of GAO's risk-adjusted range. Both GAO
and the Navy estimated planning and design as a percentage of the
overall construction cost. The difference between the two planning and
design estimates reflects the difference in the overall estimates of
construction costs.
One-Time Cost Element: Initial Dredging:
* GAO risk-adjusted range - $31.3 million to $33.1 million;
* GAO 65 percent confidence level estimate - $ 32.3 million;
* Navy point estimate - $ 46.3 million.
GAO originally used a scaled analogy based on similar dredging at
Norfolk Naval Shipyard and produced an estimate similar to The Navy's.
However, we have updated our estimate for dredging at Mayport to
reflect the cost of the contract that was awarded in August 2010. The
contract amount was $26.6 million, and the GAO estimate includes
overhead and contingency costs. The Navy stated that the contract
price came in lower than the estimate due to the widely varying nature
of costs in the dredging industry. According to a Navy official, the
dredging price is largely based on equipment availability. If
equipment happens to be available at the required time, then the
dredging cost will be lower.
One-Time Cost Element: Parking Garage:
* GAO risk-adjusted range - $21.9 million to $53.3 million;
* GAO 65 percent confidence level estimate - $ 38.5 million;
* Navy point estimate - $ 27.8 million.
The Navy's estimate is within the GAO risk-adjusted range. GAO's
estimate is based on cost per square foot, adjusted with cost factors
for supervision, inspection, and overhead; contingency; and a
geographic factor for northeast Florida. The Navy's estimate is based
on a detailed engineering estimate developed by a private engineering
firm.
One-Time Cost Element: Road Improvements:
* GAO risk-adjusted range - $9.5 million to $24.3 million;
* GAO 65 percent confidence level estimate - $18.4 million;
* Navy point estimate - $15.3 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate is based on a scaled analogy of road construction projects
for other locations. The Navy's estimate is based on a detailed
engineering estimate developed by a private engineering firm.
One-Time Cost Element: Wharf F Improvements:
* GAO risk-adjusted range - $28.3 million to $75.3 million
* GAO 65 percent confidence level estimate - $58.2 million
* Navy point estimate - $38.9 million
* The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate is based on an average of analogous wharf construction
projects from other locations. Adjustments were made for size,
geographic location, and inflation. The Navy's estimate is based on a
detailed engineering estimate developed by a private engineering firm.
One-Time Cost Element: Controlled Industrial Facility:
* GAO risk-adjusted range - $35.2 million to $94.9 million;
* GAO 65 percent confidence level estimate - $70.5 million;
* Navy estimate - $139.1 million.
The Navy's estimate is not within GAO's risk-adjusted range. The
Navy's estimate is based on a detailed engineering study developed by
a private engineering firm. The GAO estimate is based on historical
costs observed for analogous-facility construction projects at Naval
Station North Island (San Diego). These analogous costs are adjusted
to account for differences in size, inflation, geographical
differences between the two areas, and any identified complexities or
requirements unique to the Mayport construction. Those complexity
factors included construction to withstand Class 4 hurricane storm
surge and costs associated with compliance with environmental
standards and with antiterrorism and force protection. These factors
were included in GAO's estimate based on a percentage of the total
construction cost. No other specific complexity factors that would
account for the discrepancy in the two estimates have been identified
by either GAO or the Navy.
One-Time Cost Element: Ship Maintenance Support Facilities:
* GAO risk-adjusted range - $23.0 million to $59.3 million;
* GAO 65 percent confidence level estimate - $45.6 million;
* Navy estimate - $157.2 million.
The Navy's estimate is substantially outside of GAO's risk-adjusted
range. The Navy's estimate is based on a detailed engineering study
developed by a private engineering firm. The GAO estimate is based on
historical costs observed-for analogous facility construction projects
at Naval Station North Island. These analogous costs are adjusted to
account for differences in size, inflation, geographical differences
between the two areas, and any identified complexities or requirements
unique to the Mayport construction. Those complexity factors included
compliance with environmental standards and with antiterrorism and
force protection. These factors were included in GAO's estimate based
on a percentage of the total construction cost. No other specific
complexity factors that would account for the significant difference
between the two estimates have been identified by either GAO or the
Navy.
One-Time Cost Element: Initial Equipment Outfitting:
* GAO risk-adjusted range - $24.5 million to $64.4 million;
* GAO 65 percent confidence level estimate - $48.4 million;
* Navy point estimate - $73.0 million.
The Navy's estimate is outside of GAO's risk-adjusted range. GAO's
estimate is based on a scaled analogy of equipment purchased at Naval
Air Station North Island, San Diego. The Navy's estimate is based on a
detailed estimate that lists all equipment purchase requirements, with
the cost of commercial pricing estimated for each item.
One-Time Cost Element: Initial Permanent Change of Station:
* GAO risk-adjusted range - $4.7 million to $6.0 million;
* GAO 65 percent confidence level estimate - $5.5 million;
* Navy point estimate - $10.0 million.
The Navy's estimate is outside of GAO's risk-adjusted range. GAO's
estimate is based on published DOD dislocation allowance rates, by
rank, for crew members with and without dependents. It assumes a
single permanent change of station move when the homeport location is
changed. The Navy's estimate appears to be based on additional costs
other than the dislocation allowance rates, but the exact nature of
those costs was not specified.
Recurring Cost Element: Permanently Assigned Labor:
* GAO risk-adjusted range - $0.7 million to $1.9 million;
* GAO 65 percent confidence level estimate - $1.4 million;
* Navy point estimate - $1.2 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate uses the Navy's assumption of 12 full-time employees at a
rate of $100,000 per year. We confirmed that this assumption is
consistent with the staffing level and compensation for employees
working at the nuclear carrier maintenance facilities at Naval Air
Station North Island, San Diego. GAO's 65 percent confidence level
estimate is slightly higher than the Navy's point estimate because it
has been adjusted through a risk and uncertainty analysis, whereas the
Navy's estimate has not.
Recurring Cost Element: Base Operating Support:
* GAO risk-adjusted range - $0.7 million to $1.4 million;
* GAO 65 percent confidence level estimate - $1.1 million;
* Navy point estimate - $1.0 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate uses the Navy's assumptions of 10 full-time guards at $75,000
per year and $225,000 per year for facilities services based on a
fixed unit cost per square foot. The unit cost is based on an average
cost for Naval Station Mayport in fiscal year 2008. GAO's 65 percent
confidence level estimate is slightly higher than the Navy's point
estimate because it has been adjusted through a risk and uncertainty
analysis, whereas the Navy's estimate has not.
Recurring Cost Element: Facilities Sustainment:
* GAO risk-adjusted range - $0.7 million to $1.2 million;
* GAO 65 percent confidence level estimate - $1.0 million;
* Navy point estimate - $0.8 million.
The Navy's estimate is within GAO's risk-adjusted range. Both
estimates are based on facilities sustainment rates published in the
DOD Facilities Pricing Guide, a program analysis and budget tool that
lists replacement, sustainment, modernization, and operation unit
costs for DOD facilities. Adjustments are made for geographic location.
Recurring Cost Element: Facilities Restoration and Modernization:
* GAO risk-adjusted range - $2.2 million to $3.3 million;
* GAO 65 percent confidence level estimate - $2.9 million;
* Navy estimate - $6.8 million.
The Navy's estimate is outside GAO's risk-adjusted range. GAO's
estimate is based on modernization factors published in the DOD
Facilities Pricing Guide. Each factor is multiplied against the
estimated construction cost. The Navy's estimate assumes a 67-year
recapitalization rate. The modernization factors and 67-year
recapitalization rate result in very similar percentages, but because
the Navy's construction estimates are higher, the restoration and
modernization rates are correspondingly higher in the Navy's estimate.
Recurring Cost Element: Operations:
* GAO risk-adjusted range - $0.5 million to $1.4 million;
* GAO 65 percent confidence level estimate - $1.0 million;
* Navy point estimate - $0.8 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate is based on the assumptions and amounts used in the Navy's
estimate. GAO's 65 percent confidence level estimate is slightly
higher than the Navy's point estimate because it has been adjusted
through a risk and uncertainty analysis, whereas the Navy's estimate
has not.
Recurring Cost Element: Travel for Navy Shipyard Workers:
* GAO risk-adjusted range - $4.8 million to $12.2 million;
* GAO 65 percent confidence level estimate - $9.1 million;
* Navy point estimate - $7.0 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate is based on a scaled analogy of travel expenses incurred
during a 2009 planned incremental availability on the USS Ronald
Reagan. Adjustments were made for inflation and differing airfare and
per diem rates in the Jacksonville, Florida area. The Navy's estimate
is based on a programming model that breaks down the travel expense
into component costs for airfare, per diem, hotel tax, rental car, and
gas.
Recurring Cost Element: Biennial Maintenance Dredging:
* GAO risk-adjusted range - $0.1 million to $0.2 million;
* GAO 65 percent confidence level estimate - $0.1 million;
* Navy point estimate - $0.1 million.
The Navy's estimate is within GAO's risk-adjusted range. GAO's
estimate is based on the assumptions and amounts used in the Navy's
estimate. For this cost element, GAO's 65 percent confidence level
estimate is identical to the Navy's point estimate even though the
Navy's estimate was not adjusted through a risk and uncertainty
analysis like GAO's.
Recurring Cost Element: Basic Allowance for Housing:
* GAO risk-adjusted range - -$5.5 million to -$4.4 million
* GAO 65 percent confidence level estimate -$4.8 million
* Navy point estimate - -$3.2 million
The Navy's estimate is outside GAO's risk-adjusted range. Both
estimates are based on published rates, by rank, for crew members with
and without dependents. The difference between the estimates is due to
the fact that GAO's estimate assumes a crew size of 3,140, which is
consistent with the Navy's Environmental Impact Study and other
sources. The Navy's estimate assumes a crew size of 2,728, which
lowers the total savings.
Note: The estimate for this cost element is negative because the
housing allowance is less in Jacksonville, Florida than it is in
Norfolk, Virginia. This means that there is a net savings for the Navy
for this cost element.
Recurring Cost Element: Utilities:
* GAO risk-adjusted range - $0.6 million to $1.8 million;
* GAO 65 percent confidence level estimate - $1.3 million;
* Navy point estimate - $0.8 million.
The Navy's estimate is within GAO's risk-adjusted range. Both
estimates are based on the same assumed amount of power usage
multiplied by an average power cost differential. GAO's estimate uses
average power costs published by the Department of Energy for Virginia
and Florida, which is a differential of 2.51 cents. The Navy's
estimate assumes a 2.00 cent differential.
Recurring Cost Element: Recurring Permanent Change of Station:
* GAO risk-adjusted range - $1.0 million to $1.3 million;
* GAO 65 percent confidence level estimate - $1.2 million;
* Navy point estimate - N/A.
GAO's estimate is based on the permanent change of station costs
calculated as part of the one-time costs and applies that amount for
all docking planned incremental availabilities and the midlife
refueling complex overhaul. These costs are annualized over the 50-
year carrier life cycle. The Navy's estimate does not include any
permanent change of station costs associated with docking planned
incremental availabilities or the midlife refueling complex overhaul.
Recurring Cost Element: Private Sector Travel:
* GAO risk-adjusted range - $0.1 million to $1.2 million;
* GAO 65 percent confidence level estimate - $0.6 million;
* Navy point estimate - N/A.
To account for the risk that private sector shipyards may have to
bring in workers from outside of the Mayport area during surges in
workload and thereby incur travel expenses that would be passed on to
the federal government, the GAO estimate calculates potential private
sector travel costs using the public sector travel and per diem rates.
The GAO estimate assumes a 60/40 public/private ratio for maintenance
work in Mayport and sets the most likely private sector travel cost at
zero, with a high-value risk input that assumes 30 percent of the
private sector workforce has to travel from outside of the Mayport
area.
[End of section]
Appendix III: GAO's Assessment of the Navy's Cost Estimating Practices:
Characteristic: Comprehensive;
Overall assessment: Partially met;
Best practice: The cost estimate includes all life-cycle costs;
Individual assessment: Substantially met.
Best practice: The cost estimate completely defines the program,
reflects the current schedule, and is technically reasonable;
Individual assessment: Partially met.
Best practice: The cost estimate work breakdown structure is product-
oriented, traceable to the statement of work/objective, and at an
appropriate level of detail to ensure that cost elements are neither
omitted nor double-counted;
Individual assessment: Minimally met.
Best practice: The estimate documents all cost-influencing ground
rules and assumptions;
Individual assessment: Minimally met.
Characteristic: Well documented;
Overall assessment: Minimally met;
Best practice: The documentation should capture the source data used,
the reliability of the data, and how the data were normalized;
Individual assessment: Minimally met.
Best practice: The documentation describes in sufficient detail the
calculations performed and the estimating methodology used to derive
each element's cost;
Individual assessment: Minimally met.
Best practice: The documentation describes step by step how the
estimate was developed so that a cost analyst unfamiliar with the
program could understand what was done and replicate it;
Individual assessment: Minimally met.
Best practice: The documentation discusses the technical baseline
description and the data in the baseline are consistent with the
estimate;
Individual assessment: Partially met.
Best practice: The documentation provides evidence that the cost
estimate was reviewed and accepted by management;
Individual assessment: Partially met.
Characteristic: Accurate;
Overall assessment: Minimally met;
Best practice: The cost estimate results are unbiased, not overly
conservative or optimistic, and based on an assessment of most likely
costs;
Individual assessment: Not met.
Best practice: The estimate has been adjusted properly for inflation;
Individual assessment: Minimally met.
Best practice: The estimate contains few, if any, minor mistakes;
Individual assessment: Minimally met.
Best practice: The cost estimate is regularly updated to reflect
significant changes in the program so that it is always reflecting
current status;
Individual assessment: Minimally met.
Best practice: Variances between planned and actual costs are
documented, explained, and reviewed;
Individual assessment: Not met.
Best practice: The estimate is based on a historical record of cost
estimating and actual experiences from other comparable programs;
Individual assessment: Substantially met.
Characteristic: Credible;
Overall assessment: Not met;
Best practice: The cost estimate includes a sensitivity analysis that
identifies a range of possible costs based on varying major
assumptions, parameters, and data inputs;
Individual assessment: Not met.
Best practice: A risk and uncertainty analysis was conducted that
quantified the imperfectly understood risks and identified the effects
of changing key cost driver assumptions and factors;
Individual assessment: Minimally met.
Best practice: Cost drivers were cross-checked to see whether results
were similar;
Individual assessment: Not met.
Best practice: An independent cost estimate was conducted by a group
outside the acquiring organization to determine whether other
estimating methods produce similar results;
Individual assessment: Not met.
Source: GAO analysis of Navy data. GAO Cost Estimating and Assessment
Guide (GAO-09-3SP).
Note: Not met means the Navy provided no evidence that satisfies any
of the criterion; minimally met means the Navy provided evidence that
satisfies a small portion of the criterion; partially met means the
Navy provided evidence that satisfies about half of the criterion;
substantially met means the Navy provided evidence that satisfies a
large portion of the criterion; and met means the Navy provided
complete evidence that satisfies the entire criterion.
[End of table]
[End of section]
Appendix IV: Comments from the Department of Defense:
Department Of The Navy:
Office Of The Assistant Secretary:
(Energy, Installations And Environment)
1000 Navy Pentagon:
Washington, DC 20350-1000:
February 23, 2011:
Mark Wielgoszynski:
Assistant Director, Defense Capabilities and Management Team:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Wielgoszynski:
This is the Department of Defense response to the GAO draft report,
GAO-11-309, "Defense Infrastructure: Navy Can Improve the Quality of
Its Cost Estimate to Homeport an Aircraft Carrier at Naval Station
Mayport" dated January 21, 2011 (GAO Code 351521). Comments on the
report and its recommendations are enclosed.
We appreciate the opportunity to provide comments on your draft report.
Sincerely,
Signed by:
Roger M. Natsuhara:
Principal Deputy:
Enclosure:
Copy to: DoD Inspector General:
[End of letter]
GAO Draft Report Dated January 21, 2011:
GA0-11-309 (GAO Code 351521):
"Defense Infrastructure: Navy Can Improve The Quality Of Its Cost
Estimate To Homeport An Aircraft Carrier At Naval Station Mayport"
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Secretary of the Navy to take the following action: To
improve the comprehensiveness of its cost estimate, the Navy should
include all potential recurring cost, and clearly describe the ground
rules and assumptions underlying the estimation of each cost element.
DoD Response: Partially concur. While the elements cited by GAO may
contribute to improved quality in cost estimates, they may not be
ideally suited to the myriad situations in which the Navy is required,
or may be requested, to provide cost estimates to Congress, notably
formal budget submissions which require the use of prescribed exhibits
and where the costs will be spread across multiple appropriations and,
therefore, included in multiple budget justification materials.
Furthermore, Navy cost estimates used in reports to Congress typically
reflect program estimates resulting from formal, disciplined
programming/budgeting process. Additional Secretary of Defense
direction in this regard is not required unless it applies to all
Components of the Department of Defense (DoD). Details on the Navy's
process for estimating construction costs are provided in the below
technical comments. Additionally, it should be noted that, for the one
project which has been awarded (dredging), GAO's initial estimate was
comparable to the Navy's. The fact that GAO's initial dredging
estimate was essentially the same as the Navy's budget estimate
suggests that refinement of Navy estimates, as they progress through the
programming/budgeting process, leads to high quality, credible
estimates.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Secretary of the Navy to take the following action: To
improve the quality and transparency of the Navy's estimate, the Navy
should document the life cycle cost. Specifically documentation
should: identify the source data used, its reliability, and how the
data was normalized; describe the steps used in developing the overall
estimate so that it can be clearly understood and easily replicated;
an describe in sufficient detail the estimating methodology and
calculations performed to derive each element's cost.
DoD Response: Partially concur. While the elements cited by GAO may
contribute to improved quality in cost estimates, they may not be
ideally suited to the myriad situations in which the Navy is required,
or may be requested, to provide cost estimates to Congress, notably
formal budget submissions which require the use of prescribed exhibits
and where the costs will be spread across multiple appropriations and,
therefore, included in multiple budget justification materials.
Additional Secretary of Defense direction in this regard is not
required unless it applies to all DoD Components. Additionally, it
should be noted that, for the one project which has been awarded
(dredging), GAO's initial estimate was comparable to the Navy's. The
fact that GAO's initial dredging estimate was essentially the ante as
the Navy's budget estimate suggests that refinement of Navy estimate,
as they progress through the programming/budgeting process, leads to
high quality, credible estimates.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Secretary of the Navy to take the following action: To
improve the accuracy and credibility of its cost estimate, the Navy
should assign a single office with the responsibility for assembling
the overall estimate into a comprehensive and well-documented package
and for performing a sensitivity and risk and uncertainty analyses on
the overall estimate to identify the major cost drivers, extent to
which estimates could vary due to changes in key cost assumptions, and
level of confidence in the estimate.
DoD Response: Non-concur. Responsibilities for centralized supervision
and coordination of programming and budgeting within the Department of
the Navy are well-established. Navy program/budget estimates and
exhibits are prepared and submitted in accordance with OSD guidance.
Additional Secretary of Defense direction in this regard is not
required unless it applies to all DoD Components. Where such reports
include cost information, it is long-standing Navy practice to report
costs as contained in the official Future Years' Defense Program
(FYDP) where applicable. The FYDP is the culmination of the
aforementioned programming Budgeting process. Additionally, reports to
Congress are typically assigned to a central office for preparation
and coordination (which may include the compilation of inputs from
other offices). Where necessary, instructions to offices providing
inputs can include guidance on baseline assumptions (e.g., then year
versus constant dollars) to ensure consistency.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Brian J. Lepore, Director, (202) 512-4523 or leporeb@gao.gov:
Acknowledgments:
In addition to the contact listed above, key contributors to this
report include Mark J. Wielgoszynski, Assistant Director; Darnita
Akers; Shawn Arbogast; David Brown (Technomics, Inc.); Russell Bryan;
Jennifer Echard; Mary Jo LaCasse; Erik Wilkins-McKee; and Michael
Shaughnessy.
[End of section]
Footnotes:
[1] Naval Station Mayport is located northeast Florida, on the
Atlantic Coast, near Jacksonville. It is roughly 469 nautical miles
south-southwest of Norfolk, Virginia.
[2] In our report examining the military services' processes for
making basing decisions for force structure within the United States,
we provide information about the approach used by the Navy in making
its decision to homeport a nuclear-powered carrier at Mayport,
Florida. See appendix II of the report. GAO, Defense Infrastructure:
Opportunities Exist to Improve the Navy's Basing Decision Process and
DOD Oversight, [hyperlink, http://www.gao.gov/products/GAO-10-482]
(Washington, D.C.: May 11, 2010).
[3] We provided the draft report to DOD and the congressional defense
committees on January 21, 2011, and agreement was made that we issue
the report at a later date.
[4] We define these costs as full life-cycle costs, which are the net
one-time and recurring costs, both direct and indirect, that will be
incurred by the Navy in homeporting a nuclear-powered aircraft carrier
in Mayport.
[5] H.R. Rep. No. 111-491, at 507 (2010). House Report 111-491 also
directed GAO to submit two additional reports on homeporting a nuclear-
powered aircraft carrier in Mayport. See H.R. Rep. No. 111-491, at 254
and 260-261. This includes GAO, Depot Maintenance: Navy Has Revised
Its Estimated Workforce Cost for Basing an Aircraft Carrier at
Mayport, Florida, [hyperlink, http://www.gao.gov/products/GAO-11-257R]
(Washington, D.C.: Mar. 2011) and a report on the ship repair
industrial base in northeast Florida that also will released in March
2011.
[6] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: Mar. 2009).
[7] A point estimate represents the best guess or most likely value
for the cost estimate, given the underlying data. A confidence
interval describes a range of possible costs, based on a specified
probability level. The low and high values of the confidence interval
can be expressed as confidence levels that represent the probability
that the point estimate will be met. Our 80 percent confidence
interval is the range of costs from the 10 percent confidence level to
the 90 percent confidence level. At the 10 percent confidence level,
there is a 10 percent chance that the cost will be at or below the
estimate and a 90 percent chance that the cost will be above the
estimate. Likewise at the 90 percent confidence level, there is a 90
percent chance that the cost will be at or below the estimate and a 10
percent chance that the final cost will be above the estimate. Because
the Navy did not conduct a risk analysis, it is not able to identify
the confidence levels of its estimates.
[8] In accordance with the principles outlined in the GAO Cost
Estimating and Assessment Guide, the 65 percent confidence level is
appropriate for budgeting decisions.
[9] Our analysis is based on the most recent cost estimate
documentation that has been provided to us by the Navy. That
documentation consists of a report from the Chief of Naval Operations
transmitted to the congressional defense committees on December 28,
2010, plus two DOD information papers dated February 25, 2010, and
August 5, 2010. The December 2010 report provides the Navy's estimate
of both one-time and recurring costs, while the February 2010
information paper provides only the one-time costs and the August 2010
information paper outlines just the recurring costs. See appendix I for
more information.
[10] Our cost estimating methodologies primarily included parametric
estimates and scaled analogies. A parametric estimate is a method that
uses an established relationship between cost and one or more input
variables. The input variables become parameters in an equation that
is used to predict cost. Scaled analogy is a cost estimating method
that relies on using actual costs from analogous projects.
[11] Sensitivity analysis examines the effect of changing one
assumption or cost input at a time while holding all other variables
constant. A risk and uncertainty analysis provides a way to assess the
variability in the cost estimate and provides a range of possible
costs along with a level of confidence in achieving the most likely
cost.
[12] Not met means the Navy provided no evidence that satisfies any of
the criterion; minimally met means the Navy provided evidence that
satisfies a small portion of the criterion; partially met means the
Navy provided evidence that satisfies about half of the criterion;
substantially met means the Navy provided evidence that satisfies a
large portion of the criterion; and met means the Navy provided
complete evidence that satisfies the entire criterion.
[13] In addition to the road improvements on Naval Station Mayport,
information provided by a Department of Transportation official
indicates that the Jacksonville Transportation Authority, based on its
study of traffic improvements to the intersection approaching the main
gate outside of the naval station, has identified up to $8 million in
improvements that are unfunded at this time. The official indicated
that these improvements are potentially eligible for federal-aid funds.
[14] Dredging is necessary to make the Mayport basin accessible at all
times for a nuclear-powered aircraft carrier and would have to be done
whether a carrier is homeported at Mayport or not. Although Naval
Station Mayport is not currently capable of homeporting a nuclear
carrier, nuclear carriers do visit for short periods of time but can
only access the base with a light load and only during high tide due
to the depth of the turning basin.
[15] Department of the Navy, Report to Congress on Naval Station
Mayport, Florida Homeporting Alternatives (Washington, D.C.: Dec. 28,
2010). Department of Defense, Information Paper Responding to
Questions from the Government Accountability Office (Washington, D.C.:
Aug. 5, 2010). Department of Defense, Information Paper Responding to
Questions from Congressional Research Service (Washington, D.C.: Feb.
25, 2010).
[16] The Navy compiled and submitted the report in response to
direction from the House Armed Services Committee contained in H.R.
Rep. No. 111-491.
[17] The Saffir-Simpson Hurricane Wind Scale is a 1 to 5
categorization based on the hurricane's intensity at the given time. A
category 4 hurricane has sustained winds of 131 to 155 miles per hour.
[18] This 10 percent factor is based on the best available cost factor
information we could identify. Specifically, we use a published study
analyzing the effect of changes to existing residential structural
vulnerability on hurricane-induced building damage and expected
insurance losses. See Stewart et al., Hurricane Risks and Economic
Viability of Strengthened Construction, Natural Hazards Review
(February 2003).
[19] To facilitate planning for aircraft carrier maintenance, the Navy
has developed a general model of maintenance that spans the 50 years
an aircraft carrier is expected to be in the active inventory. Within
this 50 years are scheduled aircraft carrier maintenance periods known
as availabilities. There are four types of maintenance availabilities:
carrier incremental availability lasting approximately 1 month;
planned incremental availability lasting approximately 6 months;
docking planned incremental availability lasting about 10.5 months;
and a midlife refueling complex overhaul lasting about 39 months.
[20] Technomics, Inc., Arlington, Virginia.
[21] [hyperlink, http://www.gao.gov/products/GAO-09-3SP].
[22] See H.R. Rep. No. 111-491, at 507.
[23] There were some instances in which the Navy did not have any
supporting documentation for its assumptions.
[24] Although the GAO Cost Estimating and Assessment Guide lists step
7 as "develop the point estimate and compare it to an independent
estimate", our objectives led us to develop an independent estimate
for comparison to the Navy's estimate. We therefore list step 7 as
simply "develop the point estimate."
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
