Defense Transportation
Additional Information Is Needed for DOD's Mobility Capabilities and Requirements Study 2016 to Fully Address All of Its Study Objectives Gao ID: GAO-11-82R December 8, 2010The National Military Strategy of the United States calls upon the Armed Forces to retain the ability to rapidly deploy and sustain capabilities to diverse regions, and the Quadrennial Defense Review 2010 acknowledges the fundamental importance of U.S. capability to project power. The National Security Strategy identifies taking stock of capabilities as one of many ways of reducing military risk. To identify the mobility tools needed for force projection, the Department of Defense (DOD) has conducted several studies, including the fifth and most recent--the Mobility Capabilities and Requirements Study 2016 (MCRS-16). DOD issued the report in February 2010. The intent of the MCRS-16 was to provide senior leaders with a detailed understanding of the range of mobility capabilities needed for possible future strategic environments and help them make investment decisions regarding mobility systems. Specifically, the study was to examine, among other things, how changes in the mobility system affect the outcomes of major operations and to assess the associated risks. The MCRS-16 determined that with few exceptions, the projected mobility capabilities in 2016 are sufficient to support the most demanding projected requirements. The MCRS-16 reported on specific mobility issues, including the following ten mobility systems addressed in the unclassified executive summary and depicted in figures 1 and 2: Joint High Speed Vessel, Logistics Support Vessel, Intratheater Airlift, Petroleum Oil Lubricants Vessel, Containerships, Civil Air Reserve Fleet (CRAF) Passenger, CRAF Cargo, Strategic Airlift, Roll-0n/Roll-Off Vessels, and Air Refueling. Because of GAO's work assessing the 2005 Mobility Capabilities Study, we reviewed, at Congress' request, the MCRS-16 to determine the extent to which it provides useful information to decision makers. In response to congressional request, we assessed the extent to which the MCRS-16 report addressed its stated objectives. Within the context of relevant generally accepted research standards, we also examined each of the mobility issues cited above in relation to the study's objectives. While this report's executive summary is unclassified, we considered information included in the classified report of the MCRS-16, and our findings are supported by both the classified and unclassified portions of the report. According to its study plan, the MCRS-16 was to accomplish the following five objectives: (1) determine the Joint Deployment Distribution Enterprise needed to support the National Defense Strategy in the 2016 time frame; (2) identify the capabilities and requirements to deploy, employ, sustain, and retrograde joint forces in support of the National Defense Strategy; (3) determine capability gaps (shortfalls) and overlaps (excesses) associated with the programmed mobility force structure; (4) provide a risk assessment; and (5) provide insights and recommendations to support the Quadrennial Defense Review and decisions regarding future defense programs. To inform DOD's 2010 Quadrennial Defense Review and support decisions regarding future mobility force structure, the MCRS-16 developed three demanding cases of conflicts/natural disasters with multiple scenarios that occur concurrently over a 7-year period and require the use of mobility capabilities. The MCRS-16 used approved DOD planning scenarios to develop the three cases. For example, in one case, U.S. forces might be required to conduct a large land campaign and a long-term irregular warfare campaign, as well as respond to homeland defense missions. In another case, U.S. operations might include two nearly simultaneous conventional campaigns, while also supporting three nearly simultaneous domestic events and other operations.
The Mobility Capabilities and Requirements Study 2016 was to report on five study objectives. We found that two of the five study objectives were clearly addressed. However, we found that the study did not clearly or fully address the three remaining objectives. For example: (1) The MCRS-16 did not clearly address its objective to identify gaps (shortfalls) and overlaps (excesses) concerning any of the ten DOD mobility systems in figure 1 and 2. Concerning shortfalls, the MCRS-16 reported that the C-130 aircrew force level is not adequate to meet demands in at least one scenario. However, the MCRS-16 also found that the current C-130 aircraft fleet exceeds the demand for the three MCRS-16 cases that were used to define the conflicts to be modeled. DOD officials told us that there is no C-130 shortfall. As a result, it is unclear whether there is sufficient C-130 capacity when, as stated in the report, the C-130 crew force structure cannot sustain steady-state operations in combination with a conflict. Concerning excesses, the MCRS-16 suggests that there is unused capacity in Joint High Speed Vessels--a system currently in acquisition--but does not identify this unused capacity as potentially unnecessary excess or needed operational reserve. (2) The MCRS-16 study did not fully address its stated objective to include risk assessment associated with any of the ten mobility systems depicted in figure 1 or 2. DOD officials acknowledged that a risk assessment was not done for these mobility systems, but also stated that risk was considered in the warfight analysis done for the study. However, this warfight analysis risk was briefly described but not discussed in the study report. Concerning the lack of mobility systems risk assessments, the study described, for example, the Offshore Petroleum Discharge System as a critical combat enabler and stated that a single system is insufficient to meet the demands of two overlapping land campaigns. (3) The MCRS-16 did not fully address the objective to provide insights and make recommendations. The study provided some insights and made one general recommendation regarding mobility: that the department should continue to explore strategies that seek to mitigate the adverse impacts of infrastructure constraints by reducing reliance on destination infrastructure wherever possible. However, we identified other instances where explicit recommendations may have been useful. Although DOD's analysis raised questions about the potential for shortfalls and excess capabilities, the report did not make recommendations to address or further study these issues. Generally accepted research standards establish that a quality study follows its study plan, explains and documents deviations from the study plan, addresses study objectives, and presents study results in a clear manner. MCRS-16 study leaders told us that they believe the study report contains the information DOD leaders need to make mobility decisions. We agree that the study contains some useful information and is based on rigorous case studies that test the mobility system. However, we believe that additional information is needed to fully address some study objectives and make DOD's analysis more complete and relevant. Without additional information in some areas, decision makers at DOD and in Congress may not have all relevant information to ensure that defense mobility capabilities and requirements are sized most effectively and efficiently to support U.S. defense strategy. Therefore, we are making four recommendations to the Secretary of Defense: to explicitly identify shortfalls and excesses in mobility found in the MCRS-16, provide a risk assessment for the shortfalls and excesses associated with mobility systems identified in the MCRS-16, recommend mitigation strategies where necessary, and provide these analyses to decision makers in DOD and in Congress.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: William M. Solis Team: Government Accountability Office: Defense Capabilities and Management Phone: (202) 512-8365GAO-11-82R, Defense Transportation: Additional Information Is Needed for DOD's Mobility Capabilities and Requirements Study 2016 to Fully Address All of Its Study Objectives
This is the accessible text file for GAO report number GAO-11-82R
entitled 'Defense Transportation: Additional Information Is Needed for
DOD's Mobility Capabilities and Requirements Study 2016 to Fully
Address All of Its Study Objectives' which was released on December 8,
2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-11-82R:
United States Government Accountability Office:
Washington, DC 20548:
December 8, 2010:
The Honorable Adam Smith:
Chairman:
The Honorable Roscoe Bartlett:
Ranking Member:
Subcommittee on Air and Land Forces:
Committee on Armed Services:
House of Representatives:
Subject: Defense Transportation: Additional Information Is Needed for
DOD's Mobility Capabilities and Requirements Study 2016 to Fully
Address All of Its Study Objectives:
The National Military Strategy of the United States calls upon the
Armed Forces to retain the ability to rapidly deploy and sustain
capabilities to diverse regions, and the Quadrennial Defense Review
2010 acknowledges the fundamental importance of U.S. capability to
project power. The National Security Strategy identifies taking stock
of capabilities as one of many ways of reducing military risk. To
identify the mobility tools needed for force projection, the
Department of Defense (DOD) has conducted several studies, including
the fifth and most recent--the Mobility Capabilities and Requirements
Study 2016 (MCRS-16).[Footnote 1] DOD issued the report in February
2010.
The intent of the MCRS-16 was to provide senior leaders with a
detailed understanding of the range of mobility capabilities needed
for possible future strategic environments and help them make
investment decisions regarding mobility systems. Specifically, the
study was to examine, among other things, how changes in the mobility
system affect the outcomes of major operations and to assess the
associated risks. The MCRS-16 determined that with few exceptions, the
projected mobility capabilities in 2016 are sufficient to support the
most demanding projected requirements. The MCRS-16 reported on
specific mobility issues, including the following ten mobility systems
addressed in the unclassified executive summary and depicted in
figures 1 and 2: Joint High Speed Vessel, Logistics Support Vessel,
Intratheater Airlift, Petroleum Oil Lubricants Vessel, Containerships,
Civil Air Reserve Fleet (CRAF) Passenger, CRAF Cargo, Strategic
Airlift, Roll-0n/Roll-Off Vessels, and Air Refueling.
Because of GAO's work assessing the 2005 Mobility Capabilities Study,
[Footnote 2] we reviewed, at your request, the MCRS-16 to determine
the extent to which it provides useful information to decision makers.
In response to your request, we assessed the extent to which the MCRS-
16 report addressed its stated objectives. Within the context of
relevant generally accepted research standards, we also examined each
of the mobility issues cited above in relation to the study's
objectives. While this report's executive summary is unclassified, we
considered information included in the classified report of the MCRS-
16, and our findings are supported by both the classified and
unclassified portions of the report.
According to its study plan, the MCRS-16 was to accomplish the
following five objectives:
* determine the Joint Deployment Distribution Enterprise[Footnote 3]
needed to support the National Defense Strategy in the 2016 time frame;
* identify the capabilities and requirements to deploy, employ,
sustain, and retrograde joint forces in support of the National
Defense Strategy;
* determine capability gaps (shortfalls)[Footnote 4] and overlaps
(excesses)[Footnote 5] associated with the programmed mobility force
structure;
* provide a risk assessment; and:
* provide insights and recommendations to support the Quadrennial
Defense Review and decisions regarding future defense programs.
To inform DOD's 2010 Quadrennial Defense Review[Footnote 6] and
support decisions regarding future mobility force structure, the MCRS-
16 developed three demanding cases of conflicts/natural disasters with
multiple scenarios that occur concurrently over a 7-year period and
require the use of mobility capabilities. The MCRS-16 used approved
DOD planning scenarios to develop the three cases. For example, in one
case, U.S. forces might be required to conduct a large land campaign
and a long-term irregular warfare campaign, as well as respond to
homeland defense missions. In another case, U.S. operations might
include two nearly simultaneous conventional campaigns, while also
supporting three nearly simultaneous domestic events and other
operations.
We reviewed the unclassified executive summary[Footnote 7] and the
classified report of the MCRS-16, the study's terms of reference, and
study plan. Consistent with our work concerning the 2005 Mobility
Capabilities Study, we also used appropriate, relevant generally
accepted research standards in this review of the MCRS-16 to assess as
many aspects of the MCRS-16 as possible. These standards define a well-
documented and clearly presented study and were accumulated from a
number of research organizations, including DOD, GAO, and private
research centers. The standards are further detailed in enclosure II.
According to generally accepted research standards, in a well-designed
study, the study plan is followed and deviations from the study are
documented and explained. We limited our report to the extent to which
the MCRS-16 met its five study objectives.
We met with the MCRS-16 study leaders to obtain further context and
information concerning the conduct of the study as it was presented in
the report. See enclosure II for a detailed scope and methodology. We
conducted this performance audit from March 2010 to December 2010 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objective. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objective.
Summary:
The Mobility Capabilities and Requirements Study 2016 was to report on
five study objectives. We found that two of the five study objectives
were clearly addressed. However, we found that the study did not
clearly or fully address the three remaining objectives. For example:
* The MCRS-16 did not clearly address its objective to identify gaps
(shortfalls) and overlaps (excesses) concerning any of the ten DOD
mobility systems in figure 1 and 2. Concerning shortfalls, the MCRS-16
reported that the C-130 aircrew force level is not adequate to meet
demands in at least one scenario. However, the MCRS-16 also found that
the current C-130 aircraft fleet exceeds the demand for the three MCRS-
16 cases that were used to define the conflicts to be modeled. DOD
officials told us that there is no C-130 shortfall. As a result, it is
unclear whether there is sufficient C-130 capacity when, as stated in
the report, the C-130 crew force structure cannot sustain steady-state
operations in combination with a conflict. Concerning excesses, the
MCRS-16 suggests that there is unused capacity in Joint High Speed
Vessels--a system currently in acquisition--but does not identify this
unused capacity as potentially unnecessary excess or needed
operational reserve.
* The MCRS-16 study did not fully address its stated objective to
include risk assessment associated with any of the ten mobility
systems depicted in figure 1 or 2. DOD officials acknowledged that a
risk assessment was not done for these mobility systems, but also
stated that risk was considered in the warfight analysis done for the
study. However, this warfight analysis risk was briefly described but
not discussed in the study report. Concerning the lack of mobility
systems risk assessments, the study described, for example, the
Offshore Petroleum Discharge System as a critical combat enabler and
stated that a single system is insufficient to meet the demands of two
overlapping land campaigns. However, the study report did not identify
the risk associated with this lack of capability.
* The MCRS-16 did not fully address the objective to provide insights
and make recommendations. The study provided some insights and made
one general recommendation regarding mobility: that the department
should continue to explore strategies that seek to mitigate the
adverse impacts of infrastructure constraints by reducing reliance on
destination infrastructure wherever possible. However, we identified
other instances where explicit recommendations may have been useful.
Although DOD's analysis raised questions about the potential for
shortfalls and excess capabilities, the report did not make
recommendations to address or further study these issues.
Generally accepted research standards establish that a quality study
follows its study plan, explains and documents deviations from the
study plan, addresses study objectives, and presents study results in
a clear manner. MCRS-16 study leaders told us that they believe the
study report contains the information DOD leaders need to make
mobility decisions. We agree that the study contains some useful
information and is based on rigorous case studies that test the
mobility system. However, we believe that additional information is
needed to fully address some study objectives and make DOD's analysis
more complete and relevant. Without additional information in some
areas, decision makers at DOD and in Congress may not have all
relevant information to ensure that defense mobility capabilities and
requirements are sized most effectively and efficiently to support
U.S. defense strategy. Therefore, we are making four recommendations
to the Secretary of Defense: to explicitly identify shortfalls and
excesses in mobility found in the MCRS-16, provide a risk assessment
for the shortfalls and excesses associated with mobility systems
identified in the MCRS-16, recommend mitigation strategies where
necessary, and provide these analyses to decision makers in DOD and in
Congress.
In written comments on a draft of this report, DOD disagreed with our
four recommendations. In addition, DOD provided technical comments,
which we have incorporated as appropriate. DOD's comments and our
evaluation of them are discussed in detail later in this report.
Enclosure III contains the full text of DOD's letter and technical
comments, and our responses to the technical comments.
Enclosure I of this report addresses 14 mobility issues discussed in
the MCRS-16 Executive Summary.
Background:
The MCRS-16 is the second mobility capabilities study since September
11, 2001, and the first updated assessment since the 2005 Mobility
Capabilities Study.[Footnote 8] In our previous review of the 2005
Mobility Capabilities Study, we suggested that Congress and other
decision makers exercise caution in using the Mobility Capabilities
Study to make programmatic investment decisions. We recommended that
future mobility studies develop models and data for critical missions.
DOD concurred with this recommendation, and in the MCRS-16 DOD
included mobility demands for homeland defense and other missions.
Additionally, we recommended that future studies incorporate both
mobility and warfighting metrics. DOD again concurred, and in the MCRS-
16 DOD employed the million-ton-miles per day metric to measure
strategic airlift requirements. We also recommended that the Secretary
of Defense include an explanation of how ongoing and follow-on studies
and modeling and data limitations can affect findings of any
subsequent studies, but DOD did not agree with this recommendation.
At the direction of the Secretary of Defense, the MCRS-16 analyzed
aspects of the evolving National Military Strategy, including day-to-
day operations, smaller engagements requiring mobility support,
homeland defense missions, and military operations. The cosponsors of
the MCRS-16 study, U.S. Transportation Command (USTRANSCOM) and the
Office of the Secretary of Defense, Cost Assessment and Program
Evaluation, modeled a broad spectrum of military engagements
supporting notional strategic operations. The MCRS-16 used forces
listed in the 2009 President's Budget with appropriate fiscal year
2010 adjustments and compared these capabilities with the requirements
for the 2016 time frame.
In the MCRS-16, officials also considered the increased level of U.S.
military engagements around the world; an increased reliance on
airlift for moving equipment and supplies; the use of new special
equipment, such as Mine Resistant Ambush Protected vehicles; growing
use of special operations forces; establishment of the U.S. Africa
Command; and increased Army and Marine Corps troop levels. The study
was directed in part because of the National Military Strategy
recognition of the reality of long-term U.S. involvement in globally
dispersed operations and the potential for lengthy commitments to
major campaigns.
After the study was completed, DOD published an unclassified executive
summary of what are described as the major insights of the study in
addition to a classified report. DOD provided the unclassified summary
and the classified report to Congress. DOD officials told us that the
chart from the MCRS Executive Summary, "Mobility System Utilization by
MCRS-16 Case", reproduced below as figure 1, informed decision makers
concerning gaps and excesses of mobility systems by summarizing
percentages of mobility capabilities used to meet the demands of each
MCRS-16 case.
Figure 1: Mobility System Utilization by MCRS-16 Case:
[Refer to PDF for image: illustration]
Percentage of capacity used:
Case 1:
Containerships: 32%;
CRAF PAX: 55%;
JHSV: 56%;
CRAF Cargo: 57%;
Intratheater Airlift: 84%;
Strat Airlift: 91%;
Air Refueling: 103%;
POL Tanker: 118%.
Case 2:
Containerships: 24%;
CRAF PAX: 25%;
CRAF Cargo: 55%;
RORO: 63%;
Strat Airlift: 86%;
Air Refueling: 120%.
Case 3:
JHSV: 25%;
CRAF PAX: 28%;
CRAF Cargo: 30%;
LSV: 38%;
Containerships: 41%;
Intratheater Airlift: 67%;
Air Refueling: 81%;
Strat Airlift: 81%;
POL Tanker: 98%;
RORO: 100%.
Source: DOD.
JHSV: Joint High Speed Vessel.
LSV: Logistics Support Vessel.
RORO: Roll on Roll off.
[End of figure]
Assessments in the Mobility Capabilities and Requirements Study 2016
Did Not Clearly and Fully Meet Some Study Objectives:
We found that the assessments in the MCRS-16 did not clearly and fully
meet three of the five study objectives. Assessments of the ten
mobility systems depicted in figures 1 and 2 lacked some information
necessary to meet three study objectives: identifying shortfalls or
excesses, providing a risk assessment, and making recommendations. We
found that some DOD assessments were clearly presented and well
documented. However, assessments of the mobility systems lacked clear
identification of whether shortfalls or excesses are associated with
the programmed mobility force structure. Additionally, none of the
report's assessments of these ten mobility systems included
documentation of risk assessment. Also, the study made one
recommendation, but made none concerning these depicted mobility
systems. We believe that there are other instances where additional
recommendations or further insights could assist decision makers in
their review of DOD's mobility investment plans. See enclosure I for a
more detailed discussion of the mobility issues assessed in the MCRS-
16.
Study Did Not Clearly Identify Shortfalls and Excesses in Mobility
Systems Assessments:
The MCRS-16 did not clearly identify shortfalls and excesses in any
assessments of the ten mobility systems depicted in figures 1 and 2.
The MCRS-16 study plan and Executive Summary stated that identifying
shortfalls and excesses was one of five study objectives. According to
generally accepted research standards, in a well-designed study, the
study plan is followed and deviations from the study are documented
and explained. Achieving this MCRS-16 study plan objective would have
been consistent with generally accepted research standards. However,
rather than explicitly stating whether a shortfall or excess exists,
the MCRS-16 left it to decision makers to determine whether shortfalls
and excess capacity exist in the ten mobility systems. The importance
of knowing whether a shortfall exists is underscored in DOD guidance
pertaining to the requirements process. DOD guidance states that for
those capabilities where a shortfall exists, decision makers can,
among other options, accept the operational risk or seek to address
the shortfall.[Footnote 9]
Shortfalls in capacity are suggested or possible in intratheater
airlift; C-130 crew issues; air refueling tanker inventory; petroleum,
oil, and lubricants tankers; and prepositioned equipment, but not
explicitly identified as such. Similarly, excesses in capacity are
suggested but not explicitly identified in strategic airlift fleet
capacity, CRAF capacity, and Joint High Speed Vessels. As a result, it
is unclear whether unused capacity seen in figure 2 is unnecessary
excess or needed operational reserve.[Footnote 10] By not explicitly
identifying these conditions, the report does not address the
potential for damage to national security or increased costs and, as
previously mentioned, does not identify the underlying risk.
Furthermore, the study does not explain the significance of percentage
of system capacity used depicted in figure 2 below, which is our
analysis and representation of figure 1 above. Specifically, the study
does not indicate whether the used and unused capacity represents
potential gaps, excesses, or necessary operational reserves. In figure
2, where less than 100 per cent of the system was used, there is a
potential acceptable operational reserve or there is a potential
excess that could be reduced. Conversely, when in figure 2 more than
100 per cent of an available system is needed to support operations, a
potential gap is suggested. However, as noted above and despite the
data depicted on the figure, the written report does not identify
whether a shortfall or excess exists.
Figure 2: MCRS-16 Potential Gaps, Excesses, and Operational Reserves
in Mobility Systems:
[Refer to PDF for image: horizontal bar graph]
Percent of capacity used:
100% or less: Possible capacity excess or operational reserve.
Over 100%: Possible capacity gap.
Capability: Joint High Speed Vessel;
Case 1: 56%;
Case 3: 25%.
Capability: Logistics Support Vessel;
Case 1: 63%;
Case 3: 38%.
Capability: Intratheater Airlift;
Case 1: 84%;
Case 3: 67%.
Capability: Petroleum Oil Lubricants Vessel;
Case 1: 118%;
Case 3: 98%.
Capability: Containerships;
Case 1: 32%;
Case 2: 24%;
Case 3: 41%.
Capability: CRAF Passenger;
Case 1: 55%;
Case 2: 25%;
Case 3: 28%.
Capability: CRAF Cargo;
Case 1: 57%;
Case 2: 55%;
Case 3: 36%.
Capability: Strategic Airlift;
Case 1: 91%;
Case 2: 86%;
Case 3: 81%.
Capability: Roll-on/Roll-off;
Case 1: 100%;
Case 2: 63%;
Case 3: 100%.
Capability: Air Refueling;
Case 1: 103%;
Case 2: 120%;
Case 3: 81%.
Source: GAO analysis of DOD data.
Note: All data is derived from DOD data reflected in Figure 1. DOD
provided no Case 2 data for the following systems: Joint High Speed
Vessel, Logistics Support Vessel, Intratheater Airlift, and Petroleum
Oil Lubricant Vessels.
[End of figure]
Study Did Not Clearly Identify Shortfalls in Mobility Systems
Assessments:
We found the report ambiguous in describing whether capability
shortfalls existed concerning any mobility systems. For example, the
report language described the air refueling tanker inventory as "not
satisfying the peak demands" of some national security scenarios,
suggesting a tanker shortfall. (See enclosure I.) Specifically, tanker
demand in one scenario is shown to exceed planned or existing tanker
capacity by 20 percent as seen in figures 1 and 2. Also, the MCRS-16
found a need for as many as 646 air refueling tankers, but the current
inventory is 553 air refueling tankers. Finally, in testimony before
Congress in 2010,[Footnote 11] a DOD official characterized the tanker
situation depicted in the MCRS-16 as insufficient. However, DOD
officials responsible for the report told us that a tanker shortfall
does not exist despite the language and data in the report. DOD
officials explained that no critical mission went unserviced in the 6-
year period that was modeled and that the MCRS-16 did not identify
tankers as a shortfall. Figure 1, taken from the MCRS-16, and figure
2, using data taken from the MCRS-16, suggest that some shortfalls
exist since demand exceeded available capacity in two modeled
scenarios. However, the possible shortfalls suggested by these figures
are not clearly identified as shortfalls in the MCRS-16 report;
instead, the report simply notes that demands were not satisfied.
Ambiguity about whether shortfalls exist means that the study did not
meet its study objective of identifying shortfalls.
In other DOD assessments, it is similarly unclear if a capacity
shortfall exists in any mobility systems. For example, there is
ambiguity concerning a possible C-130 shortfall because the MCRS-16
states that there is sufficient aircraft but also states that the crew
structure cannot sustain steady-state operations in combination with a
conflict. Specifically, the MCRS-16 reported that the current C-130
aircraft fleet exceeds the demand for the three MCRS-16 cases, and
therefore DOD officials told us there is no C-130 shortfall. However,
DOD officials responsible for the MCRS-16 also told us that the crew
force issue was not a shortfall. We believe a shortfall exists if a C-
130 crew force cannot meet national security requirements. While the
study states that there are enough aircraft but not enough aircrews,
it does not identify this as a shortfall. Figures 1 and 2 suggest
unused C-130 capacity or a possible C-130 excess. If decision makers
at DOD and in Congress are to be informed concerning C-130 issues,
additional and clear information is required.
Study Did Not Clearly Identify Excesses in Mobility Systems
Assessments:
The MCRS-16 also did not clearly identify excess capacity in any
mobility system assessments of the mobility force structure. The MCRS
study plan and Executive Summary stated that identifying excesses was
one of the five study objectives. However, the MCRS-16 uses wording to
suggest possible excesses in specific mobility platforms but stops
short of explicitly stating whether an excess exists. Discussing the
impact of excesses on the military, the Secretary of Defense has said
that a dollar spent for capabilities excess to real needs is a dollar
taken from a capability the military does need.
We found the report ambiguous in describing whether there is excess
capability regarding the strategic airlift fleet of C-17 and C-5
aircraft. In response to earlier GAO work,[Footnote 12] DOD said that
the MCRS-16 would set the stage to address the cost-effectiveness of
the strategic airlift mix and recommend C-5 retirements. While the
MCRS-16 determined that the fleet capacity exceeded peak demand by
about 9 percent, the report did not specify whether there were too
many or too few C-17 or C-5 aircraft in the fleet mix. Specifically,
the MCRS-16 did not identify a minimum or maximum number of C-17s in
an optimum fleet or whether an excess exists. The report simply
establishes that some modeled fleet mixes produced more capacity than
was required. Similarly, the MCRS-16 did not identify any excess C-5s
and it did not identify the required number of C-5s, but the Air Force
has announced its intention to retire C-5 aircraft.[Footnote 13]
However, the MCRS-16 did not identify the most combat-effective or the
most cost-effective fleet.
Similarly, the MCRS-16 does not clearly identify whether excess
capacity exists in other important areas. For example, DOD's
assessments are unclear regarding Joint High Speed Vessels. (See
figure 2.) The MCRS-16 report found that 56 percent of Joint High
Speed Vessel capacity was used in one demanding MCRS-16 case and 25
percent in another case. However, the report did not specify whether
the unused Joint High Speed Vessel (44 percent in case 1 and 75
percent in case 2) constitutes excess capability. Moreover, the MCRS-
16 was to consider Joint High Speed Vessels in service through 2016
and only nine of these vessels are reportedly forecast to be in
service by fiscal year 2015. However, total acquisition is reportedly
forecast to include 23 vessels. Decision makers could be informed by
an MCRS-16 assessment of whether the unused Joint High Speed Vessels
capacity constitutes excess capability and whether acquisition of
additional JHSVs is warranted.
A third area where an excess capability is suggested but not confirmed
by the MCRS-16 is in CRAF. The MCRS-16 report found that the most
demanding scenarios used 55 percent of CRAF passenger assets and 57
percent of cargo assets. While CRAF demands increased over previous
usage (cargo tonnage and passenger requirements were up 24 and 15
percent, respectively), the MCRS-16 study uses wording and percentages
to suggest that planned CRAF capability may be in excess of that which
is needed. However, the study did not specify whether unused CRAF
capacity (as much as 43 percent in case 1 and 45 percent in case 2)
constitutes excess capability. Decision makers who review the CRAF
program and seek efficiencies could be informed by an MCRS-16
assessment concerning potential excess capacity.
Study Did Not Provide Any Risk Assessments of Mobility Systems:
The MCRS-16 did not achieve its study objective to provide risk
assessments for the mobility force structure. The report has no
discussion concerning mobility system risk, although risk assessment
is a specific objective in the study plan. None of the 10 DOD mobility
systems shown in figure 1 and 2 included a risk assessment.[Footnote
14] According to the National Defense Strategy, risk assessment is an
essential part of balancing risks, given limited resources, and
requires identifying the potential for damage to national security by
measuring the probability of occurrence and the consequences should
the underlying risk remain unaddressed. Military risk is defined as
DOD's ability to adequately resource and execute military operations
in support of the strategic objectives of the National Military
Strategy. There is operational risk to national security when there is
a shortfall and needs are not met, as well as military risk when
unnecessary excess capability exists. Discussing the impact of
excesses on the military, the Secretary of Defense has said that a
dollar spent for capabilities excess to real needs is a dollar taken
from a capability the military does need. According to generally
accepted research standards, in a well-designed study the study plan
is followed and deviations from the study are documented and
explained. Achieving the MCRS-16 objective on risk assessment would
have been consistent with relevant generally accepted research
standards. Further, the value of risk assessments is underscored in
the National Defense Strategy and the Quadrennial Defense Review,
which the MCRS-16 is intended to support. The strategy and the
Quadrennial Defense Review describe two types of risk relevant to
mobility force structure: operational risk and future risk.[Footnote
15] However, risk concerning mobility systems was not discussed in the
MCRS-16.
DOD and congressional decision makers have acknowledged the importance
of risk assessments, and risk assessment was included in the preceding
mobility study. DOD officials told us that the main metric used in the
MCRS-16 was whether the United States prevailed in a conflict.
However, this metric does not measure the risk associated with
mobility systems. According to DOD testimony, DOD has previously used
mobility metrics in mobility studies and quantified risk. A DOD
official said that DOD's 2005 Mobility Capabilities Study assessed
risk in the intratheater force structure and described a fleet
containing 395 to 674 C-130s as moderate risk. By comparison, DOD's
MCRS-16 reports that a fleet of 401 C-130s exceeds demands but does
not describe the level of risk associated with this size force.
Without a risk assessment, it is unclear whether resources are wasted
on excess capability. DOD officials told us that rather than reporting
risk associated with particular mobility systems, risk was considered
solely on whether warfight objectives were achieved. While warfighting
risk metrics can inform decision making concerning overall mobility
capabilities, we believe that decision makers can benefit from knowing
the risk associated with particular mobility systems as they make
force structure decisions.
Study Did Not Fully Provide Insights and Recommendations:
The MCRS-16 did not fully achieve the study objective to provide
insights and recommendations to support decisions regarding mobility
programs. The MCRS-16 study plan and Executive Summary stated that
providing insights and recommendations was one of five study
objectives. However, in some DOD assessments, the MCRS-16 uses wording
and data to suggest possible shortfalls or excesses in specific
mobility platforms but did make recommendations about necessary
mitigation. By not explicitly addressing mitigation as described in
the National Defense Strategy, the report did not fully inform
decision makers concerning the potential for damage to national
security and, as previously mentioned, did not identify underlying
risk. According to generally accepted research standards, in a well-
designed study, the study plan is followed and deviations from the
study are documented and explained. Achieving this MCRS-16 study plan
objective would have been consistent with generally accepted research
standards and may have informed decision making concerning specific
mobility systems.
The MCRS-16 provided one recommendation and some information
concerning a number of mobility systems, but it could have provided
more detailed insights that would have more fully informed decisions
regarding those systems. As previously discussed, the MCRS-16 reported
that airborne tanker demand exceeded tanker capacity by 20 percent in
one of the scenarios, noting that a modernized tanker fleet would
require fewer aircraft to meet the same demand seen in the combat
scenarios. However, the study does not report that the modernization
would have little impact based on the current production schedule
because no more than 30 modernized aircraft are forecast into the
fleet of 474 tankers in the MCRS-16 time period. Similarly, the MCRS-
16 showed that 44 percent of Joint High Speed Vessels were not used in
the most demanding scenarios, but made no recommendations concerning
the ongoing acquisition of this system. The study described the
offshore petroleum discharge system as a critical combat enabler and
stated that a single system is insufficient to meet the demands of two
overlapping land campaigns. This is a useful insight because, as
stated in the MCRS-16, the ability to prosecute two nearly
simultaneous conventional campaigns remains a cornerstone to U.S.
defense strategy. However, the MCRS-16 makes no recommendation
concerning this mobility program and whether mitigation is required.
Also, it does not identify the number of Offshore Petroleum Discharge
Systems that would be sufficient.
Another example where additional insight could have been provided
concerns CRAF and its relationship to the number and capability of
military cargo aircraft. The study reports that cargo requirements for
military aircraft amounted to 32.7 million-ton-miles per day[Footnote
16] in its highest demand. However, the study does not provide the
insight that usage of CRAF civilian cargo aircraft increased by 4.9
million-ton-miles per day. This insight may be important for decision
makers who are currently considering retiring 22 C-5 aircraft;
however, the study made no recommendation in this area. DOD officials
told us that in some instances the study avoided recommendations to
afford DOD leadership maximum flexibility. We believe that in these
instances and others, more detailed information would support the MCRS-
16 goal of informing programmatic decision making and more fully
satisfy the MCRS-16 objective of providing insights and
recommendations.
Conclusions:
While some of the MCRS-16 reflects DOD's adherence to generally
accepted research standards, it may not fully provide the level and
type of information that would allow DOD and congressional decision
makers to clearly understand what mobility systems are needed to meet
requirements, how many are needed, and what the risks are of having
too many or not enough of each asset to meet the defense strategy.
Additional information is needed to fully address some study
objectives and make DOD's report more complete. Without a clear
picture of the shortfalls and excesses that exist in the mobility
system, the potential risks inherent in those shortfalls and excesses,
and what needs to be done to mitigate those risks, decision makers at
DOD and in Congress may not have all the relevant information they
need to ensure that defense mobility capabilities and requirements are
sized most effectively and efficiently to support U.S. defense
strategy.
Recommendations for Executive Action:
To provide decision makers with information necessary to assess
defense mobility programs, we recommend that the Secretary of Defense
direct the Office of the Secretary of Defense Cost Assessment and
Program Evaluation and U.S. Transportation Command to take the
following four actions:
* Explicitly identify the shortfalls and excesses in the mobility
systems that DOD analyzed for the MCRS-16.
* Provide a risk assessment for the potential shortfalls and excesses.
* Recommend mitigation strategies and any needed changes in force
structure and planned investments resulting from the potential
shortfalls and excesses in the MCRS-16 or explain why mitigation is
not necessary.
* Provide this additional analysis to the Office of the Secretary of
Defense and senior decision makers in DOD and in Congress for their
use in further deliberations on mobility capabilities and requirements.
Agency Comments and Our Evaluation:
In commenting on a draft of this report, DOD did not agree with any of
our four recommendations. DOD also stated that "The draft report
erroneously asserts that the MCRS did not address three of its five
objectives. Specifically, the draft report states that the MCRS failed
to identify gaps and overlaps in the mobility system; did not include
a risk assessment; and did not provide insights and recommendations.
This finding demonstrates a fundamental lack of understanding, and
consequently a misrepresentation of the MCRS, despite the significant
efforts of the study team to inform GAO's assessment." DOD provided
technical comments with examples of where it believed our report
contained misleading information and factual errors. We disagree with
some of the department's technical comments regarding the facts in our
report and have addressed each of the department's comments in
enclosure III. We have revised the report to incorporate DOD's
technical comments, as appropriate. We have also added a new figure
(Figure 2) to the report along with an explanation of the information
depicted in the figure.
DOD did not agree with our recommendation that the Secretary of
Defense direct the Office of the Secretary of Defense Cost Assessment
and Program Evaluation and U.S. Transportation Command to explicitly
identify the shortfalls and excesses in the mobility systems that DOD
analyzed for the MCRS-16. DOD stated that the MCRS explicitly
identifies shortfalls and excesses in the mobility system. In its
response, DOD identified strategic airlift as an example of such an
excess. However as we noted in the report, unused capacity, which the
MCRS did identify, could be considered unnecessary excess capacity or
a necessary operational reserve. As seen in figures 1 and 2, at least
seven mobility systems in the MCRS have unused capacity, but the MCRS
does not report whether the overlap is operationally acceptable or if
alternatively an excess exists. As a result, the MCRS does not provide
insight as to whether or not these are excesses or operational reserves
- although providing this information was one of the objectives of the
MCRS. As an example, only 56 per cent of Joint High Speed Vessels
(JHSVs) are used in the most demanding scenario but the MCRS does not
characterize the remaining 44 per cent of the JHSVs as excess. We
continue to believe that explicitly identifying the shortfalls and
excesses in the mobility systems, an unmet objective of the MCRS, is
useful to decision makers and could potentially result in cost savings
and increased efficiencies.
DOD also did not agree with our second recommendation that the
Secretary of Defense direct the Office of the Secretary of Defense
Cost Assessment and Program Evaluation and U.S. Transportation Command
to provide a risk assessment for the identified potential shortfalls
and excesses. In its response, DOD stated the MCRS includes a risk
assessment which links the ability of the mobility system to support
force closure with a comprehensive set of campaign risk metrics. DOD
further stated that risk was assessed on the ability of US forces to
achieve specific campaign objectives. We believe that decision makers
need to know the risks associated with having too little or too much
of specific mobility systems and completing risk assessment was a
study objective. Despite DOD's statement that the study included
campaign risk assessments, it did not include mobility system risk
assessments for the ten mobility systems (figures 1 and 2) that have
possible excesses and shortfalls. For example, the MCRS reports that
in two of the three cases modeled, the sealift fleet had no
appreciable reserve and the result was some operational delays.
However, the MCRS makes no mention of the risk represented by these
delays since, according to the MCRS study leaders, U.S. objectives
were met in the modeled scenarios. In another example, the MCRS
reports that a single Offshore Petroleum Discharge System is
insufficient to meet the demands of two overlapping land campaigns but
it does not quantify the risk involved with this seeming shortfall. We
believe that this is important, not only because identifying risk is a
study objective, but because, as the MCRS states, the ability to
prosecute two nearly simultaneous conventional campaigns remains a
cornerstone of U.S. defense. In a previous mobility study--the
Mobility Capabilities Study 2005--DOD reported the risks associated
with the mobility assets studied. We continue to believe that
quantifying the risk associated with specific mobility systems could
help to inform decision makers who must allocate scare resources,
enabling them to address the most risk at the least cost.
DOD did not agree with our third recommendation that the Secretary of
Defense direct the Office of the Secretary of Defense Cost Assessment
and Program Evaluation and U.S. Transportation Command recommend
mitigation strategies and any needed changes in force structure and
planned investments resulting from the identified potential shortfalls
and excesses in the MCRS-16 or explain why mitigation is not
necessary. In its response, DOD stated that determining changes to
force structure, planned investments, and mitigation strategies is
beyond the scope of the MCRS. However, the MCRS explicitly states that
the goal of the study was to identify risks associated with
limitations of the mobility system and to provide insights concerning
ways to mitigate those limitations. In one instance, the MCRS found
that C-130 aircraft inventory did not satisfy the peak demands in one
of the three cases assessed because of crew issues, but made no
recommendation and provided no mitigation. As stated in our report, we
continue to believe that this study should inform decision makers
concerning the mobility systems listed in figures 1 and 2.
Specifically, how many are needed, and what the risks are of having
too many or not enough. We believe that the lack of recommendations
and suggested mitigations reflect a missed opportunity to provide DOD
officials with important information concerning mobility systems. We
believe that a study of this magnitude looking to year 2016 should
result in concrete observations and recommendations especially when
doing so is consistent with published objectives.
DOD did not agree with our fourth recommendation that the Secretary of
Defense direct the Office of the Secretary of Defense Cost Assessment
and Program Evaluation and U.S. Transportation Command to provide this
additional analysis to the Office of the Secretary of Defense and
senior decision makers in DOD and in Congress for their use in further
deliberations on mobility capabilities and requirements. In its
response, DOD stated that the MCRS met the needs of the Department's
senior leaders. We believe that the needs of decision makers within
DOD and in Congress were captured in the MCRS objectives of
identifying gaps and excesses, quantifying risks, and providing
insights and recommendations--none of which were fully met.
Accomplishing these objectives ensures that DOD can satisfy the
demands of the National Military Strategy and possibly increase
efficiencies and cost savings.
We are sending copies of this report to the Secretary of Defense and
the Commander, U.S. Transportation Command. This report also is
available at no charge on GAO's Web site at [hyperlink,
http://www.gao.gov].
If you or your staffs have any questions regarding this report, please
contact me at (202) 512-8365 or solisw@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions
to this report include Ann Borseth, Assistant Director; Jenny Hwang;
Ron La Due Lake; Greg Marchand; Charles Perdue; Richard Powelson;
Michael Silver; and Steve Woods.
Signed by:
William M. Solis:
Director:
Defense Capabilities and Management:
Enclosures - 3:
[End of section]
Related GAO Products:
Quadrennial Defense Review: 2010 Report Addressed Many but Not All
Required Items. [hyperlink, http://www.gao.gov/products/GAO-10-575R].
Washington, D.C.: April 30, 2010.
Defense Acquisitions: Strategic Airlift Gap Has Been Addressed, but
Tactical Airlift Plans Are Evolving as Key Issues Have Not Been
Resolved. [hyperlink, http://www.gao.gov/products/GAO-10-67.
Washington, D.C.: November 12, 2009.
Defense Logistics: Department of Defense's Annual Report on the Status
of Prepositioned Materiel and Equipment Can Be Further Enhanced to
Better Inform Congress. [hyperlink,
http://www.gao.gov/products/GAO-10-172R]. Washington, D.C.: November
4, 2009.
Military Airlift: DOD Should Take Steps to Strengthen Management of
the Civil Reserve Air Fleet Program. [hyperlink,
http://www.gao.gov/products/GAO-09-625]. Washington, D.C.: September
30, 2009.
Defense Acquisitions: Timely and Accurate Estimates of Costs and
Requirements Are Needed to Define Optimal Future Strategic Airlift
Mix. [hyperlink, http://www.gao.gov/products/GAO-09-50]. Washington,
D.C.: November 21, 2008.
Defense Transportation: Study Limitations Raise Questions about the
Adequacy and Completeness of the Mobility Capabilities Study and
Report. [hyperlink, http://www.gao.gov/products/GAO-06-938].
Washington, D.C.: September 20, 2006.
[End of section]
Enclosure I: Capabilities:
MCRS-16 Report: Strategic Airlift Fleet Capacity:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Million-ton-miles per day (MTMs/D): A measure of aircraft performance
that reflects how much cargo can be delivered over a given distance in
a given period of time.
Strategic airlift: The C-5 Galaxy and the C-17 Globemaster aircraft
form the core of the Department of Defense's (DOD) capabilities to
provide worldwide reach for both military and humanitarian operations.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 reported that the capacity of the department's strategic
airlift fleet exceeds its highest demand of 32.7 MTMs/D, the number of
MTMs/D that DOD determined is required to meet national security
needs. According to the MCRS-16 report, DOD's strategic airlift fleet
capacity exceeds the peak demand in all three MCRS-16 cases.
GAO's Assessment of This Item:
The MCRS-16 assessment of strategic airlift fleet capacity does not
fully address its study objectives to identify overlaps (excesses) and
provide a risk assessment. The MCRS-16 assessment is not clearly
presented or well documented, both of which are elements of generally
accepted research standards.
The strategic air fleet of 223 C-17 and 111 C-5 aircraft provides a
capability (35.9 MTMs/D) that exceeds requirements (32.7 MTMs/D) by
more than 9 percent. While the MCRS-16 determined that the fleet
capability exceeds peak demand, the report does not specify whether
there are too many C-17s or too many C-5s. DOD officials told us that
the study was not intended to determine the most cost-effective or
combat-effective fleet.
In 2008,[Footnote 17] we recommended that DOD identify the number of C-
17 aircraft and the number of C-5 aircraft needed to accomplish the
strategic airlift mission. At that time, DOD stated that the MCRS-16
would set the stage to address cost-effectiveness issues and aircraft
retirement issues. In July 2010, a DOD official described the MCRS-16
as a critical tool to be used in making difficult decisions. The MCRS-
16 did not identify any excess C-5s and it did not identify the
required number of C-5s. Similarly, the MCRS-16 did not identify a
minimum or maximum number of C-17s in an optimum fleet or whether an
excess exists”contrary to DOD's congressional testimony. However, DOD
has proposed elimination of 22 C-5As.
Additionally, the MCRS-16 assessment does not include a risk
assessment that quantifies risk associated with the strategic fleet.
According to generally accepted research standards, a well-designed
study includes a study plan that is followed and any deviations from
the study are documented and explained.
MCRS-16 Report: Intratheater Airlift:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Intratheater airlift: This airlift involves air movement and delivery
of personnel and equipment within a military theater.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that since peak demand for strategic airlift
and intratheater airlift are not concurrent, the C-17 aircraft also
can support intratheater missions usually flown by the C-130 aircraft
without adding to the peak demand for C-17s.
GAO's Assessment of This Item:
The MCRS-16 report does not fully address the C-17 aircraft's role in
intratheater missions because the assessment does not fully identify
shortfalls and it does not provide a risk assessment of intratheater
airlift. The MCRS-16 assessment is not clearly presented or well
documented, both of which are elements of generally accepted research
standards.
It is unclear whether the programmed number of C-130 aircraft can
satisfy intratheater demands without the C-17s. In the modeled
scenarios, available C-17s were used to augment intratheater missions.
However, the availability of C-17s used in intratheater missions is
conditionally based on assumptions used in modeling that are not
explicitly stated in the report. For example, a modeling assumption
concerning the amount of time between the start of conflicts may free
C-17s for the intratheater mission and this information could be
useful as decision makers consider the number of C-17s required.
Additionally, the study reported”and Department of Defense officials
confirmed”that the MCRS-16 did not consider the additional
intratheater requirement to provide airlift support directly to the
Army. As of April 2010, 30 to 40 C-130s were dedicated to this mission
in Afghanistan. The MCRS-16 report also does not identify the risk
associated with the modeled C-130 force.
The MCRS-16 assessment does not include a risk assessment that
quantifies risk associated with intratheater lift. According to
generally accepted research standards, a well-designed study includes
a study plan that is followed and any deviations from the study are
documented and explained.
MCRS-16 Report: Homeland Defense:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
(MCRS-16) report address its stated objectives?
Definition of Terms:
National Response Plan: This plan provides mechanisms for expedited
and proactive federal support to ensure that critical life-saving
assistance and incident containment capabilities are in place to
respond quickly and efficiently to catastrophic incidents.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that a combination of 12 Department of
Defense (DOD) aircraft and 36 commercial aircraft is sufficient to
meet the peak airlift requirement in support of homeland defense
scenarios, including disasters and terrorist incidents. The MCRS-16
analysis showed that using additional DOD aircraft was not faster than
using ground transportation, showing that ground transportation was
able to move forces and equipment 10 times faster than air.
GAO's Assessment of This Item:
The MCRS-16 assessment of homeland defense does not fully address its
study objective to provide a risk assessment. According to generally
accepted research standards, a well-designed study includes a study
plan that is followed and any deviations from the study are documented
and explained.
Although the Department of Transportation is the lead agency for
providing transportation in disaster areas under the National Response
Plan, DOD provides support for homeland defense missions. We found
that the report does not explain the risk associated with using 48
aircraft to service its homeland defense scenarios. In the past, DOD
has used a higher number of aircraft in response to actual domestic
crises. In response to Hurricane Katrina, it was reported that federal
officials used 346 helicopters and 68 fixed-wing aircraft to fly more
than 16,000 missions. DOD officials explained that the reduced numbers
of aircraft will satisfy mission requirements to move chemical,
biological, radiological, nuclear, and high-yield explosive response
units; the MCRS16 report did not provide additional information to
support this conclusion.
MCRS-16 Report: Ground Force End Strength:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
End strength: This is the maximum number of personnel each of the
military services is authorized to have on the last day of the fiscal
year (September 30).
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that it does not affect the peak demand for
strategic airlift to add ground force end strength to provide a larger
rotational pool of forces to sustain long-duration stability
operations.
GAO's Assessment of This Item:
The MCRS-16 assessment of ground force end strength addresses the MCRS-
16 study objectives to determine mobility requirements, identify
mobility capabilities, determine shortfalls/excesses, and provide
insights to support the Quadrennial Defense Review and decisions
regarding future defense programs. However, the MCRS-16 assessment
does not fully address its study objective to provide a risk
assessment. According to generally accepted research standards, a well-
designed study includes a study plan that is followed and any
deviations from the study are documented and explained.
The Department of Defense's assessment of end strength is clearly
presented, but risk is not addressed. The case study notes that in
general, increased end strength has an impact on force rotations and
tour lengths, but little or no effect on the number of forces
initially committed in operational plans.
MCRS-16 Report: Civil Reserve Air Fleet Capacity:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Civil Reserve Air Fleet (CRAF): In the CRAF program, the Department of
Defense (DOD) contracts for services of specific commercial aircraft
during national emergencies when the need for airlift exceeds the
capability of military aircraft.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD Reported:
The MCRS-16 report found that projected CRAF cargo capacity is
significant and greatly exceeds the requirements for all study cases.
The MCRS-16 study also reported the number of CRAF aircraft used in
support of the cases. DOD relied on CRAF as the primary means of
delivering passengers and bulk air cargo.
GAO's Assessment of This Item:
The MCRS-16 assessment of CRAF does not fully address its study
objectives to identify excesses and provide a risk assessment.
According to generally accepted research standards, a well-designed
study includes a study plan that is followed and any deviations from
the study are documented and explained.
In the MCRS-16 Executive Summary, figure 1 shows that 57 percent of
CRAF cargo capacity and 55 percent of CRAF passenger capacity was used
in the most demanding case for CRAF. However, the report does not
indicate whether the unused capacity (43 percent for cargo and 45
percent for passenger) constitutes excess capability.
We also found that the MCRS-16 increased reliance on CRAF from
previous studies, but did not directly report this increase or fully
assess the risk associated with more reliance on the availability of
commercial aircraft and a decreased use of military aircraft. In the
MCRS-16 report, CRAF cargo usage increased by 24 percent from the
capacity previously planned for DOD use and CRAF passenger usage
increased by 15 percent. In testimony before Congress, a DOD official
incorrectly stated that planned CRAF usage is roughly equivalent in
the previous mobility study and in the MCRS-16.
In 2009,[Footnote 18] we reported that DOD had not assessed the level
of risk associated with declining passenger capabilities and we
expressed concerns regarding aspects of CRAF cargo capability. The
MCRS-16 report did not address CRAF passenger capability
vulnerabilities, and the finding cited only the sufficiency of CRAF
cargo capacity.
MCRS-16 Report: C-130 Crew Issues:
GAO's Reporting Objective:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
C-130 aircraft: First produced in 1962, the C-130 aircraft perform the
tactical/intratheater portion of the airlift mission. They are capable
of operating from unimproved fields and can accommodate a wide variety
of oversized cargo.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD Reported:
The MCRS-16 report found that a fleet of 401 C-130 aircraft exceeds
the peak demand in all three analyzed cases, with the highest demand
being 335 aircraft. However, based on current total force planning
objectives, the C-130 crew force structure cannot sustain steady state
operations in combination with a long-duration irregular warfare
campaign.
GAO's Assessment of This Item:
The MCRS-16 assessment of the C-130 aircraft crew issue does not fully
address its study objectives to identify shortfalls and provide a risk
assessment. According to generally accepted research standards, a well-
designed study includes a study plan that is followed and any
deviations from the study are documented and explained.
We found the report ambiguous as to whether a capability shortfall
exists concerning C-130s. There is ambiguity concerning a possible C-
130 shortfall because the MCRS-16 states that there are sufficient
aircraft but the C-130 crew force structure cannot sustain steady-
state operations in combination with a conflict. Although the MCRS-16
report found that 401 C-130 aircraft exceeds peak demand (335
aircraft), the current crew force level is not adequate to meet peak
demand needs in one scenario. Department of Defense (DOD) officials
responsible for the MCRS-16 report told us that the crew force issue
was not a shortfall; however, in congressional testimony, a DOD
official stated that under one scenario, even if aircraft were
available, it is the crew force that is not able to sustain operations
over time. We believe that a shortfall exists if a C-130 crew force
cannot meet national security requirements. If decision makers at DOD
and in Congress are to be informed concerning C-130 issues, additional
information is required.
MCRS-16 objectives also required providing a risk assessment, but we
found that the MCRS-16 did not describe the risk associated with a C-
130 force level that cannot meet requirements. The previous mobility
study assessed moderate risk to a C-130 force structure as from 395 to
674 aircraft. The MCRS-16 does not fully explain how the requirement
was reduced to 335 aircraft.
MCRS-16 Report: Air Refueling Tanker Inventory:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Peak demand: This is the highest requirement for a mobility
capability that occurs during the period of military operations.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD Reported:
The MCRS-16 report found that the tanker inventory for 2016 does not
satisfy the peak demands of two of three specified cases (scenarios),
and air refueling demand exceeded existing capability by 20 percent in
one case and 3 percent in another case.
Additionally, the report stated that a modernized fleet would require
fewer aircraft to meet the same demand because the modernized fleet
would provide lower depot down-time and greater capability.
GAO's Assessment of This Item:
The MCRS-16 assessment of tanker inventory does not fully address the
study objectives to identify shortfalls and provide a risk assessment.
The MCRS-16 assessment is not clearly presented or well documented,
both of which are elements of generally accepted research standards.
According to generally accepted research standards, a well-designed
study includes a study plan that is followed and any deviations from
the study are documented and explained.
We found the report ambiguous as to whether a capability shortfall
exists concerning tankers. The report language described tanker
inventory as "not satisfying the peak demands" of national security
scenarios, suggesting a tanker shortfall Specifically, tanker demand
is shown to exceed planned or existing tanker capacity by 20 percent
in one case. Also, the MCRS-16 found a need for as many as 646 air
refueling tankers, but the current inventory is 553 air refueling
tankers. Additionally, in testimony before Congress, a Department of
Defense (DOD) official characterized the tanker situation as a
shortfall However, DOD officials responsible for the report told us
that a tanker shortfall does not exist despite the language used in
the report.
MCRS-16 objectives also required providing a risk assessment, but we
found”and DOD officials confirmed”that a risk assessment for tanker
inventory was not included in the report.
In testimony before Congress, a DOD official stated that the advent of
the new KC-X tanker will help address the air refueling shortfall
Similarly, the MCRS-16 asserts that a modernized tanker fleet would
require fewer aircraft. However, the assertion that a modernized
tanker fleet would address the air refueling shortfall is not
supported by the production schedule, which is forecasted to put no
more than 30 modernized aircraft into the fleet of 474 tankers in the
MCRS time period.
MCRS-16 Report: En Route Infrastructure:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Infrastructure: Infrastructure comprises the supporting facilities
vital to global distribution operations, including ports, roads,
airfields, railroads, and staging areas. U.S. Transportation
Command has noted that global infrastructure is the cornerstone for
globally projecting national security capabilities.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that en route infrastructure is sufficient in
all theaters to support the fuel requirements for deploying and
sustaining forces.
GAO's Assessment of This Item:
The MCRS-16 assessment of en route infrastructure addresses the MCRS16
study objectives to determine mobility requirements, identify mobility
capabilities, determine shortfalls/excesses, and provide insights to
support the Quadrennial Defense Review and decisions regarding future
defense programs. However, the MCRS-16 assessment does not address the
study objective to provide a risk assessment. According to generally
accepted research standards, a well-designed study includes a study
plan that is followed and any deviations from the study are documented
and explained.
MCRS-16 Report: Roll-on/Roll-off Ships:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Roll-on/roll-off ships: Roll-on/Roll-off ships have varying
capabilities and are designed to carry rolling stock and can be loaded
and unloaded by driving on and driving off.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that despite no appreciable reserves in two
of the three study cases and some operational delays, the available
sealift fleet of organic, commercial, alliance, and effective U.S.-
controlled roll-on/roll-off ships and containerships was sufficient to
meet military objectives.
GAO's Assessment of This Item:
The MCRS-16 assessment of roll-on/roll-off ships does not fully
address the study objective to identify shortfalls and provide a risk
assessment. The MCRS-16 assessment is not clearly presented or well
documented, both of which are elements of generally accepted research
standards. According to generally accepted research standards, a well-
designed study includes a study plan that is followed and any
deviations from the study are documented and explained.
The report found that roll-on/roll-off ship capacity was sufficient.
However, in two MCRS-16 modeled cases, 100 percent of roll-on/roll-off
ship capacity was used and some operational delays occurred. We noted
in the MCRS-16 report”and Department of Defense officials confirmed”
that no appreciable roll-on/roll-off ship reserves exist. We believe
that the absence of reserves could increase risk to operations, but
there was no risk assessment of demand for roll-on/roll-off ships.
MCRS-16 Report: Sealift Reserves:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Petroleum, oil, and lubricants (POL) vessels: Military Sealift Command
(MSC) operates four tankers that move fuel between commercial
refineries and Department of Defense (DOD) storage and distribution
facilities worldwide during peacetime and war. More than 90 percent of
U.S. war fighters' equipment and supplies travels by sea and these
four ships transport account for movement of more than 70 percent of
DOD needs. When more fuel-carrying capacity is required, MSC charters
additional tankers directly from the commercial market on a short-term
basis.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 reported that demand slightly exceeded capacity for POL
vessels, but the demand can be mitigated easily by gaining access to
the 1,980 tankers available in the world. In one case, the MCRS-16
assessed that POL vessel demand exceeded capacity by 18 percent.
GAO's Assessment of This Item:
The MCRS-16 assessment of sealift reserves does not fully address the
study objective to identify shortfalls and provide a risk assessment.
The MCRS-16 assessment is not clearly presented or well documented,
both of which are elements of generally accepted research standards.
According to generally accepted research standards, a well-designed
study includes a study plan that is followed and any deviations from
the study are documented and explained.
Although the MCRS-16 report found that in the most demanding case the
requirement for POL vessels exceeded demand by 18 percent, no
shortfall was reported. DOD officials told us that there is no POL
vessel shortfall; however, the conclusion is not clear based on the
report.
Some risk may be associated with DOD plans to requisition U.S.-flagged
vessels. The MCRS-16 points out that the President has the authority
in times of national emergency to requisition certain ships, but does
not detail the circumstances and consequences of requisitioning. The
MCRS16 also does not detail the extent to which U.S. strategies depend
on this requisitioning.
MCRS-16 Report: Prepositioned Equipment:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Prepositioned equipment: This equipment is forward-positioned materiel
for use by combatant commanders to respond to contingencies.
Prepositioned equipment reduces early transportation requirements and
increases global responsiveness. DOD prepositions equipment at
strategic locations around the world to enable it to field combat-
ready forces in days, rather than the weeks it would take if equipment
had to be moved from the United States to the locations of conflicts.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that prepositioned equipment was sufficient
to meet the most demanding MCRS-16 case. Additionally, equipment
currently prepositioned was not immediately used when the warfight
began. The report identified an opportunity for further study to
reevaluate the concept of employment and mix of prepositioned
equipment.
GAO's Assessment of This Item:
The MCRS-16 assessment of prepositioned equipment addresses the study
objectives to determine mobility requirements, identify mobility
capabilities, and provide insights to support the Quadrennial Defense
Review and decisions regarding future defense programs. However, the
MCRS-16 assessment does not fully address the study objectives to
identify shortfalls and provide a risk assessment. The MCRS-16
assessment is not clearly presented or well documented, both of which
are elements of generally accepted research standards. According to
generally accepted research standards, a well-designed study includes
a study plan that is followed and any deviations from the study are
documented and explained.
Low levels of prepositioned equipment”noted in the MCRS-16 report”
raise concerns because shortages could add stress to the mobility
system, but the risks associated with the shortages are not assessed.
We found the report ambiguous as to whether a capability
shortfall/excess exists concerning prepositioned equipment. While the
MCRS-16 report found that prepositioned equipment was sufficient to
meet the most demanding MCRS-16 case, it also notes that a majority of
certain types of equipment were not used immediately. This suggests
that there may be an excess in certain types of prepositioned
equipment and a shortfall in other equipment.
We agree it would be useful to reevaluate of the current concept of
employment and mix. We have previously reported information concerning
prepositioned equipment and materiel that could further inform
decision makers and we believe that shortfalls in this area remain
important.[Footnote 19]
MCRS-16 Report: Continental U.S. Infrastructure:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
U.S. infrastructure: To expeditiously transport troops and materiel to
ports of embarkation in times of crisis, the nation needs an extensive
transportation capability within its land mass. Railroads,
highways, waterways, and a fleet of railcars, buses, trucks, and
barges are vital components of the overland lift system. U.S.
Transportation Command's Surface Deployment and Distribution Command
monitors the status of the infrastructure system, including ports,
inland waterways, pipelines, and air facilities.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that the current and projected infrastructure
in the continental United States is sufficient to meet the most
demanding study case.
GAO's Assessment of This Item:
The MCRS-16 assessment of continental U.S. infrastructure addresses
the study objectives to determine mobility requirements, identify
mobility capabilities, and provide insights to support the Quadrennial
Defense Review and decisions regarding future defense programs.
However, the MCRS-16 assessment does not address the study objective
to provide a risk assessment. According to generally accepted research
standards, a well-designed study includes a study plan that is
followed and any deviations from the study are documented and
explained.
MCRS-16 Report: Joint High Speed Vessels:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Joint high speed vessel (JHSV): A JHSV is a U.S. Army or Navy surface
ship capable of transporting troops, equipment, and supplies while
operating in shallow waters and reaching speeds in excess of 35 knots
when fully loaded.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that JHSVs are critical enablers of
deployment and sustainment and are sufficient to support the most
demanding study case. In the most demanding case, the Department of
Defense used 56 percent of available capacity.
GAO's Assessment of This Item:
The MCRS-16 assessment of JHSVs does not fully address its study
objectives to identify an excess and provide a risk assessment. The
MCRS16 assessment is not clearly presented or well documented, both of
which are elements of generally accepted research standards. According
to generally accepted research standards, a well-designed study
includes a study plan that is followed and any deviations from the
study are documented and explained.
In the MCRS-16 Executive Summary, figure 1 shows that 56 percent of
JHSV capacity was used in the most demanding study case and 25 percent
in another case. However, the report does not indicate whether the
unused capacity (44 percent in case 1 and 75 percent in case 2)
constitutes excess capability. Additionally, the MCRS-16 does not
include a risk assessment of the JHSVs. The MCRS-16 was to consider
JHSVs in service through 2016 and 9 JHSVs are forecasted to be in
service by fiscal year 2015. However, total acquisition is reportedly
forecasted to include 23 vessels, costing an estimated $1.8 million
each.
MCRS-16 Report: Offshore Petroleum Discharge Systems:
GAO's Focus:
To what extent does the Mobility Capabilities and Requirements Study
2016 (MCRS-16) report address its stated objectives?
Definition of Terms:
Offshore petroleum discharge system (OPDS): An OPDS is a ship-borne
system that can provide over 1 million gallons of petroleum to the
shore where ports or terminal facilities are inadequate or
nonexistent. It supports the U.S. Army, Navy, and Marine Corps.
For more information, contact Bill Solis at (202) 512-8365 or
solisw@gao.gov.
DOD's Assessment:
The MCRS-16 report found that a single OPDS is insufficient to meet
the demands of two overlapping land campaigns.
GAO's Assessment of This Item:
The MCRS-16 assessment of OPDS capacity does not address the study
objective to provide a risk assessment. According to generally
accepted research standards, a well-designed study includes a study
plan that is followed and any deviations from the study are documented
and explained. The report does not clearly describe the risk
associated with one OPDS being insufficient to meet the demands of two
overlapping land campaigns. Department of Defense officials confirmed
that there is no risk assessment in the study about OPDS capacity.
Also, the MCRS-16 does not identify or recommend how many systems
would be sufficient.
[End of Enclosure I]
Enclosure II: Scope and Methodology:
To conduct our review of the 2016 Mobility Capabilities and
Requirements Study 2016 (MCRS-16), we reviewed and analyzed the final
MCRS-16 report, the MCRS-16 Terms of Reference, and the MCRS-16 Study
Plan. We also reviewed the National Military Strategy of the United
States of America, the National Defense Strategy of the United States
of America, and the National Security Strategy of the United States of
America. We interviewed study officials from the Office of the
Secretary of Defense, Cost Assessment and Program Evaluation, and U.S.
Transportation Command.
We relied on previous GAO work based on research literature and
Department of Defense (DOD) guidance that identified frequently
occurring, generally accepted research standards that are relevant for
defense studies, such as the MCRS-16, and that define a well-
documented and clearly presented study. Generally accepted research
standards establish that a quality study follows its study plan,
explains and documents deviations from the study plan, addresses study
objectives, and presents study results in a clear manner. During the
process of identifying relevant generally accepted research standards,
we noted that not all studies are conducted the same way. For example,
while all studies use data, not all use baseline data. Likewise, all
studies require analyses, but not all use models or simulation to
conduct analyses. For our analysis of the MCRS-16, we reviewed the
unclassified executive summary and the classified report of the MCRS-
16, the study's terms of reference, and study plan. We used
appropriate, relevant generally accepted research standards in this
review of the MCRS-16 to assess as many aspects of the MCRS-16 as
possible. We limited our report to the extent to which the MCRS-16 met
its five study objectives. Specifically, we applied sections I, II,
IV, V, and VIII shown in table 1 below.
Additionally, we reviewed research literature and DOD guidance and
identified frequently occurring, generally accepted research standards
that are characteristic of a quality study and that are relevant for
defense studies such as the MCRS-16. A number of sources were
available, and the following were our sources for these standards:
* GAO, Government Auditing Standards: 2007 Revision, [hyperlink,
http://www.gao.gov/products/GAO-07-731G] (Washington, D.C.: July 2007);
* GAO, Designing Evaluations, [hyperlink,
http://www.gao.gov/products/GAO/PEMD-10.1.4] (Washington, D.C.: May
1991);
* RAND Corporation, RAND Standards for High-Quality Research and
Analysis (Santa Monica, Calif., January 2010);
* U.S. Air Force, Office of Aerospace Studies, Analyst's Handbook: On
Understanding the Nature of Analysis (Kirtland Air Force Base: January
2000);
* U.S. Air Force, Office of Aerospace Studies, Air Force Analysis
Handbook, A Guide for Performing Analysis Studies: For Analysis of
Alternatives or Functional Solution Analysis (Kirtland Air Force Base:
July 2004);
* DOD Instruction 5000.61, DOD Modeling and Simulation (M&S)
Verification, Validation, and Accreditation (VV&A) (December 2009);
* DOD Directive 8260.1, Data Collection, Development, and Management
in Support of Strategic Analysis (January 2007); and:
* DOD Instruction 8260.2, Implementation of Data Collection,
Development, and Management for Strategic Analyses (Jan. 21, 2003).
We applied the research standards in the above documents that we
identified as relevant to the MCRS-16, as shown in table 1.
Table 1: Generally Accepted Research Standards Relevant to MCRS-16
Requirements:
Design: The study is well designed:
I. The study is well designed: Study plan, scope, and objectives
follow formal guidance?
I.a. The study is well designed: Does the study plan address specified
guidance?
I.b. The study is well designed: Was the study plan followed?
I.c. The study is well designed: Were deviations from the study plan
explained and documented?
I.d. The study is well designed: Was the study plan updated over the
course of the study and were the updates explicitly identified in the
study and updated study plan?
II. The study is well designed: Assumptions and constraints are
reasonable and consistent.
II.a. The study is well designed: Are assumptions and constraints
explicitly identified?
II.a.1. The study is well designed: Are the study assumptions
necessary and reasonable?
II.b. The study is well designed: Do the study assumptions support a
sound analysis?
II.c. The study is well designed: Are the assumptions used in analyses
common throughout the study and models?
II.d. The study is well designed: Do the assumptions contribute to an
objective and balanced research effort?
III. The study is well designed: Scenarios and threats are reasonable.
III.a. The study is well designed: Are scenarios traceable back to
formal guidance?
III.b. The study is well designed: Do scenarios represent a reasonably
complete range of conditions?
III.c. The study is well designed: Were the threats varied to allow
for the conduct of sensitivity analysis?
Execution: The study is well executed:
IV. The study is well designed: Methodology is successfully executed.
IV.a. The study is well designed: Was the study methodology executed
consistent with the (MCRS-16) study plan and schedule?
IV.b. The study is well designed: Does the methodology support
accomplishing the objectives presented in the study plan?
IV.c. The study is well designed: Were the models used to support the
analyses adequate for their intended purpose?
IV.d. The study is well designed: Were the model input data properly
generated to support the methodology?
V. The study is well designed: (Analytical) Baseline data and other
data used to support study and analyses validated, verified, and
approved.
V.a. The study is well designed: Is the (analytical) baseline fully
and completely identified and used consistently throughout the study
for the various analyses?
V.b. The study is well designed: Were data limitations identified and
was the impact of the limitations fully explained?
V.c. The study is well designed: Were the baseline data verified and
validated?
V.d. The study is well designed: Was the data verification and
validation process documented?
VI. The study is well designed: Models, simulations, and verification,
validation, and accreditation (VV&A) are reasonable.
VI.a. The study is well designed: Was a VV&A report that addresses the
models and data certification signed by the study director and
included in the report?
VI.b. The study is well designed: Were modeling and simulation
limitations identified and explained?
VI.c. The study is well designed: Has each model in the study been
described?
VI.d. The study is well designed: Are the model processes clearly
explained, documented, and understood?
VII. The study is well designed: Measures of effectiveness (MOE) and
essential elements of analysis (EEA) are addressed.
VII.a. The study is well designed: Do MOEs adhere to the guidance in
the study terms of reference?
VII.b. The study is well designed: Are the MOEs fully addressed in the
study?
VII.c. The study is well designed: Are the EEAs addressed in the study?
Presentation of results: Timely, complete, accurate, concise, and
relevant to the client and stakeholders:
VIII. The study is well designed: Presentation of results supports
findings.
VIII.a. The study is well designed: Does the report address the
objectives?
VIII.b. The study is well designed: Does the report present an
assessment that is well documented and conclusions that are supported
by the analyses?
VIII.c. The study is well designed: Are conclusions sound and complete?
VIII.d. The study is well designed: Are recommendations supported by
analyses?
VIII.e. The study is well designed: Is a realistic range of options
provided?
VIII.f. The study is well designed: Are the study results presented in
the report in a clear manner?
VIII.g. The study is well designed: Are study participants/
stakeholders (i.e., services and combatant commands) informed of the
study results and recommendations?
Source: GAO analysis of industry and DOD study and research standards.
[End of table]
We used these relevant standards as our criteria to assess the
reported MCRS-16 results. All eight key areas of the study process
were considered to have equal importance. However, not all eight areas
could be observed based on the information presented in the MCRS-16
report. Three analysts independently reviewed evidence relevant to
each subquestion in each of the eight areas, including the study
itself, the study Terms of Reference, and the MCRS-16 Study Plan. For
each of the subquestions of the study process, the analysts determined
whether (1) the evidence had no limitations or raised no concerns, (2)
the evidence had some limitations or raised some concerns, (3) the
evidence had significant limitations or raised significant concerns,
or (4) we could not determine the extent of limitations or concerns
because there was not sufficient information. The analysts then met,
compared, and discussed their individual assessments, and reached an
overall assessment for each subquestion. For any of the subquestions
of the study process for which we determined there were either "some"
or "significant" limitations or concerns, we concluded that these
aspects of the study were not consistent with the relevant generally
accepted research standards.
We conducted this performance audit from March 2010 to December 2010
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objective. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objective.
[End of Enclosure II]
Enclosure III:
Office Of The Secretary Of Defense:
Cost Assessment and Program Evaluation:
1800 Defense Pentagon:
Washington, D.C. 20301-1800:
October 28, 2010:
Mr. William Solis:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Solis:
The Department of Defense non-concurs with the GAO draft report, GAO-
1.1-82R, "Defense Transportation: Additional Information is Needed for
DOD's Mobility Capabilities and Requirements Study 2016 {MCRS) to
Fully Address All of its Study Objectives, dated October 4, 2010 (GAO
Code 351477). DoD's responses to the report's recommendations are
enclosed.
The draft report erroneously asserts that the MCRS did not address
three of its five objectives. Specifically, the draft report states
that the MCRS failed to identify gaps and overlaps in the mobility
system; did net include a risk assessment; and did not provide
insights and recommendations. This finding demonstrates a fundamental
lack of understanding, and consequently a misrepresentation of the
MCRS, despite the significant efforts on the part of the study team to
inform GAO's assessment.
The Department stands behind the adequacy, completeness, and utility
of the MCRS. The study did in fact address all five of its objectives,
and its findings and recommendations have been used to inform the
recent Quadrennial Defense Review and senior leader deliberations
undertaken as part of the Planning, Programming, Budgeting and
Execution process. The MCRS study leadership remains available to
assist your team in clarifying and correcting the draft report.
Signed by:
Scott A. Comes:
Deputy Director:
Program Evaluation:
Enclosures: As stated:
[End of letter]
GAO Draft Report Dated October 4, 2010:
GAO-11-82R (GAO CODE 351477):
"Defense Transportation: Additional Information Is Needed For DOD's
Mobility Capabilities And Requirements Study 2016 To Fully Address All
Of Its Study Objectives"
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Office of the Secretary of Defense Cost Assessment and
Program Evaluation and U.S. Transportation Command to explicitly
identify the shortfalls and excesses in the mobility systems that DoD
analyzed for the Mobility Capabilities and Requirements Study 2016
(MCRS-16).
DoD Response: DoD nonconcurs. The MCRS explicitly identifies
shortfalls and excesses in the mobility system. For example, the MCRS
states that the Departments strategic airlift fleet exceeds the peak
demand in each of the MCRS cases." The MCRS goes on to quantify the
excess as 3.2 million ton-miles per day (MTM/D) in the most demanding
MCRS case. Despite this explicit language in the MCRS report, the GAO
states that "we found the report ambiguous in describing whether there
is excess capability regarding the strategic airlift fleet of C-17 and
C-5 aircraft." The MCRS findings are clear and were used by the Air
Force to develop the FY11 President's Budget, recommending retirement
of 22 C-5A aircraft.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Office of the Secretary of Defense Cost Assessment and
Program Evaluation and U.S. Transportation Command to provide a risk
assessment for the identified potential shortfalls and excesses.
DoD Response: DoD nonconcurs. The GAO report states, "DoD officials
told us the main metric used in the MCRS-16 was whether the U.S.
prevailed in a conflict. However, this metric does not measure the
risk associated with mobility systems." The draft report is incorrect.
The MCRS includes a risk assessment which links the ability of the
mobility system to support force closure with a comprehensive set of
campaign risk metrics. Risk was assessed based on the ability of U.S.
forces to achieve specific campaign objectives. Achieving campaign
objectives is directly related to the performance of the mobility
system, because campaign objectives cannot be met if the mobility
system cannot deliver required forces in accordance with the required
timelines.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Office of the Secretary of Defense Cost Assessment and
Program Evaluation and U.S Transportation Command to recommend
mitigation strategies and any needed changes in force structure and
planned investments resulting from the identified potential shortfalls
and excesses in the MCRS-16 or explain why mitigation is not necessary.
DoD Response: DoD nonconcurs. Determining changes to force structure,
planned investments, and mitigation strategies is beyond the scope of
the MCRS. The study provides a range of capability requirements based
on its assessment of different potential demands. The Department will
continue to use studies like the MCRS to inform decisions related to
force structure and planned investments.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the Office of the Secretary of Defense Cost Assessment and
Program Evaluation and U.S. Transportation Command to provide this
additional analysis to the Office of the Secretary of Defense and
senior decision makers in DoD and in Congress for their use in further
deliberations on mobility capabilities and requirements.
DoD Response: DoD nonconcurs. The MCRS met the needs of the
Department's senior leaders by providing insights needed to assess
future capability requirements. The Department will continue to use
studies like the MCRS to inform decisions made as part of the
Planning, Programming, Budgeting and Execution process.
GAO DRAFT REPORT DATED OCTOBER 4, 2010:
GAO-11-82R (GAO CODE 351477):
"Defense Transportation: Additional Information Is Needed For DOD's
Mobility Capabilities And Requirements Study 2016 To Fully Address All
Of Its Study Objectives"
Department Of Defense Technical Comments:
1. Page 4, 1st Summary bullet: "However, it is unclear whether there
is sufficient C-130 capacity if there are insufficient crews."
Response: The MCRS clearly states that there is sufficient C-130
aircraft capacity-. The MCRS also states "based on current total force
planning objectives, the C-130 crew force structure cannot sustain
steady state operations in combination with a long duration irregular
warfare campaign." This does not mean there are "insufficient crews,"
rather it means that there are not enough crews available given
current guidance on dwell to deploy ratios. The MCRS recommended
further study in the area of crew ratio and active/reserve crew mix
for C-130s to address this finding. There is no discussion in the MCRS
report of an insufficient number of C-130 crews. [This response also
applies to page 9, 2nd paragraph and MCRS Report page 6, "C-130 Crew
Issues."] [See comment 1]
2. Page 6, 2nd Background paragraph: "The MCRS-16 used forces listed
in the 2009 President's Budget with appropriate Fiscal Year 2010
adjustments and compared these capabilities with the requirements for
the 2010-2016 timeframe."
Response: Incorrect. The MCRS focused on the 2016 timeframe, not the
2010-2016. [See comment 2]
3. Page 14, 1st paragraph: "However, the study does not provide the
insight that retirement of aircraft and lowered military (cargo)
aircraft demand is made possible by an increase usage of CRAF civilian
cargo aircraft of 4.9 million-ton-miles per day."
Response: False. The cause and effect relationship drawn by GAO
between CRAF and military aircraft is illogical given the Department's
policy to move bulk cargo and passengers by CRAF when possible because
of its cost effectiveness. Increased use of CRAF over previous studies
was driven by an increased demand for bulk cargo and passenger
movements. The MCRS noted that "CRAF cargo capacity is significant,
and greatly exceeds the requirements for all MCRS cases." The
requirement for military inter-theater aircraft is driven by the
requirement to move over and outsized equipment, which is ill-suited
for CRAF. The MCRS noted "the Department's strategic airlift fleet
exceeds the peak demand in each of the MCRS cases." [See comment 3]
4. MCRS Report Page 2, Intra-theater Airlift: "However, the
availability of C-17s used in intra-theater missions is conditionally
based on assumptions not explicitly stated in the report."
Response: False. The availability of C-17s to be used in intra-theater
missions was not based on assumptions. it was based on detailed
mobility modeling. C-17s not required for inter-theater airlift
missions were made available for intra-theater missions. [See comment
4]
5. MCRS Report Page 7, Aerial Refueling Inventory: MCRS-16 objectives
required providing a risk assessment, but we found-”and DOD officials
confirmed-”that a risk assessment for tanker inventory was not
included in the report."
Response: False. Don officials involved in the GAO investigation do
not recall confirming this incorrect assertion. The MCRS included a
risk assessment for aerial refueling and other mobility areas. [See
comment 5]
Page 9. Definition of Terms, Roll-on/Roll-off ships.
Response: This definition appears to allude to Large Medium Speed Roll-
on/roll-off ships (LMSRs) and is inaccurate. Was are a subset of the
Department's roll-on/roll-off ships (ROROs). ROROs have varying
capabilities and are designed to carry rolling stock and can be loaded
and unloaded by driving on and driving off. [See comment 6]
GAO's Responses To DOD's Technical Comments:
1. Page 4, 1st Summary bullet: "However, it is unclear whether there
is sufficient C-130 capacity if there are insufficient crews."
DOD Response: The MCRS clearly states that there is sufficient C-130
aircraft capacity. The MCRS also states "based on current total force
planning objectives, the C-130 crew force structure cannot sustain
steady state operations in combination with a long duration irregular
warfare campaign." This does not mean there are "insufficient crews,"
rather it means that there are not enough crews available given
current guidance on dwell to deploy ratios. The MCRS recommended
further study in the area of crew ratio and active/reserve crew mix
for C-130s to address this finding. There is no discussion in the MCRS
report of an insufficient number of C-130 crews. [This response also
applies to page 9, 2ND paragraph and MCRS Report page 6, "C-130 Crew
Issues."]
GAO Comment 1: DOD's technical comment makes a distinction between
"insufficient crew" numbers and "a situation where C-130 crew force
structure that cannot sustain steady state operations in combination
with a long duration irregular warfare campaign." We found that the
MCRS report has, in the same paragraph, seemingly conflicting
statements: (1) The programmed fleet of 401 C-130s exceeds the peak
demand in each of the three MCRS cases but added that (2) the C-130
crew force structure cannot sustain steady state operations in
combination with a long duration irregular warfare campaign. We find
that it is difficult to understand how the C-130 force can exceed
demands without C-130 crews, except to acknowledge that there are
enough aircraft but there are insufficient aircrews. DOD points out
that this could be mitigated by a policy change, but the MCRS does not
recommend such a change concerning crew ratios between the active and
reserves. We conclude that a potential problem exists and that,
contrary to DOD's assertion, there is not sufficient C-130 aircraft
capacity without required C-130 aircrews.
To address DOD's technical comment, we modified our report to read "it
is unclear whether there is sufficient C-130 capacity when, as stated
in the report, the C-130 crew force structure cannot sustain steady-
state operations in combination with a conflict."
2. Page 6, 2nd Background paragraph: "The MCRS-16 used forces listed
in the 2009 President's Budget with appropriate Fiscal Year 2010
adjustments and compared these capabilities with the requirements for
the 2010-2016 timeframe."
DOD Response: Incorrect. The MCRS focused on the 2016 timeframe, not
the 2010-2016.
GAO Comment 2: To address DOD's technical comment, we modified our
report to reflect the correct timeframe.
3. Page 14, 1nd paragraph: "However, the study does not provide the
insight that retirement of aircraft and lowered military (cargo)
aircraft demand is made possible by an increase usage of CRAF civilian
cargo aircraft of 4.9 million-ton-miles per day."
DOD Response: False. The cause and effect relationship drawn by GAO
between CRAF and military aircraft is illogical given the Department's
policy to move bulk cargo and passengers by CRAF when possible because
of its cost effectiveness. Increased use of CRAF over previous studies
was driven by an increased demand for bulk cargo and passenger
movements. The MCRS noted that "CRAF cargo capacity is significant,
and greatly exceeds the requirements for all MCRS cases." The
requirement for military inter-theater aircraft is driven by the
requirement to move over and outsized equipment, which is ill-suited
for CRAF. The MCRS noted "the Department's strategic airlift fleet
exceeds the peak demand in each of the MCRS cases."
GAO Comment 3: We believe that there is a relationship between the
number of organic aircraft and the amount of CRAF needed. However,
this was not the point of the GAO statement cited above. Our point is
that the report does not provide the following insight: While the MCRS
found that demand for organic aircraft is such that organic aircraft
can be retired, the need for CRAF was substantially increased. In
DOD's technical comments, the increased demand for CRAF is not
disputed and DOD does not assert that this insight was provided.
To address DOD's technical comment, we modified our report to read
"However, the study does not provide the insight that usage of CRAF
civilian cargo aircraft increased by 4.9 million-ton-miles per day."
4. MCRS Report Page 2, Intra-theater Airlift: "However, the
availability of C-17s used in intra-theater missions is conditionally
based on assumptions not explicitly stated in the report."
DOD Response: False. The availability of C-17s to be used in intra-
theater missions was not based on assumptions, it was based on
detailed mobility modeling. C-17s not required for inter-theater
airlift missions were made available for intra-theater missions.
GAO Comment 4: We believe that DOD's technical comment is incomplete.
It correctly states that availability of C-17 aircraft for the
intratheater mission is based on mobility modeling, but fails to note
that this modeling is premised on critical assumptions about
separation times between the start of overlapping conflicts. These
separation times, which, according to the study leaders, were used in
modeling, are the reason that C-17s available for the intra-theater
mission. We believe that decision makers could be better informed if
the assumptions about separation times and their role in the MCRS were
explicitly stated in the report.
To address DOD's technical comment, we modified our report to read
"However, the availability of C-17s used in intratheater missions is
conditionally based on assumptions used in modeling that are not
explicitly stated in the report. For example, a modeling assumption
concerning the amount of time between the start of conflicts may free
C-17s for the intratheater mission and this information could be
useful as decision makers consider the number of C-17s required."
5. MCRS Report Page 7, Aerial Refueling Inventory: "MCRS-16 objectives
required providing a risk assessment, but we found--and DOD officials
confirmed--that a risk assessment for tanker inventory was not
included in the report."
DOD Response: False. DOD officials involved in the GAO investigation
do not recall confirming this incorrect assertion. The MCRS included a
risk assessment for aerial refueling and other mobility areas.
GAO Comment 5: We disagree with DOD's technical comment. DOD officials
confirmed in interviews that risk was considered by use of campaign
metrics and whether the U.S. objectives were met in a scenario. The
MCRS Executive Summary makes only two allusions to risk. The first
allusion is to risk as it pertains to study scope. The second is as
follows: "The study assessed the mobility system's performance by
examining how force closures supported achievement of U.S. campaign
objectives. This was done by assessing required delivery timelines and
a comprehensive set of campaign risk metrics to determine whether
available forces met war fight objectives within desired timelines."
There is no other mention of mobility systems risk in the unclassified
Executive Summary; consequently, we raised the issue in an on-the
record interview setting and documented DOD views concerning the
issue. The DOD technical comment is particularly difficult to
understand since it refers to Air Refueling which was 120 per cent
oversubscribed - without any explanation as to the risk of having
demand exceed capacity by 20 per cent in combat scenarios. Presumably,
this is an example where risk would be discussed in some detail if
mobility systems risk was discussed in this mobility study, but there
was no such discussion. Most recently, in the Mobility Capabilities
Study 2005, mobility systems were categorized by associated risk. This
risk assessment was not done in the MCRS and no specific risk was
reported for any mobility system.
6. Page 9, Definition of Terms, Roll-on/Roll-off ships.
DOD Response: This definition appears to allude to Large Medium Speed
Roll-on/roll-off ships (LMSRs) and is inaccurate. LMSRs are a subset
of the Department's roll-on/roll-off ships (ROROs). ROROs have varying
capabilities and are designed to carry rolling stock and can be loaded
and unloaded by driving on and driving off.
GAO Comment 6: We have modified our report to reflect the appropriate
definition.
[End of section]
Footnotes:
[1] Department of Defense, Mobility Capabilities and Requirements
Study 2016 (Washington, D.C., Feb. 26, 2010).
[2] See GAO, Defense Transportation: Study Limitations Raise Questions
about the Adequacy and Completeness of the Mobility Capabilities Study
and Report, [hyperlink, http://www.gao.gov/products/GAO-06-938]
(Washington, D.C.: Sept 20, 2006).
[3] The Joint Deployment Distribution Enterprise (JDDE) is the complex
of equipment, procedures, doctrine, leaders, technical connectivity,
information, shared knowledge, organizations, facilities, training,
and materiel necessary to conduct joint distribution operations. U.S.
Transportation Command serves as the single synchronizing element on
behalf of, and in coordination with, the JDDE community and
establishes processes to support combatant commanders.
[4] According to DOD, a capability gap is the inability to achieve a
desired effect under specified standards and conditions through
combinations of means and ways to perform a set of tasks. The gap may
be the result of no existing capability, lack of proficiency or
sufficiency in existing capability, or the need to replace an existing
capability. A shortfall may result from a lack of forces, equipment,
personnel, materiel, or capability, and is reflected as the difference
between the required resources and those available to a combatant
commander. When a lack of resources would adversely affect the
command's ability to accomplish its mission, it is described as a
shortfall.
[5] For this report, overlap and excess are used interchangeably. An
overlap (excess) can occur when the military seeks to achieve a
desired effect by performing tasks under specified standards and
conditions and redundant capabilities exist to accomplish a mission or
task and the overlap is determined to be operationally undesirable or
excessive.
[6] Department of Defense, Quadrennial Defense Review Report February
2010 (Washington, D.C., Feb. 1, 2010).
[7] See Department of Defense, Mobility Capabilities and Requirements
Study 2016.
[8] The 2005 Mobility Capabilities Study was issued in December 2005.
It found that projected mobility capabilities were adequate to achieve
U.S. objectives with an acceptable level of risk during the period
from fiscal years 2007 through 2013.
[9] Chairman of the Joint Chiefs Of Staff Instruction 3170.01G, Joint
Capabilities Integration And Development System, (Mar. 1, 2009).
[10] Operational reserves can be an emergency reserve of men and/or
materiel established for the support of a specific operation.
[11] Air Mobility Programs: Hearing before the Air and Land Forces
Subcommittee of the Committee on Armed Services, 111TH Cong. (Apr. 28,
2010) (testimony of Brig. Gen. Michelle D. Johnson, USAF, Director for
Strategy, Policy, Programs, and Logistics, U.S. Transportation
Command).
[12] See GAO, Defense Acquisitions: Timely and Accurate Estimates of
Costs and Requirements Are Needed to Define Optimal Future Strategic
Airlift Mix, [hyperlink, http://www.gao.gov/products/GAO-09-50]
(Washington, D.C.: Nov. 21, 2008).
[13] According to the DOD C-5 Reliability Enhancement and Re-engining
Program (RERP), Combined Operational and Live Fire Test and Evaluation
Report of October 2010, the C-5M undergoing operational testing is
currently operationally effective but is operationally unsuitable. The
MCRS-16 used C-5Ms in calculating the ability of the strategic air
fleet.
[14] Both excess capacity and shortfall in capacity can constitute
risk. For excess capacity, there can be risk that resources needed for
higher priorities are wasted; for capacity shortfalls, there can be
risk to an operation.
[15] According to the National Defense Strategy, operational risks are
those associated with the current force executing the strategy
successfully within acceptable human, material, financial, and
strategic costs. Future challenges risks are those associated with
DOD's capacity to execute future missions successfully against an
array of prospective future challengers.
[16] According to USTRANSCOM, million-ton-miles per day is a measure
of aircraft performance and reflects how much cargo can be delivered
over a given distance in a given period of time.
[17] GAO, Defense Acquisitions: Timely and Accurate Estimates of Costs
and Requirements Are Needed to Define Optimal Future Strategic Airlift
Mix, [hyperlink, http://www.gao.gov/products/GAO-09-50] (Washington,
D.C.: Nov. 21, 2008).
[18] GAO, Military Airlift: DOD Should Take Steps to Strengthen
Management of the Civil Reserve Air Fleet Program, [hyperlink,
http://www.gao.gov/products/GAO-09-625] (Washington, D.C.: Sept. 30,
2009).
[19] GAO, Defense Logistics: Department of Defense's Annual Report on
the Status of Prepositioned Materiel and Equipment Can Be Further
Enhanced to Better Inform Congress, [hyperlink,
http://www.gao.gov/products/GAO-10-172R] (Washington, D.C.: Nov. 4,
2009).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
