Defense Logistics
DOD Input Needed on Implementing Depot Maintenance Study Recommendations Gao ID: GAO-11-568R June 30, 2011This report responds to section 322 of the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009. Section 322 required the Secretary of Defense to contract for a study on the capability and efficiency of the depots of the Department of Defense (DOD) to provide the logistics capabilities and capacity necessary for national defense. DOD placed a task order under an existing contract with LMI, Inc. (LMI) to complete the study, which was to address a range of issues specified in section 322. As required by section 322, the task order specified that the contractor submit an interim report on its study to the Senate and House Committees on Armed Services not later than 1 year after the commencement of the study and a final report not later than 22 months after the date on which the Secretary of Defense enters into the contract. LMI submitted its interim report, containing background information and summary data on the DOD depot maintenance enterprise, to the Committees on Armed Services in December 2009. The final report, containing conclusions and recommendations, was provided to the Committees on Armed Services in February 2011. Section 322 also directed GAO to provide an assessment of the feasibility of the recommendations and whether the findings were supported by the data and information examined and to submit a report to the Committees on Armed Services of the Senate and House of Representatives within 90 days of the date on which the contractor submitted its final report. In response, our objectives were to determine the extent to which (1) LMI's reports addressed the five issues and 33 subissues specified in the law, (2) the findings in LMI's final report were supported by the data and information examined, and (3) the recommendations in LMI's final report were feasible.
LMI's reports addressed 26 of the 33 subissues specified in section 322, and partially addressed the other 7 subissues. For example, within the issue of the adequacy of reports submitted to Congress on each military department's maintenance workload, the reports addressed all 4 subissues, including how accurately depot budget lines reflect depot-level workloads, the accuracy of certain depot maintenance calculations, the usefulness of current depot maintenance reporting requirements, and whether current budgetary guidelines provide sufficient flexibility during the year of execution to make best-value decisions. For each of the other issues, the LMI reports addressed some subissues and partially addressed one or more subissues. For example, the LMI report addressed 5 of the 8 subissues on current and future maintenance environments, but partially addressed the other 3: performance-based logistics, supply chain management, and private-sector depot-level capability and capacity. LMI's study was generally consistent with research standards that define a sound and complete study with regard to design, execution, and presentation. For example, the study's scope was consistent with the available guidance, the study team verified and validated the study data, and assumptions were identified in internal documents and the interim and final reports. However, we also found some areas of concern. These concerns included limited documentation on the maintenance workload and servicemember deployment statistical models used as the basis for some findings, the absence of information in the reports on the limitations present in some of the data used in the study, and unclear report organization. For example, we and officials from the military departments noted that subrecommendations were difficult to identify. Implementation of all five recommendations presented in LMI's report is feasible according to subject matter experts in the depot maintenance community whom we interviewed. The recommendations are (1) revise the statutory framework for depot maintenance, (2) link acquisition and sustainment policies, (3) strengthen the core determination process, (4) improve depot maintenance reporting, and (5) establish an independent commission or series of facilitated forums to review the major alternatives for improving organic depot maintenance management and execution. Although feasible, the interviewees told us that they believe the fifth recommendation is unnecessary because issues highlighted in the study could be addressed by existing DOD bodies. Section 322 also specified that the final report shall include comments provided by the Secretary of Defense and secretaries of the military departments on the findings and recommendations of the study, but DOD's official response did not specifically address any findings or recommendations. DOD officials indicated that the department did not provide more specific comments because DOD did not provide the study--LMI did. Our review of LMI's internal documents determined that DOD and military department officials did provide input and feedback on aspects of the study throughout its design, execution, and report presentation. Without DOD's views about LMI's findings and recommendations, Congress does not have all of the information it needs to help establish a way forward toward more effective and efficient depot-level maintenance. We are making a recommendation that DOD report to Congress with its views on LMI's recommendations, any statutory and policy changes needed to implement the recommendations, actions and timelines for accomplishing ongoing and planned actions to implement the recommendations, and estimated costs and benefits of implementing the recommendations.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Jack E. Edwards Team: Government Accountability Office: Defense Capabilities and Management Phone: (202) 512-8246GAO-11-568R, Defense Logistics: DOD Input Needed on Implementing Depot Maintenance Study Recommendations
This is the accessible text file for GAO report number GAO-11-568R
entitled 'Defense Logistics: DOD Input Needed on Implementing Depot
Maintenance Study Recommendations' which was released on June 30, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-11-568R:
United States Government Accountability Office:
Washington, DC 20548:
June 30, 2011:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Howard P. McKeon:
Chairman:
The Honorable Adam Smith:
Ranking Member:
Committee on Armed Services:
House of Representatives:
Subject: Defense Logistics: DOD Input Needed on Implementing Depot
Maintenance Study Recommendations:
This report responds to section 322 of the Duncan Hunter National
Defense Authorization Act for Fiscal Year 2009.[Footnote 1] Section
322 required the Secretary of Defense to contract for a study on the
capability and efficiency of the depots of the Department of Defense
(DOD) to provide the logistics capabilities and capacity necessary for
national defense. DOD placed a task order under an existing contract
with LMI, Inc. (LMI) to complete the study, which was to address a
range of issues specified in section 322. As required by section 322,
the task order specified that the contractor submit an interim report
on its study to the Senate and House Committees on Armed Services not
later than 1 year after the commencement of the study and a final
report not later than 22 months after the date on which the Secretary
of Defense enters into the contract. LMI submitted its interim report,
containing background information and summary data on the DOD depot
maintenance enterprise, to the Committees on Armed Services in
December 2009. The final report, containing conclusions and
recommendations, was provided to the Committees on Armed Services in
February 2011.
Section 322 also directed GAO to provide an assessment of the
feasibility of the recommendations and whether the findings were
supported by the data and information examined and to submit a report
to the Committees on Armed Services of the Senate and House of
Representatives within 90 days of the date on which the contractor
submitted its final report. In response, our objectives were to
determine the extent to which (1) LMI's reports addressed the five
issues and 33 subissues specified in the law, (2) the findings in
LMI's final report were supported by the data and information
examined, and (3) the recommendations in LMI's final report were
feasible.
Scope and Methodology:
To conduct this work, we reviewed LMI's interim and final reports and
assessed whether the LMI study addressed the issues and subissues
specified in section 322. We reviewed documentation that the LMI study
team collected and interviewed members of the study team to determine
their processes for completing the study. From prior GAO work, we
identified generally accepted research standards that define a sound
and complete study, determined which of these standards were relevant
to LMI's study, and compared characteristics of LMI's study to those
standards. We also interviewed officials from the Office of the
Secretary of Defense, the military departments, a nongeneralizable
sample of 8 of the 17 service depots and their respective commands,
and a defense industry group to obtain their perspectives on the
recommendations in LMI's final report. Further details on our scope
and methodology are provided in enclosure I, starting at page 10.
We conducted this performance audit from January 2011 to June 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
On May 16, 2011, we briefed the Committees on Armed Services of the
Senate and House of Representatives on our findings. This is our final
report and incorporates comments from DOD and LMI. Our evaluation of
the LMI study is discussed in detail in the attached briefing slides
(see enclosure I).
Summary and Recommendation for Executive Action:
LMI's reports addressed 26 of the 33 subissues specified in section
322, and partially addressed the other 7 subissues. For example, on
the issue of the adequacy of reports submitted to Congress on each
military department's maintenance workload, the reports addressed all
4 subissues, including how accurately depot budget lines reflect depot-
level workloads, the accuracy and usefulness of current depot
maintenance reporting requirements, and whether current budgetary
guidelines provide sufficient flexibility during the year of execution
to make best-value decisions. For each of the other issues, the LMI
reports addressed some subissues and partially addressed one or more
subissues. For example, the LMI report addressed 5 of the 8 subissues
on current and future maintenance environments, but partially
addressed the other 3: performance-based logistics, supply chain
management, and private-sector depot-level capability and capacity.
LMI's study was generally consistent with research standards that
define a sound and complete study with regard to design, execution,
and presentation. For example, the study's scope was consistent with
the available guidance, the study team verified and validated the
study data, and assumptions were identified in internal documents and
the interim and final reports. However, we also found some areas of
concern. These concerns included limited documentation on the
maintenance workload and servicemember deployment statistical models
used as the basis for some findings, the absence of information in the
reports on the limitations present in some of the data used in the
study, and unclear report organization. For example, we and officials
from the military departments noted that subrecommendations were
difficult to identify.
Implementation of all five recommendations presented in LMI's report
is feasible according to subject matter experts in the depot
maintenance community whom we interviewed. The recommendations are (1)
revise the statutory framework for depot maintenance, (2) link
acquisition and sustainment policies, (3) strengthen the core
determination process, (4) improve depot maintenance reporting, and
(5) establish an independent commission or series of facilitated
forums to review the major alternatives for improving organic depot
maintenance management and execution. Although feasible, the
interviewees told us that they believe the fifth recommendation is
unnecessary because issues highlighted in the study could be addressed
by existing DOD bodies. Section 322 also specified that the final
report shall include comments provided by the Secretary of Defense and
secretaries of the military departments on the findings and
recommendations of the study, but DOD's official response did not
specifically address any findings or recommendations. DOD officials
indicated that the department did not provide more specific comments
because DOD did not provide the study--LMI did. Our review of LMI's
internal documents determined that DOD and military department
officials did provide input and feedback on aspects of the study
throughout its design, execution, and report preparation. Without
DOD's views about LMI's findings and recommendations, Congress does
not have all of the information it needs to help establish a way
forward toward more effective and efficient depot-level maintenance.
Therefore, we recommend that the Secretary of Defense direct the Under
Secretary of Defense for Acquisition, Technology, and Logistics to
report to Congress with DOD's views on LMI's recommendations, any
statutory and policy changes needed to implement the recommendations,
actions and timelines for accomplishing ongoing and planned actions to
implement the recommendations, and estimated costs and benefits of
implementing the recommendations.
Agency Comments, Third-Party Views, and Our Evaluation:
In commenting on a draft of this report, DOD partially concurred with
our recommendation. DOD stated that it is reviewing the findings and
recommendations in the LMI study and believes that it is premature to
provide the results of this effort until its analysis is complete. DOD
added that a report to Congress on the Department's efforts is neither
necessary nor required. As noted earlier, section 322 specified that
LMI's final report was to include comments from DOD on the study's
findings and recommendations, but DOD did not specifically address the
findings and recommendations in its official response. Therefore, we
continue to believe that such comments would provide important context
to help Congress determine the appropriate course of action when
considering LMI's recommendations.
In commenting on a draft of this report, LMI concurred with our
acknowledgment that the study covered a broad set of depot maintenance
issues well, using a sound research approach that produced reasonable
recommendations. LMI also commented on our finding that 7 of the 33
subissues specified in section 322 were partially addressed by its
study. LMI noted that these 7 subissues were not covered in the
study's reports to the same volume as the others, but stated that this
was because the LMI study team determined that they were not critical
elements of the study's findings and recommendations. LMI stated that
these subissues were fully explored as part of the study. However, our
criterion for evaluating whether LMI addressed all 33 subissues was
the extent to which each of the subissues was covered in LMI's interim
and final reports. Therefore, we believe our finding that LMI's
reports did not fully address these 7 subissues as outlined in section
322 remains valid.
DOD's and LMI's written comments are reprinted in enclosures II and
III, respectively. Both DOD and LMI also provided technical comments
that we incorporated into this report where applicable.
We are sending copies of this report to the appropriate congressional
committees. We are also sending copies to the Secretary of Defense;
the Under Secretary of Defense for Acquisition, Technology, and
Logistics; the Secretaries of the Army, the Navy, and the Air Force;
the Commandant of the Marine Corps; and the Chief Executive Officer of
LMI, Inc. This report also is available at no charge on the GAO Web
site at [hyperlink, http://www.gao.gov].
Should you or your staff have any questions concerning this report,
please contact me at (202) 512-8246 or edwardsj@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. Key contributors to this
report are provided in enclosure IV.
Signed by:
Jack E. Edwards:
Director, Defense Capabilities and Management:
Enclosures - 4:
[End of section]
Enclosure I: Briefing to Congressional Committees:
Defense Logistics: DOD Input Needed on Implementing Depot
Maintenance Study Recommendations:
Briefing for Congressional Committees:
June, 2011:
Contents:
* Introduction;
* Mandate and Objectives;
* Scope and Methodology;
* Summary of Findings;
* Objective 1: Extent Study Addressed Specified Issues;
* Objective 2: Study Design, Execution, and Presentation;
* Objective 3: Feasibility of Recommendations;
* Conclusions;
* Recommendation for Executive Action;
* Agency Comments, Third-Party Views, and Our Evaluation;
* GAO Contact and Staff Acknowledgments;
* Related GAO Products.
Figure 1: Mechanics at Anniston Army Depot Line Up a Turret with its
Hull:
[Refer to PDF for image: photograph]
Source: U.S. Army.
[End of figure]
Introduction:
DOD's depot workload has increased to support overseas operations:
The Department of Defense's (DOD) depot maintenance enterprise employs
approximately 77,000 personnel with an annual operating budget of over
$30 billion. Some of the most in-depth and complex maintenance work is
carried out at 17 DOD depots that have an estimated facility value of
over $48 billion. Additional depot-level work is carried out by
private-sector contractors that may work in public-private
partnerships with depots or in the contractors' own facilities.
Obligations for depot-level maintenance (organic[Footnote 2] and
commercial) have increased in recent years to support overseas
operations in Iraq and Afghanistan, growing from $22.5 billion in
fiscal year 2001 to $32.3 billion in fiscal year 2009.[Footnote 3] A
large portion of this increase has been supported by Overseas
Contingency Operations funding.
The House Committee on Armed Services noted that when wartime
operations in Iraq and Afghanistan cease and supplemental funding for
depot maintenance is reduced, DOD depots may face challenges similar
to those in the post”Cold War environment where public- and private-
sector facilities competed for limited available workload.[Footnote 4]
Fiscal year 2009 National Defense Authorization Act directed DOD to
contract for a study on depot capability and efficiency:
Section 322 of the Duncan Hunter National Defense Authorization Act
for Fiscal Year 2009:[Footnote 5]
* Required the Secretary of Defense to contract for a study on the
capability and efficiency of its depots to provide the logistics
capabilities and capacity necessary for national defense. DOD placed a
task order for the study using an existing contract with LMI, Inc
(LMI).
* Included five issues, with 33 associated subissues, to be addressed
in the overall study and the final report.
* Stated that the study's final report should include recommendations
addressing issues, including:
- what would be required to maintain, in a post-reset[Footnote 6]
environment, an efficient and enduring DOD depot capability, including
appropriate changes to applicable law, and;
- the methodology of determining core logistics requirements,
including an assessment of-risk.
As outlined in the legislation, LMI prepared both an interim and final
report.
* The interim report contains background information and summary data
on the DOD depot maintenance enterprise. It is not an earlier draft of
the final report.
LMI submitted its final report, Future Capability of DOD Maintenance
Depots, to the Committees on Armed Services of the Senate and House of
Representatives on February 15, 2011. The final report contains
conclusions and recommendations.
Mandate and Objectives:
Section 322 of the 2009 National Defense Authorization Act:
* Directed us to submit a report to the Committees on Armed Services
of the Senate and House of Representatives not later than 90 days
after the date on which the final report was submitted.
* Mandated that our report contain an assessment of the feasibility of
the recommendations in the study, and whether the findings were
supported by the data and information examined.
We therefore determined the extent to which:
1. LMI's reports addressed the five issues and 33 subissues specified
in the law,
2. the findings in LMI's final report were supported by the data and
information examined, and,
3. the recommendations in LMI's final report were feasible.
Scope and Methodology:
For all three objectives, we reviewed the legislation and reports on
depots and maintenance issued by us and others; and discussed our
preliminary observations with stakeholders that included DOD, the
military services, and others such as an association representing
private-sector organizations that perform depot-level maintenance.
To determine the extent to which the LMI study addressed the
legislation-specified issues and subissues, we:
* Reviewed LMI's interim and final reports and assessed the extent to
which the reports address the five issues and their 33 subissues.
* To increase the validity of our assessments, had the initial
assessment of each issue by one analyst independently reviewed by
another analyst.
* Had the entire engagement team collectively review and approve this
decision.
To determine the extent to which the report's findings were supported
by the data and information LMI examined, we:
* Used relevant generally accepted research standards based on prior
GAO work[Footnote 7] and evaluated the study against these criteria.
* Had two team members independently review and rate the evidence as
to what, if any, limitations were present for the methods used to
conduct and report the study.
* Discussed each assessment and collectively reached agreement on the
extent to which LMI's study met the standards.
To determine the extent to which the recommendations in LMI's final
report were feasible, we took the following steps:
* Reviewed the recommendations and subrecommendations outlined in the
LMI final report.
* Interviewed subject matter experts who were officials from the
Office of the Under Secretary of Defense for Acquisition, Technology,
and Logistics (Maintenance Policy and Programs); Army, Marine Corps,
Navy, and Air Force headquarters' offices; a nongeneralizable sample
of 8 of the 17 service depots and their respective commands; and a
defense industry group in order to obtain their informed opinions of
the feasibility of the recommendations.
* Selected the eight depot locations based on the following criteria:
- At least one depot for each service.
- Maintenance support for a variety of equipment types.
- Exposure to issues discussed in the LMI reports.
- High workload.
- Previous consolidation or workforce changes.
* Analyzed the officials' responses (e.g., key examples of progress in
implementing parts of the recommendations) collected from our
interviews to determine the feasibility of each recommendation.
We conducted this performance audit from January 2011 to June 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Summary of Findings:
Objective 1: LMI's reports addressed 26 of the 33 subissues specified
in section 322, and partially addressed the other 7 subissues. For
example, within the issue of the adequacy of reports submitted to
Congress on each military department's maintenance workload, the
reports addressed all 4 subissues, including how accurately depot
budget lines reflect depot-level workloads, the accuracy of certain
depot maintenance calculations, the usefulness of current depot
maintenance reporting requirements, and whether current budgetary
guidelines provide sufficient flexibility during the year of execution
to make best-value decisions. For each of the other issues, the LMI
reports addressed some subissues and partially addressed one or more
subissues. For example, the LMI report addressed 5 of the 8 subissues
on current and future maintenance environments, but partially
addressed the other 3: performance-based logistics, supply chain
management, and private-sector depot-level capability and capacity.
Objective 2: LMI's study was generally consistent with research
standards that define a sound and complete study with regard to
design, execution, and presentation. For example, the study's scope
was consistent with the available guidance, the study team verified
and validated the study data, and assumptions were identified in
internal documents and the interim and final reports. However, we also
found some areas of concern. These concerns included limited
documentation on the maintenance workload and servicemember deployment
statistical models used as the basis for some findings, the absence of
information in the reports on the limitations present in some of the
data used in the study, and unclear report organization. For example,
we and officials from the military departments noted that
subrecommendations were difficult to identify.
Objective 3: Implementation of all five recommendations presented in
LMI's report is feasible according to subject matter experts in the
depot maintenance community whom we interviewed. The recommendations
are (1) revise the statutory framework for depot maintenance, (2) link
acquisition and sustainment policies, (3) strengthen the core
determination process, (4) improve depot maintenance reporting, and
(5) establish an independent commission or series of facilitated
forums to review the major alternatives for improving organic depot
maintenance management and execution. Although feasible, the
interviewees told us that they believe the fifth recommendation is
unnecessary because issues highlighted in the study could be addressed
by existing DOD bodies. Section 322 also specified that the final
report shall include comments provided by the Secretary of Defense and
secretaries of the military departments on the findings and
recommendations of the study, but DOD's official response did not
specifically address any findings or recommendations. DOD officials
indicated that the department did not provide more specific comments
because DOD did not provide the study”LMI-did. Our review of LMI's
internal documents determined that DOD and military department
officials did provide input and feedback on aspects of the study
throughout its design, execution, and report presentation. Without
DOD's views about LMI's findings and recommendations, Congress does
not have all of the information it needs to help establish a way
forward toward more effective and efficient depot-level maintenance.
Recommendation for Executive Action: We are making a recommendation
that DOD report to Congress with its views on LMI's recommendations,
any statutory and policy changes needed to implement the
recommendations, actions and timelines for accomplishing ongoing and
planned actions to implement the recommendations, and estimated costs
and benefits of implementing the recommendations.
Objective 1: Extent Study Addressed Specified Issues:
Overview - Study either addressed or partially addressed all 33
specified subissues:
The LMI study addressed 26 of the 33 subissues specified in section
322, and partially addressed the remaining 7.[Footnote 8] The five
major issues were as follows:
* Specific contents, including quantitative analyses; direct input
from DOD; identification of each type of maintenance activity; and
examination of relevant laws, guidance, and GAO reports.
- Four of the six subissues for this issue were addressed.
* Life-cycle sustainment strategies and implementation plans.
- Seven of the eight subissues for this issue were addressed.
* Current and future maintenance environments.
- Five of the eight subissues for this issue were addressed.
* The visibility of each military department's maintenance workload in
reports submitted to Congress.
- All four subissues for this issue were addressed.
* Specified contents for the final report, including recommendations
to ensure an efficient and enduring DOD depot capability.
- Six of the seven subissues for this issue were addressed.
Four of six content subissues were addressed, and two were partially
addressed:
Quantitative analysis of post-reset DOD depot capability-”Addressed:
Study presented the results of a quantitative analysis to make
projections of the future need for depot maintenance and the
utilization of workload capacity.
Input from Secretary of Defense and military departments-”Addressed:
Officials from the Office of the Secretary of Defense and the services”
as well as our review of LMI workpapers confirmed that they provided
input to the LMI study team. However, the DOD letter included in LMI's
final report did not identify any concerns or whether DOD concurred
with LMI's recommendations.
Input from regular and reserve components of the armed forces”-
Partially addressed: The LMI study team met with officials from
regular components of the military, including select depots, but input
from the reserve components was limited to one Army National Guard
unit.
• Identify and address each type of activity carried out at various
maintenance facilities”Partially addressed: The study identified and
addressed maintenance activities at facilities such as the major
organic depots,[Footnote 9] but did not identify or address activities
at other facilities specified in the law (such as regional sustainment-
level maintenance sites).
• Examine relevant DOD and military department guidance, including
budget guidance”-Addressed: The study presented applicable DOD
guidance in a table and discussed department guidance specific to
budgeting.
• Examine applicable laws, as well as the relevant body of GAO work”-
Addressed: The study discussed laws(e.g. 10 U.S.C. § 2464: core
logistics capabilities), recommended changes to statutes, and included
a comprehensive appendix with summaries of relevant GAO reports.
Seven of eight life-cycle sustainment strategy subissues were
addressed, and one was partially addressed:
Role of each type of maintenance activity-”Partially addressed:
Discussed organic depot, forward-deployed, and commercial activities,
but did not link the role of each type of maintenance activity to DOD
strategies and implementation plans.
Business operations-”Addressed: Discussed links between depots'
business strategies and DOD's strategic planning and management of
business operations.
Workload projection-”Addressed: Discussed possible future workload
changes and their causes, and evaluated the services' workload
projections.
Outcome-based performance management objectives”-Addressed: Discussed
systems that use outcome-based metrics, and recommended improvements.
Adequacy of information technology systems-”Addressed: Discussed how
the depots are relying on planned modernization of technology to apply
best practices.
Workforce, including skills required and development-”Addressed:
Discussed workforces for both the organic and commercial sectors of
depot-level maintenance.
Budget and fiscal planning policies-”Addressed: Discussed budget
visibility and planning processes, and the ineffectiveness of existing
budget exhibits to provide visibility.
Capital investment strategies”Addressed: Discussed how revised use of
core[Footnote 10] calculations could inform capital investment
strategies.
Five of eight subissues discussing maintenance environments were
addressed, and three were partially addressed:
Performance-based logistics-”Partially addressed: Discussed this
maintenance strategy in the context of current private-sector support
of depot-level maintenance, but did no discuss its impact on the
future depot maintenance environment.
Supply chain management-”Partially addressed: Discussed problems with
DOD's approach to integrating supply chains with maintenance depots,
but did not discuss how supply chain management should fit into the
future depot maintenance environment.
Condition-based maintenance[Footnote 11]”-Addressed: Discussed
implementation of condition-based maintenance, noting that
implementation is ongoing and uneven across the services.
Reliability-based maintenance[Footnote 11]-”Addressed: Discussed DOD
policies for this subissue, noted the services' inconsistent
implementation, and provided examples.
Consolidation and centralization-”Addressed: Discussed the different
levels of maintenance, forward-based depot capacity, and recommended
depot consolidation.
Public-private partnerships”-Addressed: Discussed the services' plans
to increase facility use by partnering with industry, and noted that
implementation is inconsistent among the services.
Private-sector depot-level capability and capacity”-Partially
addressed: Discussed future capability of commercial depot-level
maintenance (e.g., workforce and globalization issues), but did not
cover private-sector capacity because data associated with this topic
were proprietary.
Impact of proprietary technical documentation-”Addressed: Discussed
the present context for obtaining technical data rights. and
highlighted areas for improvement.
All four subissues discussing the adequate visibility of the
maintenance workload were addressed:
Whether depot budget lines in current budget displays accurately
reflect depot-level workloads”-Addressed: Discussed budget exhibits in
the context of needed improvements in their clarity and usefulness for
DOD officials and Congress, and recommended improvements to budget
reporting.
Accuracy of core and 50/50[Footnote 12] calculations-”Addressed:
Highlighted concerns about core and 50/50 calculations, concluded that
current core calculations are ineffective and destabilize future
workload capability, and recommended steps to address these concerns.
Usefulness of current depot maintenance reporting requirements for
oversight”-Addressed: Presented information on problems with reporting
(e.g., reports do not relate to outcomes), concluded that senior DOD
and service leaders and-Congress do not get timely warning of eroding
capability or workload, and recommended improving depot maintenance
reporting.
Whether current budgetary guidelines provide financial flexibility
during the execution year to permit military departments to make best-
value decisions between maintenance activities”-Addressed: Discussed
the ability of the services to realign funding for depot-level
maintenance, and noted that the LMI study team was unable to document
any instances in which the existing guidelines precluded a service
from making best-value decisions.
Six of seven subissues specified for final report were addressed, and
one was partially addressed:
Description of the depot maintenance environment”-Addressed: Described
the current workload, workforce, and funding of the depots, and
modeled anticipated future changes.
Recommendations on what would be required, in a post-reset environment
to maintain an efficient and enduring DOD depot capability-”Addressed:
Presented recommendations (such as improving core workload
determinations, linking acquisitions and sustainment, and revising
definition) to improve the depots' viability.
Recommendations for changes to applicable law appropriate for a post-
reset depot maintenance environment-”Addressed: Recommended revising
the statutory definition of depot maintenance, including a
comprehensive definition of depot-level software maintenance.
Recommendations on methodology for determining core logistics
requirements-”Addressed: Recommended strengthening the core
requirements determination process, improving how core is reported,
and using best practices from the services to improve risk assessments.
Business rules to keep DOD depots efficient and cost effective”-
Addressed: Recommended revising the core determination process to
improve its utility as a business tool.
Strategy to produce performance-driven outcomes and meet materiel
readiness goals at DOD depots”-Partially addressed: Recommended
linking depot maintenance products to operational performance, but did
not specifically address total ownership cost and repair cycle time as
specified.[Footnote 13]
Comments from Secretary of Defense and military department secretaries
on the study's findings and recommendations”-Addressed: LMI provided a
draft copy of the final report to DOD, but DOD's official letter,
included in LMI's final report, did not specifically address the
report's findings and recommendations.
Objective 2: Design, Execution, and Presentation:
Overview - Final report findings were generally supported by data
and other information, but did not meet all the research standards:
We determined that the final report findings were generally supported
as a result of our evaluation of the study's design, execution, and
presentation.
* For example, the study's scope was consistent with guidance, and
assumptions were well identified. In addition, the study team verified
and validated the study data, the report's conclusions were sound and-
complete, and the report provided realistic options.
However, we found some areas of concern.[Footnote 14]
* Specifically, areas where we found concerns included not identifying
data and model limitations in the report, a lack of documentation in
the reports regarding modeling processes, unclear report organization,
and insufficient support presented in the reports for some study
conclusions, recommendations, and subrecommendations.
To determine whether the findings in LMI's report were supported by
the data and information examined, we evaluated how adequately the
study met generally accepted research standards”on design, execution,
and presentation”identified in our prior work.[Footnote 15]
Study was generally well designed; and scope was consistent with
guidance:
LMI's study team used the language in section 322 to develop the
study's scope. They informed the Senate and House Committees on Armed
Services about the scope and approach of the study early on, and
provided periodic updates regarding the study's design.
The study team did not develop a formal study plan, but our review of
the interim report and internal documents revealed that they
developed, followed, and updated a comprehensive study approach,
including an initial scoping document. The study team explained and
documented deviations from the study approach.
* The approach included using site visits, collecting and analyzing
depot maintenance and servicemember-deployment data and other
information, and interviewing depot maintenance experts.
* LMI's study team members told us that they clarified the major issue
areas after meeting with stakeholders. LMI's study team briefed key
stakeholders, including staff for the Senate and House Committees on
Armed Services and the DOD Maintenance Executive Steering Committee,
of updates to the study approach and preliminary observations.
[Footnote 16]
* The study team originally planned to hold an industry forum and
collect private-sector data to compare with DOD's data, but they did
not to do so because they decided to focus on organic depot
maintenance. In part, this change in scope and approach was due to
LMI's inability to obtain certain types of information that industry
officials considered proprietary.
Study Design: The study identified assumptions that generally
contributed to an objective and balanced research effort:
The study identified assumptions about the future viability of organic
depots and applied those assumptions consistently throughout the
report. The final-report's findings, conclusions, and recommendations
were developed using these assumptions.
The study team assumed that:
* A drawdown of operations in Iraq and Afghanistan and dramatic force
structure cuts would lead to significant reductions in future depot
maintenance requirements, particularly for the Army and Marine Corps.
* DOD would face reduced funding support for depot maintenance,
decreasing the amount of work performed in the organic depots.
* Army and Marine Corps organic depot maintenance funding would follow
similar patterns because these services engaged in the same ground
operations.
The study team used publicly available information to identify
reasonable assumptions that are used throughout LM1's two reports to
support the analyses.[Footnote 17] The assumptions in the study
generally contributed to an objective and balanced research effort.
The clarity of the findings, conclusions, and recommendations might,
however, have been improved by identifying limitations to the study's
assumptions in the reports. For example, because of decisions made
about the scope of the study, the study team limited its
considerations of historical data to the period from fiscal years 2001
through 2009. As a result, the study team did not look at lessons
learned and other information from DOD's downsizing actions, such as
those in the 1990s.
Study Execution: Methods were adequately executed and historical data
were consistently used, but documentation was not readily available:
The study methods were adequately executed, and the historical data
were consistently used.
* As originally. planned, LMI's study team visited at least one
maintenance depot from each military service or depot type. In
addition, they held meetings with a defense industry association and a
private-sector facility conducting depot-level maintenance.
* The information provided in the interim report showed some evidence
that the study team analyzed DOD's life-cycle sustainment and
maintenance strategies consistent with the study methods. However,
LMI's internal documents lacked sufficient detail for us to assess
actions the study team took to analyze these strategies.
* The study's methods generally support accomplishing the objectives
identified in the study's approach, but we could not determine whether
the study's methods could produce the data and information necessary
to address some of the objectives.
- For example, one of the team's objectives was to examine the
strategies to enable and monitor performance-driven outcomes in order
to meet maintenance and readiness goals. The study team could not
provide documents that clearly identified actions taken to acquire the
necessary data and information for this objective.
* The interim and final reports fully and completely identify the
types of historical data presented and consistently used the
historical data in the reports' analyses (e.g., data from fiscal years
2001 through 2009).
However, some documentation of methods used in the study was not
readily available for our review and assessment.
* Some documentation was limited to data worksheets and references to
previous reports.
* For example, to gain an understanding of how the study approach was
executed we reviewed numerous internal documents, presentations,.and
legislation, and conducted interviews with the study team,, instead of
reviewing a single document describing the study methods, which was
not available.
Data verification and validation were conducted, but report did not
identify data limitations:
In accordance with International Organization for Standardization 9001
guidance,[Footnote 18] LMI maintains a quality management system that
requires quality and technical reviews of the study.
* The study team verified and validated study data and documented
verification and validation processes.
* Although not required by LMI's review process, the LMI study team
documented its efforts to validate data obtained from the military
services and solicit comments on the draft report. The study team did
not incorporate all of the military services' comments.
* The study's quality reviewer told us that he met frequently with
study team members to discuss their approach and to review data and
information the study team had gathered. He also provided verbal and
written comments to drafts of the interim and final reports, including
information on the team's verification and validation of data.
The study's quality reviewer and the study director recorded the
completion of the quality review for the final report on February 11,
2011, prior to report's issuance.
We identified concerns regarding incomplete descriptions of data
limitations and their impact.
* For example, the study identified a limitation of Air Force and Navy
aircraft and ship operating hours data for fiscal year 2009, noting
that the data were incomplete at the time of the data submission.
However, the study did-not describe the impact of this limitation or
the team's methods for addressing the limitation.
* In an interview, the study team told us that the military services'
data are sometimes not comparable, but the team did not identify this
limitation, or its methods for addressing this limitation in the
report. When asked about this concern, LMI attributed the missing
information to an internal decision to focus on strategic issues.
Models were adequate for their intended purposes, but the study did
not sufficiently explain the model limitations:
The study's models were adequate and reasonable for their intended
purposes-”to summarize simple predictive relationships among
servicemember deployment levels, maintenance labor hours, and
maintenance workforce.[Footnote 19]
The study team properly assembled and used the models' historical data
to support the study's planned approach, but internal documents and
the reports provide little information about the reliability of the
input data.
* The usefulness of these models is limited by assumptions about
deployment scenarios”-a limitation the study documentation did not
mention.
Report Presentation: Study addressed the objectives, but the final
report did not sufficiently explain model processes and organization
was unclear:
The study addressed the objectives. We found the reports addressed 26
of the 33 subissues and partially addressed the other 7 subissues (as
stated in objective 1).
However, concerns exist regarding descriptions of model processes and
the clarity of the report's organization.
* The final report's description of the models does not provide
sufficient detail for a reader to independently replicate the
analysis.[Footnote 20]
- The report focuses on the models' results with little explanation or
documentation of models' limitations or how different assumptions or
the data may have affected results.
- Members of the study team told us that they did not include detailed
descriptions of the models to avoid confusing nontechnical readers.
However, to avoid confusion, they might have alternatively presented
the details of the models in an appendix or footnotes.
* The final report includes an executive summary and topically
organized chapters, but supporting evidence for the conclusions and
recommendations is difficult to ascertain.
- For example, chapters 4, 5, and 6 of the final report present
conclusions and recommendations under separate headings, but chapters
2 and 3 present recommendations in the section titled "conclusions"
but do not include a recommendations section.
- In addition, some data in the interim report were assembled from
different sources than the data used in the corresponding sections of
the final report. The different numbers could confuse or mislead
report readers. Study team members explained that they developed the
interim report data from different sources because the study team had
not yet received requested data from the military services.
Sound conclusions and realistic options provided; concerns with
presented support for conclusions and recommendations exist:
The conclusions presented in the final report were sound and complete
in that they covered all the findings and addressed the
vulnerabilities to DOD depot maintenance that were discussed in the
final report.[Footnote 21]
The final report's discussion of the recommendations presented
realistic options. We found that members of the DOD depot maintenance
community considered all five recommendations to be feasible (as we
will discuss in objective 3).
As noted earlier, the study was generally well executed, but we have
concerns about the presentation of the support for some of the
conclusions, recommendations, and subrecommendations in the final
report.
* For example, the report presented findings showing that out of the
four military services, the Air Force and Navy's depot workloads were
stable and did not.provide examples of how "nothing happened according
to plan." Therefore, the conclusion that the depots' transitions from
peacetime to wartime support during the last decade "consistently
exhibited turbulence" may be overstated.
* Also, the report does not establish a negative effect that warrants
the subrecommendation to adopt a comprehensive definition of depot-
level software maintenance.
* In addition, Army, Navy, and Air Force officials told us that the
final report did not present specific steps for implementation, or the
benefits and unintended consequences, of the recommendations.
Objective 3: Feasibility of Recommendations:
Overview - The DOD depot maintenance community stated that LMI's
recommendations were feasible to implement:
The DOD depot maintenance community considered all five
recommendations in the LMI report to be feasible to implement. The
five general recommendations for changes to depot management practices
were:
1. Revise the statutory framework of depot maintenance.
2. Closely link acquisition and sustainment policies and outcomes with
regard to depot maintenance.
3. Strengthen the core determination process.
4. Improve depot maintenance reporting.
5. Establish an independent commission or sponsor a series of
facilitated forums to review five major alternatives for improving
organic depot maintenance management and execution.
Even though members indicated that implementation is feasible during
interviews with us, the DOD depot maintenance community stated that
establishing an independent commission to review depot maintenance is
unnecessary.
DOD's official response to the LMI study did not specifically address
the findings and recommendations.
* Section 322 specified that the study shall include comments.provided
by the Secretary of Defense and the secretaries of the military
departments on the findings and recommendations of the study.
* Officials within DOD stated that because the study_was prepared by
LMI and not the department DOD did not provide more detailed official
comments on the findings and recommendations contained in the study.
* DOD and military department officials and staff, however, provided
input and feedback on aspects of the study's design, execution, and
presentation.
Lack of specific comments could limit Congress' and DOD's ability to
improve depot-level maintenance:
Various officials within the DOD depot maintenance community provided
us with examples of initiatives under way to address some of LMI's
recommendations.
LMI's study did not specifically address challenges or costs
associated with implementing the recommendations.
Because DOD did not provide specific comments on LMI's findings and
recommendations, it is unclear as to strengths and weaknesses of LMI's
recommendations, which recommendations DOD plans to implement, and
what initiatives DOD has under way to improve depot maintenance.
Without DOD's perspectives on LMI's recommendations, Congress may be
limited in its ability to determine the actions needed to improve
depot-level maintenance.
Recommendation 1 - Revise the statutory framework of depot
maintenance:
The study recommended revising the statutory framework of depot
maintenance and also included three subrecommendations:
* Remove exclusions (e.g., modifications designed to "improve
performance") from the statutory definition of depot maintenance.
* Adopt a comprehensive definition of depot-level software maintenance.
* Require better information (e.g., informing Congress prior to
exiting Milestone B[Footnote 22] if a weapons system will not undergo
depot maintenance) regarding weapons systems that do not undergo depot
maintenance.
According to members of the DOD depot maintenance community, revising
the statutory definition of depot maintenance is feasible. Officials
noted that the current definition is open to interpretation among the
services and should be clarified.
However, members of the community had mixed opinions regarding the
feasibility of adopting the subrecommendation pertaining to a
comprehensive definition of depot-level software maintenance.
* Currently, officials noted that the services vary in terms of the
types of software maintenance (e.g., rewriting code for a weapons
system upgrade) that are performed at the depot level as well as their
interpretation of what is considered to be software maintenance.
* Air Force and Marine Corps officials told us that representatives
from across the services are working with the Office of the Secretary
of Defense to craft a revised DOD-wide definition of software
maintenance.
Recommendation 2 - Link acquisitions and sustainment:
The study recommended changes to more closely link acquisition and
sustainment policies and outcomes with regard to depot maintenance and
also included two subrecommendations:
* Designate completion of a strategic core logistics analysis as a
specific exit criterion for Milestone A review.[Footnote 23]
* Make decisions concerning the core requirements determination
process and source of repair earlier in the acquisition process.
[Footnote 24]
The members of the DOD depot maintenance community generally agreed
that linking acquisition and sustainment policies is feasible.
Officials from some services also described related steps that their
service has taken toward implementing the recommendation.
* Air Force officials told us that they have already implemented these
changes.
* Navy officials told us that the Navy and the Marine Corps have a
process in place to ensure that logistics and sustainment issues are
considered early in the acquisition/engineering process.
With regard to one of the subrecommendations, members of the community
generally agreed that core logistics analysis should be done earlier
in the acquisition process, although opinions varied as to whether
Milestone A was the appropriate stage for analysis. Examples of
feedback from service officials follow:
* Army and Navy officials stated that there is insufficient
information available regarding systems design at Milestone A to
complete a core logistics analysis.
* Army officials told us that they believe it is more appropriate to
conduct core logistics analysis at Milestone B because at that stage a
program manager is assigned and a baseline for the weapon system has
been established.
Recommendation 3 - Strengthen the core determination process:
The study recommended strengthening the core determination process and
also included three subrecommendations:
* Structure a revised core determination process to be visible and
readily understood.[Footnote 25]
* Align core and 50/50 in a single statute.
* Require all contracts that include sustainment as part of the
statement of work to provide an annual estimate of the amount of
funding expended on depot-level maintenance.
According to members of the DOD depot maintenance community,
strengthening the core determination process is feasible.
However, some officials expressed reservations or mixed views about
the feasibility of implementing two of the subrecommendations. For
example:
* Various service officials noted that aligning the core and 50/50
policies into a single statute would be challenging because core
analysis measures workload in direct labor hours while the 50/50
report measures the balance of public- and private-sector funds made
available for depot maintenance.
* As to the feasibility of requiring all contracts to identify
sustainment costs as part of the statement of work, Army officials
stated this change may not be cost-effective. Specifically, the
officials stated that this subrecommendation could make contract
negotiations difficult because the Army still uses a lot of sole-
source contracting. Marine Corps officials also told us that this
recommendation would be more useful if contractors were required to
report where the depot work was conducted.
Recommendation 4 - Improve depot maintenance reporting:
The study recommended revising depot maintenance reporting and also
included four subrecommendations:
* Provide a more complete accounting of all products and services
being purchased from the depot provider ” including procurement,
modernization, and sustainment elements.
* Make available a more complete presentation of the contributions
(e.g., work done on major product lines influencing the materiel
availability elements) of all providers of depot maintenance.
* Catalog, verify, validate, and accredit requirements determination
methods and presentation of risk for each of the major depot product
lines being resourced.
* Present depot requirements in an operational context, linking inputs
to the ability to achieve the outputs by which the services meet
combatant commander needs.
According to members of the DOD depot maintenance community, improving
depot maintenance reporting is feasible.
* Army and Navy officials asserted that their respective services each
have an effective depot maintenance reporting process, and the Navy
stated that this DOD-wide recommendation applies more to the other
services.
* Marine Corps officials stated that they were in the process of
creating a maintenance planning tool to catalog, verify, and validate
their requirements.
Recommendation 5 - Establish an independent commission:
The study recommended the establishment of an independent commission
or sponsorship of a series of facilitated forums to review the
following five major alternatives for improving organic depot
maintenance management and execution:
* Enhanced status quo (each service would continue to maintain its own
depots, and a "corporate board" built on the existing Maintenance
Executive Steering Committee would address issues such as workload
imbalances and capacity underutilization).
* Commodity executive agents (a single manager would be designated to
be responsible for the management of specified commodities across the
DOD enterprise).
* Commercial management (contractors would assume responsibility for
DOD's depots).
* Public corporation (Congress would establish an entity to oversee,
operate, and staff DOD's depots).
* Defense agency or command (the current depot structure would be
consolidated into a single agency or command).
Most members of the DOD depot maintenance community told us that the
recommendation is feasible, and some noted that prior depot
maintenance studies contained comparable recommendations.25
However, members of the community stated that the recommendation
should not be implemented.
* Air Force officials stated that the issues related to depot
maintenance are process driven, and were not related to the
organization's structure. Army officials also told us that they do not
see the benefit of establishing a consolidated agency or command
focused on depot maintenance.
* Army officials told us that DOD has the ability to facilitate a
series of forums through the Maintenance Executive Steering Committee.
They said that such forums would allow the services to share best
practices and DOD to adopt the practices that were applicable across
the services.
* Marine Corps and Navy officials also told us that the issues
highlighted in the LMI final report could be more appropriately
addressed by existing DOD bodies, such as the Maintenance Executive
Steering Committee.
Conclusions:
The LMI study provides a good basis for moving forward to enhance the
capability and efficiency of DOD's depots and provide the future
logistics capabilities and capacity necessary for national defense.
* The wide range of topics covered in the study reflect key depot-
level maintenance concerns, as outlined in legislation mandating the
LMI study.
* While LMI's reports did have some limitations with regard to
information presentation, our review showed that the rigor of the
study's design and execution typically adhered to generally accepted
research standards.
Because DOD did not provide specific comments on LMI's findings and
recommendations, it is difficult to ascertain which, if any,
recommendations DOD plans to implement. By not providing these
comments, DOD missed an opportunity to document its related ongoing
and planned initiatives and to comment on the strengths and weaknesses
of LMI s recommendations.
Without these important perspectives and added information, DOD and
military service leaders and Congress may not be optimally positioned
to decide which”if any”of LMI's recommendations to implement,
determine timetables for implementation, and take other actions that
could improve the future effectiveness and efficiency of DOD's depot-
level maintenance.
Recommendation for Executive Action:
To enhance the capability and efficiency of DOD's depots and provide
the future logistics capabilities and capacity necessary for national
defense, we recommend that the Secretary of Defense direct the Under
Secretary of Defense for Acquisition, Technology, and Logistics to
provide a report to Congress”within 90 days of publication of our
report”regarding DOD's and the military services' views on LMI's
findings and recommendations. Among other things, the report should:
* describe DOD's views on LMI's recommendations,
* specify any statutory and policy changes needed to implement the
recommendations,
* identify actions and timelines for accomplishing ongoing and planned
actions to implement the recommendations, and,
* estimate the various costs and benefits associated with implementing
the recommendations.
Agency Comments, Third-Party Views, and Our Evaluation:
In written comments on a draft of this report, DOD partially concurred
with our recommendation. DOD stated that it is reviewing the findings
and recommendations in the LMI study and believes that it is premature
to provide the results of this effort until its analysis is complete.
DOD added that a report to Congress on the Department's efforts is
neither necessary nor required. As noted earlier, section 322
specified that LMI's final report was to include comments from DOD on
the study's findings and recommendations, but DOD did not specifically
address the findings and recommendations in its official response.
Therefore, we continue to believe that such comments would provide
important context to help Congress determine the appropriate course of
action when considering LMI's recommendations.
In written comments on a draft of this report, LMI concurred with our
acknowledgment that the study covered a broad set of depot maintenance
issues well, using a sound research approach that produced reasonable
recommendations. LMI also commented on our finding that 7 of the 33
subissues specified in section 322 were partially addressed by its
study. LMI noted that these 7 subissues were not covered in the
study's reports to the same volume as the others, but stated that this
was because the LMI study team determined that they were not critical
elements of the study's findings and recommendations. LMI stated that
these subissues were fully explored as part of the study. However, our
criterion for evaluating whether LMI addressed all 33 subissues was
the extent to which each of the subissues were covered in LMI's
interim and final reports. Therefore, we believe our finding that
LMI's reports did not fully address these 7 subissues as outlined in
section 322 remains valid.
Related GAO Products:
Depot Maintenance: Navy Has Revised Its Estimated Workforce Cost for
Basing an Aircraft Carrier at Mayport, Florida. [hyperlink,
http://www.gao.gov/products/GAO-11-257R]. Washington, D.C.: March 3,
2011.
Depot Maintenance: Air Force Is Assessing Engine Maintenance Options
for Work Currently Performed at Kelly Aviation Center. [hyperlink,
http://www.gao.gov/products/GAO-11-274R]. Washington, D.C.: February
11, 2011.
Depot Maintenance: Improved Strategic Planning Needed to Ensure That
Navy Depots Can Meet Future Maintenance Requirements. [hyperlink,
http://www.gao.gov/products/GA0-10-585]. Washington, D.C.: June 11,
2010.
Depot Maintenance: Improved Strategic Planning Needed to Ensure That
Air Force Depots Can Meet Future Maintenance Requirements. [hyperlink,
http://www.gao.gov/products/GA0-10-526]. Washington, D.C.: May 14,
2010.
Depot Maintenance: Improved Strategic Planning Needed to Ensure That
Army and Marine Corps Depots Can Meet Future Maintenance Requirements.
[hyperlink, http://www.gao.gov/products/GAO-09-865]. Washington, D.C.:
September 17, 2009.
Depot Maintenance: Actions Needed to Identify and Establish Core
Capability at Military Depots. [hyperlink,
http://www.gao.gov/products/GAO-09-83]. Washington, D.C.: May 14, 2009.
Depot Maintenance: DOD's Report to Congress on Its Public-Private
Partnerships at Its Centers of Industrial and Technical Excellence
(CITEs) Is Not Complete and Additional Information Would
Be Useful. [hyperlink, http://www.gao.gov/products/GAO-08-902R].
Washington, D.C.: July 1, 2008.
Depot Maintenance: Issues and Options for Reporting on Military
Depots. [hyperlink, http://www.gao.gov/products/GAO-08-761R].
Washington, D.C.: May 15, 2008.
[End of briefing slides]
Enclosure II: Comments from the Department of Defense:
Office Of The Assistant Secretary Of Defense:
Logistics And Materiel Readiness:
3500 Defense Pentagon:
Washington, DC 20301-3500:
June 17, 2011:
Mr. Jack E. Edwards:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Edwards:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-11-568R, "Defense Logistics: DoD Input Needed on
Implementing Depot Maintenance Study Recommendations," dated May 16,
2011 (GAO Code 351555).
The Department partially concurs with the GAO recommendation and
appreciates the opportunity to comment on the GAO Draft Report
(enclosed).
My point of contact in this matter is Mr. Hal Amerau. He can be
reached at (703) 697-1903.
Sincerely,
Signed by:
Illegible, for:
Alan F. Estevez:
Principal Deputy:
Enclosure: As stated.
[End of letter]
GAO Draft Report Dated May 16, 2011:
GAO-11-568R (GAO Code 351555):
"Defense Logistics: DOD Input Needed On Implementing Depot Maintenance
Study Recommendations"
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Under Secretary of Defense for Acquisition, Technology and
Logistics to provide a report to Congress”within 90 days of
publication of our report”regarding DoD's and the military services'
views on LMI's findings and recommendations. Among other things, the
report should:
* Describe DoD's views on LMI's recommendations;
* Specify any statutory and policy changes needed to implement the
recommendations;
* Identify actions and timelines for accomplishing ongoing and planned
actions to implement the recommendations; and;
* Estimate the various costs and benefits associated with implementing
the recommendations (See page 35/GAO Draft Report.)
DOD Response: Partially concur. The Department is actively reviewing
the LMI findings and recommendations detailed in its independent study
of the capabilities and efficiencies of the maintenance depots of the
Department of Defense (DoD). The LMI study raises a number of issues
that could have significant financial, organizational, legislative,
and readiness implications. The Department intends to take the results
of the study into account as it moves forward in this complex and
important area. The Department is carefully analyzing each of the
recommendations to determine their impact on DoD capabilities and to
determine the DoD's way forward. It is premature to provide the
results of the Department's efforts until its analyses is completed.
The Department does not concur that a report to Congress on the
Department's efforts is necessary or required.
[End of enclosure]
Enclosure III: Comments from LMI, Inc.
LMI:
2000 Corporate Ridge:
McLean, VA 22102-7805:
T: 703-971-9800:
F: 703-917-7597:
[hyperlink, http://www.lmi.org]
June 15, 2011:
Dr. Jack E. Edwards:
Director, Defense Capabilities and Management:
Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Dr. Edwards:
This is LMI's response to the Government Accountability Office's draft
report GAO 11-568R, Defense Logistics: DoD Input Needed on
Implementing Depot Maintenance Study Recommendations. We appreciate
the opportunity to comment on your review of our interim and final
Duncan Hunter National Defense Authorization Act Section 322 reports
on future DoD depot capabilities. Our reports were written to
facilitate critical executive and legislative efforts to chart a fresh
course toward more adaptive, responsive, and effective depot
maintenance.
With regard to your review, we concur with your acknowledgment that
the study covered a broad set of depot maintenance issues well, using
a sound research approach that produced reasonable recommendations.
GAO found that our reports address the 33 sub-issues specified in
Section 322 and the study was generally consistent with research
standards that define a sound and complete study with regard to
design, execution, and presentation. GAO noted that, according to
subject matter experts in the depot maintenance community, the
implementation of all five recommendations presented in LMI's final
report is feasible.
We would like to comment briefly on one point associated with your
findings. According to GAO 11-568R, LMI's reports fully address 26 of
the 33 required sub-issues and partially address the other
7 sub-issues. It is our view that, through the application of a
variety of valid research techniques, we did, in fact, fully consider
all 33 sub-issues during our study process. The sub-issues GAO
identified as partially addressed are not covered in the reports to
the same volume as the other issues because they were not critical
elements of our findings and recommendations; but, be assured, they
were fully explored as part of the study. We believe the content of
our reports fully comports with the direction of the House and Senate
Committees on Armed Services. The required content of the reports,
particularly the final report, is clearly stipulated as seven areas (A”
G) in subsection (e) of the legislation.
Our Section 322 study highlights critical factors that will shape the
future of the defense depots. We believe the pragmatism of our
recommendations will be proven as DoD moves to address fundamental
changes necessary for the future management and execution of its depot
maintenance programs.
We are pleased to have produced the first comprehensive look at depot
maintenance in a decade.
Sincerely,
Signed by:
Jeffery P. Bennett:
Senior Vice President, Logistics Management:
[End of enclosure]
Enclosure IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Jack E. Edwards, (202) 512-8246 or edwardsj@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Kimberly Seay, Assistant
Director; Richard Hung; Foster Kerrison; Ron La Due Lake; Jaclyn
Nelson; Jeff Tessin; Erik Wilkins-McKee; Michael Willems; and Delia
Zee made key contributions to this report.
[End of enclosure]
Footnotes:
[1] Pub. L. No. 110-417 (2008).
[2] DOD defines organic depot activities as government-owned and
government-operated activities that perform depot-level maintenance
that are assigned to or part of a military service.
[3] Obligations funding data are in constant fiscal year 2009 dollars.
[4] H.R. Rep. No. 110-652, at 333 (2008).
[5] Pub. L No. 110-417, § 322.
[6] Reset refers to actions taken to repair, enhance, or replace
military equipment used in support of operations under way, and
associated sustainment.
[7] GA0 identified frequently occurring, generally accepted research
standards that are relevant for studies and define a sound and
complete study as part of a 2006 review. The standards drew from
several sources, including prior GAO work and external organizations
such as the RAND Corporation. See GAO, Defense Transportation: Study
Limitations Raise Questions about the Adequacy and Completeness of the
Mobility Capabilities Study and Report, [hyperlink,
http://www.gao.gov/products/GAO-06-936] (Washington, D.C.: Sept 20,
2006); and Defense Management DOD Needs to Monitor and Assess
Corrective Actions Resulting from Its Corrosion Study of the F-35
Joint Strike Fighter, [hyperlink,
http://www.gao.gov/products/GAO-11-171R] (Washington, D.C.: Dec. 16,
2010).
[8] We considered subissues to be addressed if the LMI reports
discussed all topics specified within each subissue or included
recommendations that were specified in the legislation, and partially
addressed if the reports discussed some but not all of the specified
topics.
[9] Major organic depot maintenance activities are defined as those
with more than 400 DOD civilian and U.S. uniformed military employees
engaged in depot-level maintenance operations.
[10] Core refers to a depot maintenance capability that is government-
owned and government-operated (including government personnel and
government-owned and government-operated equipment and facilities) to
ensure a ready and controlled source of technical competence and
resources necessary for effective and 13 timely response to a
mobilization, national defense contingencies, or other emergency
requirements (10 U.S.C. § 2464).
[11] Condition-based maintenance includes proactive maintenance tasks
to predict or prevent equipment failures. It is different from
reliability-based maintenance which collects and analyzes data on the
function and performance of specific equipment in order to determine
the maintenance approach needed to keep the equipment functioning
effectively and prevent future failures.
[12] Section 2466(a) of Title 10 of the United States Code states that
subject to certain exceptions, not more than 50 percent of funds made
available in a fiscal year to a military department or defense agency
for depot-level maintenance and repair may be used to contract for the
performance by nonfederal government personnel of such workload for the
military department or agency.
[13] Total ownership cost includes the costs to develop, procure,
operate, maintain, and dispose of a weapon system. Repair cycle time
is the average number of days that is required to repair a weapon
system or a major component.
[14] For the current review, we evaluated whether the evidence for
each relevant subquestion had limited or no concerns, the evidence had
concerns, or we could not determine the extent of limitations or
concerns. Concerns may raise questions about the adequacy or
completeness of the study.
[15] Each of the three general standards has a number of subquestions
(see [hyperlink, http://www.gao.gov/products/GA0-06-938] and
[hyperlink, http://www.gao.gov/products/GAO-11-171R]). However, not
all of these are applicable to every study. For example, some elements
apply to scenarios, threats, modeling, and assumptions and may be
relevant to studies that make future projections or estimates.
Similarly, some elements apply to the verification and validation of
data and may be relevant to studies that rely on quantitative analyses
as a basis for findings.
[16] The Maintenance Executive Steering Committee consists of senior
maintenance and logistics representatives throughout DOD and is
intended to serve as a mechanism for the coordinated review of DOD
maintenance policies, systems, programs, and activities.
[17] The LMI study team made these assumptions based on publicly
available information such as the Sustainable Defense Task Force
report, Debts, Deficits, and Defense: A Way Forward (June 11, 2010);
and the Washington Headquarters Services, Directorate for information
Operations and Report document. Active Duty Military Personnel
Strengths by Regional Area and by Country (Sept. 30. 2009).
[18] The International Organization for Standardization (ISO) is a
nongovernmental organization that develops standards for use in the
public and private sectors. ISO 9001 provides guidance on establishing
a quality management system that provides confidence in an
organization's ability to provide products that fulfill customer needs
and expectations.
[19] The study team could have improved the accuracy of its models by
presenting a range of predictions (based on the observed prediction
error). Larger samples and a wider range of variables also might have
improved the accuracy of the models predictions.
[20] Providing sufficient detail in a report so that a reader may
independently replicate the analysis is a common criterion for
documenting statistical or scientific research.
[21] The final report presented two vulnerabilities that could affect
the future viability of the organic depots: (1) a significant
reduction in future depot maintenance requirements, and (2) reductions
in near-term depot maintenance work and core sustaining depot
maintenance workloads.
[22] Milestone B marks the completion of the Technology Development
phase and the beginning of the Engineering and Manufacturing
Development phase of the acquisition process.
[23] In order to reach Milestone A, the point at which a program
enters the technology development phase, among other things an
analysis of alternatives is conducted and a technology development
strategy is created. We have previously recommended that DOD require
an initial core assessment early in the acquisition process. DOD
concurred with our recommendation and stated that it would revise
applicable guidance to provide more specificity on how to identify and
establish core capability during the acquisition process. See GAO,
Depot Maintenance: Actions Needed to Identify and Establish Core
Capability at Military Depots, GAO-09-83 (Washington, D.C.: May 14,
2009).
[24] We previously recommended that DOD improve core depot maintenance
policies including revising depot maintenance core policy to include a
forward look to incorporate future systems and equipment repair needs
when developing core capability requirements and a direct link to the
source-of-repair process. DOD implemented a revised core policy that
addressed our recommendation. See GAO, Defense Logistics: Actions
Needed to Overcome Capability Gaps in the Public Depot System,
[hyperlink, http://www.gao.gov/products/GA0-02-105] (Washington, D.C.:
Oct. 12, 2001).
[25] We previously recommended that DOD improve its ability to assess
core logistics capabilities with respect to fielded systems and
correct any identified shortfalls. DOD concurred with our
recommendations and planned to revise guidance on the core
determination process. See [hyperlink,
http://www.gao.gov/products/GAO-09-83].
[26] For prior studies identified by LMI, see Appendix B of LMI, Inc.,
Future Capability of DOD Maintenance Depots, LG901 M2 (McLean, Va.:
February 2011).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
