Defense Logistics
Observations on Army's Implementation of the Logistics Modernization Program Gao ID: GAO-09-852R July 8, 2009This letter formally transmits the briefing in response to House Report No. 110-652, which accompanied the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 (Pub. L. No. 110-417). The House report directed the Secretary of the Army to submit a report on Logistics Modernization Program (LMP) implementation at Army depots and the expected end-state capabilities of LMP to the Senate Committee on Armed Services, the House Committee on Armed Services, and GAO by January 31, 2009. The House report also directed us to review the report submitted by the Secretary of the Army for completeness and to provide a report to the Senate Committee on Armed Services and the House Committee on Armed Services by March 31, 2009.
GAO-09-852R, Defense Logistics: Observations on Army's Implementation of the Logistics Modernization Program
This is the accessible text file for GAO report number GAO-09-852R
entitled 'Defense Logistics: Observations on Army's Implementation of
the Logistics Modernization Program' which was released on July 8,
2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-09-852R:
United States Government Accountability Office:
Washington, DC 20548:
July 8, 2009:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives:
Subject: Defense Logistics: Observations on Army's Implementation of
the Logistics Modernization Program:
This letter formally transmits the enclosed briefing in response to
House Report No. 110-652, which accompanied the Duncan Hunter National
Defense Authorization Act for Fiscal Year 2009 (Pub. L. No. 110-417).
The House report directed the Secretary of the Army to submit a report
on Logistics Modernization Program (LMP) implementation at Army depots
and the expected end-state capabilities of LMP to the Senate Committee
on Armed Services, the House Committee on Armed Services, and GAO by
January 31, 2009. The House report also directed us to review the
report submitted by the Secretary of the Army for completeness and to
provide a report to the Senate Committee on Armed Services and the
House Committee on Armed Services by March 31, 2009. On March 31, 2009,
we provided the enclosed briefing to staff of your committees to
satisfy this mandate.
We are sending copies of this report to the appropriate congressional
committees. We are also sending copies to the Secretary of Defense; the
Deputy Secretary of Defense; the Under Secretary of Defense for
Acquisition, Technology and Logistics; and the Secretary of the Army.
This report will also be available at no charge on our Web site at
[hyperlink, http://www.gao.gov].
Please contact William M. Solis at (202) 512-8365 or solisw@gao.gov,
Asif A. Khan at (202) 512-9859 or khana@gao.gov, or Nabajyoti Barkakati
at (202) 512-4499 or barkakatin@gao.gov if you or your staff have
questions on matters discussed in this report. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report were J.
Chris Martin, Senior Technologist; David Schmitt, Assistant Director;
Evelyn Logue; Darby Smith; and Jim Melton.
Signed by:
William M. Solis:
Director, Defense Capabilities and Management:
Signed by:
Asif A. Khan:
Director, Financial Management and Assurance:
Signed by:
Nabajyoti Barkakati:
Chief Technologist:
Applied Research and Methods:
Center for Technology and Engineering:
Enclosure:
[End of section]
Enclosure:
Army‘s Implementation of the Logistics Modernization Program:
Congressional Briefing:
March 31, 2009:
Introduction:
In February 1998, the Army Materiel Command began the Logistics
Modernization Program (LMP) to replace its existing material
management systems”the Commodity Command Standard System and the
Standard Depot System. These systems have been in operation for more
than 30 years.
LMP is intended to transform logistics operations in six core
processes: order fulfillment, demand and supply planning, procurement,
asset management, material management, and financial management.
When fully implemented, LMP is expected to manage goods and services
valued at about $40 billion, including 6 million Army inventory items.
GAO has previously reported on problems the Army has encountered in its
efforts to implement LMP.[Footnote 1] The specific problems related to:
* data conversion errors that resulted in LMP containing inaccurate
unit prices or units of issue for certain materials;
* billing and collection errors whereby some customers were not billed
for costs that should have been included in their bills and other
customers were overbilled;
* requirements management and testing process did not comply with
best practices, and;
* a lack of an independent verification and validation process.
House Report 110-652 directed the Secretary of the Army to report to
the House and Senate Armed Services Committees and GAO by January 31,
2009, on the Army‘s efforts to implement LMP. The Army submitted its
report to the House and Senate Armed Services Committees and GAO on
October 21, 2008. In addition, the House report:
* stipulated that the Army‘s report on LMP address nine specific areas
and;
* directed GAO to review the Army‘s report for completeness and to
report to the House and Senate Armed Services Committees by March 31,
2009.
The nine specific areas that the Army was to report upon were:
1) expected LMP capabilities at the levels of depot production,
business operations and financial management, and Headquarters Army
Materiel Command;
2) specific LMP capabilities implemented at each depot;
3) date of expected implementation at each depot;
4) description of how LMP will forecast future maintenance capacity and
drive budgetary decisions;
5) percentage of workforce at each depot expected to be proficient on
the system;
6) strategy to educate and train depot employees on system capabilities
and the new business approach to resource planning and supply chain
management as a result of LMP implementation;
7) detailed plan for ensuring that 100 percent of each depot‘s
operating files are loaded by the planned date of implementation at
each depot;
8) leveraging of lessons learned from previous implementations; and;
9) detailed risk-mitigation strategy to support current production in
the event that LMP implementation is not as successful as planned.
Objectives:
Our objectives were as follows:
1. Determine the extent to which the Army‘s report addressed the nine
specific areas identified in the House committee report.
2. Ascertain the specific actions the Army has taken, has under way, or
has planned to address GAO‘s prior recommendations pertaining to the
Army‘s implementation of LMP.
3. Evaluate the management processes and controls that the Army
established to assess the feasibility of future LMP deployments.
Summary of Observations:
Our review and analysis of the Army‘s October 21, 2008, report to the
House and Senate Armed Services Committees found that the report
provided the required information to address each of the nine specified
areas and thereby satisfied the reporting requirement. (See page 10.)
The Army has begun to implement virtually all of GAO‘s prior
recommendations related to LMP to address issues related to data
conversion, billing and collection, requirements management and
testing, and independent verification and validation. The effective
implementation of these recommendations should help the Army to reduce
its risk to acceptable levels. (See pages 11-15.)
The Army has management processes and controls in place to assess the
feasibility of future LMP deployments, enabling it to understand the
risks associated with making a deployment decision and evaluate whether
those risks are acceptable. (See pages 16-28.)
* The Army recognizes that deploying a system such as LMP can never be
a risk-free proposition and that problems will arise after deployment
regardless of the actions taken prior to deployment.
- Ability to go back to the legacy systems if needed.
* The Army has a risk management approach in place to minimize the
effects of those problems to provide reasonable assurance that depot
operations in support of the warfighter are not significantly
disrupted.
- Enhanced help desk support to understand the ’root cause“ of problems
identified by users and help those users immediately where possible.
Background:
The Army began efforts to implement LMP in 1999, with the intent of
having the system fully deployed by June 2004.
* LMP began operations in July 2003 at Tobyhanna Army Depot, the
Communications-Electronics Command, the Army Materiel Command, and the
Defense Finance and Accounting Service. No additional sites have been
added since that time.
* The Army will decide in April 2009 whether to have additional
deployments of LMP.
As of September 2008, the Army has obligated approximately $880 million
for the development and implementation of LMP.
Objective 1 Observations: Mandated Army Report Provided Required
Information:
Our review and analysis of the Army‘s October 21, 2008, report to the
House and Senate Armed Services Committees found that the report
provided the required information to address each of the nine specified
areas and thereby satisfied the reporting requirement.
The Army‘s report provided details on (1) expected LMP capabilities,
(2) LMP capabilities implemented at depots, (3) date of expected
implementation, (4) how LMP will forecast maintenance capacity and
drive budgetary decisions, (5) percentage of workforce expected to be
proficient, (6) strategy to educate employees on LMP capabilities, (7)
a plan to ensure the depot‘s operating files are loaded into LMP, (8)
leveraging of lessons learned, and (9) a detailed risk-mitigation
strategy in the event that LMP implementation is not successful.
Objective 2 Observations: Prior GAO Recommendations:
The Army has acted to implement virtually all of GAO‘s prior
recommendations related to LMP. The effective implementation of these
recommendations should help the Army reduce its risk to acceptable
levels.
GAO made recommendations regarding problems we identified in the
following areas:
* Data conversion;
* Billing and collection;
* Requirements management and testing;
* Independent verification and validation.
Data conversion:
* We previously reported that during the first deployment, LMP contained
inaccurate unit price or unit of issue data for certain materials,
resulting in excess material being ordered and incorrect prices being
charged. Because of the inaccurate data, the Army received quantities
of parts and supplies for use in repairing military assets that were
far greater than intended.
* When Tobyhanna converted from its legacy system to LMP in July 2003,
the June 30, 2003, ending account balances in the legacy system did not
reconcile to the beginning account balances in LMP. Accurate account
balances are essential for financial reporting and preparation of
future budgets.
* The Army has since implemented a program to address the data
conversion problems caused during the first deployment, and also has a
process to evaluate whether its data conversion activities have reduced
the risks associated with this critical factor to acceptable levels.
Billing and collection:
* We previously reported that shortly after LMP was implemented,
Tobyhanna Army Depot began experiencing difficulty in accurately
billing customers and recognizing revenue.
* LMP‘s billing problems resulted in (1) customers not being billed for
costs incurred that should have been billed and (2) customers being
billed too much.
- As noted in a prior GAO report (GAO-07-860), 146 customer orders
valued at approximately $5.4 million were not billed during the January
31, 2007, billing cycle.
- Similarly, during the January 31, 2007, billing cycle about 308
customer orders amounting to about $5.8 million were shown as being
over billed, and a credit should have been issued to the customer.
- It is essential that revenue is properly recognized and customers are
billed for work performed because these are the means by which the
depots finance their day-to-day operations. Moreover, credits from the
depots to the customers can be used by the customers to meet other
funding demands.
* In addition to improving its data conversion processes, the Army
implemented technical and process improvements in December 2007. The
Army reported that these improvements reduced the number of incorrect
bills from 40 per cycle to zero. Because of time constraints, we did
not independently verify these reported results.
Requirements management and testing:
* We previously reported that the Army did not have an adequate process
for managing LMP requirements that complied with best practices. In
2007, we reported (in GAO-07-860) that the Army revised and improved
this process.
* We previously reported that the Army did not have an adequate testing
process for LMP because the tests conducted by the Army were not tied
to specific requirements, were not ’end-to-end,“ and were not performed
by users of the system and because test results were neither properly
documented nor verified and validated by an independent reviewer in
accordance with best practices.
* The Army has since revised its testing process, which we were unable
to fully assess because of time constraints. However, we found that (1)
users were conducting the test; (2) detailed test scripts were being
used; (3) an automated system was capturing test results; and (4) test
results were being evaluated daily by a group that consisted of users,
the LMP program office, and an independent contractor, which increased
the assurance that identified problems were properly evaluated.
Independent verification and validation:
* The Army has implemented an independent verification and validation
process, which should help to identify to management the issues or
weaknesses that increase the risks associated with the project.
Objective 3 Observations: Improved Management Processes:
The Army has implemented critical project management processes and
controls to identify and understand the risks associated with making a
deployment decision and evaluate whether those risks are acceptable
before deploying LMP at Corpus Christi, Letterkenny, and Redstone
Arsenal. These key processes were absent during the first deployment at
Tobyhanna.
* Improved training to address human capital needs;
* Data conversion processes developed to provide reasonable assurance
that critical data are properly converted;
* Quantitative metrics to evaluate the effectiveness of the revised
project management process have been developed;
* ’Go Live“ processes developed to rapidly evaluate and respond to
potential issues that will invariably arise during the second
deployment.
Figure: Employee Training Hours as of March 26, 2009:
[Refer to PDF for image: stacked vertical bar graph]
Actual:
Corpus Christ Army Depot: 24,280 hours;
Letterkenny Arsenal: 14,832 hours;
Redstone Arsenal: 37,120 hours;
Total: 76,232 hours.
Future:
Corpus Christ Army Depot: 20,556 hours;
Letterkenny Arsenal: 19,704 hours;
Redstone Arsenal: 71,592 hours;
Total: 111,852 hours.
Total:
Corpus Christ Army Depot: 44,836 hours;
Letterkenny Arsenal: 34,536 hours;
Redstone Arsenal: 108,712 hours;
Total: 188,084 hours.
Source: GAO analysis of Army data.
Notes: On March 19, 2009, LMP estimated that its total training hours
would be greater than the amount shown in the above chart. This
difference was caused by LMP reevaluating its course offerings and
employee training assignments, primarily at Corpus Christi Army Depot.
The LMP project office expects that the revised data will be loaded
into the training system during the week ending April 3, 2009, and that
the total number of employee training hours associated with the second
deployment will be about 195,000 employee training hours.
[End of figure]
The majority of training (about 92 percent of the total employee hours)
is expected to be delivered prior to deployment.
Some training (e.g., budget stratification reporting) will occur at
Redstone after deployment since the users will not be performing the
functions designed to be addressed by the training until deployment has
occurred, and Army officials explained that it is more effective to
train an individual when he or she is close to performing the expected
function.
Training process includes a means to evaluate the effectiveness of the
training program in delivering the expected skills.
* Participants are tested and must achieve a 70 percent score to be
considered as passing the class.
* If a person does not achieve a 70 percent score, he or she may take
other actions, such as taking the class again to achieve a passing
score.
- One-on-one help is also offered if the participant continues to
experience problems.
* Participants also provide feedback on whether they believe that the
course has helped them do their jobs.
- According to the Army, overall the participants have rated the
training program as a 4.1 out of 5, with 5 being the highest score.
LMP has conducted three data migration trial loads to identify issues
associated with migrating data from the legacy systems to LMP.
* At the end of February 2009, more than 95 percent, or about 12.5
million of the 13.1 million records, could be loaded into LMP without
issue.
* As of March 27, 2009, about 96 percent of the records loaded into LMP
successfully processed without issue. For the remaining records, Army
officials stated that they will either resolve the issues before
deployment or take actions to mitigate any adverse effects.
The Army recognizes that not all legacy data will be successfully
loaded into LMP prior to deployment.
* Historical data are needed for planning or other purposes.
- A data warehouse is being created that contains the data in the legacy
system that are not being converted--this is the same approach used at
the initial deployment site and it has served the mission needs.
- The legacy systems will remain available in an inactive state as long
as the Army considers it necessary.
* A review was made to determine which records were associated with
work that is not expected to be performed at deployment locations
during the next 3 years, and data conversion of these records will
occur after deployment, if the need arises.
* Data needed to support various LMP processes (commonly referred to
as master data) are being carefully reviewed.
- Preventive measures have been implemented to ensure that all master
data items being loaded into the system have correct base unit of
measure (BUOM) values.
-- In the initial deployment, items were loaded into LMP with incorrect
BUOM values. For example, part numbers were loaded into the system with
a BUOM with a qualitative measure such as ’box“ rather than a
quantitative measure such as ’each.“
- The Army developed a manual work-around process to provide reasonable
assurance that items already contained in LMP with incorrect BUOM will
not cause problems at the second deployment locations.
-– LMP currently has about 329 items in its records that contain
incorrect BUOM values. These errors caused reordering and billing
problems at Tobyhanna:
121 of these items are used by Corpus Christi and Letterkenny;
72 are only used by Letterkenny;
25 are only used by Corpus Christi.
-– Manual work-around processes developed at Tobyhanna have addressed
the problems and will be used at the second deployment locations until
a long-term system fix can be implemented.
Objective 3 Observations: Improved Management Processes–Quantitative
Metrics:
The Army has defined the critical business processes that must be
completed prior to making a decision on whether the deployment should
’Go Live,“ and has developed quantitative metrics to evaluate whether
this has occurred.
Examples of quantitative measures that are used include the
following:
* Validate that the critical business process testing has been
completed and that any problems that have not been addressed have
agreed-upon mitigation strategies.
* Validate that the expert users can successfully execute the
applicable processes--100 percent of the selected expert users are able
to successfully execute their respective tasks.
* Validate that end users have completed 90 percent of the required
training courses.
* Validate that the overall data accuracy rate is 95 percent, with each
deployment location having accuracy of at least 90 percent.
* Validate the interoperability, availability, and responsiveness of
LMP.
– Failures associated with exchanging data with other systems are less
than 2 percent, with an objective of 1 percent;
– Availability is at least 99.5 percent;
– Average response time is less than 2 seconds for 80 percent of the
transactions, with an objective of 2 seconds or less for 95 percent of
the transactions.
Objective 3 Observations: Improved Management Processes –’Post-Go Live“
Processes:
The Army recognizes that deploying a system such as LMP can never be a
risk-free proposition and that problems will arise after deployment
regardless of the actions taken prior to deployment. The Army has
implemented processes designed to reduce the adverse impacts associated
with those problems to acceptable levels.
* End users experiencing problems will contact expert users at their
location to help determine whether they are experiencing system
problems or whether the users need help understanding how to perform a
given task.
- Help tickets are generated, regardless of the reason, to help the
Army determine if additional training is needed or if the training
needs to be modified.
- When the training is modified, all users who have taken the course
are notified of the changes.
The Army has enhanced its help desk support to understand the ’root
cause“ of problems identified by users and help those users immediately
where possible. The Army is following best practices and will use a
three-tier help desk system to address problems that cannot be
addressed by the expert users at the deployment locations.
* Tier 1:
- Provides a single point of contact for end users;
- Gathers the necessary data to assess the reported problem;
- Determines if the problem can be resolved or escalates the problem to
the next level; for example, the Tier 1 Help Desk (1) may be aware of a
specific problem and can provide the user with the ’work-around“ that
has been developed to address the problem or (2) can determine whether
the system is operating as designed and provide the user with
instructions on how to accomplish a specific transaction;
- Verifies user satisfaction with the help desk process.
* Tier 2 has more knowledgeable support personnel and determines if the
problem can be resolved or escalates the problem to the next level.
- May recommend changes to LMP based on the change management processes
used by the program.
* Tier 3 support is used for complex problems that cannot be addressed
by the Tier 2 support or problems associated with vendor-supplied
hardware and software.
Agency Views:
We provided a draft of these briefing slides to the Departments of
Defense and the Army for their official review and comments.
On March 30, 2009, we met with officials from the Defense Business
Transformation Agency, the Office of the Secretary of Defense for
Supply Policy, the LMP Program Office, and the Department of the Army to
discuss our observations and obtain their comments on the briefing
slides. The participants agreed with our observations and the
information in the briefing slides.
Methodology:
To determine if the Army‘s report addressed the nine specific areas
identified in the House committee report, we reviewed the Army‘s report
and supporting documentation and compared them against the
requirements.
To ascertain the specific actions that the Army has taken to address
GAO‘s prior recommendations on the implementation of LMP, we reviewed
past GAO reports related to LMP implementation. We also met with Army
officials at the LMP Program Office to discuss the actions they either
have implemented or have under way to address each recommendation, and
we reviewed Army plans and policies that govern implementation. We also
traveled to an Army depot to determine if the actions were being taken.
To evaluate the management processes and controls that the Army has in
place to ascertain if LMP should be deployed, we met with Army
officials at the LMP Program Office and Corpus Christi Army Depot to
determine the processes and controls they have in place. We also
observed a training class being conducted for LMP users, and we
analyzed Army data related to training.
We conducted this performance audit from January 2009 to March 2009 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of enclosure]
Footnotes:
[1] GAO, DOD Business Systems Modernization: Billions Continue to Be
Invested with Inadequate Management Oversight and Accountability,
[hyperlink, http://www.gao.gov/products/GAO-04-615] (Washington, D.C.:
May 27, 2004); Army Depot Maintenance: Ineffective Oversight of Depot
Maintenance Operations and System Implementation Efforts, [hyperlink,
http://www.gao.gov/products/GAO-05-441] (Washington, D.C.: June 30,
2005); and DOD Business Transformation: Lack of an Integrated
Strategy Puts the Army‘s Asset Visibility System Investments at Risk,
[hyperlink, http://www.gao.gov/products/GAO-07-860] (Washington, D.C.:
July 27, 2007).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
