DOD Business Transformation
Lack of an Integrated Strategy Puts the Army's Asset Visibility System Investments at Risk Gao ID: GAO-07-860 July 27, 2007The Department of Defense (DOD) established a goal to achieve total asset visibility (TAV) over 30 years ago, but to date it has been unsuccessful. GAO was requested to (1) determine whether the Army has a systems strategy for achieving TAV, (2) determine if the Army's business system investment governance structure is consistent with DOD guidance, and (3) evaluate the Army's effort to correct previously reported problems with the Logistics Modernization Program (LMP). GAO obtained an understanding of the Army's efforts to achieve TAV, oversee and manage its business system investments, and address previously reported LMP problems.
Supply chain management has been on GAO's high-risk list since 1990. One area that has contributed to this long-standing problem has been DOD's inability to maintain control and accountability over hundreds of billions of dollars of assets. DOD plans to improve its asset management through its business system modernization. In this regard, GFEBS, the Global Combat Support System-Army (GCSS-Army), and LMP are aimed at achieving TAV within the Army. The Army estimates that it will invest approximately $5 billion to develop and implement these systems. However, this investment is being made without a clear integrated strategy. GFEBS, GCSS-Army, and LMP are not being developed in the context of a well-defined Army-wide enterprise architecture. As a result, the Army does not have an informed basis for determining if these systems will fit within the context of future Army business operations and will efficiently and effectively address the Army's long-standing weaknesses associated with the lack of asset visibility. The Army lacks a concept of operations that would describe, at a high level, (1) how the three business systems relate to each other in achieving the Army's TAV goal, and (2) how information flows from and through these systems. Moreover, GAO found that the Army's lack of a concept of operations has contributed to its failure to take full advantage of business process reengineering opportunities that are available when using an enterprise resource planning solution. Without these key foundational elements, the Army is at risk of investing about $5 billion in business systems and still not achieving DOD's and the Army's goal of TAV. Furthermore, while the Army has established a governance structure that is consistent with DOD guidance, its processes are still maturing. The Army's governance structure is designed to certify and review individual business systems rather than to evaluate these investments from a portfolio perspective. Such a perspective permits investments to be viewed in a comprehensive manner to help ensure that the organization's missions and objectives are achieved. GAO also found that the Army did not have reliable processes and analyses, such as an independent validation and verification function or economic analyses, to support its oversight of individual business systems. Until the Army's investment processes mature, it runs the risk of investing in business systems that do not provide the desired functionality and efficiency. Additionally, LMP continues to be plagued by problems that have beset the system since its implementation in July 2003. LMP continues to experience problems with accurately recognizing revenue and billing customers, which can, in part, be attributed to ineffective system testing.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-860, DOD Business Transformation: Lack of an Integrated Strategy Puts the Army's Asset Visibility System Investments at Risk
This is the accessible text file for GAO report number GAO-07-860
entitled 'DOD Business Transformation: Lack of an Integrated Strategy
Puts the Army's Asset Visibility System Investments at Risk' which was
released on July 27, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on Readiness and Management Support,
Committee on Armed Services, U.S. Senate:
United States Government Accountability Office:
GAO:
July 2007:
DOD Business Transformation:
Lack of an Integrated Strategy Puts the Army's Asset Visibility System
Investments at Risk:
GAO-07-860:
GAO Highlights:
Highlights of GAO-07-860, a report to
the Subcommittee on Readiness and Management Support, Committee on
Armed Services, U.S. Senate
Why GAO Did This Study:
The Department of Defense (DOD) established a goal to achieve total
asset visibility (TAV) over 30 years ago, but to date it has been
unsuccessful. GAO was requested to (1) determine whether the Army has a
systems strategy for achieving TAV, (2) determine if the Army‘s
business system investment governance structure is consistent with DOD
guidance, and (3) evaluate the Army‘s effort to correct previously
reported problems with the Logistics Modernization Program (LMP). GAO
obtained an understanding of the Army‘s efforts to achieve TAV, oversee
and manage its business system investments, and address previously
reported LMP problems.
What GAO Found:
Supply chain management has been on GAO‘s high-risk list since 1990.
One area that has contributed to this long-standing problem has been
DOD‘s inability to maintain control and accountability over hundreds of
billions of dollars of assets. DOD plans to improve its asset
management through its business system modernization. In this regard,
GFEBS, the Global Combat Support System-Army (GCSS-Army), and LMP are
aimed at achieving TAV within the Army. The Army estimates that it will
invest approximately $5 billion to develop and implement these systems.
However, this investment is being made without a clear integrated
strategy.
* GFEBS, GCSS-Army, and LMP are not being developed in the context of a
well-defined Army-wide enterprise architecture. As a result, the Army
does not have an informed basis for determining if these systems will
fit within the context of future Army business operations and will
efficiently and effectively address the Army‘s long-standing weaknesses
associated with the lack of asset visibility.
* The Army lacks a concept of operations that would describe, at a high
level, (1) how the three business systems relate to each other in
achieving the Army‘s TAV goal, and (2) how information flows from and
through these systems. Moreover, GAO found that the Army's lack of a
concept of operations has contributed to its failure to take full
advantage of business process reengineering opportunities that are
available when using an enterprise resource planning solution.
Without these key foundational elements, the Army is at risk of
investing about $5 billion in business systems and still not achieving
DOD‘s and the Army‘s goal of TAV.
Furthermore, while the Army has established a governance structure that
is consistent with DOD guidance, its processes are still maturing. The
Army‘s governance structure is designed to certify and review
individual business systems rather than to evaluate these investments
from a portfolio perspective. Such a perspective permits investments to
be viewed in a comprehensive manner to help ensure that the
organization‘s missions and objectives are achieved. GAO also found
that the Army did not have reliable processes and analyses, such as an
independent validation and verification function or economic analyses,
to support its oversight of individual business systems. Until the
Army's investment processes mature, it runs the risk of investing in
business systems that do not provide the desired functionality and
efficiency.
Additionally, LMP continues to be plagued by problems that have beset
the system since its implementation in July 2003. LMP continues to
experience problems with accurately recognizing revenue and billing
customers, which can, in part, be attributed to ineffective system
testing.
What GAO Recommends:
GAO makes five recommendations to DOD and the Army: (1) develop a
concept of operations for the Army; (2) develop policies, procedures,
and processes to manage investments from a portfolio perspective; (3)
establish an independent verification and validation function; (4)
require that any future General Fund Enterprise Business System (GFEBS)
economic analysis is prepared in accordance with applicable policies;
and (5) direct that LMP use an independent system test team. Overall,
DOD concurred with the recommendations and stated that it will work
diligently to close them.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-860].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact McCoy Williams at (202)
512-9095 or Keith Rhodes at (202) 512-6412.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Army Lacks an Integrated Strategy for Achieving TAV:
Army's Ability to Effectively Oversee Portfolios of Business Systems
Investment Is Not Yet Fully Developed:
LMP Illustrates Continuing Problems in Implementing Business Systems on
Time, within Budget, and with the Promised Capability:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: Army Strategy Does Not Fully Utilize Capabilities of ERP
Solution:
Appendix IV: LMP Problems Continue:
Appendix V: GAO Contacts and Staff Acknowledgments:
Tables:
Table 1: Unbilled and Credit Customer Orders at the Tobyhanna Army
Depot for November 2006 through January 2007:
Table 2: DFAS Billings to the Tobyhanna Army Depot for Accounting
Services:
Figures:
Figure 1: LMP Time Line:
Figure 2: GCSS-Army Time Line:
Figure 3: GFEBS Time Line:
Figure 4: Overview of Tiered Accountability for Army Business System
Investments:
Figure 5: Example of an ERP Vision for Accountability of the Army's
PP&E:
Figure 6: Planned GFEBS and GCSS-Army Fund Control Process:
Figure 7: GCSS-Army Funds Control Process Utilizing ERP Capabilities:
Abbreviations:
BEA: business enterprise architecture:
BTA: Business Transformation Agency:
CIO: Chief Information Officer:
COTS: commercial off-the-shelf:
CSC: Computer Sciences Corporation:
DBSMC: Defense Business Systems Management Committee:
DFAS: Defense Finance and Accounting Service:
DOD: Department of Defense:
DPAS: Defense Property Accountability System:
EA: enterprise architecture:
EAMMF: Enterprise Architecture Management Maturity Framework:
ERP: enterprise resource planning:
FFMIA: Federal Financial Management Improvement Act:
FOC: full operational capability:
GCSS-Army: Global Combat Support System-Army:
GFEBS: General Fund Enterprise Business System:
IRB: investment review board:
IT: information technology:
IV&V: independent verification and validation:
LMP: Logistics Modernization Program:
MAIS: major automated information system:
MDAP: major defense acquisition program:
OMB: Office of Management and Budget:
PBUSE: Property Book Unit Supply Enhanced:
PCA: Pre-Certification Authority:
PLM+: Product Lifecycle Management Plus:
PP&E: property, plant, and equipment:
SALE: Single Army Logistics Enterprise:
TAV: total asset visibility:
V&V: verification and validation:
United States Government Accountability Office:
Washington, DC 20548:
July 27, 2007:
The Honorable Daniel K. Akaka:
Chairman:
The Honorable John Ensign:
Ranking Member:
Subcommittee on Readiness and Management Support:
Committee on Armed Services:
United States Senate:
The Department of Defense (DOD) has continually struggled to achieve
and maintain efficient and effective management over the hundreds of
billions of dollars it has invested in tangible assets, including
inventory, supplies, and materials (inventory and related property) and
property, plant, and equipment (PP&E).[Footnote 1] DOD was responsible
for almost 72 percent ($697 billion) of the total $970 billion reported
governmentwide value for these assets, as of September 30,
2006.[Footnote 2] The nature and severity of DOD's financial and
business management system deficiencies impede the ability of DOD
managers to receive the full range of information needed to effectively
manage day-to-day operations. Of the 27 areas on GAO's high-risk
list,[Footnote 3] DOD has 8 high-risk areas of its own[Footnote 4] and
shares responsibility for 7 governmentwide high-risk areas.[Footnote 5]
Visibility over its assets has been a DOD concern for decades. If the
information contained in the asset accountability systems is not
accurate, complete, and timely, DOD's day-to-day operations could be
adversely affected by investing in inventory, for example, that is not
needed to meet current needs. When this occurs, the department may
obligate funds unnecessarily, which could lead to not having sufficient
obligational authority to purchase needed items. In recognition of the
importance of asset accountability to successful operations, the
department established a goal to achieve total asset visibility (TAV)
over 30 years ago. DOD defines TAV as the ability to provide timely and
accurate information on the location, movement, status, and identity of
units, personnel, equipment, and supplies and having the ability to act
on that information. Over the years, the military services and defense
components have undertaken numerous initiatives to achieve TAV. Within
the Army, one such initiative has been the Logistics Modernization
Program (LMP). In May 2004 and June 2005, we reported that LMP was not
providing the Army the promised capability.[Footnote 6] DOD's current
estimate for achieving TAV is 2010.
This report provides information in support of your continuing
oversight of DOD's progress towards resolving the department's long-
standing problems in achieving TAV. As you requested, our initial
effort was directed at the Army. In September 2006, the Army reported
inventory and related property of about $57 billion and PP&E over $85
billion.[Footnote 7] The Army has identified three primary system
initiatives directed at achieving TAV within the service: (1) LMP, (2)
Global Combat Support System-Army Field/Tactical[Footnote 8] (GCSS-
Army), and (3) General Fund Enterprise Business System (GFEBS). Our
objectives were to (1) determine whether the Army has developed a
business system strategy for achieving TAV, (2) determine if the Army
has effectively implemented a governance structure to oversee and
manage its business system investments in accordance with DOD guidance,
and (3) evaluate the extent to which the Army has made progress in
correcting the previously reported problems regarding LMP's
implementation.
To address the first objective, we met with Army program office
officials for GFEBS, GCSS-Army, and LMP and obtained briefings on the
intended purpose of each system. In addition, we conducted walkthroughs
and reviewed documentation related to various transactions to obtain an
understanding of how the systems would exchange data and to assess how
the Army intended to use these systems individually and collectively to
achieve TAV. To address the second objective, we reviewed guidance
issued by DOD, the Army, and the Business Transformation Agency (BTA)
related to investment management. We also obtained an understanding of
the Army's business system investment governance structure and process
for ensuring compliance with the certification and annual system review
processes required by the fiscal year 2005 National Defense
Authorization Act.[Footnote 9] To address the third objective, we
interviewed and obtained briefings from LMP program management office
officials and others, and reviewed and analyzed LMP system requirement
and testing documentation to assess the extent to which corrective
actions had been taken or are planned to address our prior
recommendations. We determined that the documentation Army prepared and
submitted through its business system governance process to the
investment review boards (IRBs) and the Defense Business Systems
Management Committee (DBSMC) as a basis for approving individual Army
business system investments was sufficiently reliable for our purposes.
Our work was performed from May 2006 through June 2007 in accordance
with U. S. generally accepted government auditing standards. Details on
our scope and methodology are included in appendix I. We requested
comments on a draft of this report from the Secretary of Defense or his
designee. We received written comments from the Deputy Under Secretary
of Defense (Business Transformation), which are reprinted in appendix
II.
Results in Brief:
The Army's current approach for developing GFEBS, GCSS-Army, and LMP
lacks several elements that are critical to the successful
implementation of integrated business systems, such as an Army-level
enterprise architecture (EA),[Footnote 10] a concept of operations, and
a portfolio-based rather than individual-project-based business system
investment review process. Without these key foundational elements, the
risk that the Army's efforts to achieve TAV will not be successful is
greatly increased. While the Army's efforts to develop these systems
and transform its logistics operations may result in incremental
improvements, without these three essential elements, they are unlikely
to achieve the efficiencies that can be attained through an integrated
business system solution. Instead, if the Army continues on its current
path, it runs the risk of investing significant resources to simply
automate its existing inefficient business processes using more current
technology.
As it now stands, the Army plans to invest about $5 billion over the
next several years to develop and implement GFEBS, GCSS-Army, and LMP
without the benefit of a well-defined Army EA. We reported in August
2006,[Footnote 11] that the Army was in the initial stages of
developing an EA. As of May 2007, this was still the case.[Footnote 12]
A well-defined EA is an essential tool for leveraging information
technology (IT) in the transformation of business and mission
operations. Our experience with federal departments and agencies has
shown that attempting to modernize systems without an EA to guide and
constrain investments often results in operations and systems that are
duplicative, not well integrated, unnecessarily costly to maintain and
interface, and ineffective in supporting mission goals.[Footnote 13]
Moreover, the development, implementation, and maintenance of an EA are
widely recognized as hallmarks of successful public and private
organizations, and their use is required by the Clinger-Cohen Act of
1996[Footnote 14] and the related guidance from the Office of
Management and Budget (OMB).[Footnote 15]
In addition to the Army's lack of an EA, the Army also lacked a concept
of operations, which outlines the Army's strategy for achieving TAV,
including how the three systems it has identified as key in attaining
TAV will interoperate. A concept of operations would provide the Army a
forum for interchange among stakeholders--such as oversight entities,
program managers, developers, and users--on major technical and
programmatic issues related to achieving TAV. Without a concept of
operations for achieving TAV, the Army is hindered in its ability to
apply an enterprise view in (1) making decisions as to how GFEBS, GCSS-
Army, and LMP will individually and collectively enhance the Army's
asset accountability, including providing TAV; and (2) determining what
changes are needed in its related business processes. Additionally, the
Army's inability to achieve TAV hinders its and DOD's efforts to
resolve the long-standing problems associated with supply chain
management, which has been on our high-risk list since 1990. Asset
visibility is one of the focus issues critical to successfully
addressing this high-risk area. Further, the Army's lack of a concept
of operations has resulted in its failure to take full advantage of
business process reengineering opportunities that are available when
using an enterprise resource planning (ERP)[Footnote 16] solution.
Rather, the Army's existing strategy perpetuates some of the cumbersome
and ineffective business processes that are currently used in its
existing legacy system environment. The benefits of an ERP solution
include streamlining of business processes and elimination of data
redundancy.
Furthermore, while the Army has established a business system
investment management governance structure that is consistent with DOD
guidance, its overall investment management approach is still maturing.
Currently, the Army's investment review process is designed to ensure
the completion of certifications and annual reviews of individual
business systems rather than to evaluate business system investments
from a portfolio management perspective. A portfolio-based perspective
permits an organization to view its business system investments in a
comprehensive manner to help ensure that the organization's missions,
strategic goals, and objectives are achieved. Moreover, we found that
the Army did not have reliable processes or analyses, such as
independent verification and validation functions or economic analyses,
to support its oversight of program management office efforts to
develop and implement business systems. Until the Army adopts a
business system investment management approach that provides for
reviewing groups of systems and making enterprise decisions regarding
how these groups will collectively interoperate to provide a desired
capability, it runs the risk of investing significant resources in
business systems that do not provide the desired functionality and
efficiency.
LMP continues to be plagued by operational problems that have beset the
system virtually since its initial implementation in July 2003. While
from a "big picture" perspective, an EA ,concept of operations, and
effective IT portfolio management are essential elements in an entity's
efforts to transform its operations, it is equally important for the
entity to have the disciplined processes needed to actually implement
individual business systems on time, within budget, and with the
promised capability. As we have previously reported, historically DOD
has had difficulty in accomplishing this goal, and LMP has been no
exception. As of September 2006, the Army reported that it had
obligated approximately $452 million to develop and implement LMP. In
May 2004 and June 2005, we reported on operational issues related to
LMP, for example, the inability to accurately recognize revenue and
bill customers--a problem that continues today. We recommended, and the
Army agreed, that the implementation of LMP should be delayed until the
operational problems we identified were resolved. While the Army is
working to resolve LMP operational issues, we continue to have concerns
about the adequacy of LMP's system testing given that the continuing
problems with LMP can, in part, be attributed to ineffective and
nonindependent system testing. Until an effective LMP testing process
is implemented, the Army will have little assurance that the corrective
actions it takes (1) are properly developed, and (2) do not introduce
additional defects into the system.
We are making five recommendations to the Secretary of Defense to
improve the department's efforts to achieve TAV and further enhance its
efforts to improve its control and accountability over business system
investments. More specifically, we recommend that the Secretary of
Defense (1) develop a concept of operations for the Army; (2) develop
policies, procedures, and processes to manage investments from a
portfolio perspective; (3) establish an independent verification and
validation function; (4) ensure the GFEBS economic analysis update is
prepared in accordance with applicable guidance; and (5) direct that
LMP use an independent system test team.
We received written comments on a draft of this report from the Deputy
Under Secretary of Defense (Business Transformation), which are
reprinted in appendix II. Overall, DOD concurred with our
recommendations and stated that it would work diligently to implement
them. The comments included two sets of specific responses to our
recommendations--one set provided by BTA and another provided by the
Army. In its comments, the Army concurred with each of the
recommendations. BTA stated that it fully agreed with our observations.
BTA, though, partially concurred with all the recommendations on the
basis that they were directed jointly to the Secretary of the Army and
the Director, BTA. BTA's comments noted that it has neither the
authority nor the responsibility to direct the actions of the Army.
We appreciate the department's willingness to address our
recommendations. With regard to BTA's concern, our recommendations do
not direct BTA to oversee or direct the Army. Rather, the
recommendations stated that the specific actions should be undertaken
jointly at the direction of the Secretary of Defense. We continue to
believe that a cooperative and effectively coordinated BTA and Army
approach to addressing our recommendations is most likely to achieve
the fundamental business system transformation necessary to achieve the
department's TAV objective.
Background:
TAV has been elusive within DOD. Timely, reliable information on the
location, quantity, and status of the department's tangible assets
could significantly improve its ability to more efficiently and
effectively deliver needed items to DOD operating forces and thereby
enhance military readiness. The department has recognized the
importance of achieving TAV and included it as part of its overall
business transformation initiative, which includes the development and
implementation of a business enterprise architecture (BEA). The Joint
Chiefs of Staff also identified TAV as one of four fundamental changes
needed to transform the department's logistics operations.[Footnote 17]
One of the primary factors contributing to DOD's inability to provide
management with TAV is DOD's outdated and ineffective management
information system environment.[Footnote 18] The Federal Financial
Management Improvement Act (FFMIA) of 1996[Footnote 19] and other
financial management reform legislation have emphasized the importance
of improving financial management, which necessarily encompasses proper
inventory management, across the federal government. Built upon the
foundation laid by the Chief Financial Officers Act of 1990,[Footnote
20] FFMIA emphasizes the need for agencies to have integrated financial
management systems that can generate timely, accurate, and useful
information to make informed decisions and to ensure accountability on
a continuous basis.[Footnote 21] Lacking such critical information,
government leaders will not be positioned to invest resources, reduce
costs, oversee programs, and hold agency managers accountable for the
manner in which government programs are operated.
Army Initiatives Aimed at Achieving TAV:
To improve control and accountability over its assets, the Army has
embarked on a multisystem integration effort that is intended to
leverage commercial ERP software and processes. This integration is the
focus of the Single Army Logistics Enterprise (SALE) initiative. SALE
is designed to integrate information technology requirements, business
processes, business rules, and data from the Army's logistics,
financial, and acquisition transactions for planning and supporting
warfighting logistics operations. According to the Army, SALE is to
provide a coordinated ERP solution built around two individual
logistics system development efforts: (1) LMP and (2) GCSS-Army. Under
the SALE vision, GCSS-Army and LMP will be integrated into a single
solution to provide an Army-wide logistics environment spanning from
"the factory to foxhole." On the financial side, the Army is developing
GFEBS to provide financial visibility over its assets. Each of these
efforts is described below.
LMP. In February 1998, the U.S. Army Materiel Command began an ERP
effort--LMP--to replace its legacy materiel and maintenance management
systems--the Commodity Command Standard System and the Standard Depot
System--with LMP. The Army has been using the existing legacy systems
for over 30 years. LMP is intended to transform the Army's Working
Capital Fund logistics operations in six core processes: order
fulfillment, demand and supply planning, procurement, asset management,
materiel maintenance, and financial management. LMP became operational
at the U.S. Army Communications and Electronics Command, Fort Monmouth,
New Jersey, and Tobyhanna Army Depot, Tobyhanna, Pennsylvania, in July
2003. The initial deployment of LMP consisted of inventory items such
as electronics; electronic repair components; and communications and
intelligence equipment such as night vision goggles, electronic
components such as circuit boards, and certain munitions such as
guidance systems included in missiles. Figure 1 shows the LMP time line
as of March 2007.
Figure 1: LMP Time Line:
[See PDF for image]
Source: DOD, LMP program office.
[End of figure]
The Communications-Electronics Life Cycle Management Command,[Footnote
22] Tobyhanna Army Depot, and the Defense Finance and Accounting
Service (DFAS) are the primary LMP users. When LMP is fully
implemented, its capacity is expected to include more than 17,000 users
at 149 locations and it will be populated with 6 million Army-managed
inventory items valued at about $40 billion. LMP is scheduled to reach
full operational capability (FOC)[Footnote 23] in fiscal year 2010. As
of September 30, 2006, the Army reported that approximately $452
million had been obligated for this system effort and estimates that it
will invest at least another $895 million in LMP.[Footnote 24]
GCSS-Army. The GCSS-Army program was initiated in 1997 to overcome
duplicative databases, poor asset visibility, and stovepiped
communications between numerous existing Army logistics systems. The
goal of GCSS-Army is to integrate multiple logistic functions by
replacing numerous legacy systems and interfaces. Since the program's
inception, it has undergone several revisions, including a change from
a custom software development to a commercial ERP solution, as well as
a change in the prime contractor. According to Army officials, it
invested approximately $95 million in the previous efforts before
adopting the ERP approach. The existing ERP effort was started in
September 2003 and is currently estimated to reach FOC during fiscal
year 2014. Figure 2 shows the time line for GCSS-Army as of March 2007.
Figure 2: GCSS-Army Time Line:
[See PDF for image]
Source: DOD, GCSS-Army program office.
[End of figure]
GCSS-Army is intended to replace 16 stovepiped, legacy logistics
systems that cover certain types of inventory and PP&E. The system is
intended to be operational at all deployable Army units, and provide
asset visibility for accountable items down to the foxhole. As of
September 30, 2006, the Army reported that it had obligated
approximately $203 million for the ERP version of GCSS-Army.
Additionally, the Army estimates that another $2.1 billion will be
invested in GCSS-Army.
Product Lifecycle Management Plus (PLM+). This initiative is the
technical enabler to integrate LMP and GCSS-Army. PLM+ is the means by
which the Army intends to achieve the SALE vision of integrated
logistics systems. PLM+ is intended to provide a single point of entry
and exit for interfaces to external systems. Furthermore, PLM+ is
intended to eliminate duplicative and costly system interfaces.
Additionally, the PLM+ implementation schedule and related funding are
aligned with that of GCSS-Army. PLM+ does not stand alone as an
independent capability. As of September 2006, the Army reported that
approximately $31 million had been obligated for PLM+.
GFEBS. While LMP is intended to provide financial control for the Army
Working Capital Fund, GFEBS is intended to provide this important
control over all non-working capital fund inventory, including that
which is reported in GCSS-Army. GFEBS is an ERP solution that was
initiated in October 2004 and is intended to serve as the Army's
general ledger system for its general fund accounting.[Footnote 25] As
such, GFEBS is intended to improve the reliability of the Army's
financial information and thereby enhance the Army's management
decision-making process. GFEBS is expected to replace 87 legacy
systems, including the 30-year old Standard Army Finance System. Figure
3 shows the GFEBS time line as of June 2007.
Figure 3: GFEBS Time Line:
[See PDF for image]
Source: DOD, GFEBS program office.
[End of figure]
The Army estimates that GFEBS will reach FOC by 2010. As of September
30, 2006, the Army reported that it had obligated $123 million for the
development of GFEBS. In addition, the Army currently estimates that it
will invest another $1.3 billion to implement GFEBS.
Overview of DOD's Investment Management Practices:
In 2005, DOD adopted a "tiered accountability" approach to improve
control and accountability over the billions of dollars it invests
annually in DOD business systems. Under this approach, executive
leadership for the direction, oversight, and execution of DOD
investments is the responsibility of several entities within DOD and
its components. As shown in figure 4 and described below, the
investment control process begins at the component level and works its
way up through a hierarchy of review and approval authorities,
depending on the size and significance of the investment.[Footnote 26]
Figure 4: Overview of Tiered Accountability for Army Business System
Investments:
[See PDF for image]
Source: Army, DOD.
[End of figure]
At the DOD enterprise level, key entities involved in maintaining
control and accountability over Army business system investments with
systems modernizations over $1 million include the DBSMC, which serves
as the highest ranking governance body for business systems
modernization activities; the Principal Staff Assistants, who serve as
the certification authorities for business system modernizations in
their respective core business missions; the investment review boards
(IRBs), which form the review and decision-making bodies for business
system investments in their respective areas of responsibility and
review each investment for BEA compliance; and BTA, which provides
support to the DBSMC and the IRBs and is responsible for leading and
coordinating business transformation efforts across the department. The
BTA is organized into seven directorates, one of which is the Defense
Business Systems Acquisition Executive--the component acquisition
executive for DOD enterprise-level (DOD-wide) business systems and
initiatives. This directorate is responsible for developing,
coordinating, and integrating DOD enterprise-level projects, programs,
systems, and initiatives--including managing resources such as funding,
personnel, and contracts for assigned systems and programs.
To implement tiered accountability within the Army, the Army designated
its Chief Information Officer (CIO) as the Army's Pre-Certification
Authority (PCA) for certification and annual reviews of business system
investments.[Footnote 27] The PCA is accountable for the component's
business system investments and acts as the component's principal point
of contact for communication with the IRBs. As such, the PCA (1)
validates that the system information for all business systems
modernizations over $1 million is complete and accessible to the IRBs,
(2) reviews development/modernization investments with total cost of $1
million or less, (3) reviews system compliance with DOD's BEA and
enterprise transition plan, (4) verifies the investment's economic
viability analysis, (5) asserts the status and validity of the
investment information by submitting a component precertification
letter to the appropriate IRB for its review, and (6) provides IT
portfolio management policy guidance and oversight of mission area/
domain IT portfolios. Below the Army enterprise level, the Army has
established six functional area domains within its business mission
area.[Footnote 28] The domains are responsible for implementing the IT
portfolio management process developed by DOD and the Army to define
and justify the portfolio of planned IT expenditures consistent with
strategic objectives and operational requirements.
GAO Has Previously Reported LMP Issues:
Our May 2004 report[Footnote 29] pointed out that the Army had not
effectively managed its implementation of LMP. The report noted that
after LMP was deployed in July 2003, operational difficulties at the
Tobyhanna Army Depot resulted in inaccurate financial management
information. More specifically, the depot was not (1) producing
accurate workload planning information, (2) generating accurate
customer bills, and (3) capturing all repair costs, necessary for the
Army to calculate accurate future repair prices. The report also
pointed out that LMP requirements (1) lacked the specific information
necessary to understand the required functionality that was to be
provided, and (2) did not describe how to determine quantitatively,
through testing or other analysis, whether the system would meet the
Army's needs. Subsequently, in June 2005,[Footnote 30] we reported that
the problems with LMP continued to prevent the Tobyhanna Army Depot
from accurately reporting on its financial operations, including gains
and losses, which adversely affected the depot's ability to accurately
set customer sales prices. Further, the report pointed out that
problems persisted with recognizing revenue and billing customers. We
recommended, and the Army agreed, that the implementation of LMP should
be delayed until the operational problems we identified were resolved.
The Army is continuing to resolve the outstanding issues.
Army Lacks an Integrated Strategy for Achieving TAV:
The Army has estimated that it will invest about $5 billion over the
next several years to complete development and implementation of GFEBS,
GCSS-Army, and LMP. However, the Army is making this significant
investment without a clear integrated strategy for how these systems
will be used to achieve TAV over hundreds of billions of dollars of
assets. As we reported in December 2004,[Footnote 31] because DOD has
not developed a clear long-range strategy, the military services will
be exposed to the risk of spending billions of dollars on duplicative,
stovepiped systems that do not support the department's business
transformation goals, including attaining TAV. More specifically, we
found that the Army's current strategy for achieving TAV, including the
implementation of these three systems, does not embrace two key
foundational elements that we had identified as essential to achieving
successful transformation of business systems and processes: an EA and
a concept of operations. Without these key foundational elements, the
Army is at risk of investing billions of dollars in business systems
that may not achieve DOD's and the Army's goal of achieving TAV.
Further, the Army's planned strategy perpetuates some of the cumbersome
and ineffective business processes and data redundancies that are
currently being used in the existing legacy system environment.
Army's Efforts to Achieve TAV Lack the Benefit of an EA:
The Army has yet to develop and implement an Army business EA to help
guide its system efforts, including those aimed at achieving TAV. DOD's
acquisition policies and guidance,[Footnote 32] as well as federal and
best practice guidance,[Footnote 33] recognize the importance of
investing in business systems within the context of an EA. GFEBS, GCSS-
Army, and LMP are not being managed and developed in the context of a
well-defined Army-wide EA. In August 2006,[Footnote 34] we reported
that the Army was in the very early stages of developing an EA. We
recommended that the Secretary of Defense ensure that the DOD
architecture programs[Footnote 35] we reviewed develop and implement
plans for fully satisfying each of the conditions in our Enterprise
Architecture Management Maturity Framework (EAMMF) for assessing and
improving EA management.[Footnote 36] In commenting on the report, DOD
agreed with our recommendation.
As detailed in our August 2006 report, we assessed the Army's efforts
to develop an EA against the criteria specified in EAMMF. Our EAMMF is
a five-stage architecture framework for managing the development,
maintenance, and implementation of an architecture and understanding
the extent to which effective architecture management practices are
being performed and where an organization is in its progression toward
having a well-managed architecture program. In short, the framework
consists of 31 core elements that relate to architecture governance,
content, use, and measurement. These elements reflect research by us
and others showing that architecture programs should be founded upon
institutional architecture commitment and capabilities, and measured
and verified products and results. Our analysis of information provided
by the Army indicated that it had satisfied only 3 percent of all
framework elements. In essence, this means that the Army is at stage 1
of developing and implementing an EA. While stage 1 agencies may have
initiated some enterprise architecture activity, these agencies'
efforts are ad hoc and unstructured, lack institutional leadership and
direction, and do not provide the management foundation necessary for
successful enterprise architecture development.
In May 2007, we reported that the Army still had not developed an
EA.[Footnote 37] As a result, the Army does not have a well-informed
basis for determining if these systems will fit within the context of
future Army business operations and will most efficiently and
effectively address the Army's long-standing weaknesses associated with
the lack of asset visibility. Improving asset visibility is critical to
addressing the problems associated with supply chain management, which
has been on our high-risk list since 1990.[Footnote 38] Asset
visibility is one of the focus areas that are a critical part of supply
chain management.[Footnote 39]
In addition, without a clear understanding of its current systems and
business processes--commonly referred to as the "As Is" environment--
and the business operations it envisions for the future--commonly
referred to as the "To Be" environment--the Army will increase the risk
that misalignments can occur that can introduce redundancies and
incompatibilities that can produce inefficiencies and require costly
and time-consuming rework to fix. GFEBS, GCSS-Army, and LMP have
proceeded without a common, institutional frame of reference (for
example, EA) that can be used to effectively manage their relationships
and dependencies. In November 2006, DOD's internal review of GFEBS by
BTA noted that the Army lacked a strategy for integrating GFEBS, GCSS-
Army, and LMP.
Army Strategy to Achieve TAV Does Not Utilize an Enterprisewide
Perspective:
The Army also lacks a concept of operations for how GFEBS, GCSS-Army,
and LMP will collectively achieve TAV. An effective concept of
operations would describe, at a high level, (1) how the three business
systems relate to each other in achieving the Army's TAV goal, and (2)
how information flows from and through these systems. Moreover, we
found that the Army's lack of a concept of operations has contributed
to its failure to take full advantage of business process reengineering
opportunities that are available when using an ERP solution. Without a
clear long-term strategy, the Army lacks a key management control for
ensuring that time frames, results-oriented performance measures, and
accountability mechanisms are established and monitored to help achieve
TAV.
Specifically, we noted that the Army's strategy perpetuates some of the
cumbersome and inefficient business processes that are currently being
used in the existing legacy system environment. One of the key benefits
of an ERP is that it can be used to streamline business processes and
eliminate data redundancy. However, some of the basic business
processes being used or expected to be used by GFEBS, GCSS-Army, and
LMP are based on the Army's existing business processes--many of which
are error prone, labor intensive, and redundant. Considering the
billions of dollars the Army is investing in these systems, it is
incumbent upon the Army to embrace and utilize the most efficient and
streamlined ERP processes to the fullest extent possible in its effort
to achieve TAV.
For example, the Army's planned strategy to improve control and
accountability over its PP&E does not take advantage of the
capabilities of an ERP solution. (See app. III for additional examples
in which the Army's TAV systems initiatives have not effectively
streamlined legacy processes related to funds control and
disbursements.) Financial data, including acquisition cost and
depreciation, for most of the Army's PP&E are currently maintained in
the Defense Property Accountability System (DPAS). PP&E accountability
data, such as location and quantity, are maintained in a variety of
legacy systems such as the Property Book Unit Supply Enhanced (PBUSE)
system--which is 1 of the 16 systems that is expected to be replaced by
GCSS-Army. In March 2007, Army Program Executive Office Enterprise
Information Systems officials informed us that the financial data--such
as acquisition cost and depreciation--for the Army's PP&E will be
transferred from DPAS to GFEBS. This maintains the existing accounting
data in an accounting system. On the other hand, the accountability
data--such as location, condition, and number of units--for PP&E assets
will initially be transferred from the current legacy system to PBUSE
or GCSS-Army, depending on whether the items belong to an installation
or an Army unit that may be deployed. The transferring of items to
PBUSE appears to be an interim solution since, according to DOD's
enterprise transition plan GCSS-Army is to replace PBUSE. However, GCSS-
Army is not expected to assume the control and accountability over
items that are recorded in PBUSE associated with the Army Working
Capital Fund or other Army organizational entities that are not
considered to be units that can be deployed (field/tactical units) in
military operations.
The Army's decision to segregate its PP&E financial and accountability
data into two separate system solutions reflects its lack of an
enterprise concept of operations and enterprise view for achieving TAV.
The Army's strategy of moving PP&E data from DPAS to PBUSE, and
ultimately to another system, means that one data conversion will take
place to move the data from DPAS to PBUSE--a system the Army is
supposed to eliminate with the implementation of GCSS-Army--followed by
another data conversion from PBUSE to yet another system, not yet
identified. Further, the Army is continuing to limit the capabilities
of LMP by transferring all information on PP&E from DPAS to PBUSE--
which as noted before is considered a legacy business system. As a
result, information on PP&E that could enhance Tobyhanna's operations,
especially in the area of workload planning, will not be available
within the LMP environment.
Thus, the approach being taken by the Army is simply to adapt the
processes used by the legacy systems rather than use the streamlined
processes inherent in the ERP solution. If the functionality of an ERP
solution was fully utilized by the Army, it would increase the
likelihood that TAV could be accomplished within each of the three
systems for the specific type of PP&E items each was responsible for.
For example, GFEBS would contain both the financial and accountability
information related to nontactical/non-working capital fund PP&E, GCSS-
Army would have all the information associated with tactical PP&E, and
LMP would have the data related to Working Capital Fund PP&E. Figure 5
illustrates one way that an ERP approach could be utilized to achieve
physical and financial accountability control of the Army's PP&E.
Figure 5: Example of an ERP Vision for Accountability of the Army's
PP&E:
[See PDF for image]
Source: GAO.
[End of figure]
While the Army's approach may result in incremental improvements in its
asset accountability, it does not take full advantage of the (1)
benefits of adopting enterprise processes and (2) functionality that is
available in the ERP solution it has selected to support those
processes. Fundamental to a successful ERP implementation is the
reengineering of an organization's business processes in a manner that
helps ensure that the right resources (people, material, machinery, and
funds) are available in the correct quantities when needed.[Footnote
40] While Army has stated that its use of ERP software would help
reengineer its business processes, we found that at least some business
processes that are being used or expected to be used in the future do
not reflect reengineered processes necessary to most effectively
implement an ERP solution. By perpetuating the use of cumbersome, error-
prone, and ineffective business processes in its asset accountability
operations, the Army will diminish its capability to achieve TAV and
improve accountability over its assets.
Army's Ability to Effectively Oversee Portfolios of Business Systems
Investment Is Not Yet Fully Developed:
While the Army has established a governance structure to oversee its
business system investments--including its asset accountability system
investments--that is consistent with DOD guidance, additional actions
are needed to enhance oversight, control, and accountability. The
Army's business system investment oversight efforts to date have
primarily focused on ensuring that business systems modernizations over
$1 million are reviewed in accordance with the criteria specified in
the fiscal year 2005 National Defense Authorization Act. Both DOD and
Army officials acknowledged that the department's and Army's investment
review processes, particularly related to their ability to review
business system investments from a portfolio perspective, are in the
early stage of maturity. Portfolio management is a conscious,
continuous, and proactive approach to allocating limited resources
among competing initiatives in light of the investments' relative
benefits. Taking an enterprise view enables an organization to consider
its investment comprehensively, so that collectively the investments
optimally address the organization's mission, goals, and objectives. In
addition, we found that the Army did not have reliable processes and
analyses to support its oversight of individual systems modernization
program efforts intended to improve asset visibility. Specifically, (1)
the Army has not established or implemented an independent verification
and validation (IV&V) process for the three systems--GFEBS, GCSS-Army,
or LMP and (2) the August 2004 GFEBS economic analysis was not prepared
in accordance with DOD and OMB guidance.[Footnote 41]
Army's Oversight of Its Business System Investments Lacks a Portfolio
Perspective:
The Army's oversight of its business system investments--including
system investments intended to achieve TAV--continues to be focused on
the review and approval of individual business system investments. In
May 2007, we reported that DOD needed to improve its policies and
procedures associated with managing its business system investments as
portfolios.[Footnote 42] More specifically, we found that DOD had not
yet progressed from project-based processes to portfolio-based
processes, a key element for effectively managing business system
transformation efforts. DOD informed us that DOD components are
responsible for developing and managing their own portfolio management
processes.
Army IT guidance for managing IT investments requires portfolios to be
managed and monitored using established quantifiable outcome-based
performance measures to determine whether to recommend continuations,
modification, or termination of individual investments within a
portfolio.[Footnote 43] However, the Army has not implemented processes
or procedures that facilitate an enterprise view toward management and
oversight of portfolios of business system investments intended to
collectively provide a specific capability or functionality. Rather,
the Army's current business system investment process focuses primarily
on reviewing and approving investment packages submitted by the
responsible program management office for individual systems
improvement efforts that have development/modernization funding over $1
million. Individual business system investments that exceed the $1
million systems modernization threshold are required to be reviewed by
the appropriate IRB and approved by the DBSMC.
Army officials acknowledged that the Army's ability to provide
portfolio-based investment review and oversight is in the early stages
of development. For example, while the Army has begun a process to
share investment package information, including cost, schedule, and
performance data, across its mission areas and domains, Army officials
acknowledged that the information provided by a program management
office, as well as the associated oversight and review, is centered
largely on the functionality and capabilities of individual systems.
For example, annual certifications for the three systems, LMP, GCSS-
Army, and GFEBS, which the Army intends to utilize collectively in
achieving TAV, are not submitted or reviewed as a portfolio, but rather
as individual system investments. The Army has not yet implemented the
processes to evaluate and improve its progress toward achieving TAV
using portfolio-based projects such as costs, schedule, performance,
and risks. As a result, the Army's ability to effectively oversee the
development of the portfolio of systems intended to collectively
provide the Army with TAV is limited. Until the Army's governance
process, policies, and procedures mature, including its ability to
apply a corporate portfolio perspective in managing and overseeing
subportfolios of business system investments intended to provide a
desired capability or function, the Army is at risk of implementing
systems that (1) do not provide the desired capability, including TAV;
(2) are stovepiped in their functionality; (3) do not interoperate in
an efficient manner; and (4) are not supported by efficient and
effective business processes.
Reliable Processes and Analyses Are Needed to Facilitate Management
Oversight of Business System Investments:
The Army has not implemented reliable processes and analyses needed to
enhance its management oversight of program management office efforts
to develop and implement its TAV, as well as other business systems on
time, within budget, and with the intended capability. Both DOD's and
the Army's business system investment oversight processes are highly
dependent upon the reliability of investment information provided by
Army program management offices. Thus, the reliability of that business
system investment information is critical to both DOD's and the Army's
ability to effectively oversee, manage, and redirect--if necessary--the
services' business system development and modernization efforts.
However, we found that the Army had not yet established an independent
verification and validation function for any of the three systems we
reviewed. In addition, the economic analysis submitted by the program
management office for GFEBS was not prepared in accordance with DOD or
OMB guidance. As a result, the economic analysis submitted by the
program management office to DOD and the Army oversight entities
justifying the Army's investment in GFEBS was based on questionable
costs and benefits.
Independent Verification and Validation Would Enhance Program
Oversight:
The Army had not yet established an IV&V function[Footnote 44] for any
of the three systems--GFEBS, GCSS-Army, and LMP. While the Army has
established a verification and validation function for LMP, it was not
an independent review because the reviewer reports directly to the LMP
program management office. Independence is a key component to reliable
verification and validation function.
Best business practices have demonstrated that use of an IV&V function
is an invaluable means to providing management reasonable assurance
that a planned system, or the portfolio of systems, will satisfy its
planned use and users. An effective IV&V review process should provide
an independent assessment to DOD and Army management of the overall
status of the project, including a discussion of any existing or
potential revisions to the project with respect to cost, schedule, and
performance. The IV&V reports should identify to management the issues
or weaknesses that increase the risks associated with the project or
portfolio to senior management so that they can be promptly addressed.
These assessments involve reviewing project documentation,
participating in meetings at all levels within the project, and
providing periodic reports and recommendations, if deemed warranted, to
senior management. The IV&V function should report on every facet of a
system project such as:
Testing program adequacy. Testing activities would be evaluated to
ensure they are properly defined and developed in accordance with
industry standards and best practices.
Critical-path analysis. A critical path defines the series of tasks
that must be finished in time for the entire project (or portfolio of
projects) to finish on schedule. Each task on the critical path is a
critical task. A critical-path analysis helps to identify the impact of
various project events, such as delays in project deliverables, and
ensures that the impact of such delays is clearly understood by all
parties involved with the project(s).
System strategy documents. Numerous system strategy documents that
provide the foundation for the system development and operations are
critical aspects of an effective system project. These documents are
used for guidance in developing documents for articulating the plans
and procedures used to implement a system. Examples of such documents
include the Life-cycle Test Strategy, Interface Strategy, and
Conversion Strategy.
Validity of GFEBS's Economic Analysis Questionable:
Our analysis of the August 2004 GFEBS economic analysis found that it
was not prepared in accordance with OMB and DOD guidance.[Footnote 45]
The purpose of an economic analysis is to give the decision maker
insight into economic factors that have a bearing on accomplishing the
stated objective of the system investment, for example implementation
of GFEBS to improve the Army's financial accountability. As such, it is
important that the economic analysis reliably identifies factors, such
as cost and performance risks and drivers that can be used to establish
priorities and allocate resource allocations. While it may be
appropriate to invest in a particular IT investment, such as GFEBS, for
reasons other than estimated economic benefits, nonetheless, the issues
we identified raise questions as whether the funds invested in GFEBS
were economically justified. Our specific concerns with the August 2004
GFEBS economic analysis are highlighted below.
* Inappropriate cost savings. At least $142 million of estimated
savings were not savings, but rather, should have been classified as
transfers--which do not equate to economic benefits. Transfers
represent shifts of control over resource allocation from one group to
another that do not result in a net change in the value of the
resources involved in the transfer. For example, we found that the Army
claimed over $88 million of savings related to transferring real
property management from the Army's legacy system to GFEBS. Since the
Army's real property management responsibilities were not eliminated,
only transferred from the legacy system environment to GFEBS, this
represents a transfer rather than an economic benefit for the Army.
Additionally, the Army claimed benefits of about $54 million related to
GFEBS reducing the length of time funds are borrowed by the Treasury to
meet the Army financial obligations by facilitating the Army's ability
to make just-in-time disbursements. OMB guidance provides that there
are no economic gains from a pure transfer payment because the benefits
to those government entities that receive such a transfer are matched
by the costs borne by those government entities that provide the
transfer.[Footnote 46] The Army's inappropriate classification of
transfers resulted in an overstatement of the economic benefits that
would be achieved by implementing GFEBS. Additionally, in November
2006, BTA's enterprise risk assessment management review of GFEBS
questioned justifying the GFEBS program on factors not fully within the
program's control.
* Lack of sensitivity analysis. The analysis did not include an
assessment of the effects of the uncertainty inherent in estimates of
GFEBS benefits and costs, as stipulated in OMB and DOD
guidance.[Footnote 47] Because an economic analysis uses estimates and
assumptions, it is critical that a sensitivity analysis be performed to
understand the effects of the imprecision in both underlying data and
modeling assumptions. This analysis is required since the estimates of
future benefits and costs are subject to varying degrees of
uncertainty. Sensitivity analysis refers to changing the value of a
given variable in a model to gauge the effect of change on model
results. It varies a single data element or assumption while holding
the other data elements and assumptions constant to determine what
amount of change in that element is required to raise or lower the
resulting benefit and cost elements. In this way, GFEBS data and
assumptions can be risk-ranked for decisionmaking and auditing. The
Army's failure to conduct a sensitivity analysis of GFEBS, as part of
its economic analysis, to identify the effect of uncertainties
associated with different assumptions increases the chance that
decisions regarding GFEBS will be made without a clear understanding of
the possible impact on GFEBS estimates of costs and benefits. Army
officials informed us that they will consider including a sensitivity
analysis of the effect of uncertainties when they update GFEBS's
economic analysis.
* Lack of documentation supporting benefits and cost assumptions. The
GFEBS program management office could not provide us with benefits and
costs data needed to replicate their analyses. As a result, we were
unable to trace the cost and benefit estimates reported in the GFEBS
economic analysis to original source documentation to assess and
validate the reliability and applicability of the data. OMB and DOD
guidance stress the importance of maintaining documentation supporting
assumptions used in a cost-benefit analysis.
LMP Illustrates Continuing Problems in Implementing Business Systems on
Time, within Budget, and with the Promised Capability:
LMP continues to be plagued by operational problems that have beset the
system virtually since its initial implementation in July 2003. While,
an EA, concept of operations, and effective IT portfolio management are
essential elements in the Army's efforts to transform its operations,
it is equally important for the Army to have disciplined processes in
place to implement its business systems on time, within budget, and
with the promised capability. Many of DOD's approximately 2,980
business systems are nonintegrated, stovepiped, and not capable of
providing departmental management and the Congress accurate and
reliable information on DOD's day-to-day operations. LMP is no
exception. As previously noted, LMP was to have reached FOC in fiscal
year 2005, but currently FOC is estimated for fiscal year 2010--a
slippage of 5 years. Further, LMP's estimated program costs have also
increased. Tobyhanna Army Depot--the only depot that is operating LMP-
-and DFAS personnel have stated that improvements have been made in
LMP's operating efficiency. However, problems in the areas of revenue
recognition and billing continue. These problems can, in part, be
attributed to ineffective system testing. The operational issues
confronting LMP and our concerns about the effectiveness of the system
testing are highlighted below and discussed in more detail in appendix
IV.
Operational Issues Continue:
Tobyhanna Army Depot continues to experience problems with LMP
accurately recognizing revenue and billing customers. These problems
have existed virtually since the implementation of LMP in July 2003.
While the Army and its contractor--Computer Sciences Corporation (CSC)-
-have made numerous attempts to fix the problem, they have not been
successful. Additionally, the accuracy of LMP financial reports
continues to be questionable. For instance, the DOD Inspector General's
audit of the Army's Working Capital Fund financial statements for
fiscal year 2006 noted that LMP was not properly recording transactions
in accordance with the U.S. Government Standard General Ledger
requirements.
For the 3-month period ending January 31, 2007, we found that LMP's
continuing billing problems resulted in (1) customers not being billed
for costs incurred that should have been billed and (2) customers being
billed too much. Based upon the billing information provided by the LMP
program management office, there were 146 customer orders valued at
approximately $5.4 million that were not billed (or recognized as
revenue) during the January 31, 2007, billing cycle. Customer billings
and the associated revenue provide the means by which the depot
finances its day-to-day operations. Similarly, during the January 31,
2007, billing cycle about 308 customer orders, amounting to about $5.8
million, were shown as being overbilled and a credit should have been
provided to the customer. Moreover, the continuing billing and revenue
recognition problem may be a factor contributing to higher depot bills
from DFAS. According to DFAS personnel, the recurring billing problems
have resulted in DFAS personnel processing Tobyhanna Army Depot's bills
manually. Based on information provided by DFAS, the number of hours
spent to provide DFAS accounting services--including billings for the
Tobyhanna Army Depot--has increased from approximately 17,800 hours in
fiscal year 2004 to over 22,600 hours in fiscal year 2006.
System Testing Remains a Concern:
Our prior audits[Footnote 48] of LMP identified significant weaknesses
with the LMP program management office's efforts to effectively
implement the requirements management and testing processes needed to
reduce risks to acceptable levels. During our current audit, we found
that the requirements management processes have improved--a critical
first step--but concern remains with respect to the adequacy of the
testing.
Our analysis of 10 selected LMP corrective actions identified specific
testing weaknesses in each of the corrective actions reviewed. For
example, none of the test scripts provided adequate information that
linked the items tested to the specific requirement being tested. This
linking is commonly referred to as traceability and is characteristic
of a disciplined testing process. The test scripts reviewed contained
headings for information on the scenario, key data requirements,
expected results, actual results, and whether the test was considered
successful. However, the actual test script did not contain the level
of specificity that clearly delineated how a specific requirement
identified in the requirement document(s) and the associated
requirement(s) in that document were being tested. As a result, it was
not possible to determine if all LMP system requirements were properly
tested. Without linking a given requirement to the tests designed to
exercise that requirement, it was impossible to obtain reasonable
assurance that (1) all the requirements were tested and (2) the applied
test provided adequate coverage for each requirement.
After discussing our concerns with the LMP program management office
and its contractor in February 2007, the LMP program office requested
that its verification and validation (V&V) contractor perform an
assessment of the LMP testing process. According to information
provided by the LMP program office and the V&V contractor, in March
2007, the review found that the LMP testing process was adequately
planned but that it did not support a clear understanding of the (1)
requirement being tested and (2) tests used to determine whether a
requirement was adequately implemented. The V&V contractor's assessment
substantiates our analysis of the LMP testing process. According to the
V&V contractor, these areas need to be addressed before the Army Test
and Evaluation Command conducts its review of LMP.
According to LMP program management office officials, the CSC is
developing and implementing a testing process that is designed to
address the concerns raised by us and the Army V&V contractor. CSC is
expected to have a testing program that clearly links the requirements
to the tests being conducted and to establish an independent test group
to ensure that the testing process is following best practices.
However, until the Army has reasonable assurance that an effective
testing process has been properly implemented, we are reiterating our
June 2005 recommendation that LMP not be deployed to additional
locations until the Army has assurance that LMP is providing the
intended functionality and, more specifically, that LMP can accurately
bill its customers and recognize revenue.
Conclusions:
The Army's efforts to develop and implement GFEBS, GCSS-Army, and LMP
may result in incremental improvements in the Army's accountability and
visibility over its billions of dollars in tangible assets. However,
the Army has not developed or utilized key management tools and
concepts necessary to successfully transform its business processes to
achieve TAV. The primary question that the Army has yet to effectively
consider and address is how these systems and associated processes,
individually and collectively, will provide the desired functionality
necessary to achieve TAV. Until the Army develops and implements an
Army EA, concept of operations, and portfolio-based management and
oversight processes, it continues to risk (1) investing billions of
dollars in asset accountability systems that may not enhance the
department's and the Army's goal of achieving TAV, and (2) further
delay DOD's efforts to remove supply chain management from our high-
risk list.
It is also important that the Army take action to implement the
disciplined processes necessary to implement asset accountability
systems on time, within budget, and with the promised capability.
Continuing problems with LMP are illustrative of the consequences in
failing to address these issues. Although LMP became operational 4
years ago, it continues to encounter operational problems. While there
have been some improvements in LMP, key functionality, such as
accurately billing customers and recognizing revenue, remains
problematic. The Army's inability to resolve these persistent LMP
problems can, in part, be attributed to inadequate system testing.
Until LMP implements an effective testing process, it will have little
assurance that the development and corrective actions it takes (1) are
properly developed, and (2) do not introduce additional defects into
the system.
Recommendations for Executive Action:
To improve the department's efforts to achieve TAV and further enhance
its efforts to improve the control and accountability over business
system investments, we recommend that the Secretary of Defense direct
the Secretary of the Army and the Director, BTA, to jointly take the
following five actions:
* Develop a concept of operations that (1) clearly defines the ERP
vision for accomplishing total asset visibility within the Army; (2)
addresses how its business systems and processes, individually and
collectively, will provide the desired functionality to achieve TAV;
and (3) determines the desired functionality among the selected
systems.
* Develop policies, procedures, and processes to support the oversight
and management of selected groupings of business systems that are
intended to provide a specific capability or functionality, such as TAV
from a portfolio perspective, utilizing indicators such as costs,
schedule, performance, and risks.
* Establish an IV&V function for GFEBS, GCSS-Army, and LMP.
Additionally, direct that all IV&V reports for each system be provided
to Army management, the appropriate IRB, and BTA.
* Require that any future GFEBS economic analysis identify costs and
benefits in accordance with the criteria specified by DOD and OMB
guidance, to include a sensitivity analysis.
* Direct that LMP utilize systems testers that are independent of the
LMP system developers to help ensure that the system is providing the
users of the system the intended capabilities.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from the Deputy
Under Secretary of Defense (Business Transformation), which are
reprinted in appendix II. Overall, DOD concurred with our
recommendations and stated that it would work diligently to implement
them. The comments included two sets of specific responses to our
recommendations--one set provided by BTA and another provided by the
Army. In regard to the response provided by the Army, it concurred with
each of the recommendations. BTA stated that it fully agreed with our
observations. BTA, though, partially concurred with all the
recommendations on the basis that they were directed jointly to the
Secretary of the Army and the Director, BTA. BTA stated that under U.S.
Code Title 10 and in accordance with DOD's tiered accountability, the
Army has complete authority for execution of its programs and
responsibility for implementing the recommendations. BTA's comments
noted that it has neither the authority nor the responsibility to
direct the actions of the Army.
We appreciate the department's willingness to address our
recommendations. With regard to BTA's concern, our recommendations do
not direct BTA to oversee or direct the Army. Rather, the
recommendations stated that the specific actions should be undertaken
jointly at the direction of the Secretary of Defense. We continue to
believe that a cooperative and effectively coordinated BTA and Army
approach to addressing our recommendations is most likely to achieve
the fundamental business system transformation necessary to achieve the
department's TAV objective. Further, the involvement of the Office of
the Secretary of Defense, including BTA, in the Army's investment
management practices is consistent with 10 U.S.C. §§ 186 and 2222,
which provide for DOD policymaking and oversight of Army business
system functions.
In regard to the responses provided by the Army, it acknowledged the
importance of a concept of operations in achieving its goal of TAV and
in optimizing its business system investments. Further, the Army
acknowledged that there are risks involved in attempting to effectively
integrate the various system efforts. Going forward, as we recommended,
it will be important that the Army develop a clear, long-term strategy
to help ensure that time frames, results-oriented performance measures,
and accountability mechanisms are established and monitored to achieve
TAV.
We are sending copies of this report to the Secretary of Defense;
Acting Secretary of the Army; Under Secretary of Defense (Acquisition,
Technology, and Logistics); Under Secretary of Defense (Comptroller);
Deputy Under Secretary of Defense (Business Transformation); Deputy
Under Secretary of Defense (Financial Management); Assistant Secretary
of Defense (Networks and Information Integration); Assistant Secretary
of the Army (Acquisition, Logistics, and Technology); Assistant
Secretary of the Army (Financial Management and Comptroller); Army
Chief Information Officer; Army Deputy Chief of Staff (Logistics);
Commander, U.S. Army Materiel Command; and other interested
congressional committees and members. Copies of this report will be
made available to others upon request. In addition, this report is
available at no charge on the GAO Web site at http://www.gao.gov.
Please contact McCoy Williams at (202) 512-9095 or williamsm1@gao.gov,
Keith A. Rhodes at (202) 512-6412 or rhodesk@gao.gov, or William M.
Solis at (202) 512-8365 or solisw@gao.gov if you or your staff have
questions on matters discussed in this report. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report are
listed in appendix V.
Signed by:
McCoy Williams:
Director:
Financial Management and Assurance:
Signed by:
Keith A. Rhodes:
Chief Technologist:
Applied Research and Methods:
Center for Technology and Engineering:
Signed by:
William M. Solis:
Director:
Defense Capabilities and Management:
[End of section]
Appendix I: Scope and Methodology:
To determine whether the Army has developed a business system strategy
for achieving total asset visibility (TAV), we met with program office
officials for the General Fund Enterprise Business System (GFEBS), the
Global Combat Support System-Army (GCSS-Army), and the Logistics
Modernization Program (LMP). We obtained briefings on the intended
purpose of each system and walkthroughs of various scenarios that
described how the systems would transmit data between them. We also met
with officials from the Office of Program Analysis and Evaluation, the
Office of the Assistant Secretary of Defense (Network and Information
Integration)/the Department of Defense (DOD) Chief Information Officer,
and the Business Transformation Agency (BTA) to identify and discuss
any issues/concerns they have related to the implementation of GFEBS
and GCSS-Army. Furthermore, we reviewed reports by the Army Audit
Agency and the DOD Inspector General to ascertain if they had
previously reported upon system weaknesses related to the three systems
and how those weaknesses would affect the Army's vision of an
integrated systems strategy.
In order to determine if the Army has effectively implemented a
governance structure to oversee and manage its business system
investment in accordance with DOD guidance, we reviewed guidance issued
by the Office of the Secretary of Defense, the Army, and BTA related to
investment management. We also obtained an understanding of the Army's
process to comply with the certification and annual system review
processes required by the fiscal year 2005 National Defense
Authorization Act.[Footnote 49] We also obtained and analyzed
documentation related to the certification and annual system review
process. Furthermore, we reviewed the Selected Capital Investment
Review reports as of February 2006 and February 2007 to ascertain if
there were any cost, schedule, and performance issues discussed in
these reports for the GFEBS, GCSS-Army, and LMP. We also reviewed
Defense Acquisition Executive Summary reports for GFEBS to ascertain if
there were any cost, schedule, and performance issues discussed in
these reports. Similar reports were not required for GCSS-Army and LMP.
Additionally, we met with BTA officials to obtain an understanding of
how the Enterprise Risk Assessment Model is to be utilized as part of
the department's investment management criteria.
To evaluate the extent to which the Army has made progress in
correcting the previously reported problems regarding LMP's
implementation, we met with officials from the LMP program office and
the Computer Sciences Corporation (CSC). Briefings were provided that
detailed the specific corrective actions taken in response to the
various issues discussed in our two previous reports.[Footnote 50] To
substantiate the corrective actions taken to address the accounting
issues, we reviewed the Defense Finance and Accounting Service (DFAS)
1307 accounting reports[Footnote 51] and discussed the issues detailed
in the reports with DFAS personnel responsible for preparing the
reports. We also reviewed the fiscal year 2006 Army Working Capital
Fund Financial Statements to ascertain the specific issues related to
LMP. Further, we met with DOD Inspector General personnel and reviewed
their workpapers to obtain an understanding of the deficiencies
discussed in the fiscal year 2006 Army Working Capital Fund Financial
Statements.
To assess whether the Army had established and implemented disciplined
processes related to requirements management and testing we:
* obtained an understanding of the Army's revised procedures for
defining requirements management frameworks and compared these
procedures to its current practices;
* reviewed guidance published by the Institute of Electrical and
Electronics Engineers and the Software Engineering Institute and
publications by experts to determine the attributes that should be used
for developing good requirements; and:
* selected 10 requirements and performed an in-depth review and
analysis to determine whether they had the attributes normally
associated with good requirements and whether these requirements traced
between the various process documents. These requirements were to have
followed the revised requirements management process.
To augment these document reviews and analyses, we discussed our
analysis with LMP program office and CSC officials. We also met with
officials from the Office of Program Analysis and Evaluation, the
Office of the Assistant Secretary of Defense (Network and Information
Integration)/DOD Chief Information Officer (CIO), and the Business
Transformation Agency to identify any issues/concerns they have related
to the implementation of LMP.
We performed our audit work from May 2006 through June 2007 in
accordance with U. S. generally accepted government auditing standards.
We conducted our work at the Office of the Assistant Secretary of
Defense (Networks and Information Integration)/DOD Chief Information
Officer; the U.S. Army Materiel Command; the Army CIO; and the Business
Transformation Agency. We visited the Army Program Executive Office
Enterprise Information Systems and GFEBS program office at Ft. Belvoir,
Virginia. We also visited the Army contractor's site for LMP, Computer
Sciences Corporation in Marlton, New Jersey--and for GCSS-Army--
Northrop Grumman in Chester, Virginia. We also visited the Tobyhanna
Army Depot, Tobyhanna, Pennsylvania and DFAS Indianapolis, Indiana. We
requested comments on a draft of this report from the Secretary of
Defense or his designee. We received written comments from the Deputy
Under Secretary of Defense (Business Transformation), which are
reprinted in appendix II.
[End of section]
Appendix II: Comments from the Department of Defense:
Note: GAO comments supplementing those in the report text appear at the
end of this appendix.
Office Of The Under Secretary Of Defense:
9000 Defense Pentagon:
Washington, DC 20301-3000:
Acquisition, Technology And Logistics:
Jul 6 2007:
Mr. McCoy William:
Director, Financial Management and Assurance:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. William:
This is the Department of Defense (DoD) response to the GAO draft
report 07-860, "DOD Business Transformation: Lack of an Integrated
Strategy Puts the Army's Asset Visibility System Investments at Risk",
dated June 6, 2007 (GAO Code 195087).
DoD hereby acknowledges receipt of the draft report. We have reviewed
the draft and appreciate the GAO's support in identifying areas of
improvement as it relates to the Department of Army's total asset
visibility. Overall, DoD concurs with the GAO's recommendations, except
where it is recommended that the Business Transformation Agency (BTA)
jointly direct the Army to implement a recommendation. Under Title 10
U.S. Code, the BTA neither has the authority nor responsibility to
direct the actions of the Army. Attached, you will find responses to
these recommendations from both the BTA and the Army.
Still, DoD agrees with the core of each recommendation, and we will
work diligently to close them. Where appropriate, the BTA will provide
assistance and support to the Army in achieving this objective.
Additionally, it is DoD's intent, at an enterprise level, to continue
maturing our portfolio management policies and procedures such that
they align with and assist the Army in addressing the recommendations
presented in the draft report.
Again, thank you for your support. DoD strongly values our partnership
with the GAO, and we appreciate your continued strides to assist us
with our business transformation efforts.
Signed for:
Paul A. Brinkley:
Deputy Under Secretary of Defense (Business Transformation):
GAO Draft Report Dated June 6, 2007 GAO-07-860 (GAO code 195087):
"DOD Business Transformation: Lack Of An Integrated Strategy Puts The
Army's Asset Visibility System Investments At Risk"
Department Of Defense Comments To The GAO Recommendations Provided By
The Business Transformation Agency:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of Business
Transformation Agency (BTA), to jointly develop a concept of operations
that:
a. clearly defines the enterprise resource planning (ERP) vision for
accomplishing total asset visibility within the Army;
b. addresses how its business systems and processes, individually and
collectively, will provide the desired functionality to achieve total
asset visibility (TAV); and:
c. determines the desired functionality among the selected systems.
(Page 42/GAO Draft Report):
DOD Response: Partially Concur:
The Department concurs with the Army's position. However, the overall
Department position is "partially concur" due to the recommendation
being directed jointly to the Secretary of the Army and the Director,
BTA. While we fully agree with GAO's observations, under U.S. Code
Title 10 and in accordance with the Department's tiered accountability
approach, the Army has complete authority for execution of its programs
and responsibility for implementing the recommendation. BTA will
continue to provide assistance and support to the Army from an
enterprise standpoint as needed.
BTA has developed the BEA and has provided specific recommendations to
the Army in addressing TAV and other related issues, and continues to
work with the Army for the integration of the proper functionality. BTA
has also evaluated the deployment of the ERP solutions in accordance
with commercial best practices and recommendations to the Army which
are designed to achieve TAV and proper financial controls.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
develop policies, procedures, and processes to support the oversight
and management of selected groupings of business systems that are
intended to provide a specific capability or functionality, such as TAY
from a portfolio perspective, utilizing indicators such as costs,
schedule, performance, and risks. (Page 43/GAO Draft Report):
DOD Response: Partially Concur:
The Department concurs with the Army's position. However, the overall
Department position is "partially concur" due to the recommendation
being directed jointly to the Secretary of the Army and the Director,
BTA. While we fully agree with GAO's observations, under U.S. Code
Title 10 and in accordance with the Department's tiered accountability
approach, the Army has complete authority for execution of its programs
and responsibility for implementing the recommendation. BTA will
continue to provide assistance and support to the Army from an
enterprise standpoint as needed.
It is the Department's intent, at an enterprise level, to continue
maturing our portfolio management policies and procedures.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA jointly
establish an independent verification and validation (IV&V) function
for General Fund Enterprise Business System (GFEBS), Global Combat
Support System-Army Field/Tactical (GCSS-Anny) and Logistics
Modernization Program (LMP). Additionally, direct that all IV&V reports
for each system be provided to Army management, the appropriate
investment review board (IRB) and BTA. (Page 43/GAO Draft Report):
DOD Response: Partially Concur:
The Department concurs with the Army's position. However, the overall
Department position is "partially concur" due to the recommendation
being directed jointly to the Secretary of the Army and the Director,
BTA. While we fully agree with GAO's observations, under U.S. Code
Title 10 and in accordance with the Department's tiered accountability
approach, the Army has complete authority for execution of its programs
and responsibility for implementing the recommendation. BTA will
continue to provide assistance and support to the Army from an
enterprise standpoint as needed.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
require that any future GFEBS economic analysis identify costs and
benefits in accordance with the criteria specified by DoD and OMB
guidance, to include a sensitivity analysis. (Page 43/GAO Draft
Report):
DOD Response: Partially Concur:
The Department concurs with the Army's position. However, the overall
Department position is "partially concur" due to the recommendation
being directed jointly to the Secretary of the Army and the Director,
BTA. While we fully agree with GAO's observations, under U.S. Code
Title 10 and in accordance with the Department's tiered accountability
approach, the Army has complete authority for execution of its programs
and responsibility for implementing the recommendation. BTA will
continue to provide assistance and support to the Army from an
enterprise standpoint as needed.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
direct the LMP utilize system testers that are independent of the LMP
system developers to help ensure that the system is providing the users
of the system the intended capabilities. (Page 43/GAO Draft Report):
DOD Response: Partially Concur:
The Department concurs with the Army's position. However, the overall
Department position is "partially concur" due to the recommendation
being directed jointly to the Secretary of the Army and the Director,
BTA. While we fully agree with GAO's observations, under U.S. Code
Title 10 and in accordance with the Department's tiered accountability
approach, the Army has complete authority for execution of its programs
and responsibility for implementing the recommendation. BTA will
continue to provide assistance and support to the Army from an
enterprise standpoint as needed.
GAO Draft Report Dated June 6, 2007 GAO-07-860 (GAO code 195087):
"DOD Business Transformation: Lack Of An Integrated Strategy Puts The
Army's Asset Visibility System Investments At Risk"
Department Of Defense Comments To The GAO Recommendations Provided By
The Army:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of Business
Transformation Agency (BTA), to jointly develop a concept of operations
that:
a. clearly defines the enterprise resource planning (ERP) vision for
accomplishing total asset visibility within the Army;
b. addresses how its business systems and processes, individually and
collectively, will provide the desired functionality to achieve total
asset visibility (TAV); and:
c. determines the desired functionality among the selected systems.
(Page 42/GAO Draft Report):
DOD Response: Concur:
* The Army is amenable to ideas that will facilitate our optimization
goals to achieve TAV. We believe that a concept of operations that
includes the three components described in Recommendation 1 will help
us attain our optimization goals. We have blueprinted our systems based
on requirements and priorities and divided that functionality among the
three ERP systems. Above all, our goal is an efficient and effective
deployment of automated business capabilities given the realities of
where the Army is today, strategically, financially, and
programmatically.
* The Army agrees that we should optimize our IT business investments
to achieve the goal of TAV. At program initiation, the Army
functionally aligned its financial and logistics ERP solutions. We
further recognize the synchronization risk between ERP programs and
manage accordingly. Hence, the Army has a single materiel developer
organization, PEO Enterprise Information Systems (PEO EIS), and a
single manager to whom the ERP Product Managers report. A recent
realignment of the Logistics Modernization Program (LMP) from an Army
Command to the PEO EIS facilitated unity of purpose, helping to
synchronize the acquisition aspects of a solution that enables TAV for
the Army. Moreover, we have established a functional cross-domain
working group between the Single Automated Logistics Enterprise (SALE)
and the Single Automated Financial Enterprise (SAFE) that meets
regularly to decide optimal implementations of business processes to
achieve visibility of Army assets that transcend Domains and the
tactical and national levels of logistics.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
develop policies, procedures, and processes to support the oversight
and management of selected groupings of business systems that are
intended to provide a specific capability or functionality, such as TAV
from a portfolio perspective, utilizing indicators such as costs,
schedule, performance, and risks. (Page 43/GAO Draft Report):
DOD Response: Concur.
While we are an Army at war, men and women are working diligently to
develop and institutionalize an oversight system for business systems
that is austere, yet effective; defined and repeatable but not
bureaucratic; and focused on the Army's ability to man, equip, train,
and maintain our Forces in the field. In this regard, the Army Business
Mission Area has allocated resources to assist in the maturation of
Army Portfolio Management (PfM).
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA jointly
establish an independent verification and validation (IV&V) function
for General Fund Enterprise Business System (GFEBS), Global Combat
Support System-Army Field/Tactical (GCSS-Army) and Logistics
Modernization Program (LMP). Additionally, direct that all IV&V reports
for each system be provided to Army management, the appropriate
investment review board (IRB) and BTA. (Page 43/GAO Draft Report):
DOD Response: Concur.
As previously stated, the Army has charged the PEO EIS to manage the
cost, schedule, and performance aspects of the GFEBS, GCSS-Army, and
LMP. The organization is governed by and accountable to the Milestone
Decision Authority according to the program baselines established. The
Army recognizes the need for a enterprise view of the risks associated
with each program. Therefore, the PEO EIS will ensure establishment of
an Independent Validation and Verification (IV&V) process for each
program that provides for risk management at a level where management
responsibility exists for each of the three ERP systems and independent
of the programs. All PEO EIS IV&V results will be provided to the
Department of the Army and relevant management organizations as
required.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
require that any future GFEBS economic analysis identify costs and
benefits in accordance with the criteria specified by DoD and OMB
guidance, to include a sensitivity analysis. (Page 43/GAO Draft
Report):
DOD Response: Concur.
* The GAO report cites $142M of questionable benefits, comprised of
"transfers". While the economics of the transfers was debated with GAO,
the real point should be that these are $142M questionable benefits of
$6,667M total benefits. GAO takes issue with not adequately justifying
2.1 % of the benefits. Assuming that the 2.1% is not legitimate,
eliminating these benefits at the very worst lowers the Benefit-Cost
Ratio from 2.35 to 2.28, which still would justify the investment.
* The GAO report cites the lack of a sensitivity analysis. The Cost
Review Board Working Group included the sensitivity analysis by adding
into the estimate the uncertainty surrounding the costs of interfaces.
While this does not show up as a titled section in the EA called
"Sensitivity", the results of the analysis were in fact included in the
GFEBS EA.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the Secretary of the Army and the Director of BTA, to jointly
direct the LMP utilize system testers that are independent of the LMP
system developers to help ensure that the system is providing the users
of the system the intended capabilities. (Page 43/GAO Draft Report):
DOD Response: Concur.
The following are GAO's comments on DOD's letter dated July 6, 2007.
GAO Comments:
1. See the "Agency Comments and Our Evaluation" section of this report.
2. In regard to the GFEBS economic analysis, the program office did not
provide the necessary support for us to replicate the estimated program
savings.
3. In regard to the sensitivity analysis, the Army's approach of adding
the uncertainty surrounding the costs of interfaces into the estimate
is not consistent with OMB, DOD, and Army guidance, as discussed in the
report. Rather, a sensitivity analysis is accomplished by changing the
numerical value of a given variable to gauge the effect of that change
on model results, such as the benefit-to-cost ratio. A sensitivity
analysis identifies key assumptions and varies a single assumption
while holding the others constant to determine what amount of change in
that assumption is required to raise or lower the resulting dominant
benefit or cost estimates by a set amount.
[End of section]
Appendix III: Army Strategy Does Not Fully Utilize Capabilities of ERP
Solution:
Fundamental to successful enterprise resource planning (ERP)
implementation is the reengineering of an organization's business
processes in a manner that helps ensure that the right resources are
available when needed. While the Army has stated that through the
successful implementation of the General Fund Enterprise Business
System (GFEBS), the Global Combat Support System-Army (GCSS-Army), and
the Logistics Modernization Program (LMP) will reengineer its business
processes, we found some business processes that have been adopted or
are expected to be adopted have not embraced the basic concept of an
ERP solution. Details concerning the Army's strategy for handling funds
control and disbursements are highlighted below.
Funds Control:
The funds control concept that will be utilized by GCSS-Army and GFEBS
is that all funds control activities will be handled by GFEBS.
Accordingly, an interface must be developed to transmit the appropriate
information between the two systems in order to ensure that all of the
information is in agreement. Figure 6 illustrates how the process is
intended to work.
Figure 6: Planned GFEBS and GCSS-Army Fund Control Process:
[See PDF for image]
Source: Army.
[End of figure]
As shown in figure 6, information will have to be transmitted from GCSS-
Army each and every time there is a transaction that is associated with
funds control. This approach does not take full advantage of the
capabilities within the ERP software that will be utilized by GCSS-
Army. Figure 7 shows an example of more efficient and streamlined funds
control process based on ERP concepts that eliminates these various
transactions having to be processed between GFEBS and GCSS-Army.
Figure 7: GCSS-Army Funds Control Process Utilizing ERP Capabilities:
[See PDF for image]
Source: Army.
[End of figure]
We were informed by GFEBS program officials that funds control for the
activities associated with GCSS-Army needed to be maintained in GFEBS
because (1) GCSS-Army was not considered an official accounting system,
(2) the version of the software initially selected by GCSS-Army could
not meet the requirements of the Federal Financial Management
Improvement Act of 1996 (FFMIA),[Footnote 52] and (3) the Army wanted
to centrally manage general fund activities. In effect, the Army's
approach mirrors the functionality in the stovepiped processes of the
legacy systems, dedicated processes to obtain information for specific
reasons rather than ensuring a given process provided the information
needed from an enterprise point of view. Furthermore, the concern that
the version of software being used by GCSS-Army had not been certified
as compliant with federal requirements can be easily addressed. Army
officials have stated that the software vendor is committed to
obtaining the necessary certifications and expects that these will be
granted before GCSS-Army is deployed. From a corporate perspective,
whether one system or two systems is the proper approach is not
necessarily the question that the Army needs to answer. Rather, the
Army needs to address how best to utilize the capabilities of the ERP
to achieve an effective funds control system.
Disbursements:
The Department of Defense (DOD) Inspector General recently reported
that LMP has been implemented without the functionality to match
proposed disbursements with corresponding obligations before making any
payments.[Footnote 53] This process is referred to as prevalidation,
which was mandated with the passage of the fiscal year 1995 Department
of Defense Appropriations Act[Footnote 54] because of DOD's long-
standing difficulty in properly matching disbursements with
corresponding obligations. Implementing LMP without this functionality
is an example of suboptimizing the system, resulting in the failure to
address a long-standing weakness in matching obligations with proposed
disbursements for the Army Working Capital Fund. This functionality is
part of the basic software package that is being used by LMP. We noted
that the Army expects to use the same ERP software package for GFEBS.
The DOD Inspector General reported that, rather than reengineering its
business processes, the Army decided to follow its existing business
processes that were inefficient and precluded the Army from being in
compliance with the fiscal year 1995 legislative requirement. As a
result of this decision, the Army has not taken full advantage of the
capabilities within the ERP software package being used to implement
LMP. Considering that the Army has already invested approximately $452
million in LMP, and as previously noted is expecting to invest hundreds
of million more, taking full advantage of its capabilities is also a
prudent financial decision. Because the Army failed to reengineer its
business processes, it continues to use an off-line database, managed
by the Defense Finance and Accounting Service (DFAS), to compare
entitlement and accounting data and provide approval for prevalidation
requests. The DOD Inspector General report noted that the database
contained errors in the accounting data, which resulted in
prevalidation failures and the need for additional research and rework
by DFAS.
A more efficient and streamlined process could be achieved by adopting
ERP processes for disbursements. For example, by utilizing ERP concepts
LMP would be the accounting system of record and responsible for
ensuring that the applicable federal requirements governing fund
control and disbursements were effectively implemented. Further, the
disbursements would be made directly by the Department of the Treasury-
-much like the process used by civilian agencies--rather than going
through DFAS. We recognize that this approach involves some technical
issues, such as building the necessary interfaces with the Department
of the Treasury. However, the commercial-off-the-shelf (COTS) package
adopted by LMP is already required by its Office of Management and
Budget certification to provide the necessary support for these
interfaces. The elimination of the DFAS interfaces, which are not
inherently supported by the COTS package, should help reduce the cost
and risk associated with this change.
[End of section]
Appendix IV: LMP Problems Continue:
The Tobyhanna Army Depot continues to experience financial management
problems because significant operational and developmental issues
related to the Logistics Modernization Program (LMP) have not been
resolved. More specifically, LMP continues to experience problems with
accurately recognizing revenue and billing customers. This problem has
existed virtually since the implementation of LMP in July 2003. While
the Army and its contractor have made numerous attempts to fix the
problem, they have not been successful. While we found that the
requirements management processes have improved--a critical first step
in reducing risk to acceptable levels[Footnote 55]--we continue to have
reservations about the adequacy of LMP's testing.
Billing Problems Continue to Plague LMP:
LMP's continuing billing problems have resulted in (1) customers not
being billed for costs incurred that should have been billed and (2)
customers being billed too much. According to information provide by
the LMP program office, 146 customer orders valued at approximately
$5.4 million were not billed during the January 31, 2007, billing
cycle. Customer orders that were not recognized as revenue and billed
represent funds that Tobyhanna Army Depot is entitled to, because the
work requested by the customer was performed. It is essential that
revenue is properly recognized and customers billed for work performed
because it is the means by which the depot finances its day-to-day
operations. Similarly, during the January 31, 2007, billing cycle about
308 customer orders amounting to about $5.8 million were shown as being
overbilled and a credit should have been issued to the customer. It is
critical that the credit be provided to the customer as soon as
possible since the customer may be able to use these funds to meet
other funding demands.
Our analysis of the January 2007 billings for Tobyhanna disclosed that,
of the 146 unbilled customer orders, 61 (about 42 percent) related to
fiscal years 2006 and 2007. Similarly, for the credits, 178 of 308
customer orders (about 58 percent) were for fiscal years 2006 and 2007.
From a dollar perspective, most of these billings also related to
fiscal years 2006 and 2007 transactions. The fact that these billing
problems have persisted clearly indicates that the various efforts by
the Army to resolve this critical aspect of the system have failed.
Table 1 shows the dollar value and the number of customer orders
related to unbilled work and credits for 3 months--November and
December 2006 and January 2007.
Table 1: Unbilled and Credit Customer Orders at the Tobyhanna Army
Depot for November 2006 through January 2007:
Month: November 2006;
Unbilled customer orders: 84;
Dollar value of unbilled customer orders: (3,161,490);
Customer orders with credits: 291;
Dollar value of customer orders with credits: 3,156,533.
Month: December 2006;
Unbilled customer orders: 92;
Dollar value of unbilled customer orders: (2,637,903);
Customer orders with credits: 301;
Dollar value of customer orders with credits: 4,772,884.
Month: January 2007;
Unbilled customer orders: 146;
Dollar value of unbilled customer orders: (5,367,798);
Customer orders with credits: 308;
Dollar value of customer orders with credits: 5,792,851.
Source: LMP program office.
[End of table]
The continuing billing problem at Tobyhanna is also a factor
contributing to higher depot bills from the Defense Finance and
Accounting Service (DFAS)--the Department of Defense's (DOD)
centralized finance and accounting organization. According to DFAS
personnel, the recurring billing and other problems have resulted in
DFAS processing Tobyhanna's bills manually and performing other manual
actions that are not required by the other depots that do not use LMP.
According to the information provided by DFAS, the number of hours
spent on DFAS accounting services billings for Tobyhanna has increased
between fiscal years 2004 and 2006. Table 2 shows the number of hours
and related amounts DFAS billed Tobyhanna in each year.
Table 2: DFAS Billings to the Tobyhanna Army Depot for Accounting
Services:
Fiscal year: 2004;
Number of hours: 17,737;
Amount billed (dollars): 1,105,168.
Fiscal year: 2005;
Number of hours: 21,197;
Amount billed (dollars): 1,360,407.
Fiscal year: 2006;
Number of hours: 22,653;
Amount billed (dollars): 1,432,777.
Source: GAO based upon data provided by DFAS.
[End of table]
Furthermore, according to DFAS personnel, while the accounting needs of
other depots requires the support of one full-time DFAS staff, DFAS
uses three full-time personnel to service Tobyhanna due to the numerous
manual workarounds associated with LMP. The manual workarounds required
to address LMP's billing issues results in higher DFAS bills for
Tobyhanna, which in turn increases the prices charged by Tobyhanna to
its customers for the work performed. Since Tobyhanna is part of the
Army Working Capital Fund, the costs incurred for accounting services
are passed on to the customer in the form of higher depot prices.
During the course of our audit, LMP program officials stated that they
were finalizing efforts to revise the billing process. At completion of
our field work, the LMP program office stated that it anticipated
having the revised process in place in the August 2007 time frame.
Since billing and revenue recognition have been such a long-standing
issue, and given that the entire process is fundamental to the
efficient and effective operation of the Working Capital Fund, it is
incumbent upon the LMP program office to closely monitor the planned
corrective actions and obtain reasonable assurance that the revised
process is meeting the Army's expectations. System testing is a
critical process that should help improve an entity's confidence that
the system will satisfy the requirements of the end user and will
operate as intended. If the planned corrective actions do not resolve
the existing problem, Tobyhanna's operations will continue to be
adversely affected. As we have previously recommended, LMP should not
be deployed to any additional locations.[Footnote 56]
Accuracy of Financial Information Reported by LMP Still a Concern:
LMP continues to experience difficulty in accurately reporting on the
results of operations. For example, the DOD Inspector General's fiscal
year 2006 financial statement audit of the Army Working Capital Fund
identified instances in which LMP was not properly recording
transactions in accordance with the U.S. Government Standard General
Ledger requirements. Further, our analysis of the DFAS monthly report
on depot operations--commonly referred to as the 1307 report[Footnote
57]--found that inaccurate account balances continue to persist in LMP.
These issues have presented many challenges that are reflected
throughout the Army Working Capital Fund financial statements as
adjustments, corrections, and footnote disclosures. More specific
details on the problems are highlighted below.
* The DOD Inspector General Report on Internal Controls and Compliance
with Laws and Regulations noted that the nonfederal accounts payable
account was misstated by approximately $175 million at the end of
fiscal year 2006. This misstatement arose because Army entities using
LMP had not resolved approximately $89 million in abnormal accounts
payable balances reported by the Supply Management business area. An
abnormal account balance is one in which the recorded amount has been
incorrectly classified. During fiscal year 2006, the LMP program office
initiated systemic corrections to reduce the reported abnormal
balances. Our analysis of the first quarter fiscal year 2007 financial
statements noted that the reported balance had been reduced to
approximately $81 million. The LMP program office was continuing to
research the issue.
* Our analysis of the DFAS 1307 reports found that abnormal balances in
accounts payable have been reported since fiscal year 2004. LMP program
officials stated that abnormal balances were primarily due to the
migration of remaining obligations on open contracts from the legacy
systems. Substantial effort was made in fiscal year 2006 to correct the
abnormal accounts payable balances that were reported by LMP, which
resulted in a reduction from $334 million at the end of fiscal year
2005 to $88 million at the end of fiscal year 2006. The Army's efforts
to clean up abnormal account balances continued in fiscal year 2007.
The Army has developed a specific plan to address the financial
management problems recognized in these statements. For example, a
Tobyhanna Army Depot on-site support team has been assembled to
complete the data cleanup, validate that all system and procedural
issues have been identified, document business processes and
procedures, and train Tobyhanna users. However, until these problems
are corrected, LMP will not be able to provide reliable and accurate
financial information to the Congress or Army management.
Adequacy of LMP Testing Continues to Be a Concern:
Our prior audits[Footnote 58] of LMP identified significant weaknesses
with LMP's efforts to effectively implement the requirements management
and testing processes needed to reduce risks to acceptable levels.
During our current audit, we found that the requirements management
processes have improved--a critical first step--but we continue to have
reservations about the adequacy of the testing. LMP program officials
stated that these testing weaknesses are being addressed and expect
that an improved testing process will be effectively implemented to
reduce risks to acceptable levels.
LMP's Requirements Management Practices Have Improved:
During the current review, Army and contractor Computer Sciences
Corporation (CSC) officials stated that they have acted upon our prior
recommendations and taken steps to improve the program's requirements
management practices. In setting forth the revised requirements
management process, LMP and CSC officials noted that the new process
generally envisioned having several documents that outlined the
requirements that needed to be addressed with each document containing
specificity needed for the intended audience.
To ascertain if the Army's stated corrective actions were being adhered
to, we selected 10 requirements for detailed review. Our analysis found
that the LMP documentation in support of the requirements management
practices was generally in compliance with the stated process. More
specifically, the requirements reviewed were generally consistent
between the documents and appeared to contain the necessary specificity
to reduce requirement-related defects to acceptable levels. Our
observations were consistent with the conclusions reached by a
subsequent review performed by the LMP verification and validation
contractor, who examined the same documentation that we analyzed.
Concerns regarding LMP Testing Processes Remain:
Our review of the stated testing processes found that many of the
attributes associated with a disciplined testing process were present.
However, based upon our review and analysis of the 10 selected
corrective actions, we still have concerns that the LMP testing process
may not be adequate since we identified specific weaknesses in the 10
corrective actions reviewed. System testing is a critical process
utilized by disciplined organizations and improves an entity's
confidence that the system will satisfy the requirements of the end
user and operate as intended. The stated testing process included the
following:
* Documenting scenario testing. A scenario is a business process that
typically consists of several events. For example, a scenario could be
developed for small purchases, which would include events such as (1)
initiating a purchase request, (2) approving the request, (3)
obligating the necessary funds, (4) ordering the item, (5) receiving
the item, and (6) paying for the item. One benefit of testing scenarios
is that it helps to ensure that information is accurately passed from
one process to another. For instance, the denial of a requisition
prevents the purchase of the item.
* Linking requirements to the test cases. Linking requirements and test
cases helps ensure that either (1) all requirements are adequately
tested or (2) the risks associated with not testing a requirement can
be properly evaluated. Because it is not economically feasible to
develop a testing program that can find every defect, it is critical
that an approach be developed to ensure that testing resources are
focused on the areas of highest risk. Accordingly, linking the
requirements makes it easier to determine which combinations of items
are not tested, thereby enabling the Army to evaluate the risks of not
conducting those tests.
* Regression testing. Regression testing is a process that helps ensure
that changes made to the system have not adversely affected
functionality that was working prior to a change being made to the
system. In essence, it is a process that retests the entire system to
ensure that the problem fixed did not have an adverse effect on other
functions within the system.
While the processes described above are an improvement over what we
have seen in the past, we continue to identify weaknesses in the
testing process. For example, none of the test scripts provided
adequate information that linked the items in the test script to a
specific requirement. This linking is commonly referred to as
traceability and is characteristic of a disciplined testing process.
The test scripts reviewed contained headings for information on the
scenario, key data requirements, expected results, actual results, and
whether the test was considered successful. However, the actual test
script failed to contain the level of specificity that clearly
delineated how a specific requirement identified in the requirement
document(s) and the associated requirement(s) in that document were
being tested. As a result, it is virtually impossible to ascertain if
the requirement was properly tested. For example, several requirements
defined rules that were expected to be implemented. Without linking a
given requirement to the tests designed to exercise that requirement,
it was impossible to obtain reasonable assurance that (1) all the
requirements had been tested and (2) the test provided adequate
coverage for each requirement.
A review of the test script for one corrective action showed that the
identified expected results were "output of report should show open
accounts receivable items for customers in the range noted." Our
analysis of the documentation disclosed that the test scripts only
tested to make sure that a report could be printed for (1) a specific
customer, (2) a consecutive range of customers (e.g., customers 1
through 10), (3) federal and nonfederal customers (the exact condition
tested was not specified since the user was only required to pick one
of the conditions), and (4) receivables associated with a specified
general ledger account. While this testing addressed the requirements
that the report be printed using these conditions, the scripts did not
provide evidence that other requirements were tested and the data
presented were valid. For example, the report was expected to put
receivables into 11 different age categories (for example 0 to 30 days,
31 to 60 days, etc.); however, no evidence was provided to show that
the balances shown on these reports (1) were consistent with known
results and (2) the accounts selected provided adequate coverage of the
ranges specified. The latter is especially important since two of the
ranges overlap. LMP and CSC officials agreed that the documentation did
not always provide the level of detail that was needed to (1) document
which requirements contained in applicable requirement documents were
being tested and (2) ensure that all of the specific tests that were
required to provide reasonable assurance that the application worked as
expected were completed.
Furthermore, we were informed that the system testing was conducted by
the developers and subject matter experts. Using developers and subject
matter experts, and depending on those individuals' comprehensive body
of knowledge as the foundation for a testing effort, carries
significant risks since this is not a recognized best practice. A basic
testing principle is that a developer should not test his or her own
work, nor should a development organization test its own work. These
testing principles are based on the concept that testing is the process
of executing a program with the intent of finding errors. Testing is
normally considered a "destructive" process while the development
activities are normally considered a "constructive" progress. It is
very difficult for a developer to change the perspective required to
successfully develop a program into the perspective necessary to
adequately test a process.[Footnote 59] This does not mean that the
developers are not involved in the testing process since a best
practice is to have them responsible for testing the code they develop
to ensure that it is ready for the next stage of testing.[Footnote 60]
Further, as noted in our May 2004 report,[Footnote 61] our analysis of
LMP's December 2003 and January 2004 project status reports identified
numerous instances in which the Army continued to experience problems
with the accuracy of data related to budgeting; workload planning and
forecasting and depot maintenance operations; and accounting records
such as customer orders, purchase orders and requisitions, obligations,
and disbursements. DFAS and Army officials acknowledged that these
problems were attributable to relying on subject matter experts to
develop tests for their respective functional areas, and not performing
end-to-end testing across the various functional areas.
After discussing our concerns with the LMP program office and CSC
officials in February 2007, the LMP program office requested the
verification and validation (V&V) contractor to perform an assessment
of the LMP testing process. According to information provided by the
LMP program office and the V&V contractor, in March 2007, the review
found that the LMP testing process was adequately defined in the
planning documentation and noted that the LMP testing philosophy was
governed by the concepts that (1) the change should operate as
intended, (2) the change should not be harmful to the existing
functional environment, (3) testing should be practical, and (4) the
testing efforts should be cost efficient. The V&V contractor analysis
of the LMP testing process also found that the testing documentation--
which was the same documentation that we analyzed did not support a
clear understanding of the (1) requirement being tested and (2) tests
used to determine whether a requirement was adequately implemented.
According to the V&V contractor, these key areas need to be addressed.
The V&V contractor assessment substantiates our analysis of the LMP
testing process.
Improved Testing Process Planned:
LMP program office officials and the V&V contractor agreed that the
testing process currently being utilized could be improved. They also
noted that the LMP program had begun implementing an initiative to
improve the testing program that should address the weaknesses we
identified. The following outlines two key components of LMP's improved
testing efforts.
* Establishment of an independent test group. This is a critical step
and, if effectively implemented, can serve as a strong foundation for
building an effective testing process. An independent testing group is
a best practice.
* Development and implementation of an automated testing program. CSC
is developing an automated testing process which was expected to reduce
the testing burden and eliminate user errors in the testing process.
For example, the user may not detect that the application did not
provide the expected result or record an error when the application did
operate as expected. These are commonly referred to as testing defects
and are to be expected when manual testing activities are conducted.
LMP program officials are of the opinion that these two actions will
help address the problems of adequately linking the requirements to the
testing and documenting the testing that is actually performed. These
actions, if effectively implemented, are a significant step forward and
can help reduce the risks associated with testing to acceptable levels.
However, the effective implementation will have to overcome a number of
past problems, including ensuring that the tests provide adequate
coverage. While developing automated tests can improve the reliability
of the testing efforts and help facilitate an effective regression
testing program, the key is whether the automated tests are testing the
correct items. In our discussions with CSC, we were informed that the
existing test scripts were the basis for developing the new automated
test. Whether this approach will provide an effective testing process
is yet to be seen. As we noted above, the test scripts that will be
used as the basis for developing the automated test scripts do not
provide the necessary link to the requirements that are being tested or
provide the detailed information necessary to write the detailed test
scripts called for in the automated process.
[End of section]
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
McCoy Williams, (202) 512-9095 or williamsm1@gao.gov:
Keith A. Rhodes, (202) 512-6412 or rhodesk@gao.gov:
William M. Solis, (202) 512-8365 or solisw@gao.gov:
Acknowledgments:
In addition to the above contacts, the following individuals made key
contributions to this report: J. Christopher Martin, Senior-Level
Technologist; Darby Smith, Assistant Director; Evelyn Logue, Assistant
Director; F. Abe Dymond, Assistant General Counsel; Beatrice Alff;
Sunny Chang; Harold Brumm, Jr; Francine DelVecchio; K. Eric Essig;
Jason Kelly; Jason Kirwan; Alyson Mahan; and Debra Rucker.
FOOTNOTES
[1] Federal Accounting Standards and the Joint Financial Management
Improvement Program (JFMIP) define inventory, supplies, and materials
as consisting of three subclassifications: (1) inventory--tangible
personal property that is (a) held for sale, (b) in the process of
production for sale, or (c) to be consumed in the production of goods
for sale or in the provision of services for a fee; (2) operating
materials and supplies--tangible personal property to be consumed in
normal operations; and (3) stockpile materials--strategic and critical
materials held due to statutory requirements for use in national
defense, conservation, or national emergencies. Property, plant, and
equipment is defined as tangible assets that have an estimated useful
life of 2 or more years, are not intended for sale in the ordinary
course of business, and are intended to be used or available for use by
the entity.
[2] The reported amounts are net of allowances and depreciation, as
applicable.
[3] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.:
January 2007).
[4] The eight specific DOD high-risk areas are (1) business
transformation, (2) business systems modernization, (3) contract
management, (4) financial management, (5) personnel security clearance
program, (6) supply chain management, (7) support infrastructure
management, and (8) weapon systems acquisition.
[5] The seven high-risk areas that DOD shares responsibility for are
(1) disability programs, (2) information sharing for homeland security,
(3) information systems and critical infrastructures, (4) interagency
contracting, (5) human capital, (6) real property, and (7) technologies
critical to national security interests.
[6] GAO, DOD Business Systems Modernization: Billions Continue to Be
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and Army Depot Maintenance:
Ineffective Oversight of Depot Maintenance Operations and System
Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005).
[7] As of September 30, 2006, the Army reported net inventory and
related net property valued at about $39 billion for the Army general
fund and about $18 billion for the Army working capital fund. For the
same period, Army also reported net PP&E valued at over $84 billion for
the Army general fund and over $1 billion for the Army working capital
fund.
[8] Field/Tactical refers to Army units that are deployable to
locations around the world such as Iraq or Afghanistan.
[9] Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-56 (Oct. 28,
2004) (codified, in part, at 10 U.S.C. §§ 186, 2222).
[10] An enterprise architecture is a blueprint for organizational
change defined in models that describe (in both business and technology
terms) how the entity operates today and how it intends to operate in
the future; it also includes a plan for transitioning to this future
state.
[11] GAO, Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006).
[12] GAO, DOD Business Systems Modernization: Progress Continues to Be
Made in Establishing Corporate Management Controls, but Further Steps
Are Needed, GAO-07-733 (Washington, D.C.: May 14, 2007).
[13] See, for example, GAO, Information Technology: FBI Is Taking Steps
to Develop an Enterprise Architecture, but Much Remains to Be
Accomplished, GAO-05-363 (Washington, D.C.: Sept. 9, 2005); Homeland
Security: Efforts Under Way to Develop Enterprise Architecture, but
Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004); DOD
Business Systems Modernization: Limited Progress in Development of
Business Enterprise Architecture and Oversight of Information
Technology Investments, GAO-04-731R (Washington, D.C.: May 17, 2004);
Information Technology: Architecture Needed to Guide NASA's Financial
Management Modernization, GAO-04-43 (Washington, D.C.: Nov. 21, 2003);
and DOD Business Systems Modernization: Important Progress Made to
Develop Business Enterprise Architecture, but Much Work Remains, GAO-03-
1018 (Washington, D.C.: Sept. 19, 2003).
[14] Pub. L. No. 104-106, div. E, tit. LI, §§ 5122, 5125, 110 Stat.
186, 683, 685 (Feb. 10, 1996) (codified at 40 U.S.C. §§ 11312, 11315).
[15] OMB Circular No. A-130, Management of Federal Information
Resources, § 8(b) (Nov. 28, 2000).
[16] An ERP solution is an automated system using commercial off-the-
shelf (COTS) software consisting of multiple, integrated functional
modules that perform a variety of business-related tasks such as
payroll, general ledger accounting, and supply chain management.
[17] DOD identified the following four fundamental changes in logistics
practices that are needed to transform its logistics operations: (1)
customer wait time, (2) time-definite delivery, (3) TAV, and (4) Web-
based, shared data environment.
[18] GAO, Defense Inventory: DOD Could Improve Total Asset Visibility
Initiative with Results Act Framework, GAO/NSIAD-99-40 (Washington,
D.C.: Apr. 12, 1999).
[19] Pub. L. No. 104-208, div. A, §101(f), title VIII, 110 Stat. 3009,
3009-389 (Sept. 30, 1996).
[20] Pub. L. No. 101-576, 104 Stat. 2838 (Nov. 15, 1990).
[21] FFMIA requires the 24 Chief Financial Officers Act departments and
agencies to implement and maintain financial management systems that
comply substantially with (1) federal financial management systems
requirements, (2) applicable federal accounting standards, and (3) the
U.S. Government Standard General Ledger at the transaction level.
[22] The name of the command changed from Communications and
Electronics Command to the Communications-Electronics Life Cycle
Management Command in February 2005.
[23] Full operational capability means that the system has been
deployed to all intended locations.
[24] The contractor has submitted claims totaling $850 million to the
Army. According to the LMP Program Office, these claims are being
reviewed by the Army contracting officer, who is expected to make a
decision by June 2007.
[25] The Army Corps of Engineers will continue to use its existing
financial system--Corps of Engineers Financial Management System.
[26] There are four tiers of business systems. Tier 1 systems include
all large, expensive system programs classified as a "major automated
information system" (MAIS) or a "major defense acquisition program"
(MDAP) and subject to the most extensive statutory and regulatory
reporting requirements. Tier 2 systems include those with modernization
efforts of $10 million or greater but that are not designated as MAIS
or MDAP or programs that have been designated as IRB interest programs
because of their effect on DOD transformation objectives. Tier 3
systems include those with modernization efforts that have anticipated
costs greater than $1 million but less than $10 million. Tier 4 systems
are those with modernization efforts that have anticipated costs of up
to $1 million.
[27] Certification reviews apply to business system development/
modernization programs with a total cost of over $1 million. This
certification review focuses on program alignment with the business
enterprise architecture and must be completed before components
obligate funds for programs. The annual review applies to all business
programs. The focus of the annual review is to determine whether the
system development effort is in compliance with the business enterprise
architecture, meeting its milestones, and addressing IRB certification
conditions.
[28] DOD defines a mission area as a defined area of responsibility
with functions and processes that contribute to mission accomplishment.
The Army maps its mission areas to the four Global Information Grid
Enterprise Services Mission Areas delineated by DOD, including:
Warfighting, Business, Enterprise Information Environment, and the DOD
portion of National Intelligence. The Business Mission Area includes
six domains: (1) acquisition, (2) financial management, (3) human
resources management, (4) logistics, (5) installations and environment,
and (6) civil works.
[29] GAO-04-615.
[30] GAO-05-441.
[31] GAO, Defense Inventory: Improvements Needed in DOD's
Implementation of Its Long-Term Strategy for Total Asset Visibility of
Its Inventory, GAO-05-15 (Washington, D.C.: Dec. 6, 2004).
[32] DOD, Department of Defense Directive Number 5000.1 (May 12, 2003);
Department of Defense Architecture Framework, Version 1.0, Volume 1
(February 2004).
[33] Pub. L. No. 104-106, div. E, tit. LI, §§ 5122, 5125; E-Government
Act of 2002, Pub. L. No. 107-347 § 101, 116 Stat. 2899, 2901-2910 (Dec.
17, 2002) codified at 44 U.S.C. §§ 3601-06); OMB Circular No. A-130, §
8(b); GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G
(Washington, D.C.: April 2003); Chief Information Officer Council, A
Practical Guide to Federal Enterprise Architecture, Version 1.0
(February 2001); and Institute of Electrical and Electronics Engineers,
Standard for Recommended Practice for Architectural Description of
Software-Intensive Systems 1471-2000 (Sept. 21, 2000).
[34] GAO-06-831.
[35] The five DOD architecture programs that were included in our audit
were the Departments of the Army, Navy, and Air Force, and the DOD
business enterprise architecture and the DOD global information grid.
[36] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G
(Washington, D.C.: April 2003).
[37] GAO-07-733.
[38] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.:
January 2007).
[39] The other two focus areas related to supply chain management are
requirements forecasting and materiel distribution.
[40] Thomas F. Wallace and Michael H. Kremzar, ERP: Making It Happen,
(New York, N.Y.: John Wiley & Sons, Inc. 2001).
[41] An economic analysis of Army's LMP initiative is not required
because it is considered a procurement of a service as opposed to an
actual system, and therefore is not subject to the criteria established
by Department of Defense Directive 5000.1 and Department of Defense
Instruction 5000.2. Subsequent to the draft of this report being
submitted to DOD for comment, we were informed that the Office of the
Deputy Assistant Secretary of the Army for Cost and Economics had
determined that GCSS-Army is economically viable. We have not evaluated
the results of that analysis.
[42] GAO, Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments, GAO-
07-538 (Washington, D.C.: May 11, 2007).
[43] U.S. Army, Army Knowledge Management (AKM) Guidance Memorandum,
Capabilities-Based Information Technology (IT) Portfolio Governance
Implementing Guidance (Jan. 5, 2006).
[44] According to the Institute of Electrical and Electronics
Engineers, verification and validation processes for projects such as
the Army ERPs can be used to determine whether (1) the products of a
given activity conform to the requirements of that activity and (2) the
software satisfies its intended use and user needs. This determination
may include analyzing, evaluating, reviewing, inspecting, assessing,
and testing software products and processes. The verification and
validation processes should assess the software in the context of the
system, including the operational environment, hardware, interfacing
software, operators, and users.
[45] OMB Circular No. A-94, Guidelines and Discount Rates for Benefit-
Cost Analysis of Federal Programs (Oct. 29, 1992); Department of
Defense Instruction 7041.3, Economic Analysis for Decisionmaking (Nov.
7, 1995); Department of the Army, U.S. Army Cost and Economic Analysis
Center, Economic Analysis Manual (February 2001).
[46] OMB Circular No. A-94, § 6(a)(4).
[47] OMB Circular No. A-94, § 9; Department of Defense Instruction
7041.3, § E3.2.2.
[48] GAO-04-615 and GAO-05-441.
[49] Ronald W. Reagan National Defense Authorization Act for Fiscal
Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-56 (Oct.
28, 2004) (codified, in part, at 10 U.S.C. §§ 186, 2222).
[50] GAO, DOD Business Systems Modernization: Billions Continue to Be
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and Army Depot Maintenance:
Ineffective Oversight of Depot Maintenance Operations and System
Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005).
[51] We reviewed the 1307 reports for March 2004, September 2004, March
2005, September 2005, March 2006, September 2006, December 2006, and
February 2007.
[52] Pub. L. No. 104-208, div. A, §101(F), title VIII, 110 Stat. 3009,
3009-389 (Sept. 30, 1996). FFMIA requires the 24 Chief Financial
Officers Act departments and agencies to implement and maintain
financial management systems that comply substantially with (1) federal
financial management system requirements, (2) applicable federal
accounting standards, and (3) the U.S. Government Standard General
Ledger at the transaction level.
[53] DOD Inspector General, Controls Over the Prevalidation of DOD
Commercial Payments, Report Number D2007-065 (Arlington, Va.: Mar. 2,
2007).
[54] Pub. L. No. 103-335, § 8137, 108 Stat. 2599, 2654-55 (Sept. 30,
1994).
[55] Acceptable levels refer to the fact that any systems acquisition
effort will have risks and will suffer the adverse consequences
associated with defects in the processes. However, effective
implementation of disciplined processes reduces the possibility of the
potential risks actually occurring and prevents significant defects
from materially affecting the cost, timeliness, and performance of the
project.
[56] GAO, Army Depot Maintenance: Ineffective Oversight of Depot
Maintenance Operations and System Implementation Efforts, GAO-05-441
(Washington, D.C.: June 30, 2005).
[57] We reviewed the 1307 reports for March 2004, September 2004, March
2005, September 2005, March 2006, September 2006, December 2006, and
February 2007.
[58] GAO, DOD Business Systems Modernization: Billions Continue to Be
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004) and GAO-05-441.
[59] Glenford J. Myers, The Art of Software Testing (New York, N.Y.:
John Wiley & Sons, Inc., 1979).
[60] Rex Black, Critical Testing Processes: Plan, Prepare, Perform,
Perfect (Boston, Mass.: Addison-Wesley, 2004).
[61] GAO-04-615.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400:
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800:
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
