DOD Business Systems Modernization
Improvements to Enterprise Architecture Development and Implementation Efforts Needed Gao ID: GAO-03-458 February 28, 2003The Department of Defense (DOD) is developing an enterprise architecture, or corporate modernization blueprint, to guide and constrain its ongoing and planned business system investments. GAO was asked to review DOD's processes and controls for developing the enterprise architecture and ensuring that ongoing IT investments are consistent with its enterprise architecture development efforts.
DOD has undertaken a challenging and ambitious task to, within 1 year, develop a department-wide blueprint for modernizing its over 1,700 timeworn, inefficient, and nonintegrated business processes and supporting information technology (IT) assets. Such a blueprint, commonly called an enterprise architecture, is an essential modernization management tool. We support the Secretary of Defense's decision to develop an architecture and the department's goal of acquiring systems that provide timely, reliable, and relevant information. Successfully doing so requires the application of effective enterprise architecture and IT investment management processes and controls. While DOD is following some of these enterprise architecture practices, it is not following others, in part because it is focused on meeting its ambitious schedule. More specifically, with respect to developing the architecture, DOD has yet to (1) establish the requisite architecture development governance structure and process controls needed to ensure that ownership of and accountability for the architecture are vested with senior leaders across the department, (2) clearly communicate to intended architecture stakeholders the purpose, scope, and approach to developing the initial and subsequent versions of the architecture, and their roles and responsibilities, and (3) define and implement an independent quality assurance process. Until it follows these practices, DOD increases the risk of developing an architecture that will be limited in scope, be resisted by those responsible for implementing it, and will not support effective systems modernization. DOD has taken initial steps aimed at improving its management of ongoing business system investments. However, DOD has yet to establish the necessary departmental investment governance structure and process controls needed to adequately align ongoing investments with its architectural goals and direction. Instead, DOD continues to allow its component organizations to make their own parochial investment decisions, following different approaches and criteria. This stove-piped decision-making process has contributed to the department's current complex, error-prone environment of over 1,700 systems. In particular, DOD has not established and applied common investment criteria to its ongoing IT system projects using a hierarchy of investment review and funding decision-making bodies, each composed of representatives from across the department. DOD also has not yet conducted a comprehensive review of its ongoing IT investments to ensure that they are consistent with its architecture development efforts. Until it takes these steps, DOD will likely continue to lack effective control over the billions of dollars it is currently spending on IT projects.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-458, DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed
This is the accessible text file for GAO report number GAO-03-458
entitled 'DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed'
which was released on February 28, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Chairman and Ranking Minority Member, Subcommittee on
Readiness and Management Support, Committee on Armed Services, U.S.
Senate:
February 2003:
DOD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation
Efforts Needed:
GAO-03-458:
GAO Highlights:
Highlights of GAO-03-458, a report to the Chairman and Ranking Minority
Member, Subcommittee on Readiness and Management Support, Committee on
Armed
Services, U.S. Senate:
February 2003:
DOD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation
Efforts Needed:
Why GAO Did This Study:
The Department of Defense (DOD) is developing an enterprise
architecture,
or corporate modernization blueprint, to guide and constrain its
ongoing and
planned business system investments. GAO was asked to review DOD‘s
processes
and controls for developing the enterprise architecture and ensuring
that
ongoing IT investments are consistent with its enterprise architecture
development efforts.
What GAO Found:
DOD has undertaken a challenging and ambitious task to, within 1 year,
develop
a departmentwide blueprint for modernizing its over 1,700 time-worn,
inefficient,
and nonintegrated business processes and supporting information
technology (IT)
assets. Such a blueprint, commonly called an enterprise architecture,
is an
essential modernization management tool. We support the Secretary of
Defense‘s
decision to develop an architecture and the department‘s goal of
acquiring systems
that provide timely, reliable, and relevant information.
Successfully doing so requires the application of effective enterprise
architecture and IT investment management processes and controls. While
DOD is
following some of these enterprise architecture practices, it is not
following
others, in part because it is focused on meeting its ambitious
schedule. More
specifically, with respect to developing the architecture, DOD has yet
to (1)
establish the requisite architecture development governance structure
and process
controls needed to ensure that ownership of and accountability for the
architecture are vested with senior leaders across the department, (2)
clearly
communicate to intended architecture stakeholders the purpose, scope,
and
approach to developing the initial and subsequent versions of the
architecture,
and their roles and responsibilities, and (3) define and implement an
independent
quality assurance process. Until it follows these practices, DOD
increases the
risk of developing an architecture that will be limited in scope, be
resisted by
those responsible for implementing it, and will not support effective
systems
modernization.
DOD has taken initial steps aimed at improving its management of
ongoing
business system investments. However, DOD has yet to establish the
necessary
departmental investment governance structure and process controls
needed to
adequately align ongoing investments with its architectural goals and
direction.
Instead, DOD continues to allow its component organizations to make
their own
parochial investment decisions, following different approaches and
criteria.
This stovepiped decision-making process has contributed to the
department‘s
current complex, error-prone environment of over 1,700 systems. In
particular,
DOD has not established and applied common investment criteria to its
ongoing
IT system projects using a hierarchy of investment review and funding
decision-
making bodies, each composed of representatives from across the
department. DOD
also has not yet conducted a comprehensive review of its ongoing IT
investments
to ensure that they are consistent with its architecture development
efforts.
Until it takes these steps, DOD will likely continue to lack effective
control
over the billions of dollars it is currently spending on IT projects.
What GAO Recommends:
To assist DOD in successfully developing an enterprise architecture and
using
it to gain control over its ongoing business system investments, we are
making
recommendations to the Secretary of Defense to ensure that DOD (1)
expands its
use of effective architecture development processes and controls and
(2)
strengthens controls over its ongoing business systems investments.
DOD concurred with our recommendations and described recently
completed, ongoing,
and planned efforts to address them.
www.gao.gov/cgi-bin/getrpt?GAO-03-458 To view the full report,
including
the scope and methodology, click on the link above. For more
information,
contact Gregory Kutz, (202) 512-9095 (kutzg@gao.gov) or Randolph Hite,
(202) 512-3439 (hiter@gao.gov).
Contents:
Letter:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Briefing to Subcommittee Staff:
Appendix II: Comments from the Under Secretary of Defense:
Appendix III: GAO Contacts and Staff Acknowledgements:
GAO Contacts:
Acknowledgments:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its
entirety without further permission from GAO. It may contain
copyrighted
graphics, images or other materials. Permission from the copyrighted
materials separately from GAO‘s product.
Letter February 28, 2003:
The Honorable John Ensign
Chairman
The Honorable Daniel K. Akaka
Ranking Minority Member
Subcommittee on Readiness and Management Support
Committee on Armed Services
United States Senate:
In May 2001,[Footnote 1] we reported that the Department of Defense
(DOD) had neither an enterprise architecture for its financial and
financial-related business operations, nor the management structure,
processes, and controls in place to effectively develop and implement
one. In September 2002, the Secretary of Defense designated improving
financial management operations, which include not only finance and
accounting but also business areas such as logistics, acquisition, and
personnel management, as 1 of the department‘s top 10 priorities. In
addition, the Secretary established a program to develop and implement
an enterprise architecture. In response to your request, we determined
whether DOD is (1) following effective processes and controls in
developing its enterprise architecture and (2) ensuring that ongoing
information technology (IT) investments are consistent with its
enterprise architecture development efforts.
On January 31, 2003, we briefed your offices on the results of this
review and the recommendations we are making to the Secretary of
Defense. This report transmits those briefing materials, including our
scope and methodology, as appendix I.
DOD has undertaken a challenging and ambitious task to, within 1 year,
develop a departmentwide enterprise architecture (blueprint) for
modernizing its business operations and systems. We support DOD‘s goals
of developing an architecture to guide and constrain its modernization
efforts and acquiring systems that provide timely, reliable, and
relevant information. Toward these goals, DOD has taken a number of
positive steps, including:
* designating improving financial management operations as 1 of its top
10 priorities;
* establishing a program office responsible for managing the enterprise
architecture development effort;
* capturing key data needed to develop the ’As Is“ architecture, such
as documenting its inventory of over 1,700 business systems; and:
* requiring DOD Comptroller review and approval of IT investments that
meet certain criteria.
Our May 2001[Footnote 2] report provided a number of fundamental steps
on how DOD should approach the development of its enterprise
architecture. At that time, we had also recommended that the department
limit business system investments until the enterprise architecture is
developed. While DOD has taken some positive actions, as specified
above, the department has yet to implement some of our recommendations
and certain best practices for developing and implementing the
architecture. The following discussion summarizes those key practices
DOD has yet to employ.
Architecture Development:
Successful architecture development requires the application of proven
management practices. Thus far, DOD has not implemented certain
practices. Specifically, DOD has yet to:
* establish the requisite architecture development governance structure
needed to ensure that ownership of and accountability for the
architecture is vested with senior leaders across the department;
* develop and implement a strategy to effectively communicate the
purpose and scope, approach to, and roles and responsibilities of
stakeholders in developing the enterprise architecture; and:
* fully define and implement an independent quality assurance process.
Not implementing these practices increases DOD‘s risk of developing an
architecture that will be limited in scope, be resisted by those
responsible for implementing it, and will not support effective systems
modernization. DOD recognizes the need to follow these practices, and
attributes its delays in doing so to tight schedule demands,
unawareness of certain best practices, and competing resource
priorities. Among other things, it plans to strengthen the architecture
governance and management structure.
Architecture Implementation and Control of Ongoing Investments:
Ensuring that ongoing IT investments are consistent with DOD‘s
architecture development efforts requires the application of proven
investment management practices. To date, DOD has not implemented
certain practices. Specifically, DOD has yet to establish an investment
management governance structure that includes:
* a hierarchy of investment review boards composed of representatives
from across the department who are assigned investment selection and
control responsibilities based on project threshold criteria;
* a standard set of investment review and decision-making criteria for
use by all boards, including criteria to ensure architectural
compliance and consistency; and:
* a specified, near-term date by which ongoing investments have to be
subjected to this investment review process, and by which decisions
should be made as to whether to proceed with each investment.
Until the investment management governance structure is established,
DOD component organizations will continue to make their own parochial
investment decisions, following different approaches and criteria. As
we have previously reported,[Footnote 3] this stovepiped decision-
making process has contributed to the department‘s current complex,
error-prone systems environment. This deeply embedded cultural
resistance to a more holistic decision-making process is a substantial
risk to successful development and implementation of the enterprise
architecture. DOD‘s leadership plans to strengthen its governance and
oversight over ongoing IT investments.
Recommendations for Executive Action:
To assist DOD in its efforts to effectively develop and implement an
enterprise architecture, and guide and constrain its business system
investments, and to address the problems discussed during the briefing,
we reiterate the recommendations that we made in our May 2001
report[Footnote 4] that DOD has yet to implement. In addition, we
recommend that the Secretary of Defense ensure that:
* the enterprise architecture executive committee members are
singularly and collectively made explicitly accountable to the
Secretary for delivery of the enterprise architecture, including
approval of each version of the architecture;
* the enterprise architecture program is supported by a proactive
marketing and communication program; and:
* the quality assurance function (1) includes the review of adherence
to process standards and reliability of reported program performance,
(2) is made independent of the program management function, and (3) is
not performed by subject matter experts involved in the development of
key architecture products.
Additionally, we recommend that the Secretary gain control over ongoing
IT investments by:
* establishing a hierarchy of investment review boards, each
responsible and accountable for selecting and controlling investments
that meet defined threshold criteria, and each composed of the
appropriate level of executive representatives, depending on the
threshold criteria, from across the department;
* establishing a standard set of criteria to include (1) alignment and
consistency with the DOD enterprise architecture and (2) our open
recommendations governing limitations in business system investments
pending development of the architecture; and:
* directing these boards to immediately apply these criteria in
completing reviews of all ongoing IT investments, and to not fund
investments that do not meet these criteria unless they are otherwise
justified by explicit criteria waivers.
Agency Comments and Our Evaluation:
In written comments on a draft of this report (see appendix II), the
Under Secretary of Defense (Comptroller) stated that the department
concurred with our recommendations and described recently completed,
ongoing, and planned efforts to address them. For example, the
department stated that it is currently developing a new architecture
development and implementation governance structure and a marketing and
communication strategy to facilitate ongoing development activities.
Additionally, the department stated that it is providing for the
independence of its quality assurance function by organizationally
moving it under the Director of Business Modernization and Systems
Integration. DOD stated that it would also require the reporting of
quality assurance information to the architecture Executive Steering
Committee. We did not verify or evaluate the extent to which the
efforts described in DOD‘s comments will address our recommendation.
Regarding the scope of quality assurance reviews, the department stated
that it has established and implemented a quality assurance function
that includes review of architecture products, program performance, and
architecture development process standards. We agree that the quality
assurance function includes review of architecture products. However,
neither during our review nor in its comments did the department
provide documentary evidence to support that its quality assurance
reviews address program performance and adherence to architecture
development process standards. Further, as stated in our report,
quality assurance function officials told us that they were never
tasked to perform such reviews.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of other Senate and House committees and subcommittees
that have jurisdiction and oversight responsibilities for the
Department of Defense. We are also sending copies to the Secretary of
Defense; the Secretary of the Army; the Secretary of the Navy; the
Secretary of the Air Force; the Under Secretary of Defense
(Comptroller); the Under Secretary of Defense (Acquisition, Technology,
and Logistics); the Under Secretary of Defense (Personnel & Readiness);
the Assistant Secretary of Defense (Command, Control, Communications,
and Intelligence); the Director of the Defense Finance and Accounting
Service; and the Director of the Office of Management and Budget.
Copies will also be available at no charge on our Web site at
www.gao.gov.
Should you or your staff have any questions on matters discussed in
this report, please contact us at (202) 512-9095 or (202) 512-3439,
respectively. We can also be reached by e-mail at kutzg@gao.gov or
hiter@gao.gov. GAO contacts and key contributors to this report are
listed in appendix III.
Gregory D. Kutz
Director, Financial Management and Assurance:
Signed by Gregory D. Kutz:
Randolph C. Hite
Director, Information Technology Architecture
and System Issues:
Signed by Randolph C. Hite:
[End of section]
Appendixes:
Appendix I: Briefing to Subcommittee Staff:
[See PDF for image]
[End of figure]
[End of section]
Appendix II: Comments from the Under Secretary of Defense:
UNDER SECRETARY OF DEFENSE:
1100 DEFENSE PENTAGON WASHINGTON, DC 20301-1100:
COMPTROLLER:
FEB 27 2003:
Mr. Gregory Kurz Director:
Financial Management and Assurance United States General Accounting
Office Washington, DC 20548:
Dear Mr. Katz,
My original reply of February 21, 2003, is rescinded. This is the
Department of Defense (DoD) response to the most recent GAO draft
report, ’DoD Business Systems Modernization: Improvements to Enterprise
Architecture Development and Implementation Efforts Needed,“ dated
February 7, 2003, (GAO Code 192070/GAO-03-458).
We concur in the recommendations of this draft report. The DoD comments
to the draft GAO recommendations are enclosed. My point of contact for
this matter is Mr. Steven Worton, Director for Business Modernization
and Systems Integration. Mr. Worton may be contacted by e-mail at
wortons@ osd.pentagon.mil or by telephone at (703) 607-3370.
Signed by Dov S. Zakheim:
Enclosure: As stated:
DoD Comments to GAO Draft Report: ’DoD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation
Efforts Needed,“ dated February 7, 2003, (GAO Code 192070/GAO-03-458):
GAO Recommendation 1: The GAO recommended that the Secretary of Defense
ensure that the enterprise architecture executive committee members are
singularly and collectively made explicitly accountable to the
Secretary for delivery of the enterprise architecture, including
approval of each version of the architecture.
DoD Comment to GAO Recommendation 1: Concur. The Department currently
is developing a governance structure that will include detailed
responsibilities for the enterprise architecture (EA) executive
committee, the Financial Management Modernization Program (FMMP)
Steering Committee, the FMMP Domain Owners, and the FMMP Domain Owners‘
Lead Agents. Included in these detailed responsibilities will be the
stipulation that the EA executive committee members are singularly and
collectively accountable to the Secretary for delivery of the
enterprise architecture. The proposed governance structure is in the
review and coordination phase. We plan for the governance structure to
be approved in March 2003.
GAO Recommendation 2: The GAO recommended that the Secretary of Defense
ensure that the enterprise architecture program is supported by a
proactive marketing and communication program.
DoD Response to GAO Recommendation 2: Concur. The Department agrees
that a proactive marketing and communication program is essential to
eliciting best participation by the cross-functional DoD
representatives managing development of the EA. In this regard, the
Department awarded a change management contract in December 2002, for
the purpose of developing a change and communication management
strategy and plan, obtaining the input and advice of the EA‘s
stakeholders, and communicating the anticipated changes to various
levels and stakeholders within the Department and to the public. We
plan on the change and communication management strategy and plan being
completed by April 30, 2003. This plan is an ongoing process which will
be further refined by the Domain Owners and adjusted as necessary.
GAO Recommendation 3: The GAO recommended that the Secretary of Defense
ensure that the quality assurance function: 1) includes the review of
adherence to process standards and reliability of reported program
performance, 2) is made independent of the program management function,
and 3) is not performed by subject matter experts involved in the
development of key architecture products.
DoD Response to GAO Recommendation 3: Concur. The Department has
established and implemented a quality assurance (QA) process that
includes architecture products, program performance using earned value
management (EVM), and process standards. The Program Office will
document the QA process currently being performed by April 30, 2003.
Further, we have a contractor providing independent verification and
validation (IV&V) assessment of all contractual deliverables, including
the architecture products.
We agree that, currently, the quality assurance (QA) function is not
independent of the program management function, but is independent of
the architecture development group. The QA function was moved from the
architecture group in January 2003 and reports directly to the
Director, Business Modernization and Systems Integration. The QA staff
is not involved in architecture development processes. We will begin
reporting Quality Assurance information to the Executive Steering
Committee to mitigate this factor beginning March 2003.
GAO Recommendation 4: The GAO recommended that the Secretary of Defense
gain control over ongoing IT investments by establishing a hierarchy of
investment review boards, each responsible and accountable for
selecting and controlling investments that meet defined threshold
criteria, and each composed of appropriate level of executive
representatives, depending on the threshold criteria, from across the
Department.
DoD Response to GAO Recommendation 4: Concur. The Governance Strategy
contains a requirement for an investment review board (IRB). The IRB
will utilize a ’portfolio management“ approach. They will consider both
new procurements and current system modifications. The details of the
operations and structure of the IRB are planned to be completed March
31, 2003. We welcome any input GAO can provide and will invite GAO to
these meetings.
GAO Recommendation 5: The GAO recommended that the Secretary of Defense
gain control over ongoing IT investments by establishing a standard set
of criteria, to include 1) alignment and consistency with the DoD
enterprise architecture, and 2) GAO‘s prior recommendations governing
limitations in business system investments pending development of the
architecture.
DoD Response to GAO Recommendation 5: Concur. DoD has taken a variety
of steps to limit IT investments that may be inconsistent with the
architecture. First, we have issued guidance for use by the FMMP
Systems Review team in the system life cycle milestone decision
process. This guidance includes standards for EA compliance, economic
and business case analysis, return on investment requirements, and
compliance with the ’Federal Financial Management Improvement Act of
1996.“ Secondly, the USD(C) has issued two memoranda that outline
system review requirements. One memorandum relates to Enterprise
Resource Planning (ERP) initiatives (August 21, 2002) and one
memorandum relates to new system development or modifications to
existing systems (October 12, 2002). These memoranda stipulate USD(C)
certification and approval requirements. Finally, the Governance
Structure will incorporate an IRB with defined processes.
GAO Recommendation 6: The GAO recommended that the Secretary of Defense
gain control over ongoing IT investments by directing investment review
boards to immediately apply the GAO defined criteria in completing
reviews of all ongoing Information Technology (IT) investments, and to
not fund investments that do not meet these criteria unless they are
otherwise justified by explicit criteria waivers.
DoD Response to GAO Recommendation 6: Concur. We intend that the
actions of the FMMP System Review team and the Governance Strategy will
lead to an improved control over IT investments.
[End of section]
Appendix III: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Cynthia Jackson, (202) 512-5086
Jenniffer Wilson, (202) 512-9192:
Acknowledgments:
In addition to the individuals named above, key contributors to this
report included Johnny Bowen, Teressa Broadie-Gardner, Christopher
DePerro, Eric Essig, Brian Johnson, Neelaxi Lakhmani, John Ledford,
Evelyn Logue, Mai Nguyen, Sanford Reigle, Darby Smith, Stacey Smith, Al
Steiner, and Randolph Tekeley.
FOOTNOTES
[1] U.S. General Accounting Office, Information Technology:
Architecture Needed to Guide Modernization of DOD‘s Financial
Operations, GAO-01-525 (Washington, D.C.: May 17, 2001).
[2] GAO-01-525.
[3] U.S. General Accounting Office, DOD Financial Management: Important
Steps Underway But Reform Will Require a Long-term Commitment, GAO-02-
784T (Washington, D.C.: June 4, 2002).
[4] GAO-01-525.
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
