DOD Business Systems Modernization
Long-Standing Weaknesses in Enterprise Architecture Development Need to Be Addressed Gao ID: GAO-05-702 July 22, 2005The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 directed the Department of Defense (DOD) to develop, by September 2005, a well-defined business enterprise architecture (BEA) and a transition plan. GAO has made numerous recommendations to assist the department in successfully doing so. As part of ongoing monitoring of the architecture, GAO assessed whether the department had (1) established an effective governance structure; (2) developed program plans, including supporting workforce plans; (3) performed effective configuration management; (4) developed well-defined BEA products; and (5) addressed GAO's other recommendations.
To effectively and efficiently modernize its nonintegrated and duplicative business operations and systems, it is essential for DOD to develop and use a well-defined BEA. However, it does not have such an architecture, and the products that it has produced do not provide sufficient content and utility to effectively guide and constrain ongoing and planned systems investments. As a result, despite spending almost 4 years and about $318 million, DOD does not have an effective architecture program. The current state of the program is due largely to long-standing architecture management weaknesses that GAO has made recommendations over the last 4 years to correct. In particular, DOD has not done the following: (1) established an effective governance structure, including an effective communications strategy, to achieve stakeholder buy-in. In particular, the structure that has been in place since 2001 lacks the requisite authority and accountability to be effective, and the key entities that made up this structure have not performed according to their respective charters; (2) developed program plans that explicitly identify measurable goals and outcomes to be achieved, nor has it defined the tasks to be performed to achieve the goals and outcomes, the resources needed to perform these tasks, or the time frames within which the tasks will be performed. DOD also has not assessed, as part of its program planning, the workforce capabilities that it needs in order to effectively manage its architecture efforts, nor does it have a strategy for doing so; (3) performed effective configuration management, which is a formal approach to controlling product parts to ensure their integrity. The configuration management plan and the charter for the configuration control board are draft; the board has limited authority; and, after 4 years of development, the department has not assigned a configuration manager; (4) developed a well-defined architecture. The latest versions of the architecture do not include products describing the "As Is" business and technology environments and a transition plan for investing in business systems. In addition, the versions that have been produced for the "To Be" environment have not had a clearly defined purpose and scope, are still missing important content, and contain products that are neither consistent nor integrated; and (5) fully addressed other GAO recommendations. DOD recognizes that these weaknesses need to be addressed and has recently assigned a new BEA leadership team. DOD also has either begun steps to or stated its intentions to (1) revise its governance structure; (2) develop a program baseline, by September 30, 2005, that will be used as a managerial and oversight tool to allocate resources, manage risks, and measure and report progress; and (3) revise the scope of the architecture and establish a new approach for developing it. However, much remains to be accomplished to establish an effective architecture program. Until it does, its business systems modernization effort will remain a high-risk program.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-702, DOD Business Systems Modernization: Long-Standing Weaknesses in Enterprise Architecture Development Need to Be Addressed
This is the accessible text file for GAO report number GAO-05-702
entitled 'DOD Business Systems Modernization: Long-standing Weaknesses
in Enterprise Architecture Development Need to Be Addressed' which was
released on July 25, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
July 2005:
DOD Business Systems Modernization:
Long-standing Weaknesses in Enterprise Architecture Development Need to
Be Addressed:
GAO-05-702:
GAO Highlights:
Highlights of GAO-05-702, a report to congressional committees:
Why GAO Did This Study:
The Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005 directed the Department of Defense (DOD) to develop, by September
2005, a well-defined business enterprise architecture (BEA) and a
transition plan. GAO has made numerous recommendations to assist the
department in successfully doing so. As part of ongoing monitoring of
the architecture, GAO assessed whether the department had (1)
established an effective governance structure; (2) developed program
plans, including supporting workforce plans; (3) performed effective
configuration management; (4) developed well-defined BEA products; and
(5) addressed GAO‘s other recommendations.
What GAO Found:
To effectively and efficiently modernize its nonintegrated and
duplicative business operations and systems, it is essential for DOD to
develop and use a well-defined BEA. However, it does not have such an
architecture, and the products that it has produced do not provide
sufficient content and utility to effectively guide and constrain
ongoing and planned systems investments. As a result, despite spending
almost 4 years and about $318 million, DOD does not have an effective
architecture program. The current state of the program is due largely
to long-standing architecture management weaknesses that GAO has made
recommendations over the last 4 years to correct. In particular, DOD
has not done the following:
* Established an effective governance structure, including an effective
communications strategy, to achieve stakeholder buy-in. In particular,
the structure that has been in place since 2001 lacks the requisite
authority and accountability to be effective, and the key entities that
made up this structure have not performed according to their respective
charters.
* Developed program plans that explicitly identify measurable goals and
outcomes to be achieved, nor has it defined the tasks to be performed
to achieve the goals and outcomes, the resources needed to perform
these tasks, or the time frames within which the tasks will be
performed. DOD also has not assessed, as part of its program planning,
the workforce capabilities that it needs in order to effectively manage
its architecture efforts, nor does it have a strategy for doing so.
* Performed effective configuration management, which is a formal
approach to controlling product parts to ensure their integrity. The
configuration management plan and the charter for the configuration
control board are draft; the board has limited authority; and, after 4
years of development, the department has not assigned a configuration
manager.
* Developed a well-defined architecture. The latest versions of the
architecture do not include products describing the ’As Is“ business
and technology environments and a transition plan for investing in
business systems. In addition, the versions that have been produced for
the ’To Be“ environment have not had a clearly defined purpose and
scope, are still missing important content, and contain products that
are neither consistent nor integrated.
* Fully addressed other GAO recommendations.
DOD recognizes that these weaknesses need to be addressed and has
recently assigned a new BEA leadership team. DOD also has either begun
steps to or stated its intentions to (1) revise its governance
structure; (2) develop a program baseline, by September 30, 2005, that
will be used as a managerial and oversight tool to allocate resources,
manage risks, and measure and report progress; and (3) revise the scope
of the architecture and establish a new approach for developing it.
However, much remains to be accomplished to establish an effective
architecture program. Until it does, its business systems modernization
effort will remain a high-risk program.
What GAO Recommends:
Beyond GAO‘s prior recommendations and in light of DOD‘s intended
changes to its BEA development efforts, GAO is making additional
recommendations to ensure that (1) DOD‘s architecture progress and
plans are fully disclosed to congressional committees, (2) GAO‘s prior
recommendations are integral to the department‘s next steps, and (3)
the department‘s plans provide for effective BEA workforce planning.
DOD concurred with GAO‘s recommendations and stated the department‘s
intent to implement them.
www.gao.gov/cgi-bin/getrpt?GAO-05-702.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD Has Yet to Implement Effective Governance and Communications, but
Improvements Are Under Way:
DOD Has Yet to Develop Program Plans and Supporting Workforce Plans,
but Intends to Make Improvements:
DOD Is Not Performing Effective Configuration Management:
DOD Has Yet to Develop a Well-Defined BEA to Guide Its Modernization
Efforts:
DOD Has Yet to Fully Address Most of Our Other Recommendations:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Status of Prior Recommendations on DOD's Development and
Maintenance of Its Business Enterprise Architecture:
Appendix III: Summary of Several Architecture Frameworks:
Appendix IV: Description of DOD Architecture Framework Products,
Version 1.0:
Appendix V: Comments from the Department of Defense:
Appendix VI: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Roles and Responsibilities of Governance Entities:
Table 2: Roles and Responsibilities of the Program Office Divisions:
Table 3: Increments for Developing the BEA:
Table 4: List of DOD's Architecture Framework Products Included in Each
Release:
Figures:
Figure 1: Simplified Diagram of the Program Office:
Figure 2: Organization Chart of the Program Office:
Figure 3: Interdependent DODAF Views of an Architecture:
Abbreviations:
ASD(NII)/CIO: Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer:
AV: all view:
BEA: business enterprise architecture:
BMMP: Business Management Modernization Program:
C4ISR: Command, Control, Communications, Computers, Intelligence,
Surveillance, and Reconnaissance (architecture framework):
CIO: Chief Information Officer:
DBSMC: Defense Business Systems Management Committee:
DOD: Department of Defense:
DODAF: Department of Defense Architecture Framework:
DO/IT: Domain Owners Integration Team:
FEA: Federal Enterprise Architecture:
FEAF: Federal Enterprise Architecture Framework:
IBM: International Business Machines:
IT: information technology:
OMB: Office of Management and Budget:
OV: operational view:
PPBE: Planning, Programming, Budgeting, and Execution:
SFIS: Standard Financial Information Structure:
SV: systems view:
TV: technical standards view:
USD(AT&L): Under Secretary of Defense for Acquisition, Technology, and
Logistics:
Letter July 22, 2005:
The Honorable John W. Warner:
Chairman:
The Honorable Carl Levin:
Ranking Minority Member:
Committee on Armed Services:
United States Senate:
The Honorable Duncan Hunter:
Chairman:
The Honorable Ike Skelton:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:
We first designated business systems modernization efforts within the
Department of Defense (DOD) as a high-risk program in 1995, and we
continue to designate it as such today.[Footnote 1] As our research on
successful public and private sector organizations has shown,
attempting such large-scale systems modernization programs without a
well-defined enterprise architecture often results in systems that are
duplicative; are not well integrated; are unnecessarily costly to
manage, maintain, and operate; and do not effectively optimize mission
performance. To help DOD transform its operations, we
recommended[Footnote 2] in 2001 that the department develop an
enterprise architecture to guide and constrain its multibillion dollar
annual investment in business systems. In July 2001, the department
initiated a business management modernization program to, among other
things, develop a business enterprise architecture (BEA).
Recognizing the importance of DOD's efforts to transform its business
operations and systems through the use of an enterprise architecture,
Congress included provisions in the National Defense Authorization Act
for Fiscal Year 2003[Footnote 3] aimed at having the department develop
and effectively implement a well-defined BEA. In addition, the Ronald
W. Reagan National Defense Authorization Act for Fiscal Year
2005[Footnote 4] again directs DOD to develop a well-defined BEA and a
transition plan by September 2005.
As part of our ongoing monitoring of the BEA, we assessed whether DOD
had (1) established an effective governance structure; (2) developed
program plans, including supporting workforce plans; (3) performed
effective configuration management; (4) developed well-defined BEA
products; and (5) addressed our other recommendations.
We performed our work from July 2004 through May 2005, in accordance
with generally accepted government auditing standards. We conducted
this review under the Comptroller General's authority and have
addressed this report to the committees of jurisdiction. Details on our
objectives, scope, and methodology are contained in appendix I.
Results in Brief:
DOD has yet to establish an effective governance structure for its
architecture development, maintenance, and implementation efforts,
including an effective communications strategy to achieve stakeholder
buy-in to the architecture. In particular, the structure that has been
in place since 2001 has lacked the requisite authority and
accountability to be effective, and the key entities making up this
structure have not performed according to their respective charters.
For example, one governance entity met only four times in 4 years,
another last met about a year ago, and a third did not include key
members--the military services--in its deliberations. Moreover, efforts
under this governance structure to outreach to and communicate with key
stakeholders, whose input and support of the BEA are critical to its
success, fell short of plans. DOD recognizes the need for change to
address these long-standing BEA governance weaknesses. Specifically,
the department established the Defense Business Systems Management
Committee (as required by the Fiscal Year 2005 Defense Authorization
Act) to direct, oversee, and approve, among other things, the BEA. In
addition, program officials stated that the department intends to
revise its communications strategy. However, much remains to be
accomplished before these steps result in establishment of an
operational governance structure. Until such a structure exists, the
department's architecture, and thus its modernization efforts, will be
at risk of failure.
DOD does not have the program plans it needs to effectively manage its
BEA efforts. It has not developed plans that explicitly identify
measurable goals and outcomes to be achieved, nor has it defined the
tasks to be performed to achieve the goals and outcomes, the resources
needed to perform the tasks, or the time frames within which the tasks
will be performed. DOD also has not assessed, as part of its program
planning, the workforce capabilities that it has in place and that it
needs to acquire to manage its architecture efforts, and it does not
have a strategy for meeting its human capital needs. Unless this
situation changes, it is unlikely that DOD will be successful in its
attempt to develop, maintain, and implement its BEA. Recognizing this
long-standing planning weakness, the department stated that it will
have an approved program plan by September 30, 2005.
DOD is not performing effective configuration management, which is a
formal process for maintaining integrity and traceability and for
controlling modifications or changes to the architecture products
throughout their life cycles. Although the department has a draft plan
that defines key steps and practices, it has not followed this plan. In
addition, the department does not have a configuration manager after
almost 4 years of architecture development. Further, while there is a
configuration control board, the board's charter has yet to be
approved, and its authority is limited to approving changes to a subset
of BEA products. Until this situation is remedied, DOD will not have
adequate assurance that it is controlling product changes and
maintaining the integrity of product content.
DOD's BEA is not well defined and, thus, provides limited utility. The
latest releases and updates of the BEA do not include products
describing DOD's current or "As Is" business and technology
environments, nor do they include a plan for investing in business
systems in a sequence that will allow it to move from this "As Is"
environment to its target or "To Be" environment. In addition, there
are no clearly defined associations between the purpose of the BEA
releases and updates produced to date for this "To Be" environment and
the department's goals and objectives. These releases are also still
missing important content that we have made recommendations to correct.
Moreover, the artifacts produced in these releases and updates have
been neither consistent with one another nor adequately integrated.
Finally, only the first release (Release 1.0) of the BEA has been
approved. Recognizing the need to develop architecture products that
have utility and provide a foundation upon which to build, program
officials have stated the department's intent to reduce the scope of
the BEA and to change the development approach from one that focuses on
producing as many products as it can within a specific time period to
one that focuses on developing high-quality products. Without such
products, DOD will not be able to effectively modernize its business
systems.
To date, DOD has not addressed most of our other prior recommendations
regarding the development and maintenance of the BEA. While the
department has taken recent actions that fully address one of these
recommendations and partially address others, much remains to be
accomplished. For example, the department has yet to develop and
implement a quality assurance plan. Until our recommendations are
addressed, the department will remain challenged in its ability to
effectively develop and implement a BEA, and, thus, its business
systems modernization will remain at high risk of failure.
Given that roughly $318 million and 4 years have been invested without
commensurate progress and results, and given the department's recent
actions intended to address some of them, we are making several
recommendations to the Secretary of Defense to ensure that (1) DOD's
architecture progress, plans for moving forward, and capability to
implement these plans are fully disclosed to congressional committees;
(2) our open recommendations are integral to DOD's next steps; and (3)
these next steps provide for effective BEA workforce planning. In its
written comments on a draft of this report, DOD concurred with our
recommendations and stated the department's intent to implement them.
Background:
DOD is a massive and complex organization. In fiscal year 2004, the
department reported that its operations involved $1.2 trillion in
assets, $1.7 trillion in liabilities, over 3.3 million military and
civilian personnel, and over $605 billion in net cost of operations.
For fiscal year 2005, the department received appropriations of about
$417 billion. Moreover, execution of its operations spans a wide range
of defense organizations, including the military services and their
respective major commands and functional activities, numerous large
defense agencies and field activities, and various combatant and joint
operational commands, which are responsible for military operations for
specific geographic regions or theaters of operations.
In executing these military operations, the department performs an
assortment of interrelated and interdependent business functions,
including logistics management, procurement, health care management,
and financial management. DOD reports that, in order to support these
business functions, it currently relies on about 4,200 systems--
including accounting, acquisition, finance, logistics, and personnel.
As we have previously reported,[Footnote 5] this systems environment is
overly complex and error prone and is characterized by (1) little
standardization across the department, (2) multiple systems performing
the same tasks, (3) the same data stored in multiple systems, and (4)
manual data entry into multiple systems. These problems continue
despite the department's spending billions of dollars annually to
operate, maintain, and modernize its business systems. DOD received
approximately $13.3 billion for fiscal year 2005 to operate, maintain,
and modernize this environment. In addition, our reports[Footnote 6]
continue to show that the department's stovepiped and duplicative
systems contribute to fraud, waste, and abuse. Of the 25 areas on GAO's
governmentwide "high-risk" list, 8 are DOD program areas, and the
department shares responsibility for 6 other high-risk areas that are
governmentwide in scope.[Footnote 7]
Because of the department's size and complexity, modernizing its
business systems is a huge management challenge that we first
designated as one of the department's high-risk areas in 1995, and we
continue to do so today.[Footnote 8] To help meet this challenge, DOD
established its business systems modernization program in 2001. As we
testified in 2003,[Footnote 9] one of the seven key elements that are
necessary to successfully execute this modernization program is to
establish and implement an enterprise architecture. Subsequently, in
its Fiscal Year 2004 Performance and Accountability Report,[Footnote
10] DOD acknowledged that deficiencies in its systems and business
processes hindered the department's ability to collect and report
financial and performance information that was accurate, reliable, and
timely. The DOD report noted that to address its systemic problems and
assist in the modernization of its business operations, the department
had undertaken the development and implementation of a BEA.
Enterprise Architecture Is Critical to Achieving Successful
Modernization:
Effective use of an enterprise architecture, or a modernization
blueprint, is a trademark of successful public and private
organizations. For more than a decade, we have promoted the use of
architectures to guide and constrain systems modernization, recognizing
them as a crucial means to a challenging goal: agency operational
structures that are optimally defined in both business and
technological environments. Congress, the Office of Management and
Budget (OMB), and the federal Chief Information Officer (CIO) Council
have also recognized the importance of an architecture-centric approach
to modernization. The Clinger-Cohen Act of 1996[Footnote 11] mandates
that an agency's CIO develop, maintain, and facilitate the
implementation of an information technology (IT) architecture. Further,
the E-Government Act of 2002[Footnote 12] requires OMB to oversee the
development of enterprise architectures within and across agencies. In
addition, OMB has issued guidance that, among other things, requires
system investments to be consistent with these architectures.
What Is an Enterprise Architecture?
An enterprise architecture provides a clear and comprehensive picture
of an entity, whether it is an organization (e.g., a federal
department) or a functional or mission area that cuts across more than
one organization (e.g., financial maagement). This picture consists of
snapshots of both the enterprise's current or "As Is" environment and
its target or "To Be" environment, as well as a capital investment road
map for transitioning from the current to the target environment. These
snapshots consist of "views," which are one or more architecture
products that provide logical or technical representations of the
enterprise.
The suite of products and their content that form a given entity's
enterprise architecture are largely governed by the framework used to
develop the architecture. Since the 1980s, various architecture
frameworks have emerged and been applied. See appendix III for a
discussion of these various frameworks.
The importance of developing, implementing, and maintaining an
enterprise architecture is a basic tenet of both organizational
transformation and systems modernization. Managed properly, an
enterprise architecture can clarify and help to optimize the
interdependencies and relationships among an organization's business
operations and the underlying IT infrastructure and applications that
support these operations. Employed in concert with other important
management controls, such as portfolio-based capital planning and
investment control practices, architectures can greatly increase the
chances that an organization's operational and IT environments will be
configured to optimize its mission performance. Our experience with
federal agencies has shown that investing in IT without defining these
investments in the context of an architecture often results in systems
that are duplicative, not well integrated, and unnecessarily costly to
maintain and interface.[Footnote 13]
DOD's Business Management Modernization Program: A Brief Description
and Chronology:
In July 2001, the Secretary of Defense established the Business
Management Modernization Program (BMMP) to improve the efficiency and
effectiveness of DOD's business operations and provide the department's
leaders with accurate and timely information through the development
and implementation of a BEA. At that time, the Secretary tasked the
Under Secretary of Defense (Comptroller), in coordination with the
Under Secretary of Defense for Acquisition, Technology, and Logistics
(USD(AT&L)) and the Assistant Secretary of Defense (Networks and
Information Integration)/Chief Information Officer (ASD(NII)/CIO), with
responsibility for overseeing the program. To accomplish this, in
October 2001, the Comptroller established governance bodies and
assigned them responsibilities associated with developing, maintaining,
and implementing the BEA. These entities and their respective roles and
responsibilities are shown in table 1. DOD is currently revising its
BEA governance structure, including recently eliminating its long-
standing governance entities. These revisions are discussed later in
this report.
Table 1: Roles and Responsibilities of Governance Entities:
Entity: Executive Committee;
Roles and responsibilities:
* Provides strategic direction to the Steering Committee;
* Champions program execution;
* Holds components[A] responsible for program results;
* Advises the Under Secretary of Defense (Comptroller) on business
modernization;
Membership:
* Cochaired by the Comptroller and the Assistant Secretary of Defense
(Networks and Information Integration)/Chief Information Officer
(ASD(NII)/CIO);
* Includes representatives from both the Office of the Secretary of
Defense and the military departments, such as the Under Secretary of
Defense for Acquisition, Technology, and Logistics (USD(AT&L)) and the
Under Secretary of the Army.
Entity: Steering Committee;
Roles and responsibilities:
* Advises the Executive Committee on program performance;
* Serves as the forum for discussion of component issues;
* Provides guidance to the program office[B];
Membership:
* Cochaired by the Principal Deputy Comptroller and the Deputy DOD CIO;
* Includes representatives from both the Office of the Secretary of
Defense and the military departments, such as the Principal Deputy
USD(AT&L) and the Assistant Secretary of the Army.
Entity: Domain Owners Integration Team;
Roles and responsibilities:
* Provides recommendations to the Steering Committee;
* Provides guidance regarding architecture updates and their effects;
* Identifies, addresses, and resolves cross-domain[C] issues, and
program governance and domain operational issues;
Membership:
* Chaired by the program office director;
* Includes representatives from the program office, senior executives
from each business domain (domain representatives) and the enterprise
information environment mission area, and military services
representatives.
Entity: Domains and Mission Area;
Roles and responsibilities:
* Implements the architecture;
* Develops and executes the transition plan;
* Oversees portfolio management activities;
* Establishes a structure to ensure the representation of the DOD
components and the appropriate federal agencies;
Membership:
* Headed by the USD(AT&L), the Comptroller, the Under Secretary of
Defense (Personnel and Readiness), and the ASD(NII)/CIO;
* Includes representatives from the DOD components, including the
military services.
Source: DOD.
[A] DOD components include the military departments, defense agencies,
and DOD field activities.
[B] In March 2005, DOD changed the name of the program office from the
Business Modernization and Systems Integration Office to the
Transformation Support Office.
[C] There are five departmental domains or business process areas: (1)
acquisition, (2) financial management, (3) human resources management,
(4) installations and environment, and (5) logistics. There is also one
mission area--enterprise information environment.
[End of table]
Also in 2001, the BMMP program office was established to execute the
program's day-to-day activities, including implementing internal
management controls and other mechanisms to provide reasonable
assurance that the office would develop and implement an effective BEA.
The office is led by a program director and comprised of seven program
divisions, each of which is headed by an assistant deputy director.
Figure 1 is a simplified diagram of the organizational structure of the
program office, and table 2 shows the roles and responsibilities of the
program divisions.
Figure 1: Simplified Diagram of the Program Office:
[See PDF for image]
[End of figure]
Table 2: Roles and Responsibilities of the Program Office Divisions:
Program division: Communications;
Roles and responsibilities: Communicate status, progress, goals, and
objectives of program to ensure that decision makers (e.g., internal
work groups and external customers, such as Congress and GAO) receive
quality information to make informed decisions about the department's
business modernization efforts.
Program division: Financial Management and Comptroller;
Roles and responsibilities: Develop and maintain the program budget;
report on program performance against the budget; and develop, oversee,
and manage all program office contracts.
Program division: Chief Technology Officer;
Roles and responsibilities: Provide information technology support for
program activities and operations, such as maintenance of the
architecture data repository, as well as ensure that the program office
is in compliance with government security policies and standards.
Program division: Enterprise Architecture;
Roles and responsibilities: Develop, extend, and improve the business
enterprise architecture (BEA); and provide quality assurance and
configuration control to ensure BEA quality and utility.
Program division: Strategic Planning;
Roles and responsibilities: Develop and maintain the strategic plan and
program baseline, develop and maintain the risk management program,
conduct quality assessments of the BEA and other program deliverables,
identify process improvement areas and ensure corrective actions are
taken, and assess program performance against program goals.
Program division: Transition Planning;
Roles and responsibilities: Develop, maintain, and implement the
transition plan; provide portfolio management assistance to the
domains; and oversee the modernization of the department's business
systems and the establishment of a consolidated systems repository.
Program division: Administrative Support Programs;
Roles and responsibilities: Provide administrative support to the
program office for human, material, financial, and technology resources
to achieve the mission, vision, and strategy.
Source: DOD.
[End of table]
Initially, DOD planned to develop the architecture in 1 year.
Subsequently, the department stated that it would develop its
architecture in three increments, each increment addressing a subset of
objectives and consisting of specific architecture releases. Table 3
shows these increments, the corresponding architecture releases, and
the planned completion dates for the increments.
To develop the architecture, DOD entered into a 5-year, $95 million
contract with International Business Machines (IBM) in April 2002,
under which the department has issued a series of task orders aimed at
developing the architecture. In 2004, DOD increased the contract amount
to $250 million; however, the contract did not provide a reason for
this increase, and program officials have yet to provide an
explanation. As of September 2004, DOD reported that it had obligated
approximately $318 million for the program, which is primarily for
contractor support.[Footnote 14]
Table 3: Increments for Developing the BEA:
Increment: 1;
Objectives:
* Achieve unqualified audit opinion for consolidated Department of
Defense (DOD) financial statements, including related processes to
achieve asset accountability and address other material weaknesses;
* Achieve total personnel visibility to include military service
members, civilian employees, military retirees, and other U.S.
personnel in a theater of operations (including contractors and other
federal employees);
Architecture release[A] and date: 2.1--April 2004; 2.2--July 2004; 2.3--
November 2004; January 2005 Update; March 2005 Update;
Original increment completion date: January 2005;
Revised increment completion date: March 2005.
Increment: 2;
Objectives:
* Align acquisition practices with government and industry best
practice benchmarks;
* Achieve total asset visibility and accurate valuation of assets
(includes operating, materials, and supplies; inventory and property;
and plant and equipment);
* Enhance force management through position accountability and
visibility (military and civilian);
* Improve military health care delivery through a more efficient health
care claims system, more accurate patient diagnostic coding, and joint
medical material asset visibility;
* Improve environmental safety and occupational health;
Architecture release[A] and date: 3.0--September 2005;
Original increment completion date: September 2005;
Revised increment completion date: No change.
Increment: 3;
Objectives:
* Implement planning, programming, budgeting, and execution (PPBE)
process improvements in accordance with Joint Defense Capabilities
Study recommendations for a capabilities-based PPBE process;
* Achieve integrated total force management;
* Improve installation management;
Architecture release[A] and date: 3.0-- September 2005;
Original increment completion date: September 2006;
Revised increment completion date: September 2005.
Source: DOD.
[A] The architecture releases for increment 1 are limited to "To Be"
architecture products. According to DOD, the architecture releases for
increments 2 and 3 will include "As Is" and "To Be" architecture
products. DOD issued the initial transition plan in May 2003, and,
according to program officials, the department currently plans to issue
the second release in September 2005.
[End of table]
Prior Reviews of DOD's Architecture Efforts Have Identified Many
Weaknesses and Challenges:
Over the last 4 years, we have identified the need for, and reviewed
DOD's efforts to develop an enterprise architecture for modernizing its
business operations and systems, and we have made a series of
recommendations to assist the department in successfully developing the
architecture and using it to guide and constrain its ongoing and
planned business systems investments.
In particular, we reported in May 2001[Footnote 15] that the department
did not have an architecture for its financial and financial-related
business operations, nor did it have the management structures,
processes, and controls in place to effectively develop and implement
one. We concluded that continuing to spend billions of dollars on new
and modified systems would result in more processes and systems that
were duplicative, not interoperable, unnecessarily costly to maintain
and interface, and not optimizing mission performance and
accountability. We made eight recommendations to the Secretary of
Defense that were aimed at providing the means for effectively
developing and implementing an architecture and limiting DOD
components' systems investments until it had a well-defined
architecture and the means to enforce it. In July 2001, DOD initiated
the BMMP.
In February 2003,[Footnote 16] we reported that the department was
following some architecture best practices, such as establishing a
program office to be responsible for managing the enterprise
architecture development effort. We also reported challenges and
weaknesses in DOD's architecture efforts. For example, we reported that
DOD had not yet (1) established a governance structure and the process
controls needed to ensure ownership of and accountability for the
architecture across the department; (2) clearly communicated to
intended stakeholders its purpose, scope, and approach for developing
the architecture; and (3) defined and implemented an independent
quality assurance process. We reiterated our earlier recommendations
and made six new recommendations aimed at enhancing DOD's ability to
develop its architecture and to guide and constrain its business
systems modernization investments.
In March 2003,[Footnote 17] we reported that DOD's draft release of its
architecture did not include a number of items that were recommended by
relevant architectural guidance, and that DOD's plans would not fully
satisfy the requirements of the National Defense Authorization Act for
Fiscal Year 2003.[Footnote 18] For example, the draft architecture did
not include a "To Be" security view, which would define the security
requirements--including relevant standards to be applied in
implementing security policies, procedures, and controls. DOD officials
generally agreed, and they stated that subsequent releases of the
architecture would provide these missing details.[Footnote 19]
In July and September 2003,[Footnote 20] we reported that the initial
release of the department's architecture, including the transition
plan, did not adequately address statutory requirements and other
relevant architectural requirements. For example, the description of
the "As Is" environment did not include (1) the current business
operations in terms of entities and people who perform the functions,
processes, and activities and (2) the locations where the functions,
processes, and activities are performed. The description of the "To Be"
environment did not include actual systems to be developed or acquired
to support future business operations and the physical infrastructure
that would be needed to support the business systems. The transition
plan did not include time frames for phasing out existing systems
within DOD's then reported inventory of about 2,300 business
systems.[Footnote 21] We concluded that the department's initial
release of the architecture did not contain sufficient scope and detail
either to satisfy the act's requirements or to effectively guide and
constrain departmentwide business systems modernization. In our
September 2003 report,[Footnote 22] we reiterated the open
recommendations that we had made in our May 2001[Footnote 23] and
February 2003[Footnote 24] reports, and we made 10 new recommendations
to, among other things, improve DOD's efforts for developing the next
release of the architecture.
In March[Footnote 25] and July 2004,[Footnote 26] we testified that
DOD's substantial long-standing financial and business management
problems adversely affected the economy, effectiveness, and efficiency
of its operations and had resulted in a lack of adequate transparency
and appropriate accountability across all major business areas.
Further, we said that substantial work remained before the BEA would
begin to have a tangible impact on improving the department's overall
business operations. We concluded that until DOD completed a number of
actions, including developing a well-defined BEA, its business systems
modernization efforts would be at a high risk of failure.
In May 2004,[Footnote 27] we reported that after 3 years of effort and
over $203 million in obligations, DOD had not made significant change
in the content of the BEA or in its approach to investing billions of
dollars annually in existing and new systems. We reported that few
actions had been taken to address the recommendations we had made in
our September 2003 report,[Footnote 28] which were aimed at improving
the department's plans for developing the next release of the
architecture and implementing the institutional means for selecting and
controlling both planned and ongoing business systems investments. We
also reported that DOD had still not adopted key architecture
management best practices that we had recommended,[Footnote 29] such as
assigning accountability and responsibility for directing, overseeing,
and approving the architecture and explicitly defining performance
metrics to evaluate the architecture's quality, content, and utility.
Further, DOD had not added the scope and detail to its architecture
that we had previously identified as missing. For example, in the
latest release of the BEA--Release 2.0--DOD did not provide sufficient
descriptive content related to future business operations and
supporting technology to permit effective acquisition and
implementation of system solutions and associated operational change.
Moreover, the department had not yet explicitly defined program plans,
including milestones, detailing how it intended to extend and evolve
the architecture to incorporate this missing content. We concluded that
the future of DOD's architecture development and implementation
activities was at risk, which in turn placed the department's business
transformation effort in jeopardy of failing. Therefore, we added that
it was imperative that the department move swiftly to implement our
open recommendations. Because many of our prior recommendations
remained open, we did not make any new recommendations in our May 2004
report,[Footnote 30] but we reiterated the open recommendations that we
had made in our May 2001, February 2003, and September 2003
reports.[Footnote 31]
In November 2004[Footnote 32] and April 2005,[Footnote 33] we testified
that for DOD to successfully transform its business operations, it
would need a comprehensive and integrated business transformation plan;
people with the skills, responsibility, and authority to implement the
plan; an effective process and related tools, such as a BEA; and
results-oriented performance measures that would link institutional,
unit, and individual personnel goals and expectations to promote
accountability for results. We testified that over the last 3 years, we
had made a series of recommendations to DOD and suggested legislative
changes that, if implemented, could help the department move forward in
establishing the means to successfully address the challenges it faces
in transforming its business operations.[Footnote 34] We also testified
that, after about 3 years of effort and over $203 million in reported
obligations, the architecture's content and the department's approach
to investing billions of dollars annually in existing and new systems
had not changed significantly, and that the program had yielded very
little, if any, tangible improvements in DOD's business operations.
DOD Has Yet to Implement Effective Governance and Communications, but
Improvements Are Under Way:
Long-standing weaknesses in DOD's BEA governance structure and
communications strategy still remain. While DOD has established a new
senior committee to oversee its business transformation efforts,
including BEA development, much remains to be accomplished before
proposed governance and communications concepts are fully defined and
implemented. Until the department has made its intended governance and
communications concept operational, the success of DOD's architecture
efforts will remain in doubt.
Long-standing Program Governance Weaknesses Remain, Although Recent
Proposals Are Intended to Address Weaknesses:
An enterprise architecture is a corporate asset that, among other
things, is intended to represent the strategic direction of the
enterprise. Accordingly, best practices[Footnote 35] recommend that to
demonstrate commitment, organizations should vest accountability and
assign responsibility for directing, overseeing, and approving the
architecture to a committee or group with representation from across
the enterprise. Sustained support and commitment by this committee to
the architecture, as well as the committee's ownership of it, are
critical to a successful enterprise architecture development effort. We
have previously recommended that DOD establish this kind of
architecture responsibility and accountability structure.[Footnote 36]
(See app. II for our prior recommendations and their current status.)
During the last 4 years, DOD has relied on three primary management
entities to govern BEA development, maintenance, and implementation--
the Executive Committee, the Steering Committee, and the Domain Owners
Integration Team (DO/IT). (See table 1 for their roles and
responsibilities.) This governance approach, however, does not assign
accountability and responsibility for directing, overseeing, and
approving the BEA to these entities either singularly or collectively.
As we reported in February 2003,[Footnote 37] the Executive and
Steering Committees were advisory in nature, and their responsibilities
were limited to providing guidance to the program office and advising
the Comptroller and the Executive Committee. Moreover, since they were
established, neither the two committees nor the DO/IT have adequately
fulfilled their assigned responsibilities, as we discuss below.
* The Executive Committee was chartered to, among other things, provide
strategic direction to the Steering Committee, champion program
execution, and hold DOD components--including the military services--
responsible for program results. To accomplish these things, the
charter states that the committee should establish a meeting schedule.
However, no schedule was established, and the committee met only four
times in over 3½ years. Moreover, no minutes of the four meetings were
prepared, according to the program's Acting Assistant Deputy Director
for Communications, and no other documentation exists to demonstrate
the committee's performance of its chartered functions. In fact, during
numerous DO/IT meetings that we attended, participants stated that the
Executive Committee was not providing strategic direction, nor was it
championing program execution.
* The Steering Committee was chartered to advise the Executive
Committee on program performance, serve as the forum for discussion of
component issues, and provide guidance to the program office. According
to the program's Acting Assistant Deputy Director for Communications,
neither Executive Committee meeting minutes nor any other documentation
exists to demonstrate that the Steering Committee advised the Executive
Committee. Moreover, during the Steering Committee meetings that we
attended over the last 4 years, we saw no evidence that this committee
either planned to or actually did advise the Executive Committee on
program performance. While we did observe in these meetings that issues
were raised and discussed, which is a chartered responsibility of the
committee, we also observed that the committee did not provide guidance
and direction to the program office during these meetings. The Steering
Committee last met about 1 year ago (June 2004).
* The DO/IT was chartered to provide recommendations to the Steering
Committee; provide guidance regarding architecture updates and their
effects; and identify, address, and resolve domain and program issues.
The charter also states that the DO/IT was to ensure that its
representation included the military services. However, during the
Steering Committee meetings that we attended, DO/IT representatives did
not provide any recommendations to the Steering Committee, nor did they
provide guidance on architecture updates and their effects. Moreover,
the DO/IT did not include military service representatives, and it did
not establish any policies and procedures for how to address and
resolve issues. As a result, issues that were identified during
meetings were not resolved. For example, in July 2004, there were
discussions regarding the lack of involvement from the services, the
lack of detail in the architecture content, and the lack of clear
understanding of the roles and responsibilities among the domains and
the services. During this meeting, however, no decisions were made
about how these issues were to be resolved, and no actions were taken
to provide recommendations to the Steering Committee for resolving the
issues. As a result, we observed the same issues being discussed,
without resolution, 5 months later.
DOD has recently taken steps to improve the program's governance
structure and, according to program officials, further steps are
planned. For example, the department has implemented the provisions in
the Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005,[Footnote 38] which are aimed at increasing senior DOD leadership
involvement in the program. In particular, it includes DOD's
establishment of the Defense Business Systems Management Committee
(DBSMC) to replace both the Executive and Steering Committees and to
serve as the highest ranking governance body overseeing business
transformation. According to the DBSMC charter, the committee is
accountable and responsible for directing, overseeing, and approving
all program activities. Specific responsibilities of the committee
include:
* establishing strategic direction and plans for the business mission
area[Footnote 39] in coordination with the warfighting and enterprise
information environment mission areas;
* approving business mission area transformation plans and initiatives
and coordinating transition planning in a documented program baseline
with critical success factors, milestones, metrics, deliverables, and
periodic program reviews;
* establishing key metrics and targets by which to track business
transformation progress;
* establishing policies and approving the business mission area
strategic plan, the transition plan for implementation for business
systems modernization, the transformation program baseline, and the
BEA; and:
* executing a comprehensive communications strategy.
Consistent with recent legislation, the DBSMC is chaired by the Deputy
Secretary of Defense, and the USD(AT&L) serves as the vice chair. Its
membership consists of senior leadership from across the department,
including the military service secretaries, the defense agency heads,
and the principal staff assistants.[Footnote 40] The department has
also moved the program office from the Comptroller to the USD(AT&L),
reporting to the Special Assistant for Business Transformation.
According to DOD, this transfer of functions and responsibilities will
allow USD(AT&L) to establish the level of activity necessary to support
and coordinate the activities of the newly established DBSMC.
Other entities may be established to support the DBSMC. According to
program officials, including the Program Director, the DO/IT has been
eliminated and may be replaced by a DOD Enterprise Transformation
Integration Group. Further, the DO/IT had identified the need for six
additional boards[Footnote 41] to assist the program office. These
boards have not yet been chartered, but potential board members have
met to discuss the boards' roles, responsibilities, and operating
procedures, as well as program issues. However, the program director
stated that not all of these boards may be established under the new
structure.
According to the Special Assistant for Business Transformation, a new
governance structure will be in place and operational by September 30,
2005. To this end, DOD officials told us that the DBSMC held its first
meeting in February 2005 to finalize its charter and member
composition, and that to move governance reforms forward, the committee
will initially hold monthly meetings.
The weaknesses in the BEA governance structure over the last 4 years,
according to program officials, are attributable to a lack of authority
and accountability in program leadership by the various management
entities (e.g., Executive and Steering Committees) and to the limited
direction provided by these entities. While DOD's recent actions are
intended to address these root causes, almost 4 years and approximately
$318 million in obligations have been invested, and the department is
still attempting to establish an effective governance structure.
Moreover, much remains to be accomplished until DOD's new governance
structure becomes an operational reality. Until it does, it is unlikely
that the department will be able to develop an effective BEA.
An Effective Communications Strategy Has Yet to Be Implemented, but
Some Activities Are Under Way:
Effective communications among architecture stakeholders are closely
aligned with effective governance. According to relevant
guidance,[Footnote 42] once initial stakeholder participation in an
architecture program is achieved, a communications strategy should be
defined and implemented to facilitate the exchange of information among
architecture stakeholders about all aspects of the program, such as
program purpose, scope, methodologies, progress, results, and key
decisions. Such communication is essential to executing an architecture
program effectively, including obtaining institutional buy-in for
developing and using the architecture for corporate decision making.
Accordingly, in 2003 we recommended[Footnote 43] that DOD provide for
ensuring stakeholder commitment and buy-in through proactive
architecture marketing and communication activities. (See app. II for
our prior recommendation and its current status.)
In response, the program office defined and approved a strategic
communications plan in March 2004. According to the plan, its purpose
is to direct the flow of information to inform, collaborate with, and
educate DOD internal and external stakeholders about the program. To
accomplish this, the plan identifies categories of stakeholders and
includes a five-phase implementation approach. The five phases are as
follows:
1. Plan Start-up: Conduct a formal tactical review, including an
assessment of current communication tools, communication procedures,
available resources, and agency or industry best practices.
2. Discovery and Assessment: Identify and verify all internal and
external stakeholders and begin defining the benchmarking and reporting
requirements.
3. Branding: Determine key messages to be communicated and select tools
for disseminating these messages.
4. Communications Planning and Execution: Execute and assess the
application of the strategy and continue to develop the communication
tools.
5. Evaluation: Evaluate the progress and overall success of the plan's
implementation, using metrics.
The strategic communications plan states that the five phases were to
have been fully implemented by December 2004. Although 5 months have
passed since the plan was to be implemented, none of the five phases
have been completed, and communications activities that have been
performed by those responsible for implementing the plan have been
limited in scope. Specifically, communications activities have focused
on raising program awareness through the distribution of posters and
press releases and the establishment of a Web site that provides
information about the program. However, an assessment to evaluate the
effectiveness of communication tools and procedures, available
resources, and agency or industry best practices in communicating the
program's purpose to the various stakeholders (e.g., the domains and
the military services) has not been performed. In addition, the
program's Acting Assistant Deputy Director for Communications stated
that a systematic approach, including metrics, to measure the
effectiveness of the communications strategy has not been defined.
According to this official, communications activities have been ad hoc.
Further, DO/IT representatives told us that the program's Web site does
not contain consistent and current information about the program; as a
result, stakeholders' understanding of the BEA is limited. Moreover,
the plan focuses first on internal stakeholders, and it defers efforts
to proactively promote understanding and buy-in with external
stakeholders, such as Congress. The plan defines the internal
stakeholders to exclude key departmental components, such as the
military services and the defense agencies. Instead, it defines
internal stakeholders to include only the program office and the
domains.
As a result, both the internal and external DOD stakeholders with whom
we spoke said that they did not have a clear understanding of the
program, including its purpose, scope, approach, activities, and
status, as well as stakeholders' roles and responsibilities. For
example, the Installation and Environment domain representative stated
that it was unclear how the program would achieve one of the program's
original goals (i.e., achieving an unqualified audit opinion by fiscal
year 2007), and what the role of this representative's domain would be
in achieving this goal. Similarly, the Acquisition domain
representative stated that the roles and responsibilities of the
domains for the program are confusing, because the domains should be
the entities that are developing the BEA, and not the program office.
According to this representative, the current approach will result in
redundancy and increased program costs.
In addition, the program office's Acting Assistant Deputy Director for
Communications acknowledged that stakeholders are confused, stating
that they do not have a good understanding of either the BEA's goals,
objectives, and intended outcomes or stakeholders' roles and
responsibilities. This official also stated that cross-domain
integration issues remain that require strong DOD leadership to move
the communications efforts beyond conducting awareness activities to
achieving departmentwide buy-in.
The Special Assistant for Business Transformation has recently begun to
better communicate with key external stakeholders, such as Congress.
For example, this official stated that department officials have met
with and intend to hold future meetings with congressional staff to
brief them on the department's plans and efforts to date. Without
effective communication with BEA stakeholders, the likelihood that DOD
will be able to effectively develop and implement its BEA is greatly
reduced.
DOD Has Yet to Develop Program Plans and Supporting Workforce Plans,
but Intends to Make Improvements:
DOD does not have the program plans that it needs to effectively manage
the development, maintenance, and implementation of its BEA. In
particular, the department has yet to specify measurable goals and
outcomes to be achieved, nor has it defined the tasks to be performed
to achieve these goals and outcomes, the resources needed to perform
the tasks, or the time frames within which the tasks will be performed.
DOD has not assessed, as part of its program planning, the workforce
capabilities that it has in place and that it needs to manage its
architecture development, maintenance, and implementation efforts, nor
does it have a strategy for meeting its human capital needs. The
absence of effective program planning, including program workforce
planning, has contributed to DOD's limited progress in developing a
well-defined architecture and clearly reporting on its progress to
date. Unless its program planning improves, it is unlikely that the
department will be successful in its attempt to develop, maintain, and
implement its BEA. Recognizing this long-standing void in planning, DOD
stated that it intends to complete, by September 30, 2005, a transition
plan that will include a program baseline that will be used to oversee
and manage program activities.
DOD Has Yet to Develop Effective Program Plans:
Architecture management guidance states that organizations should
develop and execute program plans, and that these plans should provide
an explicit road map for accomplishing architecture development,
maintenance, and implementation goals.[Footnote 44] Among other things,
effective plans should specify tasks to be performed, resources needed
to perform these tasks (e.g., funding, staffing, tools, and training),
program management and contractor roles and responsibilities, time
frames for completing tasks, expected outcomes, performance measures,
program management controls, and reporting requirements. We have
previously reported on the program's lack of effective planning and
have recommended that DOD develop BEA program plans.[Footnote 45] (See
app. II for our prior recommendation and its current status.)
Since the program was launched in 2001, DOD has operated without a
program plan. Instead, the department has set target dates for
producing a series of architecture releases as part of three generally
defined architecture increments (see table 3). However, DOD has not
clearly defined what the purpose of the respective increments are,
either individually or collectively, and it has not developed near-,
mid-, or long-term plans for producing these increments. At a minimum,
such plans would identify specific tasks (i.e., provide a detailed work
breakdown structure) for producing the architecture releases that
possess predefined content and utility. These plans would also contain
specific and reliable estimates of the time and resources it will take
to perform these tasks.
Also in lieu of program plans, the program office provided us with
documents in November 2004 that the deputy program director stated were
to address our open recommendations for (1) adding needed content and
consistency to the BEA's "As Is" and "To Be" architectural products and
(2) developing a well-defined BEA transition plan. However, the
documents that we were provided were plans for developing plans to
address our recommendations and, thus, were not documents explaining
how and when our recommendations would be addressed.
DOD has recognized the need for program planning. According to the
Acting Assistant Deputy Director for Strategic Planning, the program
office had committed to developing a program baseline by December 2004.
According to the program director, this baseline was to include program
goals, objectives, and activities as well as performance, cost, and
schedule commitments. Further, the baseline was to establish program
roles, responsibilities, and accountabilities and was to be used as a
managerial and oversight tool to allocate resources, manage risk, and
measure and report progress. However, the department has yet to develop
this program baseline. Moreover, while this program baseline would be a
useful tool for strengthening program management, it nevertheless was
not to have included all of the elements of an effective program plan,
such as a detailed work breakdown structure and associated resource
estimates. According to senior officials, including the Special
Assistant for Business Transformation and the Deputy Under Secretary of
Defense (Financial Management), the department is drafting a transition
plan that is to be approved by the DBSMC and issued by September 30,
2005. According to these officials, this plan will include a program
baseline and a BEA development approach.
The lack of well-defined program plans has contributed significantly to
the limited BEA progress that we have reported over the last several
years. Moreover, this absence of program plans has created a lack of
transparency and understanding about what is occurring on the program
and what will occur--both inside and outside the department. It has
also inhibited BEA governance entities' ability to ensure that
resources (e.g., program office, domains, and contractors) are being
effectively used to achieve worthwhile outcomes and results. Although
the contractor's work statements have provided some additional detail,
these task descriptions lack the specificity necessary to use them
effectively to monitor the contractor's progress and performance. For
example, the latest work statement includes the task "develop business
rules based on the available sources of information," which has been
included in prior work statements. However, the latest work statement
does not define the scope of this effort, nor does it define how this
latest task will support prior efforts to develop business rules. As a
result, the extent to which business rules have been developed and the
work remaining to complete the development of these rules is unclear.
Further, the relationship of this task to DOD's ability to satisfy the
objectives for increment 1 also has not been defined. These
architecture relationships need to be defined before the department can
develop explicit plans for effective BEA development.
Moreover, because there is no plan linking them together, it is not
clear how the contractor's work statements and other BEA working
groups' efforts relate to or contribute to larger BEA goals and
objectives. For example, the program office has continued to task the
contractor to develop architecture releases, although the intended use,
and thus the explicit content, of the various releases has not been
clearly linked to the goals and objectives of increment 1.
Representatives of the DOD business domains raised similar concerns
with the contractor's work statements, telling us that the work
statements have been vague and have not been linked to the specific
architecture products. According to these representatives, the
department does not know if it is investing resources on tasks that are
needed and add value to the program.
According to the deputy program director, continuous changes in the
direction and scope of the program have hindered DOD's ability to
develop effective program plans. Without such plans, DOD has been, and
will continue to be, limited in its ability to develop a well-defined
architecture on time and within budget.
DOD Has Not Performed Effective Workforce Planning:
As we have previously reported,[Footnote 46] workforce planning is an
essential component of program management. Workforce planning enables
an entity to be aware of and prepared for its current and future human
capital needs, such as workforce size, knowledge and skills, and
training. Such planning involves assessing the knowledge and skills
needed to execute the program, inventorying existing staff knowledge
and skills, identifying any shortfalls, and providing for addressing
these shortfalls. Through effective workforce planning, programs and
organizations can have the right people with the right skills doing the
right jobs in the right place at the right time. Relevant architecture
management guidance[Footnote 47] recognizes the importance of planning
for and having adequate human resources in developing, maintaining, and
implementing enterprise architectures.
DOD has yet to perform workforce planning for its BEA program.
Nevertheless, it has established a program office consisting of seven
program divisions (see fig. 1) and staffed the office with 60
government employees and approximately 300 contractor staff. In
addition, the department has assigned other staff to support the
various program government entities, such as the domains and the DO/IT,
and it has established various formal and informal working groups.
However, DOD has not taken steps to ensure that the people assigned to
the program have the right skill sets or qualifications. In particular,
DOD has not defined the requisite workforce skills and abilities that
the department needs in order to develop, maintain, and implement the
architecture. To illustrate, the program's Assistant Deputy Director
for Administrative Support told us that the position descriptions used
to staff the program office were generic and were not tailored to the
needs of an enterprise architecture program. This official added that,
as a result, a person might satisfy the qualifications contained in the
position description, but still not meet the needs of the BEA program.
In addition to not defining its workforce needs, the program also does
not have an inventory of the capabilities of its currently assigned
program workforce. For example, the Assistant Deputy Director for
Administrative Support told us that the department did not know the
number of individuals assigned to support the various governance
entities.
In the absence of an inventory of existing workforce knowledge and
skills, we requested any available information on the qualifications
and training of program staff in key leadership positions (e.g.,
assistant deputy directors). In response, the Assistant Deputy Director
for Administrative Support said that such information was not readily
available for all staff, and this official provided us with résumés for
15 program officials, 4 of whose résumés we were told were created in
response to our request for key staff qualifications and training.
Exacerbating the program's lack of workforce planning is the fact that
several key program office positions are vacant. For example, four of
the seven program division leadership positions (i.e., assistant deputy
directors' positions) are temporarily filled by persons "acting" in
this position (see fig. 2). In addition, key supporting positions
within the program divisions, such as the positions for performance
management and independent verification and validation/quality
assurance, are vacant. As a result, 1 program official is currently
acting in three positions--strategic planning/organization development
(includes risk management), performance management (earned value
management system), and independent verification and validation/quality
assurance. Further, two of the positions this person occupies are
incompatible and do not allow for appropriate separation of
duties.[Footnote 48] Specifically, this individual is responsible for
both program performance management and independent oversight of
performance management.
Figure 2: Organization Chart of the Program Office:
[See PDF for image]
[End of figure]
In addition, significant staff turnover has occurred in key program
positions. For example, the program office has had three directors in 4
years, three transition planning directors since March 2004, and four
contracting officer technical representatives responsible for the prime
contract since January 2003.
The Assistant Deputy Director for Administrative Support stated that
the program lacks valid and reliable data about its human capital needs
and current capabilities. This official told us that plans are being
developed to begin addressing this situation. For example, to begin
monitoring staff turnover, the program office recently began
maintaining a list of program staff with start dates. However, this
official also told us that the plans for improving the program's
management of its human capital were not complete, and dates for when
the plans would be complete have not been set.
The absence of effective workforce planning has contributed
significantly to DOD's limited progress to date in developing its
architecture. Unless the program's approach to human capital management
improves, it is unlikely that the department's future efforts to
develop, maintain, and implement the BEA will be successful.
DOD Is Not Performing Effective Configuration Management:
Relevant architecture guidance,[Footnote 49] including DOD
guidance,[Footnote 50] recognizes the importance of configuration
management when developing and maintaining an architecture. The purpose
of configuration management is to maintain integrity and traceability,
and control modifications or changes to the architecture products
throughout their life cycles. Effective configuration management, among
other things, enables integration and alignment among related
architecture products. As we have previously reported,[Footnote 51] an
effective configuration management process comprises four primary
elements, each of which should be described in a configuration
management plan and implemented according to the plan. In addition,
responsibility, accountability, and authority for configuration
management should be assigned to a configuration manager. The four
elements are:
* Configuration identification: Procedures for identifying,
documenting, and assigning unique identifiers (e.g., serial number and
name) to product types generated for the architecture program,
generally referred to as configuration items.
* Configuration control: Procedures for evaluating and deciding whether
to approve changes to a product's baseline configuration, generally
accomplished through configuration control boards, which evaluate
proposed changes on the basis of costs, benefits, and risks and decide
whether to permit a change.
* Configuration status accounting: Procedures for documenting and
reporting on the status of configuration items as a product evolves.
Documentation, such as historical change lists and original
architecture products, are generated and kept in a library, thereby
allowing organizations to continuously know the state of a product's
configuration and to be in a position to make informed decisions about
changing the configuration.
* Configuration auditing: Procedures for determining alignment between
the actual product and the documentation describing it, thereby
ensuring that the documentation used to support the configuration
control board's decision making is complete and correct. Configuration
audits, both functional and physical, are performed when a significant
product change is introduced, and they help to ensure that only
authorized changes are being made.
DOD has a draft configuration management plan and related procedures
that address all four of these areas. However, the plan is not being
followed. For example, according to the plan and procedures, certain
product types[Footnote 52] should be placed under configuration
management and be assigned a unique identifier. However, in one case,
the verification and validation contractor[Footnote 53] reported that
DOD had updated one of the BEA products (i.e., All View-1) that was
initially published in Release 2.3, but that this updated product was
not given a unique identifier and a new release date, and no entry was
made in the version history to enable stakeholders to differentiate
between the two versions.
Configuration naming conventions also have not been consistently
followed, resulting in the updates to a single document being given
different unique identifiers than the original document. For example,
the November 2003 configuration management plan had the unique
identifier "C0008_05_03_BMMP_Configuration_Management_Plan.doc," which
was comprised of the call number, the task number, the subtask number,
and the name of the document. However, the department later assigned
this document the following unique identifier
"Configuration_Management_Plan.doc," which did not include the call and
task numbers. Such inconsistencies could permit changes to be made to
the wrong version of a product, thereby compromising the integrity and
reliability of the information.
Consistent with relevant guidance, the procedures require that a
configuration manager be assigned and that this individual be
responsible for ensuring that the four elements are executed. However,
after almost 4 years of architecture products development, a
configuration manager has not been assigned. In addition, while the
department established a configuration control board and chartered it
to evaluate and decide whether to approve proposed product changes,
this board is not fully functioning. Specifically, the board's charter
has yet to be approved, and its authority has been limited to a subset
of BEA products. For example, its authority does not extend to the BEA
transition plan.
With respect to configuration status accounting activities that have
been conducted to ensure the integrity of product baselines,[Footnote
54] we were provided with two reports even though program officials,
including the Configuration Control Board Chair, told us that no
configuration status accounting reports exist and that neither do any
auditing procedures and processes (e.g., audit checklists, agendas, or
plans). However, one of these reports was missing key data, such as the
date of the report, the submitter, and the version of the product
currently being reviewed and, thus, was of limited use. It was also
unclear as to which version of the product was referenced in the
report, and these officials told us that the current baseline of
approved configuration items, including the configuration management
plan, is unknown. As a result, configuration items can be duplicated.
For example, the Acting Assistant Deputy Director for Communications
had a second communications plan prepared, and this official told us
that he did not know that a prior draft plan existed. Because of this,
the new strategy did not leverage any of the work that had previously
been done, and duplicative plans exist.
Program officials, including the Configuration Control Board Chair,
stated that they recognize the importance of effective configuration
management. They attributed the absence of effective configuration
management to a number of factors, including no policy or directive
requiring it and the lack of a common understanding of effective
configuration management practices.
The absence of effective configuration management raises questions
about whether changes to the BEA and other relevant products have been
justified and accounted for in a manner that maintains the integrity of
the documentation. Unless this situation is remedied, the department
will not have adequate assurance that it has maintained accountability
and ensured the consistency of information among the products it is
developing. In addition to the governance and planning weaknesses we
previously discussed, the department's lack of effective configuration
management has also contributed to the state of the BEA discussed in
the next section of this report.
DOD Has Yet to Develop a Well-Defined BEA to Guide Its Modernization
Efforts:
Despite six BEA releases and two updates, DOD still does not have a
version of an enterprise architecture that can be considered well
defined, meaning that the architecture, for example, has a clearly
defined purpose that can be linked to the department's goals and
objectives and describes both the "As Is" and the "To Be" environments;
consists of integrated and consistent architecture products; and has
been approved by department leadership. According to program officials,
the state of the BEA reflects the program's focus on meeting arbitrary
milestones rather than architecture content needs. Recognizing the need
to develop well-defined architecture products that have utility and
provide a foundation upon which to build, program officials have stated
the department's intent to change its BEA development approach,
refocusing its efforts on fewer, higher quality products. Until a BEA
development approach embodying architecture development and content
best practices is clearly defined and implemented, it is not likely
that DOD will produce an enterprise architecture that will provide
needed content and utility.
"As Is" Description, Transition Plan, and Purpose of BEA Releases Are
Missing:
As we previously discussed, the various frameworks used to develop
architecture products consistently provide for describing a given
enterprise in both logical (e.g., business, performance, application,
and information) and technical (e.g., hardware, software, and data)
terms, and for doing so for the enterprise's current or "As Is"
environment and its target or "To Be" environment; these frameworks
also provide for defining a capital investment sequencing plan to
transition from the "As Is" to the "To Be" environment. However, the
frameworks do not prescribe the degree to which the component parts
should be described to be considered correct, complete, understandable,
and usable--essential attributes of any architecture. This is because
the depth and detail of the descriptive content depends on what the
architecture is to be used for (i.e., its intended purpose and scope).
Relevant architecture guidance[Footnote 55] states that a well-defined
architecture should have a specific and commonly understood purpose and
scope, and that it should be developed in incremental releases. Using
this purpose and scope, the necessary content of architecture releases
can then be defined.
In September 2003,[Footnote 56] we reported that Release 1.0 of the BEA
was missing important content, and we made 62 recommendations to add
this content. The latest releases of the BEA (see table 4) do not
address the 32 of our 62 recommendations that are related to the "As
Is" description and the transition plan.[Footnote 57] Specifically, the
releases do not include products describing the "As Is" environment for
those areas of the enterprise that are likely to change, and they do
not include a sequencing plan that serves as a road map for
transitioning from the "As Is" state to the "To Be" state. For example,
the BEA releases do not contain a description of the "As Is"
environment that would include current business operations in terms of
the entities or people who perform the functions, processes, and
activities, and the locations where the functions, processes, and
activities are performed. It also does not describe the data or
information being used by the functions, processes, and activities. As
a result, DOD does not have a picture of its current environment to
permit development of a meaningful and useful transition plan.
The BEA releases also do not contain a transition plan that would
include information such as time frames for phasing out existing
systems within DOD's current inventory and resource requirements for
implementing the "To Be" architecture. As a result, DOD does not yet
have a meaningful and reliable basis for managing the disposition of
its existing inventory of systems or for sequencing the introduction of
modernized business operations and supporting systems. As we previously
reported,[Footnote 58] not having defined the "As Is" operations and
technology at this juncture is risky because it defers until too late
in the architecture development cycle creation of sufficient
descriptive content and context to develop an effective transition
plan. (See app. II for our prior recommendations and their current
status.) DOD's architecture framework (DODAF),[Footnote 59] which the
department is using to develop the BEA, does not require the
development of an "As Is" architectural description or a transition
plan, and thus neither has been the focus of the program. However,
according to program officials, including the Chief Architect, the
September 2005 BEA release will include both the "As Is" architectural
description and a transition plan.
In addition, DOD has not clearly linked the purpose of any of the "To
Be" architecture releases produced to date to the goals and objectives
of increment 1. Further, these releases also do not have a clearly
defined scope with respect to what business processes and supporting
systems each release would focus on. According to program officials,
the last five versions (i.e., Releases 2.1, 2.2, and 2.3 and the
January and March 2005 Updates) support the objectives of increment
1[Footnote 60] (see table 3). These objectives are broad-based
strategic goals or outcomes that DOD proposed achieving through a
series of architecture releases. However, DOD did not define how many
releases were needed to achieve each objective and how the purpose of
each release is associated with the objectives. Restated, while each
incremental objective would describe the mission outcome that DOD
intended to achieve via implementation of the series of releases that
make up that increment, the purpose of the releases was not specified
in terms of architectural questions that were related to the
objectives. To illustrate, one objective of increment 1 is to achieve
an unqualified audit opinion for consolidated DOD financial statements.
Examples of the purpose questions that would support this objective
include the following:
1. What changes need to be made to existing business processes and the
supporting systems to address material internal control weaknesses
affecting significant line items on the financial statements?
2. Where are gaps in IT support (systems functions) that need to be
addressed in order to provide the business capabilities for ensuring
that property, plant, and equipment are appropriately valued and
recorded on the financial statements?
The department did not include architecture products that would answer
these types of questions to support the increment 1 objective. As a
result, the context needed to plan and measure content sufficiency was
not established. Program officials, including the Special Assistant for
Business Transformation and the Chief Architect agreed, stating that
prior releases have not included a specific purpose and scope.
Moreover, the Chief Architect told us that the architecture releases do
not fully support increment 1's objectives, nor do they describe the
extent to which they do address the increment 1 objectives.
According to program officials, including the deputy program director
and the Chief Architect, the prior releases were developed with the
goal of producing as many architecture products as possible within a
predefined schedule. The releases were not developed to provide the
content needed to meet the defined purpose and scope of the release.
Until the department defines the intended purpose and scope of its BEA,
including its incremental releases, and ensures that the architecture
products include adequate descriptions of the "As Is" and "To Be"
environments, as well as a plan for transitioning between the two, it
will not have a well-defined architecture to guide and constrain its
systems modernization efforts.
BEA Products Are Incomplete, Inconsistent, and Not Integrated:
Architecture frameworks[Footnote 61] provide for multiple products,
each describing a particular aspect of the enterprise, such as data or
systems. Moreover, each of these products is interdependent, meaning
that they have relationships with one another that must be explicitly
defined and maintained to ensure that the products form a coherent
whole. In light of these relationships, it is important to develop the
architecture products in a logical sequence that reflects this
relationship. DODAF[Footnote 62] recognizes this need for integration
of the products that make up its three "To Be" views--operational view
(OV), systems view (SV), and technical standards view (TV). (See app.
IV for a brief description of the products that comprise each of these
views.) According to the framework, an architecture must be well
structured so that the products can be readily assembled or decomposed
into higher or lower levels of detail, as needed. It should also be
consistent--that is, information elements should be the same throughout
the architecture.
As noted in the previous section, we reported in September
2003,[Footnote 63] that Release 1.0 of the BEA was missing important
content, and we made 62 recommendations to add this content. The latest
releases of the BEA do not adequately address our 30 prior
recommendations related to the "To Be" description.[Footnote 64] For
example, these releases do not include descriptions of the actual
systems to be developed or acquired to support future business
operations and the physical infrastructure (e.g., hardware and
software) that will be needed to support the business systems. As a
result, the "To Be" environment lacks the detail needed to provide DOD
with a common vision and frame of reference for defining a transition
plan to guide and constrain capital investments and, thus, to
effectively leverage technology to orchestrate logical, systematic
change and to optimize enterprisewide mission performance. (See app. II
for details on the status of our prior recommendations.)
In addition, the respective products of each of the latest BEA
releases[Footnote 65] continue to be inconsistent and not integrated,
because key architecture products were either not developed or not
updated to reflect changes made in other products. Examples of where
Releases 2.2 and 2.3 are not consistent and integrated follow:
* In Release 2.2, the department updated the system data exchange
matrix (SV-6), which assigns attributes (e.g., timeliness) to the data
to be exchanged (e.g., Performance Metrics) between system functions--
"Manage Business Enterprise Reporting" and "Establish and Maintain
Performance Information"--to support business operations. However, the
OV-3 in Release 2.2, which shows the attributes of the information to
be exchanged to support operations, is not consistent with the
attributes defined in the SV-6. For example, in the OV-3, the attribute
referred to as "timeliness" is defined in terms of either "hours,"
"minutes," or "seconds;" however, in the SV-6, the attribute referred
to as "timeliness" is only defined in terms of either "high, medium, or
basic."
* In Releases 2.2 and 2.3, the department updated the respective
operational event-trace description product (OV-6c), which depicts when
activities are to occur within operational processes. However, the
department did not update, in either release, the operational activity
model (OV-5), which shows the operational activities (or tasks) that
are to occur and the input and output flows among these activities. For
example, the OV-6c shows the sequence of the activities to occur for
the process labeled "produce obligation reports;" however, the
activities shown in the OV-5 were not associated with this process.
The latest releases also do not provide for traceability among the
architecture products by clearly identifying the linkages and
dependencies among the products, such as associating processes (e.g.,
produce obligation reports) with activities (e.g., compare expense to
obligation) in the operational views and then associating these same
processes to systems (e.g., financial reporting system) in the systems
view. In addition, the linkage between the two functions (i.e., "Manage
Business Enterprise Reporting" and "Establish and Maintain Performance
Information") previously discussed cannot be traced to the OV-3 in
Release 2.2. This is because Release 2.2 did not include an SV-5, which
would provide a traceability of system functions back to operational
activities. The lack of an updated SV-5 also raises questions as to
whether all operational requirements are satisfied by the system
functions. In addition, the architecture products were not developed in
a logical sequence, as called for in relevant guidance and
standards[Footnote 66] (e.g., the OV-6c, which shows the timing or
sequencing of activities, was built before the OV-5, which shows the
activities that are to occur).
Further, according to the verification and validation contractor, the
department has yet to address 242 of its 299 outstanding comments since
Release 1.0. The verification and validation contractor also cited
similar concerns, as previously described, for Releases 2.2 and 2.3.
Specifically, the contractor reported that the BEA products were not
integrated, and that key products were missing or had not been updated-
-such as the operational nodes connectivity description (OV-2) and the
operational information exchange matrix (OV-3). In its report on the
January 2005 Update,[Footnote 67] the contractor stated that the
architecture products were developed in an order different from that
recommended in DODAF, and that the dependency relationships between and
among BEA products were not clearly depicted. For example, the
contractor reported that the logical data model (OV-7) was to have been
developed using the OV-3, OV-5, and OV-6c artifacts as inputs. However,
this was not the case. Instead, the OV-7 was developed using
information that may have been reverse engineered from existing systems
and architectures external to the BEA. The contractor reported that
unless these dependencies are clearly documented and depicted, new
systems may be implemented without satisfying all operational
requirements, with missing functions and interfaces or based on
obsolete data models, recreating many of the problems the modernization
is intended to resolve. The contractor also reported that the resulting
technical problems in the OV-7 could interfere with the department's
achievement of the increment 1 objectives.
The March 2005 Update also did not have fully integrated products.
Specifically, while some of the products were integrated, this
integration occurred at the highest level only and could not be found
at lower levels of decomposition (e.g., subprocesses and subactivities)
within the architecture. For example, the level of integration does not
enable the user to determine all information inputs for the activities
at all levels, nor does it clearly reflect the dissemination or use of
the information after it has been processed. In addition, this update
did not include key architecture products that are recommended by
DODAF--such as the system data exchange artifact (SV-6), which assigns
attributes (e.g., timeliness) to the data to be exchanged between
system functions and the system inventory (SV-8). The SV-8 provides a
basis for portfolio investment decisions by depicting the evolution of
systems, systems integration, and systems improvements over time. We
also found that the architecture was not user friendly in that it was
difficult to navigate. For example, the linkages among the architecture
products did not always work, thereby, requiring manual navigation
through the architecture to find the linkages. This would take hours to
do, especially since it was complicated by the fact that certain
artifacts (e.g., diagrams) could not be read online and had to be
printed.
Relatedly, as shown in table 4, the latter BEA releases have not
included all of the recommended DODAF products. DODAF recommends that
the BEA include 23 out of 26 possible architecture products to meet the
department's stated intention to use the BEA as the basis for
departmentwide business and systems modernization. However, Release 2.2
of the architecture included only 16 of the 23 recommended architecture
products, and 6 of the 16 products (OV-1, OV-2, OV-3, OV-4, OV-5, and
SV-9) were actually Release 2.0 products that had not been updated to
align with the changes that had been made to the 10 products that were
updated in this release. Similarly, Release 2.3 included only 4
products; the January 2005 Update included 6, and the March 2005 Update
included 15. According to the Chief Architect, all prior architecture
products that were not included in a specific release or update are
obsolete. For example, Release 2.2 included 16 architecture products,
of which 10 had been updated. The remaining 6 products had not been
updated, but they were still considered to be valid because they were
included in this release. This means that for all releases and updates,
only those products included in the release or update are relevant. For
example, DOD updated 15 products and included them in the March 2005
Update; therefore, as of March 2005, only these 15 products were
considered to be valid artifacts of the BEA.
Table 4: List of DOD's Architecture Framework Products Included in Each
Release:
Product title[A]: All view (AV)[H]: AV-1 Overview and Summary
Information;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: Yes;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: Yes;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: All view (AV)[H]: AV-2 Integrated Dictionary;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: Yes;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-1 High-Level Operational
Concept Graphic;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-2 Operational Node
Connectivity Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-3 Operational
Information Exchange Matrix;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-4 Organizational
Relationships Chart;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Operational view (OV)[I]: OV-5 Operational Activity
Model;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-6a Operational Rules
Model;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-6b Operational State
Transition Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Operational view (OV)[I]: OV-6c Operational Event-
Trace Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: No;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: Yes;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: Yes;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Operational view (OV)[I]: OV-7 Logical Data Model;
Recommended[B]: No;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: Yes;
Releases and dates issued: Jan: 2005 Update[D,G]: Yes;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Systems view (SV)[J]: SV-1 Systems Interface
Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Systems view (SV)[J]: SV-2 Systems Communications
Description;
Recommended[B]: No;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: No;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-3 Systems-Systems Matrix;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-4 Systems Functionality
Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Systems view (SV)[J]: SV-5 Operational Activity to
Systems Traceability Matrix;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Systems view (SV)[J]: SV-6 Systems Data Exchange
Matrix;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-7 Systems Performance
Parameters Matrix;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-8 Systems Evolution
Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: No;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-9 Systems Technology
Forecast;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Systems view (SV)[J]: SV-10a Systems Rules Model;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: No;
Releases and dates issued: 2.0,[C] Feb 2004: No;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-10b Systems State Transition
Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: No;
Releases and dates issued: 2.0,[C] Feb 2004: No;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-10c Systems Event-Trace
Description;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: No.
Product title[A]: Systems view (SV)[J]: SV-11 Physical Schema: N/A.
Product title[A]: Technical standards view (TV)[K]: TV-1 Technical
Standards Profile;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: Yes;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Product title[A]: Technical standards view (TV)[K]: TV-2 Technical
Standards Forecast;
Recommended[B]: Yes;
Releases and dates issued: 1.0, May 2003: Yes;
Releases and dates issued: 2.0,[C] Feb 2004: Yes;
Releases and dates issued: 2.1,[D] Apr 2004: No;
Releases and dates issued: 2.2,[D,E,F] July 2004: No;
Releases and dates issued: 2.3,[D] Nov 2004: No;
Releases and dates issued: Jan: 2005 Update[D,G]: No;
Releases and dates issued: Mar: 2005 Update[D,G]: Yes.
Source: GAO analysis based on DOD data.
[A] See appendix IV for a brief description of each product.
[B] Products that are recommended by DODAF based on DOD's intended use
of the BEA.
[C] According to the verification and validation contractor, the AV-1
included in this release had not been updated to align with changes the
department had made to the other products that were included in this
release.
[D] These releases do not include the "As Is" architectural description
and a transition plan.
[E] In August 2004, DOD updated this release to incorporate the
Standard Financial Information Structure (SFIS) in the OV-7 (Logical
Data Model). According to DOD, the SFIS will standardize the coding of
financial data. This updated release (Release 2.2.1) did not include
any additional architecture products.
[F] According to the AV-1 and the narrative summary, the following
architecture products had not been updated to align with changes the
department had made to the other products that were included in this
release: OV-1, OV-2, OV-3, OV-4, OV-5, and SV-9.
[G] According to program officials, the next BEA release (Release 3.0)
is due in September 2005. Until then, all releases will be referred to
as updates.
[H] All-view products are to provide for overarching aspects of the
architecture that relate to additional views (e.g., operational and
systems views) and provide the scope and context for the architecture
(e.g., to guide and constrain systems investment decisions).
[I] Operational view products are to depict the organizationwide
business environment and activities that need to occur to achieve the
"To Be" state.
[J] Systems view products are to describe the set of systems
capabilities that are to provide DOD with accurate, reliable, and
timely access to business management and associated financial
information.
[K] Technical standards view products contain the set of technology
constraints that will drive the manner of system implementation.
[End of table]
Program and contractor officials, including the Acting Assistant Deputy
Director for Transition Planning, stated that although the department's
first release of its architecture included a fairly consistent and
integrated set of architecture products, DOD's current releases do not
because the department did not update all the recommended architecture
products. These officials, including the Chief Architect, also stated
that, as a result, the utility of the architecture is limited. However,
according to key program officials, including the Special Assistant for
Business Transformation and the Chief Architect, the integration of the
architecture products was not the focus; rather, DOD's primary goal was
to produce as many products as it could within a specified time period
(see tables 3 and 4).
Recognizing these weaknesses, the Special Assistant for Business
Transformation stated that the department intends to reduce the scope
of the architecture and revise the development approach, which will be
reflected in the September 30, 2005, architecture release. However,
according to program officials, including the Special Assistant for
Business Transformation, the September 2005 BEA release will not be
comprehensive (i.e., it will not meet all the act's requirements).
Further, the department has yet to develop plans and a methodology to
execute this new focus and vet it through the department. Program
officials also stated that as a result of the new focus, they are
trying to decide which products from prior releases could be salvaged
and used.
Nevertheless, the department has spent almost 4 years and approximately
$318 million in obligations to develop an architecture that is
incomplete, inconsistent, and not integrated and, thus, has limited
utility. Until the department develops an approved, well-defined
architecture that includes a clear purpose and scope and integrated
products, it remains at risk of not achieving its intended business
modernization goals and of not having an architecture that the
stakeholders can use to guide and constrain ongoing and planned
business systems investments to prevent duplicative and
noninteroperable systems.
BEA Releases Have Not Been Approved:
Relevant architecture guidance[Footnote 68] state that architecture
versions should be approved by the committee overseeing the development
and maintenance of the architecture; the CIO; the chief architect; and
senior management, including the department head. Such approval
recognizes and endorses the architecture for what it is intended to be-
-a corporate tool for managing both business and technological change
and transformation. Consistent with guidance, DOD has stated its
intention to approve all BEA releases.
However, Release 1.0 of the BEA is the only release that DOD reports as
having been approved. As we previously reported,[Footnote 69] DOD
officials told us that Release 1.0 was approved by the former Executive
Committee, the department's CIO as a member of the Executive Committee,
and the DOD Comptroller on behalf of the Secretary of Defense in May
2003, but they also said that documentation to verify these approvals
did not exist.
Since Release 1.0, DOD has issued five additional releases and two
updates. None of these have been approved by any individual or
committee in the BEA governance structure. According to program
officials, including the Special Assistant for Business Transformation
and the Chief Architect, Release 3.0 of the BEA, which will be issued
in September 2005, will be the next release of the architecture to be
approved by the department. These officials stated that the
architecture releases have not been approved because the department did
not have a governance structure and process in place for doing so.
Without the appropriate approvals, buy-in to and recognition of the BEA
as an institutionally endorsed change management and transformation
tool is not achievable.
DOD Has Yet to Fully Address Most of Our Other Recommendations:
In addition to the governance, planning, and content issues previously
discussed, we have made other recommendations relative to DOD's ability
to effectively develop, maintain, and implement an enterprise
architecture for its business operations. To date, the department has
fully addressed one of our other recommendations, which is to report
every 6 months to the congressional committees on the status of the BEA
effort, but it has yet to fully address the remaining recommendations.
(See app. II for details on the status of these recommendations.) For
example, the department has yet to address our recommendations to:
* develop a position description for the Chief Architect that defines
requisite duties and responsibilities,
* update policies to assign responsibility and accountability for
approving BEA releases,
* update policies to address the issuance of waivers for business
systems that are not compliant with the architecture but are
nevertheless justified on the basis of documented analysis, and:
* develop and implement a quality assurance plan.
According to the program director and deputy director, the current
state of the BEA, including progress in addressing our recommendations,
reflects the program's prior focus on producing as many products as it
could within a specific time period. The focus had not been on the
content and quality of the releases, but rather on the timing of their
delivery. In contrast, our recommendations have all focused on
establishing the means by which to deliver a well-defined BEA and
ensuring that delivered releases of the architecture contain this
requisite content. Until DOD adopts the kind of approach embodied in
our recommendations, it is unlikely that it will produce a well-defined
BEA within reasonable time frames and at an affordable cost.
Conclusions:
Having and using a well-defined enterprise architecture are essential
for DOD to effectively and efficiently modernize its nonintegrated and
duplicative business operations and systems environment. However, the
department does not have such an architecture, and the architecture
products that it has produced to date do not provide sufficient content
and utility to effectively guide and constrain the department's ongoing
and planned business systems investments. This means that despite
spending almost 4 years and about $318 million to develop its BEA, the
department is not positioned to meet its legislative mandates. In our
view, the state of the architecture is due largely to long-standing
architecture management weaknesses that the recommendations we have
made over the last 4 years are aimed at correcting, as well as the
department's prior focus on producing as many products as it could
within a specific time period. To date, the department has not taken
adequate steps to implement most of our recommendations.
While recent steps to begin revamping its BEA governance structure and
to begin program planning are positive first steps and are consistent
with some of the recommendations that we made to lay a foundation for
architecture development, maintenance, and implementation, much more
remains to be accomplished. Thus, it is imperative for the department
to move swiftly to strengthen its BEA program in a manner that
incorporates our prior recommendations and recognizes its current
architecture management capabilities. Until it does, the department
will continue to put billions of dollars at risk of being invested in
systems that are duplicative, are not interoperable, cost more to
maintain than necessary, and do not optimize mission performance and
accountability.
Recommendations for Executive Action:
We recommend that the Secretary of Defense direct the Deputy Secretary
of Defense, as the chair of the DBSMC and in collaboration with DBSMC
members, to:
* immediately fully disclose the state of its BEA program to DOD's
congressional authorization and appropriations committees, including
its limited progress and results to date, as well as specific plans and
commitments for strengthening program management and producing
measurable results that reflect the department's capability to do so;
* ensure that each of our recommendations related to the BEA management
and content are reflected in the above plans and commitments; and:
* ensure that plans and commitments provide for effective BEA workforce
planning, including assessing workforce knowledge and skills needs,
determining existing workforce capabilities, identifying gaps, and
filling these gaps.
Agency Comments:
In written comments on a draft of this report signed by the Special
Assistant for Business Transformation in the Office of the Under
Secretary of Defense (Acquisition, Technology, and Logistics) and the
Deputy Under Secretary of Defense (Financial Management) (reprinted in
app. V), the department concurred with our recommendations and stated
its intent to implement them. Specifically, DOD stated that it would
(1) disclose plans, progress, and results of its BEA efforts to DOD's
congressional committees; (2) address our recommendations related to
BEA management and content; and (3) assess its workforce needs and
adjust its workforce to meet requirements.
We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the
Secretary of Defense; the Deputy Secretary of Defense; the Under
Secretary of Defense (Acquisition, Technology, and Logistics); the
Under Secretary of Defense (Comptroller); the Assistant Secretary of
Defense (Networks and Information Integration)/Chief Information
Officer; the Under Secretary of Defense (Personnel and Readiness); and
the Director, Defense Finance and Accounting Service. This report will
also be available at no charge on our Web site at [Hyperlink,
http://www.gao.gov].
If you or your staff have any questions on matters discussed in this
report, please contact me at (202) 512-3439 or [Hyperlink,
hiter@gao.gov]. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made major contributions to this report are
listed in appendix VI.
Signed by:
Randolph C. Hite:
Director:
Information Technology Architecture and Systems Issues:
[End of section]
Appendixes:
Appendix I: Objectives, Scope, and Methodology:
Our objectives were to determine whether the Department of Defense
(DOD) has (1) established an effective governance structure; (2)
developed program plans, including supporting workforce plans; (3)
performed effective configuration management; (4) developed well-
defined business enterprise architecture (BEA) products; and (5)
addressed our other recommendations.
To determine whether DOD has established an effective governance
structure for its efforts, we reviewed program documentation--such as
approved charters for the Executive and Steering Committees, the Domain
Owners Integration Team (DO/IT), the Transformation Support
Office,[Footnote 70] and the Defense Business Systems Management
Committee--and the communications strategy and supporting documents. We
compared these documents with the elements in our Enterprise
Architecture Management Maturity Framework and federal Chief
Information Officer (CIO) Council guidance.[Footnote 71]
To determine whether DOD has developed program plans, including
supporting workforce plans, we interviewed the Director and deputy
program director, and the assistant deputy directors for
communications, strategic planning, and transition planning. We also
reviewed draft plans that showed the department's intent to address our
prior recommendations for the content previously missing from the "As
Is" architecture and the transition plan. We also reviewed the
department's March 15, 2005, annual report to Congress,[Footnote 72]
briefing slides on the department's BEA development approach, and the
various statements of work for the contractor responsible for extending
and evolving the architecture. For human capital, we reviewed program
organization charts and position descriptions for key program
officials. In addition, we interviewed key program officials, such as
the assistant deputy directors for communications, strategic planning,
and enterprise architecture, to discuss their roles and
responsibilities.
To determine whether effective configuration management was being
performed, we reviewed the configuration management plan and associated
procedures,[Footnote 73] and the draft configuration control board
charter. We compared these documents with best practices, including the
federal CIO Council's Practical Guide, to determine the extent to which
DOD had adopted key management practices. In addition, we reviewed
meeting minutes to determine whether the board was operating
effectively and performing activities according to best practices. We
also interviewed program officials, including the Chief Architect and
the Configuration Control Board Chair, to discuss the process and its
effect on the department's ability to develop and maintain the BEA
products.
To determine whether DOD had developed well-defined BEA products, we
reviewed the latest BEA releases (i.e., Releases 2.2, 2.2.1, and 2.3
and the March 2005 Update)[Footnote 74] and the program's verification
and validation contractor's reports documenting its assessment of
Releases 2.2 and 2.3 and the January 2005 Update of the architecture.
To determine whether these BEA releases addressed our prior
recommendations on missing architecture content and inconsistencies, we
requested contractual change requests related to our recommendations.
Program officials, including the program director and Chief Architect,
stated that change requests to address our recommendations do not
exist. We also reviewed the verification and validation contractor's
assessment of DOD's efforts to address its outstanding comments on
prior versions of the BEA and DOD stakeholders'[Footnote 75] comments
on Release 2.2 of the BEA. Further, we reviewed DOD's approach to
developing the architecture products since Release 1.0 and compared it
with relevant guidance, such as DOD's architecture framework. We also
observed architecture walk-through sessions held by program officials
to discuss concerns and provide progress updates. In addition, we
interviewed program officials, including the Special Assistant for
Business Transformation, Deputy Under Secretary of Defense (Financial
Management), Chief Architect, the Configuration Control Board Chair,
and the verification and validation contractor to discuss the
development and maintenance of the BEA products.
To determine the status of DOD's efforts to address our other
recommendations related to BEA development and maintenance, we reviewed
program documentation, such as the draft quality assurance plan, and
compared them with the elements in our Enterprise Architecture
Management Maturity Framework. We requested updates to the Information
Technology Portfolio Management Policy and the position description for
the Chief Architect. We also interviewed program and contractor
officials, such as the Director and deputy program director, and the
assistant deputy directors for quality assurance and communications.
To augment our documentation reviews and analyses, we attended
regularly scheduled meetings, such as the DO/IT meetings, the program
execution status meetings, and configuration control board meetings. We
also held monthly teleconferences with the program and deputy program
directors to discuss any issues and to obtain explanations or
clarification on the results of our audit work.
We did not independently validate cost and budget information provided
by the department.
We conducted our work primarily at DOD headquarters in Arlington,
Virginia, and we performed our work from July 2004 through May 2005, in
accordance with generally accepted government auditing standards. We
requested comments on a draft of this report from the Secretary of
Defense or his designee. Written comments from the Special Assistant
for Business Transformation in the Office of the Under Secretary of
Defense (Acquisition, Technology, and Logistics) and the Deputy Under
Secretary of Defense (Financial Management) are addressed in the
"Agency Comments" section of this report and are reprinted in appendix
V.
[End of section]
Appendix II: Status of Prior Recommendations on DOD's Development and
Maintenance of Its Business Enterprise Architecture:
GAO-01-525: Information Technology: Architecture Needed to Guide
Modernization of DOD's Financial Operations. May 17, 2001:
GAO report information and recommendation: (1) The Secretary
immediately issue a Department of Defense (DOD) policy that directs the
development, implementation, and maintenance of a business enterprise
architecture (BEA).[A];
Implemented? Partial;
DOD comments and our assessment: DOD has developed the Information
Technology Portfolio Management policy. While this policy, in
conjunction with the overarching Global Information Grid[B] policy,
assigns responsibilities for the development, implementation, and
maintenance of the BEA, it does not provide for having accountability
for and approval of updates to the architecture processes for
architecture oversight and control and architecture review and
validation, and it does not address the issuance of waivers for
business systems that are not compliant with the BEA but are
nevertheless justified on the basis of documented analysis. Program
officials stated that the department plans to revise this policy, but
they did not provide a time frame for doing so.
GAO report information and recommendation: (2) The Secretary
immediately modify the Senior Financial Management Oversight Council's
charter to;
* designate the Deputy Secretary of Defense as the Council Chair and
the Under Secretary of Defense (Comptroller) as the Council vice-Chair;
and;
* empower the council to serve as DOD's BEA steering committee, giving
it the responsibility and authority to ensure that a DOD BEA is
developed and maintained in accordance with the DOD Command, Control,
Communications, Computers, Intelligence, Surveillance, and
Reconnaissance (C4ISR) Architecture Framework;
Implemented? Yes;
DOD comments and our assessment: We previously reported that DOD had
established the Executive and Steering Committees, which were advisory
in nature. The department also had established the Domain Owners
Integration Team (DO/IT) and stated that these three bodies were
responsible for governing the program. However, these groups had not
been assigned responsibilities for directing, overseeing, and approving
the BEA. According to key department officials, these three entities
will be replaced. Specifically, in February 2005, DOD established the
Defense Business Systems Management Committee (DBSMC), which replaced
the Executive and Steering Committees. According to its charter, the
DBSMC is accountable and responsible for the program. The department
plans to establish an underlying management structure to support the
DBSMC in carrying out its roles and responsibilities. In addition,
program officials have stated the department's intention to replace the
DO/IT with the DOD Enterprise Transformation Integration Group whose
roles and responsibilities and concept of operations have yet to be
defined.
GAO report information and recommendation: (3) The Secretary
immediately make the Assistant Secretary of Defense (Command, Control,
Communications & Intelligence), in collaboration with the Under
Secretary of Defense (Comptroller), accountable to the Senior Financial
Management Oversight Council for developing and maintaining a DOD BEA;
In fulfilling this responsibility, the Assistant Secretary appoint a
Chief Architect for DOD business management modernization and establish
and adequately staff and fund an enterprise architecture program office
that is responsible for developing and maintaining a DOD-wide BEA in a
manner that is consistent with the framework defined in the Chief
Information Officer (CIO) Council's published guide for managing
enterprise architectures. In particular, the Assistant Secretary should
take appropriate steps to ensure that the Chief Architect;
* obtains executive buy-in and support;
* establishes architecture management structure and controls;
* defines the architecture process and approach;
* develops the baseline architecture, the target architecture, and the
sequencing plan;
* facilitates the use of the architecture to guide business management
modernization projects and investments; and;
* maintains the architecture;
Implemented? Partial;
DOD comments and our assessment: The Assistant Secretary of Defense
(Networks and Information Integration)/Chief Information Officer
(ASD(NII)/CIO) is a member of the recently established DBSMC; however,
it is not known how this committee will operate; DOD established a
program office in July 2001. DOD also appointed a Chief Architect, and,
according to the department, it has adequate program funding and staff
for developing and maintaining its architecture. However, DOD has yet
to define the roles and responsibilities for the Chief Architect or
provide a time frame for doing so.
GAO report information and recommendation: (4) The ASD(NII)/CIO report
at least quarterly to the Senior Financial Management Oversight Council
on the Chief Architect's progress in developing a BEA, including the
Chief Architect's adherence to enterprise architecture policy and
guidance from the Office of Management and Budget (OMB), the CIO
Council, and DOD;
Implemented? Partial;
DOD comments and our assessment: The ASD(NII)/CIO is a member of the
recently established DBSMC; however, it is not known how this committee
will operate; The Steering Committee was briefed monthly by the program
office on various program activities until June 2004, when it held its
last meeting. As a result, the Steering Committee was not updated on
the content and status of Releases 2.2 and 2.3 and the January 2005 and
March 2005 Updates of the BEA. According to program officials, the
DBSMC held an executive session in February 2005 and its second meeting
in April 2005, and the committee will initially hold monthly meetings.
GAO report information and recommendation: (5) The Senior Financial
Management Oversight Council report to the Secretary of Defense every 6
months on progress in developing and implementing a BEA;
Implemented? Partial;
DOD comments and our assessment: The Deputy Chief Financial Officer
briefed the Secretary of Defense in November 2003 on behalf of DOD's
Comptroller, who chairs the Executive Committee. According to the
program director, the Secretary of Defense has not been briefed since
November 2003 on the department's progress in developing and
implementing the BEA.
GAO report information and recommendation: (6) The Secretary report
every 6 months to the congressional defense authorizing and
appropriating committees on progress in developing and implementing a
BEA;
Implemented? Yes;
DOD comments and our assessment: Senate Report 107-213 directs that the
department report every 6 months on the status of the BEA effort. DOD
submitted status reports on January 31 and July 31, 2003; January 31
and July 30, 2004; and March 15, 2005. The 2003 and 2004 reports were
submitted by DOD's Comptroller but were not signed by the members of
the Executive or Steering Committees. The 2005 report was signed by the
Acting Under Secretary of Defense for Acquisition, Technology, and
Logistics, who is the vice-chair of the DBSMC.
GAO-03-458: DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed.
February 28, 2003:
GAO report information and recommendation: (1) The Secretary of Defense
ensure that the enterprise architecture executive committee members are
singularly and collectively made explicitly accountable to the
Secretary for the delivery of the enterprise architecture, including
approval of each release of the architecture;
Implemented? Yes;
DOD comments and our assessment: We previously reported that DOD had
established the Executive and Steering Committees, which were advisory
in nature. The department had also established the DO/IT and stated
that these three bodies were responsible for governing the program.
However, these groups had not been assigned responsibilities for
directing, overseeing, and approving the BEA. According to key
department officials, these three entities will be replaced.
Specifically, in February 2005, DOD established the DBSMC, which
replaced the Executive and Steering Committees. According to its
charter, the DBSMC is accountable and responsible for the program. The
department plans to establish an underlying management structure to
support the DBSMC in carrying out its roles and responsibilities. In
addition, program officials have stated the department's intention to
replace the DO/IT with the DOD Enterprise Transformation Integration
Group, whose roles and responsibilities and concept of operations have
yet to be defined.
GAO report information and recommendation: (2) The Secretary of Defense
ensure that the enterprise architecture program is supported by a
proactive marketing and communication program;
Implemented? Partial;
DOD comments and our assessment: DOD has a strategic communications
plan; however, the plan has yet to be implemented. According to the
communications team, its activities have been limited to raising
awareness because it lacks the authority to fully implement the other
components of its plan, such as achieving buy-in. According to program
officials, the department intends to revise the governance structure,
including the communications strategy, in September 2005.
GAO report information and recommendation: (3) The Secretary of Defense
ensure that the quality assurance function;
* includes the review of adherence to process standards and reliability
of reported program performance,;
* is made independent of the program management function, and;
* is not performed by subject matter experts involved in the
development of key architecture products;
Implemented? Partial;
DOD comments and our assessment: DOD has established a quality
assurance function; however, this function does not address process
standards and program performance, nor is it an independent function.
Further, DOD subject matter experts continue to be involved in the
quality assurance function. Program officials stated that the
department had yet to address our recommendation, and they could not
provide a time frame for when they would begin addressing this
recommendation.
GAO-03-1018: DOD Business Systems Modernization: Important Progress
Made to Develop Business Enterprise Architecture, but Much Work
Remains. September 19, 2003:
GAO report information and recommendation: (1) The Secretary of Defense
or his appropriate designee implement the core elements in our
Enterprise Architecture Framework for Assessing and Improving
Enterprise Architecture Management that we identify in this report as
not satisfied, including ensuring that minutes of the meetings of the
executive body charged with directing, overseeing, and approving the
architecture are prepared and maintained;
Implemented? Partial;
DOD comments and our assessment: DOD has taken some actions, but these
actions do not fully address our previous concerns. For example, DOD
has;
* begun to revise its governance structure to provide for improved
management and oversight, such as establishing the DBSMC and assigning
it accountability and responsibility for directing, overseeing, and
approving the BEA; and;
* developed a configuration management plan and the related procedures,
and established a configuration control board; However, the department
has not;
* established additional governance entities to support the DBSMC and
outlined their roles and responsibilities;
* updated the policy for BEA development, maintenance, and
implementation;
* included the missing scope and detail in the BEA;
* finalized, approved, and effectively implemented the plan,
procedures, and charter governing the configuration management process;
and;
* developed specific results-oriented performance metrics.
GAO report information and recommendation: (2) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
include the 29 key elements governing the "As Is" architectural content
that our report identified as not being fully satisfied;
Implemented? No;
DOD comments and our assessment: Of the 29 elements, program officials
stated that 3 were not applicable and that it planned to address an
additional 11 by January 2005. However, these officials did not provide
any documentation to support this statement. Instead, they provided a
draft plan that shows the department's intent to develop a detailed
action plan to guide the development of an "As Is" architecture.
According to program officials, they plan to update the "As Is"
architectural description in September 2005.
GAO report information and recommendation: (3) The Secretary of Defense
or his appropriate designee update version 1.0 of the BEA to include
the 30 key elements governing the "To Be" architectural content that
our report identified as not being fully satisfied;
Implemented? No;
DOD comments and our assessment: DOD officials have provided no
evidence that this recommendation has been addressed or that it intends
to implement this recommendation.
GAO report information and recommendation: (4) The Secretary of Defense
or his appropriate designee update version 1.0 to ensure that "To Be"
architecture artifacts are internally consistent, to include addressing
the inconsistencies described in this report, as well as including user
instructions or guidance for easier architecture navigation and use;
Implemented? No;
DOD comments and our assessment: DOD officials have provided no
evidence that this recommendation has been addressed or that it intends
to implement this recommendation.
GAO report information and recommendation: (5) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
include (a) the 3 key elements governing the transition plan content
that our report identified as not being fully satisfied and (b) those
system investments that will not become part of the "To Be"
architecture, including time frames for phasing out those systems;
Implemented? No;
DOD comments and our assessment: DOD officials provided a draft plan
that shows the department's intent to develop a detailed action plan to
guide the development of the transition plan; however, the draft plan
does not provide time frames for doing so. According to program
officials, the department will issue a revised transition plan in
September 2005; but, this version will not fully address our
recommendation.
GAO report information and recommendation: (6) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
address comments made by the verification and validation contractor;
Implemented? No;
DOD comments and our assessment: According to program officials, of the
299 outstanding comments, 137 have been addressed in Release 2.3 and
earlier releases, 100 were not applicable, and the remaining 62 will be
addressed in future releases. These officials did not provide any
documentation supporting their rationale for the 100 that they
considered not applicable nor did they provide plans for addressing the
62 remaining comments. The verification and validation contractor
stated that of the 137 comments that program officials stated had been
addressed, 35 had been addressed, 22 were not applicable because they
were either duplicate or no longer relevant based on updates to prior
releases, 22 had yet to be addressed, and 58 were not assessed. The
contractor has yet to provide its assessment on the 100 comments that
DOD said were not applicable.
GAO report information and recommendation: (7) The Secretary of Defense
or his appropriate designee develop a well-defined, near-term plan for
extending and evolving the architecture and ensure that this plan
includes addressing our recommendations, defining roles and
responsibilities of all stakeholders involved in extending and evolving
the architecture, explaining dependencies among planned activities, and
defining measures of activity progress;
Implemented? No;
DOD comments and our assessment: As discussed in this report, DOD has
not developed explicit detailed plans to guide day-to-day program
activities and to enable it to evaluate its progress. According to
program officials, the department will develop a program baseline by
September 30, 2005.
Source: GAO.
[A] On May 20, 2003, the Under Secretary of Defense (Comptroller)
issued a memorandum that renamed and updated the Financial Management
Modernization Program to the Business Management Modernization Program.
In addition, the Financial Management Enterprise Architecture was
renamed the Business Enterprise Architecture.
[B] DOD defines the Global Information Grid as the globally
interconnected, end-to-end set of information, capabilities, associated
processes, and personnel for collecting, processing, storing,
disseminating, and managing information on demand to warfighters,
policy makers, and support personnel.
[End of table]
[End of section]
Appendix III: Summary of Several Architecture Frameworks:
There are various enterprise architecture frameworks that an
organization can follow. Although these frameworks differ in their
nomenclatures and modeling approaches, they consistently provide for
defining an enterprise's operations in both (1) logical terms, such as
interrelated business processes and business rules, information needs
and flows, and work locations and users, and (2) technical terms, such
as hardware, software, data, communications, and security attributes
and performance standards. The frameworks also provide for defining
these perspectives for both the enterprise's current or "As Is"
environment and its target or "To Be" environment, as well as a
transition plan for moving from the "As Is" to the "To Be" environment.
For example, John A. Zachman developed a structure or framework for
defining and capturing an architecture.[Footnote 76] This framework
provides for six windows from which to view the enterprise, which
Zachman terms "perspectives" on how a given entity operates: those of
(1) the strategic planner, (2) the system user, (3) the system
designer, (4) the system developer, (5) the subcontractor, and (6) the
system itself. Zachman also proposed six models that are associated
with each of these perspectives; these models describe (1) how the
entity operates, (2) what the entity uses to operate, (3) where the
entity operates, (4) who operates the entity, (5) when entity
operations occur, and (6) why the entity operates. Zachman's framework
provides a conceptual schema that can be used to identify and describe
an entity's existing and planned components and their relationships to
one another before beginning the costly and time-consuming efforts
associated with developing or transforming the entity.
Since Zachman introduced his framework, a number of other frameworks
has been proposed. In February 1998, DOD directed its components to use
its C4ISR Architecture Framework, Version 2.0. In August 2003, the
department released Version 1.0 of the DOD Architecture Framework
(DODAF)[Footnote 77]--an evolution of the C4ISR Architecture Framework,
which supersedes the C4ISR. The DODAF defines the type and content of
the architectural artifacts, as well as the relationships among the
artifacts that are needed to produce a useful architecture. Briefly,
the framework decomposes an architecture into three primary views:
operational, systems, and technical standards.[Footnote 78] See figure
3 for an illustration of these three views. According to DOD, the three
interdependent views are needed to ensure that IT systems support
operational needs, and that they are developed and implemented in an
interoperable and cost-effective manner.
Figure 3: Interdependent DODAF Views of an Architecture:
[See PDF for image]
[End of figure]
In September 1999, the federal CIO Council published the Federal
Enterprise Architecture Framework (FEAF), which is intended to provide
federal agencies with a common construct on which to base their
respective architectures and to facilitate the coordination of common
business processes, technology insertion, information flows, and system
investments among federal agencies. FEAF describes an approach,
including models and definitions, for developing and documenting
architecture descriptions for multiorganizational functional segments
of the federal government. Similar to most frameworks, FEAF's proposed
models describe an entity's business, the data necessary to conduct the
business, applications to manage the data, and technology to support
the applications.
More recently, the Office of Management and Budget (OMB) established
the Federal Enterprise Architecture (FEA) Program Management Office to
develop a federated enterprise architecture in the context of five
"reference models, and a security and privacy profile that overlays the
five models."
* The Business Reference Model is intended to describe the federal
government's businesses, independent of the agencies that perform them.
This model consists of four business areas: (1) services for citizens,
(2) mode of delivery, (3) support delivery of services, and (4)
management of government resources. It serves as the foundation for the
FEA. OMB expects agencies to use this model, as part of their capital
planning and investment control processes, to help identify
opportunities to consolidate information technology (IT) investments
across the federal government. Version 2.0 of this model was released
in June 2003.
* The Performance Reference Model is intended to describe a set of
performance measures for major IT initiatives and their contribution to
program performance. According to OMB, this model will help agencies
produce enhanced performance information; improve the alignment and
better articulate the contribution of inputs, such as technology, to
outputs and outcomes; and identify improvement opportunities that span
traditional organizational boundaries. Version 1.0 of this model was
released in September 2003.
* The Service Component Reference Model is intended to identify and
classify IT service (i.e., application) components that support federal
agencies and promote the reuse of components across agencies. This
model is intended to provide the foundation for the reuse of
applications, application capabilities, components (defined as "a self-
contained business process or service with predetermined functionality
that may be exposed through a business or technology interface"), and
business services. According to OMB, this model is a business-driven,
functional framework that classifies service components with respect to
how they support business and/or performance objectives. Version 1.0 of
this model was released in June 2003.
* The Data Reference Model is intended to describe, at an aggregate
level, the types of data and information that support program and
business line operations and the relationships among these types. This
model is intended to help describe the types of interactions and
information exchanges that occur across the federal government. Version
1.0 of this model was released in September 2004.
* The Technical Reference Model is intended to describe the standards,
specifications, and technologies that collectively support the secure
delivery, exchange, and construction of service components. Version 1.1
of this model was released in August 2003.
* The Security and Privacy Profile is intended to provide guidance on
designing and deploying measures that ensure the protection of
information resources. OMB has released Version 1.0 of the profile.
[End of section]
Appendix IV: Description of DOD Architecture Framework Products,
Version 1.0:
Product: All view (AV): AV-1; Product title: All view (AV): Overview
and Summary Information;
Product description: All view (AV): Executive-level summary information
on the scope, purpose, and context of the architecture.
Product: All view (AV): AV-2; Product title: All view (AV): Integrated
Dictionary;
Product description: All view (AV): Architecture data repository with
definitions of all terms used in all products.
Product: Operational view (OV): OV-1; Product title: All view (AV):
High-Level Operational Concept Graphic;
Product description: All view (AV): High-level graphical/textual
description of what the architecture is supposed to do, and how it is
supposed to do it.
Product: Operational view (OV): OV-2; Product title: All view (AV):
Operational Node Connectivity Description;
Product description: All view (AV): Graphic depiction of the
operational nodes (or organizations) with needlines that indicate a
need to exchange information.
Product: Operational view (OV): OV-3; Product title: All view (AV):
Operational Information Exchange Matrix;
Product description: All view (AV): Information exchanged between nodes
and the relevant attributes of that exchange.
Product: Operational view (OV): OV-4; Product title: All view (AV):
Organizational Relationships Chart;
Product description: All view (AV): Command structure or relationships
among human roles, organizations, or organization types that are the
key players in an architecture.
Product: Operational view (OV): OV-5; Product title: All view (AV):
Operational Activity Model;
Product description: All view (AV): Operations that are normally
conducted in the course of achieving a mission or a business goal, such
as capabilities, operational activities (or tasks), input and output
flows between activities, and input and output flows to/from activities
that are outside the scope of the architecture.
Product: Operational view (OV): OV-6a; Product title: All view (AV):
Operational Rules Model;
Product description: All view (AV): One of three products used to
describe operational activity--identifies business rules that constrain
operations.
Product: Operational view (OV): OV-6b; Product title: All view (AV):
Operational State Transition Description;
Product description: All view (AV): One of three products used to
describe operational activity--identifies business process responses to
events.
Product: Operational view (OV): OV-6c; Product title: All view (AV):
Operational Event-Trace Description;
Product description: All view (AV): One of three products used to
describe operational activity--traces actions in a scenario or sequence
of events.
Product: Operational view (OV): OV-7; Product title: All view (AV):
Logical Data Model;
Product description: All view (AV): Documentation of the system data
requirements and structural business process rules of the operational
view.
Product: Systems view (SV): SV-1; Product title: All view (AV): Systems
Interface Description;
Product description: All view (AV): Identification of systems nodes,
systems, and systems items and their interconnections, within and
between nodes.
Product: Systems view (SV): SV-2; Product title: All view (AV): Systems
Communications Description;
Product description: All view (AV): Specific communications links or
communications networks and the details of their configurations through
which systems interface.
Product: Systems view (SV): SV-3; Product title: All view (AV): Systems-
Systems Matrix;
Product description: All view (AV): Relationships among systems in a
given architecture; can be designed to show relationships of interest
(e.g., system-type interfaces, planned versus existing interfaces).
Product: Systems view (SV): SV-4; Product title: All view (AV): Systems
Functionality Description;
Product description: All view (AV): System functional hierarchies and
system functions, and the system data flow between them.
Product: Systems view (SV): SV-5; Product title: All view (AV):
Operational Activity to Systems Function Traceability Matrix;
Product description: All view (AV): Mapping of relationships between
the set of operational activities and the set of system functions
applicable to that architecture.
Product: Systems view (SV): SV-6; Product title: All view (AV): Systems
Data Exchange Matrix;
Product description: All view (AV): Characteristics of the system data
exchanged between systems.
Product: Systems view (SV): SV-7; Product title: All view (AV): Systems
Performance Parameters Matrix;
Product description: All view (AV): Quantitative characteristics of
systems and systems hardware/software items, their interfaces, and
their functions.
Product: Systems view (SV): SV-8; Product title: All view (AV): Systems
Evolution Description;
Product description: All view (AV): Planned incremental steps toward
migrating a suite of systems to a more efficient suite, or toward
evolving a current system to a future implementation.
Product: Systems view (SV): SV-9; Product title: All view (AV): Systems
Technology Forecast;
Product description: All view (AV): Emerging technologies and
software/hardware products that are expected to be available in a given
set of time frames and that will affect future development of the
architecture.
Product: Systems view (SV): SV-10a; Product title: All view (AV):
Systems Rules Model;
Product description: All view (AV): One of three products used to
describe system functionality--identifies constraints that are imposed
on systems functionality due to some aspect of systems design or
implementation.
Product: Systems view (SV): SV-10b; Product title: All view (AV):
Systems State Transition Description;
Product description: All view (AV): One of three products used to
describe system functionality--identifies responses of a system to
events.
Product: Systems view (SV): SV-10c; Product title: All view (AV):
Systems Event-Trace Description;
Product description: All view (AV): One of three products used to
describe system functionality--lays out the sequence of system data
exchanges that occur between systems (external and internal), system
functions, or human role for a given scenario.
Product: Systems view (SV): SV-11; Product title: All view (AV):
Physical Schema;
Product description: All view (AV): Physical implementation of the
Logical Data Model entities (e.g., message formats, file structures,
and physical schema).
Product: Technical standards view (TV): TV-1; Product title: All view
(AV): Technical Standards Profile;
Product description: All view (AV): Listing of standards that apply to
systems view elements in a given architecture.
Product: Technical standards view (TV): TV-2; Product title: All view
(AV): Technical Standards Forecast;
Product description: All view (AV): Description of emerging standards
and the potential impact on current systems view elements, within a set
of time frames.
Source: DOD.
[End of table]
[End of section]
Appendix V: Comments from the Department of Defense:
OFFICE OF THE UNDER SECRETARY OF DEFENSE:
ACQUISITION, TECHNOLOGY AND LOGISTICS:
3000 DEFENSE PENTAGON:
WASHINGTON, DC 20301-3000:
Mr. Randolph C. Hite:
Director, Information Technology Architecture and Systems Issues:
U.S. Government Accountability Office:
Washington, DC 20548:
JUL 8 2005:
Dear Mr. Hite:
Enclosed is the Department of Defense (DoD) response to the Government
Accountability Office (GAO) Draft Report, GAO-05-702, "DoD Business
Systems Modernization: Longstanding Weaknesses in Enterprise
Architecture Development Need to Be Addressed," dated June 7, 2005. The
Department concurs with all three of the GAO's executive
recommendations.
The Business Transformation effort of the Department of Defense, and
the activities of the Business Management Modernization Program (BMMP)
have been restructured to accelerate the transformation effort,
strengthen oversight of department wide system modernization
investments, and expand senior leadership engagement in the
transformation effort. The focus of the BMMP is to drive greater
innovation and higher levels of efficiency throughout the Business
Mission Area of the Department. This will be achieved by accelerating
the implementation of solutions that support DoD enterprise-level
capabilities to realize broader, Department-wide improvements in
business processes while continuously improving our financial
management discipline and transparency.
We look forward to on-going engagement and discussion with the GAO as
we continue our effort to transform the business operations of the
Department.
Sincerely,
Signed by:
Paul Brinkley:
Special Assistant for Business Administration:
Signed by:
Thomas Modly:
Deputy Under Secretary of Defense, Financial Management:
Enclosure: As stated:
GAO Draft Report - Dated June 7, 2005:
GAO CODE 310298/GAO-05-702:
"DoD Business Systems Modernization: Longstanding Weaknesses in
Enterprise Architecture Development Need to Be Addressed"
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
direct the Deputy Secretary of Defense, as the chair of the Defense
Business Systems Management Committee (DBSMC) and in collaboration with
the DBSMC members, to immediately fully disclose the state of its
Business Enterprise Architecture (BEA) program to DOD's congressional
authorization and appropriations committees, including its limited
progress and results to date, as well as specific plans and commitments
for strengthening program management and producing measurable results
that reflect the department's capability to do so. (p. 50/GAO Draft
Report):
DOD RESPONSE: Concur. The Department will continue to disclose plans,
progress, and results of our business transformation efforts, including
the Business Enterprise Architecture (BEA), to DOD's congressional
authorization and appropriations committees. We intend to continue our
dialog with Congress by testifying before Congressional committees,
submitting annual progress reports and dialoging on a regular basis
with Congressional members and staff.
RECOMMENDATION 2: The GAO recommended that the Secretary of Defense
direct the Deputy Secretary of Defense, as the chair of the DBSMC and
in collaboration with the DBSMC members, to ensure that each of our
recommendations related to BEA management and content are reflected in
the above plans and commitments. (p. 50/GAO Draft Report):
DOD RESPONSE: Concur. The DBSMC leadership is committed to implementing
GAO recommendations. BMMP staff continues to met with GAO staff to
bring each recommendation to closure.
RECOMMENDATION 3: The GAO recommended that the Secretary of Defense
direct the Deputy Secretary of Defense, as the chair of the DBSMC and
in collaboration with the DBSMC members, to ensure that plans and
commitments provide for effective BEA workforce planning, including
assessing workforce knowledge and skills needs, determining existing
workforce capabilities, identifying gaps, and filling these gaps. (p.
50/GAO Draft Report):
DOD RESPONSE: Concur. The Department continues to assess workforce
needs, the business transformation effort of the Department of Defense,
and the activities of the Business Management Modernization Program
(BMMP). We continue to adjust the federal and contract workforce to
meet DBSMC requirements.
[End of section]
Appendix VI: GAO Contact and Staff Acknowledgments:
GAO Contact:
Randolph C. Hite, (202) 512-3439 or [Hyperlink, hiter@gao.gov]:
Acknowledgments:
In addition to the contact named above, Cynthia Jackson, Assistant
Director; Barbara Collier; Joanne Fiorino; Neelaxi Lakhmani; Anh Le;
Freda Paintsil; Randolph Tekeley; and William Wadsworth made key
contributions to this report.
(310298):
FOOTNOTES
[1] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.:
January 2005).
[2] GAO, Information Technology: Architecture Needed to Guide
Modernization of DOD's Financial Operations, GAO-01-525 (Washington,
D.C.: May 17, 2001).
[3] Bob Stump National Defense Authorization Act for Fiscal Year 2003,
Public Law 107-314, § 1004, 116 Stat. 2458, 2629-2631 (Dec. 2, 2002).
[4] Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Public Law 108-375, § 332, 118 Stat. 1811, 1851-1856, (Oct. 28,
2004) (codified in part at 10 U.S.C. § 2222).
[5] GAO, DOD Business Systems Modernization: Improvements to Enterprise
Architecture Development and Implementation Efforts Needed, GAO-03-458
(Washington, D.C.: Feb. 28, 2003).
[6] See, for example, GAO, Defense Inventory: Opportunities Exist to
Improve Spare Parts Support Aboard Deployed Navy Ships, GAO-03-887
(Washington, D.C.: Aug. 29, 2003); Military Pay: Army National Guard
Personnel Mobilized to Active Duty Experienced Significant Pay
Problems, GAO-04-89 (Washington, D.C.: Nov. 13, 2003); and DOD Travel
Cards: Control Weaknesses Resulted in Millions of Dollars of Improper
Payments, GAO-04-576 (Washington, D.C.: June 9, 2004).
[7] GAO-05-207. The 8 specific DOD high-risk areas are (1) approach to
business transformation, (2) business systems modernization, (3)
contract management, (4) financial management, (5) personnel security
clearance program, (6) supply chain management, (7) support
infrastructure management, and (8) weapon systems acquisition. The 6
governmentwide high-risk areas are (1) disability programs, (2)
interagency contracting, (3) information systems and critical
infrastructure, (4) information sharing for homeland security, (5)
human capital, and (6) real property.
[8] GAO-05-207.
[9] GAO, Department of Defense: Status of Financial Management
Weaknesses and Progress Toward Reform, GAO-03-931T (Washington, D.C.:
June 25, 2003).
[10] DOD, Department of Defense Performance and Accountability Report,
Fiscal Year 2004 (Nov. 15, 2004).
[11] The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11312 and 11315(b)(2).
[12] The E-Government Act of 2002, Public Law 107-347 (Dec. 17, 2002).
[13] See, for example, GAO, Homeland Security: Efforts Under Way to
Develop Enterprise Architecture, but Much Work Remains, GAO-04-777
(Washington, D.C.: Aug. 6, 2004); DOD Business Systems Modernization:
Limited Progress in Development of Business Enterprise Architecture and
Oversight of Information Technology Investments, GAO-04-731R
(Washington, D.C.: May 17, 2004); Information Technology: Architecture
Needed to Guide NASA's Financial Management Modernization, GAO-04-43
(Washington, D.C.: Nov. 21, 2003); DOD Business Systems Modernization:
Important Progress Made to Develop Business Enterprise Architecture,
but Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003);
Business Systems Modernization: Summary of GAO's Assessment of the
Department of Defense's Initial Business Enterprise Architecture, GAO-
03-877R (Washington, D.C.: July 7, 2003); Information Technology: DLA
Should Strengthen Business Systems Modernization Architecture and
Investment Activities, GAO-01-631 (Washington, D.C.: June 29, 2001);
and Information Technology: INS Needs to Better Manage the Development
of Its Enterprise Architecture, GAO/AIMD-00-212 (Washington, D.C.: Aug.
1, 2000).
[14] Program officials, including the Chief Architect, stated that they
do not track and report the cost of each architecture release. These
officials stated that it would not be cost-effective to attempt to
capture this information for this review.
[15] GAO-01-525.
[16] GAO-03-458.
[17] GAO, Information Technology: Observations on Department of
Defense's Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.:
Mar. 28, 2003).
[18] Bob Stump National Defense Authorization Act for Fiscal Year 2003,
Public Law 107-314, § 1004, 116 Stat. 2458, 2629-2631 (Dec. 2, 2002).
[19] Because the review was focused on draft architecture products that
were not intended to be complete, we did not make any recommendations
in the March 2003 report. Our assessment was a specific point-in-time
review of the BEA (i.e., the Feb. 7, 2003, draft architecture).
[20] GAO-03-877R and GAO-03-1018.
[21] DOD reported that it had approximately 4,200 business systems as
of February 2005.
[22] GAO-03-1018.
[23] GAO-01-525.
[24] GAO-03-458.
[25] GAO, Department of Defense: Further Actions Needed to Establish
and Implement a Framework for Successful Financial and Business
Management Transformation, GAO-04-551T (Washington, D.C.: Mar. 23,
2004); and Department of Defense: Further Actions Needed to Establish
and Implement a Framework for Successful Business Transformation, GAO-
04-626T (Washington, D.C.: Mar. 31, 2004).
[26] GAO, Department of Defense: Long-standing Problems Continue to
Impede Financial and Business Management Transformation, GAO-04-907T
(Washington, D.C.: July 7, 2004); and Department of Defense: Financial
and Business Management Transformation Hindered by Long-standing
Problems, GAO-04-941T (Washington, D.C.: July 8, 2004).
[27] GAO-04-731R.
[28] GAO-03-1018.
[29] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-
584G (Washington, D.C.: April 2003).
[30] GAO-04-731R.
[31] GAO-01-525, GAO-03-458, and GAO-03-1018.
[32] GAO, Department of Defense: Further Actions Are Needed to
Effectively Address Business Management Problems and Overcome Key
Business Transformation Challenges, GAO-05-140T (Washington, D.C.: Nov.
18, 2004).
[33] GAO, DOD's High-Risk Areas: Successful Business Transformation
Requires Sound Strategic Planning and Sustained Leadership, GAO-05-520T
(Washington, D.C.: Apr. 13, 2005).
[34] See, for example, GAO, DOD Business Systems Modernization:
Billions Continue to Be Invested with Inadequate Management Oversight
and Accountability, GAO-04-615 (Washington, D.C.: May 27, 2004); GAO-
04-551T; and GAO-03-1018.
[35] See, for example, GAO-03-584G; and Chief Information Officer
Council, A Practical Guide to Federal Enterprise Architecture, Version
1.0 (February 2001).
[36] GAO-03-458.
[37] GAO-03-458.
[38] Ronald W. Reagan National Defense Authorization Act for Fiscal
Year 2005, Public Law 108-375, § 332, 118 Stat. 1811, 1851-1856, (Oct.
28, 2004) (codified in part at 10 U.S.C. § 2222).
[39] DOD plans to restructure the five business domains and the one
mission area into five core business mission areas: (1) Personnel
Management, (2) Weapon Systems Life-cycle Management, (3) Real Property
and Installation Life-cycle Management, (4) Material Supply and Service
Management, and (5) Financial Management.
[40] In DOD, principal staff assistants are the Under Secretaries of
Defense; the Assistant Secretaries of Defense; the General Counsel of
the Department of Defense; the Assistants to the Secretary of Defense;
and the Directors of the Office of the Secretary of Defense, or
equivalents, who report directly to the Secretary or Deputy Secretary
of Defense.
[41] The six boards are the (1) Architecture Management Board, (2)
Capabilities Transformation Board, (3) Data Management Board, (4)
Portfolio Management Board, (5) Change Management Board, and (6)
Strategic Management Board.
[42] See, for example, GAO-03-584G; and CIO Council, Federal Enterprise
Architecture, Version 1.0.
[43] GAO-03-458.
[44] See, for example, GAO-03-584G; and CIO Council, Federal Enterprise
Architecture, Version 1.0.
[45] GAO-03-1018.
[46] GAO, A Model of Strategic Human Capital Management, GAO-02-373SP
(Washington, D.C.: Mar. 15, 2002).
[47] GAO-03-584G; and CIO Council, Federal Enterprise Architecture,
Version 1.0.
[48] Separation of duties requires that key duties and responsibilities
be separated among individuals to ensure that effective checks and
balances exist to reduce the risk of error, waste, or wrongful acts or
to reduce the risk of these issues going undetected.
[49] See, for example, GAO-03-584G; CIO Council, Federal Enterprise
Architecture, Version 1.0; Electronic Industries Alliance, National
Consensus Standard for Configuration Management, ANSI/EIA-649-1998
(August 1998); Institute of Electrical and Electronics Engineers,
Standard for Application and Management of the Systems Engineering
Process 1200-1998 (Jan. 22, 1999); and the Software Engineering
Institute, Capability Maturity Model Integration, Version 1.1 (March
2002).
[50] DOD, Military Handbook 61A(SE): Configuration Management Guidance
(Feb. 7, 2001).
[51] GAO, National Guard: Effective Management Processes Needed for
Wide-Area Network, GAO-02-959 (Washington, D.C.: Sept. 24, 2002).
[52] The product types are (1) system architect encyclopedias and
architecture support products, which include data needed to produce the
operational, system, and technical views of the BEA; (2) functional
requirements, which include policies and laws that the BEA must comply
with; (3) other program deliverables, which include the quality
management plan and the data repository maintenance and support plan;
and (4) other program work products that are not required by the
performance work statement to include program processes, procedures,
and user guides.
[53] A verification and validation contractor is a neutral third-party
contractor who is responsible for conducting architecture compliance
evaluations and who provides quality assurance checking on program
information, as well as the proper implementation of the architecture
methodology.
[54] A baseline is a set of specifications or work products (e.g.,
architecture products) that has been formally reviewed or agreed upon,
that thereafter serves as the basis for further development, and that
can be changed only through change control procedures to maintain
integrity. A baseline represents the assignment of an identifier to a
configuration item and its associated entities.
[55] See, for example, GAO-03-584G; CIO Council, Federal Enterprise
Architecture, Version 1.0; DOD, Department of Defense Architecture
Framework, Version 1.0, Volume 1 (August 2003) and Volume 2 (February
2004); and Institute of Electrical and Electronics Engineers, Standard
for Recommended Practice for Architectural Description of Software-
Intensive Systems 1471-2000 (Sept. 21, 2000).
[56] GAO-03-1018.
[57] The other 30 prior recommendations pertain to the "To Be"
environment and are discussed in the next section of this report.
[58] GAO-03-1018.
[59] DOD, Department of Defense Architecture Framework, Version 1.0,
Volume 1 (August 2003) and Volume 2 (February 2004).
[60] Increment 1 objectives include (1) achieving unqualified audit
opinion for consolidated DOD financial statements and (2) achieving
total personnel visibility to include military service members,
civilian employees, military retirees, and other U.S. personnel in a
theater of operations.
[61] See, for example, GAO-03-584G; CIO Council, Federal Enterprise
Architecture, Version 1.0; Office of Management and Budget, Federal
Enterprise Architecture Business Reference Model, Version 1.0 (2002);
and Institute of Electrical and Electronics Engineers, Standard for
Recommended Practice for Architectural Description of Software-
Intensive Systems 1471-2000 (Sept. 21, 2000).
[62] DOD, Department of Defense Architecture Framework, Version 1.0,
Volume 1 (August 2003) and Volume 2 (February 2004).
[63] GAO-03-1018.
[64] The other 32 prior recommendations pertain to the "As Is"
environment and the transition plan and are discussed in the previous
section of this report.
[65] We reviewed Releases 2.2, 2.2.1, and 2.3 and the March 2005
Update.
[66] See, for example, GAO-03-584G; CIO Council, Federal Enterprise
Architecture, Version 1.0; Office of Management and Budget, Federal
Enterprise Architecture Business Reference Model, Version 1.0 (2002);
DOD, Department of Defense Architecture Framework, Version 1.0, Volume
1 (August 2003) and Volume 2 (February 2004); and Institute of
Electrical and Electronics Engineers, Standard for Recommended Practice
for Architectural Description of Software-Intensive Systems 1471-2000
(Sept. 21, 2000).
[67] IV&V Evaluation Summary Report BEA January 31, 2005, Update,
Version 2 (Feb. 1, 2005).
[68] See, for example, GAO-03-584G; and CIO Council, Federal Enterprise
Architecture, Version 1.0.
[69] GAO-03-1018.
[70] In March 2005, DOD changed the name of the program office from the
Business Modernization and Systems Integration Office to the
Transformation Support Office.
[71] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-
584G (Washington, D.C.: April 2003); and Chief Information Officer
Council, A Practical Guide to Federal Enterprise Architecture, Version
1.0 (February 2001).
[72] Department of Defense, Annual Report to Congressional Defense
Committees: Status of the Department of Defense's Business Management
Modernization Program (Mar. 15, 2005).
[73] We reviewed multiple versions of these documents during this
review. For example, we reviewed the August and November 2003 and the
April 2004 versions of the configuration management plan, as well as
the May and November 2004 versions of the configuration control
procedure document.
[74] We did not review the January 2005 Update because the department
provided this release at the same time it provided the March 2005
Update. As a result, we reviewed the content of the March release.
[75] DOD stakeholders include the program office staff, business
domains, mission area, military services, and defense agencies.
[76] J.A. Zachman, "A Framework for Information Systems Architecture,"
IBM Systems Journal 26, no. 3 (1987).
[77] DOD, Department of Defense Architecture Framework, Version 1.0,
Volume 1 (August 2003) and Volume 2 (February 2004).
[78] There are some overarching aspects of architecture that relate to
all three of the views. These overarching aspects, such as goals and
mission statements and concepts of operations, are captured in the All-
view products.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
