Organizational Transformation
Military Departments Can Improve Their Enterprise Architecture Programs Gao ID: GAO-11-902 September 26, 2011In Process
While Air Force, Army, and DON each have long-standing efforts to develop and use enterprise architectures, they have much to do before their efforts can be considered mature. GAO's enterprise architecture management framework provides a flexible benchmark against which to plan for and measure architecture program maturity and consists of 59 core elements arranged into a matrix of seven hierarchical stages. The Air Force has fully satisfied 20 percent, partially satisfied 47 percent, and not satisfied 32 percent of GAO's framework elements. The Army has fully satisfied 12 percent and partially satisfied 42 percent of the elements, with the remaining 46 percent not satisfied. Finally, DON has satisfied 27 percent, partially satisfied 41 percent, and not satisfied 32 percent of the framework elements. With respect to stages 1 through 6 of GAO's architecture framework, the military departments have generally begun establishing institutional commitments to their respective enterprise architecture efforts (stage 1), not established the management foundations necessary for effective enterprise architecture development and use (stage 2), begun developing initial enterprise architecture content (stage 3), not completed and used their initial enterprise architecture versions to achieve results (stage 4), not expanded and evolved the development and use of their respective architectures to support institutional transformation (stage 5), and taken limited steps to continuously improve their respective architecture programs and use their architectures to achieve corporate optimization (stage 6). Officials at the military departments stated that they have been limited in their ability to overcome long-standing enterprise architecture management challenges, including receiving adequate funding and attaining sufficient senior leadership understanding. Nevertheless, DOD has been provided with considerable resources for its IT systems environment, which consists of 2,324 systems. Specifically, DOD receives over $30 billion each year for this environment. Without fully developed and effectively managed enterprise architectures and a plan, the Air Force, Army, and DON lack the necessary road maps for transforming their business processes and modernizing their hundreds of supporting systems to minimize overlap and maximize interoperability. What this means is that DOD, as a whole, is not as well positioned as it should be to realize the significant benefits that a well-managed federation of architectures can afford its systems modernization efforts, such as eliminating system overlap and duplication. Because DOD is provided with over $30 billion each year for its IT systems environment, the potential for identifying and avoiding the costs associated with duplicative functionality across its IT investments is significant. GAO recommends that the military departments each develop a plan for fully satisfying the elements of GAO's framework. DOD and Army concurred and the Air Force and DON did not. In this regard, DOD stated that Air Force and DON do not have a valid business case that would justify the implementation of all the elements. However, GAO continues to believe its recommendation is warranted.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Valerie C. Melvin Team: Government Accountability Office: Information Technology Phone: (202) 512-6304GAO-11-902, Organizational Transformation: Military Departments Can Improve Their Enterprise Architecture Programs
This is the accessible text file for GAO report number GAO-11-902
entitled 'Organizational Transformation: Military Departments Can
Improve Their Enterprise Architecture Programs' which was released on
September 26, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Committee on Armed Services, U.S. Senate:
September 2011:
Organizational Transformation:
Military Departments Can Improve Their Enterprise Architecture
Programs:
GAO-11-902:
GAO Highlights:
Highlights of GAO-11-902, a report to the Committee on Armed Services,
U.S. Senate.
Why GAO Did This Study:
The Department of Defense (DOD) spends billions of dollars annually to
build and maintain information technology (IT) systems intended to
support its mission. For decades, DOD has been challenged in
modernizing its systems environment to reduce duplication and increase
integration. Such modernizations can be guided by an enterprise
architecture”a blueprint that describes an organization‘s current and
target state for its business operations and supporting IT systems and
a plan for transitioning between the two states. DOD has long sought
to employ enterprise architectures and has defined an approach for
doing so that depends in large part on the military departments
developing architectures of their own. In light of the critical role
that military department enterprise architectures play in DOD‘s
overall architecture approach, GAO was requested to assess the status
of the Departments of the Air Force, Army, and Navy (DON) enterprise
architecture programs. To do so, GAO obtained and analyzed key
information about each department‘s architecture relative to the 59
core elements contained in stages 1 through 6 of GAO‘s Enterprise
Architecture Management Maturity Framework.
What GAO Found:
While Air Force, Army, and DON each have long-standing efforts to
develop and use enterprise architectures, they have much to do before
their efforts can be considered mature. GAO‘s enterprise architecture
management framework provides a flexible benchmark against which to
plan for and measure architecture program maturity and consists of 59
core elements arranged into a matrix of seven hierarchical stages. The
Air Force has fully satisfied 20 percent, partially satisfied 47
percent, and not satisfied 32 percent of GAO‘s framework elements. The
Army has fully satisfied 12 percent and partially satisfied 42 percent
of the elements, with the remaining 46 percent not satisfied. Finally,
DON has satisfied 27 percent, partially satisfied 41 percent, and not
satisfied 32 percent of the framework elements (see table).
Table: Military Department Satisfaction of GAO‘s Framework Core
Elements:
Military department: Air Force;
Fully satisfied: 20%;
Partially satisfied: 47%;
Not satisfied: 32%.
Military department: Army;
Fully satisfied: 12%;
Partially satisfied: 42%;
Not satisfied: 46%.
Military department: DON;
Fully satisfied: 27%;
Partially satisfied: 41%;
Not satisfied: 32%.
Military department: Average;
Fully satisfied: 20%;
Partially satisfied: 44%;
Not satisfied: 37%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
With respect to stages 1 through 6 of GAO‘s architecture framework,
the military departments have generally begun establishing
institutional commitments to their respective enterprise architecture
efforts (stage 1), not established the management foundations
necessary for effective enterprise architecture development and use
(stage 2), begun developing initial enterprise architecture content
(stage 3), not completed and used their initial enterprise
architecture versions to achieve results (stage 4), not expanded and
evolved the development and use of their respective architectures to
support institutional transformation (stage 5), and taken limited
steps to continuously improve their respective architecture programs
and use their architectures to achieve corporate optimization (stage
6).
Officials at the military departments stated that they have been
limited in their ability to overcome long-standing enterprise
architecture management challenges, including receiving adequate
funding and attaining sufficient senior leadership understanding.
Nevertheless, DOD has been provided with considerable resources for
its IT systems environment, which consists of 2,324 systems.
Specifically, DOD receives over $30 billion each year for this
environment. Without fully developed and effectively managed
enterprise architectures and a plan, the Air Force, Army, and DON lack
the necessary road maps for transforming their business processes and
modernizing their hundreds of supporting systems to minimize overlap
and maximize interoperability. What this means is that DOD, as a
whole, is not as well positioned as it should be to realize the
significant benefits that a well-managed federation of architectures
can afford its systems modernization efforts, such as eliminating
system overlap and duplication. Because DOD is provided with over $30
billion each year for its IT systems environment, the potential for
identifying and avoiding the costs associated with duplicative
functionality across its IT investments is significant.
What GAO Recommends:
GAO recommends that the military departments each develop a plan for
fully satisfying the elements of GAO‘s framework. DOD and Army
concurred and the Air Force and DON did not. In this regard, DOD
stated that Air Force and DON do not have a valid business case that
would justify the implementation of all the elements. However, GAO
continues to believe its recommendation is warranted.
View [hyperlink, http://www.gao.gov/products/GAO-11-902] or key
components. For more information, contact Valerie C. Melvin at (202)
512-6304 or melvinv@gao.gov.
[End of figure]
Contents:
Letter:
Background:
Military Departments Have Begun to Develop Enterprise Architectures,
but Management and Use Can Be Improved:
Conclusions:
Recommendation for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objective, Scope, and Methodology:
Appendix II: EAMMF Table:
Appendix III: Department of the Air Force:
Appendix IV: Department of the Army:
Appendix V: Department of the Navy:
Appendix VI: Comments from the Department of Defense:
Appendix VII: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Military Department Satisfaction of Core Enterprise
Architecture Management Elements:
Table 2: Military Department Satisfaction of Stage 1 Framework
Elements:
Table 3: Military Department Satisfaction of Stage 2 Framework
Elements:
Table 4: Military Department Satisfaction of Stage 3 Framework
Elements:
Table 5: Military Department Satisfaction of Stage 4 Framework
Elements:
Table 6: Military Department Satisfaction of Stage 5 Framework
Elements:
Table 7: Military Department Satisfaction of Stage 6 Framework
Elements:
Table 8: Summary of EAMMF Version 2.0 Core Elements Categorized by
Stage:
Table 9: Air Force Satisfaction of Core Elements within Each Stage:
Table 10: Air Force Satisfaction of EAMMF Core Elements:
Table 11: Army Satisfaction of Core Elements within Each Stage:
Table 12: Army Satisfaction of EAMMF Core Elements:
Table 13: DON Satisfaction of Core Elements within Each Stage:
Table 14: DON Satisfaction of GAO EAMMF Core Elements:
Figures:
Figure 1: Simplified View of DOD Organizational Structure:
Figure 2: Conceptual Representation of DOD's Federated Architecture:
Figure 3: EAMMF Overview with Seven Stages of Maturity:
Abbreviations:
BEA: business enterprise architecture:
CIO: chief information officer:
DOD: Department of Defense:
DON: Department of the Navy:
EAMMF: Enterprise Architecture Management Maturity Framework:
IGB: Information Enterprise Governance Board:
IT: information technology:
OMB: Office of Management and Budget:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 26, 2011:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Department of Defense (DOD) spends billions of dollars each year
to maintain operational systems and processes intended to support its
mission. In this regard, the department requested about $38 billion
for its information technology (IT) investments for fiscal year 2012,
including about $25 billion in combined investments at the Departments
of the Air Force, Army, and Navy (DON).[Footnote 1] According to DOD's
systems inventory, the department's IT environment is composed of
2,324 systems and includes 338 financial management, 719 human
resource management, 664 logistics, 250 real property and
installation, and 300 weapon acquisition management systems.[Footnote
2] Of the 2,324 systems, there are 470 systems at the Air Force, 744
at the Army, and 473 at DON. For decades, DOD has been challenged in
modernizing its systems environment to reduce duplication and increase
integration between its systems.
Effective use of a well-defined enterprise architecture[Footnote 3] is
a basic tenet of successful systems modernization and associated
organizational transformation efforts, such as the one DOD has long
been seeking to accomplish. As we have previously reported,[Footnote
4] without a well-defined enterprise architecture, it is unlikely that
DOD, including its component organizations, will be able to transform
business processes and modernize supporting systems to minimize
overlap and maximize interoperability. Further, DOD's enterprise
architecture approach relies on each level of its organization (e.g.,
DOD-wide, military departments, and programs) to develop a meaningful
architecture. Accordingly, the development and use of a military
department enterprise architecture is critical for organizational
transformation and systems modernization across DOD and within each
military department.
In light of the critical role that military department architectures
play in DOD's enterprise architecture construct, you asked us to
assess the status of the Departments of the Air Force, Army, and
Navy's enterprise architecture programs. To accomplish this, we
requested key information about each department's architecture
governance, content, use, and measurement. On the basis of the
military departments' responses and supporting documentation, we
analyzed the extent to which each satisfied the 59 core elements in
our architecture maturity framework.[Footnote 5]
We conducted this performance audit at DOD and military department
offices in the Washington, D.C., metropolitan area from October 2010
through September 2011 in accordance with generally accepted
government auditing standards. Those standards require that we plan
and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on
our audit objective. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objective. Details on our objective, scope, and methodology are
provided in appendix I.
Background:
DOD is a massive and complex organization entrusted with more taxpayer
dollars than any other federal department or agency. Organizationally,
the department includes the Office of the Secretary of Defense, the
Joint Chiefs of Staff, the military departments, numerous defense
agencies and field activities, and various unified combatant commands
that are responsible for either specific geographic regions or
specific functions. (See figure 1 for a simplified depiction of DOD's
organizational structure.)
Figure 1: Simplified View of DOD Organizational Structure:
[Refer to PDF for image: organizational chart]
Top level:
* Secretary of Defense/Deputy Secretary of Defense[A].
Second level, reporting to Secretary of Defense/Deputy Secretary of
Defense:
* Department of the Army;
* Department of the Navy;
* Department of the Air Force;
* Office of the Secretary of Defense:
- DOD field activities;
- Defense agencies;
* Inspector General;
* Joint Chiefs of Staff;
* Combatant commands[B].
Source: GAO based on DOD documentation.
[A] The Deputy Secretary of Defense serves as the DOD Chief Management
Officer, who has responsibilities, under statutes and department
guidance, related to improving the efficiency and effectiveness of
business operations.
[B] The Chairman of the Joint Chiefs of Staff serves as the
spokesperson for the commanders of the combatant commands,
particularly for the administrative requirements of the commands.
[End of figure]
In support of its military operations, the department performs an
assortment of interrelated and interdependent business functions, such
as logistics management, weapons systems management, supply chain
management, procurement, health care management, and financial
management. For fiscal year 2012, the department requested about $38
billion for its IT investments, of which about $17 billion is intended
for its business systems environment and supporting IT infrastructure,
which includes systems and processes related to the management of
contracts, finances, the supply chain, support infrastructure, and
weapons systems acquisition. However, as we have previously
reported,[Footnote 6] the DOD systems environment that supports these
business functions is overly complex and error prone, and is
characterized by (1) little standardization across the department, (2)
multiple systems performing the same tasks, (3) the same data being
stored in multiple systems, and (4) the need for data to be entered
manually into multiple systems.
DOD currently bears responsibility, in whole or in part, for 14 of the
30 federal government program areas that we have designated as high
risk.[Footnote 7] Seven of these areas are specific to DOD[Footnote 8]
and the department shares responsibility for 7 other governmentwide
high-risk areas.[Footnote 9] The lack of an effective enterprise
architecture is a key contributor to its having many of these high-
risk areas. DOD's business systems modernization, which is to be
guided by the DOD Business Enterprise Architecture, is one of the high-
risk areas, and is an essential component for addressing many of the
department's other high-risk areas. For example, modernized business
systems are integral to the department's efforts to address its
financial, supply chain, and information security management high-risk
areas. A well-defined and effectively implemented enterprise
architecture is, in turn, integral to the successful modernization of
DOD's business systems.
Enterprise Architecture Is Key to Transforming Business and Mission
Operations:
An enterprise architecture is a modernization blueprint that describes
an organization's (e.g., a federal department or agency) or a
functional area's (e.g., terrorism information sharing or homeland
security) current and target state in both logical and technical
terms, as well as a plan for transitioning between the two states. As
such, it is a recognized tenet of organizational transformation and IT
management in public and private organizations. Without an enterprise
architecture, it is unlikely that an organization will be able to
transform business processes and modernize supporting systems to
minimize overlap and maximize interoperability. For more than a
decade, we have conducted work to help federal agencies improve their
architecture efforts. To this end, we developed the Enterprise
Architecture Management Maturity Framework (EAMMF), which provides
federal agencies with a common benchmarking tool for assessing the
management of their enterprise architecture efforts and developing
improvement plans.
Enterprise Architecture Description and Importance:
An enterprise can be viewed as either a single organization or a
functional area that transcends more than one organization. An
architecture can be viewed as the structure (or structural
description) of any activity. Thus, enterprise architectures are
systematically derived and captured descriptions depicted in models,
diagrams, and narratives.
More specifically, an architecture describes the enterprise in logical
terms (such as interrelated business processes and business rules,
information needs and flows, and work locations and users) as well as
in technical terms (such as hardware, software, data, communications,
security attributes, and performance standards). It provides these
perspectives both for the enterprise's current environment, and for
its target environment, and it provides a transition plan for moving
from the current to the target environment.
Enterprise architectures are a basic tenet of both organizational
transformation and IT management, and their effective use is a
recognized hallmark of successful public and private organizations.
For over a decade, we have promoted the use of architectures,
recognizing them as a crucial means to a challenging end: optimized
agency operations and performance. The alternative, as our work has
shown, is the perpetuation of the kinds of operational environments
that saddle many agencies today, in which the lack of integration
among business operations and the IT resources that support them leads
to systems that are duplicative, not well integrated, and
unnecessarily costly to maintain and interface.[Footnote 10] Employed
in concert with other important IT management controls (such as
portfolio-based capital planning and investment control practices), an
enterprise architecture can greatly increase the chances that an
organization's operational and IT environments will be configured to
optimize mission performance. Moreover, the development of agency
enterprise architectures is based on statutory requirements and
federal guidance.[Footnote 11] Further, DOD is required by statute
[Footnote 12] to develop an enterprise architecture to cover all
defense business systems, and the business transformation initiatives
of the military departments are required to develop a well-defined
enterprisewide business systems architecture.[Footnote 13]
Enterprise Architecture Approaches:
There are several approaches to structuring an enterprise
architecture, depending on the needs of the agency. In general, these
approaches provide for decomposing an enterprise into its logical
parts and architecting each of the parts in relation to enterprisewide
needs and the inherent relationships and dependencies that exist among
the parts. As such, the approaches are fundamentally aligned and
consistent with a number of basic enterprise architecture principles,
such as incremental rather than monolithic architecture development
and implementation, optimization of the whole rather than optimization
of the component parts, and maximization of shared data and services
across the component parts rather than duplication. Moreover, these
approaches are not mutually exclusive and, in fact, can all be applied
to some degree for a given enterprise, depending on the
characteristics and circumstances of that enterprise. The approaches,
which are briefly described here, are federated, segmented, and
service-oriented.
Federated:
Under a federated approach, the architecture consists of a family of
coherent but distinct member architectures that conform to an
overarching corporate (i.e., enterprise-level) or parent architecture.
This approach recognizes that each federation member has unique goals
and needs as well as common roles and responsibilities with the
members above and below it. As such, member architectures (e.g.,
component, subordinate, or subsidiary architectures) are substantially
autonomous, but they also inherit certain rules, policies, procedures,
and services from the parent architectures. A federated architecture
enables component organization autonomy while ensuring enterprise-
level or enterprisewide linkages and alignment where appropriate.
Segmented:
A segmented approach to enterprise architecture development and use,
like a federated approach, employs a "divide and conquer" methodology
in which architecture segments are identified, prioritized, developed,
and implemented. In general, segments can be viewed as logical
aspects, or "slivers," of the enterprise that can be architected and
pursued as separate initiatives under the overall enterprise-level
architecture. As such, the segments serve as a bridge between the
corporate frame of reference captured in the enterprise architecture
and individual programs within portfolios of system investments.
Service-Oriented:
Under this approach, functions and applications are defined and
designed as discrete and reusable capabilities or services that may be
under the control of different organizational entities. As such, the
capabilities or services need to be, among other things, (1) self-
contained, meaning that they do not depend on any other functions or
applications to execute a discrete unit of work; (2) published and
exposed as self-describing business capabilities that can be accessed
and used; and (3) subscribed to via well-defined and standardized
interfaces. This approach is intended to reduce redundancy and
increase integration, as well as provide the flexibility needed to
support a quicker response to changing and evolving business
requirements and emerging conditions.
DOD Has Adopted a Federated Approach to Its Enterprise Architecture:
DOD has adopted a federated strategy to develop and implement the many
and varied architectures across the department. This strategy is to
provide a comprehensive architectural description of the entire DOD
enterprise, including the relationships between and among all levels
of the enterprise (e.g., enterprise-level, mission areas, components,
and programs). Figure 2 shows a simplified conceptual depiction of
DOD's federated enterprise architecture.
Figure 2: Conceptual Representation of DOD's Federated Architecture:
[Refer to PDF for image: illustration]
Department Layer: Department of Defense.
Mission Area Layer (reporting to Department Layer):
* Warfighting Mission Area:
- Warfighting Enterprise Architecture;
* Business Mission Area:
- Business Enterprise Architecture;
* Intelligence Mission Area:
- Intelligence Enterprise Architecture;
* Enterprise Information Environment Mission Area:
- Enterprise Information Environment Enterprise Architecture.
Component Layer (reporting to Mission Area Layer):
* Military Departments;
* Defense Agencies;
* Combatant Commands.
Program Layer (reporting to Component Layer):
* Programs.
Source: GAO analysis of DOD data.
[End of figure]
DOD's Enterprise Architecture Federation Strategy describes specific
roles and responsibilities for each level of its federated
architecture. These roles and responsibilities are consistent with
DOD's tiered accountability approach to systems modernization, whereby
components (e.g., mission areas, military departments, etc.) are
responsible for defining their respective component architectures and
transition plans and program managers are responsible for developing
program-level architectures and transition plans and ensuring
integration with the architectures and transition plans developed and
executed at the component and enterprise levels. For example, each
level of the federation is responsible for developing its respective
architecture and imposing constraints on the levels below.
Accordingly, the completeness of the DOD federated enterprise
architecture depends on each level of the federation developing its
own respective enterprise architecture. Moreover, since the military
departments comprise such a large portion of the DOD enterprise, the
relative importance of their respective enterprise architectures is
significant.
GAO's Enterprise Architecture Management Maturity Framework:
In 2002, we developed version 1.0[Footnote 14] of the EAMMF to provide
federal agencies with a common benchmarking tool for planning and
measuring their efforts to improve management of their enterprise
architectures, as well as to provide OMB with a means for doing the
same governmentwide. We issued an update to the framework (version
1.1) in 2003[Footnote 15] and a new version (version 2.0) in
2010.[Footnote 16] Version 2.0 expands on prior versions based on our
experience in using them in evaluating governmentwide and agency-
specific enterprise architectures and our solicitation of comments
from federal agencies and other stakeholders on the usability,
completeness, and sufficiency of the framework. The latest version
provides a more current and pragmatic construct for viewing
architecture development and use. In this regard, it provides a
flexible benchmark against which to plan for and measure architecture
program management maturity that permits thoughtful and reasonable
discretion to be applied in using it. Restated, the framework is not
intended to be a rigidly applied "one size fits all" checklist, but
rather a flexible frame of reference that should be applied in a
manner that makes sense for each organization's unique facts and
circumstances. Specifically, depending on the size, scope, and
complexity of the enterprise, not every framework core element may be
equally applicable, not every assessment has to consider every
element, and not every assessment has to consider every element in the
same level of detail. Moreover, the framework is not intended to be
viewed as the sole benchmarking tool for informing and understanding
an organization's journey toward architecture maturity.
Version 2.0 of the framework arranges 59 core elements into a matrix
of seven hierarchical stages. Figure 3 presents a depiction of the
seven stages of maturity.
Figure 3: EAMMF Overview with Seven Stages of Maturity:
[Refer to PDF for image: illustration]
Maturity Stage 0:
No elements.
Maturity Stage 1:
Core elements.
Maturity Stage 2:
Core elements.
Maturity Stage 3:
Core elements.
Maturity Stage 4:
Core elements.
Maturity Stage 5:
Core elements.
Maturity Stage 6:
Core elements.
Source: GAO.
[End of figure]
EAMMF Stages:
Each maturity stage includes all the core elements that are resident
in the previous stages. Each stage is described in detail here.
Appendix II provides a list of core elements arranged by their
respective EAMMF stages.
Stage 0: Creating Enterprise Architecture Awareness:
At this stage, either an organization does not have plans to develop
and use an enterprise architecture or it has plans that do not
demonstrate an awareness of the management discipline needed to
successfully develop, maintain, and use an enterprise architecture.
While Stage 0 organizations may have initiated some enterprise
architecture activity, their efforts are largely ad hoc and
unstructured and lack the institutional leadership necessary for
successful development, maintenance, and use as defined in Stage 1.
Therefore, Stage 0 has no associated core elements.
Stage 1: Establishing Enterprise Architecture Institutional Commitment
and Direction:
At this stage, an organization puts in place the foundational pillars
for treating its enterprise architecture program as an institution and
for overcoming traditional barriers to its success. For example, the
organization grounds enterprise architecture development and
compliance in policy and recognizes it as a corporate asset by vesting
ownership of the architecture with top executives (i.e., lines of
business owners and chief "X" officers).[Footnote 17] As members of a
chartered architecture executive committee, these individuals are
provided with the knowledge and understanding of the architecture
concepts and governance principles needed to lead and direct the
enterprise architecture effort. Through the enterprise architecture
executive committee (hereafter referred to as the Executive
Committee), leadership is demonstrated through the approval of
enterprise architecture goals and objectives and key aspects of the
architecture's construct, such as the framework(s) to be used and the
approach for establishing the hierarchy and structure of organization
components (e.g., federation members, segments, etc.). Also during
this stage, the central figure in managing the program, the Chief
Architect, is appointed and empowered, and the integral and relative
role of the enterprise architecture vis-à-vis other enterprise-level
governance disciplines is recognized in enterprise-level policy.
Organizations that achieve this maturity stage have demonstrated
enterprise architecture leadership through an institutional commitment
to developing and using the enterprise architecture as a strategic
basis for directing its development, maintenance, and use.
Stage 2: Creating the Management Foundation for Enterprise
Architecture Development and Use:
This stage builds on the strategic leadership foundation established
in Stage 1 by creating the managerial means to the ends--an initial
version of the enterprise architecture (Stages 3 and 4) and an
evolving and continuously improving enterprise architecture (Stages 5
and 6) that can be used to help guide and direct investments and
achieve the architecture's stated purpose. For example, at this stage
the organization establishes operational enterprise architecture
program offices, including an enterprise-level program office that is
headed by the Chief Architect, who reports to the Executive Committee.
Also at this stage, the Executive Committee continues to exercise
leadership by ensuring that the Chief Architect and subordinate
architects have the funding and human capital needed to "stand up"
their respective program offices and have acquired the requisite
architecture tools (development and maintenance methodologies,
modeling tools, and repository). Organizations that achieve this stage
have largely established the program management capacity needed to
develop an initial version of the enterprise architecture.
Stage 3: Developing Initial Enterprise Architecture Versions:
At this stage, an organization is focused on strengthening the ability
of its program office(s) to develop an initial version of the
enterprise architecture while also actually developing one or more of
these versions. Among other things, steps are taken to engage
stakeholders in the process and implement human capital plans, to
include hiring and training staff and acquiring contractor expertise.
During this stage, these resources are combined with earlier acquired
tools (e.g., framework(s), methodologies, modeling tools, and
repositories) to execute enterprise architecture management plans and
schedules aimed at delivering an initial enterprise-level version of
the architecture that includes current and target views of the
performance, business, data, services, technology, and security
architectures, as well as an initial version of a plan for
transitioning from the current to the target views. Also during this
stage, one or more segment architectures or federation member
architectures are being developed using available tools and defined
plans and schedules, and progress in developing initial architecture
versions is measured by the Chief Architect and reported to the
Executive Committee. Although an organization at this maturity stage
does not yet have a version of an enterprise architecture that is
ready for implementation, it is well on its way to defining an
enterprise architecture of sufficient scope and content that can be
used to guide and constrain investments in a way that can produce
targeted results.
Stage 4: Completing and Using an Initial Enterprise Architecture
Version for Targeted Results:
At this stage, an organization has developed a version of its
enterprise-level architecture that has been approved by the Executive
Committee, to include current and target views of the performance,
business, data, services, technology, and security architectures, as
well as an initial version of a plan for transitioning from the
current to the target views. In addition, one or more segment and/or
federation member architectures have been developed and approved
according to established priorities. Moreover, the approved enterprise-
level and subordinate architectures are being used to guide and
constrain capital investment selection and control decisions and
system life-cycle definition and design decisions. Also during this
stage, a range of factors are measured and reported to the Executive
Committee, such as enterprise architecture product quality, investment
compliance, subordinate architecture alignment, and results and
outcomes. Organizations that achieve this stage of maturity have a
foundational set of enterprise-level and subordinate enterprise
architecture products that provide a meaningful basis for informing
selected investments and building greater enterprise architecture
scope, content, use, and results.
Stage 5: Expanding and Evolving the Enterprise Architecture and Its
Use for Institutional Transformation:
At this stage, the enterprise architecture's scope is extended to the
entire organization, and it is supported by a full complement of
segment and federation member architectures, all of which include
current and target views of the performance, business, data, services,
technology, and security architectures, as well as well-defined plans
for transitioning from the current to the target views. Moreover, this
suite of architecture products is governed by a common enterprise
architecture framework, methodology, and repository, thus permitting
the products to be appropriately integrated. Also at this stage, the
architecture products are continuously maintained, and major updates
of the enterprise-level architecture are approved by the head of the
organization, while subordinate architecture product updates are
approved by their corresponding organization heads or segment owners.
In addition, architecture product quality (i.e., completeness,
consistency, usability, and utility) as well as enterprise
architecture management process integrity are assessed by an
independent agent, and the results are reported to the Chief Architect
and the Executive Committee. An organization that achieves this level
of maturity has established a full suite of architecture products that
can be employed as a featured decision support tool when considering
and planning large-scale organizational restructuring or
transformation initiatives.
Stage 6: Continuously Improving the Enterprise Architecture and Its
Use to Achieve Corporate Optimization:
At this stage, an organization is focused on continuously improving
the quality of its suite of enterprise architecture products and the
people, processes, and tools used to govern their development,
maintenance, and use. By achieving this stage of maturity, the
organization has established a truly enterprisewide blueprint to
inform both "board room" strategic planning and decision making and
"on-the-ground" implementation of these changes through a range of
capital investment and maintenance projects and other enterprise-level
initiatives.
Prior Reviews of Federal Department and Agency Enterprise
Architectures:
In 2002 and 2003, we reported on the status of enterprise
architectures governmentwide, including for the Departments of the Air
Force, Army, and Navy.[Footnote 18] We found that some federal
agencies had begun to establish the management foundation needed to
successfully develop, implement, and maintain an enterprise
architecture, but that executive leadership was key to addressing
management challenges identified by enterprise architecture programs:
(1) overcoming limited executive understanding, (2) inadequate
funding, (3) insufficient number of skilled staff, and (4)
organizational parochialism. Accordingly, we made recommendations to
OMB to improve enterprise architecture leadership and
oversight.[Footnote 19] OMB responded to these recommendations by
establishing its Chief Architects Forum to, among other things, share
enterprise architecture best practices among federal agencies and by
developing an enterprise architecture assessment tool, which it used
to periodically evaluate enterprise architecture programs at federal
agencies.
In 2006, we reviewed[Footnote 20] enterprise architecture management
at 27 federal agencies and found that management improvements were
needed. Overall, most agencies had not reached a sufficient level of
maturity in their enterprise architecture development, particularly
with regard to their approaches to assessing each investment's
alignment with the enterprise architecture and measuring and reporting
on enterprise architecture results and outcomes. In addition, the
military departments comprised three of the four agencies with the
lowest overall satisfaction of key enterprise architecture management
practices.
Our 2006 report also identified that challenges facing agencies across
the federal government in developing and using enterprise
architectures are formidable. Specifically, 93 percent of federal
departments and agencies reported that they had encountered
organizational parochialism and cultural resistance to enterprise
architecture to a significant (very great or great) or moderate
extent. Other challenges reported were ensuring that the architecture
program had adequate funding (89 percent), obtaining staff skilled in
the architecture discipline (86 percent), and having department or
agency senior leaders that understand the importance and role of the
enterprise architecture (82 percent). We identified leadership as a
key to overcoming these management challenges and made specific
recommendations to individual agencies to address their challenges and
manage their programs. Since 2006, we have continued to report that
sustained top management leadership is the key to overcoming these
challenges and positioning agencies to achieve enterprise architecture-
related benefits such as improved alignment between their business
operations and the IT that supports these operations and consolidation
of their IT infrastructure environments.
Between 2005 and 2008, we reported that DOD had taken steps to comply
with key requirements of the National Defense Authorization Act for
Fiscal Year 2005[Footnote 21] relative to architecture development;
[Footnote 22] however, each report also concluded that much remained
to be accomplished relative to the act's requirements and relevant
guidance.[Footnote 23] We further reported in May 2008[Footnote 24]
that the military departments' enterprise architecture programs had
yet to advance to a level that could be considered fully mature.
Specifically, we reported that all three departments were at the
initial stage of maturity as defined in version 1.1 of GAO's
architecture maturity framework and had yet to fulfill the framework's
requirements for, among other things, establishing a management
foundation for developing, maintaining, and using the architecture. We
reported that DOD, as a whole, was not as well positioned as it should
be to realize the significant benefits that a well-managed federation
of architectures can afford its business systems modernization efforts.
More recently, we reported[Footnote 25] on the need for federal
agencies to measure and report enterprise architecture results and
outcomes as key mechanisms for identifying overlap and duplication.
Specifically, we stated that while some progress has been made in
improving management, more time is needed for agencies to fully
realize the value of having well-defined and implemented
architectures. Such value can be derived from realizing cost savings
through consolidation and reuse of shared services and elimination of
antiquated and redundant mission operations, enhancing information
sharing through data standardization and system integration, and
optimizing service delivery through streamlining and normalization of
business processes and mission operations.
Military Departments Have Begun to Develop Enterprise Architectures,
but Management and Use Can Be Improved:
The Air Force, Army, and DON each have long-standing efforts in place
to develop and use an enterprise architecture, but much remains to be
accomplished before these efforts can be considered sufficiently
mature to fully support ongoing organizational transformation and
corporate optimization efforts. Specifically, the Air Force has fully
satisfied 20 percent, partially satisfied 47 percent, and not
satisfied 32 percent of GAO's enterprise architecture framework
elements.[Footnote 26] The Army has fully satisfied 12 percent and
partially satisfied 42 percent of elements, with the remaining 46
percent not satisfied. Finally, DON has satisfied 27 percent,
partially satisfied 41 percent, and not satisfied 32 percent of
framework elements. (Table 1 summarizes each military department's
satisfaction of core enterprise architecture management elements and
detailed results are presented in appendices III, IV, and V.)
Table 1: Military Department Satisfaction of Core Enterprise
Architecture Management Elements:
Military department: Air Force;
Fully satisfied: 20%;
Partially satisfied: 47%;
Not satisfied: 32%.
Military department: Army;
Fully satisfied: 12%;
Partially satisfied: 42%;
Not satisfied: 46%.
Military department: DON;
Fully satisfied: 27%;
Partially satisfied: 41%;
Not satisfied: 32%.
Military department: Average;
Fully satisfied: 20%;
Partially satisfied: 44%;
Not satisfied: 37%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
More specifically, while the military departments have each
demonstrated that they are beginning to establish an institutional
commitment to their respective enterprise architecture by addressing
many of the elements described in Stage 1 of GAO's enterprise
architecture management framework and to develop initial enterprise
architecture content (Stage 3), they have generally not established a
well-developed enterprise architecture management foundation (Stage
2). Moreover, the departments have yet to complete and use their
initial enterprise architecture versions to achieve targeted results
(Stage 4) or expand and evolve their respective architectures to
support institutional transformation (Stage 5). Finally, the
departments have taken limited steps to continuously improve their
respective architecture programs and use their architectures to
achieve enterprisewide optimization (Stage 6). Officials at the
military departments stated that they continue to face long-standing
enterprise architecture management challenges, such as receiving
adequate funding, overcoming cultural resistance and attaining
sufficient senior leadership understanding. Nevertheless, DOD has been
provided with considerable resources for its IT systems environment.
Specifically, in recent years, DOD has been provided with over $30
billion annually for this environment. In addition, for fiscal year
2012, DOD has requested about $38 billion for its IT investments.
Without fully developed and effectively managed enterprise
architectures, the Air Force, Army, and DON do not have a sufficient
architectural basis for transforming their business processes and
modernizing their thousands of supporting systems to minimize overlap
and maximize interoperability. Consequently, DOD as a whole is not
well positioned to realize the significant benefits that a well-
managed set of architectures can contribute to its ongoing operational
and IT system modernization efforts, such as eliminating system
overlap and duplication. Because DOD is provided with over $30 billion
each year for its IT systems environment, the potential for
identifying and avoiding the costs associated with duplicative
functionality across its IT investments is significant.
Military Departments Have Each Begun to Establish an Institutional
Commitment to an Enterprise Architecture:
Stage 1 of GAO's EAMMF describes elements associated with establishing
the foundational pillars for treating the enterprise architecture as
an institution and for overcoming barriers to success. Examples of
these elements include establishing an enterprise architecture policy
and an executive committee and defining the roles and responsibilities
of key players and associated metrics to help ensure that their
respective roles and responsibilities are fulfilled.
The military departments have demonstrated that they are beginning to
establish an institutional commitment to their respective enterprise
architecture by fully satisfying 42 percent, partially satisfying 42
percent, and not satisfying 17 percent of the Stage 1 elements. Table
2 describes the extent to which each military department has satisfied
the Stage 1 elements.
Table 2: Military Department Satisfaction of Stage 1 Framework
Elements:
Military department: Air Force;
Fully satisfied: 50%;
Partially satisfied: 38%;
Not satisfied: 13%.
Military department: Army;
Fully satisfied: 25%;
Partially satisfied: 50%;
Not satisfied: 25%.
Military department: DON;
Fully satisfied: 50%;
Partially satisfied: 38%;
Not satisfied: 13%.
Military department: Average;
Fully satisfied: 42%;
Partially satisfied: 42%;
Not satisfied: 17%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 1 elements.
Policy: All three of the military departments have fully satisfied the
element associated with establishing a policy for enterprise
architecture development, maintenance, and use. Establishing such a
policy is important for, among other things, confirming an
organization's institutional commitment to an enterprise architecture
and defining the entities responsible for architecture development,
maintenance, and use.
Executive committee: Air Force and DON have fully satisfied, and Army
has partially satisfied the element that is associated with
establishing an executive committee representing the enterprise and
that is responsible and accountable for the architecture. Although the
Army has established committees responsible for some segment
architecture activities, it has not yet established an executive
committee responsible for an enterprise-level architecture.
Establishing enterprisewide responsibility and accountability is
important for demonstrating the organization's institutional
commitment to enterprise architecture and for obtaining buy-in from
across the organization. Such an executive committee also helps the
enterprise architecture effort address issues that might not be
entirely within the span of control of the organizational Chief
Architect, such as obtaining adequate funding and sufficient human
capital resources.
Performance and accountability framework: None of the military
departments has fully or partially satisfied the element associated
with establishing an enterprise architecture performance and
accountability framework that recognizes the critical roles and
responsibilities of key stakeholders and provides the metrics and
means for ensuring that roles and responsibilities are fulfilled.
Specifically, none of the military departments has defined the metrics
and means for ensuring that roles and responsibilities are fulfilled.
Successfully managing any program, including an enterprise
architecture program, depends in part on establishing clear
commitments and putting in place the means by which to determine
progress against these commitments and hold responsible parties
accountable for the results.
To their credit, each of the military departments has taken important
steps to address the Stage 1 core elements, and, as a result, has
begun to establish key institutional commitments to developing and
using an enterprise architecture. Establishing such institutional
commitments is the first step to overcoming long-standing barriers to
enterprise architecture success, such as top leadership understanding
and parochialism and cultural resistance. Without such commitments,
these barriers may continue to limit the ability of an architecture
program to contribute to efforts to improve the organization, which
may range from streamlining business processes and IT that supports a
specific organizational line of business (e.g., a segment) to larger
and more significant organizationwide improvement efforts.
Military Departments Lack Well-Developed Management Foundations for
Enterprise Architecture Development and Use:
Stage 2 of GAO's EAMMF describes elements that build on the strategic
leadership commitment established in Stage 1 by creating the
managerial means to accomplish activities in later stages, including
developing an initial version of the enterprise architecture (Stages 3
and 4) and evolving and continuously improving an enterprise
architecture (Stages 5 and 6) that can be used to help guide and
direct investments and achieve the architecture's stated purpose.
Examples of these elements include selecting automated tools,
establishing an enterprise architecture program management office,
developing a program management plan, justifying and funding program
resources, and defining human capital plans.
The military departments' satisfaction of these Stage 2 core elements
demonstrates that much remains to be accomplished to establish their
respective enterprise architecture management foundations.
Specifically, the military departments have collectively satisfied
only 17 percent, partially satisfied 47 percent, and not satisfied 37
percent of these elements. Table 3 describes the extent to which each
military department has satisfied Stage 2 framework elements.
Table 3: Military Department Satisfaction of Stage 2 Framework
Elements:
Military department: Air Force;
Fully satisfied: 10%;
Partially satisfied: 50%;
Not satisfied: 40%.
Military department: Army;
Fully satisfied: 20%;
Partially satisfied: 40%;
Not satisfied: 40%.
Military department: DON;
Fully satisfied: 20%;
Partially satisfied: 50%;
Not satisfied: 30%.
Military department: Average;
Fully satisfied: 17%;
Partially satisfied: 47%;
Not satisfied: 37%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 2 elements.
Automated tools: The Air Force, Army, and DON have each fully
satisfied the element associated with selecting automated enterprise
architecture tools. Automated tools support the creation of a holistic
view of the current and target state of the enterprise by assisting in
the process of extracting, assimilating, relating, and presenting
critical organizational information (e.g., the relationships between
business operations and associated performance metrics, information
exchanges, supporting applications and services, technology standards,
and security protocols).
Program office: All three of the military departments have partially
satisfied the element associated with establishing an enterprise
architecture program office. Although both Air Force and DON have
established a small team dedicated to enterprise architecture
development and use, they do not operate within a formally chartered
program office. In addition, while Army has established chartered
program management offices for its three primary segments, it has not
established an enterprise-level architecture program management
office. Air Force officials stated that the department does not manage
its enterprise architecture as a formal program, DON officials stated
that the department is unable to justify creating a large program
office in a fiscally constrained environment, and Army officials
recognized the need to establish an enterprise-level architecture
program management office in the future. We and the federal Chief
Information Officers Council have previously reported that enterprise
architecture development and maintenance should be managed as a formal
program.[Footnote 27] Doing so helps ensure that the enterprise
architecture program receives the appropriate attention and sufficient
funding and human capital resources needed to be successful. In
addition, establishing such an office would provide accountability for
achieving its desired results. Accordingly, the program office should
be responsible to the Executive Committee for ensuring that critical
activities that are within its span of authority and control, such as
enterprise architecture program planning and performance monitoring,
enterprise architecture development and maintenance using supporting
tools, and enterprise architecture configuration management, are
performed.
Program management plan: None of the three military departments has
fully satisfied the element associated with managing its enterprise
architecture activities according to an enterprise architecture
program management plan. To their credit, Army and DON have partially
satisfied the element. For example, DON has established a governance
plan that defines enterprise architecture management structures and
stakeholder roles and responsibilities and Army has established a
program management plan for its Network segment,[Footnote 28] which
addresses at least some enterprisewide requirements. However, Army and
DON have yet to develop comprehensive plans for managing their
architecture programs. In addition, although Air Force officials
stated that the department produces an annual roadmap from which goals
and schedules are developed, Air Force officials did not provide
evidence to demonstrate that a program management plan exists that
includes information such as management controls and accountability
mechanisms. DON officials stated that they are in the process of
developing a road map that will serve as a program management plan and
Army officials stated that the department plans to establish a program
management office that addresses key program management activities. An
enterprise architecture program management plan would provide the
range of management structures, controls, disciplines, roles, and
accountability mechanisms discussed throughout the framework as well
as descriptions of the major enterprise architecture releases or
increments to be developed. In this regard, the plan is a critical
tool for providing a bridge between more conceptual frameworks and
methodologies to the detailed and actionable work breakdown structures
and schedules. As such, it is important for the departments to develop
such a plan to ensure that the enterprise architecture program is
effectively managed.
Budgetary needs: Each of the military departments has partially
satisfied the element associated with justifying and funding
enterprise architecture budgetary needs. All of the military
departments agree that sufficient resources to establish and execute
their respective enterprise architecture programs are not available
but that the level of current funding has enabled them to continue
executing certain architectural activities. Nevertheless, DOD has been
provided with considerable resources for its IT systems environment.
Specifically, in recent years, DOD has been provided with over $30
billion annually for this environment. In addition, for fiscal year
2012, DOD has requested about $38 billion for its IT investments.
Moreover, architecture budgetary needs have not been identified and
justified through reliable cost estimating and expected program
benefits. By funding enterprise architecture as a capital investment,
an organization's leadership demonstrates its long-term commitment to
having and using an enterprise architecture to inform investment
decision making and optimize mission-facing and mission-supporting
operations. Such funding requests also establish expected enterprise
architecture program benefits that, in turn, provide justification for
department and agency enterprise architecture expenditures and
establish commitments against which enterprise architecture program
managers and department executives can be held accountable.
Human capital plans: None of the three departments has fully or
partially satisfied the element associated with developing human
capital plans that identify the knowledge, skills, and abilities
needed for enterprise architecture staff. Army officials recognized
the need for such a plan and stated that the department's updated
enterprise architecture regulation will include a requirement for such
a human capital plan. DON officials stated that a human capital plan
is not necessary due to their small number of staff and lack of a
large centralized program office. Air Force officials stated that the
department does not plan to develop such a plan because enterprise
architecture is not a formally established career field within the
federal government. However, having sufficient human capital to
successfully develop and maintain the enterprise architecture begins
with identifying human capital needs and developing a plan for
acquiring, developing, and retaining qualified staff with the
requisite knowledge, skills, and abilities. The enterprise
architecture human capital plan is the vehicle for addressing
enterprise architecture program skill gaps by, for example, training
existing staff, hiring new staff, using contractor staff, and
addressing staff retention, development, and recognition and reward.
Agencies that achieve this stage have largely established the program
management capability needed to develop an initial architecture
version. By not satisfying the majority of the elements at this stage,
which as stated creates the managerial means to the ends, the military
departments risk not being able to effectively execute higher stage
core elements. For example, an agency can begin developing initial
architecture products that describe its current and target
environment, and a plan for transitioning from its current to its
target environment (Stage 3); however, without establishing enterprise
architecture management plans to guide its enterprise architecture
efforts (Stage 2), it risks not delivering an architecture that can be
used for achieving target results and institutional transformation
(Stages 4 and 5).
Military Departments Have Begun to Develop Initial Enterprise
Architecture Content:
Stage 3 of our EAMMF describes elements associated with strengthening
the ability of a program office to develop initial versions of the
enterprise architecture by leveraging acquired resources and tools
(established in Stages 1 and 2) to execute enterprise architecture
management plans and schedules. Examples of these elements include
developing subordinate architectures, developing initial enterprise-
level architecture versions, and using the selected enterprise
architecture methodology.
The military departments' satisfaction of these Stage 3 core elements
demonstrates that they have begun to develop initial enterprise
architecture content. Specifically, the military departments have
satisfied 26 percent, partially satisfied 45 percent, and not
satisfied 29 percent of these elements. Table 4 describes the extent
to which each department has satisfied Stage 3 framework elements.
Table 4: Military Department Satisfaction of Stage 3 Framework
Elements:
Military department: Air Force;
Fully satisfied: 29%;
Partially satisfied: 50%;
Not satisfied: 21%.
Military department: Army;
Fully satisfied: 14%;
Partially satisfied: 50%;
Not satisfied: 37%.
Military department: DON;
Fully satisfied: 36%;
Partially satisfied: 36%;
Not satisfied: 29%.
Military department: Average;
Fully satisfied: 26%;
Partially satisfied: 45%;
Not satisfied: 29%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 3 elements.
Subordinate architectures: All of the military departments have
satisfied the element associated with developing one or more segment
and/or federation architecture. For example, the Air Force approved
version 1.0 of its Space Domain architecture in September 2010; the
Army is developing architectures for its Generating Force and Network
segments; and DON has developed architecture artifacts for its Net
Centric segment.[Footnote 29] As we have previously reported,[Footnote
30] successful enterprise architecture development for large, complex
federal agencies does not involve an "all-or-nothing" monolithic
approach. Rather, enterprise architecture development typically
follows a "divide and conquer" strategy in which the level of
architectural detail needed to guide and constrain individual
investments is created for distinct organizational components or
functional slices of the enterprise. In taking such an approach, the
level of architectural content that needs to be defined to
sufficiently inform high-priority, near-term system investments can be
established relatively quickly, thus allowing the benefits of the
enterprise architecture to be realized sooner rather than later.
Developing initial enterprise-level architecture content: All three of
the military departments have partially satisfied the element
associated with developing an initial enterprise-level architecture
that addresses the current and target environment and a sequencing
plan for transitioning from the current to the target environment. The
Air Force and DON approaches to developing this enterprise
architecture content involve establishing enterprise-level
architecture content that is to be further supported by lower-level
architectures, while Army has established three architecture segments
but has not yet established an enterprise-level architecture.
Consistent with these approaches, the Air Force and DON have begun to
develop initial enterprise-level architecture content, but this
content does not include separate views of the current and target
environments and neither of the two military departments has fully
established a sequencing plan that describes how the department is to
transition from its current to the target environment and is based on
an analysis of the gaps between these environments. Officials from
these departments stated that they do not plan to distinguish between
their current and target environments because they are focused on
establishing enterprise architecture content that is more immediately
usable (DON) and they have not been asked by executive management to
establish such a distinction (Air Force).
As we have previously reported, enterprise architecture development
typically occurs in an incremental fashion, whereby an initial version
is developed as the foundation on which to evolve and build
increasingly more comprehensive, detailed, and complete versions.
[Footnote 31] In addition, as we have reported,[Footnote 32]
sequencing plans should be based on an assessment of the differences
between the current and target architectures (i.e., a gap analysis).
For example, a performance gap analysis identifies performance
measures (e.g., effectiveness) of a business process, highlights which
performance measures are not being met in the current environment, and
describes performance expectations for these measures in the target
environment, thereby describing expected performance improvements to
the business process. This performance gap analysis should also
identify the business process activities or steps that need to be
changed to achieve the future performance expectations. As such, these
gap analyses identify necessary changes or adjustments in the current
environment to achieve business goals and mission outcomes expected in
the future environment, thereby serving as the support for related
investments and activities, and also as a basis for prioritization;
integration; and synchronization of decisions across the spectrum of
these investments and activities.
Using an enterprise architecture methodology: None of the military
departments has fully satisfied the element associated with developing
its enterprise architecture products according to a defined
methodology. The departments have not satisfied this element because,
among other things, none of them has established a methodology to
guide enterprise architecture development, maintenance, and use. Air
Force officials stated that efforts to develop such a methodology have
been postponed due to budget constraints and DON officials stated that
the department has not developed a methodology due to its focus on
other resource-intensive commitments, such as applying the current
enterprise architecture content. However, as stated previously in this
report, DOD has been provided with extensive resources for its IT
systems environment. Army officials stated that the department's draft
enterprise architecture regulation will call for the development of
such a methodology. It is important for the departments to develop
such a methodology, as it would provide architecture staff and
stakeholders with a shared understanding of the architecture
development approach, including defined steps, tasks, standards,
tools, techniques, and measures that are to be used to create the
specified enterprise architecture products. In addition, such a
methodology would help to ensure that enterprise architecture products
are, among other things, consistent, complete, aligned, integrated,
and usable.
An agency that achieves this stage is well on its way to defining an
enterprise architecture of sufficient scope and content that can be
used to guide and constrain investments in a way that can produce
targeted results, even though it may not yet have developed a version
of an enterprise architecture that is ready for implementation.
However, agencies that develop architectural content in Stage 3
without first addressing critical Stage 1 and 2 elements risk
developing enterprise architecture products that are not usable for
achieving target results (Stage 4).
Military Departments Have Yet to Complete and Use Initial Enterprise
Architecture Versions for Targeted Results:
Stage 4 of our EAMMF describes elements associated with completing
initial enterprise architecture versions and using the architecture to
achieve targeted results. Examples of these elements include linking
architecture to other management disciplines, measuring and reporting
the quality of enterprise architecture products, and measuring and
reporting enterprise architecture results and outcomes.
The military departments' satisfaction of these Stage 4 core elements
demonstrates that, among other things, they have yet to establish a
meaningful basis for guiding and constraining capital investment
selection and control decisions and system life cycle definition and
design decisions. Specifically, they have satisfied only 15 percent,
partially satisfied 39 percent, and not satisfied 45 percent of these
elements. Table 5 describes the extent to which each military
department has satisfied Stage 4 framework elements.
Table 5: Military Department Satisfaction of Stage 4 Framework
Elements:
Military department: Air Force;
Fully satisfied: 9%;
Partially satisfied: 55%;
Not satisfied: 36%.
Military department: Army;
Fully satisfied: 9%;
Partially satisfied: 27%;
Not satisfied: 64%.
Military department: DON;
Fully satisfied: 27%;
Partially satisfied: 36%;
Not satisfied: 36%.
Military department: Average;
Fully satisfied: 15%;
Partially satisfied: 39%;
Not satisfied: 45%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 4 elements.
Linking architecture to other management disciplines: All three
military departments have fully satisfied the element associated with
making their respective architecture programs integral to the
execution of other institutional management disciplines. Specifically,
the Army has demonstrated that its segment architecture efforts are
linked to the execution of management disciplines associated with
capability integration, acquisition, and budgeting; the Air Force
demonstrated integration with its capability integration, acquisition,
and budgeting disciplines; and DON demonstrated that its architecture
efforts are linked to the execution of its strategic planning, capital
planning, and system development efforts. Enterprise architecture is
one of several interrelated institutional management disciplines that
collectively provide the means for an organization to be successful in
meeting its mission goals and target outcomes. It is a contributor to
many of these disciplines. In particular, it provides the bridge
between strategic planning and program implementation, it informs
human capital strategic planning and capital planning and investment
control decision making, and it provides a critical underpinning to
institutional performance management. As a result, the enterprise
architecture should be an integral input into the execution of each of
these management disciplines.
Enterprise architecture quality measurement: DON has fully satisfied
the element associated with measuring and reporting the quality of its
enterprise architecture products; Air Force has partially satisfied
this element; and Army has yet to satisfy the element. Specifically,
the quality of DON enterprise architecture products is assessed by a
working group in accordance with a set of defined criteria and
submitted for final approval to an enterprise architecture approval
board. Air Force demonstrated that both its enterprise-level and
subordinate architectures are subject to quality reviews that address
completeness, usability, consistency, and accuracy and are reported to
the appropriate officials. However, the assessments were not based on
quality standards defined in an approved enterprise architecture
methodology. In addition, although Army's Network architecture segment
documentation states that quality control measures are to be used to
determine quality, reuse, compliance, and risk, related measurements
have not yet been defined. Further, according to Army officials, the
quality of its other segment architecture products is not measured and
reported. Realizing an enterprise architecture's value depends in
large part on the quality of the products or artifacts that compose
it. Accordingly, measuring and reporting the quality of enterprise
architecture products relative to defined and consistently applied
quality standards helps ensure that the enterprise architecture
program will ultimately achieve its intended purpose.
Enterprise architecture results and outcomes: None of the military
departments has partially or fully satisfied the element associated
with measuring and reporting enterprise architecture results and
outcomes, although all three have reported that they expect to realize
future benefits from their respective architecture programs. For
example, Air Force reported that it expects to achieve improved
alignment between its business operations and IT as well as improved
data integration within 2 to 5 years, and DON reported that it expects
increased infrastructure consolidation and increased use of enterprise
licenses within 2 to 5 years. What this suggests is that the real
value to the military departments from developing and using enterprise
architectures has yet to be realized. Our framework recognizes that a
key to realizing this potential is effectively managing department and
agency enterprise architecture programs. However, knowing whether
benefits and results are in fact being achieved requires having
associated measures and metrics. In this regard, it is important for
the military departments to measure and report enterprise architecture
results and outcomes. Examples of results and outcomes to be measured
include costs avoided by eliminating duplicative investments or by
reusing common services and applications and improved mission
performance through re-engineered business processes and modernized
supporting systems.
Agencies that achieve maturity Stage 4 have a foundational set of
enterprise-level and subordinate enterprise architecture products that
provide a meaningful basis for informing selected investments and
building greater enterprise architecture scope, content, use, and
results. In addition, they have begun to demonstrate initial benefits
associated with using their architecture. However, the military
departments' limited satisfaction of Stage 4 elements demonstrates
that, although they have begun to develop some architecture products
(Stage 3), they have yet to complete initial versions of those
products and use those products to achieve measurable outcomes.
Military Departments Have Yet to Expand and Evolve Enterprise
Architectures and Use Them for Organizational Transformation:
Stage 5 of our EAMMF describes elements associated with establishing a
full suite of architecture products that can be employed as a featured
decision-support tool when considering and planning large-scale
organizational restructuring or transformation initiatives. Examples
of these elements include ensuring that integrated repository tools
and common enterprise architecture framework and methodology are used
across the enterprise, enterprise-level and subordinate architectures
are extended to align with external partner architectures, and all
segment and/or federated architectures exist and are horizontally and
vertically integrated.
The military departments' satisfaction of these Stage 5 core elements
indicates that they have yet to expand and evolve the development and
use of their respective enterprise architectures to support
institutional transformation. Specifically, they have satisfied only 7
percent, partially satisfied 44 percent, and not satisfied 48 percent
of these elements. Table 6 describes the extent to which each military
department has satisfied Stage 5 framework elements.
Table 6: Military Department Satisfaction of Stage 5 Framework
Elements:
Military department: Air Force;
Fully satisfied: 11%;
Partially satisfied: 44%;
Not satisfied: 44%.
Military department: Army;
Fully satisfied: 0%;
Partially satisfied: 44%;
Not satisfied: 56%.
Military department: DON;
Fully satisfied: 11%;
Partially satisfied: 44%;
Not satisfied: 44%.
Military department: Average;
Fully satisfied: 7%;
Partially satisfied: 44%;
Not satisfied: 48%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 5 elements.
Integrated tools and common frameworks and methodologies: Each of the
military departments has partially satisfied the element associated
with ensuring that integrated repository tools and a common enterprise
architecture framework and methodology are used across the enterprise.
For example, Army demonstrated its use of enterprise architecture
tools, but has yet to establish a common enterprise architecture
framework and methodology for use across the enterprise. In addition,
DON and Air Force have established tools that can serve as a common
repository for their enterprise architecture products, but they have
not fully established common enterprise architecture methodologies to
define how architectural products will be developed. It is important
for the military departments to adopt and use a common set of tools
and a common framework and methodology. Doing so helps ensure that
architecture products are developed and used consistently across the
enterprise, which in turn further supports efforts to improve
enterprisewide architecture product quality and achieve results.
Aligning enterprise architecture to external partner architectures:
Each of the military departments has partially satisfied the element
associated with ensuring that enterprise-level and subordinate
architectures are extended to align with external partner
architectures. For example, Air Force has demonstrated that it is
aligned with the DOD Information Enterprise Architecture.[Footnote 33]
However, it has not provided evidence that its enterprise architecture
aligns with the Army and DON architectures. In addition, while DON's
enterprise architecture approach is aligned with the Joint Staff's
Joint Capability Areas,[Footnote 34] it has not provided evidence that
its enterprise architecture aligns with those of the Air Force and
Army. Such alignment is critical for achieving enterprise architecture-
related goals, such as identifying potentially redundant or
duplicative business processes or IT systems and facilitating reuse of
existing systems and services.
Integrating segment and federated architectures: While each of the
military departments has begun to develop an initial version of
subordinate architectures, none of them has satisfied the element
associated with ensuring that all segment and/or federated
architectures exist and are horizontally and vertically integrated. We
have previously reported[Footnote 35] that, in large part, achieving
this core element is a byproduct of having met many of the previously
discussed core elements related to, for example, adopting one or more
enterprise architecture approaches (e.g., federation, segmentation,
etc.) and employing enterprise architecture development, maintenance,
and management rigor and discipline. However, the military departments
have not fully satisfied critical foundational elements, such as the
element associated with establishing an enterprise architecture
development and maintenance methodology or measuring and reporting
subordinate architecture alignment with the enterprise-level
architecture. While development of subordinate architectures, as
discussed earlier, typically occurs incrementally based on
institutional needs and priorities, the ultimate goal remains to
develop each of the subordinate architectures and to ensure that they
collectively form a coherent family of parent and child architectures
that are integrated both horizontally and vertically. As with the
previously-cited example, developing such integrated architecture
products is important for supporting the organization's ability to use
these products as tools for organizational transformation by, for
example, identifying potentially redundant or duplicative business
processes or IT systems and facilitating reuse of existing systems and
services.
While enterprise architecture development and use offers the potential
to achieve important departmentwide benefits, such as increased use of
enterprise licenses and improved data integration (Stage 4),
addressing the EAMMF's Stage 5 elements expands these potential
benefits to supporting large-scale departmentwide restructuring and
transformation. Accordingly, addressing Stage 5 elements would better
position the military departments to support ongoing efforts to
identify DOD efficiencies and savings[Footnote 36] by informing
efforts to look across the military departments and identify ways in
which DOD can improve effectiveness and efficiency while continuing to
meet mission demands.
Military Departments Have Taken Limited Steps to Continuously Improve
Their Respective Enterprise Architecture Programs and Use Enterprise
Architecture for Corporate Optimization:
Stage 6 of our EAMMF describes elements that are focused on continuous
improvement of the quality of the suite of enterprise architecture
products and the people, processes, and tools used to govern their
development, maintenance, and use. Examples of these elements include
continuously improving enterprise architecture program capabilities
and products as well as enterprise architecture methodologies and
tools and ensuring that the enterprise architecture informs strategic
planning and policy formulation.
The military departments' satisfaction of these Stage 6 core elements
indicates that they have begun taking steps to improve their
respective enterprise architecture program and use their respective
enterprise architectures for enterprise-level optimization, but much
still remains to be accomplished. Specifically, they have satisfied 10
percent, partially satisfied 43 percent, and not satisfied 48 percent
of these elements. Table 7 describes the extent to which each military
department has satisfied Stage 6 framework elements.
Table 7: Military Department Satisfaction of Stage 6 Framework
Elements:
Military department: Air Force;
Fully satisfied: 14%;
Partially satisfied: 43%;
Not satisfied: 43%.
Military department: Army;
Fully satisfied: 0%;
Partially satisfied: 43%;
Not satisfied: 57%.
Military department: DON;
Fully satisfied: 14%;
Partially satisfied: 43%;
Not satisfied: 43%.
Military department: Average;
Fully satisfied: 10%;
Partially satisfied: 43%;
Not satisfied: 48%.
Source: GAO analysis of military department data.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
The following examples describe the military departments' performance
relative to selected Stage 6 elements.
Improving program capabilities and products: Air Force and DON have
each fully satisfied the element associated with ensuring that
enterprise architecture continuous improvement efforts reflect the
results of external assessments, while Army has not yet satisfied this
element. For example, according to DON officials, our 2008 assessment
of the department's enterprise architecture program has been leveraged
to make program capability and product improvements, such as
establishing a formalized enterprise architecture governance
structure; a policy for enterprise architecture development and
maintenance; an IT investment process that includes compliance
assessments with DON's architecture; and a set of criteria for
measuring the quality of its products. Similarly, Air Force leveraged
our prior report to make program improvements, such as placing
enterprise architecture products under configuration management. All
efforts to continuously improve the enterprise architecture program
capabilities and products should leverage the results of external
assessments performed by organizations external to the program,
including assessments periodically performed by us, OMB, and others to
demonstrate measurable accomplishments.
Improving methodologies and tools: Each of the military departments
has partially satisfied the element associated with continuously
improving enterprise architecture methodologies and tools. For
example, each military department has mechanisms in place to improve
existing enterprise architecture tools. However, none of the military
departments has fully developed an enterprise architecture development
and maintenance methodology that can be used as a baseline for
improvements. Continuously improving enterprise architecture
methodologies and tools helps to ensure that existing methodologies
and tools continue to support changing organizational needs.
Strategic planning and policy formulation: None of the military
departments has either fully or partially satisfied the element
associated with ensuring that the enterprise architecture is used by
executive leadership to inform organization strategic planning and
policy formulation. The enterprise architecture provides the
information needed to bridge the gap between an organization's
strategic plans and the programs it implements. As such, the
architecture has traditionally been informed and constrained by these
plans and the institutional policies that govern the plans'
implementation. As an architecture program fully matures, however, a
bidirectional relationship should exist whereby it helps to inform the
same strategic plans and institutional policies to which it is
integral to implementing. In particular, the enterprise architecture
can identify the related organizational business process, performance,
information, service, technology, and security strengths, weaknesses,
and opportunity gaps that should be considered for inclusion in
strategic plans and institutional policies. For example, emerging
technologies that are reflected in the enterprise architecture's
target view can serve as the catalyst for introducing new, or for
modifying existing, strategic goals and objectives, and/or the
timelines for achieving them.
Agencies that achieve this stage of maturity have established an
enterprisewide blueprint to inform strategic planning and decision
making and "on-the-ground" implementation of these changes through a
range of capital investment and maintenance projects and other
enterprise-level initiatives. While each of the military departments
has taken steps to establish and manage its respective enterprise
architecture program and develop initial enterprise architecture
content that could eventually be used as input into organizational
strategic planning and serve as the basis for continuous improvement
activities, incomplete program management mechanisms and enterprise
architecture content limit the extent to which either of these ends
can be achieved.
Long-Standing Enterprise Architecture Management Challenges Have Not
Been Fully Addressed:
As we have previously reported,[Footnote 37] long-standing
governmentwide enterprise architecture management challenges include
organizational parochialism and cultural resistance, ensuring adequate
funding, obtaining staff skilled in the architecture discipline, and
having department or agency senior leaders that understand the
importance and role of the enterprise architecture. Military
department officials indicated that these management challenges
continue to limit their respective enterprise architecture programs.
In particular, regarding cultural resistance, a senior Air Force
architecture official stated that the department's major programs
resist adapting to enterprisewide approaches to meeting their
technical needs. With respect to adequacy of funding, Air Force and
DON representatives provided examples of activities that could not be
completed due to a lack of funding for enterprise architecture. For
example, Air Force officials stated that the department has postponed
the development of an updated enterprise architecture methodology due
to limited funding. Nevertheless, as stated previously in this report,
DOD has been provided with considerable resources for its IT systems
environment and architecture budgetary needs have not been identified
and justified through reliable cost estimating and expected program
benefits. Concerning skilled staff, DON and Air Force officials
identified as a challenge identifying enterprise architecture staff
who possess both business and technical skills. Lastly, with regard to
senior leadership understanding, Army architecture officials cited the
difficulty of getting senior leaders to understand the importance of
having an enterprise architecture and take a holistic view of the
entire military department enterprise.
The continued existence of the management challenges described has
contributed to the status of the military departments' enterprise
architecture programs, whereby the majority of EAMMF elements have yet
to be fully satisfied. Moreover, the long-standing nature of these
challenges indicates that the departments' leaders have not yet
committed to effective development and use of enterprise architecture
as described in our EAMMF.
Conclusions:
Although the Air Force, Army, and DON each have long-standing efforts
to develop enterprise architectures, the military departments have
much to do before they have enterprise architectures that are fully
developed and effectively managed. In general, the departments have
fully satisfied certain elements related to establishing an
institutional commitment to enterprise architecture and developing
initial architecture content. However, the departments generally have
not fully satisfied framework elements that are associated with
establishing the foundation for architecture management (including the
development of a plan to manage the architecture program), completing
and using initial architecture content, expanding and evolving the
enterprise architecture, and continuously improving their
architectures. This pattern of core element satisfaction indicates
that the military departments' respective enterprise architecture
programs are at risk of achieving only limited benefits. Further, the
military departments have been limited in their ability to overcome
long-standing enterprise architecture management challenges, thus
indicating a lack of organizational commitment to effective enterprise
architecture development and use.
Without fully developed and effectively managed enterprise
architectures and a plan, the Air Force, Army, and DON do not have the
needed road maps for transforming their business processes and
modernizing their supporting systems to minimize overlap and maximize
interoperability. Further, because the military departments do not
have the robust enterprise architectures that DOD's federated
architecture approach depends on, the department at large is also
without a complete road map to effectively guide its transformation.
Establishing such a road map is critical to DOD transformation. While
DOD has been provided with considerable resources for its IT systems
environment, the department is not managing its systems in a
consistent, repeatable, and effective manner that, among other things,
maximizes mission performance while minimizing or eliminating system
overlap and duplication. Because DOD is provided with over $30 billion
each year for its IT systems environment, the potential for
identifying and avoiding the costs associated with duplicative
functionality across its IT investments is significant.
Recommendation for Executive Action:
To ensure that the military departments establish commitments to fully
develop and effectively manage their enterprise architectures, we
recommend that the Secretaries of the Air Force, Army, and Navy each
expeditiously provide to the congressional defense committees a plan
that identifies milestones for their respective department's full
satisfaction of all of our Enterprise Architecture Management Maturity
Framework elements. In the event that a military department does not
intend to fully satisfy all elements of our framework, the plan should
include a rationale for why the department deems any such element(s)
to be not applicable.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from DOD. In
the comments, which are reprinted in appendix VI, the department
partially concurred with our recommendation. Specifically, the DOD and
Army CIOs concurred with the recommendation, while the Air Force and
DON CIOs did not concur.
In this regard, DOD stated that both Air Force and DON believe that
GAO's EAMMF provides a comprehensive set of elements associated with
the development and implementation of a robust enterprise architecture
program for a federal agency or organization. The department added,
however, that the Air Force and DON do not have a valid business case
that would justify the implementation of all 59 elements of our
framework. Instead, according to DOD, in today's fiscally constrained
environment, Air Force and DON have chosen to gradually implement
selected elements of the framework which are most useful in
implementing optimized, secure, and cost effective IT systems and
capabilities.
Due to the large and complex nature of the DOD enterprise, we
determined that all 59 elements of the framework apply to the military
department enterprise architecture programs. However, our
recommendation does provide the military departments with the
flexibility of providing a rationale or business case in their plans
that would justify why the department(s) deems any of the 59 elements
to be not applicable. We do not agree that fiscal constraints are a
valid reason for limiting the Air Force and DON enterprise
architecture programs to less than full satisfaction of the framework.
DOD has been provided with over $30 billion annually for its IT
systems environment, but it is not managing its systems in a
consistent, repeatable, and effective manner that, among other things,
maximizes mission performance while minimizing or eliminating system
overlap and duplication. This, in addition to the large and complex
nature of the DOD enterprise, provides compelling reasons for them to
establish commitments to fully satisfy the framework elements. We
therefore believe our recommendation remains valid as stated.
DOD also provided technical comments on this report, which have been
incorporated as appropriate.
We are sending copies of this report to appropriate congressional
committees; the Director, Office of Management and Budget; the
Secretary of the Air Force; Secretary of the Army; and the Secretary
of the Navy. This report will also be available at no charge on our
Web site at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions on matters discussed in this
report, please contact me at (202) 512-6304 or melvinv@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix VII.
Signed by:
Valerie C. Melvin:
Director:
Information Management and Human Capital Issues:
[End of section]
Appendix I: Objective, Scope, and Methodology:
Our objective was to assess the status of enterprise architecture
efforts at the Departments of the Air Force, Army, and Navy. To
address this objective, we asked each department to identify
responsible architecture officials and requested that the identified
officials self-assess their respective department's architecture
programs relative to the 59 core elements contained in version 2.0 of
our Enterprise Architecture Management Maturity Framework (EAMMF).
[Footnote 38] Specifically, we asked the officials to indicate whether
their respective department architecture programs fully satisfied,
partially satisfied, or did not satisfy each core element. We also
asked the officials to provide documentation in support of their self
assessments. To instruct the officials in preparing their assessments,
we provided the following guidance:
To fully satisfy a core element, sufficient documentation must be
provided to permit us to verify that all applicable aspects of the
core element are met. To partially satisfy a core element, sufficient
documentation must be provided to permit us to verify that at least
some aspects of the core element are met. Core elements that are
applicable and are neither fully nor partially satisfied will be
judged to be not satisfied.
Subsequently, we independently assessed each department's architecture
program relative to the 59 EAMMF core elements using the self
assessments and supporting documentation as a starting point. We then
corroborated the assessment with supporting documentation, sought
additional information as necessary through interviews with the
departments' architecture officials, obtained and reviewed additional
documentation as appropriate, and refined our determinations about the
degree to which each core element was satisfied. In performing our
analyses, we used the same criteria for determining whether a given
core element was fully satisfied, partially satisfied, or not
satisfied that we had instructed the departments to use. Finally, we
shared with the military departments the preliminary versions of the
analyses that appear in this report as appendices III, IV, and V, and
made further adjustments, as appropriate, based on additional
discussions and supporting documentation. We also solicited
information from each department on long-standing challenges to the
success of enterprise architecture in the areas of funding, cultural
resistance, senior leadership, and staff skills. The results presented
in this report reflect the state of department and agency architecture
programs as of August 1, 2011.
In performing our analyses, we interviewed officials and supporting
contractors from the Departments of the Air Force, Army, and Navy,
including the Offices of the Chief Information Officer. To gain
additional insights into the military departments' enterprise
architecture programs, we also interviewed officials in the Office of
the Secretary of Defense.
We conducted our work in the Washington, D.C., metropolitan area from
October 2010 through September 2011, in accordance with generally
accepted government auditing standards. Those standards require that
we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: EAMMF Table:
Table 8 summarizes the framework elements in version 2.0 of our
Enterprise Architecture Management Maturity Framework (EAMMF).
Table 8: Summary of EAMMF Version 2.0 Core Elements Categorized by
Stage:
Stage: Stage 0;
Core elements:
No elements.
Stage: Stage 1: Establishing enterprise architecture institutional
commitment and direction;
Core elements:
Written and approved organization policy exists for enterprise
architecture development, maintenance, and use;
Executive committee representing the enterprise exists and is
responsible and accountable for enterprise architecture;
Executive committee is taking proactive steps to address enterprise
architecture cultural barriers;
Executive committee members are trained in enterprise architecture
principles and concepts;
Chief architect exists;
Enterprise architecture purpose is clearly stated;
Enterprise architecture framework(s) is adopted;
Enterprise architecture performance and accountability framework is
established.
Stage: Stage 2: Creating the management foundation for enterprise
architecture development and use;
Core elements:
Enterprise architecture budgetary needs are justified and funded;
Enterprise architecture program office(s) exists;
Key program office leadership positions are filled;
Program office human capital plans exist;
Enterprise architecture development and maintenance methodology exists;
Automated enterprise architecture tools exist;
Enterprise architecture program management plan exists and reflects
relationships with other management disciplines;
Work breakdown structure and schedule to develop enterprise
architecture exist;
Enterprise architecture segments, federation members, and/or extended
members have been identified and prioritized;
Program office readiness is measured and reported.
Stage: Stage 3: Developing initial enterprise architecture versions;
Core elements:
Organization business owner and CXO representatives are actively
engaged in architecture development;
Enterprise architecture human capital plans are being implemented;
Program office contractor support needs are being met;
Program office staff are trained in enterprise architecture framework,
methodology, and tools;
Methodologies and tools exist to determine investment compliance with
corporate and subordinate architectures;
Methodologies and tools exist to determine subordinate architecture
alignment with the corporate enterprise architecture;
Enterprise architecture-related risks are proactively identified,
reported, and mitigated;
Initial versions of corporate "as-is" and "to-be" enterprise
architecture and sequencing plan are being developed;
Initial version of corporate enterprise architecture describing the
enterprise in terms of performance, business, data, services,
technology, and security is being developed;
One or more segment and/or federation member architectures is being
developed;
Architecture products are being developed according to the enterprise
architecture content framework;
Architecture products are being developed according to a defined
enterprise architecture methodology;
Architecture products are being developed using enterprise
architecture tools;
Architecture development progress is measured and reported.
Stage: Stage 4: Completing and using an initial enterprise
architecture version for targeted results;
Core elements:
Executive committee has approved the initial version of corporate
enterprise architecture;
Key stakeholders have approved the current version of subordinate
architectures;
Enterprise architecture is integral to the execution of other
institutional management disciplines;
Program office human capital needs are met;
Initial versions of corporate "as-is" and "to-be" enterprise
architecture and sequencing plan exist;
Initial version of corporate enterprise architecture captures
performance, business, data, services, technology, and security views;
One or more segment and/or federation member architectures exists and
is being implemented;
Enterprise architecture product quality is measured and reported;
Enterprise architecture results and outcomes are measured and reported;
Investment compliance with corporate and subordinate architectures is
measured and reported;
Subordinate architecture alignment with the corporate enterprise
architecture is measured and reported.
Stage: Stage 5: Expanding and evolving the enterprise architecture and
its use for institutional transformation;
Core elements:
Organization head has approved current version of the corporate
enterprise architecture;
Organization component heads or segment owners have approved current
version of their respective subordinate architectures;
Integrated repository tools and common enterprise architecture
framework and methodology are used across the enterprise;
Corporate and subordinate architecture program offices operate as a
single virtual office that shares resources enterprisewide;
Corporate enterprise architecture and sequencing plan are
enterprisewide in scope;
Corporate enterprise architecture and sequencing plan are aligned with
subordinate architectures;
All segment and/or federated architectures exist and are horizontally
and vertically integrated;
Corporate and subordinate architectures are extended to align with
external partner architectures;
Enterprise architecture products and management processes are subject
to independent assessment.
Stage: Stage 6: Continuously improving the enterprise architecture and
its use to achieve corporate optimization;
Core elements:
Enterprise architecture is used by executive leadership to inform
organization strategic planning and policy formulation;
Enterprise architecture human capital capabilities are continuously
improved;
Enterprise architecture methodologies and tools are continuously
improved;
Enterprise architecture management processes are continuously improved
and reflect the results of external assessments;
Enterprise architecture products are continuously improved and updated;
Enterprise architecture quality and results measurement methods are
continuously improved;
Enterprise architecture continuous improvement efforts reflect the
results of external assessments.
Source: GAO-10-846G.
[End of table]
[End of section]
Appendix III: Department of the Air Force:
The Department of the Air Force fully satisfied 12, partially
satisfied 28, and did not satisfy 19 of the 59 elements described in
our EAMMF. Table 9 summarizes the extent to which the Air Force has
addressed the core elements described in each stage of the EAMMF.
Table 10 describes the extent to which the department satisfied each
element.
Table 9: Air Force Satisfaction of Core Elements within Each Stage:
Stage: 1;
Satisfied: 50%;
Partially satisfied: 38%;
Not satisfied: 13%.
Stage: 2;
Satisfied: 10%;
Partially satisfied: 50%;
Not satisfied: 40%.
Stage: 3;
Satisfied: 29%;
Partially satisfied: 50%;
Not satisfied: 21%.
Stage: 4;
Satisfied: 9%;
Partially satisfied: 55%;
Not satisfied: 36%.
Stage: 5;
Satisfied: 11%;
Partially satisfied: 44%;
Not satisfied: 44%.
Stage: 6;
Satisfied: 14%;
Partially satisfied: 43%;
Not satisfied: 43%.
Stage: Average;
Satisfied: 20%;
Partially satisfied: 47%;
Not satisfied: 32%.
Source: GAO analysis of information provided by Air Force.
Note: Numbers do not always add to 100 percent due to rounding.
[End of table]
Table 10: Air Force Satisfaction of EAMMF Core Elements:
Stage 1:
Core element: Core element 1: Written and approved organization policy
exists for enterprise architecture development, maintenance, and use;
Satisfied? Yes;
Our analysis: Air Force has written policies, approved by the head of
the Air Force that address enterprise architecture development,
maintenance, and use. These policies identify the major players
responsible for the Air Force's enterprise architecture efforts (e.g.,
the Air Force Chief Information Officer (CIO) and heads of the
department's Major Commands); define what an enterprise architecture
includes (i.e., a baseline architecture, a target architecture, and a
sequencing plan); and set direction on the use of enterprise
architecture (e.g., inform, guide, and support Air Force decision
making).
Core element: Core element 2: Executive Committee representing the
enterprise exists and is responsible and accountable for enterprise
architecture;
Satisfied? Yes;
Our analysis: In June 2011, the department established the Air Force
CIO Executive Council as the executive-level committee that represents
the enterprise and is responsible for approving its enterprise
architecture. The Executive Council is supported by two subordinate
boards whose responsibilities include overseeing IT capital planning
and reviewing new versions of the enterprise architecture.
Core element: Core element 3: Executive Committee is taking proactive
steps to address enterprise architecture cultural barriers;
Satisfied? Partial;
Our analysis: The Air Force Architecting Division has taken steps to
address enterprise architecture cultural barriers. For example, the
Architecting Division issued an enterprise architecture communications
plan to, among other things, improve its efforts to integrate
architecture into the department's decision-making processes. However,
Air Force did not provide evidence indicating that the enterprise
architecture Executive Committee had taken steps to address enterprise
architecture cultural barriers.
Core element: Core element 4: Executive Committee members are trained
in enterprise architecture principles and concepts;
Satisfied? Partial;
Our analysis: Enterprise architecture training is available to all Air
Force employees. However, according to Air Force enterprise
architecture officials, department executives are not required or
expected to take enterprise architecture training. Instead, they are
provided with high-level briefings on the purpose and use of
enterprise architecture from enterprise architecture subject matter
experts. According to the Enterprise Architecting Division head,
department executives do not need such training because it is the
responsibility of department architects to understand enterprise
architecture principles and concepts.
Core element: Core element 5: Chief architect exists;
Satisfied? Yes;
Our analysis: As the Air Force Chief Architect, the Director of
Policy, Planning, and Resources is responsible for developing and
maintaining the Air Force enterprise architecture. The Air Force Chief
Architect also serves as the head of the CIO Group, a subcommittee
that reports directly to the Air Force's enterprise architecture
Executive Committee, and has experience in the IT and business sides
of the organization.
Core element: Core element 6: Enterprise architecture purpose is
clearly stated;
Satisfied? Partial;
Our analysis: Air Force has defined the purpose of its enterprise
architecture, which is included in information distributed to key
stakeholders. This purpose is broadly written to support the
department's goals and objectives. However, this purpose was approved
by the CIO rather than the Air Force CIO Executive Council. According
to the recently updated Air Force enterprise architecture Executive
Committee charter, the committee will approve future versions of the
department's enterprise architecture, which have previously included
descriptions of the enterprise architecture's purpose.
Core element: Core element 7: Enterprise architecture framework(s) is
adopted;
Satisfied? Yes;
Our analysis: Air Force has established an enterprise architecture
framework that documents the suite of enterprise-level enterprise
architecture artifacts to be developed, used, and maintained. The
framework also defines the overall structure of the Air Force
enterprise architecture, such as the conceptual relationships among
the various levels of architecture (e.g., enterprise-level and segment
architectures).
Core element: Core element 8: Enterprise architecture performance and
accountability framework is established;
Satisfied? No;
Our analysis: The Air Force instruction 33-401 describes roles and
responsibilities for key enterprise architecture stakeholders, such as
the CIO, Chief Architect, and Air Force Major Commands. However, Air
Force has yet to develop a framework that provides the means and
metrics for ensuring that these roles and responsibilities are
fulfilled.
Stage 2:
Core element: Core element 9: Enterprise architecture budgetary needs
are justified and funded;
Satisfied? Partial;
Our analysis: According to Air Force officials, its annual budget
request has enabled them to maintain both government and contractor
staff. These staff have worked on activities such as creating
enterprise architecture products and the approval of segment
architectures. However, they did not provide evidence that funding
requests are based on reliable cost estimates and justified based on
expected benefits, and Air Force officials stated that the
implementation of key enterprise architecture initiatives (e.g., Air
Force Enterprise Architecture 2015) have been postponed due to limited
funds.
Core element: Core element 10: Enterprise architecture program
office(s) exists;
Satisfied? Partial;
Our analysis: Air Force has established an Architecting Division
within its Office of the CIO. This division is responsible for
performing some activities that are typically associated with a
program office. For example, the division has issued enterprise
architecture configuration management guidance and work breakdown
structures for developing new versions of its enterprise-level
architecture. However, according to Air Force officials, the
department does not consider the Architecting Division to be a program
office and it has not executed program management tasks associated
with a program office (e.g., establishing a program management plan).
Core element: Core element 11: Key program office leadership positions
are filled;
Satisfied? No;
Our analysis: Air Force has recently filled the Enterprise
Architecting Division Chief position. In addition, according to Air
Force, Core Function Lead Integrators will be assigned to lead each
segment architecture. According to Air Force officials, two have been
assigned;
however, the department has yet to provide evidence to show that these
positions have been filled. Moreover, the Architecting Division staff
are not specifically assigned to key activities such as risk
management or quality assurance.
Core element: Core element 12: Program office human capital plans
exist;
Satisfied? No;
Our analysis: According to Air Force officials, the department has not
developed an enterprise architecture human capital plan. Moreover, the
former Architecting Division head stated that the department does not
plan to develop such a plan because enterprise architecture is not a
formally established career field within the federal government.
Core element: Core element 13: Enterprise architecture development and
maintenance methodology exists;
Satisfied? Partial;
Our analysis: Although some elements of a development and maintenance
methodology can be found in existing enterprise architecture program
documents, Air Force has not yet developed an enterprise architecture
development and maintenance methodology that includes defined steps,
tasks, standards, tools, techniques, and measures that govern how the
architecture is to be developed and maintained. According to Air Force
officials, efforts to develop such a methodology have been postponed
due to budget constraints.
Core element: Core element 14: Automated enterprise architecture tools
exist;
Satisfied? Yes;
Our analysis: Air Force uses various automated enterprise architecture
tools to capture information described by its enterprise architecture
framework and enable communication of information contained in its
architecture products to enterprise architecture stakeholders. For
example, Air Force uses Troux Architect to document the relationship
of architecture products in and between each of the five main Air
Force enterprise architecture reference models.
Core element: Core element 15: Enterprise architecture program
management plan exists and reflects relationships with other
management disciplines;
Satisfied? No;
Our analysis: According to the Architecting Division, Air Force
produces an annual road map from which goals and schedules for the
division are drawn. However, a plan for managing the Air Force's
enterprise architecture program that defines the management
structures, controls, disciplines, roles, and accountability
mechanisms included in our EAMMF does not currently exist.
Core element: Core element 16: Work breakdown structure and schedule
to develop enterprise architecture exist;
Satisfied? Partial;
Our analysis: Air Force provided a work breakdown structure that
identified high-level tasks, activities, and events needed to develop
updated versions of its enterprise architecture. Air Force
Architecting Division officials stated that the work breakdown
structure, which includes a schedule, is based on prior year work
breakdown structures. However, the work breakdown structure and
schedule are not created based on input from an enterprise
architecture development methodology or program plan, as these
documents do not exist.
Core element: Core element 17: Enterprise architecture segments,
federation members, and/or extended members have been identified and
prioritized;
Satisfied? Partial;
Our analysis: Air Force has identified its enterprise architecture
subordinate architectures (i.e., segments). However, the department
has not formally prioritized these subordinate architectures.
According to Air Force, Major Commands are responsible for developing
their respective subordinate architectures when each Major Command
determines that such architecture development is needed.
Core element: Core element 18: Program office readiness is measured
and reported;
Satisfied? No;
Our analysis: Air Force does not consider its Architecting Division to
be a formal enterprise architecture program office. In addition, the
department has yet to measure the extent to which people, processes,
and tools enablers have been put in place and report this readiness
information to the enterprise architecture Executive Committee and
Chief Architect.
Stage 3:
Core element: Core element 19: Organization business owner and CXO
representatives are actively engaged in architecture development;
Satisfied? Partial;
Our analysis: Representatives of the recently established enterprise
architecture Executive Committee have been engaged in architecture
development, and an Air Force policy document requires Major Commands
and other senior officials to play a lead role in developing
subordinate architectures and other enterprise architecture products
for which they are responsible. However, Air Force has not yet
formally assigned subordinate architectures to specific Major Commands.
Core element: Core element 20: Enterprise architecture human capital
plans are being implemented;
Satisfied? No;
Our analysis: Air Force has not developed an enterprise architecture
human capital plan. Air Force enterprise architecture officials stated
that such a plan is not applicable because enterprise architecture is
not a formal career field within the federal government.
Core element: Core element 21: Program office contractor support needs
are being met;
Satisfied? Partial;
Our analysis: While Air Force has not developed an enterprise
architecture human capital plan to guide its use of contractors, it
does leverage contractors to support the development of its enterprise
architecture and has mechanisms in place to monitor contractor
performance.
Core element: Core element 22: Program office staff are trained in
enterprise architecture framework, methodology, and tools;
Satisfied? Partial;
Our analysis: According to an Air Force official, enterprise
architecture training that includes, among other things, an
understanding of enterprise architecture basics, such as common
enterprise architecture views and an introduction to the DOD
Architecture Framework and Air Force enterprise architecture is
available to staff in the Office of the Chief Architect. However,
attendance is not mandatory.
Core element: Core element 23: Methodologies and tools exist to
determine investment compliance with corporate and subordinate
architectures;
Satisfied? Partial;
Our analysis: Air Force has begun to develop elements of a methodology
to determine investment compliance with the department's enterprise
architecture. For example, according to the Air Force Architecting
Division, for each IT investment, the Architecting Division completes
a common checklist to ensure its compliance with the Air Force
enterprise architecture and provides each investment with a score of
either pass or fail. However, the Air Force's approach does not
provide for exceptions to architecture compliance on the basis of
analytical justification that are captured in documented enterprise
architecture waivers and used to update the enterprise architecture.
Core element: Core element 24: Methodologies and tools exist to
determine subordinate architecture alignment with the corporate
enterprise architecture;
Satisfied? Partial;
Our analysis: Air Force has begun to develop elements of a methodology
to determine subordinate architecture alignment with the enterprise-
level architecture. For example, each subordinate architecture is
assessed using a scorecard that evaluates its compliance with the Air
Force enterprise architecture and provides each architecture with a
numerical score. However, Air Force did not provide documentation to
support that the status of alignment (including risks) among
architectures needs to be disclosed to, among others, the Executive
Committee.
Core element: Core element 25: Enterprise architecture-related risks
are proactively identified, reported, and mitigated;
Satisfied? No;
Our analysis: The Air Force's enterprise architecture concept of
operations identifies categories of risks related to organizational
use and acceptance of enterprise architecture. However, Air Force did
not provide evidence that enterprise architecture program risks are
proactively identified, reported, and mitigated.
Core element: Core element 26: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan are being
developed;
Satisfied? Partial;
Our analysis: Air Force has developed initial versions of its
enterprise-level architecture and sequencing plan, including
architecture products such as its current business reference model.
However, the department has not developed separate current and target
enterprise architectures.
Core element: Core element 27: Initial version of corporate enterprise
architecture describing the enterprise in terms of performance,
business, data, services, technology, and security is being developed;
Satisfied? Yes;
Our analysis: Initial versions of the Air Force enterprise
architecture include performance, business, data, services, and
technology reference models. In addition, security is being addressed
in the technical reference model.
Core element: Core element 28: One or more segment and/or federation
member architectures is being developed;
Satisfied? Yes;
Our analysis: Air Force has developed subordinate architectures. For
example, version 1.0 of the Space Domain architecture was approved on
September 20, 2010.
Core element: Core element 29: Architecture products are being
developed according to the enterprise architecture content framework;
Satisfied? Yes;
Our analysis: Initial versions of the enterprise-level and subordinate
architectures are consistent with Air Force's enterprise architecture
content framework.
Core element: Core element 30: Architecture products are being
developed according to a defined enterprise architecture methodology;
Satisfied? No;
Our analysis: According to Air Force officials, development of an Air
Force-unique methodology governing how Air Force architecture products
at all levels are to be developed, maintained, and validated has been
put on hold due to budget constraints (see element 13).
Core element: Core element 31: Architecture products are being
developed using enterprise architecture tools;
Satisfied? Yes;
Our analysis: Air Force is developing architecture products using the
enterprise architecture tools described in element 14.
Core element: Core element 32: Architecture development progress is
measured and reported;
Satisfied? Partial;
Our analysis: The Air Force Architecting Division measures the state
of the enterprise-level architecture and reports this information to
the Chief Architect on a quarterly basis. According to Air Force
officials, these measurements are based on a planning document
developed annually to guide the work of the Architecting Division.
However, this planning document is not based on an enterprise
architecture program plan. In addition, according to Air Force
officials, the progress is reported to the Chief Architect and not to
other enterprise architecture stakeholders, such as the Executive
Committee. Instead, according to Air Force officials, the Air Force
has briefed enterprise architecture concepts and status of key
activities regarding the Air Force enterprise architecture to the
enterprise architecture Executive Committee.
Stage 4:
Core element: Core element 33: Executive Committee has approved the
initial version of corporate enterprise architecture;
Satisfied? No;
Our analysis: The Air Force enterprise architecture has been approved
by the CIO, but not by the Executive Committee. According to officials
and its recently updated charter, this committee, the CIO Executive
Council, will approve all subsequent versions of the Air Force
enterprise architecture.
Core element: Core element 34: Key stakeholders have approved the
current version of subordinate architectures;
Satisfied? No;
Our analysis: According to Air Force officials, the latest versions of
subordinate architectures (i.e., segment architectures) have been
approved by the Architecting Division. However, the department did not
provide evidence to demonstrate that key stakeholders have approved
the latest versions of subordinate architectures.
Core element: Core element 35: Enterprise architecture is integral to
the execution of other institutional management disciplines;
Satisfied? Yes;
Our analysis: Air Force demonstrated that enterprise architecture is
integral to the execution of other institutional management
disciplines, such as the Joint Capabilities Integration Development
System.
Core element: Core element 36: Program office human capital needs are
met;
Satisfied? No;
Our analysis: Air Force has yet to develop a human capital plan that
would identify the knowledge, skills, and abilities needed to execute
the department's enterprise architecture program plans and schedules.
According to Air Force officials, vacancies in the Architecting
Division are filled on a case-by-case basis, following OMB guidelines
for civilians and based on Air Force needs.
Core element: Core element 37: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan exist;
Satisfied? Partial;
Our analysis: Air Force has developed initial versions of its
enterprise-level architecture. However, the department has not
developed separate enterprise-level current and target enterprise
architecture products. In addition, the enterprise sequencing plan
does not include a gap analysis, which is essential to assessing the
differences between the current and target environments.
Core element: Core element 38: Initial version of corporate enterprise
architecture captures performance, business, data, services,
technology, and security views;
Satisfied? Partial;
Our analysis: Air Force has developed initial versions of its
enterprise-level architecture that captures business, performance,
data, services, technology, and security information. However, the
department has not developed separate enterprise-level current and
target enterprise architecture products.
Core element: Core element 39: One or more segment and/or federation
member architectures exists and is being implemented;
Satisfied? Partial;
Our analysis: Air Force has developed Space Domain and Agile Combat
Support architecture products. However, the department did not provide
evidence that it has implemented these architectures.
Core element: Core element 40: Enterprise architecture product quality
is measured and reported;
Satisfied? Partial;
Our analysis: Air Force demonstrated that both its enterprise-level
and subordinate architectures are subject to quality reviews that
address completeness, usability, consistency, and accuracy and are
reported to the appropriate officials. However, the assessments are
not based on an approved enterprise architecture methodology that
outlines quality expectations.
Core element: Core element 41: Enterprise architecture results and
outcomes are measured and reported;
Satisfied? No;
Our analysis: Air Force officials stated that enterprise architecture
results are measured and reported for the department's business
mission area. Specifically, the department provided its annual report
on business mission area IT investments. However, these metrics do not
demonstrate results and outcomes that measure the strategic mission
value of the Air Force enterprise architecture. Air Force officials
stated the department has yet to develop additional metrics that
demonstrate enterprise architecture results due in part to a lack of
industry-recognized enterprise architecture results metrics.
Core element: Core element 42: Investment compliance with corporate
and subordinate architectures is measured and reported;
Satisfied? Partial;
Our analysis: Air Force provided evidence showing that investment
compliance with its enterprise architecture is measured against
defined criteria. According to Air Force officials, compliance is also
reported to relevant Investment Review Boards. However, the department
did not demonstrate that waivers are issued in the event of non-
compliance or that investment compliance with subordinate
architectures is measured and reported.
Core element: Core element 43: Subordinate architecture alignment with
the corporate enterprise architecture is measured and reported;
Satisfied? Partial;
Our analysis: Air Force provided evidence that subordinate
architecture alignment with the enterprise-level architecture is
measured. However, Air Force did not provide documentation to validate
that the reports are provided to its enterprise architecture Executive
Committee and the reports do not identify areas at the subordinate
level that are different from the enterprise-level architecture and
that may require a waiver.
Stage 5:
Core element: Core element 44: Organization head has approved current
version of the corporate enterprise architecture;
Satisfied? Yes;
Our analysis: The department's CIO has approved the latest version of
the Air Force enterprise architecture. An Air Force instruction, by
order of the organization head, delegates approval of the enterprise
architecture to the CIO and Chief Architect.
Core element: Core element 45: Organization component heads or segment
owners have approved current version of their respective subordinate
architectures;
Satisfied? No;
Our analysis: According to an Air Force policy document, subordinate
architectures should be approved by designated representatives of
their architecture owners prior to their certification. In addition,
according to an official within the Air Force Architecting Division,
current versions of segment architectures were approved by either the
Air Force Chief Architect or Deputy Chief Architect. However, Air
Force officials have yet to provide evidence showing that newly-
created Service Core Function-level architectures have been approved
by their respective organization component heads or segment owners.
Core element: Core element 46: Integrated repository tools and common
enterprise architecture framework and methodology are used across the
enterprise;
Satisfied? Partial;
Our analysis: While Air Force uses an integrated enterprise
architecture repository and has established a common enterprise
architecture framework, it does not have a documented methodology
governing how its architecture products are to be developed.
Core element: Core element 47: Corporate and subordinate architecture
program offices operate as a single virtual office that shares
resources enterprisewide;
Satisfied? No;
Our analysis: Air Force has not established a formal enterprise-level
architecture program office and, according to Air Force officials, the
department's architecture offices are not designed to operate as a
single virtual office.
Core element: Core element 48: Corporate enterprise architecture and
sequencing plan are enterprisewide in scope;
Satisfied? Partial;
Our analysis: The Air Force enterprise architecture is enterprisewide
in scope. However, the department has not developed separate
enterprise-level current and target enterprise architecture products.
In addition, the Air Force sequencing plan is limited to a sequencing
of systems and the enterprise architecture does not include gaps at
the enterprise level.
Core element: Core element 49: Corporate enterprise architecture and
sequencing plan are aligned with subordinate architectures;
Satisfied? No;
Our analysis: While Air Force has criteria intended to determine if a
subordinate architecture is positioned to be federated with its
enterprise-level architecture, it has not demonstrated that its
enterprise-level architecture and sequencing plan are aligned with
subordinate architectures (see element 43).
Core element: Core element 50: All segment and/or federated
architectures exist and are horizontally and vertically integrated;
Satisfied? No;
Our analysis: Air Force did not demonstrate that all segment and/or
federated architectures exist. For example, all Service Core Function
architectures have not been fully developed. Accordingly, its
architectures are not yet horizontally and vertically integrated.
Core element: Core element 51: Corporate and subordinate architectures
are extended to align with external partner architectures;
Satisfied? Partial;
Our analysis: Air Force has begun to demonstrate that its enterprise-
level and subordinate architectures are extended to align with
external partner architectures. For example, the Air Force enterprise
architecture has been certified by DOD as meeting the requirements for
aligning with the DOD Information Enterprise Architecture. However,
Air Force did not provide evidence that its enterprise architecture
aligns with other external partner architectures (e.g., Army, DON).
Core element: Core element 52: Enterprise architecture products and
management processes are subject to independent assessment;
Satisfied? Partial;
Our analysis: The Air Force Audit Agency conducted a review of
subordinate architectures in 2011. In addition, according to a
department official, an assessment of the Air Force enterprise
architecture was performed by the DOD Federated Architecture Council
to determine whether the Air Force enterprise architecture was "fit
for federation" at DOD. However, the Air Force Audit Agency review did
not address the enterprise-level Air Force enterprise architecture. In
addition, no independent assessments of the enterprise architecture
and management processes have been performed by entities accountable
to the Air Force CIO Executive Council.
Stage 6:
Core element: Core element 53: Enterprise architecture is used by
executive leadership to inform organization strategic planning and
policy formulation;
Satisfied? No;
Our analysis: Air Force enterprise architecture officials have yet to
provide evidence demonstrating that the enterprise architecture is
used by executive leadership to inform organization strategic planning
and policy formulation.
Core element: Core element 54: Enterprise architecture human capital
capabilities are continuously improved;
Satisfied? No;
Our analysis: According to an official in the Architecting Division,
enterprise architecture human capital needs are managed on an
individual basis by the development and use of individual development
plans. However, Air Force did not provide evidence to demonstrate that
it periodically reevaluates its enterprise-level and subordinate
existing enterprise architecture human capital capabilities relative
to its future needs or uses periodic gap analyses to take proactive
steps to fill knowledge and skill gaps through training, hiring, and
contracting.
Core element: Core element 55: Enterprise architecture methodologies
and tools are continuously improved;
Satisfied? Partial;
Our analysis: Air Force has mechanisms in place to evaluate its
enterprise architecture tools. For example, the department conducted
an enterprise architecture tool assessment survey that requested
information from staff regarding tool use and performance. However,
the creation of Air Force's development and maintenance methodology
has been postponed due to budget concerns.
Core element: Core element 56: Enterprise architecture management
processes are continuously improved and reflect the results of
external assessments;
Satisfied? Partial;
Our analysis: Air Force officials provided evidence that its
enterprise-level architecture management processes were evaluated. In
addition, department officials stated that the subordinate
architecture certification process has been periodically assessed and
revised. However, Air Force did not provide evidence to validate that
it used relevant external benchmarks for either of these assessments.
Core element: Core element 57: Enterprise architecture products are
continuously improved and updated;
Satisfied? Partial;
Our analysis: Air Force officials provided information to demonstrate
that enterprise architecture products are continuously improved and
updated. For example, Air Force's configuration management plan
outlines the process for making changes to the Air Force enterprise
architecture as well as sample change requests and evidence that the
requests have been approved and reflected in an updated version of the
Air Force enterprise architecture. However, the department did not
demonstrate that it has fully implemented its configuration management
plan.
Core element: Core element 58: Enterprise architecture quality and
results measurement methods are continuously improved;
Satisfied? No;
Our analysis: Air Force did not demonstrate that it periodically
reevaluates its methods for assessing enterprise-level and subordinate
architecture quality and program results.
Core element: Core element 59: Enterprise architecture continuous
improvement efforts reflect the results of external assessments;
Satisfied? Yes;
Our analysis: Air Force demonstrated that its enterprise architecture
continuous improvement efforts reflect the results of external
assessments. For example, in our 2006 report on enterprise
architecture management maturity, we reported that Air Force
enterprise architecture products were not under configuration
management. However, as described in element 57, the enterprise-level
Air Force enterprise architecture is currently under configuration
management.
Source: GAO analysis of information provided by Air Force.
[End of table]
[End of section]
Appendix IV: Department of the Army:
The Department of the Army fully satisfied 7, partially satisfied 25,
and did not satisfy 27 of the 59 elements described in our EAMMF.
Table 11 summarizes the extent to which Army has addressed the core
elements described in each stage of the EAMMF. Table 12 describes the
extent to which the department satisfied each element.
Table 11: Army Satisfaction of Core Elements within Each Stage:
Stage: 1;
Satisfied: 25%;
Partially satisfied: 50%;
Not satisfied: 25%.
Stage: 2;
Satisfied: 20%;
Partially satisfied: 40%;
Not satisfied: 40%.
Stage: 3;
Satisfied: 14%;
Partially satisfied: 50%;
Not satisfied: 36%.
Stage: 4;
Satisfied: 9%;
Partially satisfied: 27%;
Not satisfied: 64%.
Stage: 5;
Satisfied: 0;
Partially satisfied: 44%;
Not satisfied: 56%.
Stage: 6;
Satisfied: 0;
Partially satisfied: 43%;
Not satisfied: 57%.
Stage: Average;
Satisfied: 12%;
Partially satisfied: 42%;
Not satisfied: 46%.
Source: GAO analysis of information provided by Army.
[End of table]
Table 12: Army Satisfaction of EAMMF Core Elements:
Stage 1:
Core element: Core element 1: Written and approved organization policy
exists for enterprise architecture development, maintenance, and use;
Satisfied? Yes;
Our analysis: Army has written and approved policies that address
enterprise architecture development, maintenance, and use. These
policies identify the major players responsible for Army enterprise
architecture efforts (e.g., the Army Chief Information Officer and the
Army Chief Architect); define what an enterprise architecture includes
(i.e., a baseline architecture, a target architecture, and a
sequencing plan); and set direction on the use of enterprise
architecture (e.g., eliminate unnecessary or redundant processes and
reallocate resources).
Core element: Core element 2: Executive Committee representing the
enterprise exists and is responsible and accountable for enterprise
architecture;
Satisfied? Partial;
Our analysis: Army has committees that are responsible and accountable
for two of the department's three segment architectures. However, the
department does not yet have an approved Executive Committee that
represents the enterprise and is responsible and accountable for a
departmentwide enterprise architecture. According to Army, a draft
enterprise architecture policy will establish such a committee.
Core element: Core element 3: Executive Committee is taking proactive
steps to address enterprise architecture cultural barriers;
Satisfied? Partial;
Our analysis: Army has taken initial steps to address enterprise
architecture cultural barriers by providing enterprise architecture
overview briefings and plans to provide necessary resources to
enterprise architecture activities. However, the department has not
yet encouraged the disclosure and adoption of enterprise architecture
shared services and promoted and rewarded enterprise architecture-
related collaboration across organizational boundaries.
Core element: Core element 4: Executive Committee members are trained
in enterprise architecture principles and concepts;
Satisfied? No;
Our analysis: According to Army officials, there is no formal
enterprise architecture training that provides a basic understanding
of enterprise architecture fundamentals and is appropriately tailored
toward specific Executive Committee members or subordinate
organizations.
Core element: Core element 5: Chief architect exists;
Satisfied? Yes;
Our analysis: Army has appointed the Deputy Chief of Staff for
Operations, Plans, and Training as the Army Chief Architect. The Chief
Architect is responsible for, among other things, oversight and
management of all Army architecture efforts.
Core element: Core element 6: Enterprise architecture purpose is
clearly stated;
Satisfied? Partial;
Our analysis: Army's Generating Force, Operating Force, and Network
segments have defined purpose statements. However, these statements
were not defined and approved by an enterprise-level Executive
Committee.
Core element: Core element 7: Enterprise architecture framework(s) is
adopted;
Satisfied? Partial;
Our analysis: Army has adopted the Department of Defense Architecture
Framework version 1.5 and is adopting version 2.0 as the basis for
describing its enterprise architecture products. However, the suite of
enterprise architecture products and artifacts to be developed, used,
and maintained under the architecture framework version 2.0 has not
been specified.
Core element: Core element 8: Enterprise architecture performance and
accountability framework is established;
Satisfied? No;
Our analysis: An enterprise architecture performance and
accountability framework has not been established. Specifically, an
enterprise-level approach for measuring enterprise architecture
progress, management capacity, quality, use, and results has not been
established. Further, although Army has identified the roles and
responsibilities of key stakeholders, the specific metrics and means
for ensuring that the roles and responsibilities are fulfilled and any
deviations from expectations are not documented and disclosed.
Stage 2:
Core element: Core element 9: Enterprise architecture budgetary needs
are justified and funded;
Satisfied? Partial;
Our analysis: According to Army officials, funding requests for
enterprise architecture needs are ad hoc and decentralized. While the
architecture segments have received funding, their budgetary needs
have not been fully met.
Core element: Core element 10: Enterprise architecture program
office(s) exists;
Satisfied? Partial;
Our analysis: Army has yet to establish a program office with
responsibility for the department's enterprise architecture
development and maintenance. According to Army officials, a draft
regulation is being written and will assign responsibility for
managing enterprise architecture. Existing subordinate program
management offices are chartered and have responsibility for the
Generating Force, Operating Force, and Network segment architectures.
Core element: Core element 11: Key program office leadership positions
are filled;
Satisfied? Partial;
Our analysis: Although Army has designated a Chief Architect, key
program office leadership positions such as a configuration manager,
risk manager, and quality assurance manager have not been identified
and filled. While the Generating Force, Operating Force, and Network
segment architectures have lead architects, key leadership positions
for these segment architecture programs have not been filled.
Core element: Core element 12: Program office human capital plans
exist;
Satisfied? No;
Our analysis: Army does not have an enterprise architecture program
office human capital plan. Specifically, Army has not identified the
human capital needs and developed a plan for acquiring, developing,
and retaining qualified staff with the requisite knowledge, skills,
and abilities.
Core element: Core element 13: Enterprise architecture development and
maintenance methodology exists;
Satisfied? No;
Our analysis: The Army is drafting a regulation that calls for the
development of an enterprise architecture methodology. However, the
department has not yet documented a methodology that includes defined
steps, tasks, standards, tools, techniques, and measures that govern
how its enterprise architecture is to be developed, maintained, and
validated.
Core element: Core element 14: Automated enterprise architecture tools
exist;
Satisfied? Yes;
Our analysis: Army uses various automated tools to assist in capturing
enterprise architecture information and developing, communicating,
storing, and maintaining architecture products. For example, the
department uses System Architect for the development of traditional
enterprise architecture artifacts and a repository tool.
Core element: Core element 15: Enterprise architecture program
management plan exists and reflects relationships with other
management disciplines;
Satisfied? Partial;
Our analysis: Army has not developed an enterprise architecture
program management plan. However, the Network segment has developed a
program management plan and an approach to developing its architecture
that includes management structures, controls, and institutional
management disciplines.
Core element: Core element 16: Work breakdown structure and schedule
to develop enterprise architecture exist;
Satisfied? No;
Our analysis: Army does not have a work breakdown structure to develop
its enterprise architecture or architecture segments. Additionally,
the department does not have a schedule to develop its enterprise
architecture for two of the three architecture segments. While Army
provided a high-level schedule for the Network architecture segment,
the schedule does not define the timing, sequencing, and duration of
program tasks, activities, and events.
Core element: Core element 17: Enterprise architecture segments,
federation members, and/or extended members have been identified and
prioritized;
Satisfied? Yes;
Our analysis: Army has identified and prioritized three segment
architectures: Generating Force, Operating Force, and Network in the
Army Campaign Plan. Specifically, the first priority is the Operating
Force segment with some support from the Generating Force and Network
segments. The second priority is the Generating Force segment with
some support from the Network segment. The third priority is the
Network segment.
Core element: Core element 18: Program office readiness is measured
and reported;
Satisfied? No;
Our analysis: Army does not yet have an enterprise architecture
program office. Further, the readiness of segment architecture program
offices is not measured and reported. Specifically, the Army's people,
processes, and tools elements have not been measured and have not been
shared with the Executive Committee, Chief Architect, and subordinate
architects.
Stage 3:
Core element: Core element 19: Organization business owner and CXO
representatives are actively engaged in architecture development;
Satisfied? Partial;
Our analysis: According to Army officials, organization business
owners are assigned to segment architecture program offices. For
example, Army officials stated that their General Officers (or
equivalent) are actively involved with approving Army architecture
development priorities and architecture products. However, Army has
not yet established an enterprise-level architecture program office.
Core element: Core element 20: Enterprise architecture human capital
plans are being implemented;
Satisfied? No;
Our analysis: Army does not have an enterprise architecture human
capital plan. Specifically, Army has not identified the human capital
needs or developed a plan for acquiring, developing, and retaining
qualified staff with the requisite knowledge, skills, and abilities.
Core element: Core element 21: Program office contractor support needs
are being met;
Satisfied? No;
Our analysis: According to Army officials, the segment architecture
program office contractor support needs are not being met.
Additionally, Army has not yet established an enterprise-level
architecture office.
Core element: Core element 22: Program office staff are trained in
enterprise architecture framework, methodology, and tools;
Satisfied? Partial;
Our analysis: According to Army officials, enterprise architecture
staff have attended architecture-related training. However, training
needs are not identified in a human capital management plan.
Core element: Core element 23: Methodologies and tools exist to
determine investment compliance with corporate and subordinate
architectures;
Satisfied? Partial;
Our analysis: According to Army officials, a methodology and tool
exist and are used to determine business system investment compliance
with its Generating Force segment architecture. However, automated
tools and methodologies do not yet exist for non-business system
investment compliance with the Operating Force and Network segment
architectures.
Core element: Core element 24: Methodologies and tools exist to
determine subordinate architecture alignment with the corporate
enterprise architecture;
Satisfied? No;
Our analysis: Army does not have an enterprise-level architecture to
which a methodology to determine subordinate architecture alignment
can be applied.
Core element: Core element 25: Enterprise architecture-related risks
are proactively identified, reported, and mitigated;
Satisfied? No;
Our analysis: According to Army officials, the department does not
have a formal set of risk management activities to proactively
identify, report, and mitigate enterprise architecture-related risks.
In addition, although the Army's Network segment architecture has a
methodology that includes a risk management process, this process has
yet to be implemented.
Core element: Core element 26: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan are being
developed;
Satisfied? Partial;
Our analysis: The department is developing segment architecture
products that can be used to inform the development of an enterprise-
level architecture. However, according to Army officials, enterprise-
level architecture products have not yet been developed.
Core element: Core element 27: Initial version of corporate enterprise
architecture describing the enterprise in terms of performance,
business, data, services, technology, and security is being developed;
Satisfied? Partial;
Our analysis: Army is developing segment architectures that begin to
describe the enterprise segments in terms of performance, business,
data, services, technology, and security. However, it has yet to
develop an enterprise-level architecture that describes enterprise
elements such as business rules and outcomes that all Army components
are expected to adopt.
Core element: Core element 28: One or more segment and/or federation
member architectures is being developed;
Satisfied? Yes;
Our analysis: Army is developing segment architectures. For example,
the department is developing architectures for its Generating Force
and Network segments.
Core element: Core element 29: Architecture products are being
developed according to the enterprise architecture content framework;
Satisfied? Partial;
Our analysis: Army's segment architecture products are being developed
in accordance with guidance such as the DOD architecture framework.
However, a complete enterprise architecture framework has not yet been
developed.
Core element: Core element 30: Architecture products are being
developed according to a defined enterprise architecture methodology;
Satisfied? No;
Our analysis: Army has established strategies for developing segment
architecture products. However, a defined enterprise architecture
methodology that includes steps, tasks, standards, tools, techniques,
and measures to consistently develop enterprise-level architecture
products has not yet been developed.
Core element: Core element 31: Architecture products are being
developed using enterprise architecture tools;
Satisfied? Yes;
Our analysis: Army is developing architecture products using the
enterprise architecture tools described in element 14.
Core element: Core element 32: Architecture development progress is
measured and reported;
Satisfied? Partial;
Our analysis: Progress against plans is measured and reported for
Army's Generating Force segment architecture, but not for Operating
Force and Network segments. For example, the Office of Business
Transformation produced a summary report that described the office's
efforts for fiscal year 2010.
Stage 4:
Core element: Core element 33: Executive Committee has approved the
initial version of corporate enterprise architecture;
Satisfied? No;
Our analysis: Army does not have an enterprise-level architecture that
has been approved by an executive committee.
Core element: Core element 34: Key stakeholders have approved the
current version of subordinate architectures;
Satisfied? No;
Our analysis: According to Army officials, key stakeholders, such as
business owners and executive sponsors, have not approved all major
releases of the department's subordinate architectures.
Core element: Core element 35: Enterprise architecture is integral to
the execution of other institutional management disciplines;
Satisfied? Yes;
Our analysis: Segment architectures (Generating Force, Operating
Force, and Network) are linked to the execution of other institutional
management disciplines such as the Joint Capability Integration and
Development System, Defense Acquisition System, and Planning,
Programming, Budgeting, and Execution System.
Core element: Core element 36: Program office human capital needs are
met;
Satisfied? No;
Our analysis: Army does not have a basis for meeting the enterprise
architecture human capital needs because it has not identified
staffing requirements or gaps. Further, according to Army officials,
they do not have sufficient staff to support their enterprise
architecture program.
Core element: Core element 37: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan exist;
Satisfied? Partial;
Our analysis: Army has developed initial versions of segment
architecture products that can be used to inform the development of an
enterprise-level architecture. For example, the department has
developed initial versions of Army architectures for its Generating
Force and Network segments. However, according to Army officials,
enterprise-level architecture products have not yet been developed.
Core element: Core element 38: Initial version of corporate enterprise
architecture captures performance, business, data, services,
technology, and security views;
Satisfied? Partial;
Our analysis: Army has developed initial versions of segment
architecture products that can be used to inform the development of an
enterprise-level architecture. For example, the initial version of its
Network segment architecture begins to document its performance,
business, data, services, technology, and security views. However,
according to Army officials, the enterprise-level architecture has yet
to be developed.
Core element: Core element 39: One or more segment and/or federation
member architectures exists and is being implemented;
Satisfied? Partial;
Our analysis: Army identified three segments, each with sub-segments.
However, these segments have not been fully developed or implemented
on a targeted or prioritized basis (see element 17). With respect to
implementation, the department has used segment architecture artifacts
to make decisions such as assessing data centers for closure or
sustainment and consolidating data and enterprise e-mail.
Core element: Core element 40: Enterprise architecture product quality
is measured and reported;
Satisfied? No;
Our analysis: Army does not measure or report the quality of its
enterprise architecture products. Although the Network architecture
segment documentation states that quality control measures are to be
used to determine quality, reuse, compliance, and risk, related
measurements have not yet been defined. Further, according to Army
officials, the quality of Generating Force and Operating Force
architecture products is not measured and reported.
Core element: Core element 41: Enterprise architecture results and
outcomes are measured and reported;
Satisfied? No;
Our analysis: Army does not measure and report results and outcomes of
its enterprise architecture efforts.
Core element: Core element 42: Investment compliance with corporate
and subordinate architectures is measured and reported;
Satisfied? No;
Our analysis: Measurement of investment compliance with Army
enterprise-level architecture products does not occur because such
products do not yet exist. Army officials did not provide sufficient
documentation to support their position that investment compliance
with subordinate architectures is measured and reported.
Core element: Core element 43: Subordinate architecture alignment with
the corporate enterprise architecture is measured and reported;
Satisfied? No;
Our analysis: Subordinate architecture alignment with the enterprise-
level architecture is not measured and reported. Further, Army has not
yet developed an enterprise-level architecture with which its
subordinate architectures' alignment could be measured and reported.
Stage 5:
Core element: Core element 44: Organization head has approved current
version of the corporate enterprise architecture;
Satisfied? No;
Our analysis: Army has not yet developed an enterprise-level
architecture that would be approved by the Secretary of the Army.
Core element: Core element 45: Organization component heads or segment
owners have approved current version of their respective subordinate
architectures;
Satisfied? Partial;
Our analysis: According to Army, organization component heads or
segment owners have approved the current version of the department's
subordinate architectures. However, Army officials did not provide
evidence that these approvals were based on quality measures.
Core element: Core element 46: Integrated repository tools and common
enterprise architecture framework and methodology are used across the
enterprise;
Satisfied? Partial;
Our analysis: Generating Force architecture products are currently
stored in a single repository;
however, this repository does not include all Army architecture
products. According to Army officials, Army plans to use a single
repository tool for storing all architecture products. In addition,
Army strategy calls for a common enterprise architecture framework and
methodology to be used across the enterprise but this is not yet in
place.
Core element: Core element 47: Corporate and subordinate architecture
program offices operate as a single virtual office that shares
resources enterprisewide;
Satisfied? No;
Our analysis: Army has not yet established an enterprise-level
architecture office and the department's subordinate architecture
program offices do not operate as a single virtual office that shares
resources.
Core element: Core element 48: Corporate enterprise architecture and
sequencing plan are enterprisewide in scope;
Satisfied? Partial;
Our analysis: According to Army officials, Army segments are
enterprisewide in scope. In addition, Army has established a basis for
developing enterprise-level architecture, such as the Army Operating
Concept which describes the Army's mission and future operational
environment. Further, Army has established a basis for developing an
enterprise sequencing plan. For example, the Capabilities Set process
describes concepts for prioritizing, integrating, and synchronizing
activities across the Army.
Core element: Core element 49: Corporate enterprise architecture and
sequencing plan are aligned with subordinate architectures;
Satisfied? No;
Our analysis: Army has not developed an initial version of its
enterprise-level architecture or sequencing plan, which would provide
the basis for subordinate architecture alignment.
Core element: Core element 50: All segment and/or federated
architectures exist and are horizontally and vertically integrated;
Satisfied? No;
Our analysis: Army intends to horizontally and vertically integrate
its architecture products. However, such integration has not yet
occurred.
Core element: Core element 51: Corporate and subordinate architectures
are extended to align with external partner architectures;
Satisfied? Partial;
Our analysis: Army has begun to demonstrate that its architectures are
extended to align with external partner architectures. For example,
Army has demonstrated that its Generating Force segment is aligned
with the DOD business enterprise architecture. However, the department
has not demonstrated alignment with other external partner
architectures (e.g., DON, Air Force).
Core element: Core element 52: Enterprise architecture products and
management processes are subject to independent assessment;
Satisfied? No;
Our analysis: Army's enterprise architecture products and management
processes are not subject to independent assessment.
Stage 6:
Core element: Core element 53: Enterprise architecture is used by
executive leadership to inform organization strategic planning and
policy formulation;
Satisfied? Partial;
Our analysis: According to Army officials, the segment architectures
have been used to inform a key reference handbook for senior Army
leaders. However, Army officials did not provide sufficient
documentation to clearly link its architecture products with strategic
plans and institutional policies.
Core element: Core element 54: Enterprise architecture human capital
capabilities are continuously improved;
Satisfied? No;
Our analysis: Army's enterprise architecture human capital
capabilities are not continuously improved.
Core element: Core element 55: Enterprise architecture methodologies
and tools are continuously improved;
Satisfied? Partial;
Our analysis: According to Army officials, efforts to improve
architecture tools are made but have been limited to the segment
architectures, including the Generating Force and Operating Force
segments. The officials stated that the department is drafting an
enterprise architecture policy that will provide further guidance in
this area.
Core element: Core element 56: Enterprise architecture management
processes are continuously improved and reflect the results of
external assessments;
Satisfied? No;
Our analysis: According to Army officials, the department has not
subjected its enterprise architecture management processes to periodic
reassessments by an entity that is external to the enterprise
architecture program, such as the department's internal audit function
or a contractor that is not responsible for any architecture
development, maintenance, or management activities.
Core element: Core element 57: Enterprise architecture products are
continuously improved and updated;
Satisfied? Partial;
Our analysis: According to Army officials, the Generating Force and
Operating Force segment enterprise architecture products have been
improved and updated to reflect events such as changes in legal
requirements. However, Army's Network segment architecture officials
reported that its enterprise architecture products are not
continuously improved and updated;
and the Army does not yet have a formal configuration management
process for ongoing architecture maintenance.
Core element: Core element 58: Enterprise architecture quality and
results measurement methods are continuously improved;
Satisfied? No;
Our analysis: According to Army officials, enterprise architecture
quality and results measurement methods are not continuously improved.
Core element: Core element 59: Enterprise architecture continuous
improvement efforts reflect the results of external assessments;
Satisfied? No;
Our analysis: According to Army officials, enterprise architecture
continuous improvement efforts do not reflect the results of external
assessments.
Source: GAO analysis of information provided by Army.
[End of table]
[End of section]
Appendix V: Department of the Navy:
The Department of the Navy (DON) fully satisfied 16, partially
satisfied 24, and did not satisfy 19 of the 59 elements described in
our EAMMF. Table 13 summarizes the extent to which DON has addressed
the core elements described in each stage of the EAMMF. Table 14
describes the extent to which the department satisfied each element.
Table 13: DON Satisfaction of Core Elements within Each Stage:
Stage: 1;
Satisfied: 50%;
Partially satisfied: 38%;
Not satisfied: 13%.
Stage: 2;
Satisfied: 20%;
Partially satisfied: 50%;
Not satisfied: 30%.
Stage: 3;
Satisfied: 36%;
Partially satisfied: 36%;
Not satisfied: 29%.
Stage: 4;
Satisfied: 27%;
Partially satisfied: 36%;
Not satisfied: 36%.
Stage: 5;
Satisfied: 11%;
Partially satisfied: 44%;
Not satisfied: 44%.
Stage: 6;
Satisfied: 14%;
Partially satisfied: 43%;
Not satisfied: 43%.
Stage: Average;
Satisfied: 27%;
Partially satisfied: 41%;
Not satisfied: 32%.
Source: GAO analysis of information provided by DON.
[End of table]
Table 14: DON Satisfaction of GAO EAMMF Core Elements:
Stage 1:
Core element: Core element 1: Written and approved organization policy
exists for enterprise architecture development, maintenance, and use;
Satisfied: Yes;
Our analysis: DON has written and approved policies, approved by the
head of the department that address enterprise architecture
development, maintenance, and use. These policies identify the major
players responsible for DON's enterprise architecture efforts, (e.g.,
the DON CIO and DON Chief Architect); define what the enterprise
architecture must include (i.e., the current architecture, target
architecture, and a plan to transition to the desired state); and set
direction on the use of enterprise architecture (e.g., promotion of
interoperability, public access, and IT security).
Core element: Core element 2: Executive Committee representing the
enterprise exists and is responsible and accountable for enterprise
architecture;
Satisfied: Yes;
Our analysis: DON has assigned responsibility and accountability for
directing, overseeing, and approving its architecture to a formally
chartered executive committee named the Information Enterprise
Governance Board (IGB). It includes representatives from across the
organization's units, such as research, development, and acquisition;
financial management; energy and the environment; manpower; and cyber
command.
Core element: Core element 3: Executive Committee is taking proactive
steps to address enterprise architecture cultural barriers;
Satisfied: Yes;
Our analysis: DON is taking steps to address enterprise architecture
cultural barriers, as evidenced by the recent establishment of the IGB
with the intent of increasing participation and support for the
department's enterprise architecture among senior-level staff;
the department's plans to provide training to the acquisition
community on the value of enterprise architecture;
and plans to develop a road map to address the lack of a common
understanding among stakeholders on how enterprise architecture
relates to department plans and goals.
Core element: Core element 4: Executive Committee members are trained
in enterprise architecture principles and concepts;
Satisfied: Partial;
Our analysis: DON offers training on basic enterprise architecture
fundamentals, such as DON's enterprise architecture content and
framework, at various Navy and Marine Corps locations and this
training is available to IGB members. However, this training is not
mandatory for members of the IGB. According to officials, members of
the committee are assumed to have the knowledge and experience needed
to understand the department's enterprise architecture at an
appropriate level of granularity without attending training sessions.
Core element: Core element 5: Chief architect exists;
Satisfied: Yes;
Our analysis: As the DON chief architect, the CIO leads the enterprise-
level architecture program and is responsible for enterprise
architecture development and maintenance. The chief architect is also
accountable to the IGB and has experience in the IT and business sides
of the organization.
Core element: Core element 6: Enterprise architecture purpose is
clearly stated;
Satisfied: Partial;
Our analysis: DON has defined the purpose for its enterprise
architecture, which is communicated to the stakeholders in key
documents, such as training packages. In addition, the purpose is
written to support DON's goals and objectives. However, although the
purpose was approved by executive-level officials such as the CIO, it
has not yet been approved by the IGB, which includes representatives
from across the organization's units. According to officials, the IGB
will approve future versions of the enterprise architecture purpose.
However, this responsibility has yet to be formally documented.
Core element: Core element 7: Enterprise architecture framework(s) is
adopted;
Satisfied: Partial;
Our analysis: DON has developed a version of an enterprise
architecture framework. However the framework has yet to define the
complete suite of enterprise architecture products and artifacts to be
developed or the relationships between them. Further, according to DON
documentation, the framework is not yet sufficiently flexible to serve
the needs of a large and diverse organization such as DON.
Core element: Core element 8: Enterprise architecture performance and
accountability framework is established;
Satisfied: No;
Our analysis: Although the March 2011 enterprise architecture
Executive Committee charter calls for its members to develop metrics
and feedback measures for evaluating the effectiveness of key
stakeholders (e.g., DON Deputy CIO (Navy), DON Deputy CIO (Marine
Corps), and the DON CIO) in achieving IT goals, DON officials stated
that an enterprisewide enterprise architecture performance and
accountability framework has yet to be established.
Stage 2:
Core element: Core element 9: Enterprise architecture budgetary needs
are justified and funded;
Satisfied: Partial;
Our analysis: According to DON officials, although sufficient
budgetary resources to establish and execute its enterprise
architecture program are not available due to the current fiscally
constrained environment, they have been able to maintain a small team
of government and contractor staff for the program, which has allowed
them to achieve several milestones and begin to develop and use some
architecture artifacts. Officials acknowledged, however, that
stabilizing and using the current DON enterprise architecture has been
prioritized over developing additional enterprise architecture content
since sufficient resources are not available for both activities.
Officials also stated that the level of funding needed to satisfy
enterprise architecture budgetary needs has not been identified and
justified through reliable cost estimating and expected program
benefits. Rather, architecture activities are funded out of individual
stakeholder and program budgets, and funding levels are discretionary.
Core element: Core element 10: Enterprise architecture program
office(s) exists;
Satisfied: Partial;
Our analysis: DON has established a small, dedicated team within the
CIO's office to perform activities that are typically associated with
a program office. For example, specific team members are responsible
for configuration management, program planning, performance
monitoring, and project status reporting to the department's
enterprise architecture Executive Committee. However, according to
officials, the department has no plans to establish a formally
chartered enterprise-level architecture program management office.
Officials stated that it is difficult to justify the creation of a
large enterprise architecture program office in a fiscally constrained
environment.
Core element: Core element 11: Key program office leadership positions
are filled;
Satisfied: Yes;
Our analysis: Although DON has not established a formally chartered
enterprise architecture program office, key enterprise architecture
leadership roles are being performed such as the enterprise
architecture Project Manager, Senior Technical Architect, and the
Release Agent responsible for enterprise architecture configuration
management. Additionally, key enterprise architecture governance
groups have been established to support the Chief Architect, such as
the Enterprise Architecture Working Group, to assist in developing
enterprise architecture artifacts, the Independent Verification and
Validation Working Group to assess enterprise architecture artifact
quality, and the Configuration Control Board to approve changes to
enterprise architecture artifacts.
Core element: Core element 12: Program office human capital plans
exist;
Satisfied: No;
Our analysis: According to DON officials, enterprise architecture
program staff are not hired according to a human capital plan that
would identify the knowledge, skills, and abilities that are needed
for the enterprise architecture program as well as the approach for
addressing any gaps in training, developing, and retaining existing
staff or hiring new staff.
Core element: Core element 13: Enterprise architecture development and
maintenance methodology exists;
Satisfied: Partial;
Our analysis: Although officials demonstrated that elements of an
enterprise architecture development and maintenance methodology can be
found in existing enterprise architecture program documents, they
acknowledged that a comprehensive methodology that includes defined
steps, tasks, standards, tools, techniques, and measures that govern
how the architecture is to be developed, maintained, and validated has
yet to be developed. Officials indicated that there are no specific
time frames for when a DON enterprise architecture methodology will be
developed. According to DON officials, this is due to the department's
focus on other resource-intensive commitments, such as applying the
current enterprise architecture content.
Core element: Core element 14: Automated enterprise architecture tools
exist;
Satisfied: Yes;
Our analysis: DON uses various automated enterprise architecture tools
to capture information described by its enterprise architecture
framework and to develop, communicate, store, and maintain
architecture products. For example, the department uses System
Architect for the development of traditional enterprise architecture
artifacts and an IT portfolio registry for assessing architecture
compliance assertions and waiver requests.
Core element: Core element 15: Enterprise architecture program
management plan exists and reflects relationships with other
management disciplines;
Satisfied: Partial;
Our analysis: Although a DON enterprise architecture governance plan
is in place that defines enterprise architecture management structures
and the roles and responsibilities of stakeholders, a comprehensive
plan for managing the DON enterprise architecture program that defines
the major enterprise architecture releases to be developed and
addresses key enterprise architecture management areas such as human
capital management, risk management, and information security
management has yet to be developed. According to officials, an
overarching road map is in development that will act as the enterprise
architecture program management plan until a more detailed plan is
developed. In the absence of a plan, officials stated that enterprise
architecture activities are informally managed by, for example,
discussing program priorities with subject matter experts.
Core element: Core element 16: Work breakdown structure and schedule
to develop enterprise architecture exist;
Satisfied: No;
Our analysis: DON does not currently have a work breakdown structure
that decomposes the specific tasks, activities, and events needed to
execute the department's enterprise architecture program, and a
reliable schedule that defines the timing, sequencing, and duration of
the tasks, activities, and events. According to DON officials, the
overarching road map in development that will act as the enterprise
architecture program management plan will also identify enterprise
architecture program milestones. Officials also stated that detailed
work breakdown structures are expected to be developed at a future
date to support the achievement of the milestones.
Core element: Core element 17: Enterprise architecture segments,
federation members, and/or extended members have been identified and
prioritized;
Satisfied: Partial;
Our analysis: According to DON officials and draft documentation, nine
enterprise architecture segment reference architectures have been
identified, of which three have been prioritized for initial
development. However, the identification and prioritization of these
segments have yet to be approved by the Executive Committee. According
to officials, the identification and prioritization will be discussed
with the executive committee once the enterprise architecture road map
is relatively mature.
Core element: Core element 18: Program office readiness is measured
and reported;
Satisfied: No;
Our analysis: DON has not chartered an office to manage its enterprise
architecture program. Thus, the department has yet to measure the
extent to which people, processes, and tools enablers have been put in
place and report this readiness information to the enterprise
architecture Executive Committee and Chief Architect.
Stage 3:
Core element: Core element 19: Organization business owner and CXO
representatives are actively engaged in architecture development;
Satisfied: Yes;
Our analysis: According to documentation, executive-level members,
such as the DON CIO, Chief Architect, Deputy CIOs for the Navy and
Marine Corps, and the Deputy Chief Management Officer, are actively
engaged in developing enterprise architecture products with enterprise
architecture program staff.
Core element: Core element 20: Enterprise architecture human capital
plans are being implemented;
Satisfied: No;
Our analysis: DON has yet to develop a human capital plan that
identifies the knowledge, skills, and abilities that are needed for
the enterprise architecture program as well as the approach for
addressing any gaps in training, developing, and retaining existing
staff or hiring new staff.
Core element: Core element 21: Program office contractor support needs
are being met;
Satisfied: Partial;
Our analysis: According to officials, the enterprise architecture
program is supported by two full-time contractor support staff.
Officials noted that they consider this to be reasonable considering
the current fiscally constrained environment. However, a human capital
plan is not in place to ensure that the appropriate degrees of
contractor expertise, skills, and competencies are acquired and
assimilated into the program office.
Core element: Core element 22: Program office staff are trained in
enterprise architecture framework, methodology, and tools;
Satisfied: Partial;
Our analysis: According to DON officials, enterprise architecture
training is targeted to the expected audience and available for
program staff at conferences or training sessions provided at various
Navy and Marine Corps locations;
however, attendance is not mandatory. The enterprise architecture
training package provides a basic understanding of enterprise
architecture fundamentals, including enterprise architecture content,
the enterprise architecture framework, enterprise architecture
governance, and enterprise architecture compliance assertions and
review processes.
Core element: Core element 23: Methodologies and tools exist to
determine investment compliance with corporate and subordinate
architectures;
Satisfied: Yes;
Our analysis: DON has developed methodologies and tools to determine
investment compliance with the department's enterprise architecture.
Specifically, in October 2009, the department released an updated
version of its Investment Review Process Guidance that describes the
process for assessing investment compliance with the enterprise-level
architecture on an annual basis. The methodology provides for
exceptions to architecture compliance on the basis of analytical
justifications that are (1) captured in documented enterprise
architecture waivers and (2) used to update the enterprise
architecture. In addition, according to the guide and officials, the
enterprise architecture compliance and waiver processes are fully
automated in the department's variant of the Department of Defense's
Information Technology Portfolio Repository tool.
Core element: Core element 24: Methodologies and tools exist to
determine subordinate architecture alignment with the corporate
enterprise architecture;
Satisfied: Partial;
Our analysis: According to officials, the results of subordinate
architecture alignment assessments with the enterprise-level
architecture can be captured in the Department of Defense's
Information Technology Portfolio Repository tool;
however, officials stated that a methodology does not yet exist for
developing segment reference architectures and will need to be
developed at some future date. The officials also stated that the
methodology would include specific processes, procedures, and
guidelines in order to ensure that subordinate architectures are
aligned to the enterprise-level architecture and that the verification
of alignment is expected to be conducted by DON's enterprise
architecture Independent Verification & Validation Working Group.
Core element: Core element 25: Enterprise architecture-related risks
are proactively identified, reported, and mitigated;
Satisfied: No;
Our analysis: According to officials, enterprise architecture program
risk management activities are informally conducted and not explicitly
aligned with a risk management process. DON officials stated that in
addition to acting as the enterprise architecture program management
plan, work breakdown structure, and schedule, a road map document is
being developed to address enterprise architecture risk management
activities.
Core element: Core element 26: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan are being
developed;
Satisfied: Partial;
Our analysis: Initial versions of the enterprise-level architecture
are being developed. However, the department does not expect to
document separate current and target architectures. Further, while the
department has developed a business architecture transition plan, it
does not intend to develop an enterprisewide transition plan.
Core element: Core element 27: Initial version of corporate enterprise
architecture describing the enterprise in terms of performance,
business, data, services, technology, and security is being developed;
Satisfied: Yes;
Our analysis: Initial versions of the enterprise architecture describe
the enterprise in terms of business, data, services, technology, and
security. For example, it identifies reference models for business,
data, services, technology, and security.
Core element: Core element 28: One or more segment and/or federation
member architectures is being developed;
Satisfied: Yes;
Our analysis: One or more segment and/or federation member
architectures is being developed. For example, the department has
developed artifacts for its Net Centric segment architecture.
Core element: Core element 29: Architecture products are being
developed according to the enterprise architecture content framework;
Satisfied: Partial;
Our analysis: Architecture products are being developed according to
guidelines. However, the enterprise architecture framework has yet to
define the complete suite of enterprise architecture products and
artifacts to be developed or the relationships between them.
Core element: Core element 30: Architecture products are being
developed according to a defined enterprise architecture methodology;
Satisfied: No;
Our analysis: While the department does not currently have a defined
enterprise architecture methodology, it is developing a road map for
developing such a methodology.
Core element: Core element 31: Architecture products are being
developed using enterprise architecture tools;
Satisfied: Yes;
Our analysis: DON is developing architecture products using the
enterprise architecture tools described in element 14.
Core element: Core element 32: Architecture development progress is
measured and reported;
Satisfied: No;
Our analysis: According to officials, the department does not
currently have the necessary documents in place for measuring and
reporting on architecture development progress. Specifically, as
stated, the department has yet to develop an enterprise architecture
program plan, work breakdown structure, and schedule, as well as their
associated costs;
as a result, the department's progress in executing tasks defined in
such documents cannot be measured.
Stage 4:
Core element: Core element 33: Executive Committee has approved the
initial version of corporate enterprise architecture;
Satisfied: Partial;
Our analysis: The current DON enterprise architecture has been
approved by the department's Enterprise Architecture Approval Board,
which is chaired by the department's CIO. However, the enterprise
architecture has not been approved by the department's enterprise
architecture executive committee that was established in March 2011.
While the committee has agreed to approve future versions of the
enterprise architecture, it has not yet done so.
Core element: Core element 34: Key stakeholders have approved the
current version of subordinate architectures;
Satisfied: No;
Our analysis: According to officials, nine segment reference
architectures have been identified, of which three have been
prioritized for initial development. However, the core teams with
subject matter experts have not yet been formed and key leadership
positions, including lead architects, have not been designated to
develop these segments. Officials also stated that the review and
approval of subordinate architectures will likely follow existing
procedures, with the IGB providing the final approval.
Core element: Core element 35: Enterprise architecture is integral to
the execution of other institutional management disciplines;
Satisfied: Yes;
Our analysis: The DON enterprise architecture is linked to the
execution of the department's strategic planning, capital planning and
investment control, and system development and acquisition management.
For example, the enterprise architecture is identified as a mechanism
for achieving strategic goals and objectives. In addition, assessments
of enterprise architecture compliance are conducted during annual
reviews of the department's IT investments and modernization efforts.
Core element: Core element 36: Program office human capital needs are
met;
Satisfied: No;
Our analysis: According to officials and documentation, the department
has qualified but minimal staff that perform enterprise architecture
functions. However, officials stated that more staff are needed and
program resources do not allow for additional hiring. In addition, as
stated, the department has yet to develop a human capital plan that
would identify the knowledge, skills, and abilities needed to execute
the department's enterprise architecture program plans and schedules.
Core element: Core element 37: Initial versions of corporate "as-is"
and "to-be" enterprise architecture and sequencing plan exist;
Satisfied: Partial;
Our analysis: DON has developed initial versions of enterprise-level
architecture focused on a small set of artifacts based on existing
laws, regulations, policy, and guidance. However, the department's
enterprise architecture does not distinguish between current and
target products. DON also has developed a business transition plan
that identifies legacy business systems, migration systems, and core
systems. However, it has not developed an initial version of an
enterprise-level sequencing plan.
Core element: Core element 38: Initial version of corporate enterprise
architecture captures performance, business, data, services,
technology, and security views;
Satisfied: Partial;
Our analysis: DON's initial version of its enterprise-level
architecture captures aspects of business, data, services, technology,
and security. However, it does not address performance.
Core element: Core element 39: One or more segment and/or federation
member architectures exists and is being implemented;
Satisfied: Partial;
Our analysis: DON has developed segment architecture products. For
example, the department has developed segment architecture products
for its Net Centric architecture. However, it has not completed the
development of any segment architectures. In addition, the department
did not provide evidence that segment architectures are being
implemented.
Core element: Core element 40: Enterprise architecture product quality
is measured and reported;
Satisfied: Yes;
Our analysis: The quality of DON's enterprise architecture products is
assessed by the Independent Verification and Validation Working Group
in accordance with a set of criteria and submitted for final approval
to the DON Enterprise Architecture Approval Board.
Core element: Core element 41: Enterprise architecture results and
outcomes are measured and reported;
Satisfied: No;
Our analysis: DON has yet to measure and report enterprise
architecture results and outcomes. Officials stated that a lack of
best practices for measuring the value of enterprise architecture has
inhibited the department's ability to demonstrate return on investment
to enterprise-level executives.
Core element: Core element 42: Investment compliance with corporate
and subordinate architectures is measured and reported;
Satisfied: Yes;
Our analysis: DON provided evidence showing that compliance
assessments with its enterprise architecture are measured against
criteria described in the IT investment management process. Final
approval decisions are made by the DON CIO or Deputy CIOs and placed
in the system inventory. According to officials, compliance metrics
are reported to appropriate executive-level staff and available for
review in the Department of Defense's Information Technology Portfolio
Repository.
Core element: Core element 43: Subordinate architecture alignment with
the corporate enterprise architecture is measured and reported;
Satisfied: No;
Our analysis: According to officials, core teams will be established
to develop segment reference architectures. A responsibility of these
teams will be to produce metrics for measuring segment alignment with
the enterprise-level architecture. However, officials stated that
metrics for measuring and reporting alignment are currently not in
place.
Stage 5:
Core element: Core element 44: Organization head has approved current
version of the corporate enterprise architecture;
Satisfied: Yes;
Our analysis: The DON CIO has approved and released the current
version of the department's enterprise architecture. The CIO was
delegated responsibility for overseeing the development and
maintenance of the department's architecture in policy issued by the
Secretary of the Navy.
Core element: Core element 45: Organization component heads or segment
owners have approved current version of their respective subordinate
architectures;
Satisfied: No;
Our analysis: According to officials, a small number of segment
architecture artifacts are currently in place. These artifacts were
assessed against quality measures, submitted for approval, and
released in the same manner as the DON enterprise-level architecture.
However, officials stated that the enterprise architecture program is
not yet at a level where versions of the segment architectures have
been developed and approved.
Core element: Core element 46: Integrated repository tools and common
enterprise architecture framework and methodology are used across the
enterprise;
Satisfied: Partial;
Our analysis: DON has a portal that serves as a common repository for
its enterprise architecture products. However, DON has not fully
established a common enterprise architecture framework or methodology
to define how architectural products will be developed across the
enterprise.
Core element: Core element 47: Corporate and subordinate architecture
program offices operate as a single virtual office that shares
resources enterprisewide;
Satisfied: No;
Our analysis: DON has not established a formal enterprise-level
architecture program office and subordinate architecture program
offices;
therefore, entities are not in place that could operate as a single
virtual office that shares limited resources and follows common
policies and procedures.
Core element: Core element 48: Corporate enterprise architecture and
sequencing plan are enterprisewide in scope;
Satisfied: Partial;
Our analysis: DON's enterprise-level architecture is enterprisewide in
scope. However, DON has not developed an enterprise-level sequencing
plan.
Core element: Core element 49: Corporate enterprise architecture and
sequencing plan are aligned with subordinate architectures;
Satisfied: Partial;
Our analysis: DON has begun to demonstrate that it has aligned its
enterprise architecture with its segment reference architectures. For
example, it has aligned subordinate operational activities with its
enterprise architecture capabilities. However, the department does not
have subordinate architectures for some segments. In addition, not all
segment architecture products are aligned with the enterprise-level
architecture. Moreover, it has yet to develop its enterprise
sequencing plan.
Core element: Core element 50: All segment and/or federated
architectures exist and are horizontally and vertically integrated;
Satisfied: No;
Our analysis: DON has not developed all segment and/or federated
architectures. In addition, it has yet to provide evidence that
segment and/or federated architectures are horizontally and vertically
integrated.
Core element: Core element 51: Corporate and subordinate architectures
are extended to align with external partner architectures;
Satisfied: Partial;
Our analysis: DON has begun to demonstrate that its enterprise-level
and subordinate architectures are extended to align with external
partner architectures. For example, its enterprise architecture
approach is aligned with the Joint Staff's Joint Capability Areas.
However, DON did not provide evidence that its enterprise architecture
aligns with other external partner architectures (e.g., Army, Air
Force).
Core element: Core element 52: Enterprise architecture products and
management processes are subject to independent assessment;
Satisfied: No;
Our analysis: While enterprise architecture products have undergone
verification and validation assessments, they were not conducted by an
independent body. Moreover, enterprise architecture management
processes have not been subject to independent verification and
validation.
Stage 6:
Core element: Core element 53: Enterprise architecture is used by
executive leadership to inform organization strategic planning and
policy formulation;
Satisfied: Partial;
Our analysis: DON officials demonstrated that the enterprise
architecture is informing an update to a department policy to ensure
that open source software requirements are adequately addressed.
Officials stated that the department's strategic plan is being updated
and will identify the department's enterprise architecture as a
mechanism for achieving each of the department's goals and objectives.
However, the updated strategic plan was not available for review in
order to verify the department's assertions.
Core element: Core element 54: Enterprise architecture human capital
capabilities are continuously improved;
Satisfied: No;
Our analysis: DON does not have a human capital plan in place that
identifies the enterprise architecture human capital capabilities that
are needed as well as an approach for addressing capability gaps.
Thus, the department currently lacks a foundational document that is
needed for continuous improvement of enterprise architecture human
capital capabilities.
Core element: Core element 55: Enterprise architecture methodologies
and tools are continuously improved;
Satisfied: Partial;
Our analysis: DON has automated enterprise architecture tools in place
but has not established an enterprise architecture development and
maintenance methodology. According to officials, regular reviews and
improvements are made to the department's enterprise architecture
tools that are generally based on lessons learned from using the
architecture as well as yearly reviews of the department's systems.
Core element: Core element 56: Enterprise architecture management
processes are continuously improved and reflect the results of
external assessments;
Satisfied: No;
Our analysis: DON has yet to subject its enterprise architecture
management processes to periodic reassessments against relevant
benchmarks and guidance and identify the gaps that need to be
addressed.
Core element: Core element 57: Enterprise architecture products are
continuously improved and updated;
Satisfied: Partial;
Our analysis: According to officials, the department's enterprise
architecture products have been improved, updated, and released as new
versions to reflect events such as changes in legal requirements,
emerging technologies, and governmentwide priorities. DON has also
developed a formal configuration management plan that includes a
change control process;
however, a process has not been formalized for conducting
configuration audits and reviews to ensure that only approved changes
are made to products and to maintain the integrity of the
configuration baselines.
Core element: Core element 58: Enterprise architecture quality and
results measurement methods are continuously improved;
Satisfied: No;
Our analysis: According to officials, the department is in the early
stages of developing metrics to measure enterprise architecture
quality and results and has yet to measure and report enterprise
architecture results and outcomes.
Core element: Core element 59: Enterprise architecture continuous
improvement efforts reflect the results of external assessments;
Satisfied: Yes;
Our analysis: Our 2008 assessment of the department's enterprise
architecture program has been leveraged to make program capability and
product improvements. Areas in which DON's improvement efforts reflect
the results of our assessment include establishing a formalized
enterprise architecture governance structure, a policy for enterprise
architecture development and maintenance, an IT investment process
that includes compliance assessments with DON's architecture, and a
set of criteria for measuring the quality of its products.
Source: GAO analysis of information provided by the DON.
[End of table]
[End of section]
Appendix VI: Comments from the Department of Defense:
Department Of Defense:
Chief Information Officer:
6000 Defense Pentagon:
Washington, D.C. 20301-6000:
September 13, 2011:
Ms. Valerie C. Melvin:
Director, Information Management and Human Capital Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Ms. Melvin:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-11-902, 'Organizational Transformation: Military
Departments Can Improve Their Enterprise Architecture Programs,' dated
August 22, 2011 (GAO Code 310956).
The Department appreciates the opportunity to comment. We partially
concur with GAO recommendation. Ow' rationale for our partial
concurrence is provided.
Signed by:
Teresa M. Takai:
Attachments: As stated.
[End of letter]
GAO Draft Report Dated August 22, 2011:
GAO-11-902 (GAO Code 310956):
"Organizational Transformation: Military Departments Can Improve Their
Enterprise Architecture Programs"
Department Of Defense Comments To The GAO Recommendations:
Recommendation: To ensure that the military departments establish
commitments to fully develop and effectively manage their enterprise
architectures, GAO recommends that the Secretaries of the Air Force,
Army, and Navy each expeditiously provide to the congressional defense
committees a plan that identifies milestones for their respective
department's full satisfaction of all of GAO's Enterprise Architecture
Management Maturity Framework elements. In the event that a military
department does not intend to fully satisfy all elements of GAO's
framework, the plan should include a rationale for why the military
department deems any such element(s) to be not applicable.
DoD Response: The Department of Defense partially concurs with this
recommendation. The Department of Defense Chief Information Officer
and the Department of the Army Chief Information Officer concur with
the recommendation. The Department of the Air Force (USAF) Chief
Information Officer and Department of the Navy (DON) Chief Information
Officer non-concur. Both the USAF and the DON believe that the GAO's
Enterprise Architecture Management Maturity Framework (EAMMF) provides
a very comprehensive set of elements associated with the development
and implementation of a robust enterprise architecture program for a
Federal Agency or Organization. Both the USAF and the DON, however, do
not have a valid business case that would justify the implementation
of all 59 elements of the EAMMF. In today's fiscally constrained
environment, both have chosen to gradually implement selected elements
of the EAMMF which are most useful in implementing optimized, secure,
and cost effective Information Technology systems and capabilities.
[End of section]
Appendix VII: GAO Contact and Staff Acknowledgments:
GAO Contact:
Valerie C. Melvin, (202) 512-6304 or melvinv@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Neelaxi Lakhmani and Mark
Bird, Assistant Directors; Debra Conner; Shaun Byrnes; Elena Epps;
Nancy Glover; Michael Holland; Anh Le; Lee McCracken; and Donald
Sebers made key contributions to this report.
[End of section]
Footnotes:
[1] DON includes both the Navy and the Marine Corps.
[2] This data reflects the total number of systems (IT and National
Security Systems) documented in DOD's Defense IT Portfolio Repository
system as of March 1, 2011.
[3] An enterprise architecture is a modernization blueprint that
describes the organization's current and desired state for its
business operations and supporting IT systems in both logical and
technical terms, and contains a plan for transitioning between the two
states.
[4] GAO, DOD Business Systems Modernization: Military Departments Need
to Strengthen Management of Enterprise Architecture Programs,
[hyperlink, http://www.gao.gov/products/GAO-08-519] (Washington, D.C.:
May 2008).
[5] GAO, Organizational Transformation: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 2.0),
[hyperlink, http://www.gao.gov/products/GAO-10-846G] (Washington,
D.C.: August 2010).
[6] GAO, Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, [hyperlink,
http://www.gao.gov/products/GAO-06-658] (Washington, D.C.: May 15,
2006).
[7] GAO, High-Risk Series: An Update, [hyperlink,
http://www.gao.gov/products/GAO-11-278] (Washington, D.C.: February
2011).
[8] These seven high-risk areas include DOD's overall approach to
business transformation, business systems modernization, contract
management, financial management, supply chain management, support
infrastructure management, and weapon systems acquisition.
[9] The seven governmentwide high-risk areas include disability
programs, protecting information systems and critical infrastructure,
interagency contracting, information systems and critical
infrastructure, information sharing for homeland security, human
capital, and real property.
[10] GAO, Federal Aviation Administration: Stronger Architecture
Program Needed to Guide Systems Modernization Efforts, [hyperlink,
http://www.gao.gov/products/GAO-05-266] (Washington, D.C.: Apr. 29,
2005); Homeland Security: Efforts Under Way to Develop Enterprise
Architecture, but Much Work Remains, [hyperlink,
http://www.gao.gov/products/GAO-04-777] (Washington, D.C.: Aug. 6,
2004); GAO-04-731R; Information Technology: Architecture Needed to
Guide NASA's Financial Management Modernization, [hyperlink,
http://www.gao.gov/products/GAO-04-43] (Washington, D.C.: Nov. 21,
2003); [hyperlink, http://www.gao.gov/products/GAO-03-1018]; Business
Systems Modernization: Summary of GAO's Assessment of the Department
of Defense's Initial Business Enterprise Architecture, [hyperlink,
http://www.gao.gov/products/GAO-03-877R] (Washington,, D.C.; July 7,
2003); Information Technology: DLA Should Strengthen Business Systems
Modernization Architecture and Investment Activities, [hyperlink,
http://www.gao.gov/products/GAO-01-631] (Washington, D.C.: June 29,
2001); and Information Technology: INS Needs to Better Manage the
Development of Its Enterprise Architecture, [hyperlink,
http://www.gao.gov/products/GAO/AIMD-00-212] (Washington, D.C.: Aug.
1, 2000).
[11] Clinger-Cohen Act, 40 U.S.C. § 11315(b)(2); E-Government Act, 44
U.S.C. § 3602(f)(14); Chief Information Officers Council, A Practical
Guide to Federal Enterprise Architecture, Version 1.0 (February 2001);
OMB, Information Technology Architectures, Memorandum M-97-16 (June
18, 1997), rescinded with the update of OMB Circular A-130 (Nov. 30,
2000); Improving Agency Performance Using Information and Information
Technology (Enterprise Architecture Assessment Framework v3.1) (June
2009).
[12] 10 U.S.C. § 2222(c)(1).
[13] Duncan Hunter National Defense Authorization Act for Fiscal Year
2009, Pub. L. No. 110-417, § 908(b)(2).
[14] GAO, Information Technology: Enterprise Architecture Use across
the Federal Government Can Be Improved, [hyperlink,
http://www.gao.gov/products/GAO-02-6] (Washington, D.C.: Feb. 19,
2002).
[15] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (version 1.1),
[hyperlink, http://www.gao.gov/products/GAO-03-584G] (Washington,
D.C.: April 2003).
[16] [hyperlink, http://www.gao.gov/products/GAO-10-846G].
[17] Chief "X" officer is a generic term for job titles where "X"
represents a specific specialized position that serves the entire
organization, such as the chief information officer, chief financial
officer, chief human capital officer, chief procurement officer, chief
performance officer, chief technology officer, chief information
security officer, or chief management officer.
[18] [hyperlink, http://www.gao.gov/products/GAO-02-6]; GAO,
Information Technology: Leadership Remains Key to Agencies Making
Progress on Enterprise Architecture Efforts, [hyperlink,
http://www.gao.gov/products/GAO-04-40] (Washington, D.C.: Nov. 17,
2003).
[19] The E-Government Act of 2002 assigns OMB responsibility for
overseeing the development of enterprise architectures within and
across agencies.
[20] GAO, Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, [hyperlink, http://www.gao.gov/products/GAO-06-831]
(Washington, D.C.: Aug. 14, 2006).
[21] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. § 2222).
[22] The act required DOD to, among other things, develop an
enterprise architecture to cover all defense business systems and
their related functions and activities. The act further required that
the Secretary of Defense submit an annual report to congressional
defense committees on DOD's compliance with certain requirements of
the act not later than March 15 of each year. Additionally, the act
directed us to submit to these congressional committees--within 60
days of DOD's report submission--an assessment of the department's
actions to comply with these requirements.
[23] GAO, DOD Business Systems Modernization: Progress in Establishing
Corporate Management Controls Needs to Be Replicated Within Military
Departments, [hyperlink, http://www.gao.gov/products/GAO-08-705]
(Washington, D.C.: May 15, 2008); DOD Business Systems Modernization:
Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed, [hyperlink,
http://www.gao.gov/products/GAO-07-733] (Washington, D.C.: May 14,
2007); [hyperlink, http://www.gao.gov/products/GAO-06-658]; and DOD
Business Systems Modernization: Important Progress Made in
Establishing Foundational Architecture Products and Investment
Management Practices, but Much Work Remains, [hyperlink,
http://www.gao.gov/products/GAO-06-219] (Washington, D.C.: Nov. 23,
2005).
[24] [hyperlink, http://www.gao.gov/products/GAO-08-519].
[25] GAO, Opportunities to Reduce Potential Duplication in Government
Programs, Save Tax Dollars, and Enhance Revenue, [hyperlink,
http://www.gao.gov/products/GAO-11-318SP] (Washington, D.C.: Mar. 1,
2011).
[26] Due to the large and complex nature of the DOD enterprise, we
determined that all 59 elements of the framework apply to the military
department enterprise architecture programs.
[27] See, for example, [hyperlink,
http://www.gao.gov/products/GAO-10-846G], [hyperlink,
http://www.gao.gov/products/GAO-06-831], [hyperlink,
http://www.gao.gov/products/GAO-04-40], [hyperlink,
http://www.gao.gov/products/GAO-03-584G], [hyperlink,
http://www.gao.gov/products/GAO-02-6]; Chief Information Officers
Council, A Practical Guide to Federal Enterprise Architecture, Version
1.0 (February 2001).
[28] According to Army officials, the Network segment defines
technologies, information sharing, and transport capabilities.
[29] The Space Domain is to address operational capabilities and
systems that enable space related system functions and the interaction
of those systems with corresponding air and ground systems. According
to Army officials, the Network segment defines technologies,
information sharing, and transport capabilities and the Generating
Force segment addresses the business operations needed to train, equip
and sustain the operational forces. DON's Net Centric segment is to
address human and technical connectivity and interoperability.
[30] See, for example, [hyperlink,
http://www.gao.gov/products/GAO-10-846G].
[31] See, for example, [hyperlink,
http://www.gao.gov/products/GAO-10-846G].
[32] GAO, Department of Defense: Further Actions Needed to
Institutionalize Key Business System Modernization Management
Controls, [hyperlink, http://www.gao.gov/products/GAO-11-684]
(Washington, D.C.: June 29, 2011).
[33] The Information Enterprise Architecture is to describe the
information, information resources, assets, and processes required to
share information across the Department and with mission partners.
[34] The Joint Capability Areas represent collections of similar DOD
functions that are intended to support, among other things, investment
decision making, portfolio management, and capabilities-based planning.
[35] [hyperlink, http://www.gao.gov/products/GAO-10-846G].
[36] In May 2010, the Secretary of Defense announced the need for DOD
to reduce overhead costs and subsequently called for a 5-year effort
to cut $100 billion from the department's budget in order to finance
sustainment of the current force and modernize its weapons portfolio.
[37] [hyperlink, http://www.gao.gov/products/GAO-06-831].
[38] GAO, Organizational Transformation: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 2.0),
[hyperlink, http://www.gao.gov/products/GAO-10-846G] (Washington,
D.C.: August 2010).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
