DOD Business Systems Modernization
Progress Continues to Be Made in Establishing Corporate Management Controls, but Further Steps Are Needed Gao ID: GAO-07-733 May 14, 2007In 1995, GAO first designated the Department of Defense's (DOD) business systems modernization program as "high risk," and GAO continues to do so today. To assist in addressing this high-risk area, the Fiscal Year 2005 National Defense Authorization Act contains provisions that are consistent with prior GAO recommendations. Further, the act requires the department to submit annual reports to its congressional committees on its compliance with these provisions and it directs GAO to review each report. In response, GAO assessed DOD's actions to address (1) requirements in the act and (2) GAO's recommendations that it reported as open in its prior annual report under the act. In doing so, GAO reviewed documentation and interviewed officials relative to the act and related guidance.
As part of DOD's recent efforts to strengthen management of its business systems modernization program, it has taken steps over the last year to build on past efforts and further comply with the act's requirements and relevant guidance. However, additional steps are needed. For example, the latest version of DOD's business enterprise architecture now contains information about the department's "As Is" corporate environment, which is important for effective transition planning. Further, this version represents a major step in building the family of architectures that are needed to fully satisfy the act and effectively guide and constrain thousands of system investments across all DOD component organizations. Nevertheless, GAO's reports since its last annual report under the act show that the strategy for extending the business enterprise architecture to defense components needs further definition to make it executable and the maturity of key components' architecture programs is limited. GAO has recently made recommendations to address these challenges. The updated enterprise transition plan, which is an essential component of an enterprise architecture, continues to identify systems and initiatives that are to fill business capability gaps and address DOD-wide and component business priorities contained in the business enterprise architecture. However, it does not include investments for all components and does not reflect key factors associated with properly sequencing planned investments, such as dependencies among investments and the capability to execute the plan, which GAO's existing recommendations provide for addressing. DOD has established and begun implementing the investment review structures and processes that are consistent with the act. However, it has yet to do so in a manner that is consistent with relevant guidance. In particular, it has yet to fully define the related policies and procedures needed to effectively execute both project-level and portfolio-based information technology investment management practices. GAO has recently made recommendations to address these shortcomings. DOD also continues to make progress in implementing GAO recommendations aimed at strengthening business systems modernization management. In particular, of the 14 open recommendations that GAO identified in its prior annual report under the act, 10 have either been largely implemented or subsumed by the more recent recommendations cited above. For example, DOD has implemented GAO's recommendations aimed at effectively using the assessments that have been performed by DOD's independent verification and validation contractor. Such assessments provide important information for department and congressional oversight bodies to use to better ensure the definition and institutionalization of the corporate management controls that GAO has cited as essential to addressing the DOD business systems modernization high-risk area. The department's annual reports have not included such assessments.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-733, DOD Business Systems Modernization: Progress Continues to Be Made in Establishing Corporate Management Controls, but Further Steps Are Needed
This is the accessible text file for GAO report number GAO-07-733
entitled 'DOD Business Systems Modernization: Progress Continues to Be
Made in Establishing Corporate Management Controls, but Further Steps
Are Needed' which was released on May 14, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
May 2007:
DOD Business Systems Modernization:
Progress Continues to Be Made in Establishing Corporate Management
Controls, but Further Steps Are Needed:
GAO-07-733:
GAO Highlights:
Highlights of GAO-07-733, a report to congressional committees
Why GAO Did This Study:
In 1995, GAO first designated the Department of Defense‘s (DOD)
business systems modernization program as ’high risk,“ and GAO
continues to do so today. To assist in addressing this high-risk area,
the Fiscal Year 2005 National Defense Authorization Act contains
provisions that are consistent with prior GAO recommendations. Further,
the act requires the department to submit annual reports to its
congressional committees on its compliance with these provisions and it
directs GAO to review each report. In response, GAO assessed DOD‘s
actions to address (1) requirements in the act and (2) GAO‘s
recommendations that it reported as open in its prior annual report
under the act. In doing so, GAO reviewed documentation and interviewed
officials relative to the act and related guidance.
What GAO Found:
As part of DOD‘s recent efforts to strengthen management of its
business systems modernization program, it has taken steps over the
last year to build on past efforts and further comply with the act‘s
requirements and relevant guidance. However, additional steps are
needed. For example,
* The latest version of DOD‘s business enterprise architecture now
contains information about the department‘s ’As Is“ corporate
environment, which is important for effective transition planning.
Further, this version represents a major step in building the family of
architectures that are needed to fully satisfy the act and effectively
guide and constrain thousands of system investments across all DOD
component organizations. Nevertheless, GAO‘s reports since its last
annual report under the act show that the strategy for extending the
business enterprise architecture to defense components needs further
definition to make it executable and the maturity of key components‘
architecture programs is limited. GAO has recently made recommendations
to address these challenges.
* The updated enterprise transition plan, which is an essential
component of an enterprise architecture, continues to identify systems
and initiatives that are to fill business capability gaps and address
DOD-wide and component business priorities contained in the business
enterprise architecture. However, it does not include investments for
all components and does not reflect key factors associated with
properly sequencing planned investments, such as dependencies among
investments and the capability to execute the plan, which GAO‘s
existing recommendations provide for addressing.
* DOD has established and begun implementing the investment review
structures and processes that are consistent with the act. However, it
has yet to do so in a manner that is consistent with relevant guidance.
In particular, it has yet to fully define the related policies and
procedures needed to effectively execute both project-level and
portfolio-based information technology investment management practices.
GAO has recently made recommendations to address these shortcomings.
DOD also continues to make progress in implementing GAO recommendations
aimed at strengthening business systems modernization management. In
particular, of the 14 open recommendations that GAO identified in its
prior annual report under the act, 10 have either been largely
implemented or subsumed by the more recent recommendations cited above.
For example, DOD has implemented GAO‘s recommendations aimed at
effectively using the assessments that have been performed by DOD‘s
independent verification and validation contractor. Such assessments
provide important information for department and congressional
oversight bodies to use to better ensure the definition and
institutionalization of the corporate management controls that GAO has
cited as essential to addressing the DOD business systems modernization
high-risk area. The department‘s annual reports have not included such
assessments.
What GAO Recommends:
GAO is recommending that future DOD annual reports include an
assessment by its independent verification and validation agent of the
quality of the department‘s federated family of architectures,
including the associated transition plan(s). In written comments, DOD
agreed with GAO‘s recommendation.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-733].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD Is Continuing to Improve Its Approach to Modernizing Business
Systems:
DOD Continues to Implement Our Prior Recommendations:
Conclusions:
Recommendation for Executive Action:
Agency Comments:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Status of Prior Recommendations Identified as Open in
GAO's Prior Annual Report under the Act:
Appendix III: Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management:
Appendix IV: Comments from the Department of Defense:
Appendix V: GAO Contacts and Staff Acknowledgments:
Table:
Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition:
Figures:
Figure 1: Simplified DOD Organizational Structure:
Figure 2: The Five ITIM Stages of Maturity with Critical Processes:
Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture:
Abbreviations:
ASD(NII)/CIO: Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer:
BEA: business enterprise architecture:
BEP: business enterprise priority:
BTA: Business Transformation Agency:
CIO: chief information officer:
DBSMC: Defense Business Systems Management Committee:
DOD: Department of Defense:
ETP: enterprise transition plan:
IRB: Investment Review Board:
IT: information technology:
ITIM: Information Technology Investment Management framework:
NCES: Net-Centric Enterprise Services:
OMB: Office of Management and Budget:
SOA: service-oriented architecture:
USD(AT&L): Under Secretary of Defense (Acquisition, Technology, and
Logistics):
United States Government Accountability Office:
Washington, DC 20548:
May 14, 2007:
Congressional Committees:
For decades, the Department of Defense (DOD) has been challenged in
modernizing its timeworn business systems.[Footnote 1] In 1995, we
designated DOD's business systems modernization program as high risk,
and we continue to designate it as such today.[Footnote 2] As our
research on public and private sector organizations shows, two
essential ingredients to a successful systems modernization program are
having a well-defined enterprise architecture[Footnote 3] and an
effective institutional approach to managing information technology
(IT) investments.
Accordingly, we made recommendations to the Secretary of Defense in May
2001 that included the means for effectively developing an enterprise
architecture and establishing a corporate approach to investment
control and decision making.[Footnote 4] Between 2001 and 2005, we
reported that the department's business systems modernization program
continued to lack both of these, concluding in 2005 that hundreds of
millions of dollars had been spent on a business enterprise
architecture (BEA) and investment management structures that had
limited use.[Footnote 5] Accordingly, we made more explicit
architecture and investment-related recommendations.
To assist DOD in addressing these modernization management challenges,
Congress included provisions in the Ronald W. Reagan National Defense
Authorization Act for Fiscal Year 2005[Footnote 6] that were consistent
with our recommendations. More specifically, the act required the
department to, among other things, (1) develop a BEA, (2) develop a
transition plan to implement the architecture, (3) include systems
information in its annual budget submission, (4) establish a system
investment approval and accountability structure, (5) establish an
investment review process, and (6) approve and certify any system
modernizations costing in excess of $1 million. The act further
requires that the Secretary of Defense submit an annual report to
congressional defense committees on DOD's compliance with certain
requirements of the act not later than March 15 of each year from 2005
through 2009. Additionally, the act directs us to submit--within 60
days of DOD's report submission--to congressional defense committees an
assessment of the actions taken to comply with these requirements.
As agreed with your offices, the objectives of our review were to (1)
assess the actions taken by DOD to comply with requirements of section
2222 of Title 10, U.S. Code, and (2) determine the extent DOD has
addressed our prior open recommendations for institutionalizing key
business system modernization management controls. To accomplish this,
we used our prior annual report under the act[Footnote 7] as a
baseline, analyzing whether the department had taken actions to comply
with those provisions of the act, related guidance, and the prior
recommendations that we had identified in our prior annual report as
not yet addressed. In doing this, we also relied on the results of
relevant reports that we have issued since our prior annual
report.[Footnote 8] We performed our work at DOD headquarters in
Arlington, Virginia, from March through May 2007 in accordance with
generally accepted government auditing standards. Details on our
objectives, scope, and methodology are contained in appendix I.
Results in Brief:
DOD continues to take steps to comply with legislative requirements and
related guidance pertaining to its business systems modernization high
risk area. In particular, on March 15, 2007, DOD released a new version
of its BEA, developed an updated enterprise transition plan, and issued
its annual report to Congress describing steps taken and planned
relative to the act's requirements, among other things. The steps
address several of the missing elements that we previously identified
relative to the legislative provisions and related best practices
concerning the BEA, enterprise transition plan, budgetary disclosure,
investment management, and reviews of systems costing in excess of $1
million. However, additional steps are needed to fully comply with the
act and relevant guidance. For example:
* The latest version of the BEA now contains information about the
department's "As Is" corporate environment for some enterprise priority
areas (e.g., Financial Visibility), which is important to support the
business capability gap analyses needed for transition planning;
however, it does not do this for all priority areas (e.g., Acquisition
Visibility). Moreover, while the latest version's focus on DOD-wide,
corporate policies, capabilities, rules, and standards is an essential
element to meeting the act's requirements, this version has yet to be
augmented by the DOD component organizations' subsidiary architectures
that are also essential to meeting the act's requirements and the
department's goal of having a federated family of architectures.
Compounding this are our recent reports showing the military
departments' architecture programs are not mature and the strategy that
the department has developed for federating its BEA needs more
definition to be executable.[Footnote 9] To address these limitations,
our recent reports contain additional recommendations. Once these
limitations are addressed, the architecture should provide a more
sufficient frame of reference to optimally guide and constrain DOD-wide
system investments.
* The updated transition plan continues to identify more systems and
initiatives that are to fill business capability gaps and address DOD-
wide and component business priorities and continues to provide a range
of information for each system and initiative in the plan (e.g., budget
information, performance metrics, and milestones). Further, the updated
plan also identifies legacy systems that will not be part of its target
environment. However, this latest transition plan still does not
include system investment information for all the defense agencies and
combatant commands. Moreover, the plan does not sequence the planned
investments based on a range of relevant factors, such as technology
opportunities, marketplace trends, institutional system development and
acquisition capabilities, legacy and new system dependencies and life
expectancies, and the projected value of competing investments.
According to DOD officials, they intend to address such limitations in
future versions of the transition plan. We have an existing
recommendation to the department to formalize its plans for
incrementally evolving the transition plan. Once these limitations in
the department's transition plan(s) are addressed, it will be better
positioned to effectively and efficiently migrate to a more modernized
systems environment.
* The department's fiscal year 2008 budget submission provides a range
of information on business systems, including types of information
cited in the act, such as system name, designated approval authority,
and funding to be used for development/modernization versus operations/
maintenance.
* While the department has established and begun implementing the
investment review structures and processes that are consistent with the
act, it has yet to do so in a manner that is consistent with relevant
guidance. As we recently reported,[Footnote 10] the department has yet
to fully define the related policies and procedures needed to
effectively execute both project-level and portfolio-based IT
investment management practices. For example, DOD had established an
enterprisewide IT investment board responsible for defining and
implementing its business system investment governance process, but it
had not fully defined the policies and procedures needed for oversight
of and visibility into operations and maintenance investments. To
address these investment management weaknesses, our recent report
contains additional recommendations. Once these policies and procedures
are fully defined, the risk of projects and portfolios of projects
being inconsistently and improperly selected and controlled will be
reduced, thus increasing the chances of investments meeting mission
needs in the most cost-effective manner.
* The department continues to review and approve business systems as
directed by the act. As of March 2007, the department reported that its
highest investment review body had approved 285 systems. However, the
military departments' review and approval processes are still evolving,
according to Air Force, Army, and Navy officials, and additional work
is needed to mature them. Because of the importance of the military
departments' investment management structures and processes, we have
ongoing work to determine the extent to which the Air Force and the
Navy are employing relevant investment management guidance.
In concert with the department's efforts to comply with the act, it has
also largely implemented, or our recommendations in recent reports have
otherwise subsumed, 10 of the 14 recommendations that we identified as
open in our prior annual report under the act. For example, DOD has
implemented our recommendation aimed at effectively using the results
of the BEA independent verification and validation contractor on prior
versions of the architecture. Use of an independent verification and
validation agent is an architecture management best practice for
identifying architecture strengths and weaknesses and disclosing to
department and congressional oversight bodies the information they need
to better ensure that DOD's family of architectures and associated
transition plan(s) satisfy key quality parameters. According to
department officials, they are committed to addressing all of our open
recommendations, and have actions under way and plans in place to
address the remaining 4.
To facilitate congressional oversight and promote departmental
accountability, we are recommending that the department include in its
future annual reports under the act the results of its independent
verification and validation agent's assessment of the extent to which
the department's federated family of its corporate and component
architectures, including the related transition plan(s), are complete,
consistent, understandable, and usable. The department has not included
such information in its annual reports. In written comments on a draft
of this report, signed by the Deputy Under Secretary of Defense
(Business Transformation) and reprinted in appendix IV, the department
agreed with our recommendation.
Background:
DOD is a massive and complex organization. To illustrate, the
department reported that its fiscal year 2006 operations involved
approximately $1.4 trillion in assets and $2.0 trillion in liabilities;
more than 2.9 million in military and civilian personnel; and $581
billion in net cost of operations. To date, for fiscal year 2007, the
department received appropriations of about $501 billion.
Organizationally, the department includes the Office of the Secretary
of Defense, the Chairman of the Joint Chiefs of Staff, the military
departments, numerous defense agencies and field activities; and
various unified combatant commands that are either responsible for
specific geographic regions or specific functions. (See fig. 1 for a
simplified depiction of DOD's organizational structure.)
Figure 1: Simplified DOD Organizational Structure:
[See PDF for image]
Source; GAO based on DOD documentation.
[A] The Chairman of the Joint Chiefs of Staff serves as the spokesman
for the commanders of the combatant commands, especially on the
administrative requirements of the commands.
[End of figure]
In support of its military operations, the department performs an
assortment of interrelated and interdependent business functions,
including logistics management, procurement, health care management,
and financial management. As we have previously reported,[Footnote 11]
the DOD systems environment that supports these business functions is
overly complex and error prone, and is characterized by (1) little
standardization across the department, (2) multiple systems performing
the same tasks, (3) the same data stored in multiple systems, and (4)
the need for data to be entered manually into multiple systems.
Moreover, DOD recently reported that this systems environment is
comprised of approximately 3,100 separate business systems. For fiscal
year 2007, Congress appropriated approximately $15.7 billion to DOD,
and for fiscal year 2008, DOD has requested about $15.9 billion in
appropriated funds to operate, maintain, and modernize these business
systems and associated infrastructure.
As we have previously reported,[Footnote 12] the department's
nonintegrated and duplicative systems impair DOD's ability to combat
fraud, waste, and abuse. In fact, DOD currently bears responsibility,
in whole or in part, for 15 of our 27 high-risk areas.[Footnote 13]
Eight of these areas are specific to DOD[Footnote 14] and the
department shares responsibility for 7 other governmentwide high-risk
areas.[Footnote 15] DOD's business systems modernization is one of the
high-risk areas, and it is an essential enabler to addressing many of
the department's other high-risk areas. For example, modernized
business systems are integral to the department's efforts to address
its financial, supply chain, and information security management high-
risk areas.
Enterprise Architecture and IT Investment Management Controls Are
Critical to Achieving Successful Systems Modernization:
Effective use of an enterprise architecture--a modernization blueprint-
-is a hallmark of successful public and private organizations. For more
than a decade, we have promoted the use of architectures to guide and
constrain systems modernization, recognizing them as a crucial means to
this challenging goal: optimally defined operational and technological
environments. Congress, the Office of Management and Budget (OMB), and
the federal Chief Information Officer's (CIO) Council have also
recognized the importance of an architecture-centric approach to
modernization. The Clinger-Cohen Act of 1996[Footnote 16] mandates that
an agency's CIO develop, maintain, and facilitate the implementation of
an information technology architecture. Further, the E-Government Act
of 2002[Footnote 17] requires OMB to oversee the development of
enterprise architectures within and across agencies. In addition, we,
OMB, and the CIO Council have issued guidance that emphasizes the need
for system investments to be consistent with these
architectures.[Footnote 18]
A corporate approach to IT investment management is characteristic of
successful public and private organizations. Recognizing this, Congress
enacted the Clinger-Cohen Act of 1996,[Footnote 19] which requires OMB
to establish processes to analyze, track, and evaluate the risks and
results of major capital investments in IT systems made by executive
agencies.[Footnote 20] In response to the Clinger-Cohen Act and other
statutes, OMB has developed policy and issued guidance for planning,
budgeting, acquisition, and management of federal capital
assets.[Footnote 21] We have also issued guidance in this
area,[Footnote 22] which defines institutional structures, such as
Investment Review Boards (IRB), processes for developing information on
investments (such as costs and benefits), and practices to inform
management decisions (such as whether a given investment is aligned
with an enterprise architecture).
Enterprise Architecture: A Brief Description:
An enterprise architecture provides a clear and comprehensive picture
of an entity, whether it is an organization (e.g., a federal
department) or a functional or mission area that cuts across more than
one organization (e.g., financial management). This picture consists of
snapshots of both the enterprise's current ("As Is") environment and
its target ("To Be") environment. These snapshots consist of "views,"
which are one or more interdependent and interrelated architecture
products (e.g., models, diagrams, matrices, and text) that provide
logical or technical representations of the enterprise. The
architecture also includes a transition or sequencing plan, which is
based on an analysis of the gaps between the "As Is" and "To Be"
environments; this plan provides a temporal road map for moving between
the two environments and incorporates such considerations as technology
opportunities, marketplace trends, fiscal and budgetary constraints,
institutional system development and acquisition capabilities, legacy
and new system dependencies and life expectancies, and the projected
value of competing investments.
The suite of products produced for a given entity's enterprise
architecture, including its structure and content, is largely governed
by the framework used to develop the architecture. Since the 1980s,
various architecture frameworks have been developed, such as John A.
Zachman's "A Framework for Information Systems Architecture"[Footnote
23] and the DOD Architecture Framework.[Footnote 24]
The importance of developing, implementing, and maintaining an
enterprise architecture is a basic tenet of both organizational
transformation and systems modernization. Managed properly, an
enterprise architecture can clarify and help optimize the
interdependencies and relationships among an organization's business
operations (and the underlying IT infrastructure and applications) that
support these operations. Moreover, when an enterprise architecture is
employed in concert with other important management controls, such as
portfolio-based capital planning and investment control practices,
architectures can greatly increase the chances that an organization's
operational and IT environments will be configured to optimize mission
performance. Our experience with federal agencies has shown that
investing in IT without defining these investments in the context of an
architecture often results in systems that are duplicative, not well
integrated, and unnecessarily costly to maintain and
interface.[Footnote 25]
One approach to structuring an enterprise architecture is referred to
as a federated enterprise architecture. Such a structure treats the
architecture as a family of coherent but distinct member architectures
that conform to an overarching architectural view and rule set. This
approach recognizes that each member of the federation has unique goals
and needs as well as common roles and responsibilities with the levels
above and below it. Under a federated approach, member architectures
are substantially autonomous, although they also inherit certain rules,
policies, procedures, and services from higher-level architectures. As
such, a federated architecture enables component organization autonomy
while ensuring enterprisewide linkages and alignment where appropriate.
Where commonality among components exists, there are also opportunities
for identifying and leveraging shared services.
A service-oriented architecture (SOA) is an approach for sharing
business capabilities across the enterprise by designing functions and
applications as discrete, reusable, and business-oriented services. As
such, service orientation permits sharing capabilities that may be
under the control of different component organizations. As we have
previously reported,[Footnote 26] such capabilities or services need to
be, among other things, (1) self-contained, meaning that they do not
depend on any other functions or applications to execute a discrete
unit of work; (2) published and exposed as self-describing business
capabilities that can be accessed and used; and (3) subscribed to via
well-defined and standardized interfaces. A SOA approach is thus not
only intended to reduce redundancy and increase integration, but also
to provide the kind of flexibility needed to support a quicker response
to changing and evolving business requirements and emerging conditions.
IT Investment Management: A Brief Description:
IT investment management is a process for linking IT investment
decisions to an organization's strategic objectives and business plans
that focuses on selecting, controlling, and evaluating investments in a
manner that minimize risks while maximizing the return of
investment.[Footnote 27]
* During the selection phase, the organization (1) identifies and
analyzes each project's risks and returns before committing significant
funds to any project and (2) selects those IT projects that will best
support its mission needs.
* During the control phase, the organization ensures that, as projects
develop and investment expenditures continue, they continue to meet
mission needs at the expected levels of cost and risk. If the project
is not meeting expectations or if problems arise, steps are quickly
taken to address the deficiencies.
* During the evaluation phase, actual versus expected results are
compared once a project has been fully implemented. This is done to (1)
assess the project's impact on mission performance, (2) identify any
changes or modifications to the project that may be needed, and (3)
revise the investment management process based on lessons learned.
Consistent with this guidance, our IT Investment Management framework
(ITIM)[Footnote 28] consists of five progressive stages of maturity for
any given agency relative to selecting, controlling, and evaluating its
investment management capabilities. (See fig. 2 for the five ITIM
stages of maturity.) Stage 2 critical processes lay the foundation by
establishing successful, predictable, and repeatable investment control
processes at the project level. Stage 3 is where the agency moves from
project-centric processes to portfolio-based processes and evaluates
potential investments according to how well they support the agency's
missions, strategies, and goals. Organizations implementing these
Stages 2 and 3 practices have in place selection, control, and
evaluation processes that are consistent with the Clinger-Cohen
Act.[Footnote 29] Stages 4 and 5 require the use of evaluation
techniques to continuously improve both investment processes and
portfolios in order to better achieve strategic outcomes.
Figure 2: The Five ITIM Stages of Maturity with Critical Processes:
[See PDF for image]
Source: GAO.
[End of figure]
The overriding purpose of the framework is to encourage investment
selection, control, and evaluate processes that promote business value
and mission performance, reduce risk, and increase accountability and
transparency. We have used the framework in several of our
evaluations,[Footnote 30] and a number of agencies have adopted it.
With the exception of the first stage, each maturity stage is composed
of "critical processes" that must be implemented and institutionalized
in order for the organization to achieve that stage. Each ITIM critical
process consists of "key practices"--to include organizational
structures, policies, and procedures--that must be executed to
implement the critical process. Our research shows that agency efforts
to improve investment management capabilities should focus on
implementing all lower stage practices before addressing higher stage
practices.
DOD's Institutional Approach to Business Systems Modernization:
In 2005, the department reassigned responsibility for providing
executive leadership for the direction, oversight, and execution of its
business systems modernization efforts to several entities. These
entities and their responsibilities include the Defense Business
Systems Management Committee (DBSMC), which serves as the highest
ranking governance body for business systems modernization activities;
the Principal Staff Assistants, who serve as the certification
authorities for business system modernizations in their respective core
business missions; the IRBs, which form the review and decision-making
bodies for business system investments in their respective areas of
responsibility; and the Business Transformation Agency (BTA), which is
responsible for leading and coordinating business transformation
efforts across the department. The BTA is organized into seven
directorates, one of which is the Defense Business Systems Acquisition
Executive--the component acquisition executive for DOD enterprise-
level (DOD-wide) business systems and initiatives. This office is
responsible for developing, coordinating, and integrating enterprise-
level projects, programs, systems and initiatives, including managing
resources such as fiscal, personnel, and contracts for assigned systems
and programs.
Table 1 lists these entities and provides greater detail on their
roles, responsibilities, and composition.
Table 1: DOD Business Systems Modernization Governance Entities' Roles,
Responsibilities, and Composition:
Entity: DBSMC;
Roles and responsibilities:
* Provides strategic direction and plans for the business mission
area[A] in coordination with the warfighting and enterprise information
environment mission areas;
* Recommends policies and procedures required to integrate DOD business
transformation and attain cross-department, end-to-end interoperability
of business systems and processes;
* Serves as approving authority for business system modernization;
* Establishes policies and approves the business mission area strategic
plan, the enterprise transition plan for implementation for business
systems modernization, the transformation program baseline, and the
BEA;
Composition: Chaired by the Deputy Secretary of Defense; Vice Chair is
the Under Secretary of Defense for Acquisition, Technology, and
Logistics (USD(AT&L)). Includes senior leadership in the Office of the
Secretary of Defense, the military departments' secretaries, and
defense agencies' heads, such as the Assistant Secretary of Defense
(Networks and Information Integration)/Chief Information Officer
(ASD(NII)/CIO), the Vice Chairman of the Joint Chiefs of Staff, and the
Commanders of the U.S. Transportation Command and Joint Forces Command.
Entity: Principal Staff Assistants/Certification Authorities;
Roles and responsibilities:
* Support the DBSMC's management of enterprise business IT investments;
* Serve as the certification authorities accountable for the obligation
of funds for respective business system modernizations within
designated core business missions.[B];
* Provide the DBSMC with recommendations for system investment
approval;
Composition: Under Secretaries of Defense for Acquisition, Technology,
and Logistics; Comptroller; and Personnel and Readiness.
Entity: IRBs;
Roles and responsibilities:
* Serve as the oversight and investment decision-making bodies for
those business capabilities that support activities under their
designated areas of responsibility;
* Recommend certification for all business systems investments costing
more than $1 million that are integrated and compliant with the BEA;
Composition: Includes the Principal Staff Assistants; Joint Staff;
ASD(NII)/CIO; core business mission area representatives; military
departments; defense agencies; and combatant commands.
Entity: Component Pre-Certification Authority;
Roles and responsibilities:
* Ensures component-level investment review processes integrate with
the Investment Management system;
* Identifies those component systems that require IRB certification and
prepare, review, approve, validate, and transfer investment
documentation as required;
* Assesses and precertifies architecture compliance of component
systems submitted for certification and annual review;
* Acts as the component's principal point of contact for communication
with the IRBs;
Composition: Includes the Chief Information Officer from the Air Force,
the Principal Director of Governance, Acquisition, and Chief Knowledge
Office from the Army, the Chief Information Officer from the Navy, and
comparable representatives from other defense agencies.
Entity: BTA;
Roles and responsibilities:
* Operates under the authority of the USD(AT&L) under the direction of
the Deputy Under Secretary of Defense for Business Transformation and
the Deputy Under Secretary of Defense for Financial Management;
* Maintains and updates the department's BEA and enterprise transition
plan;
* Ensures that functional priorities and requirements of various
defense components, such as the Army and Defense Logistics Agency are
reflected in the architecture;
* Ensures adoption of DOD-wide information and process standards as
defined in the architecture;
* Serves as the day-to-day management entity of the business
transformation effort at the DOD enterprise level;
* Provides support to the DBSMC and IRBs;
Composition: Comprised of seven directorates (Defense Business Systems
Acquisition Executive, Enterprise Integration, Transformation Planning
and Performance, Transformation Priorities and Requirements, Investment
Management, Warfighter Support Office, and Chief of Staff).
Source: DOD.
[A] According to DOD, the business mission area is responsible for
ensuring that capabilities, resources, and materiel are reliably
delivered to the warfighter. Specifically, the BMA addresses areas such
as real property and human resources management.
[B] DOD has five core business missions: Human Resources Management,
Weapon System Lifecycle Management, Materiel Supply and Service
Management, Real Property and Installations Lifecycle Management, and
Financial Management.
[End of table]
Tiered Accountability:
In 2005, DOD reported that it had adopted a tiered accountability
approach to business transformation. Under this approach,
responsibility and accountability for business architectures and
systems investment management are assigned to different levels in the
organization. For example, the BTA is responsible for developing the
corporate BEA, which provides the thin layer of corporate policies,
capabilities, standards, and rules. The components are responsible for
defining a component-level architecture and transition plans associated
with their own tier of responsibility and for doing so in a manner that
is aligned with (i.e., does not violate) the corporate BEA's policies,
capabilities, standards, and rules. Similarly, program managers are
responsible for developing program-level architectures and plans and
ensuring alignment with the architectures and transition plans above
them. As such, this concept allows for autonomy while also ensuring
linkages and alignment from the program level through the component
level to the enterprise level.
For business investment management, responsibility and accountability
is also tiered, meaning that it is allocated between the DOD corporate
level (i.e., Office of the Secretary of Defense) and the components
based on the amount of development/modernization funding involved and
the investment's designated "tier." More specifically, DOD corporate is
responsible for ensuring that all business systems with a development/
modernization investment in excess of $1 million are reviewed by the
IRBs for compliance with the BEA, certified by the Principal Staff
Assistants, and approved by the DBSMC. Components are responsible for
certifying development/modernization investments with total costs of $1
million or less. All DOD development and modernization efforts are also
assigned a "tier" based on acquisition category and/or the size of the
financial investment.[Footnote 31]
Summary of Fiscal Year 2005 National Defense Authorization Act
Requirements:
Congress included six provisions in the act[Footnote 32] that are aimed
at ensuring DOD's development of a well-defined BEA and associated
enterprise transition plan (ETP), as well as the establishment and
implementation of effective investment management structures and
processes. The requirements are as follows:
1. Develop a BEA that:
* includes an information infrastructure that, at a minimum, would
enable DOD to:
- comply with all federal accounting, financial management, and
reporting requirements;
- routinely produce timely, accurate, and reliable financial
information for management purposes;
- integrate budget, accounting, and program information and systems;
- provide for the systematic measurement of performance, including the
ability to produce timely, relevant, and reliable cost information;
- includes policies, procedures, data standards, and system interface
requirements that are to be applied uniformly throughout the
department; and:
- is consistent with OMB policies and procedures.
2. Develop a transition plan for implementing the architecture that
includes:
* an acquisition strategy for new systems needed to complete the
enterprise architecture;
* a list and schedule of legacy business systems to be terminated;
* a list and strategy of modifications to legacy business systems; and:
* time-phased milestones, performance metrics, and a statement of
financial and non-financial resource needs.
3. Identify each business system proposed for funding in DOD's fiscal
year budget submissions and include:
* information on each business system proposed for funding in that
budget;
* funds for current services and for business systems modernization;
and:
* the designated approval authority for each business system.
4. Delegate the responsibility for business systems to designated
approval authorities within the Office of the Secretary of Defense.
5. Require each approval authority to establish investment review
structures and processes, including a hierarchy of IRBs--each with
appropriate representation from across the department. The review
process must cover:
* review and approval of each business system by an IRB before funds
are obligated;
* at least an annual review of every business system investment;
* use of threshold criteria to ensure an appropriate level of review
and accountability;
* use of procedures for making architecture compliance certifications;
* use of procedures consistent with DOD guidance; and:
* incorporation of common decision criteria.
6. Effective October 1, 2005, DOD may not obligate appropriated funds
for a defense business system modernization with a total cost of more
than $1 million unless, the approval authority certifies that the
business system modernization:
* complies with the BEA and:
* is necessary to achieve a critical national security capability or
address a critical requirement in an area such as safety or security;
or is necessary to prevent a significant adverse effect on an essential
project in consideration of alternative solutions, and the
certification is approved by the DBSMC.
Summary of Recent GAO Reviews of DOD's Business Systems Modernization
and Business Transformation Efforts:
In November 2005[Footnote 33] and in May 2006,[Footnote 34] we reported
that DOD had partially satisfied four of the six business system
modernization requirements in the fiscal year 2005 National Defense
Authorization Act[Footnote 35] relative to architecture development,
transition plan development, budgetary disclosure, and investment
review; it had fully satisfied the requirement concerning designated
approval authorities; and it was in the process of satisfying the last
requirement for certification and approval of modernizations costing in
excess of $1 million. As a result, we concluded that the department had
made important progress in defining and beginning to implement
institutional management controls (i.e., processes, structures, and
tools), but much remained to be accomplished relative to the act's
requirements and relevant guidance, including developing component
architectures that are aligned with the corporate BEA and ensuring that
investment review and approval processes are fully developed and
institutionally implemented across all organizational levels.
Notwithstanding this progress on business systems modernization, we
also testified in November 2006[Footnote 36] that DOD continued to lack
a comprehensive, enterprisewide approach to its overall business
transformation effort. We noted that while DOD's planning and
management continued to evolve, it had yet to develop a comprehensive,
integrated, and enterprisewide plan that covered all key business
functions and contained results-oriented goals, measures, and
expectations that link organizational, unit, and individual performance
goals while also being clearly linked to DOD's overall investment
plans. We concluded that because of the complexity and long-term nature
of business transformation, the department continued to need a chief
management official with significant authority, experience, and tenure
to provide sustained leadership and integrate its overall business
transformation effort. We also concluded that without formally
designating responsibility and accountability for results, reconciling
competing priorities in investments will be difficult and could impede
DOD's progress in its transformation efforts. We are currently
assessing the department's business transformation efforts, including
an analysis of the various proposals for a chief management officer and
its response to these proposals, and plan to report our results in the
near future.
DOD Is Continuing to Improve Its Approach to Modernizing Business
Systems:
DOD continues to take steps to comply with the requirements of the act
and to satisfy relevant systems modernization management guidance. In
particular, on March 15, 2007, DOD released an update to its BEA
(version 4.1), developed an updated ETP, and issued its annual report
to Congress describing steps taken and planned relative to the act's
requirements, among other things. Collectively, these steps address
several legislative provisions and best practices concerning the
corporate architecture, transition plan, budgetary disclosure, and
investment review of systems costing in excess of $1 million that we
previously reported as missing. However, additional steps are needed to
fully comply with the act and relevant guidance. Specifically, the
department has yet to extend and evolve its corporate BEA to the
department's component organizations' (military departments and defense
agencies) architectures, fully define its IT investment management
policies and procedures, and officially establish one of the five
legislatively mandated IRBs. BTA officials agree that additional steps
are needed to fully implement the act's requirements and related system
modernization management best practices. According to BTA officials,
DOD leadership is committed to fully addressing these areas and efforts
are planned and under way to do so.
DOD Continues to Improve Its Corporate BEA, but Component Architectures
Remain a Challenge:
Among other things, the act requires DOD to develop a BEA that would
cover all defense business systems and the functions and activities
supported by defense business systems and enable the entire department
to (1) comply with all federal accounting, financial management, and
reporting requirements; (2) routinely produce timely, accurate, and
reliable financial information for management purposes; and (3) include
policies, procedures, data standards, and system interface requirements
that are to be applied throughout the department.
In 2006,[Footnote 37] we reported that the then current version of the
BEA (version 3.1) addressed several of the missing elements we had
previously identified relative to the act's requirements and relevant
guidance. However, we also reported that additional steps were needed.
On March 15, 2007, DOD released an update to its BEA (version 4.1),
which resolves several of the architecture gaps associated with the
prior version and adds content proposed by DOD stakeholders.[Footnote
38] For example, version 4.1 improves the Financial Visibility business
enterprise priority (BEP) area by including the Standard Financial
Information Structure data elements and business rules to support cost
accounting and reporting. This version also addresses, to varying
degrees, missing elements, inconsistencies, and usability issues that
we previously identified.[Footnote 39] Examples of these improvements
and remaining issues are summarized in the following text:
* This latest version contains enterprise-level information about DOD's
"As Is" architectural environment to support business capability gap
analyses. As we previously reported,[Footnote 40] such gap analyses
between the "As Is" and the "To Be" environments are essential for the
development of a well-defined transition plan. However, such gap
analyses were not previously provided for in prior versions of the BEA.
To DOD's credit, the architecture now includes "As Is" information
(e.g., problems that enterprise priorities are to address and the root
causes of each problem) for five of the six BEPs. For example, this
version identifies the "inability to record or report funds
distribution at the transaction level" as a problem for the Financial
Visibility priority area, and "stove-pipe systems" and "non-standard
forms" as the root causes. Moreover, it includes "As Is" information
about related enterprise systems, such as the Wide-area Workflow
system. However, the current version does not provide "As Is"
information for the Acquisition Visibility priority area.
* The latest version includes performance metrics for the business
capabilities within enterprise priority areas, including actual
performance relative to performance targets that are to be met. For
example, currently 26 percent of DOD assets are reported by using the
Department of the Treasury's United States Standard General
Ledger[Footnote 41] compliant formats, as compared to a target of 100
percent. However, the architecture does not describe the actual
baseline performance for operational activities, such as for the
"Manage Audit and Oversight of Contractor" operational activity. As we
have previously reported,[Footnote 42] performance models are an
essential part of any architecture and having defined performance
baselines to measure actual performance against provides the means for
knowing whether the intended mission value to be delivered by each
business process is actually being realized.
* The latest version identifies activities performed at each location/
organization and indicates which organization(s) are or will be
involved in each activity. We previously reported that prior versions
did not address the locations where specified activities are to occur
and that doing so is important because the cost and performance of
implemented business operations and technology solutions are affected
by the location and therefore need to be examined, assessed, and
decided on in an enterprise context rather than in a piecemeal, systems-
specific fashion.[Footnote 43] To DOD's credit, the latest version
includes some of this information. For example, it indicates that the
Defense Contract Management Agency is involved in the "Conduct
Acquisition Assessment" operational activity. However, not all
operational activities, such as "Authorize Return or Disposal" activity
are assigned to a location/organization. In addition, the latest
version does not include the roles and responsibilities of
organizations performing the same operational activities, which is
important to avoid duplication and inconsistency in how functions and
activities are implemented.
* The latest version includes common policies (e.g., "IRBs approve only
those system investments that are aligned with enterprise
transformation objectives and standards") and procedures (e.g.,
"Components and programs use the Architecture Compliance and
Requirements Traceability tool to illustrate how their system
investments map to applicable activities, business rules, and data in
the BEA"). It also includes business rules (e.g., "each request for
commercial export of DOD technology must be processed within 30 days
upon receipt of request from the Department of State or the Department
of Commerce") to facilitate consistent implementation of the policies
and procedures.[Footnote 44] However, the architecture does not
identify enterprise business rules for all business processes. For
example, there are no business rules for the Common Supplier Engagement
business process "Perform Acceptance Procedures for Other Goods and
Services." Moreover, the latest version continues to provide
inconsistent levels of detail for some business rules. For example,
some business rules are defined at the conceptual level (e.g.,
"ENT_Cost_Reporting") while others are defined at a more operational
level (e.g., "ENT_DOD_Obligations_Against"). Without well-defined
business rules, it is likely that policies and procedures will be
implemented inconsistently because they will be uniquely interpreted.
* The latest version provides information flows among some
organizational units, business operations, and system elements. These
information flows are intended to show what information is needed and
where and how the information moves and is shared to support mission
functions. For example, the "Financial Management Detail" operational
node connectivity diagram is a graphical depiction of the operational
nodes (or organizations) with "needlines" that indicate a need to
exchange information and identify information exchange requirements
among the financial management organizational units (e.g., between the
accounting office and commercial entitlement office operational nodes).
However, detailed operational node connectivity diagrams similar to the
"Financial Management Detail" diagram have not yet been developed for
the other core business mission areas, such as Human Resources
Management. Such information is critical for defining business service
interactions and establishing interfaces between users and systems.
Moreover, the BEA does not include information flows between the
enterprise and DOD components. Such information is important for
developing a common understanding of the semantic meaning of
information exchanges among DOD organizations.
* The latest version continues to represent the thin layer of DOD-wide
corporate architectural policies, capabilities, rules, and standards.
Having this layer is essential to a well-defined federated
architecture, but it alone does not provide the total federated family
of DOD parent and subsidiary architectures for the business mission
area that are needed to comply with the act. As we recently reported,
well-defined architectures do not yet exist for the military
departments,[Footnote 45] which constitute the largest members of the
federation. In particular, we reported that none of the three military
departments had fully developed architecture products that describe
their respective target architectural environments and developed
transition plans for migrating to a target environment, and none were
employing the full range of architecture management structures,
processes, and controls provided for in relevant guidance. Accordingly,
we made recommendations aimed at improving the management and content
of the military departments' respective architectures, which the
department agreed with.[Footnote 46] (See app. III for the specific
recommendations.)
Recognizing the need to address its component architecture challenge,
the BTA released its business mission area federation strategy and road
map in September 2006 to address how the corporate BEA would be
extended to the military departments and defense agencies. We recently
reported[Footnote 47] that this strategy provides a foundation on which
to build and align DOD's parent business architecture with the
subsidiary architectures of the military departments and defense
agencies (see fig. 3). In particular, we noted that the strategy (1)
states the department's federated architecture goals; (2) describes
federation concepts that are to be applied; and (3) includes high-level
activities, capabilities, products, and services intended to facilitate
implementation of the concepts.
Figure 3: Simplified Diagram of DOD's Business Mission Area Federated
Architecture:
[See PDF for image]
Source: GAO analysis of DOD data.
[End of figure]
However, we also reported that the strategy does not adequately define
the tasks needed to achieve the strategy's goals, including those
associated with executing high-level activities and providing related
capabilities, products, and services. Specifically, it does not
adequately address how strategy execution will be governed, including
assignment of roles and responsibilities, measurement of progress and
results, and provision of resources. Also, the strategy does not
address, among other things, how the component architectures will be
aligned with the latest version of the BEA and how it will identify and
provide for reuse of common applications and systems across the
department. Accordingly, we made recommendations aimed at better
defining the department's architecture federation plans, which the
department largely disagreed with.[Footnote 48] (See app. III for the
specific recommendations.)
According to DOD, the corporate BEA focuses on providing tangible
outcomes for a limited set of enterprise-level (DOD-wide) priorities,
and the components are responsible under the department's tiered
accountability approach for defining their respective component-level
architectures that are aligned with the corporate BEA. According to
DOD, subsequent releases of the BEA will continue to reflect this
federated approach and will define enforceable interfaces to ensure
interoperability and information flow to support decision making at the
appropriate level. To help ensure this, the BTA plans to have its BEA
independent verification and validation contractor examine architecture
federation when evaluating subsequent BEA releases. Use of an
independent verification and validation agent is an architecture
management best practice for identifying architecture strengths and
weaknesses. Through the use of such an agent, department and
congressional oversight bodies can gain information that they need to
better ensure that DOD's family of architectures and associated
transition plan(s) satisfy key quality parameters, such as
completeness, consistency, understandability, and usability, which the
department's annual reports have yet to include.
Until DOD has a well-defined family of architectures for its business
mission area, it will not fully satisfy the requirements of the act and
it will remain challenged in its ability to effectively manage its
business system modernization efforts.
DOD Continues to Expand and Update Its Enterprise Transition Plan, but
Important Elements Are Still Missing:
Among other things, the act requires DOD to develop an ETP for
implementing its BEA that includes listings of the legacy systems that
will and will not be part of the target business systems environment
and specific time-phased milestones and performance metrics.
In 2006,[Footnote 49] we reported that the prior version of the ETP
addressed several of the missing elements that we previously identified
relative to the act's requirements and relevant guidance. However, we
also reported that additional steps were needed. On March 15, 2007, DOD
released an updated version of its ETP, which provides information on
106 of what it refers to as transformational programs (systems and
initiatives) and relates these to key transformational objectives. For
example, it includes specific time-phased milestones[Footnote 50] for
about 86 business system investments and initiatives and performance
metrics for about 84 systems and initiatives. Further, the ETP
discusses progress made on business system investments over the last 6
months--including key accomplishments and milestones attained, as well
as new information on near-term activities (i.e., activities to occur
during the next 6 months). This version also addresses, to varying
degrees, missing elements that we identified in our prior
report.[Footnote 51] Examples of these improvements and remaining
issues are summarized in the following text:
* The latest version of the ETP documents the results of ongoing and
planned analyses of gaps between its "As Is" and "To Be" architectural
environments, in which capability and performance shortfalls are
described and investments (such as transformation initiatives and
systems) that are to address these shortfalls are clearly identified.
For example, it aligns the Defense Integrated Military Human Resources
System with the Personnel Visibility priority area and states that it
will provide business capability improvements that include providing
accurate and timely pay benefits for military service members and their
families anytime and anywhere. However, the gap analysis is not yet
completed for all the current BEPs. In particular, the gap analysis did
not include the Acquisition Visibility priority area. Without
identifying how business capability gaps between the baseline and
target architecture are to be addressed for all BEPs, the department's
transition plan cannot be considered sufficiently complete, and thus
its ability to support informed investment selection and control
decisions is limited.
* The latest version of the ETP provides a range of information for the
106 systems and initiatives identified, such as 3 years of budget
information for 64 of these systems and initiatives. However, the plan
has yet to address our prior finding for including system and budget
information for investments by 13 of its 15 defense agencies[Footnote
52] and for 8 of its 9 combatant commands.[Footnote 53] BTA officials
told us that information for these defense agencies and combatant
commands is not included because the ETP focused on the largest
business-related organizations in DOD (i.e., those having the majority
of the tier 1 and 2 business investments), and the majority of the
defense agencies and commands do not have investments that meet this
threshold criteria. Nevertheless, they said that they plan to include
all component tier 1 and 2 systems over the next 3 years.
* The latest version also provides performance measures for the
enterprise and component transformation programs, including key
milestones (e.g., Initial Operating Capability). However, the ETP does
not include other important information needed to understand the
sequencing of these business investments. In particular, the planned
investments in the transition plan are not sequenced based on a range
of activities that are critical to developing an effective transition
plan. More specifically, we previously reported[Footnote 54] that the
plan is largely based on a bottom-up planning process in which ongoing
programs were examined and categorized in the plan around BEPs and
capabilities, including a determination as to which programs would be
designated and managed as DOD-wide, enterprise programs versus
component programs. This bottom-up approach to developing the plan does
not explicitly reflect transition planning key practices cited in
federal guidance, such as consideration of technology opportunities,
marketplace trends, fiscal and budgetary constraints, institutional
system development and acquisition capabilities, and new and legacy
system dependencies and life expectancies, and the projected value of
competing investments.[Footnote 55] For example, many of these
investments are dependent on Net-Centric Enterprise Services
(NCES)[Footnote 56] for its core services, and as such the plans and
milestones for each should reflect the incremental capability
deployment of NCES. According to the BTA official responsible for the
ETP, the transition plan investments have not been sequenced based on
any of these considerations other than fiscal year budgetary
constraints. However, DOD officials reported that the BTA intends to
depict the dependencies in the ETP, especially program-to-program
dependencies associated with adoption of a service-oriented
architecture approach. BTA officials also said that each technology-
based sequencing decision will be governed by DOD's tiered
accountability approach to investment decision making and architecture
federation.
* The latest version of the ETP includes a listing of the legacy
systems that will not be part of the "To Be" environment and the
termination dates for many of these systems. We previously
reported[Footnote 57] that the prior version did not include a complete
listing of the legacy systems and that the termination dates for many
legacy systems, including the Personnel Records Management System,
Defense Departmental Reporting System, and Base Accounts Receivable
System, were not known, making it unclear whether or not they will be
part of the target environment. To DOD's credit, the ETP now reflects
all decisions recorded to date on these legacy system terminations.
According to the department, this list will continue to evolve as
components and IRBs make investment decisions in the future. In
addition, it provides information on legacy system migration and
retirement as a result of implementing each target system. According to
DOD, the annual report lists over 700 systems targeted for elimination
as a result of the implementation of targeted business systems, with
specific termination dates identified for over 93 percent of these
systems.
* The latest version of the ETP also includes for the first time a
discussion of how the department plans to use enterprise application
integration,[Footnote 58] including plans, methods, and tools for
reusing applications that already exist while also adding new
applications and databases. However, this discussion is nevertheless
still notional and thus lacks specifics on which investments will reuse
which applications.
According to BTA officials, a number of actions are envisioned to
address the above cited areas and further improve the ETP, such as
adding the results of capability gap analyses for all business
priorities, including tier 1 and 2 programs for all components, and
recognizing dependencies among investments. Until the ETP, or a
federated family of such plans, either directly or by reference
includes relevant information on the full inventory of investments
across the department, (and does so in a manner that reflects
consideration of the range of variables associated with a well-defined
transition plan, such as timing dependencies among investments and the
department's capability to manage them) it will not have a sufficient
basis for informed investment decision making regarding disposition of
the department's existing inventory of systems or for sequencing the
introduction of modernized systems. To ensure that the above discussed
shortcomings with the department transition plan(s) are made, we have
previously made recommendations that the department is still in the
process of addressing aimed at formalizing its plans for incrementally
improving its transition plan. (See app. II for these recommendations.)
DOD's Fiscal Year 2008 Budget Submission Includes Key Information on
Business Systems:
Among other things, the act requires DOD's annual IT budget submission
to include key information on each business system for which funding is
being requested, such as the system's designated approval authority and
the appropriation type and amount of funds associated with development/
modernization and current services (to operate and maintain the
system).
The department's fiscal year 2008/2009 budget submission includes a
range of information for business system investments requesting
funding, such as the system's (1) name, (2) approval authority, (3)
approved funding for fiscal year 2007, and (4) requested funding for
fiscal year 2008. The submission also identifies the amount of the
fiscal year 2008 request that is for development/modernization versus
operations/maintenance (i.e., current services). For example, the
Army's General Fund Enterprise Business System, the amount of
modernization funds related to "Other Procurement, Army" and "Research,
Development, Testing and Evaluation, Army" are identified. For systems
in excess of $1 million in modernization funding, the submission also
cites the DBSMC approval date, where applicable.
DOD Has Largely Established Key Investment Management Structures, but
Related Policies and Procedures Are Missing:
The act requires DOD to establish business system investment review
structures, including the previously mentioned DBSMC and five IRBs, and
processes that are consistent with the investment management provisions
of the Clinger-Cohen Act.[Footnote 59] As noted earlier, our ITIM
framework provides five progressive stages of maturity for any given
agency relative to selecting, controlling, and evaluating its IT
investments. Organizations implementing Stages 2 and 3 practices have
in place capabilities that assist in establishing selection, control,
and evaluation structures, policies, procedures, and practices that are
required by the investment management provisions of the Clinger-Cohen
Act.
In 2006, we reported that DOD had established the DBSMC and four of the
five IRBs defined in the act and that it had developed a range of
processes governing how business system investments are to be reviewed
and approved.[Footnote 60] More recently, we reported on the extent to
which the department's corporate approach to business system investment
management comports with the stages in our ITIM framework that are
associated with investment management provisions of the Clinger-Cohen
Act.[Footnote 61] In summary, we found that DOD had established
important management structures needed to manage its business system
investments, but it had not fully defined many of related policies and
procedures that our framework identified as needed to effectively
manage its business investments as individual projects (Stage 2) and as
portfolios of projects (Stage 3).
Investment Management Structures Have Been Largely Established:
DOD has largely established the organizational structures that are
associated with Stages 2 and 3 of our framework. Specifically, it has
established an enterprisewide investment board and subordinate boards
that are responsible for business systems investment governance,
including conducting investment certification and approval reviews and
annual reviews as provided for in the act. The enterprisewide board--
the DBSMC--is composed of senior executives, including the Deputy
Secretary of Defense and the ASD(NII)/CIO, as provided for in the act.
Among other things, the DBSMC is responsible for establishing and
implementing policies governing the organization's investment process
and approving lower-level investment board processes and procedures.
The subordinate boards include four IRBs that are composed of
representatives from their respective core business mission, as well as
representatives from the combatant commands, defense agencies, military
departments, and Joint Chiefs of Staff. Among other things, they are
responsible and accountable for overseeing and controlling certain
business system investments, including ensuring compliance and
consistency with the BEA. The department has also assigned
responsibility to the USD(AT&L) for managing business system portfolio
selection criteria.
Moreover, since we reported in 2006[Footnote 62] that the department
has established four of the five IRBs mandated by the act, efforts have
begun to establish the fifth. Specifically, ASD(NII)/CIO officials told
us that they are now in the process of establishing the Enterprise
Information Environment Mission Area[Footnote 63] IRB to support IT
infrastructure and information assurance activities, as required by the
act. According to these officials, the draft concept of operations for
this IRB is being revised and will subsequently be approved by the
ASD(NII)/CIO. While the IRB has not been officially established, the
officials stated that it has been in effect for about a year and added
that the chair is the DOD Deputy CIO, and its membership includes
representatives from the Defense Information Systems Agency, the DOD
mission areas, and the military departments. They also said that the
Under Secretary of Defense (Comptroller) and the Joint Chiefs of Staff
are operating in an advisory role.
Policies and Procedures Have Been Defined for Some, but Not All,
Project-Level and Portfolio-Based Investment Management Activities:
As we recently reported,[Footnote 64] DOD has defined policies and
procedures relative to several key practices in our ITIM framework that
are associated with project-level investment management (Stage 2). To
its credit, the department has, for example, documented policies and
procedures for ensuring that systems support ongoing and future
business needs through alignment with the BEA; developed procedures for
identifying and collecting information about these systems to support
DBSMC and IRB investment decision making; and assigned responsibility
for ensuring that the information collected about projects meets the
needs of DOD's investment review structures and processes. However, we
reported that it had not developed the full range of project-level
policies and procedures needed for effective investment management. In
commenting on our report, DOD stated that under DOD's tiered
accountability, these are performed at the component level, and that
departmental policies and procedures established for overseeing
execution of these practices by components are sufficient. We do not
agree. Examples of the limitations in the department's project-level
policies and procedures are summarized next, along with their
significance.
* Policies and procedures do not address how business system
investments that are past the development/modernization stage (i.e., in
operations and maintenance) are to be governed or considered by the
DBSMC or the IRBs. Given that DOD invests billions of dollars annually
in operating and maintaining business systems, this is significant.
While DOD officials stated that component-level policies and procedures
address systems that are outside of development/modernization, best
practices emphasize that the corporate investment boards should
continue to review investment cost and performance baselines throughout
their life cycles.
* Policies and procedures do not outline how the DBSMC and IRB
certification and annual review processes are to be coordinated with
other decision-support processes used at DOD, such as the Joint
Capabilities Integration and Development System; the Planning,
Programming, Budgeting, and Execution system; and the Defense
Acquisition System.[Footnote 65] Without clear linkages among these
processes, inconsistent and uninformed decision making may result.
* Procedures do not specify how the full range of cost, schedule, and
benefit data is to be used by the IRBs in certification decisions.
Without documenting how such boards are to consider cost, schedule, and
benefits factors when making these decisions, the department cannot
ensure that the boards and the DBSMC consistently and objectively
select proposals that best meet the department's needs and priorities.
* Policies and procedures do not exist that provide for sufficient
oversight and visibility into component-level investment management
activities, including component reviews of systems in operations and
maintenance and tier 4 investments. According to DOD officials, such
oversight is accomplished through the department's tiered
accountability approach. However, the department did not provide
policies and procedures defining how the DBSMC and IRBs ensure
visibility into these component processes. This is particularly
important because, according to DOD, only 285 of about 3,100 total
business systems have completed the IRB certification process and have
been approved by the DBSMC. Moreover, they said that the remaining
business systems have not been through the certification process and
have not been given a tier designation. Without policies and procedures
defining how the DBSMC and IRBs have visibility into and oversight of
all business system investments, DOD risks components continuing to
invest in systems that are duplicative, stovepiped, non-integrated, and
unnecessarily costly to manage, maintain, and operate.
DOD's policies and procedures relative to portfolio-based business
system investment management (Stage 3) are even less defined that than
those for project-level investment management. As we recently
reported,[Footnote 66] DOD has not defined any of the policies and
procedures that our ITIM framework identifies as needed for effective
portfolio management. For example, the business mission area does not
have documented policies and procedures for defining the criteria to be
used for making portfolio selection decisions, creating the portfolio
of business system investments, evaluating the performance of portfolio
investments, and conducting postimplementation reviews of these
investments. According to our ITIM framework, the development and use
of portfolio selection criteria focuses on the synergistic benefits to
be found among an agency's entire collection of investments, rather
than just from the sum of the individual investments. Moreover,
adequately documenting both the policies and the associated procedures
that provide predictable, repeatable, and reliable investment selection
and control and govern how an organization manages its IT investment
portfolio(s) is important because doing so reduces investment risk of
failure and provides the basis for having rigor, discipline, and
repeatability in how investments are selected and controlled across the
entire organization. In commenting on our recent report, DOD stated
that it intends to improve departmental policies and procedures for
business system investments by, for example, establishing a single
governance structure, but plans or time frames for doing so had not
been established.
Until DOD fully defines departmentwide policies and procedures for both
individual projects and portfolios of projects, it risks selecting and
controlling these business system investments in an inconsistent,
incomplete, and ad hoc manner, which in turn reduces the chances that
these investments will meet mission needs in the most cost-effective
manner. Accordingly, our recent report made a series of recommendations
to the department for strengthening both its project-and portfolio-
level business system investment management policies and
procedures.[Footnote 67]
DOD Continues to Approve and Review Business Systems, but Military
Departments Processes for Doing So Are Still Evolving:
The act specifies two basic requirements that took effect October 1,
2005, relative to DOD's obligation of funds for business system
modernizations costing more than $1 million. First, it requires that
these modernizations be certified by a designated approval
authority[Footnote 68] as meeting specific criteria.[Footnote 69]
Second, it requires that the DBSMC approve each of these
certifications. The act also states that failure to do so before the
obligation of funds for any such modernization constitutes a violation
of the Anti-deficiency Act.[Footnote 70] In March 2006, the department
reported that the DBSMC had approved 226 business system
modernizations, and as of March 2007, it reported that the committee
approved an additional 59 systems, for a total of 285 approved systems.
A key element of the department's approach to reviewing and approving
business systems investments is the use of "tiered accountability," in
which investment review begins at the component level and proceeds
through a hierarchy of review and approval authorities, depending on
the size and significance of the investment. Air Force, Army, and Navy
officials told us that the success of the process depends on thorough
analysis of each business system before it is submitted for higher-
level review and approval. However, they added that their respective
processes for reviewing investments are still evolving. A brief summary
of each military department's investment review activities is provided
in the following text.
Air Force:
Air Force officials report that their department is following a phased
approach to conducting reviews of about 930 business systems in
accordance with the requirements of the act. In fiscal year 2007, it is
to review all tiers 1 through 4 business systems, as well as tier 5
business systems[Footnote 71] that have operating costs, not simply
development and modernization funding, greater than $1 million. During
fiscal year 2008, the Air Force plans to review all business systems in
tiers 1 through 4 and all tier 5 systems that have operating costs
greater than $500,000. For fiscal year 2009, all business systems are
to be reviewed. According to Air Force officials, implementing a phased
approach allows time to adopt the investment management guidance set
forth in our ITIM framework.[Footnote 72] While not specifically
required by the act, Air Force officials told us that the investment
management practices that it intends to put in place for its business
systems will also be leveraged for non-business system investments
(e.g., warfighting systems). We currently have ongoing work to review
the extent to which the Air Force's business systems investment
structures and processes comport with our ITIM framework.
Army:
Army officials report that their department's primary emphasis has been
on reviewing its business system investments with funding in excess of
$1 million (i.e., tiers 1 through 3 business systems). However,
officials told us that they intend to develop a list of all business
systems that require annual reviews through January 2008 to guide
future efforts. Currently, the Army reports an inventory of 873
business systems, of which 108 are systems with development/
modernization funding in excess of $1 million, and another 765 business
system investments with funding below $1 million, including 62 with no
development/modernization funding.
Navy:
Navy officials report that their department is in the process of
conducting reviews of its 697 business systems in accordance with the
requirements of the act, although the processes being used are still
evolving. For example, Navy officials stated that the focus of the
reviews has thus far been on those systems with development/
modernization funding over $1 million. According to DOD, for fiscal
years 2006 and 2007, 54 business systems were certified by the IRBs and
approved by the DBSMC. Further, they said that greater coordination
with DOD functional areas (e.g., logistics) and ASD(NII)/CIO is needed
to improve the control and accountability over its business system
investments. We currently have ongoing work to review the extent to
which the Navy's business systems investment structures and processes
comport with our ITIM framework.
DOD Continues to Implement Our Prior Recommendations:
The act's requirements concerning the architecture, transition plan,
budgetary disclosure, and investment management structures and
processes--as discussed earlier--are consistent with the 35
recommendations that we have made since 2001, to assist the department
in developing a well-defined and useful BEA and using it to gain
control over its ongoing business system investments. To its credit,
DOD largely agreed with these recommendations and stated its commitment
to implement them. In May 2006, we reported that the department had
taken steps to fully implement 21 of the recommendations, while 14 had
yet to be fully implemented.[Footnote 73]
Since then, 10 of the 14 have either been largely implemented or have
been subsumed by our more recent recommendations and thus we are
considering them closed. (See app. II for details on the status of
these 14 recommendations; see app. III for a detailed listing of the
additional recommendations that we have made since our last annual
report under the act.) For example, DOD has addressed the core elements
in our Enterprise Architecture Management Maturity Framework[Footnote
74] relative to its corporate BEA. In particular, it has established a
chief architect who is responsible for developing the corporate BEA and
ensuring that the BEA depicts the "As Is" and "To Be" environments in
terms of business, performance, information/data, application/service,
technology, and security. As another example, the department has taken
steps to make effective use of the results of its BEA independent
verification and validation contractor on prior versions of the
architecture. As we have previously reported, using an independent
verification and validation agent is a recognized best practice because
it provides internal and external oversight bodies important
information on architecture and transition plan quality and governance.
By having and using an independent verification and validation agent,
organizations can disclose to oversight bodies independent assessments
of architecture and transition plan quality, to include completeness,
consistency, understandability, and usability, which the department has
yet to provide in its annual reports.
With respect to the remaining 4 of the 14 recommendations, actions are
under way that are intended to implement them. For example, in response
to our recommendation to develop a BEA program management plan[Footnote
75] that defines what the department's incremental improvements to the
architecture and transition plan will be, and how and when they will be
accomplished, the BTA has developed the Business Transformation
Guidance, which describes the high-level process by which incremental
improvements are identified and eventually incorporated into the
architecture. In addition, BTA officials stated that they are
developing a BEA Concept of Operations, which is to describe high-level
milestones for the BEA's use.
As another example, the BTA has established a communications team that
is responsible for achieving strategic communications objectives and
promoting external awareness of the department's vision, mission, and
progress, and BTA officials told us that this team is in the process of
developing a communications plan. According to the officials, these
efforts will address our recommendation for the BEA program to be
supported by a proactive marketing and communication program.[Footnote
76]
According to the Deputy Under Secretary of Defense (Business
Transformation), the department is committed to addressing all of our
open recommendations. It is important that the department move swiftly
in doing so because these recommendations are aimed at strengthening
architecture (and transition planning) management activities and
controlling ongoing and planned business system investments. Until it
does, the department will be challenged in its ability to effectively
guide and constrain the billions of dollars it invests annually in
thousands of business system investments.
Conclusions:
Since our last legislatively mandated report on DOD's compliance with
section 332 of the National Defense Authorization Act for Fiscal Year
2005, DOD has continued to make important progress in defining and
implementing institutional modernization management controls and
business systems budgetary disclosure, but much remains to be
accomplished. In particular, the department has yet to extend and
evolve its corporate BEA through the development of aligned subordinate
architectures for each of its component organizations, and while it has
developed a strategy for federating the BEA in this manner, this
strategy lacks the detail needed for it to be effectively implemented.
Compounding this situation is the known immaturity of the military
service architecture efforts, as well as DOD's corporate approach to
business system investment management not being governed by the range
of defined policies and procedures that are associated with effective
investment selection, control, and evaluation. Moreover, the military
departments' investment review processes are still evolving. These
architecture and investment management limitations continue to put the
billions of dollars that DOD spends each year on its thousands of
business system investments at risk.
The recommendations that we have made since we issued our last annual
report under the act are aimed at addressing these architecture and
investment management challenges. Given the demonstrated commitment of
DOD leadership to improving its business systems modernization efforts
and its recent responsiveness to our prior recommendations, we are
optimistic concerning the likelihood that the department will continue
to make progress on these fronts.
Development of a well-defined federated architecture for the business
mission area and the definition of effective business system investment
management policies and procedures across all levels of the department
are critically important in addressing the DOD business system
modernization high-risk area. However, the more formidable challenge
facing the department is how well it actually implements the
architecture and investment management controls over the years ahead on
each and every business system investment. While not a guarantee,
development of a federated BEA, including a transition plan(s), and
effective institutional business system investment management processes
can go a long way in addressing this longer-term challenge. In this
regard, it is important for the department to keep congressional
defense committees fully informed about its progress in federating the
DOD corporate BEA, to include the maturity of component organization
architecture efforts and the related transition plan(s).
Recommendation for Executive Action:
To facilitate congressional oversight and promote departmental
accountability, we recommend that the Secretary of Defense direct the
Deputy Secretary of Defense, as the chair of the DBSMC, to include in
DOD's annual report to Congress on compliance with the section 332 of
Fiscal Year 2005 National Defense Authorization Act, the results of
assessments by its BEA independent verification and validation
contractor of the completeness, consistency, understandability, and
usability of its federated family business mission area architectures,
including the associated transition plan(s).
Agency Comments:
In written comments on a draft of this report, signed by the Deputy
Under Secretary of Defense (Business Transformation) and reprinted in
appendix IV, the department agreed with our recommendation.
We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the
Secretary of Defense; the Deputy Secretary of Defense; the Under
Secretary of Defense for Acquisition, Technology, and Logistics; the
Under Secretary of Defense (Comptroller); the Assistant Secretary of
Defense (Networks and Information Integration)/Chief Information
Officer; the Under Secretary of Defense (Personnel and Readiness); and
the Director, Defense Finance and Accounting Service. Copies of this
report will be made available to other interested parties upon request.
This report will also be available at no charge on our Web site at
http://www.gao.gov.
If you or your staffs have any questions on matters discussed in this
report, please contact me at (202) 512-3439 or hiter@gao.gov, or McCoy
Williams at (202) 512-9095 or williamsm1@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major contributions
to this report are listed in appendix V.
Signed by:
Randolph C. Hite:
Director:
Information Technology Architecture and Systems Issues:
Signed by:
McCoy Williams:
Director:
Financial Management Assurance:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel Inouye:
Chairman:
The Honorable Ted Stevens:
Ranking Member:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan Hunter:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
Our objectives were to (1) assess the actions by the Department of
Defense (DOD) to comply with the requirements of section 2222 of Title
10, U.S. Code,[Footnote 77] and (2) determine the extent to which DOD
has addressed our prior open recommendations for institutionalizing key
business system modernization management controls.
For our first objective, we focused on five of the six requirements in
section 2222, and related best practices contained in federal guidance,
that we identified in our last annual report under the act as not being
fully satisfied.[Footnote 78] Generally, these five requirements are
(1) development of a business enterprise architecture (BEA), (2)
development of a transition plan for implementing the BEA, (3)
inclusion of business systems information in DOD's budget submission,
(4) establishment of business systems investment review processes and
structures, and (5) approval of defense business systems investments
with obligations in excess of $1 million. (See the Background section
of this report for additional information on the act's requirements.)
We did not include the sixth requirement because our last annual report
under the act shows that it had been satisfied. Our methodology
relative to each of the five requirements is as follows.
* To determine whether the BEA addressed the requirements specified in
the act, and related guidance, we analyzed version 4.1 of the BEA,
which was released on March 15, 2007, relative to the act's specific
architectural requirements and related guidance that our last annual
report under the act identified as not being met. We also reviewed
version 4.1 to confirm whether statements made in DOD's March 15, 2007,
annual report about the BEA's content were accurate. Also, we reviewed
and leveraged the applicable results contained in our recent reports on
major departments' and agencies' enterprise architecture programs and
on DOD's BEA federation strategy.[Footnote 79]
* To determine whether the enterprise transition plan (ETP) addressed
the requirements specified in the act, we reviewed the updated version
of the ETP, which was released on March 15, 2007, relative to the act's
specific transition plan requirements and related guidance that our
last annual report under the act identified as not being met. We also
reviewed the ETP to confirm that statements in DOD's March 15, 2007,
annual report about the content of the ETP were accurate.
* To determine whether DOD's fiscal year 2008 information technology
budget submission was prepared in accordance with the criteria set
forth in the act, we reviewed and analyzed the department report
entitled Report on Defense Business System Modernization FY 2005
National Defense Authorization Act, Section 332, prepared in February
2007 and compared the information obtained to the specific requirements
in the act.
* To determine whether DOD has established investment review structures
and processes, we focused the act's requirements that our last annual
report under the act identified as not being met, obtaining
documentation and interviewing cognizant DOD officials about efforts to
establish the one Investment Review Board (IRB) specified in the act
that had yet to be established. We also reviewed and leveraged our
recent report that assessed DOD's corporate investment approach to
managing business system investments against relevant federal
guidance.[Footnote 80]
* To determine whether the department was reviewing and approving
business system investments exceeding $1 million, we obtained the list
of business system investments certified by the IRBs and approved by
the Defense Business Systems Management Committee from the Business
Transformation Agency (BTA). We then compared the detailed information
provided with the summary information contained in the department's
March 15, 2007, report to the congressional defense committees to
identify any anomalies. We also met with representatives from the Air
Force, the Army, and the Navy to ascertain the specific actions that
were taken (or planned to be taken) in order to perform the annual
systems reviews as required by the act.
To determine the extent to which DOD has addressed our prior open
recommendations, we focused on the 14 recommendations that we
identified in our last annual report under the act as not being
implemented. We did not examine the recommendations for establishing
and implementing key business system modernization management controls
that we made since this last annual report because sufficient time had
yet to elapse for the department to have addressed them. (See app. III
for a list of the recommendations made since our last annual report
under the act.) In reviewing the 14 recommendations, we obtained and
analyzed documentation relative to corrective actions taken and
planned. Documentation that we reviewed included the DOD's March 15,
2007, annual report, updated transition plan, and BEA version 4.1. We
also compared a range of other program documentation, such as program
policies and procedures and configuration plan, to relevant elements in
our Enterprise Architecture Management Maturity Framework.[Footnote 81]
Further, we reviewed documentation regarding DOD verification and
validation contractor activities and the BTA's human capital strategy.
In addition, we reviewed the guidance establishing the IRBs and
describing the investment review, certification, and approval process.
We did not independently validate the reliability of the cost and
budget figures provided by DOD because the specific amounts were not
relevant to our findings. We conducted our work at DOD headquarters in
Arlington, Virginia, from March through May 2007 in accordance with
generally accepted government auditing standards.
[End of section]
Appendix II: Status of Prior Recommendations Identified as Open in
GAO's Prior Annual Report under the Act:
GAO report information and recommendation: GAO-01-525; Information
Technology: Architecture Needed to Guide Modernization of DOD's
Financial Operations, May 17, 2001;
(1) Until an enterprise architecture is developed and the Council is
positioned to serve as Department of Defense's (DOD) financial
management investment review board as recommended, the Secretary of
Defense limit DOD components' financial management investments to the
deployment of systems that have already been fully tested and involve
no additional development or acquisition costs; stay-in-business
maintenance needed to keep existing systems operational; management
controls needed to effectively invest in modernized systems; and new
systems or existing system changes that are congressionally directed or
are relatively small, cost-effective, and low risk and can be delivered
in a relatively short time frame;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: This recommendation has been subsumed by more recent
recommendations concerning the department's efforts to federate the
corporate business enterprise architecture (BEA), mature DOD component
organization architectures, and establish policies and procedures for
effective corporate business system investment management. (See app.
III for these more recent recommendations).
GAO report information and recommendation: GAO-03-458; DOD Business
Systems Modernization: Improvements to Enterprise Architecture
Development and Implementation Efforts Needed, February 28, 2003;
(1) The Secretary of Defense ensure that the enterprise architecture
program is supported by a proactive marketing and communication
program;
Implemented/ Closed: Yes: [Empty];
In process: X;
GAO assessment: The Business Transformation Agency (BTA) has
established a communications team that is responsible for achieving
strategic communications objectives and promoting external awareness of
the department's vision, mission, and progress. However, the department
has yet to develop a communication plan that adheres to criteria set
forth by the best practices, to include an explanation of roles and
responsibilities and details regarding evaluation, metrics, and
feedback. BTA officials told us that such a plan is currently in
development.
GAO report information and recommendation: GAO-03-1018; DOD Business
Systems Modernization: Important Progress Made to Develop Business
Enterprise Architecture, but Much Work Remains, September 19, 2003;
(1) The Secretary of Defense or his appropriate designee implement the
core elements in our Enterprise Architecture Framework for Assessing
and Improving Enterprise Architecture Management that we identify in
this report as not satisfied, including ensuring that minutes of the
meetings of the executive body charged with directing, overseeing, and
approving the architecture are prepared and maintained;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: The BTA has largely addressed the 31 core elements in
our Enterprise Architecture Management Maturity Framework in its
corporate BEA, which is the intended focus of the recommendation. For
example, the BTA has established a chief architect who is responsible
for developing and maintaining the corporate BEA and the version 4.1 of
the BEA largely provides a depiction of both the "As Is" and "To Be"
environments in terms of business, performance, information/data,
application/service, technology, and security. (See app. III for recent
recommendations aimed at having the military departments address these
core elements).
GAO report information and recommendation: (2) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
include the 29 key elements governing the "As Is" architectural content
that our report identified as not being fully satisfied; [
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO Assessment: The BTA has largely addressed these 29 key elements
relative to its corporate BEA, which is the intended focus of the
recommendation. For example, version 4.1 of the BEA contains enterprise-
level "As Is" information to support business capability gap analyses.
In addition, the architecture includes "As Is" information for five of
the six business enterprise priorities and "As Is" information for
enterprise systems, such as the Wide-area Workflow system. (See app.
III for recent recommendations aimed at effectively federating the
corporate BEA to DOD component organizations).
GAO report information and recommendation: (3) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
include the 30 key elements governing the "To Be" architectural content
that our report identified as not being fully satisfied;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: The BTA has largely addressed these 30 key elements
relative to its corporate BEA, which is the intended focus of the
recommendation. For example, version 4.1 of the BEA identifies
activities performed at each location/organization and indicates which
organization(s) is or will be involved in each activity. Furthermore,
it includes common business rules (e.g., "each request for commercial
export of DOD technology must be processed within 30 days upon receipt
of request from the Department of State or the Department of Commerce")
to facilitate consistent implementation of the architecture. (See app.
III for recent recommendations aimed at effectively federating the
corporate BEA to DOD component organizations).
GAO report information and recommendation: (4) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
include (a) the 3 key elements governing the transition plan content
that our report identified as not being fully satisfied and (b) those
system investments that will not become part of the "To Be"
architecture, including time frames for phasing out those systems;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: The BTA has largely addressed this recommendation for
its corporate or enterprise transition plan, which is the intended
focus of the recommendation. For example, the latest version of the
transition plan now documents how BEA elements (e.g., specific business
capability improvements) provide solutions to significant DOD issues or
business capability gaps (e.g., mission needs, materiel weaknesses). It
also provides performance information of DOD transformation at both the
enterprise level and component level, including performance metrics and
milestones. (See app. III for recent recommendations aimed at
effectively federating the corporate BEA, to include the transition
plan, to DOD component organizations).
GAO report information and recommendation: (5) The Secretary of Defense
or his appropriate designee update version 1.0 of the architecture to
address comments made by the verification and validation contractor;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: The verification and validation contractor reports that
all of these comments on versions 3.0 and prior versions have been
addressed.
GAO report information and recommendation: (6) The Secretary of Defense
or his appropriate designee develop a well-defined, near-term plan for
extending and evolving the architecture and ensure that this plan
includes addressing our recommendations, defining roles and
responsibilities of all stakeholders involved in extending and evolving
the architecture, explaining dependencies among planned activities, and
defining measures of activity progress;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: This recommendation has been subsumed by a later
recommendation in GAO-06-658.
GAO report information and recommendation: (7) The Secretary of Defense
or his appropriate designee limit the pilot projects to small, low-
cost, low-risk prototype investments that are intended to provide
knowledge needed to extend and evolve the architecture, and are not to
acquire and implement production version system solutions or to deploy
an operational system capability;
Implemented/ Closed: Yes: [Empty];
In process: X;
GAO assessment: According to BTA officials, the department is
continuing to assess and clarify the role of pilot projects and a
policy is to be developed relative to them. However, they did not
provide specific plans and time frames for developing and implementing
this policy.
GAO report information and recommendation: GAO-05-381; DOD Business
Systems Modernization: Billions Being Invested without Adequate
Oversight, April 29, 2005;
(1) The Secretary of Defense direct that the DBSMC develop a
comprehensive plan that addresses implementation of our previous
recommendations related to the BEA and the control and accountability
over business systems investments (at a minimum, the plan should assign
responsibility and estimated time frames for completion);
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: DOD's March 15, 2007, annual report to the
congressional defense committees identifies specific actions the
department is taking to address our open recommendations. The March
report noted that BTA has overall responsibility for ensuring that
remaining open recommendations are adequately addressed.
GAO report information and recommendation: (2) The Secretary of Defense
direct that the comprehensive plan we recommend be incorporated into
the department's second annual report due March 15, 2006, to the
defense congressional committees, as required by the Fiscal Year 2005
Defense Authorization act to help facilitate congressional oversight;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: DOD's March 15, 2006, and March 15, 2007, reports to
congressional committees included steps that DOD is taking or plans to
take to address our open recommendations.
GAO report information and recommendation: GAO-05-702; DOD Business
Systems Modernization: Long-standing Weaknesses in Enterprise
Architecture Development Need to Be Addressed, July 22, 2005;
(1) The Secretary of Defense should direct the Deputy Secretary of
Defense, as the chair of the DBSMC and in collaboration with DBSMC
members, to ensure that each of our recommendations related to the BEA
management and content are reflected in the plans and commitments;
Implemented/ Closed: Yes: X;
In process: [Empty];
GAO assessment: BTA and BEA program documentation reflects activities
and steps taken or planned to address our recommendations relative to
BEA content and management. Furthermore, the department has stated its
commitment to doing so in its annual reports to the congressional
defense committees.
GAO report information and recommendation: (2) The Secretary of Defense
should direct the Deputy Secretary of Defense, as the chair of the
DBSMC and in collaboration with DBSMC members, to ensure that plans and
commitments provide for effective BEA workforce planning, including
assessing workforce knowledge and skills needs, determining existing
workforce capabilities, identifying gaps, and filling these gaps;
Implemented/ Closed: Yes: [Empty];
In process: X;
GAO assessment: On March 21, 2007, the BTA released its Human Capital
Strategic Plan 2007-2009, which identifies BTA's goals for human
capital development and workforce planning. This strategy provides an
overview of the current workforce status in relation to those goals and
identifies several key activities for how to proceed in order to
achieve the goals. In addition, the strategy includes an initial
implementation roadmap with timelines for key activities. According to
BTA officials, the detailed plans for accomplishing key activities will
be contained in BTA's Human Capital Implementation Plan, which has yet
to be released.
GAO report information and recommendation: GAO-06-658; Business Systems
Modernization: DOD Continues to Improve Institutional Approach, but
Further Steps Needed, May 15, 2006;
(1) The Secretary of Defense direct the Deputy Secretary of Defense, as
the chair of the DBSMC, to submit an enterprise architecture program
management plan to defense congressional committees that defines what
the department's incremental improvements to the architecture and
transition plan will be, and how and when they will be accomplished,
including what (and when) architecture and transition plan scope and
content and architecture compliance criteria will be added into which
versions; the plan should also include an explicit purpose and scope
for each version of the architecture, along with milestones, resource
needs, and performance measures for each planned version;
Implemented/ Closed: Yes: [Empty];
In process: X;
GAO assessment: BTA has developed several documents that are intended
to begin addressing this recommendation. For example, it has developed
the Business Transformation Guidance, which describes the high-level
process by which incremental improvements are identified and eventually
incorporated into the BEA. In addition, BTA officials told us that they
are developing a BEA Concept of Operations, which is to describe high-
level milestones required to address the architecture's use (e.g.,
investment management, strategic decision making, oversight, system
implementation, and business case development). Notwithstanding these
steps, the department has yet to develop an architecture program
management plan that we have recommended. (See app. III for a more
recent recommendation that augments this recommendation.)
Source: GAO.
Note: See GAO, Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658
(Washington, D.C.: May 15, 2006).
[End of table]
[End of section]
Appendix III: Other Open Recommendations on Business Architectures,
Federation Strategy, and Investment Management:
GAO report information and recommendation: GAO-06-831; Enterprise
Architecture: Leadership Remains Key to Establishing and Leveraging
Architectures for Organizational Transformation, August 14, 2006.
1. The Secretary of Defense ensure that the Department of Defense
(DOD) - Global Information Grid enterprise architecture program
develops and implements plans for fully satisfying each of the
conditions in our enterprise architecture management maturity
framework.
2. The Secretary of Defense ensure that the Department of the Air Force
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
3. The Secretary of Defense ensure that the Department of the Army
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
4. The Secretary of Defense ensure that the Department of the Navy
enterprise architecture program develops and implements plans for fully
satisfying each of the conditions in our enterprise architecture
management maturity framework.
GAO report information and recommendation: GAO-07-451; Business Systems
Modernization: Strategy for Evolving DOD's Business Enterprise
Architecture Offers a Conceptual Approach, but Execution Details Are
Needed, April 16, 2007.
1. The Secretary of Defense direct the Deputy Secretary of Defense, as
the chair of the Defense Business Systems Management Committee (DBSMC),
to ensure that the appropriate DOD organizations submit a business
enterprise architecture (BEA) development management plan that
describes, at a minimum, how the business mission area architecture
federation will be governed; how the business mission area federation
strategy alignment with the DOD enterprise architecture federation
strategy will be achieved; how component business architectures'
alignment with incremental versions of the BEA will be achieved; how
shared services will be identified, exposed, and subscribed to; and
what milestones will be used to measure progress and results.
GAO report information and recommendation: GAO-07-538; Business Systems
Modernization: DOD Needs to Fully Define Policies and Procedures for
Institutionally Managing Investments, May 11, 2007.
1. The Secretary of Defense should direct the Deputy Secretary of
Defense, as the chair of the DBSMC, to ensure that well-defined and
disciplined business system investment management policies and
procedures are developed and issued. At a minimum, this should include
project-level management policies and procedures that address the
following five areas:
* instituting the investment boards, including assigning the investment
boards responsibility, authority, and accountability for programs
throughout the investment life cycle and specifying how the business
investment management system is coordinated with the Joint Capabilities
Integration and Development System, the Planning, Programming,
Budgeting, and Execution system, and the Defense Acquisition System;
* selecting new investments, including specifying how cost, schedule,
and benefit data are to be used in making certification decisions;
defining the criteria used to select investments as enterprisewide; and
establishing consistent and effective guidance for BEA compliance;
* reselecting ongoing investments, including specifying how cost,
schedule, and performance data are to be used in the annual review
process and providing for the reselection of investments that are in
operations and maintenance;
* integrating funding with the process of selecting an investment,
including specifying how the DBSMC and the investment review boards use
funding information in carrying out decisions on system certification
and approvals; and;
* overseeing IT projects and systems, including providing sufficient
oversight and visibility into component-level investment management
activities.
2. The Secretary of Defense should direct the Deputy Secretary of
Defense, as the chair of the DBSMC, to ensure that well-defined and
disciplined business system investment management policies and
procedures are developed and issued. These policies and procedures
should also include portfolio-level management policies and procedures
that address the following four areas:
* creating and modifying information technology portfolio selection
criteria for business system investments;
* analyzing, selecting, and maintaining business system investment
portfolios;
* reviewing, evaluating, and improving the performance of its
portfolio(s) by using project indicators such as cost, schedule, and
risk; and;
* conducting postimplementation reviews for all investment tiers and
directing the investment boards who are accountable for corporate
business system investments, to consider the information gathered and
to develop lessons learned from these reviews.
Source: GAO.
[End of table]
[End of section]
Appendix IV: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
3000 Defense Pentagon:
Washington, DC 20301-3000:
Acquisition, Technology And Logistics:
May 4 2007:
Mr. Randy Hite:
Director, Information Technology Architecture and Systems Issues:
Mr. McCoy Williams:
Director, Financial Management Assurance:
U.S. Government Accountability Office:
441 G Street NW:
Washington, DC 20548:
Dear Messieurs Hite and Williams:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-07-733 "DOD Business Systems Modernization: Progress
Continues to be Made in Establishing Corporate Management Controls, but
Further Steps Are Needed," dated April 26, 2007 (GAO Code 310643).
Enclosed please find the Department's response to GAO's draft report.
DoD concurs with GAO's recommendation.
We continue to view GAO's insight as a valuable asset to the
Department's defense business transformation efforts. We welcome GAO's
participation in our future efforts as the Department continues to
progress.
Signed by:
Paul A. Brinkley:
Deputy Under Secretary of Defense (Business Transformation):
Enclosure:
As stated:
GAO Draft Report Dated April 26, 2007 GAO-07-733 (GAO Codes 310643):
"DOD Business Systems Modernization: Progress Continues To Be Made In
Establishing Corporate Management Controls, But Further Steps Are
Needed"
Department Of Defense Comments To The GAO Recommendation:
Recommendation 1: The GAO recommended that the Secretary of Defense
direct the Deputy Secretary of Defense, as the chair of the Defense
Business Systems Management Committee, to include in DoD's annual
report to the Congress on compliance with the Fiscal Year 2005 National
Defense Authorization Act, the results of assessments by its Business
Enterprise Architecture independent verification and validation
contractor of the completeness, consistency, understandability, and
usability of its federated family of business mission area
architectures, including the associated transition plan(s). (p. 50/GAO
Draft Report):
DOD Response: Concur - The DoD concurs with the recommendation that the
results of these types of assessments be included in DoD's annual
report to the Congress on compliance with the Fiscal Year 2005 National
Defense Authorization Act.
Attachment:
[End of section]
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Randolph C. Hite (202) 512-3439 or hiter@gao.gov:
McCoy Williams (202) 512-9095 or williamsm1@gao.gov:
Staff Acknowledgments:
In addition to the contact persons named above, key contributors to
this report were Beatrice Alff, Karl Essig, Nancy Glover, Michael
Holland, Neelaxi Lakhmani (Assistant Director), Anh Le, Evelyn Logue,
Jacqueline Mai, John Martin, Darby Smith (Assistant Director), Debra
Rucker, and Jennifer Stavros-Turner.
FOOTNOTES
[1] Business systems support DOD's business operations, such as
civilian personnel, finance, health, logistics, military personnel,
procurement, and transportation.
[2] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.:
January 2007).
[3] An enterprise architecture, or modernization blueprint, provides a
clear and comprehensive picture of an entity, whether it is an
organization (e.g., federal department or agency) or a functional or
mission area that cuts across more than one organization (e.g.,
financial management). This picture consists of snapshots of the
enterprise's current "As Is" operational and technological environment
and its target or "To Be" environment, and contains a capital
investment road map for transitioning from the current to the target
environment. These snapshots consist of "views," which are basically
one or more architecture products that provide conceptual or logical
representations of the enterprise.
[4] GAO, Information Technology: Architecture Needed to Guide
Modernization of DOD's Financial Operations, GAO-01-525 (Washington,
D.C.: May 17, 2001).
[5] See, for example, GAO, Defense Business Transformation: A
Comprehensive Plan, Integrated Efforts, and Sustained Leadership Are
Needed to Assure Success, GAO-07-229T (Washington, D.C.: Nov. 16,
2006); Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658
(Washington, D.C.: May 15, 2006); DOD Business Systems Modernization:
Long-standing Weaknesses in Enterprise Architecture Development Need to
Be Addressed, GAO-05-702 (Washington, D.C.: July 22, 2005); DOD
Business Systems Modernization: Billions Being Invested without
Adequate Oversight, GAO-05-381 (Washington, D.C.: Apr. 29, 2005); DOD
Business Systems Modernization: Limited Progress in Development of
Business Enterprise Architecture and Oversight of Information
Technology Investments, GAO-04-731R (Washington, D.C.: May 17, 2004);
DOD Business Systems Modernization: Important Progress Made to Develop
Business Enterprise Architecture, but Much Work Remains, GAO-03-1018
(Washington, D.C.: Sept. 19, 2003); Business Systems Modernization:
Summary of GAO's Assessment of the Department of Defense's Initial
Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July
7, 2003); Information Technology: Observations on Department of
Defense's Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.:
Mar. 28, 2003); DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed,
GAO-03-458 (Washington, D.C.: Feb. 28, 2003); and GAO-01-525.
[6] Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. § 2222).
[7] GAO-06-658.
[8] GAO, Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments, GAO-
07-538 (Washington, D.C.: May 11, 2007); and Business Systems
Modernization: Strategy for Evolving DOD's Business Enterprise
Architecture Offers Conceptual Approach but Execution Details Needed,
GAO-07-451 (Washington, D.C.: Apr. 16, 2007).
[9] GAO-07-451 and Enterprise Architecture: Leadership Remains Key to
Establishing and Leveraging Architectures for Organizational
Transformation, GAO-06-831 (Washington, D.C.: Aug. 14, 2006).
[10] GAO-07-538.
[11] GAO-06-658.
[12] See, for example, GAO, DOD Travel Cards: Control Weaknesses
Resulted in Millions of Dollars of Improper Payments, GAO-04-576
(Washington, D.C.: June 9, 2004); Military Pay: Army National Guard
Personnel Mobilized to Active Duty Experienced Significant Pay
Problems, GAO-04-89 (Washington, D.C.: Nov. 13, 2003); and Defense
Inventory: Opportunities Exist to Improve Spare Parts Support Aboard
Deployed Navy Ships, GAO-03-887 (Washington, D.C.: Aug. 29, 2003).
[13] GAO-07-310.
[14] These 8 high-risk areas include DOD's overall approach to business
transformation, business systems modernization, financial management,
the personnel security clearance program, supply chain management,
support infrastructure management, weapon systems acquisition, and
contract management.
[15] The 7 governmentwide high-risk areas are (1) disability programs,
(2) ensuring the effective protection of technologies critical to U.S.
national security interests, (3) interagency contracting, (4)
information systems and critical infrastructure, (5) information-
sharing for homeland security, (6) human capital, and (7) real
property.
[16] The Clinger-Cohen Act of 1996, 40 U.S.C. § 11315(b)(2).
[17] The E-Government Act of 2002, Pub. L. No. 107-347 (Dec. 17, 2002).
[18] GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington,
D.C.: March 2004); OMB, Capital Programming Guide, Version 1.0 (July
1997); and CIO Council, A Practical Guide to Federal Enterprise
Architecture, Version 1.0 (February 2001).
[19] The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11101-11704. This act
expanded the responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act with regard to IT management. See 44
U.S.C. 3504(a)(1)(B)(vi) (OMB); and 44 U.S.C. 3506(h)(5) (agencies).
[20] We have made recommendations to improve OMB's process for
monitoring high-risk IT investments; see GAO, Information Technology:
OMB Can Make More Effective Use of Its Investment Reviews, GAO-05-276
(Washington, D.C.: Apr. 15, 2005).
[21] This policy is set forth and guidance is provided in OMB Circular
No. A-11 (Nov. 2, 2005) (section 300) and in OMB's Capital Programming
Guide, which directs agencies to develop, implement, and use a capital
programming process to build their capital asset portfolios.
[22] See, for example, GAO-04-394G; Information Technology: A Framework
for Assessing and Improving Enterprise Architecture Management (Version
1.1), GAO-03-584G (Washington, D.C.: April 2003); and Assessing Risks
and Returns: A Guide for Evaluating Federal Agencies' IT Investment
Decision-making, GAO/AIMD-10.1.13 (Washington, D.C.: February 1997).
[23] J.A. Zachman, "A Framework for Information Systems Architecture,"
IBM Systems Journal 26, no. 3 (1987).
[24] DOD, Department of Defense Architecture Framework, Version 1.0,
Volume 1 (August 2003) and Volume 2 (February 2004).
[25] See, for example, GAO, Homeland Security: Efforts Under Way to
Develop Enterprise Architecture, but Much Work Remains, GAO-04-777
(Washington, D.C.: Aug. 6, 2004); GAO-04-731R; Information Technology:
Architecture Needed to Guide NASA's Financial Management Modernization,
GAO-04-43 (Washington, D.C.: Nov. 21, 2003); GAO-03-1018; GAO-03-877R;
Information Technology: DLA Should Strengthen Business Systems
Modernization Architecture and Investment Activities, GAO-01-631
(Washington, D.C.: June 29, 2001); and Information Technology: INS
Needs to Better Manage the Development of Its Enterprise Architecture,
GAO/AIMD-00-212 (Washington, D.C.: Aug. 1, 2000).
[26] GAO, Information Technology: FBI Has Largely Staffed Key
Modernization Program, but Strategic Approach to Managing Program's
Human Capital Is Needed, GAO-07-19 (Washington, D.C.: Oct. 16, 2006).
[27] GAO-04-394G; GAO, GAO/AIMD-10.1.13; Executive Guide: Improving
Mission Performance Through Strategic Information Management and
Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1994); and Office of
Management and Budget, Evaluating Information Technology Investments, A
Practical Guide (Washington, D.C.: November 1995).
[28] GAO-04-394G.
[29] 40 U.S.C. §§ 11311-11313.
[30] GAO, Information Technology: Centers for Medicare & Medicaid
Services Needs to Establish Critical Investment Management
Capabilities, GAO-06-12 (Washington, D.C.: Oct. 28, 2005); Information
Technology: HHS Has Several Investment Management Capabilities in
Place, but Needs to Address Key Weaknesses, GAO-06-11 (Washington,
D.C.: Oct. 28, 2005); Information Technology: FAA Has Many Investment
Management Capabilities in Place, but More Oversight of Operational
Systems Is Needed, GAO-04-822 (Washington, D.C.: Aug. 20, 2004);
Information Technology: Departmental Leadership Crucial to Success of
Investment Reforms at Interior, GAO-03-1028 (Washington, D.C.: Sept.
12, 2003); Bureau of Land Management: Plan Needed to Sustain Progress
in Establishing IT Investment Management Capabilities, GAO-03-1025
(Washington, D.C.: Sept. 12, 2003); United States Postal Service:
Opportunities to Strengthen IT Investment Management Capabilities, GAO-
03-3 (Washington, D.C.: Oct. 15, 2002); and Information Technology: DLA
Needs to Strengthen Its Investment Management Capability, GAO-02-314
(Washington, D.C.: Mar. 15, 2002).
[31] As defined in the department's Investment Review Board Concept of
Operations and its Investment Certification and Annual Review Process
User Guidance, there are four tiers of business systems. Tier 1 systems
include all systems that are classified as a "major automated
information system" or a "major defense acquisition program;" tier 2
systems include those with modernization efforts of $10 million or
greater but that are not designated as a major automated information
system or a major defense acquisition program, or programs that have
been designated as IRB interest programs because of their impact on DOD
transformation objectives; tier 3 systems include those with
modernization efforts that have anticipated costs greater than $1
million but less than $10 million; and tier 4 systems are those with
modernization efforts that have anticipated costs of up to $1 million.
[32] Ronald W. Reagan National Defense Authorization Act for Fiscal
Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct.
28, 2004) (codified in part at 10 U.S.C. § 2222).
[33] GAO, DOD Business Systems Modernization: Important Progress Made
in Establishing Foundational Architecture Products and Investment
Management Practices, but Much Work Remains, GAO-06-219 (Washington,
D.C.: Nov. 23, 2005).
[34] GAO-06-658.
[35] Ronald W. Reagan National Defense Authorization Act for Fiscal
Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct.
28, 2004) (codified in part at 10 U.S.C. § 2222).
[36] GAO-07-229T.
[37] GAO-06-658.
[38] According to DOD, the BEA stakeholders include the core business
mission areas through the Business Enterprise Priorities, which
comprises Personnel Visibility, Acquisition Visibility, Common Supplier
Engagement, Materiel Visibility, Real Property Accountability, and
Financial Visibility. The department added that as the BEA evolves, the
stakeholders will include components that must federate their
architectures to the BEA, program managers who must comply with the
BEA, IRBs who use the BEA to guide and constrain investments, and
systems designers and integrators who must build and configure their
systems to comply with the BEA.
[39] GAO-06-658.
[40] GAO-06-219.
[41] The United States Standard General Ledger provides a uniform chart
of accounts and technical guidance used in standardizing federal agency
accounting.
[42] GAO-04-777 and GAO-03-584G.
[43] GAO-06-658.
[44] Business rules are important because they explicitly translate
business policies and procedures into specific, unambiguous rules that
govern what can and cannot be done.
[45] GAO-06-831.
[46] GAO-06-831.
[47] GAO-07-451.
[48] GAO-07-451.
[49] GAO-06-658.
[50] The time-phased milestones refer to milestones, such as initial
operating capability, full operating capability, technology development
phase, and system development and demonstration phase.
[51] GAO-06-658.
[52] DOD included system and budget information for the Defense
Financial and Accounting Service and Defense Logistics Agency in the
transition plan. DOD did not include this information for the following
defense agencies: (1) Missile Defense Agency, (2) Defense Advanced
Research Projects Agency, (3) Defense Commissary Agency, (4) Defense
Contract Audit Agency, (5) Defense Contract Management Agency, (6)
Defense Information Systems Agency, (7) Defense Intelligence Agency,
(8) Defense Legal Services Agency, (9) Defense Security Cooperation
Agency, (10) Defense Security Service, (11) Defense Threat Reduction
Agency, (12) National Geospatial-Intelligence Agency, and (13) National
Security Agency.
[53] DOD included system and budget information for the Transportation
Command in the transition plan. DOD did not include this information
for the (1) Central Command, (2) Joint Forces Command, (3) Pacific
Command, (4) Southern Command, (5) Space Command, (6) Special
Operations Command, (7) European Command, and (8) Strategic Command.
[54] GAO-06-219.
[55] GAO-03-584G and CIO Council, A Practical Guide to Federal
Enterprise Architecture, Version 1.0 (February 2001).
[56] NCES is intended to provide capabilities that are key to enabling
ubiquitous access to reliable decision-quality information. NCES
capabilities can be packaged into four product lines: service-oriented
architecture foundation (e.g., security and information assurance),
collaboration (e.g., application sharing), content discovery and
delivery (e.g., delivering information across the enterprise), and
portal (e.g., user-defined Web-based presentation).
[57] GAO-06-658.
[58] Enterprise application integration software is a commercial
software product, commonly referred to as middleware, to permit two or
more incompatible systems to exchange data from different databases.
[59] 40 U.S.C. § 11312.
[60] GAO-06-658.
[61] GAO-07-538.
[62] GAO-06-658.
[63] The Enterprise Information Environment Mission Area enables the
functions of the other mission areas (e.g., Warfighting Mission Area,
Business Mission Area, and Defense Intelligence Mission Area) and
encompasses communications, computing, and core enterprise service
systems, equipment, or software that provide a common information
capability or service for enterprise use.
[64] GAO-07-538.
[65] The Joint Capabilities Integration and Development System is a
need-driven management system used to identify future capabilities for
DOD; the Planning, Programming, Budgeting, and Execution process is a
calendar-driven management system for allocating resources and is
comprised of four phases--planning, programming, budgeting, and
executing--that define how budgets for each DOD component and the
department as a whole are created, vetted, and executed; and the
Defense Acquisition System is an event-driven system for managing
product development and procurement and guides the acquisition process
for DOD.
[66] GAO-07-538.
[67] GAO-07-538.
[68] Approval authorities (the USD(AT&L); the Under Secretary of
Defense (Comptroller); the Under Secretary of Defense for Personnel and
Readiness; the ASD(NII)/CIO; and the Deputy Secretary of Defense or an
Under Secretary of Defense, as designated by the Secretary of Defense)
are responsible for the review, approval, and oversight of business
systems and must establish investment review processes for systems
under their cognizance.
[69] A key condition identified in the act includes certification by
designated approval authorities that the defense business system
modernization is (1) in compliance with the enterprise architecture;
(2) necessary to achieve critical national security capability or
address a critical requirement in an area such as safety or security;
or (3) necessary to prevent a significant adverse effect on a project
that is needed to achieve an essential capability, taking into
consideration the alternative solutions for preventing such an adverse
effect.
[70] 31 U.S.C. § 1341(a)(1)(A); see 10 U.S.C. § 2222(b).
[71] According to Air Force officials, tier 5 systems only spend
current service funds.
[72] GAO-04-394G.
[73] GAO-06-658.
[74] GAO-03-584G.
[75] GAO-06-658.
[76] GAO-03-458.
[77] Ronald W. Reagan National Defense Authorization Act for Fiscal
Year 2005, Pub. L. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. § 2222).
[78] GAO, Business Systems Modernization: DOD Continues to Improve
Institutional Approach, but Further Steps Needed, GAO-06-658
(Washington, D.C.: May 15, 2006).
[79] GAO, Business Systems Modernization: Strategy for Evolving DOD's
Business Enterprise Architecture Offers Conceptual Approach, but
Execution Details Needed, GAO-07-451 (Washington, D.C.: Apr. 16, 2007);
and Enterprise Architecture: Leadership Remains Key to Establishing and
Leveraging Architectures for Organizational Transformation, GAO-06-831
(Washington, D.C.: Aug. 14, 2006).
[80] GAO, Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments, GAO-
07-538 (Washington, D.C.: May 11, 2007).
[81] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-
584G (Washington, D.C.: April 2003).
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400:
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800:
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
