Business Systems Modernization
DOD Needs to Fully Define Policies and Procedures for Institutionally Managing Investments Gao ID: GAO-07-538 May 11, 2007In 1995, GAO first designated the Department of Defense's (DOD) business systems modernization program as "high-risk," and continues to do so today. In 2004, Congress passed legislation reflecting prior GAO recommendations for DOD to adopt a corporate approach to information technology (IT) business system investment management. To support GAO's legislative mandate to review DOD's efforts, GAO assessed whether the department's corporate investment management approach comports with relevant federal guidance. In doing so, GAO applied its IT Investment Management framework and associated methodology, focusing on the framework's stages related to the investment management provisions of the Clinger-Cohen Act of 1996.
DOD has established the management structures needed to effectively manage its business system investments, but it has not fully defined many of the related policies and procedures that GAO's IT Investment Management framework defines. Specifically, the department has defined four of nine practices that call for project-level policies and procedures, and one of the five practices that call for portfolio-level policies and procedures. For example, DOD has established an enterprisewide IT investment board responsible for defining and implementing its business system investment governance process, documented policies and procedures for ensuring that systems support ongoing and future business needs, developed procedures for identifying and collecting information about these systems to support investment selection and control, and assigned responsibility to an individual or a group for managing the development and modification of the business system portfolio selection criteria. However, DOD has not fully documented business system investment policies and procedures for directing investment board operations, selecting new investments, reselecting ongoing investments, integrating the investment funding and the investment selection processes, and developing and maintaining a complete business system investment portfolio(s). Regarding project-level investment management practices, DOD officials said that these are performed at the component level, and that departmental policies and procedures established for overseeing components' execution of these practices are sufficient. For portfolio-level practices, however, these officials stated that they intend to improve departmental policies and procedures for business system investments by, for example, establishing a single governance structure, but plans or time frames for doing so have not been established. Until DOD fully defines departmentwide policies and procedures for both individual projects and portfolios of projects, it risks selecting and controlling these business system investments in an inconsistent, incomplete, and ad hoc manner, which in turn reduces the chances that these investments will meet mission needs in the most cost-effective manner.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-538, Business Systems Modernization: DOD Needs to Fully Define Policies and Procedures for Institutionally Managing Investments
This is the accessible text file for GAO report number GAO-07-538
entitled 'Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments' which
was released on May 14, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
May 2007:
Business Systems Modernization:
DOD Needs to Fully Define Policies and Procedures for Institutionally
Managing Investments:
GAO-07-538:
GAO Highlights:
Highlights of GAO-07-538, a report to congressional committees
Why GAO Did This Study:
In 1995, GAO first designated the Department of Defense‘s (DOD)
business systems modernization program as ’high-risk,“ and continues to
do so today. In 2004, Congress passed legislation reflecting prior GAO
recommendations for DOD to adopt a corporate approach to information
technology (IT) business system investment management. To support GAO‘s
legislative mandate to review DOD‘s efforts, GAO assessed whether the
department‘s corporate investment management approach comports with
relevant federal guidance. In doing so, GAO applied its IT Investment
Management framework and associated methodology, focusing on the
framework‘s stages related to the investment management provisions of
the Clinger-Cohen Act of 1996.
What GAO Found:
DOD has established the management structures needed to effectively
manage its business system investments, but it has not fully defined
many of the related policies and procedures that GAO‘s IT Investment
Management framework defines. Specifically, the department has defined
four of nine practices that call for project-level policies and
procedures, and one of the five practices that call for portfolio-level
policies and procedures (see below). For example, DOD has established
an enterprisewide IT investment board responsible for defining and
implementing its business system investment governance process,
documented policies and procedures for ensuring that systems support
ongoing and future business needs, developed procedures for identifying
and collecting information about these systems to support investment
selection and control, and assigned responsibility to an individual or
a group for managing the development and modification of the business
system portfolio selection criteria. However, DOD has not fully
documented business system investment policies and procedures for
directing investment board operations, selecting new investments,
reselecting ongoing investments, integrating the investment funding and
the investment selection processes, and developing and maintaining a
complete business system investment portfolio(s).
Regarding project-level investment management practices, DOD officials
said that these are performed at the component level, and that
departmental policies and procedures established for overseeing
components‘ execution of these practices are sufficient. For portfolio-
level practices, however, these officials stated that they intend to
improve departmental policies and procedures for business system
investments by, for example, establishing a single governance
structure, but plans or time frames for doing so have not been
established. Until DOD fully defines departmentwide policies and
procedures for both individual projects and portfolios of projects, it
risks selecting and controlling these business system investments in an
inconsistent, incomplete, and ad hoc manner, which in turn reduces the
chances that these investments will meet mission needs in the most cost-
effective manner.
Table: Policies and Procedures for Project-level and Portfolio-Level
Management:
Stage 2: Building the investment foundation: Instituting the investment
board;
Key practices executed: 1/2;
Stage 3: Developing a complete investment portfolio: Defining the
portfolio criteria;
Key practices executed: 1/2.
Stage 2: Building the investment foundation: Meeting business needs;
Key practices executed: 1/1;
Stage 3: Developing a complete investment portfolio: Creating the
portfolio;
Key practices executed: 0/1.
Stage 2: Building the investment foundation: Selecting an investment;
Key practices executed: 0/3;
Stage 3: Developing a complete investment portfolio: Evaluating the
portfolio;
Key practices executed: 0/1.
Stage 2: Building the investment foundation: Providing investment
oversight;
Key practices executed: 0/1;
Stage 3: Developing a complete investment portfolio: Conducting
postimplementation reviews;
Key practices executed: 0/1.
Stage 2: Building the investment foundation: Capturing investment
information;
Key practices executed: 2/2;
Stage 3: Developing a complete investment portfolio: [Empty];
Key practices executed: [Empty].
Stage 2: Building the investment foundation: Overall;
Key practices executed: 4/9;
Stage 3: Developing a complete investment portfolio: Overall;
Key practices executed: 1/5.
Source: GAO.
[End of table]
What GAO Recommends:
GAO recommends that DOD fully define the project and portfolio
management policies and procedures discussed in GAO‘s framework. DOD
agreed with GAO‘s overall conclusions and partially agreed with five of
GAO‘s recommendations. However, DOD disagreed with the remaining four
recommendations, stating that the department is, among other things,
already meeting the intent of these recommendations. GAO does not
agree; its recommendations focus on fully defining policies and
procedures that satisfy key practices in its framework.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-538].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite at (202)
512-3439 or hiter@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD Has Established the Structures Needed to Effectively Manage
Business System Investments, but Has Not Fully Defined Many of the
Related Policies and Procedures:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objective, Scope, and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: DOD Business Investment Management System Entities' Roles,
Responsibilities, and Composition:
Table 2: DOD's Investment Tiers:
Table 3: Stage 2 Critical Processes--Building the Investment
Foundation:
Table 4: Summary of Policies and Procedures for Stage 2 Critical
Processes--Building the Investment Foundation:
Table 5: Stage 3 Critical Processes--Developing a Complete Investment
Portfolio:
Table 6: Summary of Policies and Procedures for Stage 3 Critical
Processes--Developing a Complete Investment Portfolio:
Figures:
Figure 1: Simplified DOD Organizational Structure:
Figure 2: The Five ITIM Stages of Maturity with Critical Processes:
Figure 3: Working Relationships among DOD Business Investment
Management System Governance Entities:
Figure 4: Simplified Process Flow of Certification Reviews and
Approvals:
Figure 5: Simplified Process Flow of Annual Reviews:
Abbreviations:
ASD(NII)/CIO: Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer:
BEA: business enterprise architecture:
BMA: business mission area:
BTA: Business Transformation Agency:
DAS: Defense Acquisition System:
DBSAE: Defense Business Systems Acquisition Executive:
DBSMC: Defense Business Systems Management Committee:
DITPR: DOD Information Technology Portfolio Repository:
DOD: Department of Defense:
IRB: Investment Review Board:
IT: information technology:
ITIM: Information Technology Investment Management framework:
JCIDS: Joint Capabilities Integration and Development System:
MAIS: Major Automated Information System:
MDAP: Major Defense Acquisition Programs:
OMB: Office of Management and Budget:
OSD: Office of the Secretary of Defense:
PCA: pre-certification authority:
PPBE: Planning, Programming, Budgeting, and Execution:
USD(AT&L): Under Secretary of Defense (Acquisition, Technology, and
Logistics):
United States Government Accountability Office:
Washington, DC 20548:
May 11, 2007:
Congressional Committees:
For decades, the Department of Defense (DOD) has been challenged in
modernizing its timeworn business systems.[Footnote 1] In 1995, we
designated DOD's business systems modernization program as high risk,
and we continue to designate it as such today.[Footnote 2] As our
research on public and private sector organizations shows, one
essential ingredient to a successful systems modernization program is
having an effective institutional approach to managing information
technology (IT) investments.
In May 2001, we recommended that the department establish a corporate
approach to investment control and decision making.[Footnote 3] Between
2001 and 2005, we reported that the department's business systems
modernization program was still not being effectively managed,[Footnote
4] and we made additional investment-related recommendations. Congress
subsequently included provisions in the Ronald W. Reagan National
Defense Authorization Act for Fiscal Year 2005[Footnote 5] that
reflected our recommendations, including those for establishing and
implementing effective business system investment management structures
and processes.
Between 2005 and 2006,[Footnote 6] we reported that DOD had made
important progress in establishing and implementing these structures
and processes, but that much remained to be accomplished relative to
the act's requirements. For example, we reported that the department's
business system investment approach was not institutionalized at all
levels of the department.
To support GAO's legislative mandate to review DOD's annual report on
its business systems modernization program, and as agreed with your
offices, the objective of this review was to determine whether DOD's
corporate investment management approach comports with relevant federal
guidance. To accomplish our objective, we analyzed documents and
interviewed agency officials to determine whether DOD has developed the
structures, policies, and procedures associated with executing those
key practices in our IT Investment Management (ITIM) framework that
assist organizations in complying with the investment management
provisions of the Clinger-Cohen Act of 1996.[Footnote 7] This framework
provides a hierarchical maturity model for IT investment management and
a method for evaluating and assessing the maturity of an agency's
investment management. We performed our work at DOD headquarters in
Arlington, Virginia, from August 2006 through April 2007 in accordance
with generally accepted government auditing standards. Details on our
objective, scope, and methodology are contained in appendix I.
Results in Brief:
DOD has established the management structures needed to effectively
manage its business system investments, but it has not fully defined
many of the related policies and procedures that our framework defines.
Specifically, DOD has fully defined four of nine key practices that
call for project-level policies and procedures, and one of the five
practices that call for portfolio-level policies and procedures. For
example, regarding project-level investment, the department has (1)
established an enterprisewide investment board and subordinate boards
that are responsible for business system investment governance, (2)
documented policies and procedures for ensuring that systems support
ongoing and future business needs, (3) developed procedures for
identifying and collecting information about these systems to support
investment selection and control, and (4) assigned responsibility for
ensuring that the information collected during project identification
meets the needs of the investment management process. Regarding
portfolio-based investment, DOD has assigned responsibility to the
Under Secretary of Defense for Acquisition, Technology, and Logistics
for managing business system portfolio selection criteria.
However, DOD has not fully documented business system investment
policies and procedures related to five key project-level management
practices. For example, policies and procedures do not (1) define how
the investment selection, acquisition, and funding processes are
coordinated; (2) specify how the full range of cost, schedule, and
benefit data accessible by the Investment Review Boards (IRB) are to be
used in making selection (i.e., certification) decisions; (3) specify
how reselection decisions at the corporate level (i.e., annual review
decisions) consider investments that are in operations and maintenance;
(4) describe how funding decisions are integrated with the process of
selecting an investment at the corporate level; and (5) provide
sufficient oversight and visibility into component-level investment
management activities, including component reviews of systems in
operations and maintenance. Furthermore, DOD does not have documented
policies and procedures for (1) defining the portfolio criteria, (2)
creating the portfolio, (3) evaluating the portfolio, and (4)
conducting postimplementation reviews for all business systems.
Regarding project-level investment management practices, DOD officials
stated that these are performed at the component level, and that
departmental policies and procedures established for overseeing
execution of these practices by components are sufficient. Regarding
portfolio-level practices, however, these officials stated that they
intend to improve departmental policies and procedures for business
system investments by, for example, establishing a single governance
structure, but plans or time frames for doing so have not been
established. According to our ITIM framework, adequately documenting
both the policies and the associated procedures that govern how an
organization manages its IT investment portfolio(s) is important
because doing so provides the basis for having rigor, discipline, and
repeatability in how investments are selected and controlled across the
entire organization. Until DOD fully defines departmentwide policies
and procedures for both individual projects and portfolios of projects,
it risks selecting and controlling these business system investments in
an inconsistent, incomplete, and ad hoc manner, which in turn reduces
the chances that these investments will meet mission needs in the most
cost-effective manner.
To strengthen DOD's business system investment management capability,
we are recommending that the department fully define the policies and
procedures associated with project-level and portfolio-level investment
management as discussed in our guidance for IT investment
management.[Footnote 8]
In written comments on a draft of this report, signed by the Deputy
Under Secretary of Defense (Business Transformation) and reprinted in
appendix II, the department stated that it agreed with the report's
overall conclusions, and it described efforts under way and planned
that it said would address many of the gaps identified in the report.
In this regard, the department partially concurred with five of the
report's recommendations, adding that our recommendations and feedback
are helpful in guiding DOD's business transformation and related
improvement efforts.
However, the department disagreed with the remaining four
recommendations for two primary reasons. First, it stated that its
existing investment management structure already satisfies the intent
of these recommendations. For example, it stated that its policies
already require the provision of cost, schedule, and funding data as
part of investment certifications and annual reviews, and that a
linkage currently exists among the investment selection, acquisition,
and funding processes. We do not agree with this reasoning. Our
recommendations are not intended to address whether existing policies
or guidance provide for the use of cost, schedule, and funding data, or
whether they state that investment selection, acquisition, and funding
decision making are linked. Rather, our recommendations address the
definitions of policy, guidance, and supporting procedures that fall
short of satisfying the best practices embodied in our ITIM framework.
In the case of the above examples, while we do not question whether
investment data are provided to investment decision-making bodies, the
department's policies and procedures do not include specific decision
criteria that explain how these data are to be used to make consistent,
repeatable selection and reselection decisions across all investments.
Furthermore, while we do not question that existing guidance contains
an illustration depicting a link between investment certification and
review and other DOD decision support processes, including the funding
process, neither this guidance nor supporting procedures define how
this linkage is executed (i.e., how investment funding decisions are in
fact integrated with investment selection decisions).
Second, DOD stated that our recommendations contradict the department's
"tiered accountability" approach to investment management, in which
responsibility and accountability for business system investment
management is allocated between the Office of the Secretary of Defense
(corporate level) and DOD components (subsidiary levels) on the basis
of investment size and significance. We do not agree with the
department's reasoning. We support DOD's tiered accountability concept
because it is consistent with the hierarchical investment structures
described in our ITIM framework. Under the department's current
policies and guidance, however, most DOD investments are not subject to
corporate visibility and oversight, either because they do not involve
development/modernization (i.e., they are in operations and
maintenance) or because they do not exceed a certain dollar threshold.
Our framework recognizes that effective implementation of this concept
should include appropriate corporate visibility into and oversight of
investments, either through review and approval of those investments
that meet certain criteria or through awareness of a subordinate
board's investment management activities. Moreover, this visibility and
oversight should extend to the entire portfolio of investments,
including those that are in operations and maintenance. To ensure that
this occurs, applicable policies and procedures need to explicitly
cover all such investments and need to define how this is to be
accomplished.
Background:
DOD is a massive and complex organization. To illustrate, the
department reported that its fiscal year 2006 operations involved
approximately $1.4 trillion in assets and $2.0 trillion in liabilities,
more than 2.9 million military and civilian personnel, and $581 billion
in net cost of operations. To date, for fiscal year 2007, the
department received appropriations of about $501 billion.
Organizationally, the department includes the Office of the Secretary
of Defense (OSD), the Chairman of the Joint Chiefs of Staff, the
military departments, numerous defense agencies and field activities,
and various unified combatant commands that are responsible for either
specific geographic regions or specific functions. (See fig. 1 for a
simplified depiction of DOD's organizational structure.)
Figure 1: Simplified DOD Organizational Structure
[See PDF for Image]
Source: GAO based on DOD documentation.
[A] The Chairman of the Joint Chiefs of Staff serves as the spokesman
for the commanders of the combatant commands, especially on the
administrative requirements of their commands.
[End of figure]
In support of its military operations, the department performs an
assortment of interrelated and interdependent business functions,
including logistics management, procurement, health care management,
and financial management. As we have previously reported,[Footnote 9]
the systems environment that supports these business functions is
overly complex and error-prone, and is characterized by (1) little
standardization across the department, (2) multiple systems performing
the same tasks, (3) the same data stored in multiple systems, and (4)
the need for data to be entered manually into multiple systems.
Moreover, according to DOD, this systems environment is comprised of
approximately 3,100 separate business systems. For fiscal year 2007,
Congress appropriated approximately $15.7 billion to DOD, and for
fiscal year 2008, DOD has requested about $15.9 billion in appropriated
funds to operate, maintain, and modernize these business systems and
the associated infrastructures.
As we have previously reported,[Footnote 10] the department's
nonintegrated and duplicative systems impair DOD's ability to combat
fraud, waste, and abuse. In fact, DOD currently bears responsibility,
in whole or in part, for 15 of our 27 high-risk areas.[Footnote 11]
Eight of these areas are specific to DOD,[Footnote 12] and the
department shares responsibility for 7 other governmentwide high-risk
areas.[Footnote 13] DOD's business systems modernization is one of the
high-risk areas, and it is an essential enabler to addressing many of
the department's other high-risk areas. For example, modernized
business systems are integral to the department's efforts to address
its financial, supply chain, and information security management high-
risk areas.
IT Investment Management Is Critical to Achieving Successful Systems
Modernization:
A corporate approach to IT investment management is characteristic of
successful public and private organizations. Recognizing this, Congress
enacted the Clinger-Cohen Act of 1996,[Footnote 14] which requires the
Office of Management and Budget (OMB) to establish processes to
analyze, track, and evaluate the risks and results of major capital
investments in IT systems made by executive agencies.[Footnote 15] In
response to the Clinger-Cohen Act and other statutes, OMB has developed
policy and issued guidance for the planning, budgeting, acquisition,
and management of federal capital assets.[Footnote 16] We have also
issued guidance in this area,[Footnote 17] which defines institutional
structures, such as the IRBs; processes for developing information on
investments (such as costs and benefits); and practices to inform
management decisions (such as whether a given investment is aligned
with an enterprise architecture).
IT Investment Management: A Brief Description:
IT investment management is a process for linking IT investment
decisions to an organization's strategic objectives and business plans.
Consistent with this, the federal approach to IT investment management
focuses on selecting, controlling, and evaluating investments in a
manner that minimize risks while maximizing the return of
investment.[Footnote 18]
* During the selection phase, the organization (1) identifies and
analyzes each project's risks and returns before committing significant
funds to any project and (2) selects those IT projects that will best
support its mission needs.
* During the control phase, the organization ensures that projects, as
they develop and investment expenditures continue, meet mission needs
at the expected levels of cost and risk. If the project is not meeting
expectations or if problems arise, steps are quickly taken to address
the deficiencies.
* During the evaluation phase, expected results are compared with
actual results after a project has been fully implemented. This
comparison is done to (1) assess the project's impact on mission
performance, (2) identify any changes or modifications to the project
that may be needed, and (3) revise the investment management process
based on lessons learned.
Overview of GAO's ITIM Maturity Framework:
Our ITIM framework consists of five progressive stages of maturity for
any given agency relative to selecting, controlling, and evaluating its
investment management capabilities.[Footnote 19] (See fig. 2 for the
five ITIM stages of maturity.) This framework is grounded in our
research of IT investment management practices of leading private and
public sector organizations. The maturity stages are cumulative; that
is, to attain a higher stage, an agency must institutionalize all of
the critical processes at the lower stages, in addition to the higher
stage critical processes.
The framework can be used to assess the maturity of an agency's
investment management processes and as a tool for organizational
improvement. The overriding purpose of the framework is to encourage
investment selection and control and to evaluate processes that promote
business value and mission performance, reduce risk, and increase
accountability and transparency. We have used the framework in several
of our evaluations,[Footnote 20] and a number of agencies have adopted
it.
With the exception of the first stage, each maturity stage is composed
of "critical processes" that must be implemented and institutionalized
for the organization to achieve that stage. Each ITIM critical process
consists of "key practices"--to include organizational structures,
policies, and procedures--that must be executed to implement the
critical process. It is not unusual for an organization to perform key
practices from more than one maturity stage at the same time. However,
our research shows that agency efforts to improve investment management
capabilities should focus on implementing all lower-stage practices
before addressing higher-stage practices.
In the ITIM framework, Stage 2 critical processes lay the foundation by
establishing successful, predictable, and repeatable investment control
processes at the project level. At this stage, the emphasis is on
establishing basic capabilities for selecting new IT projects;
controlling projects so that they finish predictably within the
established cost, schedule, and performance expectations; and
identifying and mitigating exposure to risk.
Stage 3 is where the agency moves from project-centric processes to
portfolio-based processes and evaluates potential investments according
to how well they support the agency's missions, strategies, and goals.
This stage focuses on continually assessing both proposed and ongoing
projects as part of complete investment portfolios-- integrated and
competing sets of investment options. It also focuses on maintaining
mature, integrated selection (and reselection); control; and
postimplementation evaluation processes. This portfolio perspective
allows decision makers to consider the interaction among investments
and the contributions to organizational mission goals and strategies
that could be made by alternative portfolio selections, rather than to
focus exclusively on the balance between the costs and benefits of
individual investments. Organizations implementing Stages 2 and 3
practices have in place capabilities that assist in establishing
selection, control, and evaluation structures, policies, procedures,
and practices that are required by the investment management provisions
of the Clinger-Cohen Act.[Footnote 21]
Stages 4 and 5 require the use of evaluation techniques to continuously
improve both investment processes and portfolios to better achieve
strategic outcomes. At Stage 4, an organization has the capacity to
conduct IT succession activities and, therefore, can plan and implement
the deselection of obsolete, high-risk, or low-value IT investments. An
organization with Stage 5 maturity conducts proactive monitoring for
breakthrough technologies that will enable it to change and improve its
business performance.
Figure 2: The Five ITIM Stages of Maturity with Critical Processes:
[See PDF for image]
Source: GAO.
[End of figure]
Overview of DOD's Corporate Approach for Identifying, Funding, and
Acquiring All System Investments:
DOD's major system investments (i.e., weapon and business systems) are
governed by three management systems--the Joint Capabilities
Integration and Development System (JCIDS); the Planning, Programming,
Budgeting, and Execution (PPBE) system; and the Defense Acquisition
System (DAS).
* JCIDS is a need-driven, capabilities-based approach to identify
warfighting needs and meet future joint forces challenges. It is
intended to identify future capabilities for DOD; address capability
gaps and mission needs recognized by the Joint Chiefs of Staff or
derived from strategic guidance, such as the National Security Strategy
Report[Footnote 22] or Quadrennial Defense Review;[Footnote 23] and
identify alternative solutions by considering a range of doctrine,
organization, training, materiel, leadership and education, personnel,
and facilities solutions. According to DOD, the Joint Chiefs of Staff,
through the Joint Requirements Oversight Council, has primary
responsibility for defining and implementing JCIDS.
* PPBE is a calendar-driven approach that is composed of four phases
that occur over a moving 2-year cycle. The four phases--planning,
programming, budgeting, and executing--define how budgets for each DOD
component and the department as a whole are created, vetted, and
executed. As recently reported,[Footnote 24] the components start
programming and budgeting for addressing a JCIDS-identified capability
gap or mission need several years before actual product development
under DAS begins, and before OSD formally reviews the components'
programming and budgeting proposals (i.e., Program Objective
Memorandums). Once reviewed and approved, the financial details in the
Program Objective Memorandums become part of the President's budget
request to Congress. During budget execution, components may submit
program change proposals or budget change proposals, or both (e.g.,
program cost increases or schedule delays). According to DOD, the OSD
Under Secretary of Defense (Policy), the Director for Program Analysis
and Evaluation,[Footnote 25] and the Under Secretary of Defense
(Comptroller) have primary responsibility for defining and implementing
the PPBE system.
* DAS is described in the DOD Directive 5000.1 and the DOD Instruction
5000.2[Footnote 26] and establishes the procedures for the Defense
Acquisition Management Framework, which consists of three event-based
milestones associated with five key program life-cycle phases. These
five phases are as follows:
1. Concept Refinement: Intended to refine the initial JCIDS-validated
system solution (concept) and create a strategy for acquiring the
investment solution. A decision is made at the end of this phase
(milestone A decision) regarding whether to move to the next phase
(Technology Development).
2. Technology Development: Intended to determine the appropriate set of
technologies to be integrated into the investment solution by
iteratively assessing the viability of various technologies while
simultaneously refining user requirements. Once the technology has been
demonstrated in a relevant environment, a decision is made at the end
of this phase (milestone B decision) regarding whether to move to the
next phase (System Development and Demonstration).
3. System Development and Demonstration: Intended to develop a system
or a system increment and demonstrate through developer testing that
the system/system increment can function in its target environment. A
decision is made at the end of this phase (milestone C decision)
regarding whether to move to the next phase (Production and
Deployment).
4. Production and Deployment: Intended to achieve an operational
capability that satisfies the mission needs, as verified through
independent operational test and evaluation, and ensures that the
system is implemented at all applicable locations.
5. Operations and Support: Intended to operationally sustain the system
in the most cost-effective manner over its life cycle.
A key principle of DAS is that investments are assigned a category,
where programs of increasing dollar value and management interest are
subject to more stringent oversight. For example, Major Defense
Acquisition Programs (MDAP)[Footnote 27] and Major Automated
Information Systems (MAIS)[Footnote 28] are large, expensive programs
subject to the most extensive statutory and regulatory reporting
requirements and, unless delegated, are reviewed by acquisition boards
at the DOD corporate level. Smaller and less risky acquisitions are
generally reviewed at the component executive or lower levels. Another
key principle is that DAS requires acquisition management under the
direction of a milestone decision authority.[Footnote 29] The milestone
decision authority--with support from the program manager and advisory
boards, such as the Defense Acquisition Board[Footnote 30] and the IT
Acquisition Board[Footnote 31]--determines the project's baseline cost,
schedule, and performance commitments. The Under Secretary of Defense
for Acquisition, Technology, and Logistics (USD(AT&L)) has primary
responsibility for defining and implementing DAS.
DOD Business System Investments Are Subject to a Fourth Management
System:
DOD's business system investments are also governed by a fourth
management system that addresses how these investments are reviewed,
certified, and approved for compliance with the business enterprise
priorities and activities outlined by the business enterprise
architecture (BEA). For the purposes of this report, we refer to this
fourth management system as the Business Investment Management System.
This fourth management system is described in the following text in
terms of governance entities, tiered accountability, and business
system investment certification reviews and approvals. According to
DOD, these four management systems are the means by which DOD selects,
controls, and evaluates its business system investments.
Business System Investment Roles and Responsibilities:
In 2005, the department reassigned responsibility for providing
executive leadership for the direction, oversight, and execution of its
business systems modernization efforts to several entities. These
entities and their responsibilities include the following:
* The Defense Business Systems Management Committee (DBSMC) serves as
the highest-ranking governance body for business systems modernization
activities.
* The Principal Staff Assistants serve as the certification authorities
for business system modernizations in their respective core business
missions.
* The IRBs are chartered by the Principal Staff Assistants and are the
review and decision-making bodies for business system investments in
their respective areas of responsibility.[Footnote 32]
* The component pre-certification authority (PCA) is accountable for
the component's business system investments and acts as the component's
principal point of contact for communication with the IRBs.
* The Business Transformation Agency (BTA) is responsible for leading
and coordinating business transformation efforts across the department.
The BTA is organized into seven directorates, one of which is the
Defense Business Systems Acquisition Executive (DBSAE)--the component
acquisition executive for DOD enterprise-level (DOD-wide) business
systems and initiatives. This directorate is responsible for
developing, coordinating, and integrating enterprise-level projects,
programs, systems, and initiatives--including managing resources such
as fiscal, personnel, and contracts for assigned systems and programs.
Table 1 lists these entities and provides greater detail on their
roles, responsibilities, and composition. Figure 3 provides a
simplified illustration of the relationships among these entities.
Table 1: DOD Business Investment Management System Entities' Roles,
Responsibilities, and Composition:
Entity: DBSMC;
Roles and responsibilities:
* Serves as approving authority for business system certifications;
* Establishes policies and approves the business mission area (BMA)[A]
strategic plan, the transition plan for implementation for business
systems modernization, the transformation program baseline, and the
BEA;
Composition: Chaired by the Deputy Secretary of Defense; vice chair is
the USD(AT&L). Includes senior leadership in OSD; the military
departments' secretaries; and defense agencies' heads, such as the
Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer (ASD(NII)/CIO), the Vice
Chairman of the Joint Chiefs of Staff, and the commanders of the U.S.
Transportation Command and the Joint Forces Command.
Entity: Principal Staff Assistants/; Certification Authorities;
Roles and responsibilities:
* Support the DBSMC's management of enterprise business IT investments;
* Serve as the certification authorities accountable for the obligation
of funds for respective business systems modernization within
designated core business missions.[B];
* Provide the DBSMC with recommendations for system investment
approval;
Composition: Under Secretaries of Defense for Acquisition, Technology,
and Logistics; Comptroller; and Personnel and Readiness.
Entity: IRBs;
Roles and responsibilities:
* Serve as the oversight and investment decision-making bodies for
those business capabilities that support activities under their
designated areas of responsibility;
* Recommend certification for all business system investments costing
more than $1 million that are integrated and compliant with the BEA;
Composition: Includes the Principal Staff Assistants, Joint Staff,
ASD(NII)/CIO, core business mission area representatives, military
departments, defense agencies, and combatant commands.
Entity: Component PCA;
Roles and responsibilities:
* Ensures that component-level investment review processes integrate
with the investment management system;
* Identifies those component systems that require IRB certification and
prepares, reviews, approves, validates, and transfers investment
documentation as required;
* Assesses and precertifies architecture compliance of component
systems submitted for certification and annual review;
* Acts as the component's principal point of contact for communication
with the IRBs;
Composition: Includes the Chief Information Officer from the Air Force;
the Principal Director of Governance, Acquisition, and Chief Knowledge
Office from the Army; the Chief Information Officer from the Navy; and
comparable representatives from other defense agencies.
Entity: BTA;
Roles and responsibilities:
* Serves as the day-to-day management entity of the business
transformation effort at the DOD enterprise level;
* Provides support to the DBSMC and the IRBs;
* Operates under the authority of the USD(AT&L) under the direction of
the Deputy Under Secretary of Defense for Business Transformation and
the Deputy Under Secretary of Defense for Financial Management;
Composition: Comprised of seven directorates (DBSAE, Enterprise
Integration, Transformation Planning and Performance, Transformation
Priorities and Requirements, Investment Management, Warfighter Support
Office, and Chief of Staff).
Source: GAO based on DOD documentation.
[A] According to DOD, the BMA is responsible for ensuring that
capabilities, resources, and materiel are reliably delivered to the
warfighter. Specifically, the BMA addresses areas such as real property
and human resources management.
[B] DOD has five core business missions: Human Resources Management,
Weapon System Lifecycle Management, Materiel Supply and Services
Management, Real Property and Installations Lifecycle Management, and
Financial Management.
[End of table]
Figure 3: Working Relationships Among DOD Business Investment
Management System Governance Entities:
[See PDF for Image]
Source: GAO based on DOD documentation.
[End of figure]
Tiered Accountability:
According to DOD, in 2005 it adopted a tiered accountability approach
to business transformation. Under this approach, responsibility and
accountability for business investment management is allocated between
the DOD corporate (i.e., OSD) and the components on the basis of the
amount of development/modernization funding involved and the
investment's "tier." DOD corporate is responsible for ensuring that all
business systems with a development/modernization investment in excess
of $1 million are reviewed by the IRBs for compliance with the BEA,
certified by the Principal Staff Assistants, and approved by the DBSMC.
Components are responsible for certifying development/modernization
investments with total costs of $1 million or less. All DOD development
and modernization efforts are also assigned a tier on the basis of the
acquisition category or the size of the financial investment, or both.
According to DOD, a system is given a tier designation when it passes
through the certification process. Table 2 describes the four
investment tiers and identifies the associated reviewing and approving
entities.
Table 2: DOD's Investment Tiers:
Tier 1;
Tier description: MAIS and MDAPs;
Reviewing/Approving entities: IRB and DBSMC.
Tier 2;
Tier description: Exceeding $10 million in total development/
modernization costs, but not designated MAIS or MDAPs;
Reviewing/ Approving entities: IRB and DBSMC.
Tier 3;
Tier description: Exceeding $1 million and up to $10 million in total
development/modernization costs;
Reviewing/Approving entities: IRB and DBSMC.
Tier 4;
Tier description: Investment funding required up to $1 million;
Reviewing/Approving entities: Component-level review only (unless the
system or line of business it supports is designated as special
interest by the Certification Authority).
Source: DOD.
[End of table]
Business Investment Certification Reviews and Approvals:
DOD's business investment management system includes two types of
reviews for business systems: certification and annual reviews.
Certification reviews apply to new modernization projects with total
cost over $1 million. This review focuses on program alignment with the
BEA and must be completed before components obligate funds for
programs. The annual review applies to all business programs. The focus
for the annual review is to determine whether the system development
effort is meeting its milestones and addressing its IRB certification
conditions.
Certification reviews and approvals: Tiers 1 through 3 business system
investments are certified at two levels--component-level
precertification and corporate-level certification and approval. At the
component level, program managers prepare, enter, maintain, and update
information about their investments in the DOD IT Portfolio Repository
(DITPR),[Footnote 33] such as regulatory compliance reporting, an
architectural profile, and requirements for investment certification
and annual reviews. The component PCA validates that the system
information is complete and accessible on the IRB Portal, reviews
system compliance with the BEA and enterprise transition plan, and
verifies the economic viability analysis. The PCA asserts the status
and validity of the investment information by submitting a component
precertification letter to the appropriate IRB for its review.
At the corporate level, the IRB reviews the system information and
precertification letter submitted by the PCA to determine whether to
recommend investment certification. On completion of its review, a
certification memorandum is prepared and signed by the designated
certification authority[Footnote 34] that documents the IRB's system
certification decisions and any related conditions. The memorandum is
then forwarded to the DBSMC, which either approves or disapproves the
IRB's decisions and issues a memorandum containing its decisions. If
the DBSMC disapproves a system investment, it is up to the component
PCA to decide whether to resubmit the investment after it has resolved
the relevant issues. Figure 4 provides a simplified overview of the
process flow of certification reviews and approvals.
Figure 4: Simplified Process Flow of Certification Reviews and
Approvals:
[See PDF for image]
Source: GAO based on DOD documentation.
[End of figure]
Annual reviews: Tiers 1 through 4 business system investments are
annually reviewed at two levels--the component level and the corporate
level. At the component level, program managers review and update
information on all tiers of investments, both in modernization and
operations and maintenance, on an annual basis in DITPR. The updates
for Tiers 1 through 3 with system development/modernization include
cost, milestone, and risk variances and actions or issues related to
certification conditions. The PCA then verifies and submits the
information for Tiers 1 through 3 systems in development/modernization
for IRB review in an annual review assertion letter. The letter
addresses system compliance with the BEA and the enterprise transition
plan, and includes investment cost, schedule, and performance
information.[Footnote 35]
At the corporate level, the IRBs annually review certified Tiers 1
through 3 investments in development/modernization. These reviews focus
on program compliance with the BEA, program performance against cost
and milestone baselines, and progress in meeting certification
conditions. The IRBs can revoke an investment's certification when the
system has significantly failed to achieve performance commitments
(i.e., capabilities and costs). When this occurs, the component must
address the IRB's concerns and resubmit the investment for
certification. Figure 5 shows a simplified overview of the process flow
of annual reviews.
Figure 5: Simplified Process Flow of Annual Reviews:
[See PDF for image]
Source: GAO based on DOD documentation.
[End of figure]
DOD Has Established the Structures Needed to Effectively Manage
Business System Investments, but Has Not Fully Defined Many of the
Related Policies and Procedures:
According to our ITIM framework, organizations should establish the
management structures needed to manage their investments and build an
investment foundation by having defined policies and procedures for
selecting and controlling individual projects (Stage 2 capabilities),
and organizations also should manage projects as a portfolio of
investments according to defined policies and procedures, treating them
as an integrated package of competing investment options and pursuing
those that best meet the strategic goals, objectives, and mission of
the agency (Stage 3 capabilities). These Stages 2 and 3 capabilities
assist agencies in complying with the investment management provisions
of the Clinger-Cohen Act.
The department has defined four of nine practices that call for project-
level policies and procedures (see table 4) and one of the five
practices that call for portfolio-level policies and procedures (see
table 6). Specifically, it has established the management structures
contained in our ITIM framework, but it has not fully defined many of
the related policies and procedures.
With respect to project-level investment management practices, DOD
officials stated that these are performed at the component level, and
that departmental policies and procedures established for overseeing
components' execution of these practices are sufficient. With respect
to portfolio-level practices, however, these officials stated that they
intend to improve departmental policies and procedures for business
system investments by, for example, establishing a single governance
structure, but plans or time frames for doing so have not been
established. According to our ITIM framework, adequately documenting
both the policies and the associated procedures that govern how an
organization manages its IT investment portfolio(s) is important
because doing so provides the basis for having rigor, discipline, and
repeatability in how investments are selected and controlled across the
entire organization. Until DOD fully defines departmentwide policies
and procedures for both individual projects and the portfolios of
projects, it risks selecting and controlling these business system
investments in an inconsistent, incomplete, and ad hoc manner, which in
turn reduces the chances that these investments will meet mission needs
in the most cost-effective manner.
DOD Has Begun to Build a Foundation for Project-Level Investment
Management, but Key Policies and Procedures Are Not Fully Defined:
At ITIM Stage 2, an organization has attained repeatable and successful
IT project-level investment control and basic selection processes.
Through these processes, the organization can identify project
expectation gaps early and take the appropriate steps to address them.
ITIM Stage 2 critical processes include (1) defining investment board
operations, (2) identifying the business needs for each investment, (3)
developing a basic process for selecting new proposals and reselecting
ongoing investments, (4) developing project-level investment control
processes, and (5) collecting information about existing investments to
inform investment management decisions. Table 3 describes the purpose
of each of these Stage 2 critical processes.
Table 3: Stage 2 Critical Processes--Building the Investment
Foundation:
Critical process: Instituting the investment board;
Purpose: To define and establish an appropriate investment management
structure and the processes for selecting, controlling, and evaluating
investments.
Critical process: Meeting business needs;
Purpose: To ensure that investments support the organization's business
needs and meet users' needs.
Critical process: Selecting an investment;
Purpose: To ensure that a well-defined and disciplined process is used
to select new proposals and reselect ongoing investments.
Critical process: Providing investment oversight;
Purpose: To review the progress of investments, using predefined
criteria and checkpoints, in meeting cost, schedule, risk, and benefit
expectations and to take corrective action when these expectations are
not being met.
Critical process: Capturing investment information;
Purpose: To make available to decision makers information to evaluate
the impacts and opportunities created by proposed (or continuing)
investments.
Source: GAO.
[End of table]
Within these five critical processes are nine key practices that call
for policies and procedures associated with effective project-level
management. DOD has fully defined the policies and procedures needed to
ensure that four of these nine practices are performed in a consistent
and repeatable manner. Specifically, DOD has established the management
structures by instituting an enterprisewide investment board--the
DBSMC--composed of senior executives, including the Deputy Secretary of
Defense, with final approval authority over associated subsidiary
investment boards. These lower-level investment boards include
representatives from combatant commands, components, and the Joint
Chiefs of Staff. In addition, DOD's business transformation and IRB
guidance define a process for ensuring that programs support the
department's ongoing and future business needs. DOD also has policies
and procedures for submitting, updating, and maintaining investment
information in DITPR and the IRB Portal. Furthermore, the department
has assigned the component's PCA the responsibility to ensure that
specific investment information contained in the portfolio repository
and the IRB Portal is accurate and complete.
However, the policies and procedures associated with the remaining five
project-level management practices are missing critical elements needed
to effectively carry out essential investment management activities.
For example:
* Policies and procedures for instituting the investment board do not
address how investments that are past the development/modernization
stage (i.e., in operations and maintenance) are to be governed. Given
that DOD invests billions of dollars annually in operating and
maintaining business systems, this is significant. While DOD officials
stated that component-level policies and procedures address systems
outside of development/modernization, our ITIM framework emphasizes
that the corporate investment boards should continue to review
important information about an investment, such as cost and performance
baselines, throughout the investment's life cycle. In addition, the IRB
Concept of Operations and other IRB documentation do not explicitly
outline how the business investment management system is coordinated
with JCIDS, PPBE, and DAS. Without clearly defined visibility into all
investments with an understanding of decisions reached through other
management systems, inconsistent decisions may result.
* Procedures do not specify how the full range of cost, schedule, and
benefit data is used by the IRBs in making selection (i.e.,
certification) decisions. According to BTA officials, each IRB decides
how to ensure compliance and determines additional factors to consider
when making certification decisions. However, DOD did not provide us
with any supplemental policies or procedures for any of the four IRBs.
Without documenting how IRBs consider factors such as cost, schedule,
and benefits when making selection decisions, the department cannot
ensure that the IRBs and the DBSMC consistently and objectively select
proposals that best meet the department's needs and priorities.
Furthermore, while the procedures specify decision criteria that
address statutory requirements for alignment to the BEA, the criteria
allow programs to postpone demonstrating full compliance with several
BEA artifacts until the final phases of the acquisition process. As a
result, programs risk beginning production and deployment before
ensuring that a business system is fully aligned to the BEA.
* Policies and procedures do not specify how reselection decisions at
the corporate level (i.e., annual review decisions) consider
investments that are in operations and maintenance. Without an
understanding of how the IRBs are to consider these investments when
making reselection decisions, their ability to make informed and
consistent reselection and termination decisions is limited.
* Policies and procedures do not specify how funding decisions are
integrated with the process of selecting an investment at the corporate
level. Without considering component and corporate budget constraints
and opportunities, the IRBs risk making investment decisions that do
not effectively consider the relative merits of various projects and
systems when funding limitations exist.
* Policies and procedures do not exist that provide for sufficient
oversight and visibility into component-level investment management
activities, including component reviews of systems in operations and
maintenance and Tier 4 investments. According to DOD officials,
investment oversight is implemented through tiered accountability,
which, among other things, allocates responsibility and accountability
for business system investments with total costs of $1 million or less
and those in operations and maintenance to the components. However, the
department did not provide policies and procedures defining how the
DBSMC and the IRBs ensure visibility into these component processes.
This is particularly important because, according to DOD's March 15,
2007, annual report to Congress, only 285 of approximately 3,100 total
business systems have completed the IRB certification process and have
been approved by the DBSMC. DOD officials also stated that the
remaining business systems have not been through the certification
process and have not been given a tier designation. Without policies
and procedures defining how the DBSMC and the IRBs have visibility into
and oversight of all business system investments, DOD risks components
continuing to invest in systems that are duplicative, stovepiped,
nonintegrated, and unnecessarily costly to manage, maintain, and
operate.
Table 4 summarizes our findings relative to DOD's execution of the nine
practices that call for the policies and procedures needed to manage IT
investments at the project level.
Table 4: Summary of Policies and Procedures for Stage 2 Critical
Processes--Building the Investment Foundation:
Critical process: Instituting the investment board;
Key practice: 1. An enterprisewide IT investment board composed of
senior executives from IT and business units is responsible for
defining and implementing the organization's IT investment governance
process;
Rating: Executed;
Summary of evidence: DOD has instituted an enterprisewide business
system investment board--the DBSMC--composed of senior executives,
including the Deputy Secretary of Defense and the ASD(NII)/CIO. This
board is responsible for establishing and implementing policies
governing the organization's investment process and approving lower-
level investment board processes and procedures.
Key practice: 2. The organization has a documented IT investment
process directing each investment board's operations;
Rating: Not executed;
Summary of evidence: DOD's IRB Concept of Operations directs its IRBs
and includes the roles and responsibilities of the boards and
individuals involved. However, the concept of operations does not
assign the boards accountability for programs throughout the investment
life cycle (i.e., investments that are past the development/
modernization stage and in operations and maintenance). In addition,
according to our ITIM guidance, the department's investment process
should specify the manner in which investment-related processes will be
coordinated with other organizational plans, processes, and documents.
However, DOD's concept of operations does not specify how the business
investment management system is coordinated with JCIDS, PPBE, and DAS.
Critical process: Meeting business needs;
Key practice: 1. The organization has documented policies and
procedures for identifying IT projects or systems that support the
organization's ongoing and future business needs;
Rating: Executed;
Summary of evidence: DOD's Business Transformation Guidance and the
Investment Certification and Annual Review Process User Guidance define
a process for ensuring that IT business system investments support the
department's ongoing and future business needs.
Critical process: Selecting an investment;
Key practice: 1. The organization has documented policies and
procedures for selecting a new investment;
Rating: Not executed;
Summary of evidence: DOD has a two- stage selection process. The first
stage involves selection of systems using the JCIDS, DAS, and PPBE
management systems. At this level, proposals and alternatives are
viewed and prioritized for system selection. The second stage of
selection involves (1) certifying and approving Tiers 1 through 3
investments and (2) elevating certain component investments to an
enterprisewide status using the business investment management system;
While DOD's IRB Concept of Operations and its Investment Certification
and Annual Review Process User Guidance define the department's
corporate approach for certifying and approving investments, they do
not contain a structured method defining how certification decisions
are reached. For example, the guidance does not specify how cost,
schedule, and benefit data are to be used in making certification
decisions. According to our ITIM guidance, a structured selection
method should provide investment boards, business units, and IT
developers with a common understanding of the selection process,
including the cost, schedule, and benefit data used to compare and
select projects. In addition, neither the IRB Concept of Operations nor
the Investment Certification and Annual Review Process User Guidance
define the selection criteria used to elevate these investments to an
enterprisewide status; Furthermore, the BEA Compliance Guidance allows
programs to postpone demonstrating full compliance with several BEA
artifacts until the final phases of the acquisition process. In
addition, criteria for certifying compliance with the BEA are
inconsistently described in DOD documentation. For example, the BEA
Compliance Guidance provides different checkpoints for assessing
compliance during the life cycle of a program than the Business
Transformation Guidance.
Key practice: 2. The organization has documented policies and
procedures for reselecting ongoing investments;
Rating: Not executed;
Summary of evidence: DOD's IRB Concept of Operations and the Investment
Certification and Annual Review Process User Guidance define the
department's corporate approach for annually reviewing investments.
However, these documents do not include specific criteria that describe
how the IRBs make reselection decisions. For example, while DOD
officials stated that a program's risk areas (i.e., cost, schedule, and
performance) are identified and discussed by the IRB during the annual
reviews, the guidance does not specify how this information is used in
making annual review decisions. In addition, the guidance does not
provide for the reselection of investments that are in operations and
maintenance. Our ITIM guidance states that consistent qualitative and
quantitative measures are needed for analyzing a project for
reselection or, if necessary, termination. According to ITIM, the
results of this analysis can help the investment board determine the
potential risk and return of continuing to fund an ongoing project and
to prioritize projects on the basis of decision criteria.
Key practice: 3. The organization has documented policies and
procedures for integrating investment funding with investment
selection;
Rating: Not executed;
Summary of evidence: According to DOD officials and the Investment
Certification and Annual Review Process User Guidance, the IRBs are
aware of the amount of funding components have requested for a program.
However, this guidance does not specify how funding decisions are
integrated with the process of selecting an investment, and does not
specify how the DBSMC and the IRBs use this information in carrying out
decisions on system certification and approvals.
Critical process: Providing investment oversight;
Key practice: 1. The organization has documented policies and
procedures for management oversight of IT projects and systems;
Rating: Not executed;
Summary of evidence: DOD's IRB Concept of Operations and the Investment
Certification and Annual Review Process User Guidance do not provide
sufficient oversight and visibility into component-level investment
management activities, including component reviews of systems in
operations and maintenance and Tier 4 investments. For example, while
the components submit a list of systems reviewed at their levels, the
list lacks important project information, including adherence to cost,
schedule, and risk criteria. According to ITIM, to maintain adequate
oversight, the investment board should have visibility into each
project's performance and progress toward predefined cost and schedule
expectations as well as each project's anticipated benefits and risk
exposure. In addition, IRB policies and procedures do not define how
the department's management systems, JCIDS, PPBE, and DAS, are related.
Critical process: Capturing investment information;
Key practice: 1. The organization has documented policies and
procedures for identifying and collecting information about IT projects
and systems to support the investment management process;
Rating: Executed;
Summary of evidence: DOD's Investment Certification and Annual Review
Process User Guidance describes the procedures for submitting,
updating, and maintaining information in DITPR and the IRB Portal, both
of which support the business investment management system.
Key practice: 2. An official is assigned responsibility for ensuring
that the information collected during project and systems
identification meets the needs of the investment management process;
Rating: Executed;
Summary of evidence: DOD's Investment Certification and Annual Review
Process User Guidance assigns the component PCA the responsibility to
ensure investment information contained in DITPR and the IRB Portal is
accurate and complete. The guidance also assigns IRB staff
responsibility for verifying these data.
Source: GAO.
[End of table]
According to BTA officials, the IRB Concept of Operations and the
Investment Certification and Annual Review Process User Guidance are
not intended to describe the detailed approach that each IRB will use
when making certification decisions, adding that the components are
responsible for selection, annual review, budgeting, and acquisition.
While the ITIM framework does allow for multiple entities to carry out
investment selection, control, and evaluation, building a sound
investment foundation requires that the enterprisewide investment
review board has documented criteria and decision-making procedures,
clear integration among investment decision-support systems, and
policies to ensure board access to system information throughout the
life cycle for all investments. Until DOD's documented IT investment
management policies and procedures include fully defined policies and
procedures for Stage 2 activities, specify the linkages between the
various related processes, and describe how investments are to be
governed in the operations and maintenance phase, DOD risks that
investment management activities will not be carried out consistently
and in a disciplined manner. Moreover, DOD also risks selecting
investments that will not cost-effectively meet its mission needs.
DOD Has Assigned Responsibility, but Has Not Defined the Policies and
Procedures Associated with Effective Portfolio-Level Management:
At Stage 3, an organization has defined critical processes for managing
its investments as a portfolio or set of portfolios.[Footnote 36]
Portfolio management is a conscious, continuous, and proactive approach
to allocating limited resources among competing initiatives in light of
the investments' relative benefits. Taking an agencywide perspective
enables an organization to consider its investments comprehensively, so
that collectively the investments optimally address the organization's
missions, strategic goals, and objectives. Managing IT investments as
portfolios also allows an organization to determine its priorities and
make decisions about which projects to fund on the basis of analyses of
the relative organizational value and risks of all projects, including
projects that are proposed, under development, and in operation.
Although investments may initially be organized into subordinate
portfolios--on the basis of, for example, business lines or life-cycle
stages--and managed by subordinate investment boards, they should
ultimately be aggregated into enterprise-level portfolios.
According to ITIM, Stage 3 involves (1) defining the portfolio
criteria; (2) creating the portfolio; (3) evaluating (i.e., overseeing)
the portfolio; and (4) conducting postimplementation reviews. Table 5
summarizes the purpose of each of these activities.
Table 5: Stage 3 Critical Processes--Developing a Complete Investment
Portfolio:
Critical process: Defining the portfolio criteria;
Purpose: To ensure that the organization develops and maintains
portfolio selection criteria that support its mission, organizational
strategies, and business priorities.
Critical process: Creating the portfolio;
Purpose: To ensure that investments are analyzed according to the
organization's portfolio selection criteria, and to ensure that an
optimal investment portfolio with manageable risks and returns is
selected and funded.
Critical process: Evaluating the portfolio;
Purpose: To review the performance of the organization's investment
portfolio(s) at agreed- upon intervals, and to adjust the allocation of
resources among investments as necessary.
Critical process: Conducting postimplementation reviews;
Purpose: To compare the results of recently implemented investments
with the expectations that were set for them, and to develop a set of
lessons learned from these reviews.
Source: GAO.
[End of table]
DOD is executing one of the five practices within these four critical
processes that call for policies and procedures associated with
effective portfolio-level management. Specifically, DOD has issued
departmentwide guidance[Footnote 37] that assigns responsibilities to
the USD(AT&L) for managing and establishing business system investment
portfolios, including leveraging or establishing a governance forum to
oversee these business system investment portfolio activities.
However, DOD has not fully defined the policies and procedures needed
to effectively execute the remaining four portfolio management
practices relative to business system investments. Specifically, DOD
does not have policies and procedures for defining the portfolio
criteria or for creating and evaluating the portfolio. In addition,
while DOD has policies and procedures for conducting postimplementation
reviews as part of DAS, these reviews do not address systems at all
tier levels. Furthermore, there are no procedures detailing how lessons
learned from these reviews are used during investment review as the
basis for management and process improvements.
Table 6 summarizes the rating for each critical process required to
manage investment as a portfolio and summarizes the evidence that
supports these ratings.
Table 6: Summary of Policies and Procedures for Stage 3 Critical
Processes--Developing a Complete Investment Portfolio:
Critical process: Defining the portfolio criteria;
Key practice: 1. The organization has documented policies and
procedures for creating and modifying IT portfolio selection criteria;
Rating: Not executed;
Summary of evidence: DOD's IT Portfolio Management Implementation
states that the USD(AT&L) is responsible for creating and modifying
portfolio criteria (e.g., prioritization and investment tradeoffs) for
business system investments. However, the USD(AT&L) has not documented
the related policies and procedures.
Key practice: 2. Responsibility is assigned to an individual or group
for managing the development and modification of the IT portfolio
selection criteria;
Rating: Executed;
Summary of evidence: DOD's IT Portfolio Management assigns
responsibility for the business mission area portfolio management to
the USD(AT&L), who leads and manages business system investments in
coordination with the ASD(NII)/CIO, the Under Secretary of Defense
(Comptroller), and the Under Secretary of Defense (Personnel and
Readiness).
Critical process: Creating the portfolio;
Key practice: 1. The organization has documented policies and
procedures for analyzing, selecting, and maintaining the investment
portfolios;
Rating: Not executed;
Summary of evidence: DOD does not have policies and procedures for
analyzing, selecting, and maintaining business system investment
portfolios.
Critical process: Evaluating the portfolio;
Key practice: 1. The organization has documented policies and
procedures for reviewing, evaluating, and improving the performance of
its portfolio(s);
Rating: Not executed;
Summary of evidence: While the IRB Concept of Operations states that
the IRBs are responsible for reviewing factors associated with
portfolio management, such as architecture alignment and capability
delivery, there are no policies and procedures indicating how the IRBs
should use these factors and project indicators--such as cost,
schedule, and risk--to review, evaluate, and improve their portfolios.
According to our ITIM guidance for Stage 3, IRBs should use actual
investment data, such as project cost and adherence to schedule, as the
basis for reviewing and evaluating its portfolio(s) to ensure that the
overall portfolio provides the maximum benefits at a desired cost and
at an acceptable level of risk.
Critical process: Conducting postimplementation reviews;
Key practice: 1. The organization has documented policies and
procedures for conducting postimplementation reviews;
Rating: Not executed;
Summary of evidence: While DOD requires postimplementation reviews for
Tier 1 systems as part of DAS, there are no policies or procedures for
conducting them for Tiers 2 or 3 systems. Moreover, there are no
policies or procedures directing the DBSMC or IRBs, or both, which are
accountable for corporate business system investments, to consider
information gathered and to develop lessons learned from these
postimplementation reviews. According to ITIM, an effective
postimplementation review includes, among other things, how
conclusions, lessons learned, and recommended management action steps
are to be disseminated to executives and others.
Source: GAO.
[End of table]
According to BTA officials, while portfolio management is primarily a
component responsibility, they are working toward developing more
effective departmentwide portfolio management processes, but plans or
time frames for doing so have not been established. Without defining
corporate policies and procedures for managing business system
investment portfolios, DOD is at risk of not consistently selecting the
mix of investments that best supports the departmentwide mission needs
and ensuring that investment-related lessons learned are shared and
applied departmentwide.
Conclusions:
Given the importance of business systems modernization to DOD's
mission, performance, and outcomes, it is vital for the department to
adopt and employ an effective institutional approach to managing
business system investments. While the department has established
aspects of such an approach and, thus, has a foundation on which to
build, it is lacking other important elements, such as specific
policies and procedures needed for project-level and portfolio-level
investment management, including integration with DOD's other key
management systems and sufficient oversight and visibility into
operations and maintenance investments and Tier 4 investments. This
means that DOD lacks an institutional capability to ensure that it is
investing in business systems that best support its strategic needs,
and that ongoing projects meet cost, schedule, and performance
expectations. Until DOD develops this capability, the department will
be impaired in its ability to optimize business mission area
performance and accountability.
Recommendations for Executive Action:
To strengthen DOD's business system investment management capability
and address the weaknesses discussed in this report, we recommend that
the Secretary of Defense direct the Deputy Secretary of Defense, as the
chair of the DBSMC, to ensure that well-defined and disciplined
business system investment management policies and procedures are
developed and issued. At a minimum, this should include project-level
management policies and procedures that address the following five
areas:
* instituting the investment boards, including assigning the investment
boards responsibility, authority, and accountability for programs
throughout the investment life cycle and specifying how the business
investment management system is coordinated with JCIDS, PPBE, and DAS;
* selecting new investments, including specifying how cost, schedule,
and benefit data are to be used in making certification decisions;
defining the criteria used to select investments as enterprisewide; and
establishing consistent and effective guidance for BEA compliance;
* reselecting ongoing investments, including specifying how cost,
schedule, and performance data are to be used in the annual review
process and providing for the reselection of investments that are in
operations and maintenance;
* integrating funding with the process of selecting an investment,
including specifying how the DBSMC and the IRBs use funding information
in carrying out decisions on system certification and approvals; and:
* overseeing IT projects and systems, including providing sufficient
oversight and visibility into component-level investment management
activities.
These well-defined and disciplined business system investment
management policies and procedures should also include portfolio-level
management policies and procedures that address the following four
areas:
* creating and modifying IT portfolio selection criteria for business
system investments;
* analyzing, selecting, and maintaining business system investment
portfolios;
* reviewing, evaluating, and improving the performance of its
portfolio(s) by using project indicators, such as cost, schedule, and
risk; and:
* conducting postimplementation reviews for all investment tiers and
directing the investment boards, which are accountable for corporate
business system investments, to consider the information gathered and
to develop lessons learned from these reviews.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, signed by the Deputy
Under Secretary of Defense (Business Transformation) and reprinted in
appendix II, the department stated that it agreed with the report's
overall conclusions, and it described efforts under way and planned
that it said would address many of the gaps identified in the report.
In this regard, the department partially concurred with five of the
report's recommendations, adding that our recommendations and feedback
are helpful in guiding DOD's business transformation and related
improvement efforts. Nevertheless, the department disagreed with the
remaining four recommendations on the grounds that their intent had
already been met through DOD's existing business system investment
management structure and processes, or that they contradicted the
tiered accountability concept embedded in this structure and processes.
The department's comments relative to each of our project-level and
portfolio-level recommendations, along with our responses to its
comments, are provided below.
With respect to our five project-level recommendations, the department
stated that it partially agreed with two and disagreed with three.
* DOD partially agreed with our recommendation to define and implement
policies and procedures that assign the investment boards
responsibility for programs throughout the investment life cycle and
specify how the business investment management system is coordinated
with JCIDS, PPBE, and DAS. In particular, it stated that under its
tiered accountability approach to business systems investment
management, the components are currently required to review all
programs throughout their investment life cycles. We do not question
this requirement, and we recognize it in our report. However,
consistent with our ITIM framework, the corporate investment boards
should continue to review investments that meet the defined threshold
criteria throughout their life cycles (i.e., when they are in
operations and maintenance). In contrast, DOD's corporate boards focus
only on those investments that are in the development/modernization
stage. The department also stated that a linkage is currently depicted
in existing guidance among its investment selection, acquisition, and
funding processes. While we do not question that this guidance contains
an illustration depicting such a link, neither this guidance nor
supporting procedures define how this linkage is executed (e.g., how
investment funding decisions are in fact integrated with investment
selection decisions). DOD's comments appear to acknowledge this point
by stating that the department has begun to define and implement a
Business Capability Lifecycle concept, which is intended to integrate
the investment selection and acquisition management processes for Tier
1 and enterprise systems into a single oversight process that leverages
the existing IRB and DBSMC oversight framework.
* DOD partially agreed with our recommendation to define and implement
policies and procedures that specify how cost, schedule, and benefit
data are to be used in making certification and annual review
decisions; define the criteria used to select investments as
enterprisewide; and establish consistent and effective guidance for BEA
compliance. In particular, the department agreed that additional
criteria are required for selecting enterprisewide investments, noting
that initial criteria have been defined and will be incorporated in the
investment management process. However, the department did not agree
that cost, schedule, and BEA compliance information are not
sufficiently used for certification and annual review decisions, adding
that such information is required in its current policies. We do not
agree. Specifically, while we do not question whether investment data
are provided to the DBSMC and the IRBs, the department's policies and
procedures do not include specific decision criteria that explain how
these data are to be used to make consistent, repeatable selection and
reselection decisions across all investments. In addition, while BEA
compliance policies have been developed and are being used, the
guidance is not fully defined. For example, the guidance allows
programs to defer demonstrating full compliance with important BEA
artifacts until the final phases of the acquisition process, at which
time addressing instances of noncompliance would be more expensive and
difficult. Furthermore, the compliance criteria are not consistently
described in different guidance documentation. As a result, DOD risks
beginning system production and deployment before ensuring that a
system is sufficiently aligned to the BEA.
* DOD did not agree with our recommendation to define and implement
policies and procedures that provide for the reselection of investments
that are in operations and maintenance. According to DOD, components
are required by policy to annually review all business systems,
including investments for which there is no planned development or
modernization spending. We agree that the annual review process does
require this. However, consistent with our ITIM framework, the
corporate investment boards should continue to reselect investments
that meet the defined threshold criteria throughout their life cycles
(i.e., when they are in operations and maintenance). In contrast, DOD's
corporate boards focus only on reselecting those investments that are
in the development/modernization stage.
* DOD did not agree with our recommendation to define and implement
policies and procedures that specify how the corporate boards use
funding information in carrying out decisions on system certification
and approvals. In this regard, it stated that such information is
required in its current policies and considered during board
deliberations. We do not agree. Our recommendation does not address
whether existing policies or guidance provide for the collection of
this information; our recommendation addresses the definition of
policy, guidance, and supporting procedures that fall short of
satisfying the best practices embodied in our ITIM framework.
Specifically, while we do not question whether funding data are
provided to investment decision-making bodies, the department's
policies and procedures do not include specific decision criteria that
explain how these data are to be used to make consistent, repeatable
selection and reselection decisions across all investments.
* DOD did not agree with our recommendation to define and implement
policies and procedures that provide for sufficient oversight and
visibility into component-level investment management activities. In
particular, it stated that this recommendation contradicts the
department's "tiered accountability" approach to investment management.
We do not agree. Under the department's current policies and guidance,
most DOD investments are not subject to corporate visibility and
oversight, either because they do not involve development/modernization
(i.e., they are in operations and maintenance) or because they do not
exceed a certain dollar threshold. Our framework recognizes that
effective implementation of a tiered accountability concept should
include appropriate corporate visibility into and oversight of
investments, either through review and approval of those investments
that meet certain criteria or through awareness of a subordinate
board's investment management activities. Moreover, this visibility and
oversight should extend to the entire portfolio of investments,
including those that are in operations and maintenance. To ensure that
this occurs, applicable policies and procedures need to explicitly
cover all such investments and need to define how this is to be
accomplished.
With respect to our four portfolio-level recommendations, the
department stated that it partially agreed with three and disagreed
with one.
* DOD partially agreed with our recommendation to define and implement
policies and procedures for creating and modifying portfolio selection
criteria for business system investments. In particular, it stated that
while components are responsible for developing and managing their own
portfolio management processes, upcoming initiatives, such as the
Business Capability Lifecycle concept, will lead to revisions in the
department's investment review policies and procedures, such as
including portfolio selection criteria for enterprise systems that span
components. However, while these are important steps, the concept, as
defined by the department, does not apply to the thousands of
investments that are not enterprisewide.
* DOD partially agreed with our recommendation to define and implement
policies and procedures that address analyzing, selecting, and
maintaining business system investment portfolios. In particular, it
stated that the implementation of the Business Capability Lifecyle
concept will provide the corporate boards with improved visibility into
all investments in a given portfolio and a broader set of criteria for
analyzing, selecting, and maintaining business system investment
portfolios.
* DOD partially agreed with our recommendation to define and implement
policies and procedures that address reviewing, evaluating, and
improving the performance of its portfolio(s) by using cost, schedule,
and risk indicators. In particular, it stated that while such
indicators are part of the investment certification and review
processes, efforts are now under way to better understand the nature
and impact of program risks through application of an Enterprise Risk
Assessment Methodology. While we recognize the role and value of such
tools in understanding and addressing program risks, this tool is
program-specific and not portfolio-focused.
* DOD did not agree with our recommendation to define and implement
policies and procedures that address conducting postimplementation
reviews and having the corporate investment boards consider the review
results and develop lessons learned from them. In particular, it stated
that this process should not be managed by the Deputy Secretary of
Defense, and also stated that our recommendation is redundant with
postimplementation reviews currently required under OMB Circular A-
130.[Footnote 38] We do not agree with DOD's statements. Our
recommendation does not call for the Deputy Secretary to manage the
postimplementation review process. Rather, it provides for developing
policies and procedures for performing postimplementation reviews for
all tiers of business systems and having the DBSMC and IRBs, which are
the corporate investment boards, consider the information gathered from
these reviews and develop lessons learned.
We are sending copies of this report to interested congressional
committees; the Director, Office of Management and Budget; the
Secretary of Defense; the Deputy Secretary of Defense; the Under
Secretary of Defense for Acquisition, Technology, and Logistics; the
Under Secretary of Defense (Comptroller); the Assistant Secretary of
Defense (Networks and Information Integration)/Chief Information
Officer; the Under Secretary of Defense (Personnel and Readiness); and
the Director, Defense Finance and Accounting Service. Copies of this
report will be made available to other interested parties upon request.
This report will also be available at no charge on our Web site at
http://www.gao.gov.
If you or your staffs have any questions on matters discussed in this
report, please contact me at (202) 512-3439 or hiter@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix III.
Signed by:
Randolph C. Hite:
Director, Information Technology Architecture and Systems Issues:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel Inouye:
Chairman:
The Honorable Ted Stevens:
Ranking Member:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan Hunter:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Objective, Scope, and Methodology:
Our objective was to determine whether the Department of Defense's
(DOD) corporate investment management approach comports with relevant
federal guidance. Our analysis was based on the best practices
contained in GAO's Information Technology Investment Management (ITIM)
framework, and the framework's associated evaluation methodology, and
focused on DOD's establishment of departmental-level policies and
procedures for business system investments needed to assist
organizations in complying with the investment management provisions of
the Clinger-Cohen Act of 1996 (Stages 2 and 3). It did not include case
studies to verify the implementation of established policies and
procedures.
To address our objective, we asked DOD to complete a self-assessment of
its corporate investment management process and provide the supporting
documentation. We then reviewed the results of the department's self-
assessment of Stages 2 and 3 organizational commitment practices--
meaning those practices related to structures, policies, and
procedures--and compared them against our ITIM framework. We also
validated and updated the results of the self-assessment through
document reviews and interviews with officials, such as the Director of
Investment Management and the Defense Business Systems Acquisition
Executive. In doing so, we reviewed written policies, procedures, and
guidance and other documentation providing evidence of executed
practices, including the Defense Acquisition System guidance, the
Investment Review Board (IRB) Concept of Operations and Guidance, the
Business Enterprise Architecture Compliance Guidance, IRB charters and
meeting minutes, and the Business Transformation Guidance.
We compared the evidence collected from our document reviews and
interviews with the key practices in ITIM. We rated the key practices
as "executed" on the basis of whether the agency demonstrated (by
providing evidence of performance) that it had met all of the criteria
of the key practice. A key practice was rated as "not executed" when we
found insufficient evidence of all elements of a practice being fully
performed or when we determined that there were significant weaknesses
in DOD's execution of the key practice. In addition, we provided DOD
with the opportunity to produce evidence for the key practices rated as
"not executed."
We conducted our work at DOD headquarters offices in Arlington,
Virginia, from August 2006 through April 2007 in accordance with
generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
3000 Defense Pentagon:
Washington, DC 20301-3000:
Acquisition, Technology And Logistics:
May 3 2007:
Mr. Randolph Hite:
Director, Information Technology Architecture and Systems Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Hite:
This is the Department of Defense (DoD) response to the GAO draft
report 07-538, "Business Systems Modernization: DoD Needs to Fully
Define Policies and Procedures for Institutionally Managing
Investments," dated March 30, 2007, (GAO Code 310636).
The Department welcomes GAO's insight and suggestions as we continue to
strive toward meeting our shared goals of transforming defense business
practices. GAO provides valuable feedback on the Department's
achievements, highlights areas where we can improve, and helps keep our
effort on track toward achieving quality outcomes.
Attached are the Department's responses to the GAO's recommendations to
draft report GAO-07-538. The Department partially concurs on five and
non-concurs with four of the recommendations because we believe that
the existing structure established by the Department already meets the
overall intent of several of GAO's recommendations.
However, we agree with GAO's overall conclusions that DoD should
continue to improve upon its existing investment management policies
and procedures for individual business systems and programs. In fact,
the Department is now developing and implementing changes in its
investment management practices that address many of the gaps
identified by GAO in this audit report. These efforts, in the totality,
address many of the issues and illustrate preplanned BTA efforts to
ameliorate the concerns. Recent enterprise-level improvements include:
* Risk mitigation. Five of the ten business enterprise-level business
programs defined as Major Automated Information Systems (MAIS) have
been or are scheduled soon for an Enterprise Risk Assessment
Methodology (ERAM) evaluation of execution risk and alignment with
enterprise capability goals. The remainder of these 10 business MAIS
will be brought under ERAM by the end of FY 2007.
* Enterprise standards. The BTA is currently "rationalizing the
enterprise" and identifying systems as "enterprise" or "non-
enterprise". Following the initial declaration, the programs assigned
to the "enterprise" will be under the direction of Defense Business
Systems Acquisition Executive (DBSAE) and "non-enterprise" programs
will be further assigned to the appropriate component, thus examining
and assigning the programs to comport with the DoD tiered
accountability structure. While this effort is in its infancy, it
provides increased insight into programs, and the appropriate level of
portfolio management.
* Management framework. We are developing specific policy guidance to
amend the non-statutory portions of the DoD 5000 series of acquisition
regulations and the JCS 3170 to adopt a management structure tailored
to the business mission area. This framework, called the Business
Capability Lifecycle (BCL), is beginning implementation. BCL is being
designed to directly address acknowledged shortfalls in how DoD
develops and fields MAIS and enterprise-level business systems. We
expect to fully implement BCL early in FY 2008.
At the component level, the tiered accountability concept remains the
foundation for implementing portfolio management for the business
mission area. Although we agree that at an enterprise level we need to
establish the appropriate guidance and infrastructure for business
transformation, we strongly believe that delegating certain investment
management responsibilities to the component organizations provides for
a more efficient investment management process. Tiered accountability
has been embraced across DoD. This includes improving DoD's ability at
an enterprise level to maintain the appropriate level of visibility
into the component's operations.
GAO continues to be a valuable and constructive partner in the
Department's business transformation efforts. The recommendations and
feedback provided will help to further guide DoD's process of continual
improvement. We welcome GAO's insights and look forward to your
participation in our future efforts.
Signed for:
Paul A. Brinkley:
Deputy Under Secretary of Defense (Business Transformation):
GAO Draft Report Dated March 30, 2007 GAO-07-538 (GAO Code 310636):
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to institute the investment
boards, including assigning the investment boards responsibility,
authority, and accountability for programs throughout the investment
life cycle and specifying how the business investment management system
is coordinated with Joint Capabilities Integration and Development
System (JCIDS), Planning, Programming, Budgeting, and Execution (PPBE)
and Defense Acquisition System (DAS) to ensure that well-defined and
disciplined business system investment management policies and
procedures are developed and issued. (p. 34/GAO Draft Report):
DOD Response: Partially-Concur - The Department believes that the IRB/
DBSMC process and tiered accountability with the Components currently
supports accountability for programs throughout the investment
lifecycle. Further, the Department believes a linkage currently exists
between the IRB certification and review processes and many other DOD
decision support processes including JCIDS, PPBE, and Acquisition, as
depicted in the figure below from the 13 December 2006 Business
Transformation Guidance.
[See PDF for image]
[End of figure]
This linkage is also addressed in the IRB Concept of Operations
(CONOPS) (previously provided to GAO), dated 29 August 2006, in section
7.2, page 9. To further the alignment between the three processes, the
DoD has begun to implement the Business Capability Lifecycle (BCL)
concept which is scheduled to be fully implemented by FY08 and included
in the DoD 5000 and JCS 3170 rewrites scheduled for the fall of FY08.
The BCL will integrate the JCIDS and DAS, for Tier 1 and Enterprise
systems, into a single oversight process leveraging the existing IRBs
and DBSMC oversight framework. As stated in the March 2007 Annual
Report to the Congressional Defense Committees, the BCL has three
phases:
* Definition - The BCL approach requires the PSA and the functional
sponsor to collaborate to identify and clearly describe the root cause
of a business problem, long before a vendor is involved in the process.
The PSA and functional sponsor are asked to clearly explain why solving
the problem will benefit the Department and (importantly) validate
there is no existing solution. This problem statement and supporting
justification become the basis of the business case for the proposed
capability, which will be reviewed and approved by the appropriate MR
It is during this phase of the BCL that the Defense Acquisition
Executive decides whether a new program start will be approved for
funding, based on the recommendations of the IRB and members of the
DBSMC.
* Investment - After the decision is made to fund a program start, the
business case for the capability is expanded by the functional sponsor
and the candidate program office to identify the scope of the materiel
capabilities needed to solve the problem. The business case will also
define the desired outcomes for the capability, including objectives
and metrics, solution constraints and dependencies. A detailed analysis
of alternatives is conducted during this phase and included in the
business case document, which is augmented by a proposed acquisition
approach and contracting strategy.
* Execution - During the execution phase, responsibility for developing
and fielding the capability is formally assumed by the program manager.
However, the BCL concept requires that the functional sponsor remain
heavily engaged with the program office to address any issues, requests
or changes to the scope. In particular, the BCL requires that the
functional sponsor re-validate the business case (including problem
definition, expected outcomes, metrics, and costs) before each
acquisition milestone or investment decision point, such as an initial
test or the completion of the definition of a program baseline.
We are developing specific policy guidance to amend the non-statutory
portions of the DoD 5000 series of acquisition regulations and the JCS
3I70 to incorporate BCL.
Under Tiered Accountability and as system owners, Components are
responsible for:
* Overseeing program progress through the JCIDS and DAS:
* Advocating for program resources in the PPBE process.
* Coordinating with the IRBs when system certification for development/
modernization is required at key milestones in the Acquisition process.
* Managing systems that are past the development/modernization stage
through the PPBE process and the annual review process as documented in
the IRB Guidance.
The IRB CONOPS and the IRB User Guidance state that Components are
required to annually review all business systems, including those that
are in sustainment, suggesting that they perform this review as part of
an existing process such as the annual Program / Budget Formulation
phase of PPBE.The IRBs review, at least annually all business system
investments that have been previously been certified for &development
and modernization efforts over $I million dollars as required by the
FY2005 NDAA. The result is that all business systems, whether they are
under development/modernization or have been placed in sustainment, are
reviewed annually throughout their lifecycles.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to select new investments,
including specifying how cost, schedule, and benefit data are to be
used in making certification decisions; defining the criteria used to
select investments as enterprise-wide; and establishing consistent and
effective guidance for business enterprise architecture (BEA) to ensure
that well-defined and disciplined business system investment management
policies and procedures are developed and issued. (p. 35/GAO Draft
Report):
DOD Response: Partially Concur:
Partially Concur:
The BTA has defined initial criteria for selecting enterprise-wide
investments and is in the process of applying this criterion to the
enterprise systems under the Defense Business Systems Acquisition
Executive (DBSAE). This effort is defining a framework that articulates
the set of specific characteristics that are appropriate for an
enterprise-level solution.
This initiative which is referred to as "Rationalizing the Enterprise"
is scheduled to be finalized this summer and will be incorporated into
the investment management process to help the IRBs and Components
determine which business capabilities should be implemented at the
Business Mission Area (BMA) enterprise level versus those that should
be implemented at the Component level.
Non-concur:
IRB/DBSMC Policies do require cost, schedule and benefit data for
certification decisions and annual review IRB assessments. This
information is included on both the annual review and certification
dashboards. Cost, schedule and performance is assessed as "green",
"yellow" or "red" based on specified thresholds defined in policy and
benefit is assessed through non-financial and financial metrics
substantiated with an economic viability analysis. IRB decisions are
not based on any one item but a combination of factors, some of which
are measurable, and some less tangible. Cost, schedule, and performance
are the basis upon which annual reviews are conducted.
Non-Concur:
BEA Compliance policies were released April I0, 2006, which describe
the process for assessing compliance to the architecture and define the
requirements for an architecture compliance plan. This guidance has
also been enabled through the Architecture Compliance and Requirements
Traceability Tool which creates a semi-automated process for assessing
compliance and generating a Compliance Plan. It also provides metrics
which show the degree of alignment to the BEA and number of
"compliant", "non-compliant" and "compliance pending" instances.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to reselect ongoing
investments, including specifying how cost, schedule, and performance
data are to be used in the annual review process and providing for the
reselection of investments that are in operations and maintenance to
ensure that well-defined and disciplined business system investment
management policies and procedures are developed and issued. (p. 35/GAO
Draft Report):
DOD Response: Non-Concur:
As stated above, cost, schedule and performance data are used in the
annual review process.
Per the IRB CONOPS, dated 29 August 2006, in section 8.0, page 13:
* Components are required to annually review all business systems,
regardless of investment Tier, including systems for which there is no
planned development or modernization spending.
* At a minimum, as part of the annual reviews Components should make
sure that systems are assessed against the DoD BEA, ensure systems are
included in the Component or Enterprise Transition Plan, and that all
required information regarding each system has been updated in the
Department's global business systems inventory.
* Components are required to submit a letter to the IRBs on a semi-
annual basis, on a schedule consistent with the Enterprise Transition
Plan update cycle, listing all business systems that have been
reviewed. These internal Component reviews, coupled with notification
of these reviews to the CA / IRB, meet the FY 2005 NDAA annual review
requirement.
RECOMMENDATION 4: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to integrate funding with the
process of selecting an investment, including specifying how the
Defense Business Systems Management Committee (DBSMC) and the
Investment Review Board (IRB) use funding information in carrying out
decisions on system certification and approvals to ensure that well-
defined and disciplined business system investment management policies
and procedures are developed and issued. (p. 35/GAO Draft Report):
DOD Response: Non-Concur:
Funding information is integrated into the current IRB/DBSMC process;
funding information for every investment is presented to the IRB
membership and documented on both the certification and annual review
dashboards and PCA letters. Funding is an important element of the
process and is taken into consideration along with other information
(e.g. risk, benefit) during IRB/DBSMC deliberations. When there are
funding issues associated with a particular investment, they are
addressed during the IRB process, particularly during the annual review
process. If they are related to poor management/execution, the IRB/
DBSMC may recommend reprogramming actions to support better alignment
of budget to the needs of the portfolio. Each IRB decision is based on
a review of available information and unfunded requests are handled on
a case by case basis.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to oversee information
technology (IT) projects and systems, including providing sufficient
oversight and visibility into component-level investment management
activities to ensure that well-defined and disciplined business system
investment management policies and procedures are developed and issued.
(p. 35/GAO Draft Report):
DOD Response: Non-Concur - The Department's investment management
process for business systems is predicated on the tiered accountability
approach, under which DoD Components are responsible for managing their
IT investments and IT portfolios with the proviso that the cognizant
IRBs and the DBSMC provide oversight over those investments to ensure
compliance with I0 U.S.C. 2222, as added by Section 332 of the Ronald
W. Reagan National Defense Authorization Act for Fiscal Year 2005, and
other applicable laws, regulations, and policies. Under this statute
the IRBs and the DBSMC have visibility of all systems that receive in
excess of one million dollars in modernization funding.
The Department believes the GAO's recommendation contradicts the tiered
accountability approach in recommending that the Department, from a
corporate perspective, oversee Component development and issuance of
business system investment management policies and procedures. While
the Department does oversee Component business system investment
management decisions to the degree defined in the IRB CONOPS and has
issued guidance on portfolio management processes to the Components, in
accordance with tiered accountability, it does not guide or direct the
Components in the formulation of the Component-level policies and
procedures by which their investment decisions are reached.
Recommendation 6: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to create and modify IT
portfolio selection criteria for business system investments. (p. 35/
GAO Draft Report):
DOD Response: Partially Concur - The Department continues to move in
the direction of maturing its portfolio management processes. Under
Tiered Accountability, each Component is responsible for developing and
managing its own portfolio management process; however, when it is in
the best interest of DoD for a portfolio to span Components, the
appropriate IRB can establish an "Enterprise Portfolio." To date, DoD
has stood up the Distribution Process Owner (DPO) Portfolio which looks
at distribution processes and supporting business systems across all
DoD Components. The DPO is chaired by USTRANSCOM.
With the implementation of BCL, all the IRB charters, CONOPs, and
Guidance are under revision. The revised versions will clearly
articulate the criteria necessary for establishing a "Enterprise
Portfolio."
Additionally, the Department has implemented the Department of Defense
Instruction (DoDD) 8115.01 - "Information Technology Portfolio
Management", which defines the responsibilities for the management of
DoD IT investments as portfolios within the DoD Enterprise (to include
Mission Areas, Sub-portfolios, and Components).
Recommendation 7: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to analyze, select, and
maintain business system investment portfolios. (p. 35/GAO Draft
Report):
DOD Response: Partially Concur - The Department continues to move in
the direction of maturing its portfolio management processes. Under
Tiered Accountability, each Component is responsible for developing and
managing its own portfolio management process; however, when it is in
the best interest of DoD for a portfolio to span Components, the
appropriate IRB can establish an "Enterprise Portfolio." To date, DoD
has stood up the Distribution Process Owner (DPO) Portfolio which looks
at distribution processes and supporting business systems across all
DoD Components. The DPO is chaired by USTRANSCOM.
The implementation of the BCL will allow the IRBs significantly
improved visibility of all investments being made in given portfolios.
Since each investment will be accompanied by a business case, the IRBs
will have the opportunity to make investment decisions with a much
broader set of criteria than is possible at the current time.
Recommendation 8: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to review, evaluate, and
improve the performance of its portfolio(s) by using project indicators
such as cost, schedule, and risk. (p. 35/GAO Draft Report):
DOD Response: Partially Concur - Under existing IRB and DBSMC process
and procedure Ms and DBSMC currently review cost and schedule data as
part of the investment certification and annual review processes. In an
effort to understand project risk and the impact of risk on the
delivery of business capability the Department has implemented the
Enterprise Risk Assessment Methodology (ERAM). ERAM is currently being
executed on five of the ten business MAIS programs. The output of the
risk assessments will provide an analysis of the risks, impacts and
mitigation strategies for given portfolio investments enabling the IRB
to weigh risk impact along with cost, schedule and performance further
improving investment decisions.
As stated in the March 2007 Annual Report to the Congressional Defense
Committees, ERAM is a collaborative review process, bringing the
functional sponsors, the program office, and experts from the
acquisition community together. An ERAM team begins by reviewing
existing program documentation, and then conducts face-to-face
interviews with a cross-section of key program stakeholders and
managers. Based on this information, the ERAM team evaluates program
risk in seven key areas and delivers a risk mitigation plan as quickly
as possible (ideally, within five to six weeks). The seven risk areas
are:
* Strategy:
* Scope/Requirement:
* Contract:
* Technical:
* People:
* Process:
* External:
The quick turnaround is important, because the goal is to give the
sponsor and program manager targeted, actionable advice in time for
them to act to keep the program focused on delivering capability.
ERAM adheres to DoD Directive 5000 Series principles that govern
Defense acquisition activities. Ultimately, it is expected that ERAM
will help the Department improve its acquisition of capabilities by
achieving several key outcomes:
* Providing the right information needed to make sound optimized
investment decisions.
* Creating a clear path for the rapid delivery of capability.
* Reducing (or removing) burdensome Overarching Integrated Process Team
(OIPT) documentation and meeting requirements.
* Identifying program risks early enough so they can be avoided or
mitigated.
* The overall vision for ERAM is to provide a common vehicle for
collaboratively managing program risk with a focus on rapid delivery of
capability at reduced cost and schedule.
RECOMMENDATION 9: The GAO recommends that the Secretary of Defense
direct the Deputy Secretary of Defense, to conduct post implementation
reviews for all investment tiers and direct the investment boards who
are accountable for corporate business system investments, to consider
the information gathered and to develop lessons learned from these
reviews. (p. 36/GAO Draft Report):
DOD Response: Non-Concur -The Department disagrees that this process
should be managed by the Deputy Secretary of Defense. Requiring the
Deputy Secretary of Defense to perform post-implementation reviews is
redundant with The Office of Management and Budget (OMB) Circular A-
130, Chapter 8 b.(1).(d) that requires the agency "Conduct post-
implementation reviews of information systems to validate estimated
benefits and document effective management practices for broader use."
The Department will capture and leverage the lessons learned and best
management practices from these component level reviews and make them
available to the IRBs and across the Components. This also aligns with
DoD's tiered accountability approach.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Randolph C. Hite, (202) 512-3439 or hiter@gao.gov:
Staff Acknowledgments:
In addition to the contact person named above, key contributors to this
report were Neil Doherty, Nalani Fraser, Nancy Glover, Michael Holland,
Neelaxi Lakhmani (Assistant Director), Jacqueline Mai, Sabine Paul,
Niti Tandon, and Jennifer Stavros-Turner.
FOOTNOTES
[1] Business systems are information systems that include financial and
nonfinancial systems and support DOD's business operations, such as
civilian personnel, finance, health, logistics, military personnel,
procurement, and transportation.
[2] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.:
January 2007).
[3] GAO, Information Technology: Architecture Needed to Guide
Modernization of DOD's Financial Operations, GAO-01-525 (Washington,
D.C.: May 17, 2001).
[4] See, for example, GAO, DOD Business Systems Modernization: Long-
standing Weaknesses in Enterprise Architecture Development Need to Be
Addressed, GAO-05-702 (Washington, D.C.: July 22, 2005); DOD Business
Systems Modernization: Billions Being Invested without Adequate
Oversight, GAO-05-381 (Washington, D.C.: Apr. 29, 2005); DOD Business
Systems Modernization: Limited Progress in Development of Business
Enterprise Architecture and Oversight of Information Technology
Investments, GAO-04-731R (Washington, D.C.: May 17, 2004); DOD Business
Systems Modernization: Important Progress Made to Develop Business
Enterprise Architecture, but Much Work Remains, GAO-03-1018
(Washington, D.C.: Sept. 19, 2003); Business Systems Modernization:
Summary of GAO's Assessment of the Department of Defense's Initial
Business Enterprise Architecture, GAO-03-877R (Washington, D.C.: July
7, 2003); Information Technology: Observations on Department of
Defense's Draft Enterprise Architecture, GAO-03-571R (Washington, D.C.:
Mar. 28, 2003); DOD Business Systems Modernization: Improvements to
Enterprise Architecture Development and Implementation Efforts Needed,
GAO-03-458 (Washington, D.C.: Feb. 28, 2003); and GAO-01-525.
[5] Ronald W. Reagan National Defense Authorization Act for Fiscal Year
2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-1856 (Oct. 28,
2004) (codified in part at 10 U.S.C. § 2222).
[6] GAO, Defense Business Transformation: A Comprehensive Plan,
Integrated Efforts, and Sustained Leadership Are Needed to Assure
Success, GAO-07-229T (Washington, D.C.: Nov. 16, 2006); Business
Systems Modernization: DOD Continues to Improve Institutional Approach,
but Further Steps Needed, GAO-06-658 (Washington, D.C.: May 15, 2006);
and DOD Business Systems Modernization: Important Progress Made in
Establishing Foundational Architecture Products and Investment
Management Practices, but Much Work Remains, GAO-06-219 (Washington,
D.C.: Nov. 23, 2005).
[7] GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington,
D.C.: March 2004).
[8] GAO-04-394G.
[9] GAO-06-658.
[10] See, for example, GAO, DOD Travel Cards: Control Weaknesses
Resulted in Millions of Dollars of Improper Payments, GAO-04-576
(Washington, D.C.: June 9, 2004); Military Pay: Army National Guard
Personnel Mobilized to Active Duty Experienced Significant Pay
Problems, GAO-04-89 (Washington, D.C.: Nov. 13, 2003); and Defense
Inventory: Opportunities Exist to Improve Spare Parts Support Aboard
Deployed Navy Ships, GAO-03-887 (Washington, D.C.: Aug. 29, 2003).
[11] GAO-07-310.
[12] These 8 high-risk areas include DOD's (1) overall approach to
business transformation, (2) business systems modernization, (3)
financial management, (4) personnel security clearance program, (5)
supply chain management, (6) support infrastructure management, (7)
weapon systems acquisition, and (8) contract management.
[13] The 7 governmentwide high-risk areas are (1) disability programs,
(2) ensuring the effective protection of technologies critical to U.S.
national security interests, (3) interagency contracting, (4)
information systems and critical infrastructure, (5) information-
sharing for homeland security, (6) human capital, and (7) real
property.
[14] The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11101-11704. This act
expanded the responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act with regard to IT management. See 44
U.S.C. 3504(a)(1)(B)(vi) (OMB); and 44 U.S.C. 3506(h)(5) (agencies).
[15] We have made recommendations to improve OMB's process for
monitoring high-risk IT investments; see GAO, Information Technology:
OMB Can Make More Effective Use of Its Investment Reviews, GAO-05-276
(Washington, D.C.: Apr. 15, 2005).
[16] This policy is set forth and guidance is provided in OMB Circular
A-11 (Nov. 2, 2005) (section 300), and in OMB's Capital Programming
Guide, which directs agencies to develop, implement, and use a capital
programming process to build their capital asset portfolios.
[17] See, for example, GAO-04-394G; GAO, Information Technology: A
Framework for Assessing and Improving Enterprise Architecture
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003);
and Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
D.C.: February 1997).
[18] GAO-04-394G; GAO/AIMD-10.1.13; GAO, Executive Guide: Improving
Mission Performance Through Strategic Information Management and
Technology, GAO/AIMD-94-115 (Washington, D.C.: May 1994); and Office of
Management and Budget, Evaluating Information Technology Investments, A
Practical Guide (Washington, D.C.: November 1995).
[19] GAO-04-394G.
[20] GAO, Information Technology: Centers for Medicare & Medicaid
Services Needs to Establish Critical Investment Management
Capabilities, GAO-06-12 (Washington, D.C.: Oct. 28, 2005); Information
Technology: HHS Has Several Investment Management Capabilities in
Place, but Needs to Address Key Weaknesses, GAO-06-11 (Washington,
D.C.: Oct. 28, 2005); Information Technology: FAA Has Many Investment
Management Capabilities in Place, but More Oversight of Operational
Systems Is Needed, GAO-04-822 (Washington, D.C.: Aug. 20, 2004); Bureau
of Land Management: Plan Needed to Sustain Progress in Establishing IT
Investment Management Capabilities, GAO-03-1025 (Washington, D.C.:
Sept. 12, 2003); Information Technology: Departmental Leadership
Crucial to Success of Investment Reforms at Interior, GAO-03-1028
(Washington, D.C.: Sept. 12, 2003); United States Postal Service:
Opportunities to Strengthen IT Investment Management Capabilities, GAO-
03-3 (Washington, D.C.: Oct. 15, 2002); and Information Technology: DLA
Needs to Strengthen Its Investment Management Capability, GAO-02-314
(Washington, D.C.: Mar. 15, 2002).
[21] The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11311-11313.
[22] The National Security Strategy Report required by 50 U.S.C. 404a
is a comprehensive report on the national security strategy of the
United States submitted by the President to Congress.
[23] See 10 U.S.C. 118. The Quadrennial Defense Review is a
comprehensive examination of the national defense strategy, force
structure, force modernization plans, infrastructure, budget plan, and
other elements of the defense program and policies of the United States
with a view toward determining and expressing the defense strategy of
the United States and establishing a defense program for the next 20
years.
[24] GAO, Best Practices: An Integrated Portfolio Management Approach
to Weapon System Investments Could Improve DOD's Acquisition Outcomes,
GAO-07-388 (Washington, D.C.: Mar. 30, 2007).
[25] The Director for Program Analysis and Evaluation is the principal
staff assistant who conducts independent analysis for, and provides
independent advice on, all DOD program and evaluation matters to the
Secretary and Deputy Secretary of Defense.
[26] DOD Directive 5000.1, May 12, 2003 and DOD Instruction 5000.2, May
12, 2003.
[27] A MDAP is an acquisition program that is estimated by the Under
Secretary of Defense for Acquisition, Technology, and Logistics to
require an eventual total expenditure for research, development, and
test and evaluation of more than $365 million (fiscal year 2000
constant dollars) or, for procurement, of more than $2.190 billion
(fiscal year 2000 constant dollars).
[28] A MAIS is a program or initiative that is so designated by the
Assistant Secretary of Defense (Networks and Information Integration)/
Chief Information Officer or that is estimated to require program costs
in any single year in excess of $32 million (fiscal year 2000 constant
dollars), total program costs in excess of $126 million (fiscal year
2000 constant dollars), or total life-cycle costs in excess of $378
million (fiscal year 2000 constant dollars).
[29] According to DOD, the milestone decision authority is the
designated individual who has overall responsibility for an investment.
This person has the authority to approve an investment's progression in
the acquisition process and is responsible for reporting cost,
schedule, and performance results. For example, the milestone decision
authority for a MDAP program, when not delegated to the component
level, is the Under Secretary of Defense for Acquisition, Technology,
and Logistics, and the milestone decision authority for a MAIS system
is the Assistant Secretary of Defense (Networks and Information
Integration)/Chief Information Officer or a designee.
[30] The Defense Acquisition Board, chaired by the Under Secretary of
Defense for Acquisition, Technology, and Logistics, conducts reviews
for MDAPs at major program milestones and documents the decision(s)
resulting from the review in an Acquisition Decision Memorandum.
[31] The IT Acquisition Board, chaired by the Assistant Secretary of
Defense (Networks and Information Integration)/Chief Information
Officer, conducts reviews for MAIS at major program milestones and
documents the decision(s) resulting from the review in an Acquisition
Decision Memorandum.
[32] The four IRBs are for (1) Financial Management, established by the
Deputy Under Secretary of Defense for Financial Management; (2) Weapon
Systems Lifecycle Management and Materiel Supply and Services
Management; (3) Real Property and Installations Lifecycle Management,
both established by the USD(AT&L); and (4) Human Resources Management,
established by the Under Secretary of Defense for Personnel and
Readiness.
[33] DITPR is DOD's authoritative repository for certain information
about DOD's business systems, such as system names and the responsible
DOD components, that are required for the certification, approval, and
annual reviews of these business system investments.
[34] The certification authority is the designated Principal Staff
Assistant with responsibility for review, approval, and oversight of
the planning, design, acquisition, deployment, operation, maintenance,
and modernization of defense business systems.
[35] In addition, each component PCA submits a list of system names to
the IRBs on a semiannual basis, to include Tier 4 systems and systems
in operations and maintenance that have been reviewed at the component
level.
[36] Investment portfolios are integrated agencywide collections of
investments that are assessed and managed collectively on the basis of
common criteria.
[37] DOD Directive 8115.01, Information Technology Portfolio
Management, and DOD Instruction 8115.02, Information Technology
Portfolio Management Implementation.
[38] According to OMB Circular A-130, which establishes policy for the
management of federal information resources, as part of the capital
planning process, an agency must, among other things, conduct
postimplementation reviews of information systems and information
resource management processes to validate estimated benefits and costs;
document effective management practices for broader use; and document
lessons learned from the postimplementation reviews.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
