Information Technology
Treasury Needs to Strengthen Its Investment Board Operations and Oversight Gao ID: GAO-07-865 July 23, 2007The Department of the Treasury relies extensively on information technology (IT) to carry out its mission. For fiscal year 2007, Treasury requested about $2.8 billion--the third largest planned IT expenditure among civilian agencies. GAO's objectives included (1) assessing Treasury's capabilities for managing its IT investments and (2) determining any plans the agency has for improving its capabilities. GAO used its IT investment management framework (ITIM) and associated methodology to address these objectives, focusing on the framework's stages related to the investment management provisions of the Clinger-Cohen Act of 1996.
While Treasury has established many of the capabilities needed to select, control, and evaluate its IT investments, the department has significant weaknesses that hamper its ability to effectively manage its investments. Specifically, the department has executed 19 of the 38 key practices that the ITIM requires to build a foundation for IT investment management (Stage 2), including practices needed to ensure that projects support business needs and that a disciplined process exists for capturing investment information. In addition, the department has executed 11 of the 27 key practices required to manage investments as a portfolio (Stage 3), including documenting policies and procedures for conducting postimplementation reviews. However, Treasury does not have an executive investment review board--a group of executives from IT and business units that is intended to be the final decision-making authority--that is actively engaged in the investment management process. In addition, the department does not have any policies and procedures for managing its nonmajor investments, although they represent almost 70 percent of the total number of investments. Until the department addresses these weaknesses, it will not have the investment management structure needed to effectively assess and manage the risks associated with its multibillion-dollar portfolio. To its credit, Treasury has initiated efforts to improve its investment management process. For example, it has recently implemented a process for identifying major projects that should receive additional oversight. However, the department has not developed a comprehensive improvement plan that (1) is based on an assessment of strengths and weaknesses; (2) specifies measurable goals, objectives, and milestones; (3) specifies needed resources; (4) assigns clear responsibility and accountability for accomplishing tasks; and (5) is approved by senior-level management. GAO has previously reported that such a plan is instrumental in helping agencies coordinate and guide improvement efforts. Until Treasury develops this plan and the controls for implementing it, the department risks not being able to put in place an effective management process that will provide appropriate executive-level oversight for minimizing risks and maximizing returns.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-865, Information Technology: Treasury Needs to Strengthen Its Investment Board Operations and Oversight
This is the accessible text file for GAO report number GAO-07-865
entitled 'Information Technology: Treasury Needs to Strengthen Its
Investment Board Operations and Oversight' which was released on July
865, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
July 2007:
Information Technology:
Treasury Needs to Strengthen Its Investment Board Operations and
Oversight:
GAO-07-865:
GAO Highlights:
Highlights of GAO-07-865, a report to congressional requesters
Why GAO Did This Study:
The Department of the Treasury relies extensively on information
technology (IT) to carry out its mission. For fiscal year 2007,
Treasury requested about $2.8 billion”the third largest planned IT
expenditure among civilian agencies. GAO‘s objectives included (1)
assessing Treasury‘s capabilities for managing its IT investments and
(2) determining any plans the agency has for improving its
capabilities. GAO used its IT investment management framework (ITIM)
and associated methodology to address these objectives, focusing on the
framework‘s stages related to the investment management provisions of
the Clinger-Cohen Act of 1996.
What GAO Found:
While Treasury has established many of the capabilities needed to
select, control, and evaluate its IT investments, the department has
significant weaknesses that hamper its ability to effectively manage
its investments. Specifically, the department has executed 19 of the 38
key practices that the ITIM requires to build a foundation for IT
investment management (Stage 2), including practices needed to ensure
that projects support business needs and that a disciplined process
exists for capturing investment information. In addition, the
department has executed 11 of the 27 key practices required to manage
investments as a portfolio (Stage 3), including documenting policies
and procedures for conducting postimplementation reviews (see table).
However, Treasury does not have an executive investment review board”a
group of executives from IT and business units that is intended to be
the final decision-making authority”that is actively engaged in the
investment management process. In addition, the department does not
have any policies and procedures for managing its nonmajor investments,
although they represent almost 70 percent of the total number of
investments. Until the department addresses these weaknesses, it will
not have the investment management structure needed to effectively
assess and manage the risks associated with its multibillion-dollar
portfolio.
To its credit, Treasury has initiated efforts to improve its investment
management process. For example, it has recently implemented a process
for identifying major projects that should receive additional
oversight. However, the department has not developed a comprehensive
improvement plan that (1) is based on an assessment of strengths and
weaknesses; (2) specifies measurable goals, objectives, and milestones;
(3) specifies needed resources; (4) assigns clear responsibility and
accountability for accomplishing tasks; and (5) is approved by senior-
level management. GAO has previously reported that such a plan is
instrumental in helping agencies coordinate and guide improvement
efforts. Until Treasury develops this plan and the controls for
implementing it, the department risks not being able to put in place an
effective management process that will provide appropriate executive-
level oversight for minimizing risks and maximizing returns.
Table: Treasury's IT Investment Management Capabilities:
Source: GAO.
[End of table]
What GAO Recommends:
To further strengthen Treasury‘s investment management capability, GAO
recommends that the department develop and implement a plan to
establish an executive investment review board and policies and
procedures to manage nonmajor investments and address the other
weaknesses GAO identified. In e-mail comments on a draft of this
report, Treasury stated that the report reflects both Treasury‘s
shortcomings as well as progress to date and recognized the need to
take proactive steps to strengthen its investment board operations and
oversight of information technology resources and programs.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-865].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact David Powner at (202) 512-
9286 or pownerd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Treasury Has Established Many Key Practices for Managing Its
Investments, but Has Key Weaknesses with Its Board Operations and
Investment Oversight:
Treasury Does Not Have a Comprehensive Plan to Guide Its Improvement
Efforts:
Treasury CIO's Role in Managing IT Investments Has Been Mixed:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Governance Roles and Responsibilities:
Table 2: Stage 2 Critical Processes--Building the Investment
Foundation:
Table 3: Summary of Results for Stage 2 Critical Processes and Key
Practices:
Table 4: Instituting the Investment Board:
Table 5: Meeting Business Needs:
Table 6: Selecting an Investment:
Table 7: Providing Investment Oversight:
Table 8: Capturing Investment Information:
Table 9: Stage 3 Critical Processes--Developing a Complete Investment
Portfolio:
Table 10: Summary of Results for Stage 3 Critical Processes and Key
Practices:
Table 11: Defining the Portfolio Criteria:
Table 12: Creating the Portfolio:
Table 13: Evaluating the Portfolio:
Table 14: Conducting Postimplementation Reviews:
Table 15: CIO Involvement in Performing Investment Management
Responsibilities:
Figures:
Figure 1: Treasury Organizational Chart (condensed):
Figure 2: CPIC Process:
Figure 3: ITIM Stages of Maturity:
Abbreviations:
CADE: Customer Account Data Engine:
CIO: chief information officer:
CPIC: Capital Planning and Investment Control:
EA: enterprise architecture:
E-board: Treasury Executive Investment Review Board:
EVMS: earned value management system:
FinCEN: Financial Crimes Enforcement Network:
IRS: Internal Revenue Service:
IT: information technology:
ITIM: information technology investment management framework:
OA: operational analysis:
OCIO: Office of the Chief Information Officer:
OMB: Office of Management and Budget:
PIR: postimplementation review:
SaBRe: Savings Bond Replacement System:
TFIN: Treasury Foreign Intelligence Network:
TIRB: Technical Investment Review Board:
TRACS: Treasury Receivable, Accounting, and Collection System:
United States Government Accountability Office:
Washington, DC 20548:
July 23, 2007:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Sam Brownback:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
United States Senate:
The Honorable Christopher S. Bond:
United States Senate:
The Department of the Treasury relies extensively on information
technology (IT) to carry out its responsibility of promoting the
economic and financial prosperity and security of the United States.
For fiscal year 2007, the department plans to spend about $2.8 billion-
-the third largest planned IT expenditure among civilian
agencies.[Footnote 1] Given the size and significance of the
department's IT investments, you asked us to (1) assess Treasury's
capabilities for managing its IT investments, (2) determine any plans
the agency has for improving its capabilities, and (3) evaluate the
Chief Information Officer's (CIO) role in managing the department's IT
investments. We used our IT investment management framework (ITIM) and
associated methodology to address these objectives, focusing on the
framework's stages related to the investment management provisions of
the Clinger-Cohen Act of 1996.[Footnote 2]
We performed our work from August 2006 through July 2007 in accordance
with generally accepted government auditing standards. Appendix I
contains details about our objectives, scope, and methodology.
Results in Brief:
While Treasury has established many of the capabilities needed to
select, control, and evaluate its IT investments, the department has
significant weaknesses that hamper its ability to effectively manage
its investments. Specifically, the department has executed 19 of the 38
key practices that the ITIM requires to build a foundation for IT
investment management, (Stage 2) including practices needed to ensure
that projects support business needs and that a disciplined process
exists for capturing investment information. In addition, the
department has executed 11 of the 27 key practices required to manage
investments as a portfolio (Stage 3), including documenting policies
and procedures for conducting postimplementation reviews. However,
Treasury does not have an executive investment review board--a group of
executives from IT and business units that is intended to be the final
decision-making authority--that is actively engaged in the investment
management process. In addition, the department does not have any
policies and procedures for managing its nonmajor investments, although
they represent almost 70 percent of the total number of investments.
Until the department addresses these weaknesses, it will not have the
investment management structure needed to effectively assess and manage
the risks associated with its multibillion-dollar portfolio.
To its credit, Treasury has initiated efforts to improve its investment
management process. For example, it has recently implemented a process
for identifying major projects that should receive additional
oversight. However, the department has not developed a comprehensive
improvement plan that (1) is based on an assessment of strengths and
weaknesses; (2) specifies measurable goals, objectives, and milestones;
(3) specifies needed resources; (4) assigns clear responsibility and
accountability for accomplishing tasks; and (5) is approved by senior-
level management. We have previously reported that such a plan is
instrumental in helping agencies coordinate and guide improvement
efforts. Treasury officials recognize the value of having a
comprehensive plan and told us they plan to develop one once their new
assistant secretary for management is confirmed; however, a time frame
for completing the plan has not been established. Until Treasury
develops this plan and the controls for implementing it, the department
risks not being able to put in place an effective management process
that will provide appropriate executive-level oversight for minimizing
risks and maximizing returns.
The Treasury CIO's role in managing the department's IT investments has
been mixed---though it has gradually increased since September 2005,
when the department's investment management policy was issued.
Specifically, some responsibilities have been fully performed, some
have been partially performed, and others have not been performed.
To further strengthen Treasury's investment management capability, we
are recommending that the department develop and implement a plan to
establish an executive investment review board, develop policies and
procedures to manage nonmajor investments, and address the other
weaknesses we identified in this report.
In e-mail comments on a draft of this report, the Acting CIO stated
that the report reflects both Treasury's shortcomings as well as
progress to date and recognized the need to take proactive steps to
strengthen its investment board operations and oversight of information
technology resources and programs. Treasury also agreed with the need
for an executive investment review board that is actively engaged in
the investment management process and noted that nonmajor investments
have not been a priority because the major investments the department
has chosen to devote its resources to represent the more significant
portion of the portfolio in terms of dollar value, visibility to OMB
and Congress, and importance to Treasury's mission. Treasury also
commented on the department's authority to redirect funding from one
Treasury bureau to another. We incorporated these comments into our
report where appropriate.
Background:
Treasury's Mission and Organizational Structure:
The Department of the Treasury is the primary federal agency
responsible for the economic and financial prosperity and security of
the United States, and as such is responsible for a wide range of
activities, including advising the President on economic and financial
issues, promoting the President's growth agenda, and enhancing
corporate governance in financial institutions.
To accomplish its mission, Treasury is organized into departmental
offices and operating bureaus. The departmental offices are primarily
responsible for the formulation of policy and management of the
department as a whole, while the nine operating bureaus--including the
Internal Revenue Service and the Bureau of Engraving and Printing--
carry out the specific functions assigned to Treasury. Figure 1 shows
the organizational structure of the department.
Figure 1: Treasury Organizational Chart (condensed):
[See PDF for image]
Source: Department of the Treasury.
[End of figure]
Treasury's Use of Information Technology:
Information technology plays a critical role in helping Treasury meet
its mission. For example, the Internal Revenue Service relies on
information systems to process tax returns, account for tax revenues
collected, send bills for taxes owed, issue refunds, assist in the
selection of tax returns for audit, and provide telecommunications
services for business activities, including the public's toll-free
access to tax information. To modernize the systems it relies on to
carry out these functions, Treasury is engaged in a Business Systems
Modernization program.
Treasury requested $11.4 billion in the President's fiscal year 2007
budget. Of this amount, the department estimates it will spend
approximately $2.8 billion for 235 IT investments--some $2.3 billion
(about 80 percent) for 75 major investments and some $480 million
(about 20 percent) for 160 nonmajor investments.
Prior Reviews on IT Management Issues at Treasury:
Since mid-1999, we have been reviewing the Internal Revenue Service's
(IRS) progress in implementing its Business Systems Modernization
program as part of our reviews of the service's associated expenditure
plans.[Footnote 3] Our reviews have identified a number of weaknesses
in IRS's modernization management controls and capabilities and, over
the years, we have made numerous recommendations to address these
weaknesses. IRS has addressed many of our recommendations; however,
several weaknesses remain.
In January 2004, we reported[Footnote 4] as part of a governmentwide
review, that Treasury had significant weaknesses in investment
management. We noted, for example, that the department had neither
developed a capital planning and investment control guide nor developed
work processes and procedures for the agency's IT investment management
board. In addition, Treasury had not documented the alignment and
coordination of responsibilities of its various boards for decision
making related to investments, including the criteria for which
investments--including crosscutting investments--were to be reviewed by
the executive investment review board. We also reported that Treasury
did not have a department-level control process; instead, each bureau
could conduct its own reviews that address the performance of its IT
investments and corrective actions for underperforming projects. We
made several recommendations to address the weaknesses we identified.
Treasury concurred with our recommendations, stating that it recognized
its shortcomings and was working to correct them.
In July 2006,[Footnote 5] we reported on Treasury's Financial Crimes
Enforcement Network's (FinCEN) BSA Direct Retrieval and Sharing
project, a nonmajor investment,[Footnote 6] noting that FinCEN did not
always apply effective investment management processes to oversee this
project. We recommended that the director of FinCEN direct its CIO to
develop a plan for improving the agency's capabilities for overseeing
this project. FinCEN officials concurred with our findings and
recommendation.
In January 2007, in an update to our high-risk series report on the
Internal Revenue Service's Business Systems Modernization,[Footnote 7]
which we first designated as high-risk in 1995, we reported that while
the Internal Revenue Service had made progress in reducing risk with
systems modernization and financial management, improvements made have
not been sustained long enough to provide confidence that the program
is fully stable. We also reported that many challenges remain,
including improving processes for designing, developing, and delivering
modernized IT systems.
Several of Treasury's projects have been deemed to be poorly planned
and managed by the Office of Management and Budget (OMB) and have
warranted inclusion on OMB's Management Watch and High Risk
Lists.[Footnote 8]
Role of Department CIO in Investment Management:
The Clinger-Cohen Act of 1996 requires agency heads to designate the
CIO to lead reforms to help control system development risks; better
manage technology spending; and achieve real, measurable improvements
in agency performance through better management of information
resources.[Footnote 9] The agency head, through the department-level
CIO, is responsible for providing leadership and oversight for
foundational critical processes by ensuring that written policies and
procedures are established, repositories of information are created
that support investment decision making, resources are allocated,
responsibilities are assigned, and all of the activities are properly
carried out where they may be most effectively executed.
Treasury's Approach to Investment Management:
Treasury's IT investment management process is to provide the framework
for decision making and accountability required to ensure IT
investments meet the strategic and business objectives of the
department in an efficient and effective manner. In carrying out this
process, the department makes a distinction between its major and
nonmajor investments, to determine the extent and scope of the
department's investment management oversight and the level of reporting
requirements.
An IT investment is classified as major if it meets at least one of the
following criteria:[Footnote 10]
* requires special management attention because of its importance to
the mission or function of the agency, a component of the agency or
another organization;
* is for financial management and obligations of more than $500,000
annually;
* has significant program or policy implications;
* has high executive visibility;
* has high development, operating, or maintenance costs;
* has total life-cycle costs exceeding $50 million;
* has an annual budget of $5 million or more; or:
* significantly impacts more than one bureau.
Investments that do not meet at least one of these criteria are
considered to be nonmajor investments.
Several groups and individuals play a role in the department's process
to manage its IT investments at the department and bureau levels. They
are involved in all aspects of the process, including reviewing and
approving proposed investments, monitoring the investments through
implementation, and evaluating the investments once they become
operational. Table 1 identifies the groups and individuals that have a
role in this process and shows their composition and responsibilities.
Table 1: Governance Roles and Responsibilities:
Governance entity: Treasury Executive Investment Review Board; (E-
Board)[A];
Membership/description: Chaired by Treasury Deputy Secretary; co-vice-
chaired by Treasury CIO and Assistant Secretary for Management;
membership consists of bureau heads;
Example of responsibilities:
* Approves and governs major investments;
* Ensures proposed investments (IT and non-IT investments) meet
strategic, business, and technical objectives;
* Reviews periodic investment updates provided by Technical Investment
Review Board;
* Makes final decision to continue, modify, or terminate an investment
that is outside of a plus or minus 10 percent cost/schedule variance;
* Makes final decision for inclusion of investments in Treasury's IT
portfolio.
Governance entity: Technical Investment Review Board (TIRB);
Membership/description: Chaired by Treasury CIO: membership consists of
bureau CIOs;
Example of responsibilities:
* Makes recommendations on technical and funding matters to the E-
board;
* Recommends policy on Capital Planning and Investment Control (CPIC),
shared infrastructure, enterprise architecture, and security issues;
* Conducts periodic reviews of the portfolio and key investments;
* Evaluates major investment adherence to Treasury and OMB capital
planning criteria;
* Assesses investment alignment with Treasury's architecture and
procurement standards.
Governance entity: IT Governance subcouncils;
Membership/description: Membership consists of four standing
committees--Capital Planning, Enterprise Architecture, Security, and
Telecommunications;
Example of responsibilities:
* Acts as liaison between the CIO and the bureaus to communicate and
assist in the implementation of standards and guidelines;
* Provides input into the development of departmentwide standards for
CPIC, Enterprise Architecture (EA), and security;
* Supports TIRB by providing leadership in formulating and implementing
CPIC policies and programs;
* Provides a forum for bureaus to discuss CPIC issues and requirements
and make recommendations to TIRB.
Governance entity: Treasury Capital Planning and Investment Control
(CPIC) team;
Membership/description: Membership consists of Treasury CIO personnel,
known as desk officers;
Example of responsibilities:
* Responsible for investment management oversight of the CPIC process;
* Develops bureau-level IT portfolio expertise and provides input and
recommendations to bureaus, Treasury CIO, and TIRB;
* Serves as points of contact for bureau CPIC coordinators and oversees
one or more bureaus;
* Responsible for scoring Exhibit 300s and coordinating information
sharing with Treasury's budget office and other critical partners.
Governance entity: Bureau CPIC coordinators;
Membership/description: CPIC coordinators from each of the nine
Treasury bureaus;
Example of responsibilities:
* Serves as the bureau's single point of contact to Treasury's CPIC
team;
* Disseminates information, instructions, and due dates to bureau
investment project managers;
* Coordinates all IT- related, bureau-specific input to bureau's Chief
Financial Officer organizations and Treasury's CPIC team.
Source: GAO analysis of Treasury data.
[A] This board currently does not exist; however, according to Treasury
officials, the department has initiated efforts to re-establish it.
[End of table]
Reviews by TIRB and the department's executive investment review board
focus on IT investments that are defined as major strategic investments
for the department. To support this process, Treasury uses an automated
portfolio management tool for collecting and maintaining data during
the four phases of the process. Various forms in the tool are available
for staff to enter new and updated data on Treasury's IT investments.
Process for Managing Investments:
In September 2005, the department issued a Capital Planning and
Investment Control Policy Guide defining a four-phase process for
managing its IT investments.[Footnote 11] These phases consist of
preselect, select, control, and evaluate. Completing the requirements
of one phase is necessary before moving on to the subsequent phase.
Each phase is to be overseen by Treasury's executive investment review
board, which ultimately approves or rejects an investment's advancement
to the next phase.
* Preselect phase is the annual process by which potential new major
investments seeking funding in the upcoming budget year are approved to
move into the select phase and are considered for inclusion in the
department's budget request. Only major IT investments are promoted
through the preselect process and reviewed at the departmental level.
During this phase, an investment's business owner is to document the
business need for the investment and describe its anticipated alignment
with bureau, Treasury, and e-government initiatives,[Footnote 12] and
the President's Management Agenda[Footnote 13] strategic goals. The
CPIC team is then expected to review and validate the preselect data
and pass on its assessment and recommendation to TIRB, which is to
provide recommendations to the department's executive investment review
board. Once a major investment is approved by the executive investment
review board, it moves forward to the select phase. The department's
bureaus have the exclusive responsibility for the preselection of
nonmajor investments within their respective bureaus, and the bureaus'
executive leadership must approve a nonmajor investment in order for it
to enter the select phase.
* Select phase is the process by which new and existing major IT
investments seeking funding in the upcoming budget year are annually
screened, scored, and selected for inclusion in Treasury's IT
investment portfolio. In this phase, Treasury is to ensure that only IT
investments that best support its mission, investment principles, and
approach to EA are chosen and that the investment owners have taken
steps to be successful, such as having a qualified project manager and
analyzing risks. As in the preselect phase, the CPIC team is expected
to review and validate that all data is complete, score each investment
based on Treasury's investment principles, and submit its findings and
recommendations to TIRB. TIRB, in turn, is to review the scoring
results and provide its recommendations to Treasury's executive
investment review board, which is then to select which investments will
be included in the department's IT investment portfolio that is
ultimately submitted to OMB for funding considerations. Investments do
not technically exit the select phase until they are terminated, since
they must be reviewed annually for reselection. The bureaus are
responsible for conducting their own select process for nonmajor
investments.
* Control Phase ensures, through timely oversight, quality control, and
executive review, that IT investments are managed in a disciplined and
consistent manner. This phase is characterized by Treasury's Office of
the CIO initiating quarterly control reviews, which focus on ensuring
that an investment's projected benefits are being realized; that cost,
schedule, and performance goals are being met; that risks are minimized
and managed; and that the investment continues to meet strategic goals.
Through Office of the CIO quarterly data calls, bureau project managers
are to update data as of the end of the previous quarter for cost and
schedule, performance measures, and risk assessments for both major and
nonmajor investments. This updated data is to be entered into the
department's automated IT portfolio management tool, which the bureau
project managers and the bureau CIOs are to certify for accuracy using
a certification form within the tool. Next, Treasury's CPIC team is to
evaluate the data and provide feedback to the bureaus through the
bureaus' CPIC coordinators, giving the bureaus an opportunity to
remediate missing or erroneous data. For major investments, the CPIC
team is then expected to summarize the results, including identifying
corrective actions planned, for presentation to TIRB. TIRB is to review
the results for potential risk factors, such as schedule or cost
slippages or major technical problems, before forwarding its
recommendation to Treasury's executive investment review board. The
executive investment review board is to review TIRB's recommendations
before making a decision to continue, accelerate, modify, suspend, or
terminate investments. While control data are captured for nonmajor
investments, the department leaves it to the bureaus to conduct their
own oversight process for these investments. However, TIRB and the
executive investment review board may choose to review these
investments on a random sample basis.
In July 2006, Treasury adopted procedures for establishing an Internal
Watch List of major investments at risk of not meeting established
goals.[Footnote 14] The criteria for placement on this list include:
1. cost or schedule variances greater than plus or minus 10 percent for
two consecutive quarters;
2. lack of validation of project manager's qualifications by the bureau
CIO;
3. lack of a current certification and accreditation;[Footnote 15] or:
4. duplication of another investment within the department or with any
of the President's e-government initiatives or lines of
business.[Footnote 16]
Treasury's Office of the CIO is to make this determination, and
investments on this list are subject to additional reporting
requirements, including development of an action plan to remediate the
noncompliant conditions. Bureau CIOs are to report monthly to the
Treasury CIO on the status of these investments. Once all requirements
have been met and the Treasury CIO concurs, the investment can be
removed from the list.
* Evaluation phase involves an annual process to determine how well
major investments are performing once they become operational. This
process is to occur in the first quarter of the fiscal year and is
composed of two subprocesses--the postimplementation review (PIR) and
the operational analysis (OA). The age and the life cycle stage of the
investment determine which of these two subprocesses is conducted on an
investment.
The purpose of the PIR is to assess the performance of an investment
that has been fully developed and has moved into the operational and
maintenance stage of its life cycle. An investment's project manager is
to initiate a PIR 6 to 18 months after an investment has moved into its
operational and maintenance stage. During a PIR, an investment's actual
performance is compared to its expected performance to identify lessons
learned for improving both the investment and Treasury's CPIC process.
The PIR is also intended to measure the strategic impact, user
satisfaction, and whether the investment is meeting cost, schedule, and
performance metrics. The results of the PIR are to be documented in
Treasury's portfolio management tool. Once the PIR is completed,
Treasury's CPIC team is to evaluate the results, provide feedback to
the project manager and the respective bureau management, and provide
summary information to TIRB. TIRB, in turn, is to report lessons
learned from the PIRs conducted and any recommendations it may have to
the department's executive investment review board in order to promote
the lessons learned across the department's IT investment portfolio.
* The purpose of the OA is to identify those investments in operations
and maintenance for which PIRs have been conducted that are likely to
require modification, acceleration, replacement, or retirement, and to
help determine the remaining useful life of an investment. However,
because of the newness of Treasury's PIR requirement and the age of
certain investments that have been in the operations and maintenance
stage of their life cycle, a PIR may not have been performed on these
investments prior to the required OA. Similar to a PIR, in conducting
the OA, Treasury focuses on two key areas: (1) program objectives,
looking at alignment to cost, schedule, and strategic goals; and (2)
meeting user needs. In determining how well the investment aligns to
program objectives, data are to be captured on an annual basis---most
likely from established sources, such as the quarterly control reviews
and annual select phase process. To determine whether user needs are
still being met by the investment, the investment's project manager, in
coordination with the investment's business owner, is to solicit user
input, using such means as a survey, focus groups, or regular user
group meetings. The results of the OA are to be documented in
Treasury's portfolio management tool and can entail recommending the
investment continue operations as is, be modified, or be terminated.
Based on further analysis by the CPIC team, a review meeting may be
scheduled to discuss the results and the recommendations. The results
of these meetings are to be shared with TIRB and the executive
investment review board, as appropriate. Prior to exiting the
evaluation phase, the executive investment review board must approve
the disposal, retirement, or replacement of major investments.
Figure 2 shows the schedule of select, control, and evaluate activities
that take place throughout the year.
Figure 2: CPIC Process:
[See PDF for image]
Source; Department of the Treasury.
[A] Budget year is a term used in the budget formulation process that
refers to the fiscal year for which the budget is being considered,
that is, with respect to a session of Congress, the fiscal year of the
government that starts on October 1 of the calendar year in which that
session of Congress begins.
[B] E-board--Executive Investment Review Board.
[C] TIRB--Technical Investment Review Board.
[End of figure]
ITIM Maturity Framework:
To provide a method for evaluating and assessing how well an agency is
selecting and managing its IT resources, GAO developed the Information
Technology Investment Management framework (ITIM).[Footnote 17] The
ITIM is a maturity model composed of five progressive stages of
maturity that an agency can achieve in its investment management
capabilities. It was developed on the basis of our research into the IT
investment management practices of leading private-and public-sector
organizations. In each of the five stages, the framework identifies
critical processes for making successful IT investments. The maturity
stages are cumulative; that is, in order to attain a higher stage, the
agency must have institutionalized all of the critical processes at the
lower stages in addition to the higher stage critical processes.
The framework can be used to assess the maturity of an agency's
investment management processes and as a tool for organizational
improvement. The overriding purpose of the framework is to encourage
investment processes that increase business value and mission
performance, reduce risk, and increase accountability and transparency
in the decision process. We have used the framework in several of our
evaluations,[Footnote 18] and a number of agencies have adopted it.
These agencies have used ITIM for purposes ranging from self-assessment
to redesign of their IT investment management processes.
ITIM's five maturity stages represent the steps toward achieving stable
and mature processes for managing IT investments. Each stage builds on
the lower stages, and the successful attainment of each stage leads to
improvement in the organization's ability to manage its investments.
With the exception of Stage 1, each maturity stage is composed of
critical processes that must be implemented and institutionalized in
order for the organization to achieve that stage.[Footnote 19] These
critical processes are further broken down into key practices that
describe the types of activities an organization should be performing
to successfully implement each critical process. It is not unusual for
an organization to be performing key practices from more than one
maturity stage at the same time, but efforts to improve investment
management capabilities should focus on implementing all lower stage
practices before addressing higher stage practices.
In the ITIM, Stage 2 critical processes lay the foundation for sound IT
investment processes by helping the agency to attain successful,
predictable, and repeatable investment control processes at the project
level. Specifically, Stage 2 encompasses building a sound investment
management foundation by establishing basic capabilities for selecting
new IT projects. It involves developing the capability to control
projects so that they finish predictably within established cost and
schedule expectations and have the capability to identify potential
exposures to risk and put in place strategies to mitigate that risk. It
also involves instituting an IT investment board,[Footnote 20] which
includes defining its membership, guidance policies, operations, roles,
responsibilities, and authorities for one or, if applicable, more IT
investment boards within the organization, and, if appropriate, each
board's support staff. The basic selection processes established in
Stage 2 lay the foundation for more mature selection capabilities in
Stage 3, which represents a major step forward in maturity. In this
stage, the agency moves from project-centric processes to a portfolio
approach, evaluating potential investments by how well they support the
agency's mission, strategies, and goals.
Stage 3 requires that an organization continually assess both proposed
and ongoing projects as parts of a complete investment portfolio--an
integrated and competing set of investment options. It focuses on
establishing a consistent, well-defined perspective on the IT
investment portfolio and maintaining mature, integrated selection (and
reselection), control, and evaluation processes, which are to be
evaluated during PIRs. This portfolio perspective allows decision
makers to consider the interaction among investments and the
contributions to organizational mission goals and strategies that could
be made by alternative portfolio selections, rather than focusing
exclusively on the balance between the costs and benefits of individual
investments.
Stages 4 and 5 require the use of evaluation techniques to continuously
improve both the investment portfolio and the investment processes in
order to better achieve strategic outcomes. At Stage 4 maturity, an
organization has the capacity to conduct IT succession activities and,
therefore, can plan and implement the deselection of obsolete, high-
risk, or low-value IT investments. An organization with Stage 5
maturity conducts proactive monitoring for breakthrough information
technologies that will enable it to change and improve its business
performance. Organizations that have implemented Stages 2 and 3 have in
place capabilities that assist in establishing the selection, control,
and evaluation processes that are required by the Clinger-Cohen Act of
1996.[Footnote 21] Stages 4 and 5 define key attributes that are
associated with the most capable organizations.
Figure 3 shows the five ITIM stages of maturity and the critical
processes associated with each stage.
Figure 3: ITIM Stages of Maturity:
[See PDF for image]
Source: GAO.
[End of figure]
As defined by the model, each critical process consists of key
practices that must be executed to implement the critical process.
Treasury Has Established Many Key Practices for Managing Its
Investments, but Has Key Weaknesses with Its Board Operations and
Investment Oversight:
In order to have the capabilities to effectively manage IT investments,
an agency, at a minimum, should (1) build an investment foundation by
putting basic, project-level control and selection practices in place
(Stage 2 capabilities) and (2) manage its projects as a portfolio of
investments, treating them as an integrated package of competing
investment options and pursuing those that best meet the strategic
goals, objectives, and mission of the agency (Stage 3 capabilities).
These practices may be executed at various organizational levels of the
agency, including at the component level. However, overall
responsibility for their success remains at the department level.
Therefore, at a minimum, the department should effectively oversee
component agencies' IT investment management processes.
While Treasury has established many of the capabilities needed to
select, control, and evaluate its IT investments, the department has
significant weaknesses that hamper its ability to effectively manage
its investments. Specifically, the department has executed 19 of the 38
key practices that the ITIM requires to build a foundation for IT
investment management (Stage 2), including practices needed to ensure
that projects support business needs and that a disciplined process
exists for capturing investment information. In addition, the
department has executed 11 of the 27 key practices required to manage
investments as a portfolio (Stage 3), including documenting policies
and procedures for conducting postimplementation reviews.
However, Treasury does not have an executive investment review board--
a group of executives from IT and business units that is intended to be
the final decision-making authority--that is actively engaged in the
investment management process. According to the Associate CIO for
Capital Planning and Information Management, while efforts to establish
an executive investment review board have been initiated, these efforts
have been stymied by changes in executive leadership. In addition, the
department does not have any processes in place for managing its
nonmajor investments, although they represent about 70 percent of the
total number of investments. According to officials, nonmajor
investments have not been a priority because the department has instead
chosen to devote its resources to major investments, which represent
about 80 percent of its IT expenditures. While it is reasonable to
focus attention on major investments, nonmajor investments represent a
significant amount of funding (about $480 million) and constitute the
bulk of most bureaus' investment portfolio and therefore also require a
certain level of oversight. Until the department addresses these
weaknesses, it will not have the investment management structure needed
to effectively assess and manage the risks associated with its
multibillion-dollar portfolio.
In addition, until the department addresses these weaknesses, it will
not have assurance that key investment management decisions are
benefiting from the contribution of executives who are in the best
position to make the full range of decisions needed to enable the
agency to meet its mission most effectively. In addition, the
department will not be able to ensure that it is effectively assessing
and managing the risks associated with nonmajor investments costing
hundreds of millions of dollars.
Treasury Has Established Many of the Foundational Practices Needed to
Manage its Investments:
At the ITIM Stage 2 level of maturity, an organization has attained
repeatable, successful IT project-level investment control and basic
selection processes. Through these processes, the organization can
identify expectation gaps early and take the appropriate steps to
address them. According to ITIM, critical processes at Stage 2 include
(1) defining IT investment board operations, (2) identifying the
business needs for each IT investment, (3) developing a basic process
for selecting new IT proposals and reselecting ongoing investments, (4)
developing project-level investment control processes, and (5)
collecting information about existing investments to inform investment
management decisions. Table 2 describes the purpose of each of these
Stage 2 critical processes.
Table 2: Stage 2 Critical Processes--Building the Investment
Foundation:
Critical process: Instituting the investment board;
Purpose: To define and establish an appropriate IT investment
management structure and the processes for selecting, controlling, and
evaluating IT investments.
Critical process: Meeting business needs;
Purpose: To ensure that IT projects and systems support the
organization's business needs and meet users' needs.
Critical process: Selecting an investment;
Purpose: To ensure that a well-defined and disciplined process is used
to select new IT proposals and reselect ongoing investments.
Critical process: Providing investment oversight;
Purpose: To review the progress of IT projects and systems, using
predefined criteria and checkpoints, in meeting cost, schedule, risk,
and benefit expectations and to take corrective action when these
expectations are not being met.
Critical process: Capturing investment information;
Purpose: To make available to decision makers information to evaluate
the impacts and opportunities created by proposed (or continuing) IT
investments.
Source: GAO.
[End of table]
Because of management attention that has recently been given to IT
investment management, Treasury has put in place half of the key
practices needed to establish the investment foundation. These include
all of the key practices associated with identifying and collecting
information to support investment decisions and some of the key
practices for ensuring that projects and systems support organizational
needs and meet users' needs as well as for selecting new
proposals[Footnote 22] and reselecting ongoing investments.
However, because no executive investment review board currently exists
(see details in next section), the department has not executed many of
the key practices for instituting the investment board. In addition,
because of its limited involvement in managing nonmajor investments,
the department has not executed many of the key practices related to
providing investment oversight. Treasury officials stated that the
management turnover present a challenge to establishing an executive
investment review board. They also acknowledged the need for a process
to oversee nonmajor investments, particularly in light of the recent
failure of the BSA Direct project.
Table 3 summarizes the status of Treasury's Stage 2 critical processes,
showing how many associated key practices the department has executed.
Table 3: Summary of Results for Stage 2 Critical Processes and Key
Practices:
Critical process: Instituting the investment board;
Key practices executed: 3;
Total required by critical process: 8;
Percentage of key practices executed: 38.
Critical process: Meeting business needs;
Key practices executed: 2;
Total required by critical process: 7;
Percentage of key practices executed: 29.
Critical process: Selecting an investment;
Key practices executed: 6;
Total required by critical process: 10;
Percentage of key practices executed: 60.
Critical process: Providing investment oversight;
Key practices executed: 2;
Total required by critical process: 7;
Percentage of key practices executed: 29.
Critical process: Capturing investment information;
Key practices executed: 6;
Total required by critical process: 6;
Percentage of key practices executed: 100.
Critical process: Total;
Key practices executed: 19;
Total required by critical process: 38;
Percentage of key practices executed: 50.
Source: GAO.
[End of table]
Treasury Does Not Have an Executive Investment Review Board:
The establishment of decision-making bodies or boards is a key
component of the IT investment management process. At the Stage 2 level
of maturity, organizations define one or more boards, provide resources
to support the boards' operations, and appoint members who have
expertise in both operational and technical aspects of proposed
investments. The boards should operate according to a written IT
investment process guide that is tailored to the organization's unique
characteristics, thus ensuring that consistent and effective management
practices are implemented across the organization. The organization
selects board members to ensure they are knowledgeable about policies
and procedures for managing investments. Organizations at the Stage 2
level of maturity also take steps to ensure that executives and line
managers support and carry out the decisions of the investment board.
According to ITIM, organizations should (1) establish an enterprisewide
IT investment board composed of senior executives from IT and business
units, (2) have a documented IT investment process directing each
investment board's operations, and (3) ensure that the enterprisewide
investment board has oversight responsibilities for the development and
maintenance of the organization's documented IT investment process.
(The complete list of key practices is provided in table 4.)
Treasury has executed three of the eight key practices for this
critical process. For example, the department has documented an IT
investment process that directs investment board operations. In
addition, adequate resources are provided to support board operations.
However, Treasury currently does not have an executive investment
review board composed of senior executives from IT and business units
that is actively engaged in the investment management process.
According to officials, such a board was established in 2005 but
stopped functioning at the prompting of the assistant secretary for
management because it was considered inefficient. In 2006, another
executive investment review board structure was proposed under a new
assistant secretary for management, but, according to the Associate CIO
for Capital Planning and Information Management, it was not
implemented, due to yet another change in executive leadership.
Officials told us that one of the challenges in establishing the board
has been the constant turnover in Treasury's management. They noted
that many of the management positions, including the assistant
secretary for management position, are currently being filled by
temporary or "acting" officials, who may be replaced soon. Until the
department establishes an executive investment review board with senior
executives from IT and business units, its investment management
process will not benefit from the contribution of those executives who
are in the best position to make the full range of decisions needed for
the department to meet its mission most effectively.
Table 4 shows the rating for each key practice required to implement
the critical process for instituting the investment board at the Stage
2 level of maturity and summarizes the evidence that supports these
ratings.
Table 4: Instituting the Investment Board:
Type of practice: Organizational commitments;
Key practice: 1. An enterprisewide IT investment board composed of
senior executives from IT and business units is responsible for
defining and implementing the organization's IT investment governance
process;
Rating: not executed;
Summary of evidence: According to Treasury's CPIC guide, the
department's investment management structure includes an executive
investment review board that is responsible for defining and
implementing Treasury's IT investment governance process. However, this
board does not exist to perform this practice.
Key practice: 2. The organization has a documented IT investment
process directing each investment board's operations;
Rating: executed;
Summary of evidence: Treasury's CPIC guide outlines the IT investment
process that directs the operations of the executive investment review
board and TIRB, which are part of the investment management structure.
The guide specifies the roles of key entities involved in the
organization's IT investment process and explains procedures for
assigning responsibility for decision making for IT investments. The
CPIC guide specifies that the bureaus retain decision-making authority
for nonmajor IT investments, while adhering to the department-level IT
investment management process.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, are provided for supporting the operations of each IT investment
board;
Rating: executed;
Summary of evidence: Although the executive investment review board
does not exist, adequate resources are provided to support its
operations, including TIRB and CPIC office staff, with bureau desk
officers that are responsible for, among other things, aiding in
compiling relevant IT investment management data for the board's
review.
Key practice: 2. The board members understand the organization's IT
investment management policies and procedures and the tools and
techniques used in the board's decision-making process;
Rating: not executed;
Summary of evidence: Treasury has in place informal mechanisms they use
to keep executives informed of the department's IT investment
management policies, procedures, tools, and techniques, including
presentations given to the bureaus regarding the CPIC process. However,
no executive investment review board exists to perform this key
practice.
Key practice: 3. Each board's span of authority and responsibility is
defined to minimize overlaps or gaps among the boards;
Rating: executed;
Summary of evidence: According to Treasury's CPIC guide and officials,
although the executive investment review board does not exist, its
defined responsibilities include defining and implementing the
department's IT investment governance process.
Type of practice: Activities;
Key practice: 1. The enterprisewide investment board has oversight
responsibilities for the development and maintenance of the
organization's documented IT investment process;
Rating: not executed;
Summary of evidence: According to officials, Treasury's executive
investment review board is supposed to be involved in the development
and maintenance of the department's documented IT investment process
through TIRB, which provides investment management policy change
recommendations to the board for approval. However, this board does not
exist to perform this activity.
Key practice: 2. Each investment board operates in accordance with its
assigned authority and responsibility;
Rating: not executed;
Summary of evidence: The Treasury CPIC guide outlines the roles and
responsibilities of the department's executive investment review board;
however, this board does not exist to perform this activity.
Key practice: 3. The organization has established management controls
for ensuring that investment boards' decisions are carried out;
Rating: not executed;
Summary of evidence: The Treasury CPIC Team is responsible for ensuring
that board decisions are carried out. However, the executive investment
review board does not exist to perform this activity.
Source: GAO.
[End of table]
Treasury Has a Process for Ensuring Projects Are Aligned with Business
Needs:
Defining business needs for each IT project helps to ensure that
projects and systems support an organization's business needs and meet
users' needs. This critical process ensures that an organization's
business objectives and its IT management strategy are linked.
According to ITIM, effectively meeting business needs requires, among
other things, (1) documenting business needs with stated goals and
objectives, (2) identifying specific users and other beneficiaries of
IT projects and systems, (3) providing adequate resources to ensure
that projects and systems support the organization's business needs and
meet users' needs, and (4) periodically evaluating the alignment of IT
projects and systems with the organization's strategic goals and
objectives. (The complete list of key practices is provided in table
5.)
Treasury has executed two of the seven key practices for ensuring
business needs are met. Specifically, Treasury has a documented
business mission, with stated goals and objectives in its Treasury
Strategic Plan for fiscal years 2003 through 2008. In addition,
resources are devoted to ensuring that IT projects and systems support
the organization's business needs and meet users' needs, including
Treasury's portfolio management tool, several subcouncils, an Exhibit
300 scoring guide to help develop major IT investments business cases,
and training manuals on the use of the portfolio management tool
contained in an online resource called the CPICResource Link.
Treasury's weaknesses in this area stem mostly from the fact that,
while the department has delegated the management of nonmajors to the
bureaus, it has no mechanism for ensuring that bureaus are effectively
carrying out associated activities. In addition, while Treasury's
system development life-cycle methodology requires user involvement in
projects' life cycle, the investment management process does not have
any steps for ensuring this is done. By not ensuring that bureaus are
effectively aligning nonmajor investments with business needs, Treasury
is incurring the risk that investments that make up approximately 20
percent of their IT budget and represent the majority of their
investments may not be supporting the department's priorities. In
addition, without an executive investment review board actively
involved in the process, Treasury cannot be assured it is making the
best decisions regarding investments' ability to support ongoing and
future business needs.
Table 5 shows the rating for each key practice required to implement
the critical process for meeting business needs at the Stage 2 level of
maturity and summarizes the evidence that supports these ratings.
Table 5: Meeting Business Needs:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for identifying IT projects or systems that support the
organization's ongoing and future business needs;
Rating: not executed;
Summary of evidence: Treasury has policies and procedures for ensuring
that major IT projects and systems support the department's ongoing and
future business needs in its CPIC guide and the preselect section of
its Enterprise Architecture Guidance. While Treasury has delegated the
management of nonmajor investments to the bureaus, it does not have a
mechanism for ensuring that the bureaus have policies and procedures to
address this critical process.
Type of practice: Prerequisites;
Key practice: 1. The organization has a documented business mission
with stated goals and objectives;
Rating: executed;
Summary of evidence: The Treasury Strategic Plan for fiscal years 2003
through 2008 defines the agency's mission goals and objectives. The
plan defines goal categories, goals, and objectives linked to the
goals.
Key practice: 2. Adequate resources, including people, funding, and
tools, are provided for ensuring that IT projects and systems support
the organization's business needs and meet users' needs;
Rating: executed;
Summary of evidence: Treasury has adequate resources for ensuring that
its IT projects and systems support the organization's business needs
and meet users' needs. They include a portfolio management tool, TIRB,
and several subcouncils. Also, Treasury has an Exhibit 300 scoring
guide to help develop business cases and training manuals on the use of
the portfolio management tool contained in an online resource called
the CPICResource Link.
Type of practice: Activities;
Key practice: 1. The organization defines and documents business needs
for both proposed and ongoing IT projects and systems;
Rating: not executed;
Summary of evidence: The preselect and select processes defined in
Treasury's CPIC guide specify how Treasury defines and documents
business needs for both proposed and ongoing major IT projects and
systems. Major investments business needs are documented within the
portfolio management tool. All of the major investments we reviewed--
TFIN, SaBRe, and CADE--had documented their business needs within the
portfolio management tool. For nonmajor investments, Treasury has
delegated this key practice to the bureaus; however, the department
does not have a mechanism for ensuring the bureaus are effectively
executing it.
Key practice: 2. The organization identifies specific users and other
beneficiaries of IT projects and systems;
Rating: not executed;
Summary of evidence: Users are supposed to be identified in the
preselect and select phases, as outlined in the performance measurement
section of Treasury's CPIC guide. The guide states that the following
information be documented with regard to the users: identify who will
use the system, describe the principal business task they will perform,
and describe how they will use the system to help them perform their
principal business task. We verified that the three major projects we
reviewed--TFIN, SaBRE, and CADE--had identified the users of their
system in the portfolio management tool. For nonmajor investments,
Treasury has delegated this key practice to the bureaus; however, the
department does not have a mechanism for ensuring the bureaus are
effectively executing it.
Key practice: 3. Users participate in project management throughout an
IT project's or system's life cycle;
Rating: not executed;
Summary of evidence: Treasury's system development life-cycle
methodology requires user involvement throughout projects' life cycle.
For example, users are to be involved in quality and assurance and
configuration management. Treasury's investment management process,
however, does not include steps to ensure that this activity is
actually being performed until investments are in operations and
maintenance.
Key practice: 4. The investment board periodically evaluates the
alignment of its IT projects and systems with the organization's
strategic goals and objectives and takes corrective actions when
misalignment occurs;
Rating: not executed;
Summary of evidence: According to Treasury's CPIC guide, the investment
board is supposed to evaluate the alignment of major IT projects and
systems with the organization's strategic goals and objectives and take
corrective action when misalignment occurs during the select phase.
However, since the executive investment review board does not exist,
this activity is not being performed. For the nonmajor investments,
Treasury has delegated this key practice to the bureaus; however, the
department does not have a mechanism for ensuring the bureaus are
effectively executing it.
Source: GAO.
[End of table]
Treasury Has Processes to Select Major Investments but Is Not
Effectively Selecting Nonmajor Investments:
Selecting new IT proposals and reselecting ongoing investments requires
a well-defined and disciplined process to provide the agency's
investment boards, business units, and developers with a common
understanding of the process and the cost, benefit, schedule, and risk
criteria that will be used both to select new projects and to reselect
ongoing projects for continued funding. According to ITIM, this
critical process requires, among other things, (1) providing adequate
resources for investment selection activities; (2) making funding
decisions for new proposals according to an established process; and
(3) using a defined selection process to select new investments and
reselect ongoing investments. (The complete list of key practices is
provided in table 6.)
Treasury has executed 6 of the 10 key practices associated with
selecting an investment. Treasury's portfolio management tool contains
a form for entering select data and provides staff, such as project
managers and CPIC desk officers, with information to help manage the
select process. We verified that three of the systems we reviewed--
TFIN, CADE, and SaBRe--did, in fact, use the select form in the
portfolio management tool for entering select data. The department has
aligned funding decisions with the budget process for new and ongoing
investments through the department's budget formulation process, which
is used to select both enterprisewide and bureau investments. Treasury
has also documented criteria for analyzing, prioritizing, selecting,
and reselecting new and ongoing major investments that address its
strategic goals and its IT strategic goals, value, and risk. The
criteria are incorporated into the department's portfolio management
tool and adjusted within the tool to reflect organizational objectives.
However, the executive investment review board that is supposed to make
final selection and reselection decisions does not exist. Treasury
officials state that, as part of the budget formulation process, the
results of the select process are approved by executives and that the
results of the fiscal year 2008 select process were approved by a group
of executives, including the Treasury Assistant Secretary for
Management and other department and bureau executives, prior to being
forwarded to OMB. The officials recognized, however, that this group
was convened only for that purpose and did not include business (i.e.,
mission) representation from the bureaus.
In addition, Treasury has delegated the selection and reselection of
the nonmajor systems to the bureaus; however, as previously noted,
Treasury does not have a mechanism for ensuring that the bureaus are
effectively carrying out these activities. Without such a mechanism,
Treasury cannot have assurance that investments that make up
approximately 20 percent of its budget and represent the majority of
investments are being consistently and objectively selected and
reselected.
Table 6 shows the rating for each key practice required to implement
the critical process for selecting an investment at the Stage 2 level
of maturity and summarizes the evidence that supports these ratings.
Table 6: Selecting an Investment:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for selecting new IT proposals;
Rating: not executed;
Summary of evidence: Treasury has documented policies and procedures
for selecting major investments in its CPIC guide. The selection of
nonmajor investments is delegated to the bureaus. However, Treasury has
no mechanism for ensuring the bureaus have effective selection policies
and procedures.
Key practice: 2. The organization has documented policies and
procedures for reselecting ongoing IT investments;
Rating: not executed;
Summary of evidence: Treasury has documented policies and procedures
for reselecting major investments in its CPIC guide. The reselection of
nonmajor investments is delegated to the bureaus. However, Treasury has
no mechanism for ensuring the bureaus have effective reselection
policies and procedures.
Key practice: 3. The organization has policies and procedures for
integrating funding with the process of selecting an investment;
Rating: executed;
Summary of evidence: The CPIC guide calls for the budget process to be
aligned with the investment management process. The process for doing
so is by integrating the select data calls with the budget exercises
through the use of the CPIC calendar. Additionally, treasury officials
stated that acquisition processes are also entered into the portfolio
management tool, which helps to align the funding with the select
process.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, are provided for identifying and selecting IT projects and
systems;
Rating: executed;
Summary of evidence: Adequate resources are provided for identifying
and selecting major IT projects and systems. They include the desk
officers, CPIC team, the CPIC subcouncils, and the department's
portfolio management tool, which contains forms for selecting IT
projects and systems. Nonmajor investments are selected by the bureaus.
Key practice: 2. Criteria for analyzing, prioritizing, and selecting
new IT investment opportunities have been established;
Rating: executed;
Summary of evidence: Treasury has established criteria for analyzing,
prioritizing, and selecting enterprise and bureau IT investments. They
include strategic alignment, EA alignment, and cost, schedule, benefit,
and risk factors.
Key practice: 3. Criteria for analyzing, prioritizing, and reselecting
IT investment opportunities have been established;
Rating: executed;
Summary of evidence: Treasury has established criteria for analyzing,
prioritizing, and reselecting new IT investments for Treasury and its
bureaus. They include strategic alignment, EA alignment, and cost,
schedule, benefit, and risk factors.
Key practice: 4. A mechanism exists to ensure that the criteria
continue to reflect organizational objectives;
Rating: executed;
Summary of evidence: Treasury reviews and adjusts the select criteria
through a working group called the Select Phase Optimization Working
Group. This group meets and discusses changes to the select criteria
and updates the guidance and the portfolio management tool to reflect
the changes.
Type of practice: Activities;
Key practice: 1. The organization uses its defined selection process,
including predefined selection criteria, to select new IT investments;
Rating: not executed;
Summary of evidence: Treasury's CPIC guide outlines the select process
and directs all bureaus to use the reselect process to select new major
investments. However, the executive investment review board that is
supposed to make final selection decisions does not exist. In addition,
the selection of nonmajor investments is delegated to the bureaus, but
the department has no process for ensuring the bureaus are effectively
carrying out selection activities.
Key practice: 2. The organization uses the defined selection process,
including predefined selection criteria, to reselect ongoing IT
investments;
Rating: not executed;
Summary of evidence: Treasury's CPIC guide outlines the select process
and directs all bureaus to use the select process to reselect new major
investments. However, the board that is supposed to make final
reselection decisions does not exist. In addition, reselection of
nonmajor investments is delegated to the bureaus, but the department
has no process for ensuring the bureaus are effectively carrying out
reselection activities.
Key practice: 3. Executives' funding decisions are aligned with
selection decisions;
Rating: executed;
Summary of evidence: Treasury makes funding decisions for new and
ongoing investments through the department's budget formulation
process, which is used to reselect major ongoing enterprise and bureau
investments.
Source: GAO.
[End of table]
Treasury Is Not Effectively Overseeing Its Investments:
An organization should effectively oversee its IT projects throughout
all phases of their life cycles. An investment board should observe
each project's performance and progress toward predefined cost and
schedule expectations as well as each project's anticipated benefits
and risk exposure. This does not mean that a departmental board should
micromanage each project to provide effective oversight; rather, it
means that the departmental board should be actively involved in all IT
investments and proposals that are high cost or high risk or have
significant scope and duration and, at a minimum, should have a
mechanism for maintaining visibility of all investments. The board
should also use early warning systems that enable it to take corrective
actions at the first sign of cost, schedule, and performance slippages.
According to ITIM, effect project oversight requires, among other
things, (1) having written policies and procedures for management
oversight; (2) developing and maintaining an approved management plan
for each IT project; (3) making up-to-date cost and schedule data for
each project available to the oversight boards; (4) having regular
reviews by each investment board of each project's performance against
stated expectations; and (5) ensuring that corrective actions for each
underperforming project are documented, agreed to, implemented, and
tracked until the desired outcome is achieved. (The complete list of
key practices is provided in table 7.)
Treasury has executed two of the seven key practices associated with
effective project oversight. Treasury has adequate resources to support
the executive investment review board for this critical process. The
TIRB conducts quarterly control reviews of IT investments and can make
recommendations to the executive investment review board based on these
reviews. The department uses an automated portfolio management tool for
the collection and maintenance of information to support the
department's quarterly control reviews. Treasury's CPIC team, composed
of Office of the Chief Information Officer (OCIO) personnel, assists
the bureaus in compiling data on their respective IT portfolios,
reviewing the data for accuracy and completeness prior to submission to
TIRB for its quarterly control reviews. In addition, the bureaus have
CPIC coordinators, each of which serve as a single point of quality
control for their respective bureaus before information is released to
OCIO's CPIC team and provide assistance in addressing CPIC team
comments received during the department's quarterly control reviews. In
addition, we verified that cost, schedule, benefits, and risk
expectations were documented for the four projects we reviewed: CADE,
SaBRe, TFIN, and TRACS. All four projects maintained project management
plans or other documents that captured this information.
However, although the department has written policies and procedures
for management oversight of its investments, including its Capital
Planning and Investment Control Policy Guide and its Earned Value
Management Policy Guide, these policies and procedures are centered on
the department's major investments. Treasury leaves oversight of its
nonmajor investments to the bureaus. According to Treasury officials,
the department has thus far focused on the major investments because
they represent about 80 percent of its IT expenditures. Until the
department develops a mechanism for TIRB and its executive investment
review board to periodically conduct nonmajor portfolio reviews, as
indicated in its CPIC guide, or develops a mechanism for ensuring that
the bureaus are doing so, the department risks not being able to
identify investment problems when it is easier and less costly to
resolve them.
In addition, because the executive investment review board does not
exist, Treasury is not executing any of the activities associated with
providing investment oversight. Specifically, there is no executive
investment review board to receive actual investment performance data,
review the performance of projects and systems against expectations,
and ensure that appropriate actions are taken to correct or terminate
underperforming projects. The TIRB is currently carrying out these
activities. However, without the involvement of an executive investment
review board, these reviews are being performed without the corporate
perspective that is useful in determining the impact individual project
decisions may have on other projects and on the attainment of
organizational goals and objectives.
Table 7 shows the rating for each key practice required to provide
investment oversight and summarizes the evidence that supports these
ratings.
Table 7: Providing Investment Oversight:
Type of practice: Organizational commitment;
Key practice: 1. The organization has documented policies and
procedures for management oversight of IT projects and systems;
Rating: not executed;
Summary of evidence: Treasury has documented policies and procedures
for major investments in its CPIC guide and its Earned Value Management
Policy Guide. These guides specify the oversight responsibilities of
TIRB and the department's executive investment review board. Treasury
has delegated management oversight of nonmajor investments to the
bureaus. However, the department does not have any mechanism to ensure
the bureaus have effective policies and procedures for carrying out
this process.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, are provided for IT project oversight;
Rating: executed;
Summary of evidence: Treasury has adequate resources for providing IT
project oversight. Specifically, TIRB conducts quarterly control
reviews of IT investments and can make recommendations to the executive
investment review board based on these reviews. The CPIC team in the
department's Office of the CIO assists the bureaus in compiling data on
the bureaus' IT investment portfolios for the quarterly TIRB control
process. The bureaus' CPIC coordinators serve as the bureaus' single
point of contact to the CPIC team, providing a point of quality control
before information is released to the CPIC team. Also, the department
has an automated tool to facilitate the collection and maintenance of
information to support the agency's quarterly control process.
Key practice: 2. IT projects and systems, including those in steady
state (operations and maintenance), maintain approved project
management plans that include expected cost and schedule milestones and
measurable benefit and risk expectations;
Rating: executed;
Summary of evidence: Treasury guidance requires all projects to have a
project plan documenting expected cost, schedule, benefit, and risk.
Project managers are to track performance measures such as cost,
schedule, and risk against the project management plan to support the
control process. The four case study projects we reviewed maintained
project management plans or other documents that contain this
information.
Type of practice: Activities;
Key practice: 1. Data on actual performance (including cost, schedule,
benefit, and risk performance) are provided to the appropriate IT
investment board;
Rating: not executed;
Summary of evidence: Treasury's CPIC guide calls for data on actual
performance of major systems to be provided to both TIRB and the
executive investment review board. For the three major projects in our
case studies (CADE, SaBRe, and TFIN), we verified that actual
performance data were provided to TIRB. While TIRB receives this
information on a quarterly basis, the executive investment review board
that is supposed to make recommendations does not exist. In addition,
this activity is delegated to the bureaus for the nonmajor investments,
but Treasury has no mechanism for ensuring that the bureaus are
effectively carrying out the review.
Key practice: 2. Using verified data, each investment board regularly
reviews the performance of IT projects and systems against stated
expectations;
Rating: not executed;
Summary of evidence: During Treasury's quarterly control reviews, TIRB
reviews the performance of major IT investments against expectations
based on data provided by the bureaus. Following its review, TIRB can
make recommendations to the department's executive investment review
board. However, the department has not provided us with documentation
on the results of TIRB reviews. Also, the executive investment review
board does not exist to perform this activity. Treasury is in the
process of restructuring this board. In addition, the department has
delegated oversight of nonmajor investments to the bureaus, but does
not have a process in place for ensuring that the bureaus are
effectively carrying out this activity for nonmajor investments.
Key practice: 2. For each underperforming IT project or system,
appropriate actions are taken to correct or terminate the project or
system in accordance with defined criteria and the documented policies
and procedures for management oversight;
Rating: not executed;
Summary of evidence: The department's TIRB is provided information on
the status of IT investments, including information on underperforming
investments and corrective actions planned. Following its review, TIRB
makes recommendations to the department's executive investment review
board. However, the department has not provided us with documentation
on the results of TIRB reviews. Also, the executive investment review
board does not exist. Treasury is in the process of restructuring this
board. In addition, the department has delegated oversight of nonmajor
investments to the bureaus but does not have a process in place for
ensuring that the bureaus are effectively carrying out this activity
for nonmajor investments.
Key practice: 4. The investment board regularly tracks the
implementation of corrective actions for each underperforming project
until the actions are completed;
Rating: not executed;
Summary of evidence: Because an executive investment review board does
not exist, this key practice is not being performed. Also, the
department has delegated oversight of nonmajor investments to the
bureaus but does not have a process in place for ensuring that the
bureaus are effectively carrying out this activity for nonmajor
investments.
Source: GAO.
[End of table]
Treasury Has a Structured Process for Capturing Investment Information:
To make good IT investment decisions, an organization must be able to
acquire pertinent information about each investment and store that
information in a retrievable format. During this critical process, an
organization identifies its IT assets and creates a comprehensive
repository of investment information. This repository provides
information to investment decision makers to help them evaluate the
potential impacts and opportunities created by proposed or continuing
investments. It can provide insights into major IT cost and management
drivers and trends. The repository can take many forms and need not be
centrally located, but the collection method should, at a minimum,
identify each IT investment and its associated components. This
critical process may be satisfied by the information contained in the
organization's current enterprise architecture (EA), augmented by
additional information--such as financial information and information
on risk and benefits--that the investment board may require to ensure
that informed decisions are being made. According to ITIM, effectively
managing this repository requires, among other things, (1) developing
written policies and procedures for identifying and collecting the
information; (2) assigning responsibilities for ensuring that the
information being collected meets the needs of the investment
management process; (3) identifying IT projects and systems and
collecting relevant information to support decisions about them; and
(4) making the information easily accessible to decision makers and
others. (The complete list of key practices is provided in table 8.)
Treasury has in place all six key practices associated with capturing
investment information. For example, the department's Capital Planning
and Investment Control Policy Guide and Earned Value Management Policy
Guide define the policies and procedures for identifying and collecting
information to support its investment management process and, according
to Treasury officials, the Associate CIO for Capital Planning and
Information Management is assigned responsibility for ensuring that the
information collected meets the needs of the investment management
process. Also, the department has adequate resources for supporting the
process, including the Office of the CIO's CPIC team, which is
responsible for reviewing the information for accuracy and completeness
before it is presented to TIRB for review prior to making its
recommendations to the executive investment review board for final
decisions. It also maintains an automated portfolio management tool for
collecting and maintaining information on its IT investments. This tool
is used by department and bureau components for updating information on
their projects in response to data calls for the information required
by TIRB to conduct its quarterly reviews.
Table 8 shows the rating for each key practice required to implement
this Stage 2 critical process and summarizes the evidence that supports
these ratings.
Table 8: Capturing Investment Information:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for identifying and collecting information about IT projects
and systems to support the investment management process;
Rating: executed;
Summary of evidence: Treasury's CPIC guide and its Earned Value
Management Policy Guide have documented policies and procedures for
identifying and collecting information to support the investment
management process. This includes the use of a portfolio management
tool to collect and maintain information on IT investments.
Key practice: 2. An official is assigned responsibility for ensuring
that the information collected during project and systems
identification meets the needs of the investment management process;
Rating: executed;
Summary of evidence: According to Treasury officials, the Associate CIO
for Capital Planning and Information Management is the official
responsible for ensuring that the information collected meets the needs
of the investment management process.
Type of practice: Prerequisite;
Key practice: 1. Adequate resources, including people, funding, and
tools, are provided for identifying IT projects and systems and
collecting relevant investment information about them;
Rating: executed;
Summary of evidence: The department has adequate resources for meeting
this key practice, including Treasury's CPIC team, which assists the
bureaus in compiling the relevant information on IT investments. Each
bureau has a CPIC coordinator who serves as a point of quality control
before information is released to the department level. Treasury also
has an automated portfolio management tool to identify and collect
information on the department's and bureaus' IT investments.
Type of practice: Activities;
Key practice: 1. The organization's IT projects and systems are
identified, and specific information is collected to support decisions
about them;
Rating: executed;
Summary of evidence: Treasury uses a portfolio management tool for
maintaining information on its IT investments. Various forms within
this tool are used to collect information on Treasury's major and
nonmajor IT investments during the preselect, select, and control
phases of the department's CPIC process. Treasury's CPIC team is
responsible for reviewing the information for accuracy and
completeness. We verified that information on our four case study
projects was collected to support the IT investment management process.
Key practice: 2. The information that has been collected is easily
accessible and understandable to decision makers and others;
Rating: executed;
Summary of evidence: Treasury maintains information on its IT
investments in its portfolio management tool. For example, a summary of
each major investment is provided to TIRB as part of the quarterly
control review process.
Key practice: 3. The information repository is used by investment
decision makers and others to support investment management;
Rating: executed;
Summary of evidence: The portfolio management tool (the department's
information repository) is used by TIRB decision makers and others to
support investment management. For example, the bureaus use this tool
to update the information required for TIRB's quarterly control
reviews. The CPIC team is responsible for reviewing the information in
the tool for accuracy and completeness prior to consideration by TIRB.
Source: GAO.
[End of table]
Treasury Lacks Key Capabilities Needed to Manage IT Investments as a
Portfolio, and It Has Not Conducted Postimplementation Reviews:
Once an agency has attained Stage 2 maturity, it needs to implement
critical processes for managing its investments as a portfolio (Stage
3). An IT investment portfolio is an integrated, agencywide collection
of investments that are assessed and managed collectively based on
common criteria. Managing investments as a portfolio is a conscious,
continuous, and proactive approach to allocating limited resources
among an organization's competing initiatives in light of the relative
benefits expected from these investments. Taking an agencywide
perspective enables an organization to consider its investments
comprehensively, so that collectively the investments optimally address
the organization's mission, strategic goals, and objectives. Managing
IT investments as a portfolio also allows an organization to determine
its priorities and make decisions about which projects to fund and
continue to fund based on analyses of the relative organizational value
and risks of all projects, including projects that are proposed, under
development, and in operation. Although investments may initially be
organized into subordinate portfolios--based on, for example, business
lines or life cycle stages--and managed by subordinate investment
boards, they should ultimately be aggregated into this enterprise-level
portfolio.
According to the ITIM, Stage 3 maturity includes (1) defining the
portfolio criteria, (2) creating the portfolio, (3) evaluating the
portfolio, and (4) conducting postimplementation reviews. Table 9
summarizes the purpose of each critical process in Stage 3.
Table 9: Stage 3 Critical Processes--Developing a Complete Investment
Portfolio:
Critical process: Defining the portfolio criteria;
Purpose: To ensure that the organization develops and maintains IT
portfolio selection criteria that support its mission, organizational
strategies, and business priorities.
Critical process: Creating the portfolio;
Purpose: To ensure that IT investments are analyzed according to the
organization's portfolio selection criteria and to ensure that an
optimal IT investment portfolio with manageable risks and returns is
selected and funded.
Critical process: Evaluating the portfolio;
Purpose: To review the performance of the organization's investment
portfolios at agreed-upon intervals and to adjust the allocation of
resources among investments as necessary.
Critical process: Conducting postimplementation reviews;
Purpose: To compare the results of recently implemented investments
with the expectations that were set for them and to develop a set of
lessons learned from these reviews.
Source: GAO.
[End of table]
Treasury has executed 11 of the 27 key practices required by Stage 3.
For example, the department has a working group in place that is
responsible for managing the development and modification of the
department's IT portfolio selection criteria. In addition, it has
documented criteria to regularly assess its portfolio performance
expectations through its portfolio tool. However, many key practices
still need to be executed before Treasury can effectively manage its IT
investments from a portfolio perspective. For example, the department
has only addressed 3 of the 7 practices for evaluating the portfolio
and 2 of the 6 practices for conducting PIRs. Until Treasury fully
implements the critical processes associated with managing its
investments as a complete portfolio, it will not have the data it needs
to make informed decisions about competing investments.
Table 10 summarizes the status of Treasury's Stage 3 critical processes
and shows how many associated key practices the department has
executed.
Table 10: Summary of Results for Stage 3 Critical Processes and Key
Practices:
Critical process: Defining the portfolio criteria;
Key practices executed: 4;
Total required by critical process: 7;
Percentage of key practices executed: 57.
Critical process: Creating the portfolio;
Key practices executed: 2;
Total required by critical process: 7;
Percentage of key practices executed: 29.
Critical process: Evaluating the portfolio;
Key practices executed: 3;
Total required by critical process: 7;
Percentage of key practices executed: 43.
Critical process: Conducting postimplementation reviews;
Key practices executed: 2;
Total required by critical process: 6;
Percentage of key practices executed: 33.
Critical process: Total;
Key practices executed: 11;
Total required by critical process: 27;
Percentage of key practices executed: 41.
Source: GAO.
[End of table]
Treasury Has Portfolio Selection Criteria but Lacks Documented Policies
and Procedures for Modifying Them:
To manage IT investments effectively, an organization needs to
establish rules or portfolio selection criteria for determining how to
allocate scarce funding to existing and proposed investments. Thus,
developing an IT investment portfolio requires defining appropriate
cost, benefit, schedule, and risk criteria with which to evaluate
individual investments in the context of all other investments. To
ensure that the organization's strategic goals, objectives, and mission
will be satisfied by its investments, the criteria should have an
enterprisewide perspective. Further, if an organization's mission or
business needs and strategies change, criteria for selecting
investments should be re-examined and modified as appropriate.
Portfolio selection criteria should be disseminated throughout the
organization to ensure that decisions concerning investments are made
in a consistent manner and that this critical process is
institutionalized. To achieve this result, project management personnel
and others should be aware of the criteria and address the criteria in
funding submissions for projects. Resources required for this critical
process typically include the time and attention of executives involved
in the process, adequate funding, and supporting tools. (The complete
list of key practices is provided in table 11.)
Treasury has executed four of the seven key practices associated with
defining the portfolio selection criteria. For example, according to
Treasury officials, the department has adequate resources for portfolio
selection activities, including the Associate CIO for Capital Planning
and Information Management, the CPIC team, the CPIC subcouncil, which
is responsible for managing the development and modification of the IT
portfolio selection criteria, as well as a portfolio management tool.
In addition, project management personnel and other stakeholders are
made aware of the portfolio selection criteria through Treasury's CPIC
team, and the department's internal Web site.
Despite these important steps in defining portfolio selection criteria,
weaknesses remain. Specifically, the department has not developed
policies or procedures for modifying the portfolio selection criteria
to reflect changes to its strategic initiatives. In addition, because
Treasury does not have an executive investment review board, the
activities that call for this board to review and approve the portfolio
selection criteria are not being performed. Reviews of the portfolio
selection criteria are performed by the department's CPIC subcouncil,
which forwards its reviews to TIRB for approval of the criteria. Until
Treasury fully defines and implements the practices required for
defining the portfolio selection criteria, it will not have the tools
it needs to effectively select the mix of investments that best meet
the department's mission needs considering resource and funding
constraints.
Table 11 shows the rating for each key practice required to create a
portfolio and summarizes the evidence that supports these ratings.
Table 11: Defining the Portfolio Criteria:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for creating and modifying IT portfolio selection criteria;
Rating: not executed;
Summary of evidence: The department has documented policies and
procedures for creating the IT portfolio selection criteria. However,
the policies and procedures do not address how these criteria are to be
modified.
Key practice: 2. Responsibility is assigned to an individual or group
for managing the development and modification of the IT portfolio
selection criteria;
Rating: executed;
Summary of evidence: A Treasury CPIC subcouncil working group is
responsible for managing the development and modification of the IT
portfolio selection criteria.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, have been committed to portfolio selection criteria activities;
Rating: executed;
Summary of evidence: Adequate resources have been committed for
portfolio selection criteria activities, according to officials. The
resources include the Associate CIO for Capital Planning and
Information Management, the CPIC team, and the CPIC subcouncil.
Key practice: 2. A working group has been designated to be responsible
for developing and modifying the IT portfolio selection criteria;
Rating: executed;
Summary of evidence: Treasury has established a CPIC subcouncil working
group that is responsible for developing and modifying the portfolio
selection criteria.
Type of practice: Activities;
Key practice: 1. The enterprisewide investment board approves the core
IT portfolio selection criteria, including cost, benefit, schedule, and
risk criteria, based on the organization's mission, goals, strategies,
and priorities;
Rating: not executed;
Summary of evidence: According to officials, TIRB has been approving
the portfolio selection criteria. However, the CPIC guide states that
the executive investment review board is responsible for approving the
IT portfolio selection criteria, but Treasury does not have an
executive investment review board.
Key practice: 2. Project management personnel and other stakeholders
are aware of the portfolio selection criteria;
Rating: executed;
Summary of evidence: Project management personnel and other
stakeholders are made aware of the portfolio selection criteria through
Treasury's CPIC team, and the department's internal Web site.
Key practice: 3. The enterprisewide investment board regularly reviews
the IT portfolio selection criteria, using cumulative experience and
event-driven data, and modifies the criteria as appropriate;
Rating: not executed;
Summary of evidence: Treasury does not have an executive investment
review board to conduct portfolio selection criteria reviews. As a
result, the CPIC subcouncil reviews the portfolio selection criteria,
and TIRB approves them.
Source: GAO.
[End of table]
Treasury Lacks Documented Policies and Procedures for Analyzing and
Maintaining its Portfolio:
At Stage 3, organizations create a portfolio of IT investments to
ensure that IT investments are analyzed according to the organization's
portfolio selection criteria and to ensure that an optimal IT
investment portfolio with manageable risks and returns is selected and
funded. According to ITIM, creating the portfolio requires
organizations to, among other things, document policies and procedures
for analyzing, selecting, and maintaining the portfolio; provide
adequate resources, including people, funding, and tools for creating
the portfolio; and capture the information used to select, control, and
evaluate the portfolio and maintain it for future reference. In
creating the portfolio, the investment board must also (1) examine the
mix of new and ongoing investments and their respective data and
analyses and select investments for funding and (2) approve or modify
the performance expectations for the IT investments they have selected.
(The complete list of key practices is provided in table 12.)
Treasury has executed two of the seven key practices associated with
creating the portfolio. For example, the department has adequate
resources for creating its portfolio, including the CPIC subcouncil and
the use of the department's portfolio management tool. In addition,
information is captured and maintained for future reference in the
department's portfolio management tool. The information in the tool is
used to select, control, and evaluate all major IT portfolio
investments.
Nevertheless, Treasury has weaknesses in the way it creates a
portfolio. First, it does not have a complete set of policies and
procedures that address this critical process. Even though the
department has policies and procedures for selecting the IT portfolio
criteria, it lacks policies and procedures for using the criteria to
analyze and maintain the department's IT investment portfolio. Second,
since the department does not have an executive investment review
board, board members are not knowledgeable about creating a portfolio.
In addition, information comparing the performance of IT investments
against expectations is not currently being provided to the board
because Treasury does not have one. Even though TIRB board selects IT
investments based on data associated with the mix of new and ongoing
major investments, this activity is not done for nonmajors, and there
is not an executive investment review board to select the IT
investments. Moreover, the executive investment board does not approve
or modify the performance expectations of the selected IT investments.
Unless Treasury defines and implements the practices for creating a
comprehensive portfolio of IT, it will not be able to determine whether
it has selected the mix of investments that best meets its needs and
considers resource and funding constraints.
Table 12 shows the rating for each key practice required to create a
portfolio and summarizes the evidence that supports these ratings.
Table 12: Creating the Portfolio:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for analyzing, selecting, and maintaining the investment
portfolio;
Rating: not executed;
Summary of evidence: While Treasury's CPIC guide documents policies and
procedures for selecting the portfolio, the department does not have
documented policies and procedures for analyzing and maintaining the
investment portfolio.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, are provided for the process of creating the portfolio;
Rating: executed;
Summary of evidence: Adequate resources, including the CPIC subcouncil
and a portfolio management tool, are provided for creating the
portfolio.
Key practice: 2. Board members are knowledgeable about the process of
creating a portfolio;
Rating: not executed;
Summary of evidence: While TIRB members who are involved in creating
the department's portfolio are knowledgeable about this process,
Treasury does not have an executive investment review board.
Key practice: 3. The investment board is provided with information
comparing project and system performance with expectations;
Rating: not executed;
Summary of evidence: While TIRB is provided with information comparing
project performance with expectations for major investment during the
quarterly reviews, Treasury does not have an executive investment
review board.
Type of practice: Activities;
Key practice: 1. Each IT investment board examines the mix of new and
ongoing investments and their respective data and analyses and selects
investments for funding;
Rating: not executed;
Summary of evidence: While the CPIC policy guide calls for the
executive investment review board to examine the mix of new and ongoing
major investments and to select IT investments for funding, Treasury
does not have an executive investment review board. In addition, for
nonmajor investments, Treasury has delegated this oversight
responsibility to the bureaus but does not have a mechanism to ensure
that the bureaus are effectively performing this responsibility.
Key practice: 2. Each investment board approves or modifies the
performance expectations for its selected IT investments;
Rating: not executed;
Summary of evidence: TIRB approves and modifies the performance
expectations for selected IT investments. However, Treasury does not
have an executive investment review board that is responsible for this
activity.
Key practice: 3. Information used to select, control, and evaluate the
portfolio is captured and maintained for future reference;
Rating: executed;
Summary of evidence: Information from Treasury's portfolio management
tool is used to capture and maintain investment information for the
select, control, and evaluate process and for future reference.
Source: GAO.
[End of table]
Treasury Does Not Have Documented Policies for Evaluating Its
Portfolio:
This critical process builds on the Stage 2 critical process--Providing
Investment Oversight--by adding the elements of portfolio performance
to an organization's investment control capacity. Compared with less
mature organizations, Stage 3 organizations will have the foundation
they need to control the risks faced by each investment and to deliver
benefits that are linked to mission performance. In addition, a Stage 3
organization will have the benefit of performance data generated by
Stage 2 processes. Executive-level oversight of risk management
outcomes and incremental benefit accumulation provides the organization
with increased assurance that each IT investment will achieve the
desired results. (The complete list of key practices is provided in
table 13.)
Treasury is executing three of the seven key practices for this
critical process by providing adequate resources for reviewing the
portfolio, including the use of a portfolio tool that captures data on
cost, schedule, and risk and is used to produce scorecards on a
quarterly basis that summarizes portfolio data. The performance data
are consolidated in the portfolio tool and used by TIRB. The CPIC staff
is responsible for ensuring that the data are consistent with the
portfolio performance criteria and that it is modified as needed. For
example, based on OMB guidance, the department has added and modified
criteria related to the Exhibit 300s, EA, and earned value management
reporting requirements. In addition, Treasury uses the portfolio tool
to collect portfolio performance data in a consistent manner that
aligns with Treasury's portfolio performance criteria.
Despite these strengths, the department has yet to develop policies and
procedures that address the review, evaluation, and improvement of its
IT portfolio performance. In addition, TIRB members are not
consistently provided with oversight review information for nonmajor IT
investments by bureaus even though nonmajors make up about 70 percent
of the department's total number of projects. Also, while the
department has a process in place for ensuring that adjustments are
made to major investments in response to actual portfolio performance,
it does not have a process in place to ensure that the bureaus make the
necessary adjustments to their nonmajor investments on a consistent
basis. Until Treasury executes all the key practices associated with
this critical process, senior executives will not have the information
they need to determine whether the investments they have selected are
delivering mission value at the expected cost and risk.
Table 13 shows the rating for each key practice required to implement
the critical process for portfolio performance oversight at the Stage 3
level of maturity and summarizes the evidence that supports these
ratings.
Table 13: Evaluating the Portfolio:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for reviewing, evaluating, and improving the performance of
its portfolios;
Rating: not executed;
Summary of evidence: Treasury does not have documented policies and
procedures for reviewing, evaluating, and improving the performance of
its IT portfolio as a whole.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, have been provided for reviewing the investment portfolio and
its projects;
Rating: executed;
Summary of evidence: Treasury has adequate resources to review its
investment portfolio and projects. They include: project managers, the
CPIC team, and the portfolio management tool.
Key practice: 2. Board members are familiar with the process for
evaluating and improving the portfolio's performance;
Rating: not executed;
Summary of evidence: This key practice is not executed because Treasury
does not have an executive investment review board.
Key practice: 3. Results of relevant Providing Investment Oversight
reviews from Stage 2 are provided to the investment board;
Rating: not executed;
Summary of evidence: While Treasury's policy specifies that the
department's executive investment review board is to receive the
results of relevant oversight reviews from Stage 2, it does not have an
executive investment review board to perform this key practice.
Key practice: 4. Criteria for assessing portfolio performance are
developed, reviewed, and modified at regular intervals to reflect
current performance expectations;
Rating: executed;
Summary of evidence: Treasury has criteria to regularly assess
portfolio performance expectations. Portfolio performance criteria are
developed and modified using the department's portfolio management tool
that incorporates performance expectations such as cost and schedule.
Type of practice: Activities;
Key practice: 1. IT portfolio performance measurement data are defined
and collected consistent with portfolio performance criteria;
Rating: executed;
Summary of evidence: Treasury has a process for collecting portfolio
performance data that are defined and collected consistent with
Treasury's portfolio performance criteria.
Key practice: 2. Adjustments to the IT investment portfolio are
executed in response to actual portfolio performance;
Rating: not executed;
Summary of evidence: Treasury has a process to ensure that adjustments
are made to its major investment portfolio in response to actual
portfolio performance. For its nonmajor investments, however, Treasury
delegates this activity to the bureaus, but the department does not
have a mechanism to ensure that the bureaus are effectively carrying
out this activity.
Source: GAO.
[End of table]
Treasury Has Not Institutionalized a Postimplementation Review Process:
The purpose of a PIR is to evaluate an investment after it has been
completely developed (that is, after its transition from the
implementation phase to the operations and maintenance phase) in order
to validate actual investment results. This review is conducted to (1)
examine differences between estimated and actual investment costs and
benefits and possible ramifications for unplanned funding needs in the
future and (2) extract "lessons learned" about the investment selection
and control processes that can be used as the basis for management
improvements. Similarly, PIRs should be conducted for investment
projects that were terminated before completion, to readily identify
potential management and process improvements. (The complete list of
key practices is provided in table 14.)
Treasury has executed two of the six key practices for conducting PIRs.
According to officials, in fiscal year 2006, the department finished
revising its PIR policies and procedures as part of the last phase of
its CPIC process, the evaluate phase. The PIR guidance states that PIRs
are to be conducted 6 to 18 months after the investment has been
deployed (transitioned into the steady state life-cycle stage) or after
the investment has rolled out major functionality. In addition, the
department's portfolio tool (PIR form) requires that reviews measure
user satisfaction, achievement of strategic goals, and whether the
investment met cost, schedule, and performance goals. The CPIC guidance
also stipulates that project managers are responsible for conducting
the reviews and collecting the information needed to document lessons
learned, and who is responsible for approving the final PIR
recommendations.
Nevertheless, the department has not yet performed any PIRs since the
CPIC policy was issued and therefore has not performed any of the
activities associated with this critical process. Treasury officials
stated that, since the issuance of their PIR policy, they have not
conducted any PIRs because they have not had any investments
transitioning from the development phase into the steady state phase.
In 2005, the department conducted pilot PIRs on two major IT
investments. Of the two, one review met its goals and the other review
was recommended for a follow-up PIR because it was unable to provide
information on customer satisfaction, benefits analysis, and systems
performance due to schedule delays. Until PIRs are conducted on a
regular basis with senior executive management involvement, Treasury
will not be able to effectively evaluate the results of its IT
investments to determine whether continuation, modification, or
termination of an IT investment would be necessary in order to meet
stated Treasury mission objectives.
Table 14 shows the rating for each key practice required to conduct
PIRs and summarizes the evidence that supports these ratings.
Table 14: Conducting Postimplementation Reviews:
Type of practice: Organizational commitments;
Key practice: 1. The organization has documented policies and
procedures for conducting PIRs;
Rating: executed;
Summary of evidence: Treasury's CPIC guide documents policies and
procedures for conducting PIRs.
Type of practice: Prerequisites;
Key practice: 1. Adequate resources, including people, funding, and
tools, have been provided for conducting PIRs;
Rating: executed;
Summary of evidence: Treasury has adequate resources for conducting
PIRs, including the PIR form in its portfolio management tool, project
managers, and the CPIC team.
Key practice: 2. Individuals assigned to the investment board to
conduct PIRs should be familiar with both the policies and the
procedures for conducting such reviews;
Rating: not executed;
Summary of evidence: Individuals are not assigned to the executive
investment review board to conduct PIRs. Treasury's CPIC guidance
states that PIRs will be conducted by a project manager 6 to 18 months
after the investment transitions from the development life-cycle stage
to the operational stage.
Type of practice: Activities;
Key practice: 1. The investment board identifies which projects will
have a PIR conducted;
Rating: not executed;
Summary of evidence: According to the CPIC guide, all investments are
subject to PIRs 6 to 18 months after becoming operational. However,
Treasury has not conducted any PIRS because no investments have
transitioned from the developmental life-cycle stage to the operational
stage.
Key practice: 2. Quantitative and qualitative investment data are
collected, evaluated for reliability, and analyzed during the PIRs;
Rating: not executed;
Summary of evidence: Treasury has not conducted any PIRs since
documenting its PIR policies because no major investments have
transitioned from the developmental life-cycle stage to the operational
stage.
Key practice: 3. Lessons learned and recommendations for improving the
investment process are developed during the PIR, documented, and then
distributed to all stakeholders;
Rating: not executed;
Summary of evidence: Treasury has not conducted any PIRs since
documenting its PIR policies because no major investments have
transitioned from the developmental life-cycle stage to the operational
stage.
Source: GAO.
[End of table]
Treasury Does Not Have a Comprehensive Plan to Guide Its Improvement
Efforts:
We have previously reported that to effectively implement IT
investments management processes, organizations need to be guided by a
plan that (1) is based on an assessment of strengths and weaknesses;
(2) specifies measurable goals, objectives, and milestones; (3)
specifies needed resources; (4) assigns clear responsibility and
accountability for accomplishing tasks; and (5) is approved by senior-
level management. Such a plan is instrumental in helping agencies
coordinate and guide improvement efforts.
Treasury has initiated efforts to improve its investment management
process.
* Treasury has contracted for a review of the CPIC governance process
at each of its bureaus that entails performing portfolio investment
validation and evaluation on the bureaus' major investments. The
reviews involve visiting the respective bureaus to verify key CPIC
documentation, the health of their governance and investment processes,
and their compliance with the department's CPIC process. These reviews
are to provide the department with a better understanding of the
bureau's processes and help the department identify opportunities for
investment management improvements. The reviews also are to provide the
department with greater confidence in the investment information being
provided by the bureaus.
* In April 2007, Treasury issued an Internal Watch List that identifies
major investments at risk of not meeting established goals. Among the
criteria for placement on this list is cost or schedule variances
greater than plus or minus 10 percent for two consecutive quarters. The
department's Office of the CIO is responsible for overseeing the
Internal Watch List. Investments placed on this list are subject to
additional reporting requirements, including development of an action
plan to remediate the investment's noncompliant conditions. Bureaus are
to report on the status of their corrective actions to the CIO monthly.
Once the corrective actions have been implemented and the CIO concurs,
the investment may be removed from the list. According to officials, as
of May 2007, bureaus were beginning to submit their corrective action
plans to the CIO. The Internal Watch List process should improve
project oversight by providing greater assurance that actions are taken
to address deficiencies.
Although Treasury has initiated these efforts, the department has not
developed a comprehensive plan with the characteristics listed above
that would help guide improvements to its investment management
process. Treasury officials recognize the value of having a
comprehensive plan and told us they plan to develop one once their new
assistant secretary for management is confirmed; however, a time frame
for completing the plan has not been established. Until Treasury
develops this plan, the department risks not being able to put in place
an effective management process that will provide appropriate executive-
level oversight for minimizing risks and maximizing returns.
Treasury CIO's Role in Managing IT Investments Has Been Mixed:
The Clinger-Cohen Act, E-Government Act of 2002,[Footnote 23] and
implementing guidance from OMB provide a number of investment
management responsibilities to CIOs that generally entail working with
the agency head and senior managers to define and implement processes
for selecting, controlling, and evaluating investments. Our IT
investment management framework defines practices that are consistent
with these provisions. Because CIOs are to carry out their investment
management functions with the support of an enterprisewide investment
review board, many of the responsibilities we used to evaluate the
Treasury CIO's role relate to key practices discussed earlier in the
report as part of our evaluation of the department's investment
management capabilities.
The Treasury CIO's[Footnote 24] role in managing the department's IT
investments has been mixed, although it has gradually increased since
September 2005, when the department's CPIC policy was issued.
* Many responsibilities have been fully performed, including
responsibilities for establishing investment management policy, several
associated with selecting investments, and some associated with
controlling investments.
* Several responsibilities have been partially performed--including
some associated with selecting investments, and others associated with
controlling investments--either because the department has not extended
them to nonmajor investments or because some activities have not yet
been completed.
* A few responsibilities--most of them associated with controlling
investments--have not yet been performed, primarily because they are
just getting under way and have yet to produce results.
Table 15 shows the CIO's role in performing key investment management
responsibilities.
Table 15: CIO Involvement in Performing Investment Management
Responsibilities:
General.
Investment management responsibility: Implement investment governance
process as a member of executive investment review board;
Role in performing responsibility: While the CIO plays a significant
role in implementing Treasury's investment governance process, he is
not operating as a member of an executive investment review board. (As
noted in the report, this board currently does not exist.);
CIO involvement: Not performed.
Investment management responsibility: Provide oversight of development
and maintenance of documented investment process;
Role in performing responsibility: In the absence of an executive
investment review board, the CIO has been carrying out this
responsibility as head of TIRB. TIRB, for example, approved the CPIC
guidance first issued in September 2005;
CIO involvement: Fully performed.
Investment management responsibility: Develop comprehensive earned
value management policy;
Role in performing responsibility: The CIO issued an earned value
management policy to the department in December 2005;
CIO involvement: Fully performed.
Selecting investments.
Investment management responsibility: Approve selection criteria
(including portfolio selection criteria);
Role in performing responsibility: The selection criteria are first
defined in the CPIC policy, which the CIO issued in September 2005.
Changes to the selection criteria are approved by TIRB, which the CIO
chairs;
CIO involvement: Fully performed.
Investment management responsibility: Regularly review and modify
selection criteria (including portfolio selection criteria), as
appropriate;
Role in performing responsibility: Changes to the selection criteria
are approved by TIRB, which the CIO chairs;
CIO involvement: Fully performed.
Investment management responsibility: Use defined selection process to
select/reselect investments; Role in performing responsibility: TIRB,
which the CIO chairs, uses the defined selection process to select/
reselect major investments. The CIO is not involved in the selection/
reselection of nonmajor investments; CIO involvement: Partially
performed.
Investment management responsibility: Align funding decisions with
investment selection decisions;
Role in performing responsibility: The CIO works with other executives,
including the Assistant Secretary for Management/Chief Financial
Officer, to make funding decisions that are aligned with investment
selection decisions;
CIO involvement: Fully performed.
Investment management responsibility: Ensure qualified project managers
are assigned to all projects;
Role in performing responsibility: During the quarterly control
reviews, TIRB determines whether projects have qualified project
manager, in accordance with OMB guidance. The CIO issued a memo to
bureau CIOs in December 2005 requiring them to certify project manager
qualifications. In April 2007, the CIO issued a memo specifying
criteria for identifying major projects that will be subject to
additional CIO oversight and reporting requirements. These criteria
include lack of a validation of project managers' qualifications by the
bureau CIO. According to officials, as of May 2007, this process was
just getting under way;
CIO involvement: Partially performed.
Investment management responsibility: Leverage interagency and
governmentwide investments to support common missions;
Role in performing responsibility: The CIO oversees this activity (it
is carried out by EA staff);
CIO involvement: Fully performed.
Investment management responsibility: Use information repository to
support executive decision-making reselection;
Role in performing responsibility: TIRB uses information from the
department's repository to inform its selection decisions and
recommendations to executives;
CIO involvement: Fully performed.
Investment management responsibility: Ensure all investments have
acceptable business cases;
Role in performing responsibility: For the fiscal year 2008 budget
formulation process, the Office of the CIO instituted several policies
aimed at improving the quality of these business cases, including
requiring bureau project managers and CIOs to certify the accuracy of
the data in their business cases, and establishing an independent
validation program to examine both bureau CPIC processes and selected
Exhibit 300s. This program is currently under way;
CIO involvement: Partially performed.
Investment management responsibility: Evaluate the alignment of IT
projects/systems with strategic goals and objectives and provide
corrective actions if needed;
Role in performing responsibility: The TIRB, which the CIO chairs,
performs this activity for major projects as part of the select
process. The CIO does not carry out this activity for nonmajor
projects;
CIO involvement: Partially performed.
Controlling investments.
Investment management responsibility: Approve/modify the performance
expectations of selected investments;
Role in performing responsibility: For major investments, the CIO
carries out this responsibility by approving the business cases and
other documents that specify performance expectations and approving
baseline change requests. The CIO does not carry out this
responsibility for nonmajor investments;
CIO involvement: Partially performed.
Investment management responsibility: Conduct integrated baseline
reviews on contracts with an earned value management system (EVMS)
requirement;
Role in performing responsibility: According to the Associate CIO for
Capital Planning and Information Management and the Director for
Capital Planning and Investment Control, this responsibility has been
delegated to the bureaus. Because this responsibility involves working
with contract officer technical representatives, the office of the CIO
has engaged the Office of the Chief Procurement Officer. The two
offices are currently working to develop guidance;
CIO involvement: Partially performed.
Investment management responsibility: Receive data on actual cost and
schedule performance;
Role in performing responsibility: The CIO--as head of TIRB--receives
data on actual cost and schedule performance of major investments on a
quarterly basis. The CIO does not carry out this responsibility for
nonmajor investments;
CIO involvement: Partially performed.
Investment management responsibility: Review, on a regular basis, the
performance of IT projects against expectations using verified data;
Role in performing responsibility: TIRB reviews the performance of
major IT projects against expectations, using verified data as part of
its quarterly reviews. The CIO does not carry out this responsibility
for nonmajor investments;
CIO involvement: Partially performed.
Investment management responsibility: Manage and measure projects to a
10 percent variance of baseline using EVMS;
Role in performing responsibility: The TIRB quarterly reviews of
performance data include a measure of 10 percent variance of baseline
using EVMS. The CIO, however, issued a memo in April 2007 requiring
projects experiencing cost or schedule variances greater than plus or
minus 10 percent for two consecutive quarters to develop an action plan
to remediate the condition and report to the CIO on the status of
actions taken on a monthly basis. According to Treasury officials, as
of May 2007, this process was just getting under way;
CIO involvement: Not performed.
Investment management responsibility: Take corrective actions for
underperforming IT projects;
Role in performing responsibility: In April 2007, the CIO issued a memo
regarding the identification of major projects to be placed on an
Internal Watch List based on not meeting certain criteria for two
consecutive quarters. These projects are to develop corrective actions
and report to the CIO on the status of these actions on a monthly
basis. According to officials, as of May 2007, this process was just
getting under way;
CIO involvement: Not performed.
Investment management responsibility: Track implementation of
corrective actions on projects;
Role in performing responsibility: In April 2007, the CIO issued a memo
regarding the identification of major projects to be placed on an
Internal Watch List based on not meeting certain criteria for two
consecutive quarters. These projects are to develop corrective actions
and report to the CIO on the status of these actions on a monthly
basis. According to officials, as of May 2007, this process was just
getting under way;
CIO involvement: Not performed.
Investment management responsibility: Use information repository to
support control decisions;
Role in performing responsibility: TIRB uses information from the
department's repository to make control decisions and investment
recommendations to executives;
CIO involvement: Fully performed.
Investment management responsibility: Coordinate "high risk" project
identification with OMB;
Role in performing responsibility: The CIO worked with OMB to identify
its initial list of high-risk projects and continues to provide updates
of this list on a quarterly basis;
CIO involvement: Fully performed.
Investment management responsibility: Assess, confirm, and document the
performance of high-risk projects;
Role in performing responsibility: Every quarter, the CIO submits to
OMB a report that assesses, confirms, and documents the performance of
the department's high-risk projects;
CIO involvement: Fully performed.
Evaluating investments.
Investment management responsibility: Identify IT projects for
postimplementation reviews;
Role in performing responsibility: According to Treasury's CPIC
policies, postimplementation reviews are required for all projects 6 to
18 months after they become operational. According to officials, Office
of the CIO staff keep track of when projects reach that phase. These
officials also note, however, that no project has become eligible for
PIRs since the CPIC policy was issued;
CIO involvement: Not performed.
Source: GAO.
[End of table]
The CIO's involvement in managing the department's investments has
strengthened the investment management process. For example, by
regularly reviewing and modifying investment selection criteria, as
appropriate, to reflect organizational objectives, the CIO, as Chair of
the TIRB, has helped ensure investments supporting organizational goals
are selected.
However, several responsibilities have not been fully performed. For
example, several responsibilities for selecting and controlling
investments have not been performed for nonmajor investments. As
discussed earlier in the report, Treasury officials stated they have
not made the nonmajor investments a priority because they have instead
chosen to devote their resources to the major investments, which
represent about 80 percent of the department's IT expenditures. As
noted earlier, while it is reasonable to focus on the major
investments, the nonmajor investments also require a certain level of
oversight, given the significant amount of funding (about $480 million)
and number of investments (160) involved. Because several
responsibilities have not been fully performed, there is increased risk
that investments will not be effectively managed.
Conclusions:
Given the importance of IT to Treasury's mission, it is vital that the
department manage its investments effectively. To its credit, because
of the attention that has recently been given to investment management,
Treasury has established many of the practices needed to build the
investment foundation and manage its projects as a portfolio and, as
such, has made progress since we examined the department's process as
part of our governmentwide review 3 years ago. However, the absence of
an executive investment review board actively engaged in the investment
management process and the department's limited involvement in the
management of nonmajor investments are significant weaknesses that
hamper the department's ability to effectively manage its investments.
As a result, the department cannot ensure that it is managing the mix
of investments that will maximize returns to the organization, taking
into account the appropriate level of risk.
Critical to Treasury's success going forward will be the development
and implementation of a plan that (1) is based on the assessment of
strengths and weaknesses identified in this report; (2) specifies
measurable goals, objectives, and milestones; (3) specifies needed
resources; (4) assigns clear responsibility and accountability for
accomplishing tasks; and (5) is approved by senior management. Without
such a plan and procedures for implementing it, it will be difficult
for the department to maintain steady progress in improving its
investment management process. As a result, Treasury will continue to
be challenged in its ability to make informed and prudent investment
decisions in managing its annual multibillion-dollar IT budget.
By fully performing selected investment management responsibilities,
the CIO has taken positive steps toward strengthening the department's
process for selecting, controlling, and evaluating investments.
However, the department's investments will continue to be at risk as
long as there are responsibilities that are partially performed or not
performed.
Recommendations for Executive Action:
To strengthen Treasury's investment management capability, we recommend
that the Secretary of the Department of the Treasury direct the
Assistant Secretary for Management, in collaboration with the CIO, to
develop and implement a plan to address the following two actions:
(1) Establish an executive investment review board, composed of
executives representing IT and business units, that would be actively
engaged in the investment management process.
(2) Develop and implement policies and procedures to manage nonmajor
investments.
We also recommend that the plan include actions to address the
weaknesses in eight critical processes identified in this report,
beginning with those identified in our Stage 2 analysis and continuing
with those identified in our Stage 3 analysis. The plan should, at a
minimum, provide for fully implementing the following:
In Stage 2:
* instituting the investment board,
* meeting business needs,
* selecting an investment, and:
* providing investment oversight.
In Stage 3:
* defining the portfolio criteria,
* creating the portfolio,
* evaluating the portfolio, and:
* conducting postimplementation reviews.
In developing the plan, the Secretary of the Department of the Treasury
should direct the Chief Information Officer to ensure that the plan
draws together ongoing and additional efforts needed to address the
weaknesses identified in this report, including those relating to the
CIO's role in performing investment management responsibilities. The
plan should also (1) specify measurable goals, objectives, and
milestones; (2) specify needed resources; (3) assign clear
responsibility and accountability for accomplishing tasks; and (4) be
approved by senior management. In implementing the plan, the Chief
Information Officer should ensure that the resources are available to
carry out the plan and that progress is measured and reported
periodically to the Secretary of the Department of the Treasury.
Agency Comments and Our Evaluation:
In e-mail comments on a draft of this report, the Acting CIO stated
that the report reflects both Treasury's shortcomings as well as
progress to date and recognized the need to take proactive steps to
strengthen its investment board operations and oversight of information
technology resources and programs. Treasury also commented on the need
for an executive review board, nonmajor investments, and the
department's authority to redirect funding from one Treasury bureau to
another.
Regarding the need for an executive investment review board, Treasury
noted that, in addition to the Technical Investment Review Board
chaired by the CIO, an E-Board consisting of Treasury executives
previously existed. We acknowledge the establishment of these boards in
our report but emphasize that there currently is no executive
investment review board composed of executives from IT and business
units that is actively engaged in the investment management process.
The department recognizes this in its comments, stating that it agrees
it needs to reconstitute its executive board such that it is actively
engaged in the investment management process.
Regarding nonmajor investments, Treasury commented that nonmajor
investments have not been a priority because the major investments the
department has chosen to devote its resources to represent the more
significant portion of the portfolio in terms of dollar value,
visibility to OMB and Congress, and importance to Treasury's mission.
We recognize the importance of the major investments in our report and
acknowledge that it is reasonable to focus attention on these
investments. Nevertheless, we maintain that nonmajor investments should
require a certain level of oversight given the amount of funding
involved (about $480 million in estimated expenditures for fiscal year
2007) and the fact that they represent the bulk of most bureaus
investment portfolio. Treasury also stated that its CPIC guide contains
guidance on managing nonmajor IT investments and that the department
conducts quarterly control reviews of all IT investments, both major
and nonmajor. While the guide requires all IT investments to comply
with its provisions, it clearly states that the select phase described
applies to major investments and that bureaus are responsible for
conducting their own select process for nonmajor investments. In
addition, while, as we note in the report, Treasury requires bureaus to
report on the cost, schedule, and performance of its nonmajor
investments on a quarterly basis, this information is not provided to
TIRB for review. Treasury noted that it is currently developing
guidance and reporting requirements for nonmajors that integrates
enterprise architecture and capital planning.
In its comments, Treasury also noted that the department's ability to
exercise effective management of its IT portfolio requires that the CIO
(as chairman of the Technical Investment Review Board) be empowered to
make recommendations to the executive board concerning IT budgetary
requests across the department. Additionally, the executive board needs
to be empowered to make decisions across organizational lines on behalf
of the department. Treasury added that, currently, neither the Treasury
Department, including the Acting CIO, nor the executive board has the
prerogative (authority) to redirect IT funding from one Treasury bureau
to another. While this particular authority was not the subject of our
review, we agree that not having it could present a challenge to
effectively managing the IT portfolio. Nevertheless, effective
portfolio management requires the collective decisionmaking of
executives from both IT and business units, which highlights the
importance of having an executive investment review board that is
actively engaged in the investment management process.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of other Senate and House committees that have
authorization and oversight responsibilities for Treasury and other
interested congressional committees; the Director of the Office of
Management and Budget; the Secretary of the Treasury; the Assistant
Secretary for Management and Chief Financial Officer; and the Chief
Information Officer. We also will make copies available to others upon
request. In addition, the report will be available at no charge on the
GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please
contact me at (202) 512-9286 or pownerd@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report are
listed in appendix II.
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
The objectives of our review were to (1) assess the Department of the
Treasury's capabilities for managing its IT investments, (2) determine
any plans Treasury might have for improving those capabilities, and (3)
evaluate the CIO's role in managing the department's IT investments.
To address our first objective, we reviewed the results of the
department's self-assessment of Stages 2 and 3 practices using our IT
investment management framework and validated and updated the results
of the self-assessment through document reviews and interviews with
officials. We reviewed written policies, procedures, and guidance and
other documentation providing evidence of executed practices, including
Treasury's Capital Planning and Investment Control Policy Guide, Earned
Value Management Policy Guide, Exhibit 300 Scoring Guide, Alternative
Analysis Policy Guide, FY06 IT Portfolio Alignment Summary, IT
Modernization Blueprint Volume 2: IT Strategic Plan, portfolio
management tool guidance, and various memorandums. We also reviewed
TIRB and CIO Council meeting materials. In addition, we conducted
interviews with officials from the Office of the CIO, whose main
responsibility is to oversee and ensure that Treasury's IT investment
management process is implemented and followed.
We compared the evidence collected from our document reviews and
interviews to the key practices in ITIM. We rated the key practices as
"executed" on the basis of whether the agency demonstrated (by
providing evidence of performance) that it had met the criteria of the
key practice. A key practice was rated as "not executed" when we found
insufficient evidence of a practice during the review or when we
determined that there were significant weaknesses in Treasury's
execution of the key practice. In addition, Treasury was provided with
the opportunity to produce evidence for key practices rated as "not
executed." We did not assess progress in establishing the capabilities
found in Stages 4 and 5 because the department acknowledged it had not
executed the key practices in these higher maturity stages.
To determine the level of guidance the department is providing to its
bureaus, we interviewed officials and obtained written responses from
the Bureau of the Public Debt, Financial Management Service, and the
Internal Revenue Service (IRS) to determine the level of investment
management guidance and oversight that is provided by the department.
As part of our analysis, we selected one enterprisewide and three
bureau-level IT projects as case studies to verify that the critical
processes and key practices were being applied. The projects selected
(1) are in different life-cycle phases, (2) represent a mix of
headquarters and component bureau investments, (3) support different
functional areas, and (4) required different levels of funding. The
four projects are described as follows:
1. Customer Account Data Engine (CADE). The database initiative is the
foundation for managing taxpayer accounts in IRS's Business Systems
Modernization[Footnote 25] effort. CADE is being incrementally
designed, developed, and implemented to form the data foundation for a
modernized IRS by replacing the Individual Master File[Footnote 26] and
its related applications with new technology, new applications, and new
databases. The system's purpose is to enable IRS tax specialists to
post transactions and update taxpayer account and return data using an
online interface tool. Updates are to be available daily to authorized
personnel who have access to this data, which provide a complete,
timely, and accurate account of the individual taxpayer's information.
The project is a major investment and has an estimated life-cycle cost
of over $1.3 billion.
2. Savings Bond Replacement System (SaBRe). SaBRe supports two of the
President's Management Agenda initiatives: financial performance and
expanded e-government. It processes cash and security transactions that
result when accrued savings bonds are sold or redeemed by Federal
Reserve Bank processing sites or by financial institutions and
corporate entities designated as fiscal agents. Federal Reserve Bank
processing sites consolidate and report to SaBRe daily issue and
retirement transactions generated by processing cash and security
transactions. SaBRe processes the transactions, updates electronic
records that are used for customer service, and reports daily financial
transactions for inclusion in the Daily Treasury Statement. The project
is a major investment and has an estimated life-cycle cost of over $57
million.
3. Treasury Receivable, Accounting, and Collection System (TRACS).
TRACS is to provide Treasury's Financial Management Service with a tool
for supporting its Payment Business Line for the accounting, debt
billing, collection, and reporting requirements associated with
Treasury's check claims business process. It is to aid in the
processing of check claims accounting, authorization of payments,
issuing of bills, debt collection, and funds transfers from and to
federal program agencies. Currently all funding for TRACS will be used
to maintain and enhance the system. The project is a nonmajor
investment and has an estimated life-cycle cost of over $11 million
through fiscal year 2012.
4. Treasury Foreign Intelligence Network (TFIN). TFIN exists to assist
Treasury analysts in their ongoing efforts to provide meaningful
intelligence to senior Treasury management as well as to other agencies
within the intelligence community. It was originally built as a
customized in-house network over 10 years ago. In early fiscal year
2005, Treasury identified a need to modernize TFIN due to the age of
the system, outdated components, and performance issues, and to address
Treasury's expanding mission in the fight against terrorism. The system
is currently listed as a major department-level development,
modernization, and enhancement effort, with total estimated life-cycle
costs of $43 million.
For these projects, we reviewed project management documentation, such
as project plans, and status reports. We also obtained investment
information from the bureau officials responsible for managing the
projects.
To address our second objective, we obtained and evaluated documents
showing what management actions had been taken and what initiatives had
been planned by the agency. This documentation included the IT
Modernization Blueprint Volume 2, IT Strategic Plan, The Department of
the Treasury's Strategic Plan, and a contractor work request for an
independent validation and verification of Treasury's capital planning
program support process. We also interviewed officials from the Office
of the CIO to determine efforts undertaken to improve IT investment
management processes.
To address our third objective, we reviewed legislation, including the
Clinger-Cohen Act of 1996 and the E-Government Act of 2002, and OMB
guidance to determine the roles and responsibilities of CIOs regarding
investment management. We also reviewed the practices laid out in GAO's
IT investment management framework. We reviewed documentation and
conducted interviews with Treasury officials, including the Associate
CIO for Capital Planning and Information Management, to determine the
extent of the CIO's involvement in selecting, controlling, and
evaluating the department's IT investments. We conducted our work at
Treasury headquarters in Washington, D.C., from August 2006 through
July 2007 in accordance with generally accepted government auditing
standards.
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner, (202) 512-9286 or pownerd@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Sabine Paul, Assistant
Director; William Barrick; Camille Chaires; Neil Doherty; Nancy Glover;
and Tomas Ramirez; made key contributions to this report.
FOOTNOTES
[1] Office of Management and Budget, Report on Information Technology
(IT) Spending for the Federal Government for Fiscal Years 2006, 2007,
2008 (Washington, D.C., May 2007).
[2] 40 U.S.C. §§ 11312-11313.
[3] See, for example, GAO, Business Systems Modernization: Internal
Revenue Service's Fiscal Year 2007 Expenditure Plan, GAO-07-247
(Washington, D.C.: Feb.15, 2007).
[4] GAO, Information Technology Management: Governmentwide Strategic
Planning,Performance, Measurement, and Investment Management Can Be
Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004).
[5] GAO, Information Technology Management: Observations on the
Financial Crimes Enforcement Network's (FinCEN's) BSA Direct Retrieval
and Sharing (BSA Direct R&S) Project, GAO-06-947R (Washington, D.C.:
July 14, 2006).
[6] According to officials, this investment was classified as nonmajor
until August 2006.
[7] GAO, High-Risk Series: An Update, GAO-07-310 (Washington, D.C.:
January 2007).
[8] OMB determines projects to be included on its Management Watch List
based on an evaluation of Exhibit 300 business cases that agencies
submit for major projects as part of the budget development process.
The high-risk list consists of projects identified by the agencies with
the assistance of OMB, using specific criteria established by OMB, and
that are reported quarterly by the agencies to OMB.
[9] 40 U.S.C. §§ 11312, 11313, 11315.
[10] The first five criteria are OMB criteria outlined in OMB Circular
A-11 for determining major investments. The remaining three criteria
are Treasury-specific criteria.
[11] The policy document has been updated a few times since it was
issued. The most recent update was issued in October 2006.
[12] The President's e-government initiatives are intended to improve
services to citizens, to increase the efficiency and effectiveness of
the government, and to provide savings to the taxpayer.
[13] The President's Management Agenda, announced in 2001, is a
strategy for improving the management of the federal government,
focusing on five areas of management weaknesses across the government.
One of these areas involves expanded use of electronic government for
better serving the public.
[14] In August 2005, OMB initiated an effort for agencies to improve IT
project planning and execution. Through this effort, agencies are to
identify "high risk projects" using specific criteria established by
OMB and report quarterly to OMB on each project's performance noted
shortfalls and planned corrective actions to address the shortfalls.
The criteria Treasury used to establish its internal watch list mirrors
the list of shortfalls OMB requires agencies to report on.
[15] Certification is the comprehensive evaluation of the management,
operational, and technical security controls in an information system
to determine the effectiveness of these controls and identify existing
vulnerabilities. Accreditation is the official management decision to
authorize operation of an information system. This authorization
explicitly accepts the risk remaining after the implementation of an
agreed-upon set of security controls.
[16] Similarly to the e-government initiatives, the line of business
initiatives are intended to improve services to citizens, to increase
the efficiency and effectiveness of the government, and to provide
savings to the taxpayer.
[17] GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington,
D.C.: March 2004).
[18] GAO, Information Technology: DLA Needs to Strengthen Its
Investment Management Capability, GAO-02-314 (Washington, D.C.: Mar.
15, 2002); United States Postal Service: Opportunities to Strengthen IT
Investment Management Capabilities, GAO-03-3 (Washington, D.C.: Oct.
15, 2002); Information Technology: Departmental Leadership Crucial to
Success of Investment Reforms at Interior, GAO-03-1028 (Washington,
D.C.: Sept. 12, 2003); Bureau of Land Management: Plan Needed to
Sustain Progress in Establishing IT Investment Management Capabilities,
GAO-03-1025 (Washington, D.C.: Sept. 12, 2003); Information Technology:
FAA Has Many Investment Management Capabilities in Place, but More
Oversight of Operational Systems Is Needed, GAO-04-822 (Washington,
D.C.: Aug. 20, 2004); Information Technology: HHS Has Several
Investment Management Capabilities in Place, but Needs to Address Key
Weaknesses, GAO-06-11 (Washington, D.C.: Oct. 28, 2005); Information
Technology: Centers for Medicare & Medicaid Services Needs to Establish
Critical Investment Management Capabilities, GAO-06-12 (Washington,
D.C.: Oct. 28, 2005); Information Technology: DHS Needs to Fully Define
and Implement Policies and Procedures for Effectively Managing
Investments, GAO-07-424 (Washington, D.C.: Apr. 27, 2007).
[19] Stage 1 is typified by the absence of an organized, executable,
and consistently applied IT investment management process.
[20] An IT investment board is a decision-making body, made up of
senior program, financial, and information officials, that is
responsible for making decisions about IT projects and systems on the
basis of comparisons and trade-offs among competing projects and has an
emphasis on meeting mission goals.
[21] The Clinger-Cohen Act of 1996, 40 U.S.C. §§ 11312.
[22] According to ITIM, new proposals include both (1) previously
submitted IT proposals that were not originally selected for funding
and (2) IT proposals that have never been submitted.
[23] Pub. L. No. 107-347 (Dec. 17, 2002)
[24] We are referring to both the current CIO who has been acting since
January 2007 and the former CIO.
[25] The Business Systems Modernization is a highly complex,
multibillion-dollar effort to modernize IRS's technology and related
business processes.
[26] The Individual Master File is IRS's database that stores various
types of taxpayer account information. This database includes
individual, business, employee plans, and exempt organizations data.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400:
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800:
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
