Information Technology
OMB Can More Effectively Use Its Investment Reviews Gao ID: GAO-05-571T April 21, 2005Federal spending on information technology (IT) is over $60 billion this year and is expected to continue to rise. Accordingly, it is essential that federal IT investments are managed efficiently. Of the 1,200 major IT projects in the President's Budget for Fiscal Year 2005, OMB stated that it had placed about half--621 projects, representing about $22 billion--on a Management Watch List to focus attention on mission-critical IT investments that need management improvements. GAO was asked to testify on the findings and recommendations made in a report that it recently completed (GAO-05-276), which describes and assesses OMB's processes for (1) placing projects on its Management Watch List and (2) following up on corrective actions established for projects on the list.
For the fiscal year 2005 budget, OMB developed processes and criteria for including investments on its Management Watch List. In doing so, it identified opportunities to strengthen investments and promote improvements in IT management. However, it did not develop a single, aggregate list identifying the projects and their weaknesses. Instead, OMB officials told GAO that to identify projects with weaknesses, individual analysts used scoring criteria that the office established for evaluating the justifications for funding that federal agencies submit for major projects. These analysts, each of whom is typically responsible for several federal agencies, were then responsible for maintaining information on these projects. To derive the total number of projects on the list for fiscal year 2005, the office polled its individual analysts and compiled the result. However, OMB officials told GAO that because they did not see such an activity as necessary, they did not compile a single list. Accordingly, OMB has not fully exploited the opportunity to use its watch list as a tool for analyzing IT investments on a governmentwide basis. OMB asked agencies to take corrective actions to address weaknesses associated with projects on the Management Watch List, but it did not develop a structured, consistent process for deciding how to monitor agency corrective actions. According to OMB officials, decisions on monitoring of progress were typically made by the staff with responsibility for reviewing individual agency budget submissions, depending on the staff's insights into agency operations and objectives. Because it did not consistently require or monitor agency follow-up activities, OMB did not know whether the project risks that it identified through its Management Watch List were being managed effectively, potentially leaving resources at risk of being committed to poorly planned and managed projects. In addition, because it did not consistently monitor the follow-up performed on projects on the Management Watch List, OMB could not readily tell GAO which of the 621 projects received follow-up attention. To help enable OMB to take advantage of the potential benefits of using the Management Watch List as a tool for analyzing and following up on investments, GAO's report included recommendations that OMB develop a single, aggregate Management Watch List and that it develop and use criteria for prioritizing and monitoring the projects on the list. GAO also recommended that the office use the prioritized list for reporting to the Congress as part of its statutory reporting responsibilities. In commenting on a draft of this report, OMB did not agree that the aggregated governmentwide list recommended by GAO is necessary for adequate oversight and management. However, GAO continues to believe that an aggregated Management Watch List would contribute to OMB's ability to analyze IT investments governmentwide and track progress in addressing deficiencies.
GAO-05-571T, Information Technology: OMB Can More Effectively Use Its Investment Reviews
This is the accessible text file for GAO report number GAO-05-571T
entitled 'Information Technology: OMB Can More Effectively Use Its
Investment Reviews' which was released on April 21, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
For Release on Delivery:
Expected at:
10 a.m. EDT on Thursday,
April 21, 2005:
GAO:
Testimony:
Before the Committee on Government Reform,
House of Representatives:
GAO-05-571T:
Information Technology:
OMB Can More Effectively Use Its Investment Reviews:
Statement of David A. Powner,
Director, Information Technology:
Management Issues:
GAO Highlights:
Highlights of GAO-05-571T, a testimony before the Committee on
Government Reform, House of Representatives.
Why GAO Did This Study:
Federal spending on information technology (IT) is over $60 billion
this year and is expected to continue to rise. Accordingly, it is
essential that federal IT investments are managed efficiently. Of the
1,200 major IT projects in the President‘s Budget for Fiscal Year 2005,
OMB stated that it had placed about half”621 projects, representing
about $22 billion”on a Management Watch List to focus attention on
mission-critical IT investments that need management improvements.
GAO was asked to testify on the findings and recommendations made in a
report that it recently completed (GAO-05-276), which describes and
assesses OMB‘s processes for (1) placing projects on its Management
Watch List and (2) following up on corrective actions established for
projects on the list.
What GAO Found:
For the fiscal year 2005 budget, OMB developed processes and criteria
for including investments on its Management Watch List. In doing so, it
identified opportunities to strengthen investments and promote
improvements in IT management. However, it did not develop a single,
aggregate list identifying the projects and their weaknesses. Instead,
OMB officials told GAO that to identify projects with weaknesses,
individual analysts used scoring criteria that the office established
for evaluating the justifications for funding that federal agencies
submit for major projects. These analysts, each of whom is typically
responsible for several federal agencies, were then responsible for
maintaining information on these projects. To derive the total number
of projects on the list for fiscal year 2005, the office polled its
individual analysts and compiled the result. However, OMB officials
told GAO that because they did not see such an activity as necessary,
they did not compile a single list. Accordingly, OMB has not fully
exploited the opportunity to use its watch list as a tool for analyzing
IT investments on a governmentwide basis.
OMB asked agencies to take corrective actions to address weaknesses
associated with projects on the Management Watch List, but it did not
develop a structured, consistent process for deciding how to monitor
agency corrective actions. According to OMB officials, decisions on
monitoring of progress were typically made by the staff with
responsibility for reviewing individual agency budget submissions,
depending on the staff‘s insights into agency operations and
objectives. Because it did not consistently require or monitor agency
follow-up activities, OMB did not know whether the project risks that
it identified through its Management Watch List were being managed
effectively, potentially leaving resources at risk of being committed
to poorly planned and managed projects. In addition, because it did not
consistently monitor the follow-up performed on projects on the
Management Watch List, OMB could not readily tell GAO which of the 621
projects received follow-up attention.
To help enable OMB to take advantage of the potential benefits of using
the Management Watch List as a tool for analyzing and following up on
investments, GAO‘s report included recommendations that OMB develop a
single, aggregate Management Watch List and that it develop and use
criteria for prioritizing and monitoring the projects on the list. GAO
also recommended that the office use the prioritized list for reporting
to the Congress as part of its statutory reporting responsibilities. In
commenting on a draft of this report, OMB did not agree that the
aggregated governmentwide list recommended by GAO is necessary for
adequate oversight and management. However, GAO continues to believe
that an aggregated Management Watch List would contribute to OMB‘s
ability to analyze IT investments governmentwide and track progress in
addressing deficiencies.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-571T].
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact David Powner at (202) 512-9286 or
pownerd@gao.gov.
[End of Section]
Mr. Chairman and Members of the Committee:
Thank you for the opportunity to participate in the Committee's hearing
on processes that the Office of Management and Budget (OMB) has
developed as part of its efforts to identify and follow up on
information technology (IT) projects that need management improvements.
As you know, the President's Budget for Fiscal Year 2005 requested over
$60 billion to fund IT, and that figure is expected to rise throughout
the rest of the decade. OMB stated that of the nearly 1,200 major IT
projects in the fiscal year 2005 budget, it had placed about half--621
projects, representing about $22 billion--on its Management Watch List.
For fiscal year 2006, 342 of 1,087 IT projects (representing about $15
billion) were placed on the watch list.
At your request, we performed a review of OMB's processes for (1)
placing projects on its Management Watch List and (2) following up on
corrective actions established for projects on the list. Today I am
summarizing the findings and recommendations of that report, which is
being released today.[Footnote 1]
Results in Brief:
For the fiscal year 2005 budget, OMB developed processes and criteria
for including IT projects (investments) on its Management Watch List.
In doing so, it identified opportunities to strengthen investments and
promote improvements in IT management. However, OMB did not develop a
single, aggregate list identifying the projects and their weaknesses.
Instead, according to OMB officials, individual OMB analysts assigned
scores to the justifications for funding (known as exhibit 300s) that
are submitted by federal agencies. (These scores were based on criteria
established in the office's Circular A-11.) OMB delegated individual
analysts on its staff, each of whom is typically assigned
responsibility for several federal agencies, with maintaining, for
their respective agencies, information for the IT projects included on
the list. To derive the total number of projects on the list that OMB
reported (621 for fiscal year 2005), OMB polled its individual analysts
and compiled the numbers. According to OMB officials, they did not
construct a single list of projects meeting their watch list criteria
because they did not see such an activity as necessary for performing
OMB's predominant mission: to assist in overseeing the preparation of
the federal budget and to supervise agency budget administration. Thus,
OMB did not exploit the opportunity to use the list as a tool for
analyzing IT investments on a governmentwide basis, limiting its
ability to identify and report on the full set of IT investments
requiring corrective actions.
OMB asked agencies to take corrective actions to address weaknesses
associated with projects on the Management Watch List, but it did not
develop a structured, consistent process for deciding how to follow up
on these actions. According to OMB officials, decisions on follow-up
and monitoring of specific projects were typically made by the OMB
staff with responsibility for reviewing individual agency budget
submissions, depending on the staff's insights into agency operations
and objectives. Because it did not consistently monitor the follow-up
performed, OMB could not tell us which of the 621 projects identified
on the fiscal year 2005 list received follow-up attention, and it did
not know whether the specific project risks that it identified through
its Management Watch List were being managed effectively. This approach
could leave resources at risk of being committed to poorly planned and
managed projects. Thus, OMB was not using its Management Watch List as
a tool for improving IT investments on a governmentwide basis and
focusing attention where it was most needed.
To enable OMB to take advantage of the potential benefits of using the
Management Watch List as a tool for analyzing and following up on IT
investments, we recommended in our report that OMB develop a single,
aggregate Management Watch List, and that it develop and use criteria
for prioritizing and monitoring the projects on the list. We also
recommended that the office use the prioritized list for reporting to
the Congress as part of its statutory reporting responsibilities. In
commenting on a draft of this report, OMB did not agree that the
aggregated governmentwide list recommended by GAO is necessary for
adequate oversight and management. However, GAO continues to believe
that an aggregated Management Watch List would contribute to OMB's
ability to analyze IT investments governmentwide and track progress in
addressing deficiencies.
Background:
The President's Budget for Fiscal Year 2005 identified approximately
$60 billion for IT projects. In that budget, OMB stated that, of
approximately 1,200 major IT projects, about half--621 projects,
representing about $22 billion--were on a Management Watch List. In
testimony in March 2004,[Footnote 2] OMB officials explained that the
fiscal year 2005 budget process required agencies to successfully
correct project weaknesses and business case deficiencies of projects
on the Management Watch List; otherwise, OMB would limit agencies'
spending on new starts and other developmental activities.
In the most recent budget, that for fiscal year 2006, OMB continued its
use of a Management Watch List. This budget includes 1,087 IT projects,
totaling about $65 billion. Of this total, 342 projects, representing
about $15 billion, are on the Management Watch List. The budget also
stated that projects on the Management Watch List had to address
performance, security, or other related issues before funding would be
obligated in fiscal year 2006.
According to OMB officials, the office identifies projects for the
Management Watch List through their evaluation of justifications for
funding that agencies submit for major IT projects as part of the
budget development process. This evaluation is carried out as part of
OMB's predominant mission: to assist the President in overseeing the
preparation of the federal budget and to supervise budget
administration in executive branch agencies. OMB is also responsible
for evaluating the effectiveness of agency programs, policies, and
procedures; assessing competing funding demands among agencies; and
setting funding priorities. Finally, OMB is responsible for overseeing
and coordinating the administration's policies regarding procurement,
financial management, information, and regulations. In each of these
three areas of responsibility, OMB's role is to help improve
administrative management, to develop better performance measures and
coordinating mechanisms, and to reduce unnecessary burden on the public.
To drive improvement in the implementation and management of IT
projects, the Congress enacted the Clinger-Cohen Act in 1996, which
expanded the responsibilities of the agencies and OMB under the
Paperwork Reduction Act.[Footnote 3] Under the act, agencies are
required to engage in capital planning and performance-and results-
based management. OMB is required to establish processes to analyze,
track, and evaluate the risks and results of major capital investments
in information systems made by executive agencies. OMB is also required
to report to the Congress on the net program performance benefits
achieved as a result of major capital investments in information
systems that are made by executive agencies.[Footnote 4]
In response to the Clinger-Cohen Act and other statutes, OMB developed
section 300 of Circular A-11. This section provides policy for
planning, budgeting, acquisition, and management of federal capital
assets and instructs agencies on budget justification and reporting
requirements for major IT investments.[Footnote 5] Section 300 defines
the budget exhibit 300 as a document that agencies submit to OMB to
justify resource requests for major IT investments. This reporting
mechanism (part of the budget formulation and review process) is
intended to enable an agency to demonstrate to its own management, as
well as to OMB, that it has employed the disciplines of good project
management; developed a strong business case for the investment; and
met other Administration priorities in defining the cost, schedule, and
performance goals proposed for the investment. The exhibit 300 includes
information that is intended, among other things, to help OMB and the
agencies identify and correct poorly planned or performing investments
(i.e., investments that are behind schedule, over budget, or not
delivering expected results) and real or potential systemic weaknesses
in federal information resource management (e.g., project manager
qualifications).
According to OMB's description of its processes, agencies' exhibit 300
business cases are reviewed by OMB analysts from its four statutory
offices--the Offices of E-Government and Information Technology (e-
Gov), Information and Regulatory Affairs (OIRA), Federal Financial
Management, and Federal Procurement Policy--and its Resource Management
Offices (RMO). Within OIRA, each of about 12 analysts is responsible
for overseeing IT projects for a specific agency or (more commonly)
several agencies. According to OMB officials, the OIRA and e-Gov
analysts, along with RMO program examiners, evaluate and score agency
exhibit 300 business cases as part of the development of the
President's Budget. The results of this review are provided to agencies
through what is called the "passback" process. That is, OMB passes the
requests back to agencies with its evaluation, which identifies any
areas requiring remediation.
The integrity of this review process presupposes that the exhibit 300s
are accurate. In response to a request from this committee, we are
currently reviewing the quality of the information that underlies
exhibit 300s at several agencies. We will be reporting on this work in
the fall of this year.
OMB Established Processes and Criteria for Identifying Weak Projects,
but It Did Not Use an Aggregate List to Perform Its Analysis or
Oversight:
According to OMB officials, including the Deputy Administrator of OIRA
and the Chief of the Information Technology and Policy Branch, OMB
staff identified projects for the Management Watch List through their
evaluation of the exhibit 300s that agencies submit for major IT
projects as part of the budget development process. The OMB officials
added that the scoring of agency exhibit 300s is based on guidance in
OMB Circular A-11[Footnote 6] that is intended to ensure that agency
planning and management of capital assets are consistent with OMB
policy and guidance.
As described in Circular A-11, the scoring of a business case consists
of individual scoring for 10 categories, as well as a total composite
score of all the categories. (Examples of these 10 categories are
performance goals, security and privacy, performance-based management
system--including the earned value management system[Footnote 7]--and
support of the President's Management Agenda.) According to Circular A-
11, scores range from 1 to 5, with 5 indicating investments whose
business cases provided the best justification and 1 the least.
OMB officials said that, for fiscal year 2005, an IT project was placed
on the Management Watch List if its exhibit 300 business case received
a total composite score of 3 or less, or if it received a score of 3 or
less in the areas of performance goals, performance-based management
systems, or security and privacy, even if its overall score was a 4 or
5. OMB reported that agencies with weaknesses in these three areas were
to submit remediation plans addressing the weaknesses.
According to OMB management, individual analysts were responsible for
evaluating projects and determining which projects met the criteria to
be on the Management Watch List for their assigned agencies. To derive
the total number of projects on the list that were reported for fiscal
year 2005, OMB polled the individual analysts and compiled the numbers.
OMB officials said that they did not aggregate these projects into a
single list describing projects and their weaknesses, because they did
not see such an activity as necessary in performing OMB's predominant
mission. Further, OMB officials stated that the limited number of
analysts involved enabled them to explore governmentwide issues using
ad hoc queries and to develop approaches to address systemic problems
without the use of an aggregate list. They pointed at successes in
improving IT management, such as better compliance with security
requirements, as examples of the effectiveness of their current
approach.
Nevertheless, OMB has not fully exploited the opportunity to use its
Management Watch List as a tool for analyzing IT investments on a
governmentwide basis. According to the Clinger-Cohen Act, OMB is
required to establish processes to analyze, track, and evaluate the
risks and results of major IT capital investments made by executive
agencies, which aggregation of the Management Watch List would
facilitate. Without aggregation, OMB's ability to conduct
governmentwide analysis is limited, since no governmentwide dataset
exists--only a set of subordinate datasets in the hands of individual
analysts. In addition, each time an up-to-date report is required, OMB
must query all its analysts to assemble an aggregate response; thus,
the office cannot efficiently identify, analyze, and report on the full
set of IT investments requiring corrective actions.
OMB's Monitoring of Projects Was Inconsistent, and Agency Follow-up
Activities Were Not Tracked Centrally:
OMB asked agencies to take corrective actions to address weaknesses
associated with projects on the Management Watch List, but it did not
develop a structured, consistent process or criteria for deciding how
to follow up on these actions. Instead, OMB officials, including the
Deputy Administrator of OIRA and the Chief of the Information
Technology and Policy Branch, said that the decision on whether and how
to follow up on a specific project was typically made jointly between
the OIRA analyst and the RMO program examiner who had responsibility
for the individual agency, and that follow-up on specific projects was
driven by a number of factors, only one of which was inclusion on the
Management Watch List. According to these officials, those Management
Watch List projects that did receive specific follow-up attention
received feedback through the passback process, through targeted
evaluation of remediation plans designed to address weaknesses, and
through the apportioning of funds so that the use of budgeted dollars
was conditional on appropriate remediation plans being in
place.[Footnote 8]
These officials also said that follow-up of some Management Watch List
projects was done through quarterly e-Gov Scorecards; these are reports
that use a red/yellow/green scoring system to illustrate the results of
OMB's evaluation of the agencies' implementation of e-government
criteria in the President's Management Agenda. OMB determines the
scores in quarterly reviews, in which it evaluates agency progress
toward agreed-upon goals along several dimensions. The e-gov scores are
part of the input to the quarterly reporting on the President's
Management Agenda.
OMB officials also stated that those Management Watch List projects
that did receive follow-up attention were not tracked centrally, but
only by the individual OMB analysts. Accordingly, OMB could not readily
tell us which of the 621 watch list projects for fiscal year 2005 were
followed up on, nor could it use the list to describe the relationship
between its follow-up activities and the changes in the numbers of
projects on the watch list between fiscal year 2005 (621 projects) and
fiscal year 2006 (342 projects).
OMB does not have specific criteria for prioritizing follow-up on
Management Watch List projects. Without specific criteria, OMB staff
may be agreeing to commit resources to follow up on projects that did
not represent OMB's top priorities from a governmentwide perspective.
For example, inconsistent attention to OMB priorities, such as earned
value management, could undermine the objectives that OMB set in these
areas. In addition, major projects with significant management
deficiencies may have continued to absorb critical agency resources.
In order for OMB management to have assurance that IT program
deficiencies are addressed, it is critical that corrective actions
associated with Management Watch List projects be monitored. Such
monitoring is instrumental in ensuring that agencies address and
resolve weaknesses found in exhibit 300s, which may indicate underlying
weaknesses in project planning or management. Tracking agency follow-up
activities is essential to enabling OMB to determine progress on both
specific projects and governmentwide trends. Without tracking specific
follow-up activities, OMB could not readily ascertain whether the risks
that it identified through its Management Watch List were being managed
effectively; if they were not, funds were potentially being spent on
poorly planned and managed projects.
In summary, OMB's scoring of agency IT budget submissions and
identification of weaknesses has resulted in opportunities to
strengthen investments. However, the office has not taken the next
step--to develop a single, aggregate list identifying the projects and
their weaknesses--and it has not developed a structured, consistent
process for deciding how to follow up on corrective actions. OMB's
approach does not fully exploit the insights developed through the
scoring process, and it may leave unattended weak projects consuming
significant budget dollars. Developing an aggregated list would help
OMB to realize more fully the potential benefits of using the
Management Watch List as a tool for monitoring and analyzing IT
investments governmentwide. Accordingly, in our report we recommended
that the Director of OMB take the following four actions:
* Develop a central list of projects and their deficiencies.
* Use the list as the basis for selecting projects for follow-up and
for tracking follow-up activities;
* to guide follow-up, develop specific criteria for prioritizing the IT
projects included on the list, taking into consideration such factors
as the relative potential financial and program benefits of these IT
projects, as well as potential risks.
* Analyze the prioritized list to develop governmentwide and agency
assessments of the progress and risks of IT investments, identifying
opportunities for continued improvement.
* Report to the Congress on progress made in addressing risks of major
IT investments and management areas needing attention.
In commenting on a draft of this report, OMB's Administrator of the
Office of E-Government and Information Technology expressed
appreciation for our review of OMB's use of its Management Watch List.
However, the Administrator disagreed with our assessment that an
aggregated governmentwide list is necessary to perform adequate
oversight and management, and that OMB does not know whether risks are
being addressed. Nonetheless, based on OMB's inability to easily report
which of the 621 investments on the Management Watch List remained
deficient or how much of the $22 billion cited in the President's
Budget remained at risk, we continue to believe that an aggregate list
would facilitate OMB's ability to track progress.
Mr. Chairman, that concludes my testimony. I would be pleased to answer
any questions that you and the other Members of the Committee may have.
Contact and Acknowledgements:
For further information, please contact David A. Powner at (202) 512-
9286 or Lester Diamond at (202) 512-7957. We can also be reached by e-
mail at pownerd@gao.gov or diamondl@gao.gov. Key contributors to this
testimony were William G. Barrick, Barbara Collier, Lester Diamond, and
Sandra Kerr.
FOOTNOTES
[1] GAO, Information Technology: OMB Can Make More Effective Use of Its
Investment Reviews, GAO-05-276 (Washington, D.C.: Apr. 15, 2005).
[2] On March 3, 2004, OMB's Deputy Director for Management and its
Administrator for Electronic Government and Information Technology
testified at a hearing conducted by the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census,
Committee on Government Reform, House of Representatives. The hearing
topic was "Federal Information Technology Investment Management,
Strategic Planning, and Performance Measurement: $60 Billion Reasons
Why."
[3] 44 U.S.C. § 3504(a)(1)(B)(vi) (OMB); 44 U.S.C. § 3506(h)(5)
(agencies).
[4] These requirements are specifically described in the Clinger-Cohen
Act, 40 U.S.C. § 11302(c).
[5] OMB Circular A-11 defines a major IT investment as an investment
that requires special management attention because of its importance to
an agency's mission or because it is an integral part of the agency's
enterprise architecture, has significant program or policy
implications, has high executive visibility, or is defined as major by
the agency's capital planning and investment control process.
[6] These scoring criteria are presented in Office of Management and
Budget Circular A-11, Part 7, Planning, Budgeting, Acquisition, and
Management of Capital Assets (July 2004).
[7] Earned value management is a project management tool that
integrates the investment scope of work with schedule and cost elements
for investment planning and control. This method compares the value of
work accomplished during a given period with that of the work expected
in the period. Differences in expectations are measured in both cost
and schedule variances.
[8] The authority for apportioning funds is specifically described in
the Clinger-Cohen Act, 40 U.S.C. § 11303(b)(5)(B)(ii).
