Information Technology
Federal Agencies Need to Strengthen Investment Board Oversight of Poorly Planned and Performing Projects Gao ID: GAO-09-566 June 30, 2009The federal government expects to spend about $71 billion for information technology (IT) projects for fiscal year 2009. Given the amount of money at stake, it is critical that these projects be planned and managed effectively to ensure that the public's resources are being invested wisely. This includes ensuring that they receive appropriate selection and oversight reviews. Selection involves identifying and analyzing projects' risks and returns and selecting those that will best support the agency's mission needs; oversight includes reviewing the progress of projects against expectations and taking corrective action when these expectations are not being met. GAO was asked to determine whether (1) federal departments and agencies have guidance on the role of their department-level investment review boards in selecting and overseeing IT projects and (2) these boards are performing reviews of poorly planned and poorly performing projects. In preparing this report, GAO reviewed the guidance of 24 major agencies and requested evidence of department-level board reviews for a sample of 41 projects that were identified as being poorly planned or poorly performing.
The 24 major federal agencies have guidance calling for department-level investment review boards to select and oversee IT investments. However, while all of the agencies had department-level boards, the board membership for the Departments of Commerce and Labor did not include business unit (i.e., mission) representation as called for by IT investment management best practices. Without business unit representation on their department-level boards, these agencies will not have assurance that the boards include those executives who are in the best position to make the full range of investment decisions necessary for them to carry out their missions most effectively. About half of the projects GAO examined did not receive selection or oversight reviews. Specifically, 12 of the 24 projects GAO reviewed that were identified by OMB as being poorly planned (accounting for $4.9 billion in the President's fiscal year 2008 budget request or two-thirds of the funding represented by the 24 projects) did not receive a selection review, and 13 of 28 poorly performing projects GAO reviewed (amounting to about $4.4 billion or 93 percent of the funding represented by the 28 projects) did not receive an oversight review by a department-level board. Agencies provided several reasons for not performing department-level board reviews, including some which were not consistent with sound management practices. Furthermore, 6 of the 11 projects in the sample identified as being both poorly planned and poorly performing, with over $3.7 billion in funding in the President's fiscal year 2008 budget request, received neither a selection review nor an oversight review. Without consistent involvement of department-level review boards in selecting and overseeing projects that have been identified as poorly planned or poorly performing, agencies incur the risk that these projects will not improve, potentially leading to billions of federal taxpayer dollars being wasted.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-09-566, Information Technology: Federal Agencies Need to Strengthen Investment Board Oversight of Poorly Planned and Performing Projects
This is the accessible text file for GAO report number GAO-09-566
entitled 'Information Technology: Federal Agencies Need to Strengthen
Investment Board Oversight of Poorly Planned and Performing Projects'
which was released on July 30, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
June 2009:
Information Technology:
Federal Agencies Need to Strengthen Investment Board Oversight of
Poorly Planned and Performing Projects:
GAO-09-566:
GAO Highlights:
Highlights of GAO-09-566, a report to congressional requesters.
Why GAO Did This Study:
The federal government expects to spend about $71 billion for
information technology (IT) projects for fiscal year 2009. Given the
amount of money at stake, it is critical that these projects be planned
and managed effectively to ensure that the public‘s resources are being
invested wisely. This includes ensuring that they receive appropriate
selection and oversight reviews. Selection involves identifying and
analyzing projects‘ risks and returns and selecting those that will
best support the agency‘s mission needs; oversight includes reviewing
the progress of projects against expectations and taking corrective
action when these expectations are not being met.
GAO was asked to determine whether (1) federal departments and agencies
have guidance on the role of their department-level investment review
boards in selecting and overseeing IT projects and (2) these boards are
performing reviews of poorly planned and poorly performing projects. In
preparing this report, GAO reviewed the guidance of 24 major agencies
and requested evidence of department-level board reviews for a sample
of 41 projects that were identified as being poorly planned or poorly
performing.
What GAO Found:
The 24 major federal agencies have guidance calling for department-
level investment review boards to select and oversee IT investments.
However, while all of the agencies had department-level boards, the
board membership for the Departments of Commerce and Labor did not
include business unit (i.e., mission) representation as called for by
IT investment management best practices. Without business unit
representation on their department-level boards, these agencies will
not have assurance that the boards include those executives who are in
the best position to make the full range of investment decisions
necessary for them to carry out their missions most effectively.
About half of the projects GAO examined did not receive selection or
oversight reviews. Specifically, 12 of the 24 projects GAO reviewed
that were identified by OMB as being poorly planned (accounting for
$4.9 billion in the President‘s fiscal year 2008 budget request or two-
thirds of the funding represented by the 24 projects) did not receive a
selection review, and 13 of 28 poorly performing projects GAO reviewed
(amounting to about $4.4 billion or 93 percent of the funding
represented by the 28 projects) did not receive an oversight review by
a department-level board. Agencies provided several reasons for not
performing department-level board reviews, including some which were
not consistent with sound management practices. Furthermore, 6 of the
11 projects in the sample identified as being both poorly planned and
poorly performing, with over $3.7 billion in funding in the President‘s
fiscal year 2008 budget request, received neither a selection review
nor an oversight review (see table below). Without consistent
involvement of department-level review boards in selecting and
overseeing projects that have been identified as poorly planned or
poorly performing, agencies incur the risk that these projects will not
improve, potentially leading to billions of federal taxpayer dollars
being wasted.
Table: Poorly Planned and Performing Projects That Received No
Department-Level Board Review (Dollars in millions):
Agency: Education;
IT investment: Common Services for Borrowers;
FY 2008 request: $15.
Agency: Homeland Security;
IT investment: DHS-Infrastructure;
FY 2008 request: $1,071.
Agency: Homeland Security;
IT investment: CBP Secure Border Initiative (SBI) net;
FY 2008 request: $1,000.
Agency: Treasury;
IT investment: Enterprise IT Infrastructure Optimization Initiative;
FY 2008 request: $1,638.
Agency: Treasury;
IT investment: Integrated Collection System;
FY 2008 request: $9.
Agency: Nuclear Regulatory Commission;
IT investment: National Source Tracking System;
FY 2008 request: $4.
Agency: Total;
FY 2008 request: $3,737.
Source: GAO analysis of agency data.
[End of table]
What GAO Recommends:
GAO is making recommendations to selected agencies to improve their
department-level board representation and selection and oversight
processes. In comments on a draft of the report, 11 agencies generally
agreed with the recommendations and one did not.
View [hyperlink, http://www.gao.gov/products/GAO-09-566] or key
components. For more information, contact David A. Powner at (202) 512-
9286 or pownerd@gao.gov.
[End of section]
Contents:
Letter:
Background:
Major Federal Agencies Have Guidance for Selection and Oversight of IT
Investments, but Two Agency Boards Lack Business Unit Representation:
Many Projects Did Not Receive a Department-Level IRB Selection or
Oversight Review:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Commerce:
Appendix III: Comments from the Department of Defense:
Appendix IV: Comments from the Department of Education:
Appendix V: Comments from the Department of Homeland Security:
Appendix VI: Comments from the Department of Housing and Urban
Development:
Appendix VII: Comments from the Department of the Interior:
Appendix VIII: Comments from the Department of Justice:
Appendix IX: Comments from the Department of Labor:
Appendix X: Comments from the Department of the Treasury:
Appendix XI: Comments from the Department of Veterans Affairs:
Appendix XII: Comments from the National Aeronautics and Space
Administration:
Appendix XIII: Comments from the Nuclear Regulatory Commission:
Appendix XIV: Comments from the Social Security Administration:
Appendix XV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Project Selection Reviews by Department-Level IRBs:
Table 2: Project Oversight Reviews by Department-Level IRBs:
Table 3: Department-Level Reviews Received by Poorly Planned and Poorly
Performing Projects:
Figures:
Figure 1: Frequency of Department-Level IRB Oversight Reviews:
Figure 2: Percentage of Projects That Received a Selection Review by a
Department-Level IRB:
Figure 3: Percentage of Projects That Received an Oversight Review by a
Department-Level IRB:
Abbreviations:
CFO: chief financial officer:
CIO: chief information officer:
IRB: investment review board:
IT: information technology:
ITIM: information technology investment management:
NASA: National Aeronautics and Space Administration:
OMB: Office of Management and Budget:
PBO: performance-based organization:
SBA: Small Business Administration:
SBI: Secure Border Initiative:
USAID: U.S. Agency for International Development:
USPTO: U.S. Patent and Trademark Office:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
June 30, 2009:
Congressional Requesters:
Federal government expenditures for information technology (IT)
investments have exceeded $60 billion each year since fiscal year 2004,
and the government expects to spend about $71 billion for IT projects
in fiscal year 2009. Given the amount of money at stake, it is critical
that IT projects be planned and managed effectively to ensure that the
public's resources are being invested wisely.
To this end, the Office of Management and Budget (OMB), which plays a
key role in directing and overseeing the federal government's IT
investments, established a Management Watch List[Footnote 1] of major
IT projects identified as poorly planned and also required the major
federal departments and agencies to identify high-risk projects that
are performing poorly.[Footnote 2] In addition, GAO and OMB have long
endorsed having agencies establish a disciplined process for their
executives to participate in selecting and overseeing projects, among
other things. Selecting projects involves identifying and analyzing
risks and returns before committing any significant funds to them and
selecting those that will best support the agency's mission needs.
[Footnote 3] Overseeing projects involves reviewing the progress of
projects against expectations and taking corrective action when these
expectations are not being met.
Given the large number and dollar value of projects that are identified
as being poorly planned and poorly performing every year, you asked us
to determine whether (1) federal departments and agencies have guidance
on the role of their department-level investment review boards (IRB) in
selecting and overseeing IT projects and (2) these boards are actually
performing selection and oversight reviews of poorly planned and poorly
performing projects.
To address the first objective, we reviewed the investment management
guidance of 24 major agencies[Footnote 4] to determine the role
department-level IRBs are expected to play in selecting and overseeing
IT projects, updating the findings from our 2004 governmentwide review
of agencies' use of key investment management practices.[Footnote 5] We
also reviewed the composition of the boards to determine whether they
included senior executives from both IT and business units. To address
the second objective, we identified a sample of 48 (subsequently
reduced to 41) projects that were identified as being poorly planned
according to OMB's Management Watch List or reported as being poorly
performing on the High-Risk List. For each project, we requested and
analyzed evidence of department-level IRB reviews during the time
period when the projects were on the OMB lists.
We conducted this performance audit from January 2008 to June 2009 in
Washington, D.C., in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
Further details on our objectives, scope, and methodology are provided
in appendix I.
Background:
OMB plays a key role in helping federal agencies manage their IT
investments by working with them to better plan, justify, and determine
how much they need to spend on IT projects and how to manage approved
projects. In particular, the Clinger-Cohen Act[Footnote 6] of 1996
requires OMB to establish processes to analyze, track, and evaluate the
risks and results of major capital investments in information systems
made by federal agencies and report to Congress on the net program
performance benefits achieved as a result of these investments.
[Footnote 7] In addition, the Clinger-Cohen Act places responsibility
for managing IT investments with the heads of agencies[Footnote 8] and
establishes chief information officers to advise and assist agency
heads in carrying out this responsibility.[Footnote 9]
To help carry out its oversight role and assist the agencies in
carrying out their responsibilities, OMB developed its Management Watch
List[Footnote 10] in 2003 and its High-Risk List in 2005 to focus
executive attention and to ensure better planning and tracking of the
major IT investments. The Management Watch List identifies projects at
federal agencies that are poorly planned, i.e., projects with
weaknesses in their funding justifications, which are known as exhibit
300s. Because of the focus on the funding justifications, projects on
the Management Watch List specifically concern the process by which
agencies select projects to invest in. OMB places projects on the High-
Risk List when they require special attention from oversight
authorities and the highest level of agency management. These projects
are not necessarily "at risk" of failure, but may be on the list
because of one or more of the following four reasons:
* The agency has not consistently demonstrated the ability to manage
complex projects.
* The project has exceptionally high development, operating, or
maintenance costs, either in absolute terms or as a percentage of the
agency's total IT portfolio.
* The project is being undertaken to correct recognized deficiencies in
the adequate performance of an essential mission program or function of
the agency, a component of the agency, or another organization.
* Delay or failure of the project would introduce for the first time
unacceptable or inadequate performance or failure of an essential
mission function of the agency, a component of the agency, or another
organization.
The High-Risk List also includes projects that are performing poorly
(i.e., high-risk projects with reported performance shortfalls). High-
risk projects are identified as having performance shortfalls if one or
more of the following performance evaluation criteria are not met: (1)
establishing baselines with clear cost, schedule, and performance
goals; (2) maintaining the project's cost and schedule variances within
10 percent; (3) assigning a qualified project manager; and (4) avoiding
duplication by leveraging inter-agency and governmentwide investments.
Projects on the High-Risk List, therefore, require disciplined and
effective oversight to ensure that performance shortfalls, if any, are
addressed.
The Management Watch List and High-Risk List were intended to be
instrumental in helping both OMB and the agencies to identify and
improve oversight of poorly planned and poorly performing projects. We
have issued several reports, made recommendations for improvements, and
testified over the past 4 years on the effectiveness of these
processes.[Footnote 11] Last year, for example, we reported that, as of
July 2008, OMB and the 24 major federal agencies identified 352 IT
projects--totaling about $23.4 billion--as being poorly planned (on the
Management Watch List).[Footnote 12] Also last year, agencies reported
that 87 of their high-risk projects (totaling about $4.8 billion) were
poorly performing. In addition, 26 projects (totaling about $3 billion)
were considered both poorly planned and poorly performing.[Footnote 13]
OMB took several steps to address our recommendations to improve the
identification and oversight of Management Watch List and High-Risk
List projects; however, further action is needed, including, for
example, identifying the deficiencies (i.e., performance shortfalls)
associated with the high-risk projects. On April 28, 2009, we testified
that the future of the Management Watch List and High-Risk List was
uncertain because OMB officials stated that they had not decided if the
agency plans to continue to use these lists. We noted that OMB needs to
decide if it is going to continue to use the Management Watch List and
High-Risk List and, if not, that OMB should promptly implement other
appropriate mechanisms to help direct and oversee IT investments in the
future.[Footnote 14] In response, the Federal Chief Information Officer
testified that OMB would determine how to better oversee poorly planned
and performing projects by the end of June 2009.
Investment Management Framework Calls for Boards to Select and Oversee
IT Investments:
Federal agencies face significant challenges in planning for and
managing their IT systems and networks. These challenges can be
addressed, in part, by the use of systematic management processes to
select, control, and evaluate the investments. To further support the
implementation of such processes, we developed an IT investment
management (ITIM) framework[Footnote 15] for agencies to use. It is
based on our research of IT investment management practices of leading
private and public sector organizations and can be used to determine
both the status of an agency's current IT investment management
capabilities and the additional steps that are needed to establish more
effective processes. The framework consists of progressive stages of
maturity for any given organization relative to its selection and
oversight responsibilities. We have used the framework in many of our
reports,[Footnote 16] and a number of agencies have adopted it.
The ITIM maturity framework cites the establishment of "one or more IT
investment management boards" as a fundamental step in establishing a
mature capital planning process.[Footnote 17] The framework states that
a departmentwide IT investment review board (IRB) composed of senior
executives from both IT and business units should be responsible for
defining and implementing the department's IT investment governance
process. This department-level IRB is to provide selection and
oversight of department IT projects to ensure that the department's
portfolio of projects meets mission needs at expected levels of cost
and risk. Selecting projects involves identifying and analyzing
projects' risks and returns before committing any significant funds to
them and selecting those that will best support the agency's mission
needs; overseeing projects involves reviewing the progress of projects
against expectations and taking corrective action when these
expectations are not being met.
To ensure that agencies' department-level boards are using a
disciplined selection and oversight process, the ITIM framework also
states that, among other things, the department-level board should:
select new investments and reselect ongoing investments; perform
regular reviews of each project's performance against stated
expectations; and receive data associated with a project's actual
performance (including cost, schedule, benefit, and risk performance).
Importantly, according to the ITIM framework, while these functions can
be performed by subordinate boards, the department-level IRBs must
maintain ultimate responsibility for and visibility into the
subordinate boards' activities.
Prior Reviews Have Identified Weaknesses in Executive-Level Board
Involvement in Selection and Oversight:
We have previously reported that federal agencies face challenges in
effectively managing their IT investments. Specifically, in January
2004, we reported that, although most of the major agencies in our
review had IRBs responsible for defining and implementing their
investment management processes, the agencies did not always have the
mechanisms in place for these boards to effectively control their
investments.[Footnote 18] We made recommendations to the agencies
regarding those practices that were not fully in place. More recently,
in 2008, we reported that the Social Security Administration had not
fully developed policies and procedures for management oversight of its
IT projects and systems, such as elevating problems to the department-
level IRB. We also reported that the Social Security Administration had
not tracked corrective actions for underperforming investments and had
not reported the actions to the department-level IRB.[Footnote 19] To
address these weaknesses, we recommended that the agency strengthen and
expand the board's oversight responsibilities for underperforming
projects and evaluations of projects and establish a mechanism for
tracking corrective actions for underperforming investments.
Major Federal Agencies Have Guidance for Selection and Oversight of IT
Investments, but Two Agency Boards Lack Business Unit Representation:
The 24 major federal agencies have guidance calling for department-
level IRBs to select and oversee IT investments pursuant to OMB
guidance required by the Clinger-Cohen Act, and specified in practices
laid out in the ITIM framework. However, while all of the agencies had
department-level IRBs, the board membership for two agencies did not
include business unit (i.e., mission) representation.
Agency Guidance Calls for Department-Level IRBs to Select Projects:
Each of the agencies had documented guidance that called for a
department-level IRB to perform selection of the projects to be
included in the agency's IT investments. For example, according to the
Department of the Treasury's guidance, its department-level IRB is to
consider investment scoring results and recommendations that are
provided to it by the Chief Information Officer Council (a subordinate
board) and select which investments will be included in Treasury's IT
investment portfolio. The Department of Transportation's recently
issued IT investment management policy delegates responsibility for
project selection, as well as project oversight, to its component-level
investment review boards, but requires its components to establish and/
or document the existence of their boards, specifies the roles and
responsibilities these boards are to have, and establishes specific
metrics to be used by the department-level IRB to measure the
performance of the component boards.
Agency Guidance Calls for Department-Level IRBs to Oversee Projects:
As with project selection, each of the agencies had documented guidance
that called for the department-level IRB to conduct an oversight
reviews of projects, and the frequency of these reviews varied (see
figure 1 for a breakdown of the frequency of oversight reviews
specified in agencies' guidance).
Figure 1: Frequency of Department-Level IRB Oversight Reviews:
[Refer to PDF for image]
Annually: 2;
Semiannually: 1;
Quarterly: 14;
Monthly: 3;
Varies: 4.
Source: GAO analysis of agency data.
[End of figure]
For 20 of the 24 agencies, the guidance allowed the delegation of
oversight reviews to other entities. In these cases, the agencies had
guidance in place to help ensure that these other entities were
effectively carrying out their responsibilities. At the remaining four
agencies--the National Science Foundation, Small Business
Administration, Department of State, and the U.S. Agency for
International Development --project oversight was to be primarily
performed by the department-level IRB. By having guidance specifying
department-level IRB selection and oversight of projects, agencies
recognize the importance of involving those who have the ultimate
responsibility and accountability for the organization's success in key
project decisions.
Two Agencies' Department-Level Boards Lack Business Unit
Representation:
It should be noted, however, that while all of the agencies had
guidance requiring department-level IRBs to be responsible for
selecting and overseeing projects, the boards at the Departments of
Commerce and Labor did not include senior executives from business
units (e.g., line or mission units) as called for in the ITIM
framework[Footnote 20]. Specifically, these boards consisted of
executives from IT and other department mission support units, such as
the Chief Financial Officer, Director of Budget, or Controller, as well
as administrative officers, but did not have appropriate line or
mission representation from the organizations' business units. We have
previously reported that because allocating resources among major IT
investments may require fundamental trade-offs among a multitude of
business objectives, portfolio management decisions are essentially
business decisions and therefore require sufficient business
representation on the department-level IR[Footnote 21]B.:
The two agencies with boards that did not include senior executives
from business units offered the following rationales for this practice.
* The Department of Commerce reported that it does not include
nontechnical program representatives on its department-level IRB
because it would be impractical to have fair representation of all 12
of the major agencies and the dozens of major programs comprising the
department. In addition, Commerce reported that it is run on a
federated basis, putting responsibility on each of the department's
operating units to prioritize its own investments in determining which
should be reviewed by the department. Finally, Commerce stated that it
does not prioritize among investments from its different operating
units; instead, departmental officials work with each operating unit to
ensure that the investment and investment strategy being recommended is
optimum for meeting that operating unit's mission. We have previously
reported that using this approach of giving responsibility to
subordinate units should include appropriate department-level
involvement, either through review and approval of their investments
that meet certain criteria or through awareness of the subordinate
unit's investment management activities.[Footnote 22] We believe that
this corporate visibility should be provided by a board composed of
executives from both business and IT units to ensure that decisions
made are in the best interest of the entire department. In addition,
while Commerce's practice may not be to prioritize among the
investments at the department level, the department has ultimate
responsibility for the success of its operating units' investments and
the department-level IRB should therefore include business
representation to ensure that decisions made are in the best interest
of the agency.
* The Department of Labor reported that the senior IT and
administrative executives who serve on its department-level IRB, have
in-depth, detailed, and expert knowledge of their units' missions and
business objectives and are capable of representing their units'
interests. However, we have previously reported that IT and
administrative executives responsible for mission support functions do
not constitute sufficient business representation because, by virtue of
their responsibilities, they are not in the best position to make
business decisions.[Footnote 23]
* Until these agencies adjust their board memberships to include
representation from their business units, they will not have assurance
that the department-level IRB includes those executives who are in the
best position to make the full range of decisions needed to enable the
agency to carry out its mission most effectively.
Many Projects Did Not Receive a Department-Level IRB Selection or
Oversight Review:
Although all the major agencies had guidance calling for a department-
level IRB selection or oversight review, many of the projects we
examined did not receive one of these reviews. Specifically, 12 of the
24 projects identified by OMB as being poorly planned in 2007
(accounting for about $4.9 billion) did not receive a selection review,
and 13 of 28 poorly performing projects in 2007[Footnote 24] (amounting
to about $4.4 billion) did not receive an oversight review by the
department-level IRB. Furthermore, 6 of the 11 projects identified as
being both poorly planned and poorly performing, with nearly $3.7
billion in funding in the President's fiscal year 2008 budget request,
received neither a selection review nor an oversight review.
Half of the Poorly Planned Projects Did Not Receive a Selection Review
by a Department-Level IRB:
Of the 24 poorly planned projects in 2007 that we reviewed, 12 projects
did not receive a selection review, while 12 were reviewed by the
department-level IRB.[Footnote 25] The requested funding level for
these 24 poorly planned projects was about $7.3 billion. The 12
projects that were reviewed by a department-level IRB accounted for
approximately $2.4 billion, while the 12 projects not reviewed
accounted for about $4.9 billion, about two thirds of the total
requested funding for the 24 projects (see figure 2 and table 1).
Figure 2: Percentage of Projects That Received a Selection Review by a
Department-Level IRB:
[Refer to PDF for image: two pie-charts]
Projects reviewed: 50% (12);
Projects not reviewed: 50% (12);
Projects reviewed: 33% ($2,385,000,000);
Projects not reviewed: 67% ($4,925,000,000).
Source: GAO analysis of agency data.
[End of figure]
We assessed five projects as not having received department-level IRB
selection reviews because the agencies did not provide evidence of such
reviews. Agencies offered varying reasons for why selection reviews had
not been performed for the remaining seven. Table 1 shows whether
projects we reviewed received a selection review from the department-
level IRB and lists reported reasons why no review was performed, where
applicable.
Table 1: Project Selection Reviews by Department-Level IRBs:
Agency: Agriculture;
IT investment/project: Consolidated Infrastructure, Office Automation &
Telecom;
FY 2008 request: $843 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Agriculture;
IT investment/project: Modernize & Innovate the Delivery of Agriculture
Systems (MIDAS);
FY 2008 request: $151 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Commerce;
IT investment/project: U.S. Patent and Trademark Office (USPTO) Patent
Automation Program;
FY 2008 request: $91 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Project not required to
be reviewed by department-level IRB because it belongs to the USPTO, a
performance-based organization.
Agency: Defense;
IT investment/project: Defense Information System for Security;
FY 2008 request: $65 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Education;
IT investment/project: Common Services for Borrowers;
FY 2008 request: $15 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Project not required to
be reviewed by department-level IRB because it is under the oversight
of the Federal Student Aid Executive Leadership Team.
Agency: General Services Administration;
IT investment/project: Federal Supply Service 19;
FY 2008 request: $31 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Health & Human Services;
IT investment/project: Centers for Medicare & Medicaid Services IT
Infrastructure;
FY 2008 request: $126 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Health & Human Services;
IT investment/project: Food and Drug Administration Consolidated
Infrastructure;
FY 2008 request: $102 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Homeland Security;
IT investment/project: DHS-Infrastructure;
FY 2008 request: $1,071 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: DHS did not provide
evidence of a selection review for this project.
Agency: Homeland Security;
IT investment/project: CBP-Secure Border Initiative (SBI) net;
FY 2008 request: $1,000 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: DHS did not provide
evidence of a selection review for this project.
Agency: Labor;
IT investment/project: New Core Financial Management System (NCFMS);
FY 2008 request: $12 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: National Aeronautics and Space Administration;
IT investment/project: NASA Office Automation, IT Infrastructure,
Telecommunications;
FY 2008 request: $548 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: NASA did not provide
evidence that a selection review had been performed by the appropriate
department-level review board.
Agency: NASA;
IT investment/project: JSC Software Development/Integration Laboratory;
FY 2008 request: $132 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: NASA did not provide
evidence that a selection review had been performed by the appropriate
department-level review board.
Agency: NASA;
IT investment/project: Earth Observing System Data Info System;
FY 2008 request: $131 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: NASA did not provide
evidence that a selection review had been performed by the appropriate
department-level review board.
Agency: Nuclear Regulatory Commission;
IT investment/project: National Source Tracking System (NSTS);
FY 2008 request: $ million4;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Lower-level board
performed project selection review.
Agency: Nuclear Regulatory Commission;
IT investment/project: Infrastructure Services and Support;
FY 2008 request: $52 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Lower-level board
performed project selection review.
Agency: Office of Personnel Management;
IT investment/project: Electronic Questionnaire for Processing (eQIP)
and Fingerprint Transaction System (FTS);
FY 2008 request: $17 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Small Business Administration;
IT investment/project: Business Development Management Information
System;
FY 2008 request: $0[A];
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Transportation;
IT investment/project: Combined IT Infrastructure;
FY 2008 request: $234 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: No reason provided by
Transportation.
Agency: Treasury;
IT investment/project: Enterprise IT Infrastructure Optimization
Initiative;
FY 2008 request: $1,638 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Department-level board
was not active.
Agency: Treasury;
IT investment/project: Integrated Collection System;
FY 2008 request: $9 million;
Dept. IRB selection review? No;
Reported reason for lack of selection review: Department-level board
was not active.
Agency: Veterans Affairs;
IT investment/project: VistA-Legacy;
FY 2008 request: $352 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Veterans Affairs;
IT investment/project: VistA Imaging;
FY 2008 request: $41 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Veterans Affairs;
IT investment/project: IT Infrastructure;
FY 2008 request: $645 million;
Dept. IRB selection review? Yes;
Reported reason for lack of selection review: Not applicable.
Agency: Total;
IT investment/project: All 24 projects;
FY 2008 request: $7,310 million;
Dept. IRB selection review? 24.
Agency: Total;
IT investment/project: Projects receiving selection review;
FY 2008 request: $2,385 million;
Dept. IRB selection review? 12.
Agency: Total;
IT investment/project: Projects not receiving selection review;
FY 2008 request: $4,925 million;
Dept. IRB selection review? 12.
Source: GAO analysis of agency data.
[A] Project funding request was less than $500,000, which rounds to $0
in millions.
[End of table]
Following are details on the reasons why the 12 projects did not
receive a department-level IRB review:
* A project belonging to Commerce's USPTO was not reviewed by the
department-level IRB, according to the agency, because the USPTO is a
performance-based organization (PBO),[Footnote 26] and therefore its
projects are not required to be reviewed by the department-level IRB.
According to the legislation that established the USPTO as a PBO, the
office is subject to the policy direction of the Secretary of Commerce,
but it otherwise retains responsibility for decisions regarding the
management and administration of its operations and exercises
independent control of its budget allocations and expenditures,
personnel decisions and processes, procurements, and other
administrative and management functions.
* According to the Department of Education, the Common Services for
Borrowers project did not receive a selection review by the department-
level board because it is under the oversight of the Federal Student
Aid Executive Leadership Team. In written comments on a draft of this
report, however, the department stated that it plans to bring all of
its IT investments under the department-level board's oversight.
* The Department of Homeland Security did not provide evidence of a
selection review for its two projects but noted that it was
reengineering its investment management process to include department-
level IRB reviews of projects at key milestone decision points.
* Although NASA stated that its three projects were governed by
oversight bodies, the documentation provided did not show evidence that
reviews had been performed by the appropriate department-level review
board.
* At the Nuclear Regulatory Commission, a lower-level board performed
the selection reviews. According to the agency's guidance, the
department-level board should have performed the reviews. It stated
that this board only gets involved when the lower-level board believes
issues need to be elevated. However, NRC's guidance does not specify
when issues need to be elevated to the department-level IRB. In
addition, the agency did not provide any examples of cases when issues
had been elevated to the department-level IRB.
* Officials from the Department of Transportation's Office of the Chief
Information Officer could not provide a reason why a department-level
board selection review of its projects had not been performed. In
commenting on a draft of this report, the agency stated that it planned
to have this project reviewed in detail by its departmental-level
board.
* The Department of the Treasury's projects did not receive a
department-level IRB selection review because this board was not active
during the time frame we considered during our review. The department,
however, has since then reestablished its department-level IRB.
About Half of the Poorly Performing Projects Did Not Receive an
Oversight Review by the Department-Level IRB:
About half of the poorly performing projects in 2007 we reviewed did
not receive an oversight review by a department-level IRB. Of the 28
projects, 13 did not receive an oversight review by the department-
level IRB, while 15 did. The President's requested fiscal year 2008
funding for the 28 projects totaled approximately $4.7 billion. The 15
projects that received a review represented approximately $0.3 billion,
or 7 percent of the total $4.7 billion funding request, while the 13
poorly performing projects that were not reviewed totaled nearly $4.4
billion, or 93 percent of the total requested funding. (See figure 3
and table 2.)
Figure 3: Percentage of Projects That Received an Oversight Review by a
Department-Level IRB:
[Refer to PDF for image: two pie-charts]
Projects reviewed: 54% (15);
Projects not reviewed: 46% (13);
Projects reviewed: 7% ($337,000,000);
Projects not reviewed: 933% ($4,414,000,000).
Source: GAO analysis of agency data.
[End of figure]
Table 2 shows whether projects received oversight reviews, as well as
reported reasons why no review was performed, where applicable.
Table 2: Project Oversight Reviews by Department-Level IRBs:
Agency: Agriculture;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Modernize & Innovate the Delivery of
Agriculture Systems;
FY 2008 request: $151 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Commerce;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Financial Management Line of Business
Migration;
FY 2008 request: $0[A];
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Defense;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Integrated Acquisition Environment (IAE)
Shared Services Provider - Past Performance Information Retrieval
System (PPIRS);
FY 2008 request: $10 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Below financial threshold
required for review by board.
Agency: Defense;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Defense Information System for Security;
FY 2008 request: $65 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Project being
rebaselined.
Agency: Education;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Common Services for Borrowers;
FY 2008 request: $15 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Project not required to
be reviewed by department-level IRB because it is under the oversight
of the Federal Student Aid Executive Leadership Team.
Agency: Education;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: ADvance (Aid Delivery);
FY 2008 request: $65 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Project not required to
be reviewed by department-level IRB because it is under the oversight
of the Federal Student Aid Executive Leadership Team.
Agency: Environmental Protection Agency;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: FM LoB--Migration;
FY 2008 request: $0[A];
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Environmental Protection Agency;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: eRulemaking;
FY 2008 request: $1 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Health & Human Services;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Federal Health Architecture--Managing
Partner;
FY 2008 request: $4 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Homeland Security;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: DHS-Infrastructure;
FY 2008 request: $1,071 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: While DHS provided
evidence that a lower-level board had agreed to submit this project to
the department-level IRB for review, the agency did not provide
evidence that this review had been performed.
Agency: Homeland Security;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: CBP Secure Border Initiative (SBI) net;
FY 2008 request: $1,000 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: While DHS stated that
this project had received an oversight review by the department-level
board IRB, it did not provide sufficient evidence to support this.
Agency: Homeland Security;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: SEI/NPPD US-VISIT;
FY 2008 request: $462 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: While DHS stated that
this project had received an oversight review by the department-level
board IRB, it did not provide sufficient evidence to support this.
Agency: Housing & Urban Development;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Integrated Financial Management Improvement
Program;
FY 2008 request: $22 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Interior;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: MMS--OCS Connect;
FY 2008 request: $14 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Justice;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: FBI Sentinel[B];
FY 2008 request: $57 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Labor;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: EFAST2;
FY 2008 request: $19 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Labor;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: New Core Financial Management System
(NCFMS);
FY 2008 request: $12 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: National Aeronautics and Space Administration;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Integrated Enterprise Management-Core
Financial;
FY 2008 request: $22 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Nuclear Regulatory Commission;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: National Source Tracking System (NSTS);
FY 2008 request: $4 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Review performed by lower-
level board.
Agency: Small Business Administration (SBA);
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Business Development Management Information
System;
FY 2008 request: $0[A];
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: SBA;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Disaster Credit Management System;
FY 2008 request: $13 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: State;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: State Messaging and Archive Retrieval
Toolset;
FY 2008 request: $10 million;
Dept. IRB oversight review? Yes;
Reported reason for lack of oversight review: Not applicable.
Agency: Treasury;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Enterprise IT Infrastructure Optimization
Initiative;
FY 2008 request: $1,638 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Department-level board
was not active.
Agency: Treasury;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Treasury Automated Auction Processing
System;
FY 2008 request: $32 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Department-level board
was not active.
Agency: Treasury;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Integrated Collection System;
FY 2008 request: $9 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Department-level board
was not active.
Agency: U.S. Agency for International Development;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: JAMS System;
FY 2008 request: $12 million;
Dept. IRB oversight review?
Yes; Reported reason for lack of oversight review: Not applicable.
Agency: U.S. Agency for International Development;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: HSPD-12;
FY 2008 request: $2 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Project has not proceeded
due to lack of funding.
Agency: Veterans Affairs;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: VistA Imaging;
FY 2008 request: $41 million;
Dept. IRB oversight review? No;
Reported reason for lack of oversight review: Department-level board
does not review projects in operations and maintenance.
Agency: Total;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: All 28 projects;
FY 2008 request: $4,751 million;
Dept. IRB oversight review? 28.
Agency: Total;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Projects receiving oversight review;
FY 2008 request: $337 million;
Dept. IRB oversight review? 15.
Agency: Total;
Poorly performing project: high-risk project with performance
shortfalls in 2006 or 2007: Projects not receiving oversight review;
FY 2008 request: $4,414 million;
Dept. IRB oversight review? 13.
Source: GAO analysis of agency data.
[A] Project funding request was less than $500,000, which rounds to $0
in millions.
[B] We included the Sentinel project in our sample because it was
reported as having a performance shortfall (a schedule variance of 14%)
in the Department of Justice's high-risk report for September 2007. We
have performed several reviews of Sentinel and recognized FBI's recent
efforts to improve the project's management. For example, in July 2007,
we reported that the FBI had established and was following effective
processes to proactively identify and mitigate program risks before
they have chance to become actual cost, schedule, or performance
problems (GAO-07-912). More recently, we reported that FBI was
employing five key acquisition methods that should increase the chances
of cost effectively delivering required Sentinel capabilities on time
(GAO-08-1014).
[End of table]
Agencies provided several reasons why the 13 projects did not receive
oversight reviews, including some which were not consistent with sound
management practices:
* One Defense project's funding was below the financial threshold
required for a review by the department-level IRB, consistent with the
agency's guidance. However, in May 2007 and May 2009, we reported that
DOD's guidance and practices did not provide for sufficient oversight
and visibility into component-level investment management activities,
including component reviews of investments such as this
project.[Footnote 27] We made recommendations to DOD to address these
weaknesses, which DOD has yet to fully implement.
* Another Defense project was reportedly being rebaselined (meaning
that its cost, schedule, and performance goals were being modified to
reflect a change in the scope of the work) and therefore had not
received a review by the department-level IRB. This project, however,
continues to be funded and therefore could have benefited from a
department-level oversight review.
* According to the Department of Education, the two projects we
reviewed did not receive oversight reviews by the department-level IRB
because they were under the oversight of the Federal Student Aid
Executive Leadership Team. As noted earlier, in written comments on a
draft of this report, the department stated it plans to bring all of
its IT investments under the department-level board's oversight.
* While DHS provided evidence that a lower-level board had agreed to
submit the DHS-Infrastructure Project to the department-level IRB for
review, the agency did not provide evidence that this review had been
performed. The department also stated that SBInet and US-VISIT projects
had received an oversight review by the department-level IRB, but did
not provide sufficient evidence to support this, including information
presented to the board for review. In March 2009, however, DHS
officials told us that they had recently made changes to their
investment review process and, as part of these changes, were planning
to improve the documentation associated with department-level IRB
reviews.
* A Nuclear Regulatory Commission project should have received a review
by the department-level IRB according to the agency's guidance, but
officials told us that, in practice, this board only gets involved when
the lower-level board elevates issues. However, agency officials were
unable to provide us with any examples where the lower-level board had
elevated issues about the project to the IRB.
* The Department of the Treasury's projects did not receive a
department-level IRB oversight review because this board was not active
during the time frame we considered during our review. The department,
however, has since then reestablished its department-level IRB.
* According to the U.S. Agency for International Development, its
project did not receive an oversight review because it has not been
able to proceed due to lack of funding. We agree that an oversight
review was not warranted since there was no activity on the project.
* A Veterans Affairs project was not reviewed because the IRB is not
required to review projects in the operations and maintenance stage.
Instead, oversight of projects in this stage is the responsibility of
the Office of the Chief Information Officer. However, the IRB does not
oversee this office's review activities. According to the ITIM
framework, boards should ensure projects are reviewed throughout their
life cycle. In addition, they must maintain ultimate responsibility for
and visibility into the activities of groups that carry out their
functions.[Footnote 28]
About Half of the Projects That Were Both Poorly Planned and Poorly
Performing Received Neither a Selection Review Nor an Oversight Review:
Six of the 11 projects that were identified as being both poorly
planned and poorly performing in 2007 did not receive a selection or an
oversight review by the departmental-level IRB. Funding requests for
fiscal year 2008 for these 6 projects accounted for about $3.7 billion
(see table 3).
Table 3: Department-Level Reviews Received by Poorly Planned and Poorly
Performing Projects:
Agency: Agriculture;
IT investment: Modernize & Innovate the Delivery of Agr. Systems
(MIDAS);
FY 2008 request: $151 million;
Review(s) received: Selection and oversight.
Agency: Defense;
IT investment: Defense Information System for Security;
FY 2008 request: $65 million;
Review(s) received: Selection.
Agency: Education;
IT investment: Common Services for Borrowers;
FY 2008 request: $15 million;
Review(s) received: Neither.
Agency: Homeland Security;
IT investment: DHS-Infrastructure;
FY 2008 request: $1,071 million;
Review(s) received: Neither.
Agency: Homeland Security;
IT investment: CBP-Secure Border Initiative (SBI) net;
FY 2008 request: $1,000 million;
Review(s) received: Neither.
Agency: Labor;
IT investment: New Core Financial Management System (NCFMS);
FY 2008 request: $12 million;
Review(s) received: Selection and oversight.
Agency: Nuclear Regulatory Commission;
IT investment: National Source Tracking System (NSTS);
FY 2008 request: $4 million;
Review(s) received: Neither.
Agency: Small Business Administration;
IT investment: Business Development Management Information System;
FY 2008 request: $0;
Review(s) received: Selection and oversight.
Agency: Treasury;
IT investment: Enterprise IT Infrastructure Optimization Initiative;
FY 2008 request: $1,638 million;
Review(s) received: Neither.
Agency: Treasury;
IT investment: Integrated Collection System;
FY 2008 request: $9 million;
Review(s) received: Neither.
Agency: Veterans Affairs;
IT investment: VistA Imaging;
FY 2008 request: $41 million;
Review(s) received: Selection.
Agency: Total;
IT investment: All 11 projects;
FY 2008 request: $4,006 million.
Agency: Total;
IT investment: Projects receiving neither review;
FY 2008 request: $3,737 million.
Source: GAO analysis of agency data.
[End of table]
Without consistent involvement of department-level IRBs in selecting
and overseeing projects that have been identified as poorly planned or
poorly performing, agencies incur the risk that these projects will not
improve, which could lead to potentially billions of federal taxpayer
dollars being wasted.
Conclusions:
Department-level investment review boards' involvement in selecting and
overseeing their agencies' IT projects is critical to ensuring that
these projects meet mission needs and that federal funds are not
wasted. To their credit, the 24 major federal agencies have established
guidance calling for department-level boards to perform project
selection and oversight reviews. However, department-level boards for
two agencies did not include representation from their business units
and therefore did not have assurance that the board included all of the
executives who are in the best position to make the full range of
decisions needed to enable the agency to carry out its mission most
effectively.
While having selection and oversight guidance is a good step, it is
only worthwhile if effectively implemented. The fact that many poorly-
planned or performing projects were not reviewed by department-level
boards is particularly alarming considering that they represent, in
total, about $6 billion in funding and that the Management Watch List
and High-Risk List were established specifically to draw management
attention to such projects. Until agencies ensure that their department-
level review boards are consistently involved in selecting and
overseeing these projects, they will continue to incur the risk that
the projects will not improve and that potentially billions of federal
taxpayer dollars will be wasted.
Recommendations for Executive Action:
To ensure that IT projects are effectively managed, we are making
recommendations to the agencies whose practices were not consistent
with sound management practices. Specifically, we recommend that:
* the Secretaries of Commerce and Labor ensure their department-level
review boards include business unit (i.e., mission) representation;
* the Chairman of the Nuclear Regulatory Commission direct the
Executive Director for Operations to define conditions for elevating
issues related to project selection and oversight to its department-
level IRB; and:
* the Secretary of Veterans Affairs define and implement
responsibilities for the department-level IRB to oversee projects in
operations and maintenance.
In addition, we are recommending that the Secretaries of the
Departments of Defense, Education, Homeland Security, Transportation,
Treasury, and Veterans Affairs, the Administrator for the National
Aeronautics and Space Administration, the Chairman of the Nuclear
Regulatory Commission, and the Administrator for the U.S. Agency for
International Development ensure that the projects that are identified
in this report as not having received departmental-IRB selection or
oversight reviews receive these reviews.
Agency Comments and Our Evaluation:
We sent a draft of this report to the 24 major agencies and received a
response from 20.[Footnote 29] Of these 20, 15 provided comments, and 5
stated they did not have any comments (we had not made any
recommendations to these agencies, which were the Department of Health
and Human Services, the Department of State, the Environmental
Protection Agency, the National Science Foundation, and the Office of
Personnel Management). Of the 15 agencies that provided comments, 11
generally agreed with our recommendations, and 1 (the Department of
Justice) did not. Three agencies (the Department of Housing and Urban
Development, the Department of the Interior, and the Social Security
Administration) provided views on various aspects of our report.
Several agencies also provided technical comments, which we
incorporated as appropriate.
The agencies' comments and our response are summarized below:
* In written comments on a draft of the report, the Department of
Commerce's Chief Information Officer, addressing our recommendation
that the department ensure that its department-level review board
include business unit (i.e. mission) representation, stated that the
department had modified the membership structure of its investment
review board to provide operating unit management with latitude in
identifying senior managers most able to provide effective
representation and, as a result had broadened its membership to include
chief financial officers from certain operating units as well as the
Deputy Director of the Bureau of the Census. The Department of
Commerce's comments are printed in appendix II.
* In written comments on a draft of the report, the Department of
Defense's Deputy Chief Information Officer concurred with our
recommendation to ensure that the Defense Information System for
Security receive an oversight review, stating that, going forward, it
will ensure that the project receives all required IRB reviews. The
department partially concurred with our recommendation to ensure its
Integrated Acquisition Environment Shared Services Provider-Past
Performance Information Retrieval System receive an oversight review,
stating, as indicated in the report, that the project is below the
threshold required for department-level IRB oversight. The department
stated, however, that the project will be brought before the
appropriate department-level IRB for compliance review if, and when it
meets the financial threshold. The department also provided technical
comments which we have incorporated as appropriate. The Department of
Defense's comments are printed in appendix III.
* In written comments on a draft of the report, the Department of
Education's Chief Information Officer, agreed with our recommendation
to ensure that the two projects we identified in the report as not
having received departmental-level IRB selection or oversight reviews
receive such reviews, stating that the IRB will review the investments,
render decisions as appropriate, and incorporate the results in the IT
portfolio currently under review. The department also noted that, while
the projects we reviewed were under the oversight of the Federal
Student Aid's Executive Leadership Team, they would be brought under
the department's oversight along with all other investments. The
department disagreed with the statement that the projects reviewed did
not receive a selection or oversight review, stating that they had been
selected and reviewed by the Federal Student Aid's Executive Leadership
Team. In our report, we have clarified the discussion of these reviews
by the Executive Leadership Team where appropriate. The Department of
Education's comments are reprinted in appendix IV.
* In written comments on a draft of this report, the Department of
Homeland Security's Director for Departmental GAO/OIG Liaison Office
agreed with the recommendation to conduct department-level reviews of
the three programs we reviewed and provided evidence of department
Acquisition Review Board reviews for these programs during fiscal year
2008. The department disagreed with the assertion that the department-
level review boards were not active in overseeing the three projects we
examined during our review and provided decision memoranda--three of
which we had not been provided before--as evidence of reviews by the
boards in place for 2007, the time period we considered. However, in
our report, we do not state that the department-level boards were not
active. Rather, we note that the department did not provide sufficient
evidence of department-level IRB reviews. We did not change our
assessments for the three projects because the additional documentation
received still did not provide sufficient evidence documenting the 2007
reviews.
The documentation we have seen from more recent reviews more completely
documents departmental-level IRB reviews and we have noted this in our
report. The department also provided technical comments. The
department's comments are reprinted in appendix V.
* In written comments on a draft of this report, the Acting Chief
Information Officer of the Department of Housing and Urban Development
stated that the department-level IRB will maintain its disciplined
process for program executives to participate in selecting and
overseeing projects. We did not make any recommendations to the
department. The Department of Housing and Urban Development's comments
are reprinted in appendix VI.
* In written comments on a draft of this report, the Department of the
Interior's Deputy Assistant Secretary for Budget and Business
Management agreed with our conclusions that consistent involvement of
department-level review boards in selecting and overseeing projects,
particularly poorly performing projects, is important in safeguarding
federal taxpayer dollars. The department also asked that the definition
of high-risk projects reflect the fact that some investments designated
as such are performing within acceptable thresholds but require
heightened awareness and oversight by investment review boards because
of their importance. To address this comment, we have added OMB's
criteria for designating projects as high-risk to our report
background. We did not make any recommendations to the Department of
the Interior. The Department of the Interior's comments are reprinted
in appendix VII.
* In written comments on a draft of this report, the Department of
Justice's Assistant Attorney General for Administration disagreed with
our recommendation that it ensure its department-level review board
include business unit representation and provided clarification on the
role and responsibilities of the Deputy Attorney General who chairs the
board and on the participation of component executives in the board's
decisionmaking process. Based on this clarification, we agree that the
board provides adequate business unit representation. We have noted
this change in our report and removed the related recommendation. In
its comments, the department also took issue with our use of the term
"poorly performing" to characterize the projects we reviewed. We are
not implying as the department states that these projects are "near
failing." We have clarified our use of the term in the report and, in
the case of the Sentinel project--which we have reviewed--acknowledged
progress made in managing the project. The Department of Justice's
comments are reprinted in appendix VIII.
* In written comments on a draft of this report, the Department of
Labor's Assistant Secretary for Administration and Management addressed
our recommendation to ensure that its department-level review board
include business unit representation by acknowledging that the board
does not include senior executives from business units and stating
that, while it believes the executives on the board effectively
represented the business interests of their respective organizations,
it will consider appropriate and efficient steps for including senior
executives from business units as part of the board's process. The
Department of Labor's comments are reprinted in appendix IX.
* In e-mail comments on a draft of this report, the Department of
Transportation's Director of Audit Relations addressed our
recommendation to ensure that the projects we identified as not having
received department-level IRB selection or oversight reviews receive
these reviews by stating that actions are underway to schedule a summer
IRB meeting to review the entire budget year 2011 portfolio of IT
investments, and that the Combined IT Infrastructure investment which
we reviewed is expected to be reviewed in detail.
* In written comments on a draft of this report, the Department of the
Treasury's Deputy Assistant Secretary for Information Systems and Chief
Information Officer addressed our recommendation to ensure that the
projects we identified as not having received department-level IRB
selection or oversight reviews receive these reviews by noting recent
efforts to reconstitute a department-level Executive Investment Review
Board, increase the oversight role of its Chief Information Officer
Council, and remediate weaknesses associated with the three projects we
reviewed. The Department of the Treasury's comments are reprinted in
appendix X.
* In written comments on a draft of this report, the Secretary of the
Department of Veterans Affairs concurred with our recommendations to
define and implement responsibilities for the department-level IRB to
oversee projects in operations and maintenance by noting that the
Programming and Long Term Issues Board will include operational
programs/projects in its program reviews for fiscal year 2010. The
department also concurred with our recommendation to ensure that the
project which we identified as not having received department-level IRB
oversight reviews receive these reviews and stated that it will address
actions to ensure this in its plan to address our recommendation. The
Department of Veterans Affairs' comments are reprinted in appendix XI.
* In written comments on a draft of this report, the National
Aeronautics and Space Administration's Associate Deputy Administrator
partially concurred with our recommendation that projects which are
identified in this report as not having received department-level IRB
selection or oversight reviews receive these reviews stating that the
departmental board will continue to review major IT investments that
are not highly specialized in nature (this includes two of the four
projects we reviewed), while another governing body will maintain
responsibility for ensuring the overall successful performance of
NASA's program portfolio, including the highly specialized IT
investments. We received information about the second governing body
after we sent our report to NASA for comment. During the comment
period, the agency also provided us additional documentation on the
projects we reviewed. After reviewing this documentation, we have
changed the reported reason column in table 1 from "department-level
board was not active (i.e., it had not yet been established)" to "NASA
did not provide evidence that a selection review had been performed by
the appropriate department-level IRB" for the three projects we
reviewed for selection. In addition, we changed the department-level
IRB review column in table 2 for the Integrated Financial Management
Improvement program from a "no" to a "yes." NASA's comments are
reprinted in appendix XII.
* In written comments on a draft of this report, the Nuclear Regulatory
Commission's Deputy Executive Director for Corporate Management, Office
of the Executive Director for Operations, agreed with our
recommendation to define conditions for elevating issues related to
project selection and oversight to its department-level IRB stating
that the commission will review and enhance the existing guidance for
project selection and oversight to ensure that its process is compliant
with the intent of the Clinger-Cohen Act. This will include updating
the Information Technology Business Council charter for project
oversight reviews to include any necessary changes to the process or
criteria for review by the Information Technology Senior Advisory
Council. The commission also agreed with our recommendation to ensure
that the National Source Tracking System which we identified as not
having received a selection or oversight review by the department-level
IRB receive such review. The Nuclear Regulatory Commission's comments
are reprinted in appendix XIII.
* In written comments on a draft of this report, the Commissioner of
the Social Security Administration asked that we remove the Information
Technology Operations Assurance project we reviewed from our report
because it is not a poorly planned or poorly performing project. During
the agency comment period, we informed the agency that we would be
removing the project from our sample, and, based on clarification
provided by the Associate Chief Information Officer that the project
reported a positive cost variance, agreed that it should not be
considered poorly performing. We did not make any recommendations to
the agency. The Social Security Administration's comments are reprinted
in appendix XIV.
* In e-mail comments on a draft of this report, the U.S. Agency for
International Development concurred with our recommendation to ensure
that the project which we identified as not having received a
department-level IRB oversight review receive this review. The agency
noted, however, that the review might not occur if the project is not
funded.
As agreed with your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies to other
interested congressional committees, the Director of the Office of
Management and Budget, and other interested parties. The report also
will be available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov]. Should you or your offices have questions on
matters discussed in this report, please contact me at (202) 512-9286
or at pownerd@gao.gov. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made key contributions to this report are listed
in appendix XV.
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
List of Requesters:
The Honorable Joseph I. Lieberman:
Chairman:
The Honorable Susan M. Collins:
Ranking Member:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Thomas R. Carper:
Chairman:
The Honorable John McCain:
Acting Ranking Member:
Subcommittee on Federal Financial Management, Government Information,
Federal Services, and International Security:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Tom Coburn, M.D.
United States Senate:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
Our objectives were to determine whether (1) federal departments/
agencies have guidance on the role of their department-level investment
review boards (IRB) in selecting and overseeing information technology
(IT) projects and (2) these boards are performing selection and
oversight reviews of poorly planned and performing projects.
To address the first objective, we reviewed the investment management
guidance (including policy documents and board charters) of each of 24
agencies listed in the Chief Financial Officers (CFO) Act of
1990[Footnote 30] (referred to in our report as "the 24 major
agencies"). In reviewing the guidance, we determined the role
department-level IRBs are expected to play in selecting and overseeing
IT projects, updating the findings from our 2004 governmentwide review
of agencies' use of key investment management practices.[Footnote 31]
We also reviewed the composition of the boards to determine whether
they included senior executives from both IT and business (i.e.,
mission) units, in accordance with the GAO IT Investment Management
framework which identifies the key practices for creating and
maintaining successful investment management processes. [Footnote 32]
For the second objective, we selected a sample of 48 IT projects that
were identified as being poorly planned according to the Office of
Management and Budget's Management Watch List [Footnote 33] or reported
as poorly performing on the High-Risk Lists[Footnote 34] or both. To
provide a governmentwide perspective, we attempted to select one
project from the 2007 Management Watch List and one project from the
High-Risk List with performance shortfalls during 2007 for each of the
24 major agencies. We focused on the high-risk projects with
performance shortfalls in the areas of cost and schedule since we had
reported in September 2007 that these were the most frequently reported
shortfalls.[Footnote 35] To obtain broader representation of agencies
with high-risk projects, we also selected three High-Risk projects that
had performance shortfalls in 2006. From these lists, we selected those
projects with the highest funding levels according to the fiscal year
2008 President's budget request. When an agency had a project on only
one of the lists (i.e., only the Management Watch List or High-Risk
List), we selected at least 2 projects from that list. For example, we
selected 2 high-risk projects with shortfalls for the Environmental
Protection Agency because the agency did not have any projects on the
Management Watch List for the time frame we considered.
Our selection process resulted in 26 projects from the Management Watch
List, totaling about $7.4 billion in the fiscal year 2008 budget
request, and 33 projects from the High-Risk List, totaling about $5.2
billion in the fiscal year 2008 budget request. Eleven of these
projects, totaling about $4 billion, were on both lists. The Department
of Energy and the National Science Foundation did not have any projects
on the Management Watch List or on the High-Risk List with shortfalls
and, therefore, we did not select any projects from these agencies. We
removed two Management Watch List projects and five high-risk projects
from our initial sample after sending the draft report to agency
comment because we determined after further review and discussion with
agencies that these projects had not been on the Management Watch List
during 2007 or reported negative cost or schedule variances exceeding
10 percent between December 2006 and December 2007. This brought our
sample of Management Watch List projects to 24 projects, totaling about
$7.3 billion in the fiscal year 2008 budget request and 28 high-risk
projects totaling about $4.7 billion in the fiscal year 2008 budget
request and the number of projects on both lists to 11 projects
totaling $4 billion in the fiscal year 2008 budget request.
To determine whether department-level IRBs were performing selection
and oversight reviews of poorly planned and performing projects, we
requested evidence of board reviews for the 48 projects in our sample
during the time they were either on the Management Watch List or High-
Risk List. We analyzed the documentation obtained, and, when reviews
had not been performed, we followed up with agencies to determine why
the required reviews were not performed. For the oversight reviews, we
determined whether project cost, benefit, schedule and risk data had
been provided to the board, but we did not assess the reliability of
this information.
We conducted this performance audit from January 2008 to June 2009 in
Washington, D.C., in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Commerce:
United States Department Of Commerce:
Chief Information Officer:
Washington, DC 20230:
June 22, 2009:
Ms. Sabine R. Paul:
Assistant Director, Information Technology Management Issues:
Government Accountability Office:
441 G Street. N.W.
Washington, DC 20548:
Dear Ms. Paul:
Thank you for the opportunity to review the draft report, "Information
Technology: Federal Agencies Need to Strengthen Investment Board
Oversight of Poorly Planned and Poorly Performing Projects, GAO-04-
566." This draft report provides an informative assessment of
procedures used across the Federal Government to support department-
level investment review boards. Specific comments on the content of the
draft report are enclosed.
Sincerely,
Signed by:
Suzanne Hilding:
Enclosure:
[End of letter]
Enclosure: Department of Commerce Comments on the Government
Accountability Office's Draft Report "Information Technology: Federal
Agencies Need to Strengthen Investment Board Oversight of Poorly
Planned and Performing Projects, GAO-09-566"
On pages 12 and 13 of the draft report, the Government Accountability
Office (GAO) identifies the Department of Commerce (DOC) as one of
three agencies with investment review boards that do not include senior
executives from their business units. For purposes of clarity, it
should be noted that membership on DOC's investment review board does
include representation from across the Department.
At its inception, the investment review board included members from the
operating units, which have principal responsibility for implementing
mission-related programs, as well as Departmental offices with
oversight responsibility for information technology (IT) and various
administrative functions. It was--and still is--co-chaired by the
Department's Chief Information Officer (CIO) and the Chief Financial
Officer and Assistant Secretary for Administration (CFO/ASA). finder
its initial charter, chief information officers served as their
operating units representative on the board. The largest operating
units held permanent positions while smaller operating units held term
appointments that changed on a rotating basis. Program officials and
other individuals were included in board activities as needed to
appropriately inform the discussion of any agenda item.
During the course of GAO's review, .DOC modified the membership
structure of its investment review board to provide operating unit
management with latitude in identifying senior managers most able to
provide effective representation. As a result, operating unit
membership has broadened to include chief financial officers from
certain operating units and the Deputy Director of the Bureau of the
Census. The board is still co-chaired by the CIO and CFO/ASA, and
includes active participation by individuals from their organizations
with extensive experience with an array of administrative functions and
IT. The board retains the ability to obtain advice as needed from
individuals with other program, technical, or administrative expertise.
We believe that this interdisciplinary approach complies with GAO's
overall recommendation for improving departmental review boards and
oversight processes.
[End of section]
Appendix III: Comments from the Department of Defense:
Department Of Defense:
Chief Information Officer:
6000 Defense Pentagon:
Washington, DC 20301-6000:
June 22, 2009:
Mr. David A. Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548:
Dear Mr. Powner:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-09-566, "Information Technology: Federal Agencies Need to
Strengthen Investment Board Oversight of Poorly Planned and Performing
Projects, dated May 27, 2009 (GAO Code 310862).
Enclosed are the Department's responses to the Draft GAO Report GAO-09-
566. The Department concurs with the recommendation for the Defense
Information System for Security (DISS) and partially concurs with the
recommendation for the Integrated Acquisition Environment (IAE) Shared
Services Provider (SSP) Initiative. Supporting justification is
enclosed.
The Department welcomes GAO's insights and recommendations, and is
committed to ensuring that all IT projects receive the appropriate
selection and oversight reviews. Thank you for the opportunity to
comment on the Draft GAO Report.
Sincerely,
Signed by:
David M. Wennergren:
DoD Deputy Chief Information Officer:
Enclosure: As stated:
[End of letter]
GAO Draft Report Dated May 27, 2009:
GAO-09-566 (GAO Code 310862):
"Information Technology: Federal Agencies Need To Strengthen Investment
Board Oversight Of Poorly Planned And Performing Projects:
Department Of Defense Comments To The GAO Recommendation:
Recommendation: The GAO recommended that the Secretary of the
Department of Defense ensure that the projects which are identified in
this report as not having received departmental Investment Review Board
selection or oversight reviews receive these reviews.
DOD Response: Partially Concur. The DoD concurs with the Defense
Information System for security and partially concurs with the
Integrated Acquisition Environment (IAE) Shared Services Provider -
Past Performance Information Retrieval System (PPIRS). Following is the
explanation:
* Defense Information System for Security (DISS): Concur. The
Department is committed to ensuring appropriate information technology
selection and oversight reviews are conducted. It is important to note
that the specific system identified in this report, DISS, as not having
undergone an investment review board (IRB) oversight review was denied
FY 2008 modernization funding requested during its 2007 selection
review and therefore did not require a subsequent IRB oversight review.
Since that time, as noted in the report, DISS went through a
rebaselining process, during which, the overall Joint Security
Clearance Reform effort, of which DISS is a part, was overseen by
Department of Defense, Director of National Intelligence, Office of
Management and Budget, and the Office of Personnel Management senior
leadership to include the supporting Information Technology elements.
Following rebaselining, DISS received another review by the
departmental-IRB and approval for modernization funding for FY 2009.
Going forward, the Department will ensure that DISS continues to
undergo all required reviews.
* Integrated Acquisition Environment (IAE) Shared Services Provider -
Past Performance Information Retrieval System (PPIRS): Partially
Concur. IAE is a federal-wide E-Government (E-Gov) Initiative that is
managed by the General Services Administration, of which PPIRS is just
one of multiple systems. The Department's Business Transformation
Agency (BTA) manages the PPIRS program as a Shared Service Provider
(SSP) on behalf of the federal government.
OMB guidance issued to all federal agencies dated February 26, 2007,
required all E-Gov and Line of Business Initiatives be included on the
OMB High Risk List (HRL), due to the high visibility and government-
wide impact of these initiatives. As a result, PPIRS was included on
the HRL as an IAE Shared Service Provider starting in Q I FY 2007.
As indicated in the report, the PPIRS modernization budget is
significantly less than the threshold that requires DoD-level IRB
oversight. However, PPIRS does receive BTA level quarterly program
reviews to ensure compliance with the Department's investment review
requirements. If, and when PPIRS meets the financial threshold, it will
be brought before the appropriate departmental-IRB for compliance
review.
Recommend that Table 2 (p. 21/GAO Draft Report) be updated to reflect
the "Integrated Acquisition Environment (IAE) Shared Services
Provider - Past Performance Information Retrieval System (PPIRS)" vice
the "Integrated Acquisition Environment (IAE) Shared Services
Provider."
[End of section]
Appendix IV: Comments from the Department of Education:
United States Department Of Education:
Office Of The Chief Information Officer:
The Chief Information Officer:
400 Maryland Ave., S.W.,
Washington, D.C. 20202-4580:
[hyperlink, http://www.ed.gov]
"Our mission is to ensure equal access to education and to promote
educational excellence throughout the Nation."
June 16, 2009:
Mr. David A. Powner:
Director:
Information Technology Management Issues:
Government Accountability Office:
Washington, DC 20548:
Dear Mr. Powner:
I am writing to respond to recommendations made in the Government
Accountability Office (GAO) draft report "Federal Agencies Need to
Strengthen Investment Board Oversight of Poorly Planned and Performing
Projects" (GAO-09-566). This report focused on the existence and
operation of structures and processes that support Investment Review
Board (IRB) activities, specifically those related to the selection and
oversight of information technology (IT) investments.
The Department appreciates the opportunity to review and respond to the
draft report and recognizes that it is critical to plan and manage IT
projects effectively to ensure that limited resources are invested
appropriately. The two projects reviewed are under the oversight of the
Federal Student Aid (FSA) Executive Leadership Team (ELT). The ELT
reviews and provides oversight for FSA-managed investments before
delivering the results to the Department's IRB.
The Department has an operating IRB that meets as needed and a working
group of executives -the Planning and Investment Review Working Group
(PIRWG) chartered by the IRB that meets monthly to provide oversight
and review of investments. In past years, the PIRWG made investment
recommendations to the IRB, and the FSA portfolio was added to the
Department's IT budget submission, based on decisions by the FSA ELT.
The Department is in agreement with your finding that the cited
investments received neither a selection review nor an oversight review
by the Department's IRB. However, the Department does not agree with
the statement that they did not receive a selection or oversight review
because the cited investments were selected and reviewed by the FSA
ELT.
1 am pleased to note that you found our IRB and investment review
processes otherwise appropriate. Going forward I am leveraging our
current IT Investment Management structure and processes to bring all
investments under Department oversight. Specifically, this year (unlike
past years) all FSA investments are being reviewed by the Department's
PIRWG consistent with the plan and schedule for all other investments.
All FSA investments will be included in a single portfolio
recommendation to the Department's IRB. There were no recommendations
directed exclusively to Education, but regarding the recommendation
that applies:
Recommendation: In addition, we are recommending that the Secretaries
of the Departments of Defense, Education, Homeland Security,
Transportation, Treasury, and Veterans Affairs, and the General
Services Administration, National Aeronautics and Space Administration,
Nuclear Regulatory Commission, and U.S. Agency for International
Development ensure that the projects which are identified in this
report as not having received departmental-IRB selection or oversight
reviews receive these reviews.
Response:
The IRB will review the investments, render decisions as appropriate
and incorporate the results in the IT portfolio currently under review.
Again, I appreciate the opportunity to respond to the GAO report. If
you or your staff members have any questions regarding our response,
please contact me at (202) 401-0896 or Danny.Harris@ed.gov.
Sincerely,
Signed by:
Danny A. Harris, Ph.D.
[End of section]
Appendix V: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
June 22, 2009:
Mr. David A. Powner:
Director:
Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Powner:
Re: GAO-09-566 Information Technology: Federal Agencies Need to
Strengthen Investment Board Oversight of Poorly Planned and Performing
Projects (GAO Job Code 310862):
The Department of Homeland Security (DHS) appreciates the opportunity
to review and comment on the U.S. Government Accountability Office's
(GAO's) draft report referenced above. The GAO came to several
conclusions with regard to the status of executive oversight at DHS.
The Department agrees with some of these assertions and disagrees with
others; we appreciate the opportunity to clarify.
DHS disagrees with the assertion that the Department-level review
boards were not active in overseeing the three identified programs
during the period GAO reviewed. The Department is forwarding Investment
Decision Memoranda for the Secure Border Initiative Technology Program
(SBInet), US-VISIT and the DHS IT Infrastructure Transformation Program
(ITP) supporting the actions of its Departmental Executive Review
Boards in place at that time -the Investment Review Board and the Joint
Requirements Council. In addition, the Department has explained to GAO
examiners that each of the three programs underwent Department-level
review in 2007 via the Program Review Boards led by the Deputy
Secretary via the Programming, Planning, Budget and Execution (PPB&E)
process.
The Department would also like to note that the DHS IT Infrastructure
Transformation Program completed the enterprise development of network,
email and data center platforms in 2008 and the program office was
stood down. Components are completing their migrations to the new
platforms under the supervision of the Chief Information Officer and
the CIO Council. The ITP only breached performance targets as a result
of the impact of Hurricane Katrina on Gulf Coast operations; it has not
requested funds over its approved baseline.
In addition, page 26 of the draft report indicates that the DHS ITP
received neither a selection review nor an oversight review. The ITP
received selection and oversight reviews prior to the period of GAO's
study and received a selection and oversight review by the Department's
Joint Requirements Council on April 26, 2006.
Recommendations:
The Department agrees with the recommendation to conduct Department-
level review of the three DHS programs and has provided evidence to GAO
of the conduct of several DHS Acquisition Review Board reviews for
these programs during FY 2008.
Sincerely,
Signed by:
Jacqueline L. Lacasse, for:
Jerald E. Levine:
Director:
Departmental GAO/OIG Liaison Office:
[End of section]
Appendix VI: Comments from the Department of Housing and Urban
Development:
U.S. Department Of Housing And Urban Development:
Chief Information Officer:
Washington, DC 20410-1000:
June 17, 2009:
Mr. David A. Powner:
Director:
Information Technology Management Issues:
Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Powner:
Thank you for the opportunity to comment on the Government
Accountability Office (GAO) draft report, entitled "Information
Technology Federal Agencies Need to Strengthen Investment Board
Oversight of Poorly Planned and Performing Projects" (GAO-09-566).
The Department of Housing and Urban Development (HUD) reviewed the
draft report. I am pleased that GAO issued no recommendations for HUD.
The Department is meeting GAO's standards by (1) establishing guidance
on the role of HUD's department-level Investment Review Boards (IRBs)
in selecting and overseeing IT projects, and (2) performing reviews on
any poorly performing projects.
HUD's department-level IRB will maintain this disciplined process for
Program executives to participate in selecting and overseeing projects,
as endorsed by GAO and the Office of Management and Budget (OMB).
If you have any questions or require additional information, please
contact Stephen A. Hill, Acting Director, Investments, Strategy, Policy
and Management at (202) 402-8346.
Sincerely,
Signed by:
Lynn Allen:
Acting Chief Information Officer:
[End of section]
Appendix VII: Comments from the Department of the Interior:
United States Department of the Interior:
Office Of The Secretary:
Washington, DC 20240:
June 17, 2009:
Sabine Paul:
Assistant Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, D.C 20548:
Dear Ms. Paul:
Thank you for providing the Department of the Interior the opportunity
to review and comment on the draft Government Accountability Office
Report entitled "Information Technology: Federal Agencies Need to
Strengthen Investment Board Oversight of Poorly Planned and Performing
Projects," (GAO-09-566). While there were no findings or
recommendations for the Department of the Interior, we appreciate
participating in this assessment, as we work to continuously improve
and mature our information technology investment management practices.
We agree with GAO's conclusions and overall recommendation that
consistent involvement of department-level review boards in selecting
and overseeing projects, particularly poorly performing projects, is
important in safeguarding federal taxpayer dollars.
In reviewing the draft report, we would like to point out one statement
that needs clarification. Page six, paragraph one of the report states
that the "High-Risk List includes projects that are performing
poorly... (i.e., projects experiencing performance shortfalls, meaning
that they do not meet one or more of four performance evaluation
criteria, including cost or schedule variances exceeding 10 percent)."
While poorly performing projects are a key focus of the High-Risk List,
according to the Office of Management and Budget, "projects on the High
Risk List are those requiring special attention from the highest level
of agency management, but aren't projects necessarily 'at risk' of
failure" (see [hyperlink,
http://www.whitehouse.aov/omb/oubpress/2008/102308_vueit.htm]). An
example of this at Interior is our Geospatial One-Stop investment. This
investment is on the High-Risk List because it is a federal-wide
initiative of high importance and visibility, but is not in danger of
failing. It receives regular oversight by our investment review boards
and is performing within acceptable cost and schedule variances.
We believe that statements in the report that define the High-Risk List
should reflect the fact that some investments included are performing
within acceptable tolerances, but require heightened awareness and
oversight by investment review boards because of their importance.
If you have any questions, or need additional information, please
contact Sylvia Burns, Office of the Chief Information Officer,
Portfolio Management Division, at svlvia_burns@ios.doi.gov or (202) 208-
4109.
Sincerely,
Signed by:
Illegible, for:
Pamela K. Haze:
Deputy Assistant Secretor} for Budget and Business Management:
[End of section]
Appendix VIII: Comments from the Department of Justice:
U.S. Department of Justice:
Washington, DC 20530:
June 22, 2009:
Mr. David A. Powner:
Director, Information Technology Management:
United States Government Accountability Office:
Washington, DC 20548:
Dear Mr. Powner:
The Department of Justice has reviewed the Government Accountability
Office's (GAO) draft report, "Information Technology: Federal Agencies
Need to Strengthen Investment Board Oversight of Poorly Planned and
Performing Projects," (GAO-09-566) and provides the
following comments on the report's conclusions, findings and
recommendations. The Department concurs with most of what the GAO
found. However, we take issue with the following.
The Recommendation:
The Department disagrees with the auditors' conclusion that led to the
following recommendation.
The Secretaries of Commerce, Labor, and Justice ensure their department-
level review boards include business unit (i.e., mission)
representation.
This recommendation appears to be based on a GAO misconception that
Department business unit representatives do not participate directly in
the decisions made by the Department's investment Review Board (DIRB).
The GAO reached this conclusion, it says, from its observation that
business unit representatives are not among the standing members of the
DIRB and, consequently, they do not vote on DIRB matters. In its draft
report, the GAO notes that it reviewed the DIRB Charter and reports
that summarized what transpired at meetings of the DIRB. Although much
of what the GAO observed is accurate, the GAO report is silent on many
facts that, had they been considered, show that the DIRB includes
business from the Department. Moreover, these people play key roles in
decisions by the DIRB as the following demonstrates:
a. First, included among the DIRB membership is the most senior
business manager at the Department--the Deputy Attorney General. Also,
he is not just a member: the Deputy Attorney General is the chairman of
the DIRB. In addition to his direct participation, he exercises
significant authority in DIRB decision making. The Deputy Attorney
General is second only to the Attorney General in "formulating and
implementing Department policies and programs and in providing overall
supervision and direction to all organizational units of the
Department." 28 C.F.R. Section 0, 15(b).
b. Second, the DIRB is empowered to include business representatives in
its deliberations and the DIRB exercises that power, facts not
mentioned in the draft report. The DIRB Charter authorizes the DIRB to
invite into its deliberations executives from Department business units
responsible for information technology (IT) projects under DIRB review.
Furthermore, the DIRB periodically invites executives from other
Department components for the purposes of ensuring transparency and a
comprehensive understanding of the IT activity. When they attend,
executives participate fully: they express concerns and raise issues,
share their views on user customer expectations regarding the
investment under review, comment on any project risks, and help
evaluate the effectiveness of the program management team. In summary,
these executives fully participate in the DIRB deliberations, and their
views substantially influence a project's assessment. The fact that
only official DIRB members vote does not rule out consideration of the
views of these business representatives.
The vote by the DRB is advisory; the Deputy Attorney General, as the
Chairman of the DIRB, has the final authority to approve or reject the
Board' recommendation(s), and to dictate changes, if he deems any are
necessary.
c. Third, the Department believes it would be unwise to extend DIRB
voting authority to the business unit representatives. The Department
chose to limit voting authority to those DIRB members identified in the
Charter. The Department believes that extending a vote to a
representative from the business unit sponsoring a project would create
the appearance of bias, if not permitting that person to influence the
Board's review. The issue of voting rights was evaluated when the DIRB
was organized and chartered. The role of business unit executives was
limited for the reasons already explained.
GAO Inclusion of List with Unnecessary Inflammatory Subtitle:
Finally, the Department believes wording chosen by the GAO unfairly
mischaracterized Department IT projects. On Table 2 (at page 21) the
GAO lists a number of IT projects under the heading "Poorly performing
project: High risk project with performance short falls in 2006
and 2007." The term "Poorly performing" is inherently negative and,
with respect to the projects of the Department of Justice, improperly
used. These projects are not near failing. Nor are issues from 2006, as
shown in your chart on page 22, representative of the current status of
these projects.
For example, the GAO included on its list the Sentinel project, an
undertaking by the Federal Bureau of Investigation (FBI). The Office of
Management and Budget (OMB) created the High Risk List. In one of its
publications, the OMB cautions "Projects on the High Risk List are
those requiring special attention from the highest level of agency
management, but are not necessarily 'at risk' of failure. Nevertheless,
the GAO equates projects on the OMB High Risk list with "performance
shortfalls." In fact, Sentinel was placed on the OMB High Risk List
because of its high cost and importance to the FBI mission. The GAO
should know that the Sentinel project has achieved operational
successes. In its most recent audit of Sentinel (see, GAO Report No. 08-
1014), the GAO applauded Sentinel for implementing five key methods for
acquiring commercial information technology solutions and the GAO went
so far as to suggest that the Department adopt these methods as
standard practices.
Similarly, the Unified Financial Management system is moving ahead as
expected. The Drug Enforcement Administration became the second
Department component to fully implement the new system when it "went
live" worldwide in January 2009. The Federal Bureau of Investigation
(FBI), has implemented the Contract Writing Tool. The Bureau of
Alcohol, Tobacco, Firearms, and Explosives (ATF) recently successfully
completed Phase 1 of its implementation of the system. The Federal
Bureau of Prisons (BOP) is on schedule executing a regional rollout of
UFMS Acquisitions Functionality, with two out of four groups going live
in June and the remainder scheduled to complete in July. For the BOP
implementation, the UFMS program was able to react rapidly and provide
an earlier-than-planned implementation when BOP found that their legacy
application's failure was imminent.
The GAO should modify the table heading, to more properly convey the
high visibility and importance of these projects rather than using the
current terms which connote pending failure.
The Department appreciates this opportunity to comment on the draft
report prepared by the GAO.
Should you have any questions regarding this topic, please do not
hesitate to contact Richard Theis, DOJ Audit Liaison, on 202-514-0469.
Sincerely,
Signed by:
Michael H. Allen, for:
Lee J. Lofthus:
Assistant Attorney General for Administration:
[End of section]
Appendix IX: Comments from the Department of Labor:
U.S. Department of Labor:
Office of the Assistant Secretary for Administration and Management:
Washington, DC 20210:
David A. Powner:
Director:
Office of Information Technology Management Issues:
Government Accountability Office:
441 G Street, NW:
Washington, D.C. 20548:
Dear Mr. Powner:
Thank you for the opportunity to review and comment on the Government
Accountability Office's (GAO) draft report titled: Information
Technology: Federal Agencies Need to Strengthen Investment Board
Oversight of Poorly Planned and Performing Projects (GAO-09-566).
GAO correctly reflects the Department's view that its information
technology investment review board--in Labor referred to as the
Technical Review Board (TRB)--is comprised of senior IT and
administrative executives from each of the Department's agencies,
bureaus and offices who have in-depth, detailed and expert knowledge of
their units' missions and business objectives.
The draft report observes, however, that Labor's staffing for its TRB
does not in all respects comport with GAO's previous government-wide
recommendation that IT investment boards should also include executives
from the business units. In the draft report, GAO reasons that
"...IT and administrative executives responsible for mission support
functions do not constitute Sufficient business representation because,
by virtue of their responsibilities, they are not in the best position
to make business decisions."
We acknowledge that Labor's TRB does not include senior executives from
business units. however, as stated during the review, it is our
experience that the executives on Labor's Board perform very
effectively in representing the business interests of their respective
organizations.
To the extent that the draft report is intended to associate "poorly
planned and poorly performing" IT projects with management oversight,
the report should acknowledge that the Department has a very robust IT
investment review process that includes:
* Earned Value Management reporting for major IT development programs,
which includes monthly reporting that highlights cost and schedule
variances;
* Quarterly IT program reviews (currently 62 programs are reviewed)
that monitor cost, schedule, and performance, as well as enterprise
architecture and IT security requirements; and;
* Corrective Action Plan requirement for IT investments that approach
or exceed the ten percent variance that specifics how the program
manager will correct variances.
In our experience, these management controls provided effective,
regular monitoring of the performance of IT investments against planned
progress and expectations, as well as timely warning of when corrective
action is needed.
With the forgoing in mind, the Department will consider appropriate and
efficient steps for including senior executives from business units as
part of the TRB process.
Should you, or a member of your staff, have any questions, please
contact Tom Wiesner, Deputy Chief Information Officer, at (202) 693-
4200 or at Wiesner.Thomas@dol.gov.
Sincerely,
Signed by:
T. Michael Kerr:
Assistant Secretary for Administration and Management:
[End of section]
Appendix X: Comments from the Department of the Treasury:
Department Of The Treasury:
Washington, Dc 20220:
June 17, 2009:
David A. Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street N.W.
Washington, D.C. 20515:
Dear Mr. Powner:
Thank you for the opportunity to comment on proposed report GAO-09-566,
Information Technology - Federal Agencies Need to Strengthen Investment
Board Oversight of Poorly Planned and Performing Projects before
Finalizing.
In January 2008, recognizing the need to strengthen executive
engagement and oversight of the IT portfolio, the Department formally
re-constituted a Department-level Executive Investment Review Board (E-
Board) chaired by the Deputy Secretary and the Assistant Secretary for
Management/CFO. We also modified the CIO Council charter to increase
its oversight role. The E-Board met in February, June, and November of
2008, with pre-meetings by the CIO Council, and reviewed the FY 2009 IT
portfolio as well as proposed FY 2010 investments. The Board also
focused on investments on the OMB Management Watch List and OMB High
Risk List, as well as investments with notable cost and schedule
variances.
I am pleased to note that two of the three Treasury Department
investments GAO highlights, the Treasury Automated Auction Processing
System (TAAPS) and the Integrated Collection System (ICS), which were
placed on the OMB Management Watch List in September 2007 due to cost
and schedule variances in the Exhibit 300's, were remediated
successfully in early 2008 and removed from the list. As a result of
the reaffirmed mission/business need and the Department's confirmation
that the investments were within 10% of cost and schedule goals for all
developmental activities, both projects were considered worthy of
continuation and selected for inclusion in the Treasury IT portfolio.
The Department is currently reassessing how it populates Exhibit 300's
to ensure that this data is accurately presented and reported.
The third project highlighted in the GAO report, the Enterprise IT
Infrastructure Optimization Project (EITIO), was one of only four major
IT investments (of 65) remaining on the OMB Management Watch List by
the end of FY 2008. EITIO is an OMB-mandated consolidation of all
Treasury IT infrastructure projects. The amalgamated approach produces
a composite cost and schedule variance that is not a valid indicator of
planning or management on individual IT projects. Since IT
infrastructure is integral to the successful performance of the
Treasury mission, EITIO was deemed worthy of continuation and selected
for inclusion in the Treasury IT portfolio.
Finally, to strengthen oversight and transparency of federal IT
investments, we note that OMB will launch the IT Dashboard website at
the end of June 2009. The IT Dashboard will provide agencies and the
public the ability to view the details of federal IT investments online
and to track their progress over time. This tool will further ensure
that the management of IT investments remains at the forefront of
agency priorities.
Thank you for considering our comments and additional information. If
you have any questions, please contact Ms. Diane Litman, Associate
Chief Information Officer for Planning and Management, at 202-622-7704.
Sincerely,
Signed by:
Michael D. Duffy:
Deputy Assistant Secretary for Information Systems and Chief
Information Officer:
[End of section]
Appendix XI: Comments from the Department of Veterans Affairs:
The Secretary Of Veterans Affairs:
Washington:
June 16, 2009:
Mr. David A. Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Powner:
The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, INFORMATION TECHNOLOGY.•
Federal Agencies Need to Strengthen Investment Board Oversight of
Poorly Planned and Performing Projects (GAO-09-566) and concurs with
GAO's recommendations.
The enclosure specifically addresses each of GAO's recommendations to
the Department. VA appreciates the opportunity to comment on your draft
report.
Sincerely,
Signed by:
Eric K. Shinseki:
Enclosure:
[End of letter]
Enclosure: Department Of Veterans Affairs (VA) Comments To GAO Draft
Report, "Information Technology: Federal Agencies Need to Strengthen
Investment Board Oversight of Poorly Planned and Performing Projects"
(GAO-09-566):
GAO Recommendations:
Recommendation 1: The Secretary of Veterans Affairs define and
implement responsibilities for the department-level IRB to oversee
projects in operations and maintenance.
Response: Concur. VA now has processes in place to review all
investments. While the information technology leadership board is the
primary/senior executive information technology investment review board
(IRB), the programming and long term issues board is responsible for
oversight and assessment of major information technology investments
(program reviews). The intent is to include operational
programs/projects in the program reviews for fiscal year 2010.
Recommendation 2: The Secretary of Veterans Affairs ensure that the
projects which are identified in this report as not having received
departmental-IRB selection or oversight reviews receive these reviews.
Response: Concur. The Department will provide, in its 60 day letter, a
description of the actions it will take to implement this
recommendation.
[End of section]
Appendix XII: Comments from the National Aeronautics and Space
Administration:
National Aeronautics and Space Administration:
Office of the Administrator:
Washington, DC 20546-0001:
June 16, 2009:
Mr. David A. Powner:
Director, Information Technology Management Issues:
United States Government Accountability Office:
Washington, DC 20548:
Dear Mr. Powner:
Thank you for the opportunity to review and comment on your draft
report entitled, "Federal Agencies Need to Strengthen Investment Board
Oversight of Poorly Planned and Performing Projects" (GAO-09-566).
In the draft report, GAO makes a total of four recommendations intended
to ensure that information technology (IT) projects are effectively
managed. Of the four recommendations communicated in the report, one is
addressed to NASA, specifically:
Recommendation 4: We are recommending that the Secretaries of the
Departments of Defense, Education, Homeland Security, Transportation,
Treasury, and Veterans Affairs, and the General Services
Administration, National Aeronautics and Space Administration, Nuclear
Regulatory Commission, and U.S. Agency for International Development
ensure that the projects which are identified in this report as not
having received departmental-Investment Review Board selection or
oversight reviews receive these reviews.
Response: Partially concur. The NASA Information Technology Strategy
and Investment Board (IT SIB) was chartered on March 11, 2008, to
review and approve all significant IT investments that are not highly
specialized in nature. Highly specialized IT is defined as IT that is
an embedded component of a flight system, experiment, simulator. ground
support equipment, or mission control center. Two of the four projects
identified are subject to the review of the NASA IT SIB: (1) NASA's
Office Automation, IT Infrastructure, Telecommunications (OAIIT), and
(2) NASA's Integrated Enterprise Management Program (TEMP) - Core
Financial. These steady state investments, along with NASA's other
major IT investments, were presented to the IT SIB in June 2008 for
confirmation to continue. The NASA IT SIB will conduct a review of
major investments again in June 2009, as part of the Planning,
Programming, Budget, and Execution process. Prior to 2008, review and
oversight of OAIIT was conducted by the NASA Chief Information Office's
(CIO) Board in concert with periodic face-to-face meetings of the
Board; oversight of TEMP was provided by the Program Management Council
(PMC) and transferred to the Operations Management Council in 2007.
The two other projects identified in the report are considered highly
specialized IT: (1) JSC Software Development/Integration Laboratory,
and (2) Earth Observing System Data Information System (EOSDIS). Life-
cycle management of highly specialized IT projects is in accordance
with NASA Procedural Requirement (NPR) 7120.5, "Space Flight Program
and Project Management Requirements" or NPR 7120.8, "NASA Research and
Technology Program and Project Management Requirements" and is subject
to applicable governance structures there under. The EOSDIS program is
subject to the OMB Program Assessment Rating Tool (PART) and is
specifically governed by the NASA PMC upon referral from Earth Science
Flight Program Reviews. The JSC Software Development/integration
Laboratory is a critical capability funded by the Space Shuttle
Program, Space Station Program, and Constellation Program, which are
subject to OMB PART, as well as oversight by the NASA PMC, upon
referral by the Program Control Boards. Therefore, review of these
projects by the IT SIB is unnecessary, as well as inconsistent with
NASA policy and procedures.
In summary, the NASA IT SIB will continue to review major IT
investments that are not highly specialized in nature, while the NASA
PMC will maintain responsibility for ensuring the overall successful
performance of NASA's program portfolio, including the highly
specialized IT investments there under.
My point of contact for this matter is Gary Cox, Associate CIO for
Policy and Investment. He maybe contacted by e-mail at
Gary.Cox-l@nasa.gov or by telephone at (202) 358-0413.
Sincerely,
Signed by:
Charles H. Scales:
Associate Deputy Administrator:
[End of section]
Appendix XIII: Comments from the Nuclear Regulatory Commission:
United States:
Nuclear Regulatory Commission:
Washington, DC 20555-0001:
June 18, 2009:
Mr. David A. Powner, Director:
Information Technology Management Issues:
U.S. Government Accountability Office:
Washington, D.C. 20548:
Dear Mr. Powner:
Thank you for the opportunity to provide comments on the Government
Accountability Office (GAO) draft report titled: "Information
Technology: Federal Agencies Need to Strengthen Investment Board
Oversight of Poorly Planned and Performing Projects (GAO-09-566)."
The study found that two U.S. Nuclear Regulatory Commission (NRC)
Information Technology (IT) projects, National Source Tracking System
(NSTS) and Infrastructure Services and Support, did not receive a
selection review by the department-level Investment Review Board. The
report also found that the NSTS had not received an oversight review by
the department-level Investment Review Board. The study considered the
Information Technology Senior Advisory Council (ITSAC) as NRC's
department-level Investment Review Board and the Information Technology
Business Council (ITBC) as the lower-level review board. However, in
practice, both the ITSAC and the ITBC are comprised of executives from
the NRC's major offices and both function as and should be considered
department-level investment review boards.
The NRC agrees with the findings and the recommendations in the report.
The NRC will review and enhance the existing guidance for project
selection and oversight to ensure that the agency process is compliant
with the intent of the Clinger-Cohen Act. This will include
reauthorizing on a periodic basis the role of the ITBC as the agency-
level board responsible for project selection and oversight reviews.
Additionally, the NRC will update the ITBC charter for project
oversight reviews to include any necessary changes to the process or
criteria for review by the ITSAC. At a minimum, the updated process
will require a project oversight review at key checkpoints identified
during the initial business case approval process and will also require
further review by the ITSAC if the project meets specified criteria.
Finally, as GAO recommended, the NSTS will have an oversight review by
the ITSAC.
Please change the language in the "Recommendations' section on page 27
of the report as follows. The recommendation that reads 'the
Commissioner of the Nuclear Regulatory Commission define conditions for
elevating issues related to project selection and oversight to its
department-level;' should be changed to "the Executive Director for
Operations of the Nuclear Regulatory Commission define conditions for
elevating issues related to project selection and oversight to its
department-level;" In addition the 3rd recommendation should be
addressed to the Chairman of the Nuclear Regulatory Commission.
Sincerely,
Signed by:
Illegible, for:
Darren B. Ash:
Deputy Executive Director for Corporate Management:
Office of the Executive Director for Operations:
[End of section]
Appendix XIV: Comments from the Social Security Administration:
Social Security:
The Commissioner:
Social Security Administration:
Baltimore, MD 21235-0001:
June 23, 2009:
Mr. David A. Powner:
Director, Information Technology Management Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, D.C. 20548:
Dear Mr. Powner:
Thank you for the opportunity to review and comment on the Government
Accountability Office (GAO) draft report, "Information Technology:
Federal Agencies Need to Strengthen Investment Board Oversight of
Poorly Planned and Performing Projects" (GAO-09-566). Our comments on
the report are attached.
If you have any questions, please contact me or have your staff contact
Candace Skurnik, Director, Audit Management and Liaison Staff at (410)
965-4636.
Sincerely,
Signed by:
Michael J. Astrue:
Enclosure:
[End of letter]
Comments On The Government Accountability Office (GAO) Draft Report,
"Information Technology: Federal Agencies Need To Strengthen Investment
Board Oversight Of Poorly Planned And Performing Projects" (GAO-09-566)
We have reviewed your governmentwide report, "Information Technology:
Federal Agencies Need to Strengthen Investment Board Oversight of
Poorly Planned and Performing Projects." We offer the following
comments for your consideration.
While you do not make any recommendations for us to consider, we are
concerned that our Information Technology Operations Assurance (ITOA)
project is included in the report and is identified as a poorly planned
or poorly performing project. We do not believe ITOA should be included
in the report as a poorly planned or performing project.
Your review selected projects that appear on the Office of Management
and Budget's (OMB) Management Watch List or OMB's High-Risk List. Our
ITOA project appears on OMB's High-Risk List. According to the White
House website [hyperlink,
http;//www.whitehouse.gov/omb/pubpressl2008/041708_it.html], "Projects
on the High Risk List are those requiring special attention from the
highest level of agency management, but aren't projects necessarily at
risk of failure." ITOA meets this definition-it warrants "special
attention from the highest level of agency management," but it is not
at risk of failure. ITOA's presence on the High Risk List does not
indicate that it is a poorly performing or planned project.
In the report, you used the High Risk List as a source of possible IT
projects to select fur the audit. It appears that you applied your own
criteria to the projects to identify some of them as poorly performing.
One of your criterion is "maintaining the project's cost and
schedule variances within 10 percent." We believe that you used this
criterion to select ITOA for the report. However, this criterion does
not distinguish the reason for the variance. This distinction is
important since a variance could indicate good management oversight if,
for example, contract awards come in lower than anticipated or work is
ahead of schedule or below budget, rather than indicate poor
performance.
Specifically, the ITOA project experienced a positive cost variance
that should not be characterized as a shortfall. When the General
Services Administration had trouble acquiring and developing the
property necessary for the Durham Support Center, we quickly adapted
the IT project schedule to match the new construction schedule,
ensuring that the equipment was at the right place at the right time.
Our management controls allowed us to keep the IT project on track and
generated a positive cost variance.
You should consider the reason for the ITOA project cost variance and
remove this project from the report because it is not a poorly planned
or poorly performing project.
[End of section]
Appendix XV: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner, (202) 512-9286, or pownerd@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, Sabine R. Paul, Assistant
Director; William G. Barrick; Neil J. Doherty; Nancy E. Glover; Robert
G. Kershaw; Lee A. McCracken; Tomas Ramirez; and Kevin C. Walsh made
key contributions to this report.
[End of section]
Footnotes:
[1] GAO, Information Technology: OMB Can Make More Effective Use of Its
Investment Reviews, [hyperlink, http://www.gao.gov/products/GAO-05-276]
(Washington, D.C.: Apr. 15, 2005).
[2] GAO, Information Technology: Management and Oversight of Projects
Totaling Billions of Dollars Need Attention, [hyperlink,
http://www.gao.gov/products/GAO-09-624T] (Washington, D.C.: Apr. 28,
2009).
[3] The selection process does not only apply to new projects. It
should be repeated each time funds are allocated to projects (this is
often referred to as "reselection").
[4] We are using "24 major agencies" to refer to 24 agencies listed in
the Chief Financial Officers (CFO) Act of 1990 (31 U.S.C. §901(b)).
They are the Departments of Agriculture, Commerce, Defense, Education,
Energy, Health and Human Services, Homeland Security, Housing and Urban
Development, the Interior, Justice, Labor, State, Transportation, the
Treasury, and Veterans Affairs; the Environmental Protection Agency,
General Services Administration, National Aeronautics and Space
Administration, National Science Foundation, Nuclear Regulatory
Commission, Office of Personnel Management, Small Business
Administration, Social Security Administration, and U.S. Agency for
International Development.
[5] GAO, Information Technology Management: Governmentwide Strategic
Planning, Performance Measurement, and Investment Management Can Be
Further Improved, [hyperlink, http://www.gao.gov/products/GAO-04-49]
(Washington, D.C.: Jan. 12, 2004).
[6] Division E of Pub. L. No. 104-106, February 10, 1996, now codified
as 40 U.S.C. Subtitle III--Information Technology Management, Chapters
111, 113, 115, and 117. The law, initially titled the Information
Technology Management Reform Act of 1996 along with the Federal
Acquisition Reform Act of 1996, was later renamed the 'Clinger-Cohen
Act' in Pub. L. No. 104-208, September 30, 1996.
[7] 40 U.S.C. § 11302(c).
[8] 40 U.S.C. § 11313.
[9] 40 U.S.C. § 11315.
[10] [hyperlink, http://www.gao.gov/products/GAO-05-276].
[11] [hyperlink, http://www.gao.gov/products/GAO-05-276]; GAO,
Information Technology: Agencies and OMB Should Strengthen Processes
for Identifying and Overseeing High Risk Projects, [hyperlink,
http://www.gao.gov/products/GAO-06-647] (Washington, D.C., June 15,
2006); Information Technology: Improvements Needed to More Accurately
Identify and Better Oversee Risky Projects Totaling Billions of
Dollars, [hyperlink, http://www.gao.gov/products/GAO-06-1099T]
(Washington, D.C.: Sept. 7, 2006); Information Technology: Further
Improvements Needed to Identify and Oversee Poorly Planned and
Performing Projects, [hyperlink,
http://www.gao.gov/products/GAO-07-1211T] (Washington, D.C.: Sept. 20,
2007); Information Technology: Agencies Need to Establish Comprehensive
Policies to Address Changes to Projects' Cost, Schedule, and
Performance Goals, [hyperlink, http://www.gao.gov/products/GAO-08-925]
(Washington, D.C.: July 31, 2008); and [hyperlink,
http://www.gao.gov/products/GAO-09-624T].
[12] GAO, Information Technology: OMB and Agencies Need to Improve
Planning, Management, and Oversight of Projects Totaling Billions of
Dollars, [hyperlink, http://www.gao.gov/products/GAO-08-1051T]
(Washington, D.C.: July 31, 2008).
[13] [hyperlink, http://www.gao.gov/products/GAO-08-1051T].
[14] [hyperlink, http://www.gao.gov/products/GAO-09-624T].
[15] GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, [hyperlink,
http://www.gao.gov/products/GAO-04-394G] (Washington, D.C: Mar. 1,
2004).
[16] GAO, Information Technology: SSA Has Taken Key Steps for Managing
Its Investments, but Needs to Strengthen Oversight and Fully Define
Policies and Procedures, [hyperlink,
http://www.gao.gov/products/GAO-08-1020] (Washington, D.C.: Sept. 12,
2008); Information Technology: DHS Needs to Fully Define and Implement
Policies and Procedures for Effectively Managing Investments,
[hyperlink, http://www.gao.gov/products/GAO-07-424] (Washington, D.C.:
Apr. 27, 2007); Information Technology: Treasury Needs to Strengthen
its Investment Board Operations and Oversight, [hyperlink,
http://www.gao.gov/products/GAO-07-865] (Washington, D.C.: July 23,
2007); Information Technology: Centers for Medicare and Medicaid
Services Needs to Establish Critical Investment Management
Capabilities, [hyperlink, http://www.gao.gov/products/GAO-06-12]
(Washington, D.C.: Oct. 28, 2005); Information Technology: HHS Has
Several Investment Management Capabilities in Place, but Needs to
Address Key Weaknesses, [hyperlink,
http://www.gao.gov/products/GAO-06-11] (Washington, D.C.: Oct. 28,
2005); Information Technology: FAA Has Many Investment Management
Capabilities in Place, but More Oversight of Operational Systems Is
Needed, [hyperlink, http://www.gao.gov/products/GAO-04-822]
(Washington, D.C.: Aug. 20, 2004); Bureau of Land Management: Plan
Needed to Sustain Progress in Establishing IT Investment Management
Capabilities, [hyperlink, http://www.gao.gov/products/GAO-03-1025]
(Washington, D.C.: Sept. 12, 2003); Information Technology:
Departmental Leadership Crucial to Success of Investment Reforms at
Interior, [hyperlink, http://www.gao.gov/products/GAO-03-1028]
(Washington, D.C.: Sept. 12, 2003); United States Postal Service:
Opportunities to Strengthen IT Investment Management Capabilities,
[hyperlink, http://www.gao.gov/products/GAO-03-3] (Washington, D.C.:
Oct. 15, 2002); and Information Technology: DLA Needs to Strengthen Its
Investment Management Capability, [hyperlink,
http://www.gao.gov/products/GAO-02-314] (Washington, D.C.: Mar. 15,
2002).
[17] [hyperlink, http://www.gao.gov/products/GAO-04-394G].
[18] [hyperlink, http://www.gao.gov/products/GAO-04-49].
[19] [hyperlink, http://www.gao.gov/products/GAO-08-1020].
[20] According to the ITIM framework, agencies should establish an
enterprisewide IT IRB composed of senior executives from IT and
business units.
[21] [hyperlink, http://www.gao.gov/products/GAO-06-11].
[22] GAO, Business Systems Modernization: DOD Needs to Fully Define
Policies and Procedures for Institutionally Managing Investments,
[hyperlink, http://www.gao.gov/products/GAO-07-538] (Washington, D.C.:
May 11, 2007).
[23] [hyperlink, http://www.gao.gov/products/GAO-06-11].
[24] Three of the 28 poorly performing projects we selected reported
performance shortfalls in 2006.
[25] In some cases, the department-level IRBs' selection review
consisted in approving selections made by other entities, including
lower-level boards or component agencies.
[26] A PBO is a government program, office, or other discrete
management unit with strong incentives to manage for results. The
organization commits to specific measurable goals with targets for
improved performance. In exchange, the PBO is allowed more flexibility
to manage its personnel and procurement.
[27] [hyperlink, http://www.gao.gov/products/GAO-07-538] and GAO,
Business Systems Modernization: Recent Slowdown in Institutionalizing
Key Management Controls Needs to Be Addressed, [hyperlink,
http://www.gao.gov/products/GAO-09-586] (Washington, D.C.: May 18,
2009).
[28] [hyperlink, http://www.gao.gov/products/GAO-04-394G].
[29] We did not receive a response from the Department of Agriculture,
the Department of Energy, the General Services Administration, or the
Small Business Administration.
[30] 31 U.S.C. §901(b).
[31] [hyperlink, http://www.gao.gov/products/GAO-04-49].
[32] [hyperlink, http://www.gao.gov/products/GAO-04-394G].
[33] The Management Watch List identifies projects that OMB determines
to be "poorly planned." When we began our review at the beginning of
2008, OMB had not yet released the fiscal year 2008 Management Watch
List.
[34] High-risk projects are identified as having performance shortfalls
if one or more of the following performance evaluation criteria are not
met: (1) establishing baselines with clear cost, schedule, and
performance goals; (2) maintaining the project's cost and schedule
variances within 10 percent; (3) assigning a qualified project manager;
and (4) avoiding duplication by leveraging inter-agency and
governmentwide investments.
[35] [hyperlink, http://www.gao.gov/products/GAO-07-1211T].
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
