Information Technology
Agencies Need to Improve the Accuracy and Reliability of Investment Information Gao ID: GAO-06-250 January 12, 2006Each year, agencies submit to the Office of Management and Budget (OMB) a Capital Asset Plan and Business Case--the exhibit 300--to justify each request for a major information technology (IT) investment. The exhibit's content should reflect controls that agencies have established to ensure good project management, as well as showing that they have defined cost, schedule, and performance goals. It is thus a tool to help OMB and agencies identify and correct poorly planned or performing investments. In its budget and oversight role, OMB relies on the accuracy and completeness of this information. GAO was asked to determine the extent to which selected agencies have underlying support for the information in their fiscal year 2006 exhibit 300s. From five major departments having over $1 billion in IT expenditures in that year, GAO chose for analysis 29 exhibits for projects that supported a cross section of federal activities.
Underlying support was often inadequate for information provided in the exhibit 300s reviewed. Three general types of weaknesses were evident. All exhibit 300s had documentation weaknesses. Documentation either did not exist or did not fully agree with specific areas of the exhibit 300. For example, both these problems occurred in relation to calculations of financial benefits for most investments. In addition, for 23 of the 29 investments, information on performance goals and measures was not supported by explanations of how agencies had initially measured their baseline levels of performance (from which they determine progress) or how they determined the actual progress reported in the exhibit 300. Agencies did not always demonstrate that they complied with federal or departmental requirements or policies with regard to management and reporting processes. For example, 21 investments were required to use a specific management system as the basis for the cost, schedule, and performance information in the exhibit 300, but only 6 did so following OMB-required standards. Also, none had cost analyses that fully complied with OMB requirements for cost-benefit and cost-effectiveness analyses. In contrast, most investments did demonstrate compliance with information security planning and training requirements. In sections that required actual cost data, these data were unreliable because they were not derived from cost-accounting systems with adequate controls. In the absence of such systems, agencies generally derived cost information from ad hoc processes. Officials from the five agencies (the Departments of Agriculture, Commerce, Energy, Transportation, and the Treasury) attributed these shortcomings in support to lack of understanding of a requirement or how to respond to it. Agency officials mentioned in particular insufficient guidance or training, as well as lack of familiarity with particular requirements. The weaknesses in the 29 exhibit 300s raise questions regarding the sufficiency of the business cases for these major investments and the quality of the projects' management. Without adequate support in key areas, OMB and agency executives may be depending on unreliable information to make critical decisions on IT projects, thus putting at risk millions of dollars. Further, although the 29 examples cannot be directly projected to the over one thousand business cases developed each year across the federal government, the results suggest that the underlying causes for the weaknesses identified need attention. These weaknesses and their causes are also consistent with problems in project and investment management that are pervasive governmentwide, including at such agencies as the Departments of Defense, Health and Human Services, and Homeland Security, as documented in reports by GAO and others.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-250, Information Technology: Agencies Need to Improve the Accuracy and Reliability of Investment Information
This is the accessible text file for GAO report number GAO-06-250
entitled 'Information Technology: Agencies Need to Improve the Accuracy
and Reliability of Investment Information' which was released on
February 13, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
January 2006:
Information Technology:
Agencies Need to Improve the Accuracy and Reliability of Investment
Information:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-250]:
GAO Highlights:
Highlights of GAO-06-250, a report to congressional requesters:
Why GAO Did This Study:
Each year, agencies submit to the Office of Management and Budget (OMB)
a Capital Asset Plan and Business Case”the exhibit 300”to justify each
request for a major information technology (IT) investment. The
exhibit‘s content should reflect controls that agencies have
established to ensure good project management, as well as showing that
they have defined cost, schedule, and performance goals. It is thus a
tool to help OMB and agencies identify and correct poorly planned or
performing investments. In its budget and oversight role, OMB relies on
the accuracy and completeness of this information. GAO was asked to
determine the extent to which selected agencies have underlying support
for the information in their fiscal year 2006 exhibit 300s. From five
major departments having over $1 billion in IT expenditures in that
year, GAO chose for analysis 29 exhibits for projects that supported a
cross section of federal activities.
What GAO Found:
Underlying support was often inadequate for information provided in the
exhibit 300s reviewed. Three general types of weaknesses were evident:
* All exhibit 300s had documentation weaknesses. Documentation either
did not exist or did not fully agree with specific areas of the exhibit
300. For example, both these problems occurred in relation to
calculations of financial benefits for most investments. In addition,
for 23 of the 29 investments, information on performance goals and
measures was not supported by explanations of how agencies had
initially measured their baseline levels of performance (from which
they determine progress) or how they determined the actual progress
reported in the exhibit 300.
* Agencies did not always demonstrate that they complied with federal
or departmental requirements or policies with regard to management and
reporting processes. For example, 21 investments were required to use a
specific management system as the basis for the cost, schedule, and
performance information in the exhibit 300, but only 6 did so following
OMB-required standards. Also, none had cost analyses that fully
complied with OMB requirements for cost-benefit and cost-effectiveness
analyses. In contrast, most investments did demonstrate compliance with
information security planning and training requirements.
* In sections that required actual cost data, these data were
unreliable because they were not derived from cost-accounting systems
with adequate controls. In the absence of such systems, agencies
generally derived cost information from ad hoc processes.
Officials from the five agencies (the Departments of Agriculture,
Commerce, Energy, Transportation, and the Treasury) attributed these
shortcomings in support to lack of understanding of a requirement or
how to respond to it. Agency officials mentioned in particular
insufficient guidance or training, as well as lack of familiarity with
particular requirements.
The weaknesses in the 29 exhibit 300s raise questions regarding the
sufficiency of the business cases for these major investments and the
quality of the projects‘ management. Without adequate support in key
areas, OMB and agency executives may be depending on unreliable
information to make critical decisions on IT projects, thus putting at
risk millions of dollars.
Further, although the 29 examples cannot be directly projected to the
over one thousand business cases developed each year across the federal
government, the results suggest that the underlying causes for the
weaknesses identified need attention. These weaknesses and their causes
are also consistent with problems in project and investment management
that are pervasive governmentwide, including at such agencies as the
Departments of Defense, Health and Human Services, and Homeland
Security, as documented in reports by GAO and others.
What GAO Recommends:
To improve the accuracy and value of exhibit 300s, GAO is making
recommendations aimed at improving guidance and training in exhibit 300
requirements and at ensuring limitations on reliability are disclosed
and mitigated. In response to a draft of this report, the agencies
agreed with the findings or had no comment. OMB accepted the findings
but stated that ultimate responsibility for the accuracy and
reliability of this information lies with the agencies.
www.gao.gov/cgi-bin/getrpt?GAO-06-250.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Dave Powner at (202) 512-
9286 or pownerd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Exhibit 300s Were Generally Not Based on Adequate Support:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Appendix II: Comments from the Office of Management and Budget:
Appendix III: Descriptions of Investments Reviewed:
Department of Agriculture:
Department of Commerce:
Department of Energy:
Department of Transportation:
Department of the Treasury:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Description of Key Sections in the Exhibit 300 and General
Documentation Typically Used as Support:
Table 2: Financial Funding Data for ePermits:
Table 3: Financial Funding Data for CFMS:
Table 4: Financial Funding Data for IAS:
Table 5: Financial Funding Data for PCIT:
Table 6: Financial Funding Data for PCIMS:
Table 7: Financial Funding Data for STARS II:
Table 8: Financial Funding Data for AWIPS:
Table 9: Financial Funding Data for CLASS:
Table 10: Financial Funding Data for ECON:
Table 11: Financial Funding Data for AES:
Table 12: Financial Funding Data for NWSTG:
Table 13: Financial Funding Data for SOCC/CDA:
Table 14: Financial Funding Data for ESnet:
Table 15: Financial Funding Data for eCMS:
Table 16: Financial Funding Data for IPABS-IS:
Table 17: Financial Funding Data for LSN:
Table 18: Financial Funding Data for LANL ERP:
Table 19: Financial Funding Data for ASCM:
Table 20: Financial Funding Data for DOT Financial System
Consolidation:
Table 21: Financial Funding Data for NTD:
Table 22: Financial Funding Data for NEXCOM:
Table 23: Financial Funding Data for SASO:
Table 24: Financial Funding Data for WAAS:
Table 25: Financial Funding Data for CADE:
Table 26: Financial Funding Data for DMAS:
Table 27: Financial Funding Data for EMS:
Table 28: Financial Funding Data for GWA:
Table 29: Financial Funding Data for SCRIPS:
Table 30: Financial Funding Data for SPS:
Figure:
Figure 1: Breakdown by Civilian Agencies of Planned $35 Billion in
Fiscal Year 2006 IT Investments:
Abbreviations:
CIO: Chief Information Officer:
BRM: Business Reference Model:
DHS: Department of Homeland Security:
DOD: Department of Defense:
EVM: earned value management:
FISMA: Federal Information Security Management Act:
HHS: Department of Health and Human Services:
IT: information technology:
ITIM: information technology investment management:
NIST: National Institute of Standards and Technology:
OMB: Office of Management and Budget:
Letter January 12, 2006:
The Honorable Tom Davis:
Chairman:
Committee on Government Reform:
House of Representatives:
The Honorable Adam H. Putnam:
House of Representatives:
Each year, the Office of Management and Budget (OMB) plays a central
role in determining how much the government plans to spend for
information technology (IT) and how these funds are allocated. The IT
budget is not insignificant: federal agencies requested over $65
billion in fiscal year 2006. While these investments are critical to
achieving the goals of the federal government, for the past 3 years OMB
has highlighted in the President's Budget tens of billions of dollars
of IT investments that are at risk. In the most recent budget, for
fiscal year 2006, nearly 25 percent of the funds requested, totaling
about $15 billion, were considered by OMB to be at risk.
A key component of OMB's management and oversight of the IT budget
process is the exhibit 300, also known as the Capital Asset Plan and
Business Case, which is developed by agencies and reviewed by both
agencies and OMB. OMB sets forth requirements for the exhibit 300 in
its circular A-11.[Footnote 1] According to this guidance, agencies are
required to perform analyses and provide documentation to support
decisions on proposed major IT investments. The exhibit 300 is the
means to accomplish this task: it is a reporting mechanism intended to
enable an agency to demonstrate to its own management, as well as OMB,
that it has employed the disciplines of good project management;
developed a strong business case for the investment; and met other
Administration priorities in defining the cost,schedule, and
performance goals proposed for the investment. The exhibit 300
comprises eight key sections, which cover spending, performance goals
and measures, analysis of alternatives,[Footnote 2] risk inventory and
assessment, acquisition strategy, project (investment) and funding
plan, enterprise architecture, and security and privacy. When
considering IT investments to recommend for funding, OMB relies on the
accuracy and completeness of the information reported in the exhibit
300s.
This report responds to your request that we determine the extent to
which selected agencies have underlying support for the information in
their fiscal year 2006 exhibit 300s. To accomplish this objective, we
reviewed exhibit 300s from the fiscal year 2006 budget submission, as
well as supporting documentation, for 29 major IT investments at five
departments.[Footnote 3] We compared information in each exhibit 300
with the supporting documentation on the corresponding investment.
Further details on our objective, scope, and methodology are provided
in appendix I. Our work was conducted between March and November 2005
in accordance with generally accepted government auditing standards.
Results in Brief:
Underlying support for the information provided in the exhibit 300 was
often inadequate. Examination of the exhibit 300s and the supporting
materials revealed three types of weaknesses.
* All exhibit 300s had documentation weaknesses. Either documentation
did not exist for specific areas of the exhibit 300, or it did not
fully agree with the exhibit 300. For 23 of the 29 investments, for
example, information in the performance goals and measures section was
not supported by explanations of how agencies had initially measured
their baseline levels of performance (from which they determine
progress) or how they determined the actual progress that was reported
in the exhibit 300. In the case of risk assessment, supporting
documentation for about 75 percent of the investments did not address
OMB's required risk categories. Additionally, the analysis of
alternatives for most investments either lacked supporting
documentation to justify the calculations of financial benefits in the
exhibit 300, or the documentation did not agree with what was reported.
* Agencies did not always demonstrate that they complied with federal
or departmental requirements or policies with regard to management and
reporting processes. For example, 21 investments were required to use
an earned value management (EVM) system[Footnote 4] as the basis for
the cost, schedule, and performance information provided in the exhibit
300, but only 6 investments used an EVM process that followed OMB-
required standards. In addition, none of the investments under review
had cost analyses that fully complied with OMB requirements for
completing cost-benefit and cost-effectiveness analyses. An exception
was the information security section, for which most investments had
security plans and indications that security awareness training had
been conducted.
* For those sections that required actual cost data, including the
summary of spending and project and funding plan sections, the data
were unreliable because they were not derived from cost-accounting
systems with adequate controls. In the absence of adequate cost-
accounting systems, agencies generally derived cost information from ad
hoc processes used by project managers.
Agency officials attributed the shortcomings in support to lack of
understanding of a requirement or how to respond to it. Agency
officials mentioned in particular insufficient guidance or training, as
well as lack of familiarity with particular requirements, such as the
EVM process. If underlying support is inadequate in key areas, OMB and
agency executives are depending on unreliable information to monitor
the management of major IT projects and to make critical decisions on
their funding, thus putting at risk millions of dollars in investments.
These weaknesses and their causes are also consistent with problems in
project and investment management that are pervasive governmentwide,
including at such agencies as the Departments of Defense (DOD), Health
and Human Services (HHS), and Homeland Security (DHS), as documented in
reports by GAO and others.
We are making recommendations to OMB aimed at improving guidance and
training in exhibit 300 requirements and at ensuring that limitations
on the reliability of information in exhibit 300s are disclosed and
mitigated.
In written comments, OMB accepted the findings of the draft report,
while expressing concern that, by directing our recommendations to OMB
rather than to the agencies, we were suggesting that OMB rather than
the agencies is responsible for data accuracy and employee training. We
do not intend to make this suggestion; we place significant
responsibility on agencies, as reflected in our recommendation that OMB
instruct agencies to determine the extent to which the information
contained in each exhibit 300 is accurate and reliable, to disclose
weaknesses, and to describe their approach to mitigating these
weaknesses. This recommendation clearly places responsibility on the
agencies for assessing the quality of their budget information and the
processes that produced this information. Our recommendations are
directed to OMB because of its responsibility for providing
governmentwide leadership in information resources management, pursuant
to the Clinger-Cohen Act, and because they address findings relating to
OMB-required budget documents.
In addition, OMB expressed concern that the recommendations do not
focus on how well agencies fulfill their underlying information
resources management responsibilities. Our view is that our
recommendation on disclosing and mitigating weaknesses does address
these underlying responsibilities. The report specifically addresses
the reliability of exhibit 300s as support for agency and OMB decision
making and clearly states that the lack of documentation and inadequate
support may indicate underlying management weaknesses. Requiring
agencies to disclose and mitigate associated weaknesses will
necessitate that agencies examine and address their approach to
fulfilling information resources management responsibilities.
Technical comments from the Departments of Energy and Transportation
were incorporated as appropriate. The Departments of Agriculture,
Commerce, Transportation, and the Treasury provided no comments on a
draft of this report.
We address these comments more fully in the Agency Comments and Our
Evaluation section of this report. We have reproduced the written
comments in appendix II.
Background:
The President's Budget for Fiscal Year 2006 included 1,087 IT projects,
totaling about $65 billion. The planned expenditures at the civilian
agencies comprised about $35 billion of that total cost. In particular,
the five departments in our review made up about one-third of the
civilian planned expenditures (see fig. 1).
Figure 1: Breakdown by Civilian Agencies of Planned $35 Billion in
Fiscal Year 2006 IT Investments:
[See PDF for image]
[End of figure]
OMB plays a key role in overseeing these IT investments and how they
are managed, stemming from its predominant mission: to assist the
President in overseeing the preparation of the federal budget and to
supervise budget administration in executive branch agencies. In
helping to formulate the President's spending plans, OMB is responsible
for evaluating the effectiveness of agency programs, policies, and
procedures; assessing competing funding demands among agencies; and
setting funding priorities. To carry out these responsibilities, OMB
depends on agencies to collect and report accurate and complete
information; these activities depend in turn on agencies having
effective IT management practices.
Improvements in IT Management Are Goals of Laws and Guidance:
To drive improvement in the implementation and management of IT
projects, the Congress enacted the Clinger-Cohen Act in 1996, which
expanded the responsibilities of OMB and the agencies that had been set
under the Paperwork Reduction Act.[Footnote 5] The Clinger-Cohen Act
requires that agencies engage in capital planning and performance-and
results-based management.[Footnote 6] The act also requires OMB to
establish processes to analyze, track, and evaluate the risks and
results of major capital investments in information systems made by
executive agencies. OMB is also required to report to the Congress on
the net program performance benefits achieved as a result of major
capital investments in information systems that are made by executive
agencies.[Footnote 7]
With regard to OMB's responsibilities in this area, we recently issued
a report[Footnote 8] that provided recommendations to improve OMB's
processes for monitoring high-risk IT investments. Since that report
was released, OMB has issued additional guidance outlining steps that
agencies must take for all high-risk projects to better ensure improved
execution and performance as well as promote more effective
oversight.[Footnote 9]
In response to the Clinger-Cohen Act and other statutes, OMB developed
policy for planning, budgeting, acquisition, and management of federal
capital assets. This policy is set forth in OMB Circular A-11 (section
300) and in OMB's Capital Programming Guide (supplement to Part 7 of
Circular A-11), which directs agencies to develop, implement, and use a
capital programming process to build their capital asset portfolios.
Among other things, OMB's Capital Programming Guide directs agencies
to:
* evaluate and select capital asset investments that will support core
mission functions that must be performed by the federal government and
demonstrate projected returns on investment that are clearly equal to
or better than alternative uses of available public resources;
* institute performance measures and management processes that monitor
actual performance and compare to planned results; and:
* establish oversight mechanisms that require periodic review of
operational capital assets to determine how mission requirements might
have changed and whether the asset continues to fulfill mission
requirements and deliver intended benefits to the agency and customers.
Among OMB's goals in requiring the use of a capital programming process
is to assist agencies in complying with a number of results-oriented
requirements. Key requirements include those set by:
* the Federal Acquisition Streamlining Act of 1994,[Footnote 10] which
(1) requires agencies to establish cost, schedule, and measurable
performance goals for all major acquisition programs and (2)
establishes that agencies should achieve on average 90 percent of those
goals;
* the Government Performance and Results Act of 1993,[Footnote 11]
which establishes the foundation for budget decision making to achieve
strategic goals in order to meet agency mission objectives; and:
* the Federal Information Security Management Act of 2002,[Footnote 12]
which requires agencies to integrate IT security into their strategic
and operational planning processes, such as the capital planning and
enterprise architecture processes at the agency.
OMB is aided in its responsibilities by the Chief Information Officers
(CIO) Council as described by the E-Government Act of 2002.[Footnote
13] The council is designated the principal interagency forum for
improving agency practices related to the design, acquisition,
development, modernization, use, operation, sharing, and performance of
federal government information resources. Among the specific functions
of the CIO Council are the development of recommendations for the
Director of OMB on government information resources management policies
and requirements and the sharing of experiences, ideas, best practices,
and innovative approaches related to information resources management.
The CIO Council has issued several guides on capital planning and
investment management over the past several years.[Footnote 14]
To further support the implementation of IT capital planning practices,
we have developed an IT investment management (ITIM) framework[Footnote
15] that agencies can use in developing a stable and effective capital
planning process, as required by statute and directed in OMB's Capital
Programming Guide. Consistent with the statutory focus on
selecting,[Footnote 16] controlling,[Footnote 17] and
evaluating[Footnote 18] investments, this framework focuses on these
processes in relation to IT investments specifically. It is a tool that
can be used to determine both the status of an agency's current IT
investment management capabilities and the additional steps that are
needed to establish more effective processes. Mature and effective
management of IT investments can vastly improve government performance
and accountability. Without good management, such investments can
result in wasteful spending and lost opportunities for improving
delivery of services to the public.
The ITIM framework lays out a coherent collection of key practices
that, when implemented in a coordinated manner, can lead an agency
through a robust set of analyses and decision points that support
effective IT investment management. The framework explicitly calls for
consideration of cost, schedule, benefit, and risk objectives,
including the development of analyses such as return on investment and
a risk management plan. The framework also describes the criticality of
tracking progress using valid and complete data. The guidance laid out
in the ITIM framework is consistent with the requirements of OMB's
Circular A-11 and matches it in many instances. For example, among the
requirements on the exhibit 300 is that agencies indicate that the
investment has been reviewed and approved by the responsible oversight
entity. The agency investment review board is a critical element of the
ITIM framework, and the expectation for the board to select and oversee
IT investments is explicit.
In previous work using our IT investment management framework, we
reported that the use of IT investment management practices by agencies
was mixed. For example, a few agencies that have followed the ITIM
framework in implementing capital planning processes have made
significant improvements.[Footnote 19] In contrast, however, we and
others have continued to identify weaknesses at agencies in many areas,
including immature management processes to support both the selection
and oversight of major IT investments and the measurement of actual
versus expected performance in meeting established IT performance
measures.[Footnote 20] For example:
* We recently reported that the HHS senior investment board does not
regularly review component agencies' IT investments, leaving close to
90 percent of its discretionary investments without an appropriate
level of executive oversight.[Footnote 21] To remedy this weakness, we
recommended that the department (1) establish a process for the
investment board to regularly review and track the performance of a
defined set of component agency IT systems against expectations and (2)
take corrective actions when these expectations are not being met.
* At DHS, we determined that the department's draft information
resources management strategic plan did not include fully defined goals
and performance measures.[Footnote 22] To address this weakness, we
recommended that the department establish IT goals and performance
measures that, at a minimum, address how information and technology
management contributes to program productivity, the efficiency and
effectiveness of agency operations, and service to the public.
* A recent review by the DOD Inspector General[Footnote 23] determined
that over 90 percent of the business cases submitted to OMB in support
of the DOD fiscal year 2006 budget request did not completely respond
to one or more data elements addressing security funding, certification
and accreditation, training and security plans, and enterprise
architecture. The DOD Inspector General concluded that, as a result,
these submissions continued to have limited value and did not
demonstrate that the department was effectively managing its proposed
IT investments for fiscal year 2006.
Exhibit 300 Supports OMB and Agency Oversight of IT Management:
Besides providing policy for planning, budgeting, acquisition, and
management of federal capital assets, section 300 of OMB's Circular A-
11 instructs agencies on budget justification and reporting
requirements for major IT investments.[Footnote 24] Section 300 defines
the budget exhibit 300, also called the Capital Asset Plan and Business
Case, as a document that agencies submit to OMB to justify resource
requests for major IT investments. According to OMB, only priority
capital asset investments that comply with the policies for good
capital programming, as described in the Capital Programming Guide,
will be recommended for funding in the President's Budget.
The exhibit 300 was established as a source of information on which to
base both quantitative decisions about budgetary resources consistent
with the Administration's program priorities and qualitative
assessments about whether the agency's planning, acquisition,
management, and use of capital assets (investments) are consistent with
OMB policy and guidance. The types of information included in the
exhibit 300 are intended, among other things, to help OMB and the
agencies identify and correct poorly planned or performing investments
(i.e., investments that are behind schedule, over budget, or not
delivering expected results) and real or potential systemic weaknesses
in federal information resource management (such as a shortage of
sufficiently qualified project managers).
According to Circular A-11, the information in the exhibit 300 allows
the agency and OMB to review and evaluate each agency's IT spending and
to compare IT spending across the federal government. Further, the
information helps the agency and OMB to provide a full and accurate
accounting of IT investments for the agency, as required by the
Paperwork Reduction Act and the Clinger-Cohen Act.
The exhibit 300 is required for all assets, though certain sections
apply only to information technology. Table 1 provides a description of
the key sections of the exhibit 300, as well as examples of the types
of documentation that provide support for the data summarized in the
exhibit 300 (although the supporting documentation may take other
forms). This support may be derived from a variety of sources,
including financial management systems and management processes that
agencies carry out to comply with federal requirements and guidelines
(such as the Federal Information Security Management Act of 2002 and
the Federal Enterprise Architecture),[Footnote 25] as well as from
analyses carried out specifically in support of the management of the
investment.
Table 1: Description of Key Sections in the Exhibit 300 and General
Documentation Typically Used as Support:
Section name: Summary of Spending table;
Description: Provides an overview of the costs for planning,
acquisition, maintenance, and full- time employees for the previous,
current, and budget fiscal years; it also includes a summary of these
costs for earlier years and estimated costs for future years;
Examples of supporting documentation: Financial reports.
Section name: Performance Measures and Goals;
Description: Describes the link between the agency's annual goals and
mission and how the investment will meet those goals. This section
illustrates the performance measures and results of the investments;
Examples of supporting documentation: Annual performance plan and/or
annual performance budget; IT strategic plan.
Section name: Analysis of Alternatives;
Description: Provides a summary of the comparison of viable alternative
solutions that includes a general rationale and analysis of the
monetized benefits for each alternative presented;
Examples of supporting documentation: Cost- benefit analysis or cost-
effectiveness analysis.
Section name: Risk Inventory and Assessment;
Description: Provides a summary of the investment's risk assessment,
showing the active management of 19 risk elements that OMB requires to
be considered;
Examples of supporting documentation: Risk management plan, risk
reports.
Section name: Acquisition Strategy;
Description: Provides a description of the acquisition strategy used
and mitigation efforts to ensure minimal risk to the government;
Examples of supporting documentation: Acquisition plan.
Section name: Project (Investment) and Funding Plan;
Description: Provides a summary of the investment's status in
accomplishing baseline cost and schedule goals through the use of an
earned value management (EVM) system or operational analysis, depending
on the life-cycle stage;
Examples of supporting documentation: Cost performance reports,
integrated baseline review, time-phased performance measurement
baseline, work breakdown structure, and operational analysis.
Section name: Enterprise Architecture;
Description: Demonstrates that the investment is included in the
agency's enterprise architecture and capital planning investment
control process. Illustrates the agency's capability to align the
investment to the Federal Enterprise Architecture;
Examples of supporting documentation: Investment- specific artifacts,
including as-is and to-be architectures, migration plan, documented
approval by an enterprise architecture review committee.
Section name: Security and Privacy;
Description: Provides a description of an investment's security and
privacy issues. It summarizes the agency's ability to manage security
at the system or application level. Additionally, it demonstrates
compliance with the certification and accreditation process, as well as
the mitigation of IT security weaknesses;
Examples of supporting documentation: Certification and accreditation
packages, security plans, security training log, and system-level
incident handling procedures.
Source: GAO analysis based on OMB data.
[End of table]
According to OMB guidance, the life-cycle stage of the asset affects
what is reported on the exhibit 300:
* New investments (i.e., proposed for budget year or later, or in
development) must be justified based on the need to fill a gap in the
agency's ability to meet strategic goals and objectives with the lowest
life-cycle costs of all possible alternatives and provide risk-adjusted
cost and schedule goals and measurable performance benefits.
* Mixed life-cycle investments (i.e., investments that are operational
but include some developmental effort, such as a technology refresh)
must demonstrate satisfactory progress toward achieving baseline cost,
schedule, and performance goals using an EVM system.[Footnote 26]
* Operational investments (i.e., steady state) must demonstrate, among
other things, how close actual annual operating and maintenance costs
are to the original life-cycle cost estimates; whether the technical
merits of the investment continue to meet the needs of the agency and
customers; and that an analysis of alternatives was performed with a
future focus.
OMB requires agencies to transmit exhibit 300s electronically, using a
predefined format. To meet this requirement and facilitate the
aggregation of elements of the exhibits from various sources throughout
the organization, many agencies use software applications to compile
their exhibits 300s. Besides aggregating portions of the exhibit 300,
these tools are designed to also perform certain calculations, such as
return on investment and those required for earned value analysis.
Exhibit 300s Were Generally Not Based on Adequate Support:
Although the agencies reported that all 29 exhibit 300s had been
approved by their investment review boards (as required), in many
instances, support for the information provided was not adequate.
(Details on the 29 investment projects described in the exhibit 300s
that we reviewed are provided in app. III.) Three types of problems
were evident.
* First, all exhibit 300s had documentation weaknesses. For example,
each investment lacked documentary support for one or more of the
following: Analysis of Alternatives, Risk Inventory and Assessment, and
Performance Measures and Goals. In other cases, the supporting material
that was provided to us did not match the information in the exhibit
300.
* Second, agencies did not always demonstrate (for example, in the
Security and Privacy and the Project and Funding Plan sections) that
they complied with federal requirements or policies with regard to
management and reporting processes.
* Finally, information in some sections (such as the Summary of
Spending table and the Project and Funding Plan) could not be relied
upon because the numbers were not derived using repeatable processes or
reliable systems.
Agency officials attributed the absence of adequate support for their
exhibit 300s to lack of understanding of the requirements or of how to
respond to them. Agency officials mentioned in particular insufficient
guidance or training, as well as lack of familiarity with particular
requirements, such as the EVM process. If underlying support is
inadequate in key areas, OMB and agency executives are depending on
unreliable information to monitor the management of major IT projects
and to make critical decisions on their funding, thus putting at risk
millions of dollars in investments.
Underlying Documentation Was Lacking or Did Not Support the Exhibit
300:
OMB Circular A-11 states that agencies must justify funding requests
for major acquisitions by demonstrating, among other things, measurable
performance benefits, comprehensive risk mitigation and management
planning, and positive return on investment for the planned investment.
Agencies are instructed to establish performance metrics (including
baselines from which progress can be measured) to ensure that project
managers are accountable in meeting expected performance goals and that
projects are aligned with the agencies' strategic goals. Agencies are
also expected to manage investment risk through a robust risk
management program; according to OMB's guidance, agencies need to
actively manage risks from initial concept throughout the life cycle of
each investment. To demonstrate a positive return on investment for the
selected alternative and identify a project's total lifetime cost, OMB
requires agencies to compare alternatives and report summary cost
information for investments (including calculations for payback period
and net present value). Documents produced in the performance of these
activities provide evidence that they were carried out as required.
* Performance measures. The investments did not usually demonstrate the
basis for the performance measure information provided in the exhibit
300. Only 6 of the 29 investments had documentation to support how
agencies initially measured their baseline levels of performance, from
which they measured progress toward the agency's strategic goals. In
most cases, the investments lacked documentation describing the levels
of performance that had been achieved or how these results actually
helped meet agency strategic needs. The absence of documentation in
these cases could indicate a systemic weakness in agency performance
management practices, since well-developed practices should provide the
expected support. This finding is consistent with our prior work where
we determined that agencies were generally not measuring actual versus
expected performance in meeting IT performance goals.[Footnote 27] Weak
performance management practices reduce the ability of agency
executives to track investment performance in meeting performance
objectives and raise the risk that investments will not be well aligned
with agency strategic objectives.
* Risk management. About 75 percent of the investments were unable to
demonstrate that they were actively addressing the risk elements that
OMB specifies in Circular A-11, or how they had determined that any of
those risks were not applicable. In addition, documentation of risk
management that was provided had significant weaknesses. In one case, a
risk management plan was approximately 9 years old and had not been
updated, and for three investments, the risk documentation addressed
only the project development phase, even though the systems had exited
that phase and were in full operation.
* Analysis of alternatives. All 29 investments reported cost
information in the analysis of alternatives section of the exhibit 300.
However, in about 72 percent of the exhibit 300s reviewed, either
supporting documentation was missing for this cost information, or
information in the documentation did not agree with that in the exhibit
300.
In cases where investments lacked documentation to support information
reported in the performance and risk areas, project officials
frequently told us that they had filled out these sections of the
exhibit 300 to satisfy the reporting requirement, relying on their own
knowledge of the investment rather than any project documentation.
However, such an approach is not consistent with the requirement for
providing accurate information in compliance with OMB capital
programming and capital planning and investment control policies. In
addition, several project officials told us that they believed some of
the 19 risk management areas required in the exhibit 300 were not
applicable to their investment, but they reported on those categories
nonetheless to fulfill the requirement. Although the guidance instructs
agencies to indicate whether the risk category was not applicable,
officials stated that their impression is that "not applicable"
responses might lower the evaluation of their investments and reduce or
eliminate their funding.
Further, agency officials generally responded that the training they
received for preparing the exhibit 300 was not sufficient. For example,
one agency commented that agencies would benefit from targeted OMB
training that would address agency-specific questions. Several agencies
stressed that OMB training should occur earlier in the budget cycle. In
addition, one agency said that it needed OMB training on preparing each
section of the exhibit 300.
Overall, the lack of documentation supporting the exhibit 300s raises
questions regarding the sufficiency of the business case for the
investment and the quality of the projects' management.
Agencies Did Not Always Comply or Provide Evidence of Compliance with
Federal Requirements and Policies:
Compliance with OMB and other federal guidance and related federal laws
helps ensure that agency investments are managed in a manner consistent
with the intent of the Congress and that key information is available
to OMB and agency managers on which they can base informed decisions.
* The security section of the exhibit 300 requires that agencies
demonstrate that they have developed information security plans in
accordance with the Federal Information Security Management Act of 2002
(FISMA); according to FISMA, these plans must include rules of behavior
for system use, technical security controls, and procedures for
incident handling--that is, how to respond to system security breaches.
In addition, agencies ensure that employees and contractors receive
security awareness training.[Footnote 28] Guidance from the National
Institute of Standards and Technology (NIST) supports FISMA by
outlining the necessary components of key security documentation,
including security plans, certification and accreditation packages, and
security controls testing.
* For the analysis of alternatives section, OMB's instructions for the
exhibit 300 cite the Clinger-Cohen Act, which requires agencies to
complete a cost-benefit analysis for new IT investments, and OMB
Circular A-94, which outlines requirements for completing cost-benefit
and cost-effectiveness analyses, including the comparison of at least
three alternatives, a discussion of assumptions for each alternative,
and an analysis of uncertainty (a sensitivity assessment to raise
awareness of the potential for unforeseen impacts on the investment).
* For the project and funding plan section, OMB Circular A-11 provides
guidance that requires an agency to have in place a process for
monitoring the investment's status in accomplishing baseline cost and
schedule goals.
For the 29 investments, agency compliance with the FISMA and NIST
requirements described above was mixed. For example, about 86 percent
of all investments could demonstrate, based on documentation, that
security awareness training had been conducted for employees and
contractors and that a mechanism for tracking completion of security
awareness training had been established. In addition, 21 of the 22
operational investments (for which information security plans are
required) had security plans that addressed areas such as the rules of
behavior for system use and technical security controls. In contrast,
about 77 percent of these 22 investments did not provide support
describing how incident handling activities would be performed at a
system level, such as detecting, reporting, and mitigating risks
associated with security incidents.
While the compliance of security documentation with federal
requirements was mixed, the documented support for the analysis of
alternatives and the project and funding plan areas of the exhibit 300
provided little assurance that investments complied with applicable
guidance and laws. None of the investments had cost analysis
documentation that fully complied with Circulars A-94 and A-11 criteria
(lacking, for example, a comparison of at least three alternatives, a
discussion of assumptions for each alternative, or an analysis of
uncertainty).
Project officials attributed deficiencies in the analysis of
alternatives to, among other things, a lack of understanding of what
was expected for reporting in the exhibit 300. In a few instances,
officials noted that they believed that their investments were excluded
from meeting the federal requirements because the investments were near
the end of their operational or, in some cases, useful life cycles. OMB
guidance on analysis of alternatives does not differentiate between
operational and developmental investments; nonetheless, one agency's
internal guidance explicitly states that no analysis of alternatives is
necessary for investments in the steady state (that is, operational).
However, a forward-looking analysis of alternatives for operational
investments can help agencies recognize when an alternative solution
may be more efficient or effective than the current investment, thereby
freeing scarce resources to be reallocated.
The agencies' lack of compliance with OMB guidelines for analysis of
alternatives, including the cost-benefit analysis, leaves senior
executive managers at risk of making poor investment management
decisions on incomplete and sometimes inaccurate information.
For the project and funding plan section of the exhibit 300, OMB
Circular A-11 provides guidance on the information to be provided,
which depends upon the state of the investment (i.e., new, mixed life
cycle, or steady state). According to this guidance, information
presented in the project and funding plan is to be derived from one of
two types of analysis: for steady state investments, an operational
analysis, and for new and mixed life-cycle investments, an analysis
based on an EVM process that is compliant with ANSI/EIA-748-A.[Footnote
29] Operational analysis is a method for assessing the technical merits
of an existing investment in meeting user needs, while EVM is a method
for assessing the value of work performed compared to its actual cost
during development of an investment.
Of the eight steady state investments we reviewed, only two had
conducted an operational analysis. Furthermore, only one of those had
documented procedures that were in accordance with OMB's Capital
Programming Guide criteria, such as addressing user needs and technical
performance. In most cases for which no operational analyses were in
place, agency officials commented that OMB guidance describing how to
perform an operational analysis was at such a high level of generality
that they found it difficult to follow. Instead of attempting to devise
and perform an operational analysis, therefore, they implemented
variations on an EVM process. However, these implementations of EVM did
not address topics required for the operational analysis, such as user
needs and technical performance. Unless they address these topics,
agencies may not have the information they need to determine, among
other things, whether investments are performing as intended and
meeting user needs.
Similarly, of the 21 new and mixed life-cycle investments required to
use EVM, only 6 used an EVM process that generally followed the ANSI
standard.[Footnote 30] Since fiscal year 2002, OMB has required the use
of EVM as a project management tool. The ANSI standard is intended to
ensure that data produced by an EVM process are reliable so as to allow
objective reports of project status, produce early warning signs of
impending schedule delays and cost overruns, and ultimately provide
unbiased estimates of anticipated costs at completion. If agencies do
not implement EVM processes that follow the ANSI standard, they have
reduced assurance that the information used for tracking the cost,
schedule, and performance of the investment is reliable.
For the remaining 15 investments that did not have EVM processes
following the required standard, project officials commented that EVM
was relatively new to them and that they did not understand how to
implement an ANSI-compliant process at the time of the fiscal year 2006
submission. At the time of our review, all five departments stated that
they were working toward implementing compliant processes.
To OMB's credit, it recognized the need for improvement in the
execution of agencies' IT projects and has issued clarifying guidance
on the implementation of EVM.[Footnote 31] This guidance, issued in
August 2005, could be expected to have an impact on the exhibit 300s
prepared for fiscal year 2008. Under this guidance, agencies are
instructed, among other things, to:
* develop comprehensive agency policies for using EVM to plan and
manage development activities for major IT investments no later than
December 31, 2005;
* include a provision and clause in major acquisition contracts or
agency in-house project charters directing the use of an EVM system
compliant with the required standard; and:
* provide documentation demonstrating that the contractor's or agency's
in-house EVM system complies with the required standard and conduct
periodic surveillance reviews.
Additionally, the Civilian Agency Acquisition Council and the Defense
Acquisition Regulations Council published in the Federal Register a
proposed amendment[Footnote 32] to the Federal Acquisition Regulation
(FAR Case 2004-019) to standardize EVM contract policy across the
government.
In previous work,[Footnote 33] we have reported that EVM can have a
significant impact on the success of an IT acquisition because it
heightens visibility into whether a program is on target with respect
to cost, schedule, and technical performance. Therefore, it is
important that the process is implemented properly to maximize its
value as a project management tool. If it is not implemented
effectively, agency executives and OMB risk making poor investment
decisions based on inaccurate and potentially misleading EVM
information.
Cost Data Supporting Business Cases Were Unreliable:
Accurate and timely cost management information is critical for federal
managers to understand the progress of major projects and vital in
developing meaningful links among budget, accounting, and performance.
The Federal Financial Management Improvement Act of 1996 [Hyperlink,
http://frwebgate.access.gpo.gov/cgi-
bin/getdoc.cgi?dbname=104_cong_public_laws&docid=f:publ208.104.pdf]
emphasizes the need for agencies to have systems that are able to
generate reliable, useful, and timely information for decision-making
purposes and to ensure accountability on an ongoing basis.[Footnote 34]
In previous work, we have reported on the lack of adherence to federal
accounting standards throughout the federal government and have made
recommendations that agencies improve cost-accounting systems.[Footnote
35]
At every agency, cost information reported in the 29 exhibit 300s was
derived from ad hoc processes rather than from cost-accounting systems
with adequate controls to ensure accountability. This condition had
impact in two particular areas of the exhibit 300--the summary of
spending table and the project and funding plan section:
* Figures for dollars expended for the prior year (in this case, fiscal
year 2004) were not reliable. In all cases, documentation provided to
support prior year cost figures in the summary of spending table showed
that the information was derived from ad hoc sources, such as
spreadsheet estimates, handwritten figures, or e-mails. Therefore, the
cost data reported in the exhibit 300 are not verifiable.
* Information in the project and funding plans was also unreliable for
the 21 new and mixed life-cycle investments required to use EVM. As
discussed earlier, 15 of these investments reported cost figures based
on EVM processes that did not follow the ANSI standard; because the
standard was not followed, these processes did not have the controls
necessary to ensure that the data they produced were reliable. The
other 6 investments had ANSI-compliant EVM processes in place for the
contractor component of the investment costs, but the government
component of the investment costs was derived from ad hoc systems (such
as tracking government costs in spreadsheets based on project managers'
own records); thus, that portion of the data was not reliable, lending
a degree of unreliability to the overall EVM reports.
The lack of accurate cost figures limits decision makers' ability to
determine the actual resources expended on an investment, and therefore
inhibits their ability to make fully informed decisions on whether to
proceed. Without reliable systems that meet federal standards,
government agencies cannot produce reliable cost-based performance
information.
Conclusions:
The usefulness of the exhibit 300 business case as a mechanism to
support the selection and oversight of federal IT investments is
undercut by the kinds of weaknesses displayed in the 29 business cases
that we reviewed. Although we cannot directly project these examples to
the more than one thousand business cases developed each year across
the federal government, our results suggest that the issues raised need
attention. The shortcomings in guidance and training are likely to be
widespread, and so the weaknesses may extend beyond the specific
examples identified here. The kinds of weaknesses displayed and the
causes behind them are consistent with the pervasive problems with
project and investment management that we have documented in numerous
prior reports.
The absence of documentary support in the cases reviewed raises
questions regarding the sufficiency of the justification provided for
the investment and undermines the management decisions being made based
on it. More troubling, it may indicate an underlying weakness in the
management of the investment, particularly since several sections of
the exhibit are specifically designed to capture information from
systems used in project management, such as those that support EVM and
financial management. In many cases, inadequate support raises
questions regarding the adequacy of an agency's management processes
and internal controls, which strongly affect the reliability of the
information presented to decision makers. Further, in view of the
inaccuracies in the cases reviewed, it is evident that agencies are not
taking sufficient actions to ensure the accuracy of the information in
the exhibit 300s. To make reasonable decisions, management needs to be
aware of limitations in the data on which they rely and thus be able to
take steps to mitigate the risks involved.
Collectively, our findings raise questions on whether fundamental
project management processes are in place, whether project managers are
adequately trained in these processes, and whether they receive
sufficient guidance on these processes and on preparing all areas of
the exhibit 300. At a minimum, this situation undermines the usefulness
of the exhibit 300 as a mechanism to communicate to OMB and agency
executives that the project team has employed the disciplines of good
project management. By reporting information that is not supported by
documentation, the exhibit 300 can create the misleading appearance
that investments are being managed properly, when in fact they are not.
In addition, OMB has relied on these exhibits to identify and oversee
high-risk projects; thus, our finding that the data being presented to
OMB may not be reliable or accurate further complicates its oversight.
While OMB is applying more rigor to its oversight processes through
such processes as its tracking of high-risk investments, these advances
may be undermined by inaccurate or unreliable data used in decision
making. Unless these weaknesses are addressed, OMB, agency executives,
and Congress will not have assurance that key decisions to pursue and
oversee the $65 billion in IT investments are being made based on
accurate and reliable information.
Recommendations for Executive Action:
To improve the accuracy and validity of exhibit 300s for major IT
investments and to increase the value of using the information they
provide in selection, oversight, and resource allocation decisions, we
are making three recommendations.
1. Because decision makers should be aware of any weaknesses in the
processes used to develop the information in the exhibit 300s, we are
recommending that the Director of OMB direct agencies to determine the
extent to which the information contained in each exhibit 300 is
accurate and reliable. Where weaknesses in accuracy and reliability are
identified, the agency should be required to disclose them and explain
the agency's approach to mitigating them.
In addition, to help ensure that agency personnel completing exhibit
300s better understand their responsibilities, we recommend that the
Director of OMB take the following additional actions:
2. In advance of OMB's next issuance of the Circular A-11 update,
develop and promulgate clearer and more explicit guidance for sections
of the exhibit 300 business case that cause confusion, including
addressing weaknesses identified in this report (as indicated below)
and consulting with agency personnel having responsibility for
completing exhibit 300s across government to identify other areas of
confusion. At a minimum, the guidance should do the following:
* Provide a more detailed description of the requirements for
completing an operational analysis, as referred to in the supplement to
Part 7 of Circular A-11, the Capital Programming Guide.
* Address or clarify possible flexibilities and alternative approaches
available to agencies in completing their exhibit 300s: for example,
whether the analysis of alternatives section of the exhibit 300 needs
to be updated every year for steady state investments and whether all
risk areas are relevant for all investments.
3. Provide for training of agency personnel responsible for completing
exhibit 300s. This training should go beyond a description of changes
from prior years' guidance and include working through examples for a
variety of investments. In developing the training, OMB should consult
with agencies to identify deficiencies that the training should
address.
In implementing these recommendations, OMB should work with the CIO
Council to develop the necessary guidance and implement an effective
training program to ensure governmentwide acceptance of these changes.
Because we have outstanding recommendations aimed at enhancing OMB's
audit guidance related to federal cost-accounting systems,[Footnote 36]
we are not making any new recommendations in this report regarding
federal cost accounting.
Agency Comments and Our Evaluation:
We provided a draft of this report to OMB and the five agencies whose
exhibit 300s we reviewed.
In written comments received on December 23, 2005, the Administrator of
OMB's Office for E-Government and Information Technology accepted the
findings of the draft report. OMB described two of our three
recommendations and expressed three concerns: first, that our report
does not address the need for agencies to ensure the accuracy of their
IT investment requests; second, that the report focuses on the way
agency employees fill out OMB's exhibit 300s and not on the underlying
management responsibilities; and third, that by directing our
recommendations to OMB rather than to the agencies, we could be seen as
suggesting that OMB and not the agencies are responsible for data
accuracy and employee training.
OMB's concern regarding data accuracy is addressed by our first
recommendation: that the Director of OMB instruct agencies to determine
the extent to which the information contained in each exhibit 300 is
accurate and reliable, to disclose weaknesses, and to describe the
agency's approach to mitigating these weaknesses. This recommendation
clearly places responsibility on the agencies for assessing the quality
of their budget information and the processes that produced this
information.
With respect to OMB's concern that the recommendations do not focus on
how well agencies fulfill their underlying information resources
management responsibilities, our view is that our recommendation on
disclosing and mitigating weaknesses does address these underlying
responsibilities. The report specifically addresses the exhibit 300s
and the reliability of these documents when used as support in the
agencies' and OMB's decision-making processes. As our report clearly
states, the lack of documentation may indicate an underlying weakness
in the management of the investment. In many cases inadequate support
raises questions about the investments' program management and internal
controls. Requiring agencies to disclose and mitigate associated
weaknesses presupposes that agencies examine and address their approach
to fulfilling information resources management responsibilities.
Regarding OMB's third concern, we do not intend to suggest that
agencies are not responsible and accountable for the weaknesses we
describe. We place significant responsibility on agencies to manage
their information assets effectively, as reflected in our first
recommendation and in the large number of evaluations that we have
previously conducted at individual agencies and the recommendations
resulting, some of which are still outstanding. In this report,
however, our recommendations are directed to OMB because they address
findings relating to OMB-required budget documents, and OMB has
statutory responsibility for providing information resources management
guidance governmentwide.
Regarding OMB's comment that agencies be held responsible for employee
training in information resources management, we agree that agencies
are responsible for such training. However, as agencies indicated
during the review, additional training by OMB would be helpful,
especially in the understanding of OMB's requirements for the exhibit
300. This is also consistent with OMB's responsibility under the E-
Government Act of 2002[Footnote 37] to identify where current training
does not satisfy the personnel needs related to information technology
and information resource management.
The Deputy Associate Chief Information Officer for Information
Technology Reform of the Department of Energy provided largely
technical comments, which we incorporated as appropriate. The Director
of Audit Relations of the Department of Transportation also provided
technical comments that were incorporated as appropriate. The
Departments of Agriculture, Commerce, Transportation, and the Treasury
provided no comments.
The written comments from OMB are reproduced in appendix II.
As agreed with your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies of this report
to the Secretaries of the Departments of Agriculture, Commerce, Energy,
Transportation, and the Treasury and the Director of Office of
Management and Budget. We will also make copies available to others
upon request. In addition, this report will be available at no charge
on the GAO Web site at [Hyperlink, http://www.gao.gov].
If you have any questions on matters discussed in this report, please
contact me at (202) 512-9286. I can also be reached by e-mail at
[Hyperlink, pownerd@gao.gov]. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Other contacts and key contributors to this report
are listed in appendix IV.
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
[End of section]
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Our objective was to ascertain the extent to which selected agencies
have underlying support for the information described in their fiscal
year 2006 exhibit 300s as submitted to the Office of Management and
Budget (OMB) in September 2004. To address our objective, we reviewed
the supporting documentation for 29 exhibit 300s from agencies and
components from the Departments of Agriculture, Commerce, Energy,
Transportation, and the Treasury.[Footnote 38]
We selected the five departments for our review on the basis of two
criteria. First, to ensure that we examined significant investments, we
selected departments that expected to spend $1 billion or more on
information technology (IT) investments in fiscal year 2006. Second, of
those agencies with significant investments, we further narrowed our
selection of agencies to those with the first and second largest number
of IT investments in each of three categories of the federal
government's Business Reference Model (BRM): Services for
Citizens,[Footnote 39] Support Delivery of Services,[Footnote 40] and
Management of Government Resources.[Footnote 41] We did this to ensure
that the agencies under review reflect the primary business operations
performed by the federal government. We excluded the Mode of Delivery
Business Area because we found investments in this area to be largely
from one agency, the Department of Defense (DOD). (In general, Mode of
Delivery describes the mechanisms the government uses to achieve its
purposes: Services for Citizens.) (We excluded DOD and the Department
of Homeland Security (DHS) from our selection, because the Defense
Inspector General recently performed an extensive review of exhibit
300s,[Footnote 42] and we have both completed and ongoing work on
several major IT investments at DHS).[Footnote 43] This process
resulted in the selection of the five departments mentioned above.
To make our selection of IT investments from the five departments, we
used OMB capital planning and budget documentation to identify a mix of
investments. Specifically, we chose IT investments that (1) supported
government operations across each of the three BRM business areas
identified above and (2) reflected different stages of investment
(e.g., new, mixed life cycle, and steady state). Initially, we selected
a total of 30 investments (i.e., 6 investments from each department).
However, one IT investment was dropped from our total of 30 selected
investments because we determined during our review that OMB and the
agency had cancelled its funding.
To determine the extent of each investment's underlying support, we
developed a set of questions regarding the types of analysis and
documentation that were associated with the information provided in
each of the major sections of OMB's exhibit 300.[Footnote 44] Using our
set of questions, we met with agency officials for each selected
investment to collect and analyze investment documentation associated
with each exhibit 300 area in our evaluation. We further compared the
documentation against the exhibit 300 to ascertain whether the
documentation agreed with what the investment reported in the exhibit
300. Where federal requirements, laws, and other guidelines were cited
in Circular A-11, we also used these to assess the extent to which
agencies and components had complied with specific documentation
requirements as prescribed in these sources (including National
Institute of Standards and Technology (NIST) guidance, OMB circulars,
and OMB memorandums).
In areas where federal directives were cited in the exhibit 300, we
conducted limited reliability testing; these areas included security,
analysis of alternatives, and the project and funding plan. In our
evaluation of security documentation, we used criteria set forth in
NIST guidance to assess whether the major components were present in
key documents, which included the security plan and system-level
incident handling procedures. For security awareness training, we
identified whether training was conducted and tracked but did not
assess its content. In our evaluation of the analysis of alternatives,
we used criteria from OMB Circular A-94 to assess whether the major
components were present in the cost-benefit or cost-effectiveness
analysis. In cases where investment managers told us that their earned
value management (EVM) processes were in conformance with ANSI/EIA-748-
A (for our evaluation of the project and funding plan sections), we
used criteria from ANSI/EIA-748-A to assess whether key EVM processes
were in place. We did not test the quality of the documentation in
these areas of evaluation.
Regarding the reliability of cost data, we did not test the adequacy of
agency or contractor cost-accounting systems. Our evaluation was based
on what we were told by the agencies and the information they could
provide (to the extent to which they had information).
We performed our work at the agencies' offices in the Washington, D.C.,
metropolitan area. We conducted our review between March and November
2005 in accordance with generally accepted government auditing
standards.
[End of section]
Appendix II: Comments from the Office of Management and Budget:
EXECUTIVE OFFICE OF THE PRESIDENT:
OFFICE OF MANAGEMENT AND BUDGET:
WASHINGTON, D. C. 20503:
Mr. David A. Powner:
Director:
IT Management Issues:
Government Accountability Office:
441 G Street, SW:
Washington, DC 20548:
Dear Mr. Powner:
Thank you for the opportunity to comment on the Government
Accountability Office's (GAO) draft report titled, "Information
Technology: Agencies Need to Improve the Accuracy and Reliability of
Investment Management" (GAO-06-250).
We appreciate GAO's effort to determine the extent to which selected
agencies have underlying support for their justifications for
investments in information technology (Exhibit 300s). In the draft
report, GAO made two recommendations to the Office of Management and
Budget (OMB).
First, the report recommends in its next update of Circular A-11, OMB
provide clearer and more explicit guidance for those areas of the
Exhibit 300s some agency officials found confusing. Specifically, the
report suggests a more detailed description of the requirements for
completing an operational analysis and address possible flexibilities
and alternative approaches available to the agencies in completing
their Exhibit 300s including whether certain sections must be updated
every year.
Second, the report recommends OMB provide training of agency personnel
responsible for completing Exhibit 300s and work with the CIO Council
to do so. This recommendation suggests the training go beyond a
description of changes from prior year's guidance.
We accept the findings in the draft report, but have significant
concerns with the focus of the resulting recommendations. We are also
concerned the draft report does not address the primary way agencies
can ensure the accuracy and appropriateness of their IT investment
requests, i.e., through reviews by internal investment review boards or
other similar reviews by senior agency personnel.
With respect to the draft report's recommendations, we believe they are
inconsistent with the report's findings because they focus on improving
the way agency employees prepare a form (i.e., the Exhibit 300) and not
how well agencies fulfill their underlying Information Resources
Management (IRM) responsibilities found in law and policy.
Moreover, by directing the recommendations solely to OMB, the draft
report can be viewed as suggesting OMB and not the agencies are
responsible and accountable for data accuracy and employee training. In
fact, it could encourage agencies to wait to address any deficiencies
in agency programs until OMB takes some specific action in response to
the recommendations. We assume this is not the intent of the report as
employee IRM training is prominently featured as an agency
responsibility in GAO's Executive Guide for Information Technology
Investment Management as well as in both the Clinger-Cohen Act of 1996
and the Paperwork Reduction Act of 1995.
Therefore, to the extent certain agency employees are uncertain of
their IRM responsibilities and how to perform them (and thus are unable
to properly complete the Exhibit 300 form), each agency must take the
appropriate action including ensuring agency employees take advantage
of the many training opportunities available such as those identified
on the CIO Council's website and participate in the CIO IT Workforce
Committee. Past activities of the Committee include the development of
an IT Workforce Capability Guidance, issuance of IT training guidance,
development of an IT Roadmap, the hosting of IT quarterly forums, and
the IT Exchange Program.
Finally on this point, we suggest the draft report be clarified to
place greater emphasis on agency responsibilities to train their
employees and include associated recommendations to the agencies.
As your draft report suggests, OMB will continue to work with the
agencies and the CIO Council to help improve agency employee
understanding of their IRM responsibilities including planning for
information technology projects. Clearly, those employees who
understand their responsibilities are far better equipped to accurately
complete Exhibit 300 investment justification forms. As we do each year
in preparing OMB Circular A-11, we will work with the agencies and the
CIO Council to identify the extent to which additional guidance is
necessary and the appropriate form of such guidance. I would like to
note however, at the request of agency CIOs, for the past two years OMB
did not make significant changes to those portions of Circular A-11
addressing IT investments. The CIOs expressed their understanding of
the guidance and suggested to us stability would promote improved data
quality.
Thank you for the opportunity to review and comment on your draft
report on this important issue.
Sincerely,
Signed by:
Karen S. Evans:
Administrator for E-Government And Information Technology:
Office of Management and Budget:
[End of section]
Appendix III: Descriptions of Investments Reviewed:
The following provides additional detail on the agencies and
investments that we reviewed as part of this audit. We reviewed a total
of 29 investments at five departments: Agriculture, Commerce, Energy,
Transportation, and the Treasury.[Footnote 45] The selected departments
account for the first and second largest number of IT investments in
each of three categories of the federal government's Business Reference
Model: Services for Citizens,[Footnote 46] Support Delivery of
Services,[Footnote 47] and Management of Government Resources.[Footnote
48]
According to OMB guidance, the life-cycle stage of the asset affects
what is reported on the exhibit 300:
* New investments (i.e., proposed for budget year or later, or in
development) must be justified based on the need to fill a gap in the
agency's ability to meet strategic goals and objectives with the lowest
life-cycle costs of all possible alternatives and must provide risk-
adjusted cost and schedule goals and measurable performance benefits.
* Mixed life-cycle investments (i.e., investments that are operational
but include some developmental effort, such as a technology refresh)
must demonstrate satisfactory progress toward achieving baseline cost,
schedule, and performance goals using an EVM system.[Footnote 49]
* Operational investments (i.e., steady state) must demonstrate, among
other things, how close actual annual operating and maintenance costs
are to the original life-cycle cost estimates, whether the technical
merits of the investment continue to meet the needs of the agency and
customers, and that an analysis of alternatives was performed with a
future focus.
Department of Agriculture:
System: Comprehensive Electronic Permit System (ePermits):
Brief description: This system is expected to automate processes to
allow the Department of Agriculture to issue, track, and rapidly verify
the validity of a federal permit allowing the importation of plants and
animals. It is also expected to assist the public by allowing
applicants to apply for permits, check the status of permit
applications, and receive permits online.
Investment stage: New:
Business Reference Model category: Services for Citizens:
Table 2: Financial Funding Data for ePermits:
Millions of dollars.
Fiscal year: FY 04;
Total: $4.8;
Development: $4.8;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $3.3;
Development: $3.3;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $2.2;
Development: $2.2;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Corporate Financial Management Systems (CFMS):
Brief description: This investment is designed to represent the entire
portfolio of current corporate financial management and administrative
payment systems for the department. It is a corporatewide solution for
financial management reform and systems integration that provides tools
for program and financial managers to manage and evaluate federal
programs.
Investment stage: Steady state:
Business Reference Model category: Management of Government Resources:
Table 3: Financial Funding Data for CFMS:
Millions of dollars.
Fiscal year: FY 04;
Total: $64.6;
Development: $0.0;
Steady state: $64.6.
Fiscal year: FY 05;
Total: $62.0;
Development: $0.0;
Steady state: $62.0.
Fiscal year: FY 06;
Total: $61.7;
Development: $0.0;
Steady state: $61.7.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Integrated Acquisition System (IAS):
Brief description: This system is intended to be a single
enterprisewide acquisition management system to support a strategic and
more standardized acquisition management process for Agriculture. It is
expected to provide a real-time interface to the department's core
financial system, reliable data, and a shortened time for acquiring
goods and services.
Investment stage: Mixed life cycle:
Business Reference Model category: Management of Government Resources:
Table 4: Financial Funding Data for IAS:
Millions of dollars.
Fiscal year: FY 04;
Total: $35.9;
Development: $20.5;
Steady state: $15.4.
Fiscal year: FY 05;
Total: $27.1;
Development: $11.6;
Steady state: $15.5.
Fiscal year: FY 06;
Total: $30.6;
Development: $14.5;
Steady state: $16.1.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Phytosanitary Certificate Issuance and Tracking System (PCIT):
Brief description: This system is expected to establish a new process
to collect and track phytosanitary certificates issued by the
department, which attest to compliance with import regulations of
importing countries. It is also intended to provide better service to
users by reducing the need for repetitive data entry from applicants
and enabling certifying officials to deliver certificates in a timelier
manner.
Investment stage: New:
Business Reference Model category: Services for Citizens:
Table 5: Financial Funding Data for PCIT:
Millions of dollars.
Fiscal year: FY 04;
Total: $0.9;
Development: $0.9;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $3.6;
Development: $3.6;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $2.5;
Development: $2.5;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Processed Commodity Inventory Management System (PCIMS):
Brief description: This system is designed to support the annual
acquisition, tracking, and distribution of commodities acquired by
Agriculture for domestic and foreign food assistance programs by
providing financial and program management, reporting, and control to
track commodity requests against purchases and distributions from
inventory.
Investment stage: Steady state:
Business Reference Model category: Management of Government Resources:
Table 6: Financial Funding Data for PCIMS:
Millions of dollars.
Fiscal year: FY 04;
Total: $12.9;
Development: $4.3;
Steady state: $8.6.
Fiscal year: FY 05;
Total: $12.2;
Development: $3.7;
Steady state: $8.6.
Fiscal year: FY 06;
Total: $12.7;
Development: $2.8;
Steady state: $9.9.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Store Tracking and Redemption System Redesign (STARS II):
Brief description: This system is intended to support the department's
Food-Stamp program mission by tracking and monitoring food
coupon/electronic benefit redemption activities and regulatory
violations by businesses and associated administrative actions related
to enforcement of penalties, among other things. This initiative is
expected to replace the current legacy system, which has been in place
since 1993.
Investment stage: New:
Business Reference Model category: Support Delivery of Services:
Table 7: Financial Funding Data for STARS II:
Millions of dollars.
Fiscal year: FY 04;
Total: $12.7;
Development: $12.4;
Steady state: $0.3.
Fiscal year: FY 05;
Total: $7.0;
Development: $5.2;
Steady state: $1.8.
Fiscal year: FY 06;
Total: $6.4;
Development: $0.0;
Steady state: $6.4.
Source: OMB FY2006 Exhibit 53.
[End of table]
Department of Commerce:
System: Advanced Weather Interactive Processing System (AWIPS):
Brief description: This system is designed to be an interactive
computer system that integrates all meteorological and hydrological
data and all satellite and radar data to enable the forecaster to
prepare and issue more accurate and timely forecasts and warnings.
Investment stage: Mixed life cycle:
Business Reference Model category: Services for Citizens:
Table 8: Financial Funding Data for AWIPS:
Millions of dollars.
Fiscal year: FY 04;
Total: $49.2;
Development: $14.0;
Steady state: $35.2.
Fiscal year: FY 05;
Total: $49.5;
Development: $12.7;
Steady state: $36.8.
Fiscal year: FY 06;
Total: $52.2;
Development: $14.1;
Steady state: $38.1.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Comprehensive Large Array-Data Stewardship System (CLASS):
Brief description: This system is expected to provide an integrated
solution to weather and water data archive and access, including an
access portal with search, browse, and geospatial capabilities for
users to obtain environmental data, contributing to improvements in
prediction capabilities.
Investment stage: New:
Business Reference Model category: Services for Citizens:
Table 9: Financial Funding Data for CLASS:
Millions of dollars.
Fiscal year: FY 04;
Total: $3.2;
Development: $3.2;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $9.6;
Development: $9.6;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $7.6;
Development: $7.6;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Economic Census and Surveys (ECON):
Brief description: This investment is designed to provide statistical
programs that count and profile U.S. businesses and government
organizations through the gathering of surveys and principal economic
indicators in order to conduct research and technical studies.
Investment stage: Steady state:
Business Reference Model category: Support Delivery of Services:
Table 10: Financial Funding Data for ECON:
Millions of dollars.
Fiscal year: FY 04;
Total: $53.6;
Development: $0.0;
Steady state: $53.6.
Fiscal year: FY 05;
Total: $34.0;
Development: $0.0;
Steady state: $34.0.
Fiscal year: FY 06;
Total: $51.4;
Development: $0.0;
Steady state: $51.4.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Improve the Automated Export System (AES):
Brief description: The current system is designed to expedite monthly
statistics on international trade, remedy shortcomings in export
statistics, and help to control the export of weapons or other
hazardous items that could be a threat to U.S. national security or
public welfare. The proposed initiative is designed to improve the
current system to handle electronic filing of all export transactions,
incorporate an electronic manifest system, and provide for verification
of export information reported on export transactions.
Investment stage: Mixed life cycle:
Business Reference Model category: Support Delivery of Services:
Table 11: Financial Funding Data for AES:
Millions of dollars.
Fiscal year: FY 04;
Total: $0.0;
Development: $0.0;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $0.0;
Development: $0.0;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $6.6;
Development: $6.6;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: National Weather Service Telecommunication Gateway (NWSTG)
System:
Brief description: The current system is designed to collect and
distribute raw and processed hydrometeorological data and products,
disseminating weather observations and guidance to a national and
international community of customers. Improvements to current system
are expected to provide sufficient performance, capacity, and
catastrophic backup capability to meet current and future demands for
data.
Investment stage: Mixed life cycle:
Business Reference Model category: Support Delivery of Services:
Table 12: Financial Funding Data for NWSTG:
Millions of dollars.
Fiscal year: FY 04;
Total: $13.2;
Development: $5.5;
Steady state: $7.7.
Fiscal year: FY 05;
Total: $13.0;
Development: $5.0;
Steady state: $8.0.
Fiscal year: FY 06;
Total: $11.1;
Development: $0.8;
Steady state: $10.3.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Satellite Operations Control Center Command and Data
Acquisition (SOCC/CDA):
Brief description: This system is designed to command and control
Commerce's operational environmental satellites and to acquire and
manage the weather and water data the satellites collect, in order to
provide support functions that are not available commercially, such as
real-time hurricane support.
Investment stage: Mixed life cycle:
Business Reference Model category: Services for Citizens:
Table 13: Financial Funding Data for SOCC/CDA:
Millions of dollars.
Fiscal year: FY 04;
Total: $11.7;
Development: $11.7;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $10.5;
Development: $6.1;
Steady state: $4.4.
Fiscal year: FY 06;
Total: $12.6;
Development: $7.7;
Steady state: $5.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
Department of Energy:
System: Energy Sciences Network (ESnet):
Brief description: This project is designed to support scientific
research by providing an interoperable, effective, and reliable
communications infrastructure and network services to the Department of
Energy research facilities.
Investment stage: Mixed life cycle:
Business Reference Model category: Services for Citizens:
Table 14: Financial Funding Data for ESnet:
Millions of dollars.
Fiscal year: FY 04;
Total: $18.9;
Development: $11.3;
Steady state: $7.6.
Fiscal year: FY 05;
Total: $18.3;
Development: $9.5;
Steady state: $8.8.
Fiscal year: FY 06;
Total: $18.3;
Development: $9.3;
Steady state: $9.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: E-content Management System (eCMS):
Brief description: This system is expected to be an enterprisewide,
integrated document and records management system that will include
portal accessibility and integration with knowledge management tools in
order to improve decision and service delivery quality and serve as a
resource for operations management.
Investment stage: New:
Business Reference Model category: Support Delivery of Services:
Table 15: Financial Funding Data for eCMS:
Millions of dollars.
Fiscal year: FY 04;
Total: $2.3;
Development: $2.3;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $2.7;
Development: $2.7;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $6.4;
Development: $4.1;
Steady state: $2.3.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Integrated Planning, Accountability, and Budgeting System
Information System (IPABS-IS):
Brief description: This system is designed to support the routine
collection and reporting needs of Energy for life-cycle planning,
budget formulation, and project and budget execution.
Investment stage: Steady state:
Business Reference Model category: Support Delivery of Services:
Table 16: Financial Funding Data for IPABS-IS:
Millions of dollars.
Fiscal year: FY 04;
Total: $3.1;
Development: $0.0;
Steady state: $3.1.
Fiscal year: FY 05;
Total: $2.9;
Development: $0.0;
Steady state: $2.9.
Fiscal year: FY 06;
Total: $2.8;
Development: $0.0;
Steady state: $2.8.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Licensing Support Network (LSN):
Brief description: This is a Web-based system that is intended to make
relevant documentary material supporting the Nuclear Regulatory License
Application available to users, as part of the requirements of the
Nuclear Waste Policy Act.
Investment stage: Steady state:
Business Reference Model category: Services for Citizens:
Table 17: Financial Funding Data for LSN:
Millions of dollars.
Fiscal year: FY 04;
Total: $39.6;
Development: $39.6;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $13.6;
Development: $0.0;
Steady state: $13.6.
Fiscal year: FY 06;
Total: $9.6;
Development: $0.0;
Steady state: $9.6.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Los Alamos National Laboratory Enterprise Project (LANL ERP):
Brief description: This investment is intended to identify, design, and
implement the systems, processes, and controls related to financial
management, human resources, supply chain management, facilities
maintenance, information management, project management, and
manufacturing in order to lower costs and provide more efficient
operations and improved management.
Investment stage: Mixed life cycle:
Business Reference Model category: Management of Government Resources:
Table 18: Financial Funding Data for LANL ERP:
Millions of dollars.
Fiscal year: FY 04;
Total: $43.2;
Development: $39.3;
Steady state: $3.9.
Fiscal year: FY 05;
Total: $45.5;
Development: $41.3;
Steady state: $4.2.
Fiscal year: FY 06;
Total: $43.7;
Development: $41.3;
Steady state: $2.4.
Source: OMB FY2006 Exhibit 53.
[End of table]
Department of Transportation:
System: Asset Supply Chain Management (ASCM):
Brief description: This investment is intended to provide the
Department of Transportation (DOT) with asset management and supply
chain management information systems to track and manage over $21
billion in federal government assets. Reducing the number of
information systems, optimizing supply chain operations, and
streamlining business operations of employees are expected to result in
reduced costs to the agency.
Investment stage: Mixed life cycle:
Business Reference Model category: Management of Government Resources:
Table 19: Financial Funding Data for ASCM:
Millions of dollars.
Fiscal year: FY 04;
Total: $5.0;
Development: $5.0;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $6.0;
Development: $6.0;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $13.2;
Development: $13.2;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: DOT Financial System Consolidation:
Brief description: This program is expected to consolidate several
major and nonmajor DOT financial systems to interface or integrate all
related systems in order to eliminate redundant data and processes.
Investment stage: New:
Business Reference Model category: Management of Government Resources:
Table 20: Financial Funding Data for DOT Financial System
Consolidation:
Millions of dollars.
Fiscal year: FY 04;
Total: $51.0;
Development: $6.2;
Steady state: $44.8.
Fiscal year: FY 05;
Total: $64.5;
Development: $11.1;
Steady state: $53.3.
Fiscal year: FY 06;
Total: $44.1;
Development: $0.2;
Steady state: $44.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: National Transit Database (NTD):
Brief description: This system is designed to collect performance data
from over 640 local transit agencies for the purpose of reporting
statistical data on the U.S. transit industry.
Investment stage: Steady state:
Business Reference Model category: Services for Citizens:
Table 21: Financial Funding Data for NTD:
Millions of dollars.
Fiscal year: FY 04;
Total: $2.2;
Development: $0.0;
Steady state: $2.2.
FY 05;
Total: $3.7;
Development: $0.0;
Steady state: $3.7.
Fiscal year: FY 06;
Total: $3.7;
Development: $0.0;
Steady state: $3.7.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Next Generation Air/Ground Communications (NEXCOM):
Brief description: This system is intended to provide air
pilot/controller voice and data communications by utilizing a digital-
based air/ground communication system.
Investment stage: New:
Business Reference Model category: Services for Citizens:
Table 22: Financial Funding Data for NEXCOM:
Millions of dollars.
Fiscal year: FY 04;
Total: $28.7;
Development: $28.7;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $29.5;
Development: $29.5;
Steady state: $0.1.
Fiscal year: FY 06;
Total: $33.8;
Development: $33.5;
Steady state: $0.3.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: System Approach for Safety Oversight (SASO):
Brief description: This system is expected to consolidate the agency's
28 oversight systems on aviation regulatory compliance into 5
integrated aviation safety risk management systems. Its intended
purpose is to allow applicable government agencies and the aviation
industry to use common system safety applications and databases for
managing and overseeing flight safety.
Investment stage: New:
Business Reference Model category: Services for Citizens:
Table 23: Financial Funding Data for SASO:
Millions of dollars.
Fiscal year: FY 04;
Total: $7.2;
Development: $7.2;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $7.3;
Development: $7.3;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $10.7;
Development: $10.7;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Wide-Area Augmentation System (WAAS):
Brief description: This is a navigation system that is designed to
provide navigation across the entire United States for all classes of
aircraft in all flight operations, including en-route navigation,
airport departures, and airport arrivals including precision landing
approaches in all weather conditions.
Investment stage: Mixed life cycle:
Business Reference Model category: Services for Citizens:
Table 24: Financial Funding Data for WAAS:
Millions of dollars.
Fiscal year: FY 04;
Total: $105.6;
Development: $99.4;
Steady state: $6.2.
Fiscal year: FY 05;
Total: $122.6;
Development: $99.2;
Steady state: $23.4.
Fiscal year: FY 06;
Total: $124.4;
Development: $100.0;
Steady state: $24.4.
Source: OMB FY2006 Exhibit 53.
[End of table]
Department of the Treasury:
System: Customer Account Data Engine (CADE):
Brief description: This system is part of a modernization program that
is expected to provide the Department of the Treasury with the
capability to manage its tax accounts utilizing new technology,
applications, and databases. This system is designed to create
applications for daily posting, settlement, maintenance, refund
processing, and issue detection for taxpayer tax account and return
data to improve customer service and compliance.
Investment stage: Mixed life cycle:
Business Reference Model category: Support Delivery of Services:
Table 25: Financial Funding Data for CADE:
Millions of dollars.
Fiscal year: FY 04;
Total: $100.6;
Development: $100.6;
Steady state: $0.0.
Fiscal year: FY 05;
Total: $109.9;
Development: $109.9;
Steady state: $0.0.
Fiscal year: FY 06;
Total: $109.9;
Development: $109.9;
Steady state: $0.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Debt Management Accounting System (DMAS):
Brief description: This system is designed to be a financial accounting
system for activities associated with Treasury's debt collection
program to track funds recovered by the agency, post these funds to the
proper account in an accurate and timely manner, and transfer moneys
due to the appropriate government agencies. The system is also designed
to record the general ledger activity and produce operational,
management, and standard external reports.
Investment stage: Mixed life cycle:
Business Reference Model category: Support Delivery of Services:
Table 26: Financial Funding Data for DMAS:
Millions of dollars.
Fiscal year: FY 04;
Total: $4.4;
Development: $2.3;
Steady state: $2.1.
Fiscal year: FY 05;
Total: $4.1;
Development: $1.8;
Steady state: $2.3.
Fiscal year: FY 06;
Total: $4.2;
Development: $1.9;
Steady state: $2.4.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Electronic Management System (EMS):
Brief description: This system is designed to be a front-end processing
system that receives, validates, stores, forwards to mainframe
electronic filing systems, and acknowledges electronic files containing
tax documents. The system is intended to receive returns from third
parties, acknowledge the receipt of information, format the information
for mainframe processing, provide acknowledgements to the third
parties, and send state return data to participating states.
Investment stage: Steady state:
Business Reference Model category: Management of Government Resources:
Table 27: Financial Funding Data for EMS:
Millions of dollars.
Fiscal year: FY 04;
Total: $11.8;
Development: $3.1;
Steady state: $8.8.
Fiscal year: FY 05;
Total: $9.1;
Development: $2.0;
Steady state: $7.1.
Fiscal year: FY 06;
Total: $10.3;
Development: $2.0;
Steady state: $8.2.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Governmentwide Accounting and Reporting Modernization (GWA):
Brief description: This system is designed to produce accurate,
accessible, and timely governmentwide financial information through the
streamlining of reports and the reduction of the reconciliation burden
on government agencies in order to minimize the amount of labor
necessary to transfer financial information.
Investment stage: Mixed life cycle:
Business Reference Model category: Management of Government Resources:
Table 28: Financial Funding Data for GWA:
Millions of dollars.
Fiscal year: FY 04;
Total: $7.7;
Development: $5.6;
Steady state: $2.1.
Fiscal year: FY 05;
Total: $7.8;
Development: $5.5;
Steady state: $2.3.
Fiscal year: FY 06;
Total: $7.8;
Development: $5.3;
Steady state: $2.5.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Service Center Recognition/Images Processing System (SCRIPS):
Brief description: This system is intended to be a data capture,
management, and storage system used to process tax documents
automatically in order to meet mandated timelines and processing
requirements for various tax forms and the Federal Tax Deposits, which
directly impacts revenue brought into the federal treasury.
Investment stage: Steady state:
Business Reference Model category: Support Delivery of Services:
Table 29: Financial Funding Data for SCRIPS:
Millions of dollars.
Fiscal year: FY 04;
Total: $13.0;
Development: $0.0;
Steady state: $13.0.
Fiscal year: FY 05;
Total: $13.7;
Development: $0.0;
Steady state: $13.7.
Fiscal year: FY 06;
Total: $15.0;
Development: $0.0;
Steady state: $15.0.
Source: OMB FY2006 Exhibit 53.
[End of table]
System: Secure Payment System (SPS):
Brief description: This system is designed to be a browser-based
Internet version of the current Electronic Certification System, which
will allow federal program agencies to submit certified requests for
payment disbursement online. It is intended to provide a more secure
payment process, increase the ability to protect sensitive financial
and privacy data, and improve the financial performance of federal
program agencies by providing program agencies a method of providing
financial data to Treasury.
Investment stage: Steady state:
Business Reference Model category: Support Delivery of Services:
Table 30: Financial Funding Data for SPS:
Millions of dollars.
Fiscal year: FY 04;
Total: $4.3;
Development: $0.0;
Steady state: $4.3.
Fiscal year: FY 05;
Total: $3.1;
Development: $0.0;
Steady state: $3.1.
Fiscal year: FY 06;
Total: $3.1;
Development: $0.0;
Steady state: $3.1.
Source: OMB FY2006 Exhibit 53.
[End of table]
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner, (202) 512-9286, [Hyperlink, pownerd@gao.gov].
Acknowledgments:
In addition to the contact named above, the following people made key
contributions to this report: Carol Cha, Barbara Collier, Joseph Cruz,
Lester Diamond, Valerie Hopkins, Sandra Kerr, Linda Lambert, Tammi
Nguyen, Chris Owens, Mark Shaw, Kevin Walsh, and Martin Yeung.
(310479):
FOOTNOTES
[1] OMB Circular No. A-11, Preparation, Submission, and Execution of
the Budget, Part 7 (July 2004).
[2] An analysis of alternatives compares viable alternative solutions
and includes a general analysis of the benefits for each alternative
presented.
[3] The five departments were the Departments of Agriculture, Commerce,
Energy, Transportation, and the Treasury.
[4] EVM is a project management tool that integrates the investment
scope of work with schedule and cost elements for investment planning
and control. This method compares the value of work accomplished during
a given period with that of the work expected in the period.
Differences in expectations are measured in both cost and schedule
variances. OMB requires agencies to use EVM as part of their
performance-based management system for any investment under
development or with system improvements under way.
[5] 44 U.S.C. § 3504(a)(1)(B)(vi) (OMB); 44 U.S.C. § 3506(h)(5)
(agencies).
[6] 40 U.S.C. § 11312; 40 U.S.C. § 11313.
[7] These requirements are specifically described in the Clinger-Cohen
Act, 40 U.S.C. § 11302(c).
[8] GAO, Information Technology: OMB Can Make More Effective Use of Its
Investment Reviews, GAO-05-276 (Washington, D.C.: Apr. 15, 2005).
[9] OMB Memorandum, M-05-23 (Aug. 4, 2005).
[10] 41 U.S.C. § 263.
[11] 5 U.S.C. § 306; 31 U.S.C. § 1115.
[12] 44 U.S.C. § 3544 (a)(1)(C).
[13] 44 U.S.C. § 3603.
[14] Chief Information Officers Council, Smart Practices in Capital
Planning (October 2000); A Summary of First Practices and Lessons
Learned in Information Technology Portfolio Management (Washington,
D.C.: March 2002); and Value Measuring Methodology (Washington, D.C.:
October 2002).
[15] GAO, Information Technology Investment Management: A Framework for
Assessing and Improving Process Maturity, GAO-04-394G (Washington,
D.C.: March 2004).
[16] During the selection phase, the organization (1) identifies and
analyzes each project's risks and returns before committing significant
funds to any project and (2) selects those IT projects that will best
support its mission needs.
[17] During the control phase, the organization ensures that, as
projects develop and investment expenditures continue, the project is
continuing to meet mission needs at the expected levels of cost and
risk. If the project is not meeting expectations or if problems have
arisen, steps are quickly taken to address the deficiencies.
[18] During the evaluation phase, actual versus expected results are
compared once projects have been fully implemented. This is done to (1)
assess the project's impact on mission performance, (2) identify any
changes or modifications to the project that may be needed, and (3)
revise the investment management process based on lessons learned.
[19] These agencies include the Departments of Agriculture, Commerce,
and the Interior.
[20] For example, GAO, Information Technology: Centers for Medicare &
Medicaid Services Needs to Establish Critical Investment Management
Capabilities, GAO-06-12 (Washington, D.C.: Oct. 28, 2005); Information
Technology Management: Governmentwide Strategic Planning, Performance
Measurement, and Investment Management Can Be Further Improved, GAO-04-
49 (Washington, D.C.: Jan. 12, 2004); Information Technology:
Departmental Leadership Crucial to Success of Investment Reforms at
Interior, GAO-03-1028 (Washington, D.C.: Sept. 12, 2003); and United
States Postal Service: Opportunities to Strengthen IT Investment
Management Capabilities, GAO-03-3 (Washington, D.C.: Oct. 15, 2002).
[21] GAO, Information Technology: HHS Has Several Investment Management
Capabilities in Place, but Needs to Address Key Weaknesses, GAO-06-11
(Washington, D.C.: Oct. 28, 2005).
[22] GAO, Department of Homeland Security: Formidable Information and
Technology Management Challenge Requires Institutional Approach, GAO-
04-702 (Washington, D.C.: Aug. 27, 2004).
[23] Department of Defense Office of Inspector General, Information
Technology Management: Reporting of Department of Defense Capital
Investments for Information Technology in Support of the Fiscal Year
2006 Budget Submission, D-2005-083 (Arlington, Va.: June 10, 2005).
[24] OMB Circular A-11 defines a major IT investment as an investment
that requires special management attention because of its importance to
an agency's mission or because it is an integral part of the agency's
enterprise architecture, has significant program or policy
implications, has high executive visibility, or is defined as major by
the agency's capital planning and investment control process.
[25] The Federal Enterprise Architecture is a comprehensive business-
driven blueprint of the entire federal government. It consists of a set
of interrelated "reference models" designed to facilitate cross-agency
analysis and the identification of duplicative investments, gaps, and
opportunities for collaboration within and across agencies. The Federal
Enterprise Architecture includes 39 lines of business that describe
activities of the government, such as education, income security, and
supply chain management.
[26] Recent OMB guidance directed agencies to implement earned value
management on major IT investments, in an effort to meet baseline cost,
schedule, and performance goals.
[27] GAO-04-49.
[28] 44 U.S.C. § 3544 (6).
[29] The ANSI/EIA-748-A standard is composed of 32 criteria that
address five basic categories of project management practices:
organization, planning and budgeting, accounting considerations,
analysis and management reports, and revisions and data maintenance.
[30] We reviewed the investments to determine whether the ANSI-required
EVM processes were in place. We did not assess the quality of those
processes.
[31] OMB Memorandum, M-05-23 (Aug. 4, 2005).
[32] Federal Register Vol. 70, No. 67 (Apr. 8, 2005).
[33] GAO, Defense Acquisitions: Improved Management Practices Could
Help Minimize Cost Growth in Navy Shipbuilding Programs, GAO-05-183
(Washington, D.C.: Feb. 28, 2005); Polar-Orbiting Environmental
Satellites: Information on Program Cost and Schedule Changes, GAO-04-
1054 (Washington, D.C.: Sept. 30, 2004); and NASA: Lack of Disciplined
Cost-Estimating Processes Hinders Effective Program Management, GAO- 04-
642 (Washington, D.C.: May 28, 2004).
[34] 31 U.S.C. § 3512 note.
[35] GAO, Financial Management: Achieving FFMIA Compliance Continues to
Challenge Agencies, GAO-05-881 (Washington, D.C.: Sept. 20, 2005);
Managerial Cost Accounting Practices: Leadership and Internal Controls
Are Key to Successful Implementation, GAO-05-1013R (Washington, D.C.:
Sept. 2, 2005); and Financial Management: Sustained Efforts Needed to
Achieve FFMIA Accountability, GAO-03-1062 (Washington, D.C.: Sept. 30,
2003).
[36] GAO, Financial Management: FFMIA Implementation Critical for
Federal Accountability, GAO-02-29 (Washington, D.C.: Oct. 1, 2001);
Financial Management: FFMIA Implementation Necessary to Achieve
Accountability, GAO-03-31 (Washington, D.C.: Oct. 1, 2002); Financial
Management: Sustained Efforts Needed to Achieve FFMIA Accountability,
GAO-03-1062 (Washington, D.C.: Sept. 30, 2003); and Financial
Management: Improved Financial Systems Are Key to FFMIA Compliance, GAO-
05-20 (Washington, D.C.: Oct. 1, 2004).
[37] Pub. L. 107-347, Title II, § (209)(b)(1)(B) (Dec. 17, 2002).
[38] Results from nonprobability samples cannot be used to make
inferences about a population because in a nonprobability sample, some
elements of the population being studied have no chance or an unknown
chance of being selected as part of the sample.
[39] The Services for Citizens Business Area describes the mission and
purpose of the U.S. government in terms of the services it provides
both to and on behalf of the American citizen. It includes the delivery
of citizen-focused, public, and collective goods and/or benefits as a
service and/or obligation of the federal government to the benefit and
protection of the nation's general population.
[40] Support Delivery of Services provides the critical policy,
programmatic, and managerial foundation to support federal government
operations.
[41] Management of Government Resources refers to the back office
support activities that enable the government to operate effectively.
[42] DOD OIG, Information Technology: Reporting of DoD Capital
Investments for Information Technology, D-2004-081 (May 7, 2004).
[43] These investments include the U.S. Visitor and Immigrant Status
Indicator Technology (a program intended to strengthen management of
the pre-entry, entry, status, and exit of foreign nationals who travel
to the United States); the Automated Commercial Environment (a new
trade processing system planned to support the movement of legitimate
imports and exports and strengthen border security); Atlas (a program
intended to modernize Immigration and Customs Enforcement's IT
infrastructure); and Secure Flight (a new airline passenger screening
system).
[44] In addition to basic information about the investment, the exhibit
300 has the following sections: a Summary of Spending table,
Performance Measures and Goals, Analysis of Alternatives, Risk
Inventory and Assessment, Acquisition Strategy, Project (Investment)
and Funding Plan, Enterprise Architecture, and Security and Privacy.
[45] The financial data on each investment was obtained from the FY2006
Exhibit 53 which is available on the OMB Web site. System investment
descriptions and stage information were gathered from the Exhibit 300s
provided by the Office of the Chief Information Officer (CIO) at the
respective agencies.
[46] The Services for Citizens Business Area describes the mission and
purpose of the U.S. government in terms of the services it provides
both to and on behalf of the American citizen. It includes the delivery
of citizen-focused, public, and collective goods and/or benefits as a
service and/or obligation of the federal government to the benefit and
protection of the nation's general population.
[47] Support Delivery of Services provides the critical policy,
programmatic, and managerial foundation to support federal government
operations.
[48] Management of Government Resources refers to the back office
support activities that enable the government to operate effectively.
[49] Recent OMB guidance directed agencies to implement EVM on major IT
investments, in an effort to meet baseline cost, schedule, and
performance goals.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
