Financial Management Systems
Lack of Disciplined Process Puts Effective Implementation of Treasury's Governmentwide Financial Report System at Risk Gao ID: GAO-06-413 April 21, 2006For the past 9 years, since the first audit of the consolidated financial statements of the U.S. government (CFS), one of the major impediments to our ability to render an opinion on the CFS is that the federal government has not had adequate system, controls, and procedures to properly prepare the CFS. To address some of the internal control weaknesses identified in our audit report, Treasury began developing the Governmentwide Financial Report System (GFRS). The goal of this new system is to directly link information from federal agencies' audited financial statements to amounts reported in the CFS, a concept that we strongly support. We reported internal control weaknesses and GAO recommendations regarding the preparation of the CFS, along with progress made in this area in a separate report. This report provides our assessment of Treasury's ongoing effort to develop and implement GFRS and makes recommendations for reducing the risks associated with the development of GFRS.
Treasury's development of GFRS is a positive initiative and Treasury has made progress towards preparing the CFS based on agencies' audited financial statements, which has been one of our principal concerns. Treasury, though, has not yet effectively implemented the disciplined system development processes necessary to provide reasonable assurance that GFRS will meet all of its performance, schedule, and cost goals and result in the most efficient and effective means of preparing the CFS. Specifically, Treasury has not developed a concept of operations or any other document that adequately defines or documents the expected performance of GFRS; developed a detailed project plan and schedule through completion of GFRS; developed a budget justification for GFRS in its Capital Asset Plan and Business Case (commonly referred to as the Exhibit 300), as called for in Office of Management and Budget (OMB) Circular No. A-11; and implemented the disciplined processes necessary to effectively manage the GFRS project, which has contributed to usability problems encountered by its users. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. In our work at certain other federal agencies, we have found that project deficiencies such as those we have identified with the GFRS project have led to a range of problems, from increased cost and reduced functionality to system failure. Going forward, it will be important that Treasury better mitigate its risks so that long-standing internal control weaknesses regarding the preparation of the CFS can be eliminated and, more importantly, Treasury ends up with a system that fully meets its and agencies' needs.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-413, Financial Management Systems: Lack of Disciplined Process Puts Effective Implementation of Treasury's Governmentwide Financial Report System at Risk
This is the accessible text file for GAO report number GAO-06-413
entitled 'Financial Management Systems: Lack of Disciplined Processes
Puts Effective Implementation of Treasury's Governmentwide Financial
Report System at Risk' which was released on April 21, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Secretary of the Treasury:
April 2006:
Financial Management Systems:
Lack of Disciplined Processes Puts Effective Implementation of
Treasury's Governmentwide Financial Report System at Risk:
GAO-06-413:
GAO Highlights:
Highlights of GAO-06-413, a report to the Secretary of the Treasury:
Why GAO Did This Study:
For the past 9 years, since the first audit of the consolidated
financial statements of the U.S. government (CFS), one of the major
impediments to our ability to render an opinion on the CFS is that the
federal government has not had adequate system, controls, and
procedures to properly prepare the CFS. To address some of the internal
control weaknesses identified in our audit report, Treasury began
developing the Governmentwide Financial Report System (GFRS). The goal
of this new system is to directly link information from federal
agencies‘ audited financial statements to amounts reported in the CFS,
a concept that we strongly support. We reported internal control
weaknesses and GAO recommendations regarding the preparation of the
CFS, along with progress made in this area in a separate report. This
report provides our assessment of Treasury‘s ongoing effort to develop
and implement GFRS and makes recommendations for reducing the risks
associated with the development of GFRS.
What GAO Found:
Treasury‘s development of GFRS is a positive initiative and Treasury
has made progress towards preparing the CFS based on agencies‘ audited
financial statements, which has been one of our principal concerns.
Treasury, though, has not yet effectively implemented the disciplined
system development processes necessary to provide reasonable assurance
that GFRS will meet all of its performance, schedule, and cost goals
and result in the most efficient and effective means of preparing the
CFS. Specifically, Treasury has not
* developed a concept of operations or any other document that
adequately defines or documents the expected performance of GFRS;
* developed a detailed project plan and schedule through completion of
GFRS;
* developed a budget justification for GFRS in its Capital Asset Plan
and Business Case (commonly referred to as the Exhibit 300), as called
for in Office of Management and Budget (OMB) Circular No. A-11; and
* implemented the disciplined processes necessary to effectively manage
the GFRS project, which has contributed to usability problems
encountered by its users.
A disciplined software development and acquisition process can maximize
the likelihood of achieving the intended results (performance) within
established resources (costs) on schedule. Because of this
relationship, these factors can be viewed as an equilateral triangle as
shown below.
[See PDF for Image]
Source: GAO:
[End of Figure]
In our work at certain other federal agencies, we have found that
project deficiencies such as those we have identified with the GFRS
project have led to a range of problems, from increased cost and
reduced functionality to system failure. Going forward, it will be
important that Treasury better mitigate its risks so that long-standing
internal control weaknesses regarding the preparation of the CFS can be
eliminated and, more importantly, Treasury ends up with a system that
fully meets its and agencies‘ needs.
What GAO Recommends:
GAO is making three recommendations focused on reducing the risks
associated with the development of GFRS. Treasury stated that it
concurs with the recommendations in this report and is working to adopt
them.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Gary Engel at (202) 512-
3406 or [Hyperlink engelg@gao.gov].
[End of Section]
Contents:
Letter:
Results in Brief:
Background:
Scope and Methodology:
Effective Implementation of the Disciplined Processes Are Key to
Reducing Project Risks:
Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its
Performance, Schedule, and Cost Goals:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Department of Treasury:
Figures:
Figure 1: Percent of Effort Associated with Undisciplined Projects:
Figure 2: Software Development Trade-off Triangle:
Figure 3: Relationship between Requirements Development and Testing:
[End of Section]
United States Government Accountability Office:
Washington, DC 20548:
April 21, 2006:
The Honorable John W. Snow:
The Secretary of the Treasury:
For the past 9 years, we have reported that the federal government did
not have adequate systems, controls, and procedures to properly prepare
its consolidated financial statements. Many of the weaknesses in
internal control that contributed to our continuing disclaimers of
opinion were identified by federal agency financial statement auditors
during their audits of federal agencies' financial statements and were
reported in detail with recommendations to the federal agencies in
separate reports. However, some of the internal control weaknesses were
identified during our tests of the Department of the Treasury's
(Treasury) process for preparing the consolidated financial statements
of the U.S. government (CFS). The ability to prepare the CFS has been a
long-standing challenge for Treasury's Financial Management Service. To
address some of the internal control weaknesses identified in our audit
report,[Footnote 1] Treasury began developing the Governmentwide
Financial Report System (GFRS). The goal of this new system is to
directly link information from federal agencies' audited financial
statements to amounts reported in the CFS, a concept that we strongly
support and is a necessary prerequisite for our reliance on the
individual agency audits in auditing the CFS. Once Treasury is able to
achieve adequate systems, controls, and procedures to properly prepare
its consolidated financial statements, a major impediment to our
ability to audit the CFS would be eliminated.
This report provides our assessment of Treasury's ongoing effort to
develop and implement GFRS. Although the implementation of any
information system, such as GFRS, is never risk-free, organizations
that follow and effectively implement accepted best practices in
systems development and implementation (commonly referred to as
disciplined processes) have been shown to reduce these risks to
acceptable levels. The term acceptable levels acknowledges the fact
that any system acquisition has risks and will suffer the adverse
consequences associated with defects. A disciplined software
development and acquisition process can maximize the likelihood of
achieving the intended results (performance) within established
resources (costs) on schedule. Effective implementation of the
disciplined processes reduces the potential for risks to occur and
helps prevent those that do occur from having any significant adverse
impact on the cost, timeliness, and performance of the project. Our
reviews have found, like those performed by others, that the
effectiveness of the processes used to manage the project provide
valuable insight into whether a project will meet its cost, schedule,
and performance objectives.
We last briefed your staff on our initial assessment of this
development effort in September 2004. This report updates our work and
focuses on whether Treasury has effectively implemented disciplined
processes necessary to reduce the risks to the GFRS project to
acceptable levels and accordingly provide reasonable assurance that
GFRS will meet its performance, schedule, and cost goals. We reported
internal control weaknesses and our recommendations regarding the
preparation of the CFS, along with progress made in this area in a
separate report.[Footnote 2]
Results in Brief:
For fiscal years 2004 and 2005 Treasury's GFRS was able to capture
certain agency financial information from agencies' audited financial
statements. At the same time, the system was not yet at the stage of
development that it could be used to compile the consolidated financial
statements from the information that was captured. Therefore, for
fiscal years 2004 and 2005 Treasury continued to have to primarily use
manual procedures to prepare the CFS as GFRS continued to be developed.
We identified some risks in Treasury's development of GFRS.
We found that Treasury had not yet effectively implemented the
disciplined processes necessary to provide reasonable assurance that
GFRS will meet its performance, schedule, and cost goals. Specifically,
Treasury had not:
² developed a concept of operations or any other document that
adequately defines or documents the expected performance of GFRS;
² developed a detailed project plan and schedule through completion of
GFRS;
² developed a budget justification for GFRS in its Capital Asset Plan
and Business Case (commonly referred to as the Exhibit 300), as called
for in Office of Management and Budget (OMB) Circular No. A-
11;[Footnote 3] and:
² implemented the disciplined processes necessary to effectively manage
the GFRS project, which has contributed to usability problems
encountered by its users.
In our work at certain other federal agencies, we have found that not
effectively implementing disciplined processes has led to a range of
problems, from increased cost and reduced functionality to system
failure. Going forward, it will be important that Treasury better
mitigate its risks so that long-standing internal control weaknesses
regarding the preparation of the CFS can be eliminated and, more
importantly, Treasury ends up with a system that fully meets its and
agencies' needs. This report includes three recommendations to the
Secretary of the Treasury and the Fiscal Assistant Secretary focused on
reducing the risks associated with the development of GFRS.
In commenting on a draft of this report, Treasury stated that it
concurs with our recommendations and is working to adopt them. Treasury
also stated that it plans to develop a concept of operations for GFRS
and recently started to develop a budget justification for GFRS in a
Capital Asset Plan and Business Case. Further, Treasury stated that
although it uses manual procedures to prepare the CFS, it does not
agree that manual processes necessarily create internal control
weaknesses. We agree. Our discussion of Treasury's manual processes was
not intended to imply that using manual processes will by itself lead
to internal control weaknesses; rather, it was to highlight that
Treasury, as a user of GFRS, has to perform manual compilation
procedures because GFRS was not yet at the stage of development that it
could be used to compile the consolidated financial statements from the
agency financial information that was captured.
Background:
Treasury began implementing a new process for preparing the CFS using
GFRS during fiscal year 2004. GFRS is an application that is being
developed to meet the business needs of Treasury to prepare the CFS.
Treasury has stated that the goal of the new system is to be able to
directly link information from federal agencies' audited financial
statements to amounts reported in the CFS and to address our
recommendations[Footnote 4] regarding the preparation of the CFS. For
the fiscal year 2004 and 2005 reporting process, GFRS had enough
functionality to capture certain federal agency financial information
from federal agencies' audited financial statements. We also found that
Treasury made progress in demonstrating that amounts in the Balance
Sheet and the Statement of Net Cost were consistent with federal
agencies' audited financial statements prior to eliminating
intragovernmental activity and balances. GFRS, though, was not yet at
the stage of development that it could be used to compile the CFS from
the information that was captured. Treasury's plans had been to
evaluate and consider the manual processes used during fiscal year 2004
to develop the necessary requirements needed for the development of the
compilation module of GFRS. Although some enhancements to GFRS were
implemented for fiscal year 2005, Treasury continued to primarily use
manual procedures to prepare the most recently issued CFS.
As part of the concept of directly linking information from agencies'
audited financial statements to amounts reported in the CFS, Treasury
has classified 35 federal agencies as verifying[Footnote 5] agencies
and approximately 125 as nonverifying agencies and required these
agencies to enter data in GFRS. Verifying agencies are required to
reclassify certain of their audited financial statement line items to
the appropriate CFS line items within GFRS. Both verifying and
nonverifying agencies are required to enter certain of their financial
statement notes and other requested data into GFRS. Treasury then uses
this information to prepare the CFS. GFRS users at verifying and
nonverifying agencies include financial statement preparers, Chief
Financial Officers (CFOs), Inspectors General (IGs), and contracted
independent public accountants. GFRS grants certain permissions to
different user roles which limit each type of user to designated
locations within the system. Only certain roles have permission to
input, modify, and approve data.
Scope and Methodology:
As part of our annual audit of the CFS, we evaluate Treasury's
financial reporting procedures and related internal control, including
the development of GFRS. Our work was performed in accordance with U.S.
generally accepted government auditing standards. In fiscal years 2004
and 2005, GFRS was used to collect federal agencies' audited financial
statement data compiled in the CFS.
To assess Treasury's implementation of disciplined processes in the
development of GFRS, we reviewed industry standards and best practices
from the Institute of Electrical and Electronics Engineers (IEEE),
Software Engineering Institute (SEI), OMB Circular No. A-11, and prior
GAO reports. Beginning in fiscal year 2004, we reviewed and analyzed
available GFRS documentation and made inquiries of Treasury's GFRS
project team and other Treasury personnel to determine whether Treasury
has effectively implemented disciplined processes necessary to reduce
the risks to the GFRS project to acceptable levels and accordingly
provide reasonable assurance that GFRS will meet its performance,
schedule, and cost goals. We briefed Treasury on our initial assessment
of this development effort in September 2004. We reported internal
control weaknesses related to GFRS in our fiscal year 2004 audit
report[Footnote 6] and made a recommendation related to GFRS in our
fiscal year 2004 internal control report.[Footnote 7] In fiscal year
2005, we reviewed documentation related to the planned enhancements for
GFRS for 2005 and made inquiries of Treasury's GFRS project team and
other Treasury personnel to further determine whether Treasury has
effectively implemented disciplined processes. We again reported
internal control weaknesses related to GFRS in our fiscal year 2005
audit report.
We requested comments on a draft of this report from the Secretary of
the Treasury or his designee. Treasury's comments are reprinted in
appendix I and are also discussed in the Agency Comments and Our
Evaluation section.
Effective Implementation of the Disciplined Processes Are Key to
Reducing Project Risks:
Disciplined processes, which are fundamental to successful systems
development and implementation efforts, have been shown to reduce the
risks associated with software development and acquisition to
acceptable levels. A disciplined software development and acquisition
process can maximize the likelihood of achieving the intended results
(performance) within established resources (costs) on schedule.
Although there is no standard set of practices that will ever guarantee
success, several organizations, such as SEI[Footnote 8] and
IEEE,[Footnote 9] as well as individual experts, have identified and
developed the types of policies, procedures, and practices that have
been demonstrated to reduce development time and enhance effectiveness.
The key to having a disciplined system development effort is to have
disciplined processes in multiple areas, including project planning and
management, requirements management, configuration management, risk
management, quality assurance, and testing. Effective processes should
be implemented in each of these throughout the project life cycle
because change is constant. Effectively implementing the disciplined
processes necessary to reduce project risks to acceptable levels is
hard to achieve because a project must effectively implement several
best practices, and inadequate implementation of any one may
significantly reduce or even eliminate the positive benefits of the
others.
As part of the disciplined processes, acquiring and implementing a new
financial management system such as GFRS requires a methodology that
starts with a clear definition of the organization's mission and
strategic objectives and ends with a system that meets specific
information needs. We have seen many system efforts fail because
federal agencies started with a general need, but did not define in
precise terms (1) the specific problems they were trying to solve, (2)
what their operational needs were, and (3) what specific information
requirements flowed from these operational needs. Instead, they plunged
into the acquisition and implementation process in the belief that
these specifics would somehow be defined along the way. The typical
result was that systems were delivered well past anticipated
milestones; failed to perform as expected; and, accordingly, were over
budget because of required costly modifications.
Figure 1 shows how organizations that do not effectively implement the
disciplined processes lose the productive benefits of their efforts as
a project continues through its development and implementation cycle.
Although undisciplined projects show a great deal of productive work at
the beginning of the project, the rework associated with defects begins
to consume more and more resources. In response, processes are adopted
in the hopes of managing what later turns out, in reality, to have been
unproductive work. Generally, these processes are "too little, too
late" and rework begins to consume more and more resources because
sufficient foundations for building the systems were not established or
not established adequately. Experience has shown that projects for
which disciplined processes are not implemented at the beginning are
forced to implement them later when it takes more time and they are
less effective.[Footnote 10]
Figure 1: Percent of Effort Associated with Undisciplined Projects:
[See PDF for image]
Source: Reproduced by permission from Steve McConnell, Professional
Software Development: Shorter Schedules, Higher Quality Products, More
Successful Projects, Enhanced Careers(Boston, Mass: Pearson Education
Inc., 2004).
[End of figure]
As shown in figure 1, a major consumer of project resources in
undisciplined efforts is rework (also known as thrashing). Rework
occurs when the original work has defects or is no longer needed
because of changes in project direction. Disciplined organizations
focus their efforts on reducing the amount of rework because it is
expensive. Fixing a requirements defect after the system is released
costs anywhere from 10 to 100 times as much as fixing it when the
requirements are defined.[Footnote 11] As shown in figure 1, projects
that are unable to successfully address their rework will eventually
only be spending their efforts on rework and the associated processes
rather than on productive work. In other words, the project will
continually find itself reworking items.
A disciplined software development and acquisition process can maximize
the likelihood of achieving the intended results (performance) within
established resources (costs) on schedule. Because of this
relationship, these factors can be viewed as an equilateral triangle as
shown in figure 2.
Figure 2: Software Development Trade-off Triangle:
[See PDF for image]
Source: GAO.
[End of figure]
The performance corner of the triangle is also referred to as the
product corner. This corner includes quality and all other product-
related attributes including complexity, usability, maintainability,
and defect rate. For example, beyond a certain point, it is difficult
to increase performance without changing the cost and/or schedule
attributes. Similarly, a schedule cannot be drastically compressed
without degrading the quality of the software project or increasing the
cost of development. Schedule, cost, and performance must be in balance
for a project to succeed. The following sections of this report address
our observations regarding the performance, schedule, and cost of
Treasury's GFRS development effort.
Lack of Disciplined Processes Puts GFRS at Risk of Not Meeting Its
Performance, Schedule, and Cost Goals:
Treasury's development of GFRS is a positive initiative and Treasury
has made progress towards preparing the CFS based on agencies' audited
financial statements, which is one of our principal concerns. While we
support the goals of GFRS, we are concerned that Treasury has not yet
effectively implemented a number of disciplined processes that have
been shown to reduce project risks to acceptable levels. Specifically,
we noted that Treasury has not yet implemented the types of project
management processes necessary to provide the information required to
assess the progress of GFRS and whether it is meeting its cost,
schedule, and performance objectives. First, Treasury has not described
its vision for the project in a manner that can be used to guide the
development efforts. Although disciplined projects use a concept of
operations document to serve this purpose, Treasury has not yet
developed such a document. Treasury also has not developed a detailed
project plan or schedule through the completion of GFRS or followed
OMB's guidance for major investment planning, budgeting, and
acquisition. Some adverse impacts associated with these process
weaknesses have already been realized. Specifically, significant
usability issues have arisen and little meaningful project management
information is available to assess the project performance.
Treasury Has Not Developed a Concept of Operations:
Treasury has not adequately defined or documented the expected
functionality for GFRS. One approach for describing the functionality
that is expected from a system is to document the vision in a concept
of operations. According to IEEE Standard 1362-1998, a concept of
operations document is normally one of the first documents produced
during a disciplined development effort because it describes system
characteristics for a proposed system from the user's viewpoint. This
is important because a good concept of operations document can be used
to communicate overall quantitative and qualitative system
characteristics to the user, developer, and other organizational
elements typically involved in a systems development effort. This
allows the reader to understand the user organizations, missions, and
organizational objectives from an integrated systems point of view.
Until such a document is developed, it is virtually impossible for the
project to develop a firm foundation for its requirements and testing
activities. Experience has shown that poor requirements[Footnote 12]
are a major factor associated with projects that do not meet their
cost, schedule, and performance objectives. Furthermore, without good
requirements, it is impossible to develop a disciplined testing
program. Figure 3 shows the relationship between requirements and
testing and how the concept of operations provides the foundation for
the requirements.
Figure 3: Relationship between Requirements Development and Testing:
[See PDF for image]
Source: GAO.
[End of figure]
The Treasury project team stated they consider the November 2001
Working Group Report on the Financial Report of the United States
Government Preparation Process to be their concept of operations. Our
analysis noted at least two reasons why this document would not be
acceptable for that purpose. First, the November 2001 report resembles
an alternatives analysis where Treasury weighed several options to
change its current process for preparing the CFS. The report does not
address the objectives of a concept of operations because it does not
outline the business processes that are expected to be implemented in
the new system. For example, when we compared the November 2001 report
to the current functionality of GFRS, we found that the November 2001
report stated that federal agencies would be required to submit an
analysis of changes in net position and to split net position between
intragovernmental and operations with the public. However, over 2 years
ago Treasury stated to GAO that they would not be requiring federal
agencies to split their net position and did not provide an alternative
solution for addressing this key net position weakness affecting the
preparation of the CFS. Second, the November 2001 report was issued
prior to the Federal Accounting Standards Advisory Board's issuance of
Statement of Federal Financial Accounting Standard No. 24, Selected
Standards for the Consolidated Financial Report of the United States
Government, which requires two additional financial statements to be
included in the Financial Report--Reconciliation of Net Operating Cost
and Unified Budget Deficit and Statement of Changes in Cash Balance
from Unified Budget and Other Activities. Because the November 2001
report could not contemplate the preparation of these two statements,
it is at least partially obsolete in terms of reporting requirements
for two of the five primary consolidated financial statements.
Going forward, an effective concept of operations could help Treasury
reach its goal of having GFRS, when fully implemented, resolve several
continuing weaknesses that we have identified during our CFS audit
work, including providing additional functionality that would allow
Treasury to eliminate some of the manual processes and provide
improvements from its previous process. Treasury has not yet developed
and documented in a concept of operations document or any other
document (1) the corrective actions for financial reporting weaknesses
that the system is expected to address and (2) the functionality it
needs GFRS to provide when it is completed to implement these
corrective actions. Accordingly, Treasury does not have adequate
assurance that the resulting system will have the needed functionality
to address the weaknesses that Treasury management may expect the
system to eliminate or reduce the time it takes Treasury to produce the
CFS. This basic information is critical for the project team to
effectively develop a complete set of high-level requirements for GFRS
and reach agreement with Treasury management on the functionality GFRS
is expected to provide and how to determine when GFRS is complete and
has ultimately met management's expectations.
Treasury Has Not Developed a Detailed Project Plan and Schedule through
Completion of GFRS:
Treasury has not clearly defined what GFRS will encompass and when that
functionality is expected to be delivered. This basic information
should serve as the foundation to support realistic project scheduling
efforts, so that Treasury could (1) determine the work needed to
complete the project and (2) make informed decisions on when that work
can be expected to be completed.
While we understand that GFRS will take several years to develop, a
"blueprint" guiding these efforts has not been developed. Rather than
using best practices to schedule out the functionality to be developed
and implemented over the entire development period, each year Treasury
decides on the high-level requirements to be developed and implemented
only during that given year. These requirements are not "placed in
context" to the overall system goals and needs. This approach presents
the following issues.
² It is unclear how the requirements selected for implementation each
year move GFRS closer to providing the necessary functionality to
address the material weaknesses Treasury expects the system to correct.
² Without a detailed project plan and schedule through completion of
GFRS, it is unclear whether the project is capable of meeting its
established time frames and goals, how to determine the progress made
with a given year's efforts, or how to estimate the funding necessary
to develop the system.
These weaknesses have already had adverse project impacts.
Specifically, even once the requirements for a given year have been
"defined", we found that little meaningful project scheduling was used.
For example, Treasury provided us with a list of its milestones for the
fiscal year 2005 development of GFRS; however, this documentation did
not provide information such as the detailed tasks to be performed,
duration of the tasks, person responsible for completing each task, or
how these development efforts link to the overall functionality
required for GFRS. Accordingly, Treasury did not have the basic project
management information necessary to determine whether its efforts were
meeting the project's cost, schedule, and performance objectives.
In our September 2004 briefing to Treasury on our initial assessment of
the GFRS development effort, we noted that Treasury faced the risk of
rework by using a single-year scheduling approach rather than
developing a schedule for the entire development period. We stated that
implementing any of management's high-level requirements for the
current year of development could cause significant rework of the
functionality that was implemented in the prior year. Disciplined
organizations focus their efforts on reducing the amount of rework
because it is expensive. Without a detailed project plan and schedule,
the program does not have a process that is designed to avoid doing
things twice and runs a much greater risk that it will cost additional
time, money, and resources.
Impacts of Not Following OMB Guidance:
Treasury has not yet developed a budget justification for GFRS in a
Capital Asset Plan and Business Case (commonly referred to as the
Exhibit 300), called for in OMB Circular No. A-11. The OMB circular
provides instructions on how to prepare the necessary budget
justification and reporting requirements for major information
technology investments,[Footnote 13] such as GFRS. The Exhibit 300
helps demonstrate to federal agency management and OMB that the federal
agency has employed the disciplines of good project management,
represented a strong business case for the investment, and met other
administration priorities to define the proposed cost, schedule, and
performance goals for the investment if funding approval is obtained.
The business case should include security, privacy, and enterprise
architecture, and provide the effectiveness and efficiency gains
planned by the business lines and functional operations. According to
Treasury personnel, it was a Treasury policy decision not to prepare an
Exhibit 300 for submission to OMB.
The circular states that an "exhibit 300 must be submitted for all
major investments in accordance with this section." The circular also
notes that "[g]ood budgeting requires appropriations for the full risk
adjusted costs of asset acquisition be enacted in advance to help
ensure that all costs and benefits are fully taken into account when
decisions are made about providing resources." We found that Treasury
has not yet determined the cost of GFRS to date or estimated its cost
through completion. Instead, Treasury's project team stated that, to
date, they have used funds remaining from other Treasury projects to
pay for the development of GFRS.[Footnote 14] According to Treasury's
project team, the development of GFRS is guided, limited, and
constrained by the funds that they can "beg, borrow, and steal" from
other Treasury development efforts. As noted in OMB Circular No. A-11,
when "capital assets are funded in increments, without certainty if or
when future funding will be available, it can and occasionally does
result in poor planning, acquisition of assets not fully justified,
higher acquisition costs, project (investment) delays, cancellation of
major investments, the loss of sunk costs, or inadequate funding to
maintain and operate the assets."
GFRS Has Some Usability Issues:
Effectively implementing disciplined processes helps ensure the system
meets the users' needs. To date, GFRS has had some significant
usability issues. In February 2005, Treasury held a forum with GFRS
federal agency users, including agency CFO, IG, and independent public
accountant staff, to discuss lessons learned from the fiscal year 2004
GFRS reporting process and planned changes to GFRS for fiscal year
2005. At the forum, the users noted the lack of user-friendly reporting
capabilities in GFRS. As we noted in our September 2004 briefing to
Treasury, user-friendly reporting capabilities were not available to
federal agency users for fiscal year 2004 and such functionality was
critical to the ultimate usefulness of the system. The process of
viewing the information entered into GFRS is especially burdensome for
the 32 federal agencies[Footnote 15] required to have this information
audited. GFRS does not provide the ability for an agency to develop one
report containing all of the information required to be entered into
GFRS. Instead, users are required to run over 35 separate reports in
GFRS, which has proven to be time-consuming. Compounding this is that
agencies have only a very short period of time to enter the data into
GFRS. Specifically, federal agencies are required to have their audited
financial information in the GFRS closing package by November 18--just
3 days after their audited financial statements are due to OMB.
We believe the lack of user-friendly reporting capabilities is a result
of Treasury not effectively implementing disciplined processes.
Treasury had not defined types of reporting that would be available to
the users and when it would be available. Treasury's project team
stated that it is now trying to address this issue and has acknowledged
that a lack of an adequate understanding of users' reporting needs
contributed to the problem. They also noted the lack of funding as
another reason this issue has not been fully addressed. In our view, if
Treasury had implemented disciplined processes during the planning of
GFRS, such as developing a concept of operations and implementing an
effective requirements management process, this issue (1) would have
been identified and (2) actions could have been taken at the outset to
develop a viable solution earlier in the process rather than waiting
until significant amounts of development work had been performed.
In our discussions with the Treasury project team over the reporting
issue, we were told that they are concerned that providing more robust
reporting capabilities to users could cause the system to fail and
"crash" should all of the GFRS users run these reports at the same
time. In our September 2004 briefing, we noted that one of the
challenges to Treasury was adequately testing GFRS to ensure that it
could function properly when all federal agencies use the system at the
same time, which includes allowing users to run all needed GFRS
reports. Treasury's project team told us that they were unable to test
GFRS capacity because they had not identified or developed all the
needed user-friendly reporting capabilities.
We believe other usability issues exist that directly affect Treasury
as a user of GFRS as well. For example, currently, Treasury cannot rely
on the system to compile the CFS from the information that is captured.
Therefore, for 2 years in a row, Treasury primarily used manual
procedures to prepare the CFS. While Treasury's initial plans were to
evaluate and consider the manual processes used during fiscal year 2004
to develop the necessary requirements needed for the development of the
compilation module of GFRS, this capability has not been added and it
is unclear when or if it will be available.
Conclusions:
The implementation of any major system, such as GFRS, is not a risk-
free proposition. However, organizations that follow and effectively
implement accepted best practices in systems development and
implementation have been shown to reduce these risks to acceptable
levels. Treasury has not yet done so. If it continues on this path, it
runs an unnecessary risk of building a system that may be more costly
and take longer to deploy, while not providing all of the intended
system functionality. Going forward, it will be important that Treasury
take the actions necessary to implement the disciplined processes that
provide reasonable assurance that GFRS will achieve Treasury's ultimate
goal of preparing the CFS with a direct link to federal agencies'
audited financial statements and addressing long-standing internal
control weaknesses. A key first step will be for Treasury to develop
and document its view of how GFRS will operate, including how it will
be used to address significant internal control weaknesses.
Recommendations for Executive Action:
We recommend that the Secretary of the Treasury direct the Treasury
Fiscal Assistant Secretary to take the following three actions to
reduce the risks associated with the development of GFRS:
² Follow the budget justification and reporting processes set out in
OMB Circular No. A-11 and ensure that funding needs are fully addressed.
² Develop and adopt a concept of operations that fully describes the
functionality expected to be provided by GFRS and the material internal
control weaknesses that are expected to be addressed by the system.
² Develop and effectively implement the disciplined processes necessary
to properly manage the development of GFRS, including the development
of a detailed project plan and schedule through the completion of the
system.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, which are reprinted in
appendix I, Treasury stated that our report clearly points out
recommendations for reducing the risks associated with the continued
development of GFRS. Treasury also stated that it concurs with our
recommendations and is working to adopt them. Further, Treasury stated
that it plans to develop a concept of operations for GFRS and recently
started to develop a budget justification for GFRS in a Capital Asset
Plan and Business Case.
Treasury provided additional comments regarding Treasury's manual
procedures used in preparing the CFS. In particular, Treasury stated
that although it uses manual procedures to prepare the CFS, it does not
agree that manual processes necessarily create internal control
weaknesses. We agree. Our discussion of Treasury's manual processes was
not intended to imply that using manual processes will lead to internal
control weaknesses, but rather to highlight that Treasury, as a user of
GFRS, has to perform manual compilation procedures because GFRS was not
yet at the stage of development that it could be used to compile the
consolidated financial statements from the agency financial information
that was captured. Our understanding is that one of the automated
functions Treasury expected GFRS to perform was the compilation of the
CFS from the information captured rather than performing these
functions manually. This capability has not yet been added to GFRS, and
it is unclear when or if it will be available because, as we have
reported, Treasury does not currently have a concept of operations that
defines or documents the expected functionality for GFRS.
This report contains recommendations to the Secretary of the Treasury.
The head of a federal agency is required by 31 U.S.C. 720 to submit a
written statement on actions taken on these recommendations. You should
submit your statement to the Senate Committee on Homeland Security and
Governmental Affairs and the House Committee on Government Reform
within 60 days of the date of this report. A written statement must
also be sent to the House and Senate Committees on Appropriations with
the agency's first request for appropriations made more than 60 days
after the date of the report.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of the Senate Committee on Homeland Security and
Governmental Affairs; the Subcommittee on Federal Financial Management,
Government Information, and International Security, Senate Committee on
Homeland Security and Governmental Affairs; the House Committee on
Government Reform; and the Subcommittee on Government Management,
Finance, and Accountability, House Committee on Government Reform. In
addition, we are sending copies to the Fiscal Assistant Secretary of
the Treasury. Copies will be made available to others upon request.
This report is also available at no charge on GAO's Web site at
[Hyperlink=http://www.gao.gov].
We appreciate the courtesy and cooperation extended to us by your staff
throughout our work. We look forward to continuing to work with your
offices to help improve systems development efforts in the federal
government. If you have any questions about the contents of this
report, please contact Gary Engel, Director, Financial Management and
Assurance, who may be reached at (202) 512-3406 or by e-mail at:
engelg@gao.gov, or Keith Rhodes, Chief Technologist, Applied Research
and Methods, who may be reached at (202) 512-6412 or by e-mail at
rhodesk@gao.gov. Key contributors to this letter were Chris Martin,
Lynda Downing, and Katherine Schirano.
:Signed By
Gary T. Engel:
Director:
Financial Management and Assurance:
[Signed By]
Keith A. Rhodes:
Chief Technologist:
Applied Research and Methodology Center for Engineering and Technology:
[End of section]
Appendix I: Comments from the Department of Treasury:
Department Of The Treasury:
Financial Management Service:
Washington, D.C. 20227:
Commissioner:
April 3, 2006:
Mr. Gary T. Engel:
Director, Financial Management and Assurance: Government Accountability
Office:
Washington, DC 20548:
Dear Mr. Engel:
Thank you for the opportunity to comment on GAO's draft report, GAO-06-
413, Lack of Disciplined Processes Puts Effective Implementation of
Treasury's Governmentwide Financial Report System at Risk.
Your report clearly pointed out 3 recommendations for reducing the
risks associated with the continued development of the Governmentwide
Financial Report System (GFRS). We concur with these recommendations
and are working to adopt them. We plan to develop a Concept of
Operations for GFRS and we recently started to develop a budget
justification for GFRS in a Capital Asset Plan and Business Case. While
we acknowledge that our system development process could be more
disciplined and structured, the GFRS has been very successful in
capturing the audited financial statement information from the agency
for both FY04 and FY05. GFRS has also been very effective in
reclassifying this agency data for compilation into the Financial
Report (FR). In addition, the internal control security weaknesses
identified by GAO in FY05 have been addressed by adopting and
performing an Oracle Security Baseline Review. The Security Baseline
details the requirements that FMS has set forth to protect the
application.
There are two other items we would like to call to your attention in
the section titled "GFRS has some usability issues".
1. We have put in place a structured process to solicit, evaluate and
prioritize agency requests for enhancements to GFRS and, to that
extent, have strived to add and modify functionality as soon as
reasonably possible within a limited development time frame and budget
constraints. Our enhancements for GFRS are planned through FY07 and we
will maintain a plan for the next several years. For FY 2006, we are
accommodating two major agency requests; the roll-over of prior year
reported data into the prior year fields and the ability to print
multiple reports. In addition, every year, we conduct new user training
during the summer and provide a training class on the new functionality
for existing users, when applicable. In addition, we conducted a forum
for the CFO and IG representatives to evaluate possible future
improvements in GFRS.
2. We agree that some manual processes are used to prepare the FR.
However, we do not agree with the notion that manual processes
necessarily create internal control weaknesses. Instead it is necessary
to look at the specific controls that are used to ensure the integrity
of the entire process, both the automated and the manual processes. How
we accomplish a task should not be judged only on the manual procedures
involved but also by taking into account the internal controls that we
have in place to mitigate the risks involved and ensure the integrity
of the data. We have spoken to representatives from major corporations
such as ALCOA and Martin Marietta regarding the processes they use to
prepare their financial statements. Many of them use a combination of
automated and manual processes and they agree that some manual
procedures are often necessary and by themselves do not create internal
control weaknesses.
In conclusion, we will continue to work with the members of your team
to resolve any security and system development concerns.
Sincerely,
[Signed By]
Richard L. Gregg:
Cc: Donald Hammond:
Cc: Linda Combs, OMB:
Cc: Danny Wersel, OMB:
(198415):
[End of section]
FOOTNOTES:
[1] Department of the Treasury, 2005 Financial Report of the United
States Government (Washington, D.C.: December 2005). This report
includes GAO's audit report.
[2] GAO, Financial Audit: Significant Internal Control Weaknesses
Remain in Preparing the Consolidated Financial Statements of the U.S.
Government, GAO-06-415 (Washington, D.C.: Apr. 21, 2006).
[3] Section 300 of OMB Circular No. A-11, Preparation, Submission, and
Execution of the Budget (Nov. 2, 2005), sets forth requirements for
federal agencies for planning, budgeting, acquiring, and managing
information technology capital assets.
[4] GAO, Financial Audit: Process for Preparing the Consolidated
Financial Statements of the U.S. Government Continues to Need
Improvement, GAO-05-407 (Washington, D.C.: May 4, 2005).
[5] Treasury defines the verifying agencies as the 24 Chief Financial
Officer (CFO) Act agencies, Export-Import Bank of the United States,
Farm Credit System Insurance Corporation, Federal Communications
Commission, Federal Deposit Insurance Corporation, National Credit
Union Administration, U.S. Postal Service, Pension Benefit Guaranty
Corporation, Railroad Retirement Board, Securities and Exchange
Commission, Smithsonian Institution, and Tennessee Valley Authority.
Treasury also refers to the verifying agencies as "significant"
agencies.
[6] Department of the Treasury, 2004 Financial Report of the United
States Government (Washington, D.C.: December 2004). This report
includes GAO's audit report.
[7] GAO-05-407.
[8] SEI is a federally funded research and development center operated
by Carnegie Mellon University and sponsored by the Department of
Defense. The SEI objective is to provide leadership in software
engineering and in the transition of new software engineering
technologies into practice.
[9] IEEE is a nonprofit, technical professional association that
develops standards for a broad range of global industries including the
information technology and information assurance industries. The IEEE
standards program serves the global needs of industry, government, and
the public. It also works to assure the effectiveness and high
visibility of the standards program both within IEEE and throughout the
global community.
[10] Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash: Microsoft Press, 1996).
[11] Steve McConnell, Code Complete, Second Edition (Redmond, Wash:
Microsoft Press, 2004).
[12] Requirements are the specifications that system developers and
program managers use to design, develop, and acquire a system. They
need to be unambiguous, consistent with one another, verifiable, and
directly traceable to higher level business or functional requirements.
It is critical that requirements flow directly from the organization's
concept of operations.
[13] OMB Circular No. A-11, section 300 defines a major investment as a
system or project requiring special management attention because of its
importance to the mission or function of the agency, a component of the
agency or another organization; is for financial management and
obligates more than $500,000 annually; has significant program or
policy implications; has high executive visibility; has high
development, operating, or maintenance costs; or is defined as major by
the agency's capital planning and investment control process.
[14] We did not review Treasury's alternative funding sources.
[15] The Treasury Financial Manual states that the Inspector General or
their contracted independent public accountant for each verifying
agency, except those agencies with a year end other than September 30
(Federal Deposit Insurance Corporation, National Credit Union
Administration, and Farm Credit System Insurance Corporation), must
opine on the closing package data, entered by the Chief Financial
Officer into GFRS, as to its consistency with the comparative, audited,
consolidated, department-level financial statements. Of the 35
verifying agencies, 32 are currently required to have an audit of their
closing package.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
