Financial Management Systems
HHS Faces Many Challenges in Implementing Its Unified Financial Management System Gao ID: GAO-04-1089T September 30, 2004GAO has previously reported on systemic problems the federal government faces in achieving the goals of financial management reform and the importance of using disciplined processes for implementing financial management systems. As a result, the Subcommittee on Government Efficiency and Financial Management, House Committee on Government Reform, asked GAO to review and evaluate the agencies' plans and ongoing efforts for implementing financial management systems. The results of GAO's review of the Department of Health and Human Services' (HHS) ongoing effort to develop and implement the Unified Financial Management System (UFMS) are discussed in detail in the report Financial Management Systems: Lack of Disciplined Processes Puts Implementation of HHS' Financial System at Risk (GAO-04-1008). In this report, GAO makes 34 recommendations focused on mitigating risks associated with the project. In light of this report, the Subcommittee asked GAO to testify on the challenges HHS faces in implementing UFMS.
HHS had not effectively implemented several disciplined processes, which are accepted best practices in systems development and implementation, and had adopted other practices, that put the project at unnecessary risk. Although the implementation of any major system is not a risk-free proposition, organizations that follow and effectively implement disciplined processes can reduce these risks to acceptable levels. While GAO recognized that HHS had adopted some best practices related to senior level support, oversight, and phased implementation, GAO noted that HHS had focused on meeting its schedule to the detriment of disciplined processes. GAO found that HHS had not effectively implemented several disciplined processes to reduce risks to acceptable levels, including requirements management, testing, project management and oversight using quantitative measures, and risk management. Compounding these problems are departmentwide weaknesses in information technology management processes needed to provide UFMS with a solid foundation for development and operation, including investment management, enterprise architecture, and information security. GAO also identified human capital issues that significantly increase the risk that UFMS will not fully meet one or more of its cost, schedule, and performance objectives, including staffing and strategic workforce planning. HHS stated that it had an aggressive implementation schedule, but disagreed that a lack of disciplined processes is placing the UFMS program at risk. GAO firmly believes if HHS continues to follow an approach that is schedule-driven and shortcuts key disciplined processes, it is unnecessarily increasing its risk. GAO stands by its position that adherence to disciplined processes is crucial, particularly with a project of this magnitude and importance. HHS indicated that it plans to delay deployment of significant functionality associated with its UFMS project for at least 6 months. This decision gives HHS a good opportunity to effectively implement disciplined processes to enhance the project's opportunity for success.
GAO-04-1089T, Financial Management Systems: HHS Faces Many Challenges in Implementing Its Unified Financial Management System
This is the accessible text file for GAO report number GAO-04-1089T
entitled 'Financial Management Systems: HHS Faces Many Challenges in
Implementing Its Unified Financial Management System' which was
released on September 30, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Government Efficiency and Financial
Management, Committee on Government Reform, House of Representatives:
For Release on Delivery Expected at 2:00 p.m. EDT Thursday, September
30, 2004:
Financial Management Systems:
HHS Faces Many Challenges in Implementing Its Unified Financial
Management System:
Statement of Jeffrey C. Steinhoff, Managing Director, Financial
Management and Assurance:
Keith A. Rhodes, Chief Technologist, Applied Research and Methodology
Center for Engineering and Technology:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-1089T]:
GAO Highlights:
Highlights of GAO-04-1089T, a testimony before the Subcommittee on
Government Efficiency and Financial Management, Committee on
Government Reform, House of Representatives.
Why GAO Did This Study:
GAO has previously reported on systemic problems the federal
government faces in achieving the goals of financial management reform
and the importance of using disciplined processes for implementing
financial management systems. As a result, the Subcommittee on
Government Efficiency and Financial Management, House Committee on
Government Reform, asked GAO to review and evaluate the agencies‘ plans
and ongoing efforts for implementing financial management systems.
The results of GAO‘s review of the Department of Health and Human
Services‘ (HHS) ongoing effort to develop and implement the Unified
Financial Management System (UFMS) are discussed in detail in the
report Financial Management Systems: Lack of Disciplined Processes Puts
Implementation of HHS‘ Financial System at Risk (GAO-04-1008). In this
report, GAO makes 34 recommendations focused on mitigating risks
associated with the project. In light of this report, the Subcommittee
asked GAO to testify on the challenges HHS faces in implementing UFMS.
What GAO Found:
HHS had not effectively implemented several disciplined processes,
which are accepted best practices in systems development and
implementation, and had adopted other practices, that put the project
at unnecessary risk. Although the implementation of any major system
is not a risk-free proposition, organizations that follow and
effectively implement disciplined processes can reduce these risks to
acceptable levels. While GAO recognized that HHS had adopted some best
practices related to senior level support, oversight, and phased
implementation, GAO noted that HHS had focused on meeting its schedule
to the detriment of disciplined processes.
GAO found that HHS had not effectively implemented several disciplined
processes to reduce risks to acceptable levels, including
* requirements management,
* testing,
* project management and oversight using quantitative measures, and
* risk management.
Compounding these problems are departmentwide weaknesses in information
technology management processes needed to provide UFMS with a solid
foundation for development and operation, including
* investment management,
* enterprise architecture, and
* information security.
GAO also identified human capital issues that significantly increase
the risk that UFMS will not fully meet one or more of its cost,
schedule, and performance objectives, including
* staffing and
* strategic workforce planning.
HHS stated that it had an aggressive implementation schedule, but
disagreed that a lack of disciplined processes is placing the UFMS
program at risk. GAO firmly believes if HHS continues to follow an
approach that is schedule-driven and shortcuts key disciplined
processes, it is unnecessarily increasing its risk. GAO stands by its
position that adherence to disciplined processes is crucial,
particularly with a project of this magnitude and importance.
HHS indicated that it plans to delay deployment of significant
functionality associated with its UFMS project for at least 6 months.
This decision gives HHS a good opportunity to effectively implement
disciplined processes to enhance the project‘s opportunity for
success.
www.gao.gov/cgi-bin/getrpt?GAO-04-1089T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Jeffrey Steinhoff (202)
512-2600, steinhoffj@gao.gov or Keith Rhodes (202) 512-6412,
rhodesk@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
We are pleased to be here today to discuss the efforts by the
Department of Health and Human Services (HHS) to develop and implement
its Unified Financial Management System (UFMS). We would like to thank
the Subcommittee for having this hearing. Hearings such as this one
today foster meaningful financial management reform. Our work focused
on whether the UFMS project was being managed in a way that best
ensures long-term success of this important project. At the time of our
review, the complete implementation of UFMS was targeted for 2007 and
the estimated total project cost was over $700 million.[Footnote 1] Not
only must the system ultimately replace 5 accounting systems, but it
must also interface with about 110 other systems. By any measure, this
is a major undertaking that brings with it a degree of risk. Risk,
though, can be managed and reduced to acceptable levels through the use
of disciplined processes, which in short, represent best practices in
system development and implementation that have proven their value in
the past.
Our report,[Footnote 2] which was prepared at the request of the
Subcommittee and is being released at this hearing, discusses in detail
the issues we identified with the UFMS project and includes 34
recommendations that focus on mitigating project risk. Our testimony
today[Footnote 3] will (1) highlight the importance of adhering to
disciplined processes for a system development and implementation
effort such as UFMS, (2) summarize our findings on HHS' management of
the UFMS project, and (3) provide our perspective on actions needed for
HHS to mitigate the risk to this project and move forward.
Disciplined Processes Are Key to Successful System Implementation:
The ability to produce the information needed to efficiently and
effectively manage the day-to-day operations of the federal government
and provide accountability to taxpayers and the Congress has been a
long-standing challenge for federal agencies. To help address this
challenge, many agencies are in the process of replacing their core
financial systems as part of their financial management system
improvement efforts. Although the implementation of any major system is
not a risk-free proposition, organizations that follow and effectively
implement disciplined processes can reduce these risks to acceptable
levels. The use of the term acceptable levels acknowledges the fact
that any systems acquisition has risks and will suffer the adverse
consequences associated with defects. However, effective
implementation of the disciplined processes reduces the potential for
risks to occur and helps prevent those that do occur from having any
significant adverse impact on the cost, timeliness, and performance of
the project. A disciplined software development and acquisition process
can maximize the likelihood of achieving the intended results
(performance) within established resources (costs) on schedule.
Although there is no standard set of practices that will ever guarantee
success, several organizations, such as the Software Engineering
Institute (SEI)[Footnote 4] and the Institute of Electrical and
Electronic Engineers (IEEE),[Footnote 5] as well as individual experts
have identified and developed the types of policies, procedures, and
practices that have been demonstrated to reduce development time and
enhance effectiveness. The key to having a disciplined system
development effort is to have disciplined processes in multiple areas,
including project planning and management, requirements management,
configuration management, risk management, quality assurance, and
testing. Effective processes should be implemented in each of these
areas throughout the project life cycle because change is constant.
Effectively implementing the disciplined processes necessary to reduce
project risks to acceptable levels is hard to achieve because a project
must effectively implement several best practices, and inadequate
implementation of any one practice may significantly reduce or even
eliminate the positive benefits of the others.
Successfully acquiring and implementing a new financial management
system requires a process that starts with a clear definition of the
organization's mission and strategic objectives and ends with a system
that meets specific information needs. We have seen many system efforts
fail because agencies started with a general need, such as improving
financial management, but did not define in precise terms (1) the
specific problems they were trying to solve, (2) what their operational
needs were, and (3) what specific information requirements flowed from
these operational needs. Instead, they plunged into the acquisition and
implementation process in the belief that these specifics would somehow
be defined along the way. The typical result was that systems were
delivered well past anticipated milestones; failed to perform as
expected; and, accordingly, were overbudget because of required costly
modifications.
Undisciplined projects typically show a great deal of productive work
at the beginning of the project, but the rework associated with defects
begins to consume more and more resources.[Footnote 6] In response,
processes are adopted in the hopes of managing what later turns out to
have been unproductive work. Generally, these processes are "too
little, too late" because sufficient foundations for building the
systems were not established or not established adequately. Experience
has shown that projects for which disciplined processes are not
implemented at the beginning are forced to implement them later when it
takes more time and the processes are less effective.[Footnote 7]
A major consumer of project resources in undisciplined efforts is
rework (also known as thrashing). Rework occurs when the original work
has defects or is no longer needed because of changes in project
direction. Disciplined organizations focus their efforts on reducing
the amount of rework because it is expensive. Experts have reported
that fixing a defect during the testing phase costs anywhere from 10 to
100 times the cost of fixing it during the design or requirements
phase.[Footnote 8] Projects that are unable to successfully address
their rework will eventually only be spending their time on rework and
the associated processes rather than on productive work. In other
words, the project will continually find itself reworking items.
HHS Had Not Effectively Implemented Disciplined Processes, Information
Technology Management Practices, and Human Capital Planning:
We found that HHS had adopted some best practices in its development of
UFMS. The project had support from senior officials and oversight by
independent experts, commonly called independent verification and
validation (IV&V) contractors. We also view HHS' decision to follow a
phased implementation to be a sound approach.
However, at the time of our review, HHS had not effectively implemented
several disciplined processes essential to reducing risks to acceptable
levels and therefore key to a project's success, and had adopted other
practices that put the project at unnecessary risk. HHS officials told
us that they had carefully considered the risks associated with
implementing UFMS and that they had put in place strategies to manage
these risks and to allow the project to meet its schedule within
budget. However, we found that HHS had focused on meeting its schedule
to implement the first phase of the new system at the Centers for
Disease Control and Prevention (CDC) in October 2004, to the detriment
of disciplined processes and thus had introduced unnecessary risks that
may compromise the system's cost, schedule, and performance. We would
now like to briefly highlight a few of the key disciplined processes
that HHS had not fully implemented at the time of our review. These
matters are discussed in detail in our report.
* Requirements management. Requirements are the specifications that
system developers and program managers use to design, develop, and
acquire a system. Requirements management deficiencies have
historically been a root cause of systems that do not meet their cost,
schedule, and performance objectives. Effective requirements
management practices are essential for ensuring the intended
functionality will be included in the system and are the foundation for
testing. We found significant problems in HHS' requirements management
process and that HHS had not developed requirements that were clear and
unambiguous.
* Testing. Testing is the process of executing a program with the
intent of finding errors.[Footnote 9] Without adequate testing, an
organization (1) is taking a significant risk that substantial defects
will not be detected until after the system is implemented and (2) does
not have reasonable assurance that new or modified systems will
function as planned. We found that HHS faced challenges in implementing
a disciplined testing program, because, first of all, it did not have
an effective requirements management system that produced clear,
unambiguous requirements upon which to build its testing efforts. In
addition, HHS had scheduled its testing activities, including those for
converting data from existing systems to UFMS, late in the
implementation cycle leaving little time to correct defects identified
before the scheduled deployment in October 2004.
* Project management and oversight using quantitative measures. We
found that HHS did not have quantitative metrics that allowed it to
fully understand (1) its capability to manage the entire UFMS effort;
(2) how problems in its management processes would affect the UFMS
cost, schedule, and performance objectives; and (3) the corrective
actions needed to reduce the risks associated with the problems
identified with its processes. Such quantitative measures are essential
for adequate project management oversight. Without such information,
HHS management can only focus on the project schedule and whether
activities have occurred as planned, not on whether the substance of
the activities achieved their system development objectives. As we note
in our report, this is not an effective practice.
* Risk management. We noted that HHS routinely closed its identified
risks on the premise that they were being addressed. Risk management is
a continuous process to identify, monitor, and mitigate risks to ensure
that the risks are being properly controlled and that new risks are
identified and resolved as early as possible. An effective risk
management process is designed to mitigate the effects of undesirable
events at the earliest possible stage to avoid costly consequences.
In addition, HHS' effectiveness in managing the processes associated
with its data conversion and UFMS interfaces will be critical to the
success of this project. For example, CDC's ability to convert data
from its existing systems to the new system will be crucial to helping
ensure that UFMS will provide the kind of data needed to manage CDC's
programs and operations. The adage "garbage in garbage out" best
describes the adverse impact. Furthermore, HHS expects that once UFMS
is fully deployed, the system will need to interface with about 110
other systems, of which about 30 system interfaces are needed for the
CDC deployment. Proper implementation of the interfaces between UFMS
and the other systems it receives data from and sends data to is
essential for providing data integrity and ensuring that UFMS will
operate as it should and provide the information needed to help manage
its programs and operations.
Compounding these UFMS-specific problems are departmentwide weaknesses
we have previously reported in information technology (IT) investment
management,[Footnote 10] enterprise architecture,[Footnote 11] and
information security.[Footnote 12] Specifically, HHS had not
established the IT management processes needed to provide UFMS with a
solid foundation for development and operation. Such IT weaknesses
increase the risk that UFMS will not achieve planned results within the
estimated budget and schedule. We will now highlight the IT management
weaknesses that HHS must overcome:
* Investment management. HHS had weaknesses in the processes it uses to
select and control its IT investments. Among the weaknesses we
previously identified, HHS had not (1) established procedures for the
development, documentation, and review of IT investments by its review
boards or (2) documented policies and procedures for aligning and
coordinating investment decision making among its investment management
boards. Until HHS addresses weaknesses in its selection or control
processes, IT projects like UFMS will face an increased likelihood that
the projects will not be completed on schedule and within estimated
costs.
* Enterprise architecture. While HHS is making progress in developing
an enterprise architecture that incorporates UFMS as a central
component, most of the planning and development of the UFMS IT
investment had occurred without the guidance of an established
enterprise architecture. An enterprise architecture is an
organizational blueprint that defines how an entity operates today and
how it intends to operate in the future and invest in technology to
transition to this future state. Our experience with other federal
agencies has shown that projects developed without the constraints of
an established enterprise architecture are at risk of being
duplicative, not well integrated, unnecessarily costly to maintain and
interface, and ineffective in supporting missions.
* Information security. HHS had not yet fully implemented the key
elements of a comprehensive security management program and had
significant and pervasive weaknesses in its information security
controls. The primary objectives of information security controls are
to safeguard data, protect computer application programs, prevent
unauthorized access to system software, and ensure continued
operations. Without adequate security controls, UFMS cannot provide
reasonable assurance that the system is protected from loss due to
errors, fraud and other illegal acts, disasters, and incidents that
cause systems to be unavailable.
Finally, we believe it is essential that an agency take the necessary
steps to ensure that it has the human capital capacity to design,
implement, and operate a financial management system. We found that
staff shortages and limited strategic workforce planning have resulted
in the project not having the resources needed to effectively design,
implement, and operate UFMS. We identified the following weaknesses:
* Staffing. HHS had not filled positions in the UFMS Program Management
Office that were identified as needed. Proper human capital planning
includes identifying the workforce size, skills mix, and deployment
needed for mission accomplishment and to create strategies to fill the
gaps. Scarce resources could significantly jeopardize the project's
success and have led to several key UFMS deliverables being
significantly behind schedule.
* Strategic workforce planning. HHS had not yet fully developed key
workforce planning tools, such as the CDC skills gap analysis, to help
transform its workforce so that it can effectively use UFMS. Strategic
workforce planning focuses on developing long-term strategies for
acquiring, developing, and retaining an organization's total workforce
(including full-and part-time federal staff and contractors) to meet
the needs of the future. Strategic workforce planning is essential for
achieving the mission and goals of the UFMS project. By not identifying
staff with the requisite skills to operate such a system and by not
identifying gaps in needed skills and filling them, HHS has not
optimized its chances for the successful implementation and operation
of UFMS.
Action Is Needed to Mitigate Risk:
To address the range of problems we have just highlighted, our report
includes 34 recommendations that focus on mitigating the risks
associated with this project. We made 8 recommendations related to the
initial deployment of UFMS at CDC that are specifically tied to
implementing critical disciplined processes. In addition, we
recommended that until these 8 recommendations are substantially
addressed, HHS delay the initial deployment. The remaining 25
recommendations were centered on developing an appropriate foundation
for moving forward and focused on (1) disciplined processes, (2) IT
security controls, and (3) human capital issues.
In its September 7, 2004, response to a draft of our report, HHS
disagreed regarding management of the project and whether disciplined
processes were being followed. In its comments, HHS characterized the
risk in its approach as the result, not of a lack of disciplined
processes, but of an aggressive project schedule. From our perspective,
this project demonstrated the classic symptoms of a schedule-driven
effort for which key processes had been omitted or shortcutted, thereby
unnecessarily increasing risk. As we mentioned at the outset of our
testimony, this is a multiyear project with an estimated completion
date in fiscal year 2007 and a total estimated cost of over $700
million.[Footnote 13] With a project of this magnitude and importance,
we stand by our position that it is crucial for the project to adhere
to disciplined processes that represent best practices. Therefore, in
order to mitigate its risk to an acceptable level, we continue to
believe it is essential for HHS to adopt and effectively implement our
34 recommendations.
In commenting on our draft report, HHS also indicated that actions had
either been taken, were under way, or were planned that address a
number of our recommendations. In addition, HHS subsequently contacted
us on September 23, 2004, to let us know that it had decided to delay
the implementation of a significant amount of functionality associated
with the CDC deployment from October 2004 until April 2005 in order to
address the issues that had been identified with the project. HHS also
provided us with copies of IV&V reports and other documentation that
had been developed since our review. Delaying implementation of
significant functionality at CDC is a positive step forward given the
risks associated with the project. This delay, by itself, will not
reduce the risk to an acceptable level, but will give HHS a chance to
implement the disciplined processes needed to do so.
HHS will face a number of challenges in the upcoming 6 months to
address the weaknesses in its management of the project that were
discussed in our report. At a high level, the key challenge will be to
implement an event driven project based on effectively implemented
disciplined processes, rather than a schedule-driven project. It will
be critical as well to address the problems noted in the IV&V reports
that were issued during and subsequent to our review. If the past is
prologue, taking the time to adhere to disciplined processes will pay
dividends in the long term.
Mr. Chairman, this concludes our statement. We would be pleased to
answer any questions you or other members of the Subcommittee may have
at this time.
Contacts and Acknowledgments:
For further information about this statement, please contact Jeffrey C.
Steinhoff, Managing Director, Financial Management and Assurance, who
may be reached at (202) 512-2600 or by e-mail at
[Hyperlink, steinhoffj@gao.gov], or Keith A. Rhodes, Chief
Technologist, Applied Research and Methodology Center for Engineering
and Technology, who may be reached at (202) 512-6412 or by e-mail at
[Hyperlink, rhodesk@gao.gov]. Other key contributors to this testimony
include Kay Daly, Michael LaForge, Chris Martin, and Mel Mench.
(193057):
FOOTNOTES
[1] The costs for this financial management system improvement effort
can be broken down into four broad areas: (1) National Institutes of
Health (NIH); (2) Centers for Medicare and Medicaid Services (CMS); (3)
all other HHS entities including the Centers for Disease Control and
Prevention (CDC); and (4) a system to consolidate the results of HHS'
financial management operations. HHS estimated that it would spend
about $110 million for NIH, $393 million for CMS, and $210 million for
the remaining HHS organizations. HHS has not yet developed an estimate
of the costs associated with integrating these efforts into a unified
financial management system.
[2] GAO, Financial Management Systems: Lack of Disciplined Processes
Puts Implementation of HHS' Financial System at Risk, GAO-04-1008
(Washington, D.C.: Sept. 23, 2004).
[3] This testimony is based on our report and does not assess HHS'
other financial management improvement efforts at the National
Institutes of Health (NIH) and Centers for Medicare and Medicaid
Services (CMS).
[4] SEI is a federally funded research and development center operated
by Carnegie Mellon University and sponsored by the U.S. Department of
Defense. The SEI objectives are to provide leadership in software
engineering and in the transition of new software engineering
technologies into practice.
[5] IEEE develops standards for a broad range of global industries
including the information technology and information assurance
industries.
[6] Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash.: Microsoft Press, 1996).
[7] McConnell, Rapid Development: Taming Wild Software Schedules.
[8] McConnell, Rapid Development: Taming Wild Software Schedules.
[9] Glenford J. Myers, The Art of Software Testing (John Wiley & Sons,
Inc., 1979).
[10] GAO, Information Technology Management: Governmentwide Strategic
Planning, Performance Measurement, and Investment Management Can Be
Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004).
[11] GAO, Information Technology: Leadership Remains Key to Agencies
Making Progress on Enterprise Architecture Efforts, GAO-04-40
(Washington, D.C.: Nov. 17, 2003).
[12] GAO, Information Security: Agencies Need to Implement Consistent
Processes in Authorizing Systems for Operation, GAO-04-376 (Washington,
D.C.: June 28, 2004).
[13] This includes the eventual incorporation of CMS and NIH.
