Financial Management Systems
Additional Efforts Needed to Address Key Causes of Modernization Failures Gao ID: GAO-06-184 March 15, 2006Billions of dollars have been spent governmentwide to modernize financial management systems that have often exceeded budgeted cost, resulted in delays in delivery dates and did not provide the anticipated system functionality when implemented. GAO was asked to identify (1) the key causes for financial management system implementation failures, and (2) the significant governmentwide initiatives currently under way that are intended to address the key causes of financial management system implementation failures. GAO was also asked to provide its views on actions that can be taken to help improve the management and control of agency financial management system modernization efforts.
GAO's work has linked financial management system implementation failures to three recurring themes: (1) disciplined processes, (2) human capital management, and (3) other information technology (IT) management practices. The predictable result of not effectively addressing these three areas has been numerous agency systems throughout the federal government that did not meet their cost, schedule, and performance objectives. Problems related to disciplined processes included requirements management, testing, data conversion and system interfaces, and risk and project management. Human capital management issues included strategic workforce planning, human resources, and change management. Other areas of IT management identified as problems included enterprise architecture, investment management, and information security. The Office of Management and Budget (OMB) has undertaken a number of initiatives to reduce the risks associated with acquiring and implementing financial management systems and addressing long-standing financial management problems. Some of these initiatives are in collaboration with others and are broad-based attempts to reform financial management operations governmentwide. First, OMB has developed and continues to evolve Federal Enterprise Architecture products and has required a mapping of agency architectures to this federal architecture. Another key OMB initiative is referred to as the financial management line of business which established centers of excellence to consolidate financial management activities for major agencies through cross-servicing arrangements. Finally, certain financial management activities and responsibilities have been reassigned to OMB, the Financial Systems Integration Office, and a Chief Financial Officers Council Committee. OMB's initiatives for reforming financial management systems governmentwide could help address the key causes of system implementation failures, but further actions are needed to fully define and implement the processes necessary to successfully complete these initiatives. OMB has correctly recognized the need to implement financial management systems as a governmentwide solution, rather than individual agency stove-piped efforts designed to meet a given entity's needs. Based on industry best practices, GAO believes that four concepts are integral to OMB's approach and key to successfully implementing financial management systems: a concept of operations provides the foundation, standard business processes promote consistency, a strategy for implementing the financial management line of business, and disciplined processes to help ensure successful implementations. GAO recognizes that implementing these concepts is a complex undertaking and raises a number of issues that have far-reaching implications for the government and private sector application service providers.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-184, Financial Management Systems: Additional Efforts Needed to Address Key Causes of Modernization Failures
This is the accessible text file for GAO report number GAO-06-184
entitled 'Financial Management Systems: Additional Efforts Needed to
Address Key Causes of Modernization Failures' which was released on
March 15, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
March 2006:
Financial Management Systems:
Additional Efforts Needed to Address Key Causes of Modernization
Failures:
GAO-06-184:
GAO Highlights:
Highlights of GAO-06-184, a report to congressional requesters:
Why GAO Did This Study:
Billions of dollars have been spent governmentwide to modernize
financial management systems that have often exceeded budgeted cost,
resulted in delays in delivery dates and did not provide the
anticipated system functionality when implemented. GAO was asked to
identify (1) the key causes for financial management system
implementation failures, and (2) the significant governmentwide
initiatives currently under way that are intended to address the key
causes of financial management system implementation failures. GAO was
also asked to provide its views on actions that can be taken to help
improve the management and control of agency financial management
system modernization efforts.
What GAO Found:
GAO‘s work has linked financial management system implementation
failures to three recurring themes: (1) disciplined processes, (2)
human capital management, and (3) other information technology (IT)
management practices. The predictable result of not effectively
addressing these three areas has been numerous agency systems
throughout the federal government that did not meet their cost,
schedule, and performance objectives. Problems related to disciplined
processes included requirements management, testing, data conversion
and system interfaces, and risk and project management. Human capital
management issues included strategic workforce planning, human
resources, and change management. Other areas of IT management
identified as problems included enterprise architecture, investment
management, and information security.
The Office of Management and Budget (OMB) has undertaken a number of
initiatives to reduce the risks associated with acquiring and
implementing financial management systems and addressing long-standing
financial management problems. Some of these initiatives are in
collaboration with others and are broad-based attempts to reform
financial management operations governmentwide. First, OMB has
developed and continues to evolve Federal Enterprise Architecture
products and has required a mapping of agency architectures to this
federal architecture. Another key OMB initiative is referred to as the
financial management line of business which established centers of
excellence to consolidate financial management activities for major
agencies through cross-servicing arrangements. Finally, certain
financial management activities and responsibilities have been
reassigned to OMB, the Financial Systems Integration Office, and a
Chief Financial Officers Council Committee.
OMB‘s initiatives for reforming financial management systems
governmentwide could help address the key causes of system
implementation failures, but further actions are needed to fully define
and implement the processes necessary to successfully complete these
initiatives. OMB has correctly recognized the need to implement
financial management systems as a governmentwide solution, rather than
individual agency stove-piped efforts designed to meet a given entity‘s
needs. Based on industry best practices, GAO believes that four
concepts are integral to OMB‘s approach and key to successfully
implementing financial management systems:
* a concept of operations provides the foundation,
* standard business processes promote consistency,
* a strategy for implementing the financial management line of
business, and
* disciplined processes to help ensure successful implementations.
GAO recognizes that implementing these concepts is a complex
undertaking and raises a number of issues that have far-reaching
implications for the government and private sector application service
providers.
What GAO Recommends:
To help reduce the risks associated with financial management system
implementation efforts, GAO recommends that the Director of OMB place a
high priority on the four concepts and underlying key issues needed to
help facilitate the implementation of the financial management line of
business and realignment initiatives across the government. OMB agreed
with GAO‘s recommendations and described its planned approach and steps
underway to improve financial management system modernization efforts.
www.gao.gov/cgi-bin/getrpt?GAO-06-184.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact McCoy Williams at (202)
512-9095 or Keith Rhodes at (202) 512-6412.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Agencies' Failure to Follow Best Practices in Three Key Areas Has
Hampered Successful Implementation of Financial Management Systems:
Federal Initiatives Under Way to Improve System Implementations:
Broad-Based Actions Needed to Implement Financial Management Systems
Governmentwide:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: IG Reports Reviewed:
Appendix III: Disciplined Processes:
Appendix IV: Comments from the Office of Management and Budget:
Appendix V: GAO Contacts and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Building Blocks for Financial Management Systems
Governmentwide and Summary of Key Issues:
Table 2: Problems Related to Disciplined Processes in Implementing
Financial Management Systems:
Table 3: Problems Related to Strategic Human Capital Management in
Implementing Financial Management Systems:
Table 4: Problems Related to Other IT Management Practices in
Implementing Financial Management Systems:
Table 5: OMB Initiatives to Reform Federal Financial Management System
Implementations:
Table 6: Federal Enterprise Architecture Reference Models:
Figures:
Figure 1: Percentage of Effort Associated with Undisciplined Projects:
Figure 2: Relationship between Requirements Development and Testing:
Abbreviations:
CDC: Centers for Disease Control and Prevention:
CFO: chief financial officer:
CIO: chief information officer:
COTS: commercial off-the-shelf:
Customs: U.S. Customs and Border Protection:
DOD: Department of Defense:
ERP: enterprise resource planning:
FFMIA: Federal Financial Management Improvement Act:
FSIO: Financial Systems Integration Office:
HHS: Department of Health and Human Services:
IEEE: Institute of Electrical and Electronic Engineers:
IG: inspector general:
Interior: Department of the Interior:
IRS: Internal Revenue Service:
IT: information technology:
JFMIP: Joint Financial Management Improvement Program:
NASA: National Aeronautics and Space Administration:
OFFM: Office of Federal Financial Management:
OMB: Office of Management and Budget:
OPM: Office of Personnel Management:
SEI: Software Engineering Institute:
VA: Department of Veterans Affairs:
United States Government Accountability Office:
Washington, DC 20548:
March 15, 2006:
The Honorable Todd R. Platts:
Chairman:
The Honorable Edolphus Towns:
Ranking Minority Member:
Subcommittee on Government Management, Finance, and Accountability:
Committee on Government Reform:
House of Representatives:
The federal government has long been plagued by financial management
system modernization efforts that have failed to meet their cost,
schedule, and performance goals. While agencies anticipate that the new
systems will provide reliable, useful, and timely data to support
managerial decision making, our work and that of others has shown that
has often not been the case. Modernizing financial management systems
is expensive but critical to instituting strong financial management as
called for by the Chief Financial Officers (CFO) Act of 1990,[Footnote
1] Federal Financial Management Improvement Act of 1996 (FFMIA),
[Footnote 2] and other financial management reform legislation. The CFO
Act calls for the improvement of financial management systems in
departments and major agencies throughout the federal government to
achieve the systematic measurement of performance, the development of
cost information, and the integration of program, budget, and financial
information for management reporting. FFMIA builds on the foundation
laid by the CFO Act by reflecting the need for CFO Act agencies to have
financial management systems that can generate reliable, useful, and
timely information with which to make fully informed decisions and to
ensure accountability on an ongoing basis.
Billions of dollars have been spent on developing and implementing
financial management systems throughout the federal government. These
systems support the interrelationships and interdependencies between
budget, cost, and management functions. Financial management systems
are critical for producing complete, reliable, timely, and consistent
financial information for use by the executive branch of the federal
government and the Congress in the financing, management, and
evaluation of federal programs. Many efforts are under way to implement
new core financial systems[Footnote 3] and supporting financial
management systems such as logistics, acquisition, and human resources.
However, recent efforts to modernize financial management systems have
often exceeded budgeted cost, resulted in delays in delivery dates, and
did not provide the anticipated system functionality and performance.
For example, as we testified in May 2004,[Footnote 4] the National
Aeronautics and Space Administration (NASA) was on its third attempt in
12 years to modernize its financial management processes and systems
and has spent about $180 million on two failed prior attempts to
implement core financial systems. NASA's current effort is estimated to
cost about $983 million. In another case, the Navy largely wasted
approximately $1 billion on four pilot Enterprise Resource
Planning[Footnote 5] (ERP) program efforts, without marked improvement
in its day-to-day operations, which resulted in four more stove-piped
systems that did not enhance overall efficiency at the Department of
Defense (DOD).[Footnote 6] The Navy is now working on a new project to
consolidate these four systems into one at an additional cost of $800
million.
Because of your concern about failures at some agencies to successfully
modernize or implement financial management systems, you asked us to
identify (1) the key causes for financial management system
implementation failures, which we define as financial management system
improvement efforts that did not meet cost, schedule, or performance
goals, and (2) the significant governmentwide initiatives currently
under way that are intended to address the key causes of financial
management system implementation failures. You also asked us to provide
our views on actions that can be taken to help improve the management
and control of agency financial management system modernization
efforts.
This report is based on our prior reports over the last 5 years that
focused on financial management system implementation efforts. We also
reviewed selected inspector general (IG) reports dealing with financial
management system implementations. We interviewed key CFO Council and
Office of Management and Budget (OMB) officials and reviewed their
existing oversight policies related to financial management systems as
well as related current initiatives under way. We did not evaluate the
federal government's overall information technology (IT) strategy or
whether a particular agency selected the most appropriate financial
management system. Our work for this report was performed in
Washington, D.C., from January 2005 through October 2005 in accordance
with U.S. generally accepted government auditing standards. Details on
our scope and methodology are included in appendix I and a list of
related IG reports that we reviewed are included in appendix II. The
background section describes elements of IT management, including
certain disciplined processes, human capital and other IT management
practices, and appendix III provides additional information on the
disciplined processes. Other related GAO reports are listed at the end
of this report.
Results in Brief:
From our analysis of prior reports, we identified several key causes of
financial management system implementation failures within three
recurring themes related to agencies not following best practices in
(1) systems development and implementation efforts (commonly referred
to as disciplined processes), (2) human capital management, and (3)
other IT management practices. Although the implementation of any major
system will never be a risk-free proposition, organizations that follow
and effectively implement disciplined processes, along with effective
human capital and other IT management practices, can reduce these risks
to acceptable levels. Our review of over 40 prior GAO and IG reports
found problems associated with the failure to effectively implement
disciplined processes in the areas of requirements management, testing,
data conversion and system interfaces, risk management, and project
management that can impact how a system functions and how it performs.
For example, ill-defined or incomplete requirements have been
identified by many experts as a root cause of system failure. As a case
in point, we recently reported[Footnote 7] that the initial deployment
of a new Army system intended to improve depot operations was still not
meeting user needs, and the Army expected to invest about $1 billion to
fully deploy the system. One reason that users had not been provided
with the intended systems capabilities was a breakdown in the
requirements management process. As a consequence, the Army implemented
error-prone, time-consuming manual workarounds to minimize disruption
to critical operations, and the financial management operations
continued to be affected by systems problems. Human capital management
problems were also identified as critical to successfully implementing
a new financial management system. Agencies have faced challenges in
implementing financial management systems due to human capital
management issues related to strategic workforce planning, human
resources, and change management. By not identifying staff with the
requisite skills to implement such systems and by not identifying gaps
in needed skills and filling them, agencies reduced their chances of
successfully implementing and operating new financial management
systems. Finally, deficiencies in other IT management practices have
hindered modernization efforts, including problems related to
enterprise architecture, investment management, and information
security management practices.
As the federal organization with key responsibility for federal
financial management systems, OMB has undertaken a number of
initiatives intended to reduce the risks associated with acquiring and
implementing financial management systems and addressing long-standing
financial management problems. Some of these initiatives are in
collaboration with the Chief Information Officers (CIO) and CFO
councils. OMB has recognized the need for standardization and including
key stakeholders in new work groups to develop systems requirements and
processes. While OMB has taken steps to accomplish its initiatives,
they are generally at the early stages of implementation, and a firm
foundation to address the long-standing problems that have impeded
success has not yet been established. OMB initiatives are under way in
the following key areas:
* Federal Enterprise Architecture-to build a comprehensive business-
driven (rather than technology focused) blueprint of the entire federal
government.
* Lines of Business-to develop business-driven common solutions that
span across the federal government, such as consolidating duplicative
financial management systems by using centers of excellence to provide
services.
* Joint Financial Management Improvement Program (JFMIP)[Footnote 8]
Realignment-to realign responsibilities for overseeing, developing,
testing, and publishing core financial systems requirements, including
the development of standard business processes.
OMB has developed the Federal Enterprise Architecture, which continues
to evolve, to maximize technology investments, but as we have
previously testified, questions remain about the Federal Enterprise
Architecture, including how it relates to agencies' enterprise
architectures.[Footnote 9] Regarding the financial management line of
business, OMB has developed an approach for outsourcing financial
management systems to a limited number of application service
providers,[Footnote 10] such as OMB designated centers of excellence or
private sector entities. With this initiative, OMB has correctly
recognized that enhancing the government's ability to implement
financial management systems that are capable of providing accurate,
reliable, and timely information on the results of operations needs to
be addressed as a governmentwide solution, rather than individual
agency stove-piped efforts designed to meet a given entity's needs.
This is a significant change in how agencies acquire new systems and
raises numerous complex issues that have far-reaching implications for
the government and private sector application service providers.
Therefore, strong executive support will be essential for these
modernization efforts. In addition, the actions resulting from the
realignment of the JFMIP in December 2004 can help streamline financial
management improvement efforts by providing additional policy and
oversight. However, OMB has not yet fully defined and implemented the
processes necessary to successfully complete the financial management
line of business and JFMIP realignment initiatives.
Specifically, based on industry best practices, we identified four key
concepts that are not yet fully developed in OMB's initiatives and
related processes. Careful consideration of these four concepts, each
one building upon the next, will be integral to the success of OMB's
initiatives and will help break the cycle of failure in implementing
financial management systems. The four concepts are: (1) developing a
concept of operations, (2) defining standard business processes, (3)
developing a strategy for ensuring that agencies are migrated to a
limited number of application service providers in accordance with
OMB's stated approach, and (4) defining and effectively implementing
disciplined processes necessary to properly manage the specific
projects. Table 1 summarizes the key issues raised in each of the four
areas.
Table 1: Building Blocks for Financial Management Systems
Governmentwide and Summary of Key Issues:
Building blocks for financial management systems governmentwide:
Concept of operations;
Key issues to be addressed:
* Defining financial management systems for consistent use in the
federal government;
* Establishing how development will result in a governmentwide solution
rather than individual agency stove-piped efforts;
* Linking to Federal Enterprise Architecture in user-friendly terms;
* Obtaining reliable information on the costs of federal financial
management system investments.
Building blocks for financial management systems governmentwide:
Standard business processes;
Key issues to be addressed:
* Developing governmentwide standard business processes to meet the
needs of federal agencies based on best practices;
* Encouraging agencies to adopt new processes, rather than automating
old ways of doing business;
* Providing consistency across government agencies and application
service providers;
* Supporting the processes of agencies that have unique needs.
Building blocks for financial management systems governmentwide:
Strategy for implementing the financial management line of business;
Key issues to be addressed:
* Assisting agencies in adopting a change management strategy that
reduces the risks of moving to this approach;
Focusing agency financial management system investment decisions on the
benefits of standard processes and application service providers;
* Facilitating the decision-making process used by agencies to select a
provider;
* Incorporating strategic workforce planning.
Building blocks for financial management systems governmentwide:
Disciplined processes;
Key issues to be addressed:
* Incorporating industry standards and best practices into
governmentwide guidance related to financial management system
implementation efforts;
* Reducing the risks and costs associated with data conversion and
interface efforts;
* Developing an oversight process.
Source: GAO.
[End of table]
While OMB has taken a number of steps to address these issues, more
remains to be done to facilitate the implementation of the financial
management line of business and JFMIP realignment initiatives across
the government. We make recommendations in this report regarding fully
integrating the four concepts into OMB's approach to help reduce the
risks associated with financial management system implementation
efforts. In written comments on a draft of this report, the Controller
of OMB agreed with our recommendations and described the approach and
steps that OMB is taking to improve financial management system
modernization efforts. OMB's comments are discussed further in the
Agency Comments and Our Evaluation section and reprinted in appendix
IV.
Background:
OMB plays a central role in setting federal financial management policy
and guidance. The CFO Act of 1990 established OMB's Office of Federal
Financial Management (OFFM), which has responsibility to provide
overall direction and leadership to the executive branch on financial
management matters by establishing financial management policies and
requirements, and by monitoring the establishment and operation of
federal government financial management systems. Among the key issues
OFFM addresses in addition to financial management systems, are agency
and governmentwide financial reporting, asset management, grants
management, improper payments, performance measurement, single audits,
and travel and purchase cards. Within OFFM, the Federal Financial
Systems Branch is responsible for orchestrating all of the elements of
the financial systems governmentwide into a coherent, coordinated
architecture. These elements include:
* agency financial management systems and JFMIP standards;
* interfaces between agency financial systems and other systems that
support business processes (e.g., human resources systems, procurement
systems, databases supporting performance management);
* common financial management services, including e-Travel, e-Learning,
Contractor Central Registry, Intragovernmental Payment and Collection
System, and Electronic Certification System; and:
* governmentwide accounting and other data consolidation systems.
Another office in OMB, the Office of Electronic Government and
Information Technology, has responsibility for providing overall
leadership and direction to the executive branch on electronic
government. In particular, this OMB office oversees implementation of
IT throughout the federal government, including monitoring and
consulting on agency technology efforts; advising the OMB Director on
the performance of IT investments, as well as identifying opportunities
for joint agency and governmentwide IT projects; and overseeing the
development of enterprise architectures within and across agencies,
which is being fulfilled through the Federal Enterprise
Architecture.[Footnote 11] This office also shares statutory IT
management responsibilities with the Office of Information and
Regulatory Affairs, which OMB was required to establish under the
Paperwork Reduction Act of 1995.[Footnote 12]
Finally, OMB is the preparer of the President's budget and provides
instructions to executive branch agencies to submit budget-related
information in accordance with the requirements of OMB Circular No. A-
11, Preparation, Submission and Execution of the Budget. OMB is
responsible for reviewing and evaluating IT spending across the federal
government and uses the IT spending information submitted by the
agencies during the budget formulation process to review requests for
agency financial management systems and other IT spending. Major agency
IT investments are reported to OMB individually. OMB Circular No. A-11
defines a major IT investment as a system or project that requires
special management attention because of its importance to an agency's
mission, or has significant program or policy implications, among other
criteria. Financial management systems costing more than $500,000
annually are considered major IT investments. OMB Circular No. A-11
also requires agencies to use Exhibit 300, Capital Asset Plan and
Business Case, to describe the business case for the investment, which
serves as the primary means of justifying IT investment proposals as
well as managing IT investments once they are funded.
Elements of IT Management:
Best practices are tried and proven methods, processes, techniques, and
activities that organizations define and use to minimize risks and
maximize chances for success. As we have previously reported,[Footnote
13] using best practices related to IT acquisitions can result in
better outcomes--including cost savings, improved service and product
quality, and ultimately, a better return on investment. We and others,
such as the Software Engineering Institute (SEI),[Footnote 14] have
identified and promoted the use of a number of best practices
associated with acquiring IT systems. For the purposes of this report,
we have identified various elements of IT management and categorized
them as disciplined processes, human capital and other IT management
practices that are critical elements for minimizing the risks related
to financial management system implementations. These areas are
interrelated and interdependent, collectively providing an agency with
a comprehensive understanding both of current business approaches and
of efforts (under way or planned) to change these approaches and a
means to implement those changes. Understanding the relationships among
these areas can help an agency determine how it is applying its
resources, analyze how to redirect these resources in the face of
change, implement such redirections, and measure success. With this
decision-making capability, the agency is better positioned to deploy
financial management systems and direct appropriate responses to
unexpected changes in its environment. The following sections provide
additional background information on the key elements of IT management
discussed in this report, including disciplined processes, human
capital and other IT management practices.
Disciplined Processes:
Disciplined processes are fundamental to successful systems
implementation efforts and have been shown to reduce the risks
associated with software development and acquisition to acceptable
levels. A disciplined software development and acquisition process can
maximize the likelihood of achieving the intended results (performance)
within established resources (costs) on schedule. Although there is no
standard set of practices that will ever guarantee success, several
organizations, such as the SEI and the Institute of Electrical and
Electronic Engineers (IEEE),[Footnote 15] as well as individual
experts, have identified and developed the types of policies,
procedures, and practices that have been demonstrated to reduce
development time and enhance effectiveness. The key to having a
disciplined system development effort is to have disciplined processes
in multiple areas, including requirements management, testing, data
conversion and system interfaces, configuration, risk and project
management, and quality assurance. Effective processes should be
implemented in each of these areas throughout the project life cycle
because change is constant. Effectively implementing the disciplined
processes necessary to reduce project risks to acceptable levels is
difficult to achieve because a project must effectively implement
several best practices, and inadequate implementation of any one may
significantly reduce or even negate the positive benefits of the
others.
Figure 1 shows how organizations that do not effectively implement the
disciplined processes lose the productive benefits of their efforts as
a project continues through its development and implementation cycle.
Although undisciplined projects show a great deal of what appears to be
productive work at the beginning of the project, the rework associated
with defects begins to consume more and more resources. In response,
processes are adopted in the hopes of managing what later turns out, in
reality, to have been unproductive work. Generally, these processes are
"too little, too late" because sufficient foundations for building the
systems were not done or not done adequately. Experience in both the
private sector and the government has shown that projects for which
disciplined processes are not implemented at the beginning then must be
implemented later, when it takes more time and they are less
effective.[Footnote 16]
Figure 1: Percentage of Effort Associated with Undisciplined Projects:
[See PDF for image]
[End of figure]
As shown in figure 1, a major consumer of project resources in
undisciplined efforts is rework (also known as thrashing). Rework
occurs when the original work has defects or is no longer needed
because of changes in project direction. Disciplined organizations
focus their efforts on reducing the amount of rework because it is
expensive. Fixing a requirements defect after the system is released
costs anywhere from 10 to 100 times the cost of fixing it when the
requirements are defined.[Footnote 17] Projects that do not
successfully address rework will eventually spend even more effort on
rework and the associated processes rather than on productive work. In
other words, the project will continually require reworking items.
Human Capital Management:
People--human capital--are a critical element to transforming
organizations to meet the challenges of the 21st century. Recognizing
this, we first added strategic human capital management as a
governmentwide high-risk issue in January 2001,[Footnote 18] and
although progress has been made, continued to include it on the latest
high-risk list issued in January 2005.[Footnote 19] Strategic human
capital management for financial management projects includes
organizational planning, staff acquisition, and team development. Human
capital planning is necessary for all stages of the system
implementation. It is important that agencies incorporate strategic
workforce planning by (1) aligning an organization's human capital
program with its current and emerging mission and programmatic goals
and (2) developing long-term strategies for acquiring, developing, and
retaining an organization's total workforce to meet the needs of the
future. This incorporates a range of activities from identifying and
defining roles and responsibilities, to identifying team members, to
developing individual competencies that enhance performance. It is
essential that an agency take the necessary steps to ensure that it has
the human resources to design, implement, and operate a financial
management system. In addition, organizational change management, which
is the process of preparing users for the business process changes that
usually accompany implementation of a new system, is another important
human capital element.
Strategic workforce planning is essential for achieving the mission and
goals of financial management system projects. As we have
reported,[Footnote 20] there are five key principles that strategic
workforce planning should address:
* Involve top management, employees, and other stakeholders in
developing, communicating, and implementing the strategic workforce
plan.
* Determine the critical skills and competencies that will be needed to
achieve current and future programmatic results.
* Develop strategies that are tailored to address gaps in the number,
deployment, and alignment of human capital approaches for enabling and
sustaining the contributions of all critical skills and competencies.
* Build the capability needed to address administrative, educational,
and other requirements important to support workforce planning
strategies.
* Monitor and evaluate the agency's progress toward its human capital
goals and the contribution that human capital results have made toward
achieving programmatic results.
Having adequate and sufficient human resources with the requisite
training and experience to successfully implement a financial
management system is another critical success factor. According to OMB,
qualified federal IT project managers are our first line of defense
against the cost overruns, schedule slippage, and poor performance that
threaten agencies' ability to deliver efficient and effective services
to citizens. In July 2004, OMB issued a memorandum[Footnote 21] to help
agencies comply with fiscal year 2005 budget guidance that instructed
agencies to ensure "by September 30, 2004, all major projects are
managed by project managers qualified in accordance with CIO Council
guidance."[Footnote 22] The CIO Council's Federal IT Project Manager
Guidance Matrix and Federal IT Project Management Validation define
levels of complexity for IT projects/systems, identify appropriate
competencies and experience, suggest education and training sources,
and serve as a tool for validating IT project manager credentials. IT
project managers are expected to achieve and demonstrate baseline
skills in applicable competency areas listed in the Office of Personnel
Management (OPM) Interpretive Guidance for Project Manager Positions.
The OMB memorandum also required agencies to submit a plan to meet the
guidance on project manager qualifications and document the approach,
milestones, and schedule. The plans should also follow OPM's Workforce
Planning Model and Human Capital Assessment and Accountability
Framework.
Changing an organization's business processes is not an easy task.
Managing culture and process change in large, diverse, organizationally
and geographically decentralized agencies is a much greater challenge.
Frequently, the greatest difficulties lie not in managing the technical
or operational aspects of change, but in managing the human dimensions
of change. Some experts caution that unless planning and accountability
for change management are given a separate focus, the efforts will not
be managed well. Management roles in implementing a new system include
establishing business goals, realistic expectations, accountability,
and leading cultural change necessary to accept the capabilities of a
new system. During the implementation phase especially, agency
executives must be in the forefront in dealing with the social,
psychological, and political resistance to changing the way work is
done. Executives must also recognize that their own roles and
responsibilities may need to undergo change as well.
Other IT Management Practices:
Weaknesses in other IT management processes also increase the risks
associated with financial management system implementation efforts.
Developing an enterprise architecture, establishing IT investment
management policies, and addressing information security weaknesses are
critical to ensuring successful system implementation.
OMB Circular No. A-130,[Footnote 23] which establishes executive branch
policies pursuant to the Paperwork Reduction Act of 1995[Footnote 24]
and the Clinger-Cohen Act of 1996[Footnote 25] among other laws,
requires agencies to use architectures. A well-defined enterprise
architecture provides a clear and comprehensive picture of the
structure of any enterprise by providing models that describe in
business and technology terms how the entity operates today and
predicts how it will operate in the future. It also includes a plan for
transitioning to this future state. Enterprise architectures are
integral to managing large-scale programs. Managed properly, an
enterprise architecture can clarify and help optimize the
interdependencies and relationships among an organization's business
operations and the underlying IT infrastructure and applications that
support these operations. Employed in concert with other important
management controls, architectures can greatly increase the chances
that organizations' operational and IT environments will be configured
to optimize mission performance. To aid agencies in assessing and
improving enterprise architecture management, we issued guidance
establishing an enterprise architecture management framework.[Footnote
26] The underpinning of this framework is a five-stage maturity model
outlining steps toward achieving a stable and mature process for
managing the development, maintenance, and implementation of an
enterprise architecture.
IT investment management provides for the continuous identification,
selection, control, life-cycle management, and evaluation of IT
investments. The Clinger-Cohen Act lays out specific aspects of the
process that agency heads are to implement to maximize the value of the
agency's IT investments. In addition, OMB and GAO have issued
guidance[Footnote 27] for agencies to use in implementing the Clinger-
Cohen Act requirements for IT investment management. For example, we
issued guidance establishing an IT investment management
framework.[Footnote 28] This framework is also a maturity model
composed of five progressive stages of maturity that an agency can
achieve in its IT investment management capabilities. These stages
range from creating investment awareness to developing a complete
investment portfolio to leveraging IT for strategic outcomes. The
framework can be used both to assess the maturity of an agency's
investment management processes and as a tool for organizational
improvement.
The Federal Information Security Management Act of 2002[Footnote 29]
provides the overall framework for ensuring the effectiveness of
information security controls that support federal operations and
assets and requires agencies and OMB to report annually to the Congress
on their information security programs. OMB Circular No. A-130 also
requires agencies to protect information commensurate with the risk and
magnitude of the harm that would result from the loss, misuse, or
unauthorized access to or modification of such information. The
reliability of operating environments, computerized data, and the
systems that process, maintain, and report these data is a major
concern to federal entities that have distributed networks that enable
multiple computer processing units to communicate with each other. Such
distributed networks increase the risk of unauthorized access to
computer resources and possible data alteration. Effective
departmentwide information security controls will help reduce the risk
of loss due to errors, fraud, and other illegal acts, disasters, or
incidents that cause systems to be unavailable. Inadequate security and
controls can adversely affect the reliability of the operating
environments in which financial management systems and their
applications operate.
Agencies' Failure to Follow Best Practices in Three Key Areas Has
Hampered Successful Implementation of Financial Management Systems:
We reviewed numerous prior GAO and IG reports and identified several
problems related to agencies' implementation of financial management
systems in three recurring and overarching themes: disciplined
processes, human capital and other IT management practices. Simply put,
the agencies were not following best practices in these three critical
areas. The predictable result of not effectively addressing these three
areas has been numerous agency systems throughout the federal
government that did not meet their cost, schedule, and performance
objectives. We have issued governmentwide reports on other IT
management practices including agencies' enterprise
architecture,[Footnote 30] IT investment management,[Footnote 31] and
information security[Footnote 32] and therefore will not be addressing
those issues further in this report. However, broad-based actions are
needed to address the problems repeatedly experienced at the agencies
as they continue to struggle to implement new financial management
systems. Many of the systems we reviewed had at least one problem in
each of the three critical areas. While there was some overlap in these
three areas, we selected examples that best illustrate the specific
problems in each area.
Disciplined Processes Have Not Been Fully Used:
From our review of over 40 prior reports, we identified a number of key
problem areas in disciplined processes related to requirements
management, testing, data conversion and system interfaces, risk
management, and project management activities. Inadequate
implementation of disciplined processes can manifest itself in many
ways when implementing a financial management system and the failure to
properly implement disciplined processes in one area can undermine the
work in all the other areas and cause significant problems. Table 2
summarizes and provides examples for some of the problems we identified
from prior reports that can be expected when agencies do not
effectively implement the disciplined processes necessary to manage
their financial management system implementation projects.
Table 2: Problems Related to Disciplined Processes in Implementing
Financial Management Systems:
Agency/related report(s): U.S. Customs and Border Protection (Customs)
(GAO-05-267);
Key problem area(s): Project management;
Observations: More than 3 years into its second attempt, Customs had
relaxed system quality standards and started new phases despite system
defects. Correcting such defects would consume resources (e.g., people)
at the expense of later system releases.
Agency/related report(s): Department of Defense;
* Army Logistics Modernization Program (GAO-05-441);
Key problem area(s): Requirements, Testing, Data conversion and system
interfaces;
Observations: Tobyhanna Army Depot could not accurately report on its
financial operations, which also affect the depot's ability to set
prices. Subsequent deployments of the system costing $1 billion have
been delayed;
* Defense Integrated Military Human Resources System (GAO-05-189);
Key problem area(s): Requirements, Project management;
Observations: DOD accepted the design of the first system phase in
November 2004 and was proceeding with development, but program
responsibility was diffused and requirements were not complete;
* Navy Enterprise Resource Planning (GAO-05-858);
Key problem area(s): Data conversion and system interfaces, Project
management;
Observations: The Navy largely wasted about $1 billion in four pilot
efforts that were not interoperable and started a new project to
converge them into a single program which is expected to cost another
$800 million.
Agency/related report(s): Department of Health and Human Services (HHS)
(GAO-04-1008) (GAO-04-1089T);
Key problem area(s): Requirements, Testing, Data conversion and system
interfaces, Risk management, Project management;
Observations: HHS had not developed sufficient quantitative measures
for determining the impact of many of the process weaknesses and did
not determine until less than 1 month before the scheduled deployment
date that the $210 million project should be delayed by 6 months.
Agency/related report(s):
* Department of the Interior (Interior) (Bureau of Indian Affairs)
(GAO/AIMD-00-259);
Key problem area(s): Requirements, Testing, Data conversion and system
interfaces;
Observations: Over 5 years after the project was first fielded, only
one function was considered successfully implemented, and Interior was
looking for a replacement system.
Agency/related report(s): Internal Revenue Service (IRS) (GAO-02-356)
(GAO-05-46) (GAO-05-566) (GAO-05-774);
Key problem area(s): Risk management, Project management;
Observations: Total life-cycle costs for full deployment of the initial
release of a new core accounting system had increased by almost $74
million, and project completion had been delayed by 15 months because
of an inability to timely resolve key system design, integration, and
performance issues.
Agency/related report(s): National Aeronautics and Space Administration
(GAO-04-255) (GAO-04-754T) (GAO-05-799R);
Key problem area(s): Requirements, Testing, Project management;
Observations: After a total of 12 years and about $180 million on two
prior failed efforts, NASA was on its third attempt at modernizing its
financial systems and still could not produce auditable financial
statements or specific information for managing NASA projects.
Agency/related report(s): Office of Personnel Management (GAO-05-237);
Key problem area(s): Requirements, Risk management, Project management;
Observations: OPM planned to award the contract for a system to process
retirement claims at the end of January 2005 with implementation by the
end of fiscal year 2008 at a total cost of about $294 million despite
the lack of disciplined processes in key areas. OPM had not awarded the
contract at the end of our field work.
Agency/related report(s): Department of Transportation (Transportation
IG, FI-2001-074 and FI-2005-009);
Key problem area(s): Testing, Data conversion and system interfaces;
Observations: The Department of Transportation transitioned to a new
accounting system in fiscal year 2004, but the system was not able to
account for expected loan repayments from grantees, which were valued
at $604 million on September 30, 2004.
Agency/related report(s): Department of Veterans Affairs (VA) (VA IG,
04-01371-177);
Key problem area(s): Testing Data conversion and system interfaces,
Project management;
Observations: Patient services and medical center operations were
interrupted when supplies were not available because of inaccurate
inventory data that had been transferred to its new financial system.
After numerous problems, VA halted implementation of the system for
which it reported to have spent almost $250 million.
Source: GAO analysis based on prior GAO and IG reports.
[End of table]
The following provides more specific details on three of the examples
of financial management system implementation problems related to the
lack of disciplined processes.
* In May 2004, we first reported[Footnote 33] our concerns with the
requirements management and testing processes used by the Army in the
implementation of the Logistics Modernization Program and the problems
being encountered after it became operational in July 2003. At the time
of our initial report, the Army decided that future deployments would
not go forward until they had reasonable assurance that the deployed
system would operate as expected for a given deployment. However, as we
reported in June 2005,[Footnote 34] the Army had not effectively
addressed its requirements management and testing problems and data
conversion weaknesses had hampered the Army's ability to address the
problems that need to be corrected before the system can be fielded to
other locations. For example, the system cannot properly recognize
revenue nor bill customers. Data conversion problems resulted in
general ledger account balances that were not properly converted to the
new system in July 2003, and these differences remained unresolved
almost 18 months later. These weaknesses adversely affected the Army's
ability to set the prices for the work performed at the Tobyhanna Army
Depot. In addition, data conversion problems resulted in excess items
being ordered and shipped to Tobyhanna. As noted in our June 2005
report, three truckloads of locking washers (for bolts) were mistakenly
ordered and received, and subsequently returned, because of data
conversion problems. As a result of the problems, the Army has
implemented error-prone, time-consuming manual workarounds as a means
to minimize disruption to critical operations; however, the depot's
financial management operations continue to be adversely affected by
systems problems.
* NASA has struggled to implement a modern integrated financial
management system. After two failed efforts over 12 years and about
$180 million, NASA embarked on a third effort that is expected to cost
about $983 million. We have previously identified problems and made
recommendations to NASA related to requirements, testing, and project
management as well as problems with human capital and other IT
management issues related to this effort. For example, NASA had not
implemented quantitative metrics to help gauge the effectiveness of its
requirements management process. Such metrics would be particularly
important for NASA to address the root causes of system defects and be
reasonably assured that its processes would result in a system that
meets its business needs. However, in our September 2005
report,[Footnote 35] we found that overall progress implementing our
recommendations had been slow. From our perspective, of the 45
recommendations we made in prior reports, NASA had taken sufficient
action to close 3 recommendations and had partially implemented 13, but
29 recommendations remained open. Furthermore, in November 2004, NASA's
independent auditor reported that NASA's new financial system, which
was implemented in June 2003, could not produce auditable financial
statements for fiscal year 2004 and did not comply with the
requirements of FFMIA.[Footnote 36] Key areas of concern included the
core financial module's inability to (1) produce transaction-level
detail in support of financial statement account balances, (2) identify
adjustments or correcting entries, and (3) correctly and consistently
post transactions to the right accounts.
* In August 2004, the VA IG reported[Footnote 37] that the effect of
transferring inaccurate data to its new core financial system at a
pilot location interrupted patient care and medical center operations.
This raised concerns that similar conversion problems would occur at
other VA facilities if the conditions identified were not addressed and
resolved nationwide prior to roll out. Some of the specific conditions
the IG noted were that contracting and monitoring of the project were
not adequate, and the deployment of the new system encountered multiple
problems including those related to software testing, data conversion
and system interfaces, and project management. When the new financial
system was deployed at the pilot location in October 2003, it did not
function as project managers had expected because of inaccurate or
incomplete vendor and inventory system data. As a result of these
problems, patient care was interrupted by supply outages and other
problems. The inability to provide sterile equipment and needed
supplies to the operating room resulted in the cancellation of 81
elective surgeries for a week in both November 2003 and February 2004.
In addition, the operating room was forced to operate at two-thirds of
its prior capacity. Because of the serious nature of the problems
raised with the new system, VA management decided to focus on
transitioning back to the previous financial management software at the
pilot location and assemble a senior leadership team to examine the
results of the pilot and make recommendations to the VA Secretary
regarding the future of the system.
Human Capital Management Problems Impede Financial Systems Development
and Deployment:
Effective human capital management is critical to the success of
systems implementations. As we previously reported in our Executive
Guide: Creating Value Through World-class Financial Management,
[Footnote 38] having staff with the appropriate skills is key to
achieving financial management improvements, and managing an
organization's employees is essential to achieving results. By not
identifying staff with the requisite skills to implement such systems
and by not identifying gaps in needed skills and filling them, agencies
reduce their chances of successfully implementing and operating new
financial management systems. For example, in our prior report on
building the IT workforce,[Footnote 39] we found that in the 1990s the
initial rounds of downsizing were set in motion without considering the
longer-term effects on agencies' IT performance capacity. Additionally,
a number of individual agencies drastically reduced or froze their
hiring efforts for extended periods. Consequently, following a decade
of downsizing and curtailed investments in human capital, federal
agencies face skills, knowledge, and experience imbalances, especially
in their IT workforces. Without corrective action, this situation will
worsen, especially in light of the numbers of federal civilian workers
becoming eligible to retire in the coming years. In this regard, we are
emphasizing the need for additional focus on key problem areas we
identified from prior reports including strategic workforce planning,
human resources, and change management. Examples for some of the human
capital management problems we identified in prior reports that hamper
the implementation of new financial management systems are summarized
in table 3.
Table 3: Problems Related to Strategic Human Capital Management in
Implementing Financial Management Systems:
Agency/related report(s): U.S. Customs and Border Protection (GAO-05-
267);
Key problem area(s): Strategic workforce planning Human resources
Change management;
Observations: A human capital strategy that provided both near-and long-
term solutions to the program office's human capital capacity
limitations was needed. Key change management actions were not being
implemented. The schedule was extended by 3 years and estimated costs
increased by about $1 billion.
Agency/related report(s): Department of Health and Human Services (GAO-
04-1008);
Key problem area(s): Strategic workforce planning Human resources;
Observations: Strategic workforce planning was incomplete and ongoing
staff shortages had played a role in key deliverables being
significantly behind schedule.
Agency/related report(s): Department of the Interior (Bureau of Indian
Affairs) (GAO/AIMD-00-259);
Key problem area(s): Change management;
Observations: Without taking time to reexamine and revise its business
processes, Interior was not able to maximize the potential benefits of
the new system and instead may perpetuate outmoded ways of doing
business.
Agency/related report(s): Internal Revenue Service (GAO-05-46) (GAO- 05-
774);
Key problem area(s): Strategic workforce planning Human resources;
Observations: IRS had not defined or implemented a human capital plan
for obtaining, developing, and retaining requisite human capital
resources and experienced significant cost increases and schedule
delays.
Agency/related report(s): National Aeronautics and Space Administration
(GAO-04-118);
Key problem area(s): Human resources;
Observations: Personnel shortages at Marshall Space Flight Center for
several months affected the core financial project and resulted in
additional costs of nearly $400,000 for extra hours worked.
Agency/related report(s): Office of Personnel Management (GAO-05-237);
Key problem area(s): Change management;
Observations: OPM had not developed a detailed transition plan to help
prepare users for changes to their job responsibilities. The award of
the contract for the new system was delayed because OMB asked to review
a revised business case for the new system.
Agency/related report(s): Department of Veterans Affairs (VA IG, 04-
01371-177);
Key problem area(s): Human resources;
Observations: Conversion to the new system was disrupted because
management did not ensure that inventory management staff were trained
as required. The duties of the Project Director and Contracting Officer
Technical Representative were too onerous for one individual to
adequately manage.
Source: GAO analysis based on prior GAO and IG reports.
[End of table]
The following provides more specific details on two of the examples of
the types of human capital management problems we found.
In May 2002, we first reported[Footnote 40] that the Customs
Modernization Office did not have the people in place to perform
critical system acquisition functions and did not have an effective
strategy for meeting its human capital needs. Customs had decided to
compress its time frame for delivering its new system from 5 to 4 years
and was taking a schedule-driven approach to acquiring the system
because of the system's national importance. This exacerbated the level
of project risk by introducing more overlap among incremental system
releases and stretching critical resources. In our most recent report
issued in March 2005,[Footnote 41] we found that although Customs had
developed a staffing plan, it had not been approved and was already out
of date because the modernization office subsequently implemented a
reorganization that transferred government and contractor personnel to
the modernization office. We also observed that changes in roles and
responsibilities had the modernization office and the contractor
sharing development duties of the new system. Finally, Customs
developed a revised organizational change approach with new change
management activities, but key actions associated with the revised
approach were not planned for implementation because the funding
request for fiscal year 2005 did not fully reflect the revised
approach. In July 2004, Customs extended delivery of the last release
from fiscal year 2007 to fiscal year 2010, adding a new release for
screening and targeting, and increasing the life-cycle cost estimate by
about $1 billion to $3.1 billion. The new schedule reflected less
overlap between future releases. While Customs, which is now under the
Department of Homeland Security, has taken important actions to help
address release-by-release cost and schedule overruns that we
previously identified, we concluded that it was unlikely that these
actions would prevent the past pattern of overruns from recurring
because the Department of Homeland Security had relaxed system quality
standards, so that milestones were being passed despite material system
defects, and because correcting these defects will ultimately require
the program to expend resources, such as people and test environments,
at the expense of later system releases (some of which are now under
way).
We reported, in September 2004,[Footnote 42] that staff shortages and
limited strategic workforce planning resulted in HHS not having the
resources needed to effectively design and operate its new financial
management system. HHS had taken the first steps in strategic workforce
planning. For example, the Centers for Disease Control and Prevention
(CDC), where the first deployment was scheduled, was the only operating
division that had prepared a competency report, but a skills gap
analysis and training plan for CDC had not been completed. In addition,
many government and contractor positions on the implementation project
were not filled as planned. For example, an independent verification
and validation contractor reported that some key personnel filled
multiple positions and their actual available time was inadequate to
perform the allocated tasks. As a result, some personnel were
overworked, which, according to the independent verification and
validation contractor could lead to poor morale. The organization chart
for the project showed that the project team was understaffed and that
several integral positions were vacant or filled with part-time
detailees. While HHS and the systems integrator had taken measures to
acquire additional human resources for the implementation of the new
financial management system, we concluded that scarce resources could
significantly jeopardize the project's success and lead to several key
deliverables being significantly behind schedule. In September 2004,
HHS decided to delay its first scheduled deployment at CDC by 6 months
in order to address these and other issues identified with the project.
Other IT Management Practices Were Not Fully Implemented:
We identified a number of key problems related to other IT management
practices. Specifically, we found that in planning and developing new
financial management systems, agencies had not adequately considered
their existing IT management processes and framework. Through our
research into IT management best practices and our evaluation of agency
IT management performance, we have identified a set of essential and
complementary management disciplines.[Footnote 43] These include key
areas where we found problems such as enterprise architecture,
investment management, and information security, among others. Using
the results of this research and evaluation, we have developed various
management frameworks and guides and reported on numerous IT management
weaknesses at individual agencies. Table 4 summarizes and provides
examples for some of the key problems we found described in prior
reports on financial management system implementations related to other
IT management areas not previously discussed.
Table 4: Problems Related to Other IT Management Practices in
Implementing Financial Management Systems:
Agency/related report(s): Department of Defense (GAO-04-731R) (GAO-05-
140T) (GAO-05-381) (GAO-05-702);
Key problem area(s): Enterprise architecture Investment management;
Observations: Recent legislation pertaining to defense business
systems, enterprise architecture, accountability, and modernization, if
properly implemented, should improve system investment activities.
However, DOD's transformation efforts have not adequately addressed key
underlying causes of past reform failures.
Agency/related report(s): Department of Health and Human Services (GAO-
04-1008);
Key problem area(s): Enterprise architecture Investment management
Information security;
Observations: HHS planned and developed its new system using the
agency's existing IT management processes that had known weaknesses in
enterprise architecture, investment management, and information
security.
Agency/related report(s): Department of the Interior (Bureau of Indian
Affairs) (GAO/AIMD-00-259);
Key problem area(s): Enterprise architecture;
Observations: Not having a complete information systems architecture to
guide its new system and other projects was a major challenge for
Interior.
Agency/related report(s): National Aeronautics and Space Administration
(GAO-04-754T) (GAO-05-799R);
Key problem area(s): Enterprise architecture;
Observations: Key architecture management processes were not
established, the architecture was missing important content, and NASA
had already implemented system components not mapped to the
architecture.
Agency/related report(s): Office of Personnel Management (GAO-05-237);
Key problem area(s): Investment management Information security;
Observations: OPM lacked policies and procedures for guiding the
investment board's oversight responsibilities and had not developed
specific security plans.
Agency/related report(s): Small Business Administration (SBA/IG-3-32);
Key problem area(s): Information security;
Observations: The system was not fully secure and potential breaches of
security could occur and go undetected. Due to cost issues for
implementing phase I, which exceeded the entire $6.4 million budget for
full implementation, remaining phases were put on hold.
Source: GAO analysis based on prior GAO and IG reports.
[End of table]
The following provides more specific details on two of the examples of
other problems related to IT management that have had an impact on
financial management system implementation projects.
* For several years, we have reported that deficiencies in DOD's
enterprise architecture and IT investment management policies are
contributing factors to DOD's stovepiped, duplicative, and
nonintegrated systems environment. In May 2004, we reported[Footnote
44] that we had not seen any significant change in the content of DOD's
architecture or in DOD's approach to investing billions of dollars
annually in existing and new systems. Few actions had been taken to
address prior recommendations, which were aimed at improving DOD's
plans for developing the next version of the architecture and
implementing the institutional means for selecting and controlling both
planned and ongoing business systems investments. In April 2005, we
reported[Footnote 45] that DOD still did not have an effective
departmentwide management structure for controlling business
investments despite DOD requesting over $13 billion in fiscal year 2005
to operate, maintain, and modernize its existing duplicative business
systems. In addition, because DOD lacked a well-defined business
enterprise architecture and transition plan, billions of dollars
continued to be at risk of being spent on systems that would be
duplicative, not interoperable, cost more to maintain than necessary,
and would not optimize mission performance and accountability. In July
2005, we reported[Footnote 46] that despite spending almost 4 years and
about $318 million, DOD still did not have an effective architecture
program, and as a result its modernization program remained a high
risk.
* We reported, in February 2005,[Footnote 47] that OPM had implemented
selected processes in the areas of systems acquisition, investment
management, and information security; however, many processes were not
sufficiently developed, were still under development, or were planned
for future development. Although OPM had an executive steering
committee chaired by the deputy associate director of the Center for
Retirement and Insurance Services that acted as an IT investment
management board for the new retirement system, program officials were
not aware of formal policies or procedures guiding the board's
oversight responsibilities or activities. Agency officials stated that
they would define such a governance structure for the retirement system
project during the contract award process. In addition, the agency had
not yet developed security plans for the licensed technology and data
conversion portions of the new system. Agency officials said they did
not have detailed security requirements for the licensed technology
portion of the new system, although the request for proposals
identified the need for high-level security requirements. They planned
to develop detailed security requirements after awarding the licensed
technology contract to a vendor. Without fully developed security plans
and security requirements for the licensed technology and data
conversion portions of the new system, OPM increased the risk that both
it and its vendors would not meet information security needs for these
portions of the program expected to be implemented in fiscal year 2008.
Federal Initiatives Under Way to Improve System Implementations:
As the federal organization with key responsibility for federal
financial management systems, OMB has undertaken a number of
initiatives related to acquiring and implementing financial management
system capabilities. Some of these initiatives are in collaboration
with the CIO and CFO Councils and are broad-based attempts to reform
financial management operations across the federal government. While
reforming federal financial management is an undertaking of tremendous
complexity, it presents great opportunities for improvements in
financial management system implementations and related business
operations.
Notably, OMB has developed and continues to evolve governmentwide
Federal Enterprise Architecture products and has required a mapping of
agency architectures to this federal architecture as part of the budget
review process. Another key OMB initiative is referred to as the lines
of business and promotes streamlining common systems to enhance the
government's performance and services, such as establishing centers of
excellence to consolidate financial management activities for major
agencies through cross-servicing arrangements. The advantages of this
approach are many, including the implementation of standard business
processes and focusing system acquisition, development, and maintenance
activities at select agencies or entities with experience that have the
necessary resources to reduce the risks associated with such efforts.
Furthermore, certain activities and responsibilities performed by JFMIP
prior to its termination have been reassigned to OMB's OFFM, the
Financial Systems Integration Office, and a CFO Council Committee
providing guidance and oversight. However, as discussed in the next
section, we identified four key concepts that are not yet fully
developed and integrated in OMB's initiatives and related processes.
Table 5 highlights some of the foremost initiatives under way at OMB
and their potential strengths.
Table 5: OMB Initiatives to Reform Federal Financial Management System
Implementations:
Initiative: Federal Enterprise Architecture to build a comprehensive
business-driven blueprint of the entire federal government;
Potential strengths:
* Business driven;
* Proactive and collaborative across the federal government;
* Architecture improves the effectiveness and efficiency of government
information resources.
Initiative: Lines of Business to develop business-driven common
solutions that span across the federal government, such as
consolidating duplicative financial management systems using centers of
excellence to provide services;
Potential strengths:
* Enhance process improvements;
* Achieve cost savings;
* Standardize business processes and data models;
* Promote seamless data exchange between federal agencies;
* Strengthen internal controls;
* Reduce development risks.
Initiative: JFMIP Realignment to realign responsibilities for
overseeing, developing, testing, and publishing core financial systems
requirements, including the development of standard business processes;
Potential strengths:
* Eliminate duplicative roles;
* Streamline financial management improvement efforts consistent with
statutory requirements;
* Define standard business processes and system requirements;
* Improve interoperability and data consistency.
Source: GAO analysis.
[End of table]
Federal Enterprise Architecture:
In 2002, OMB established the Federal Enterprise Architecture Program
Management Office to develop a Federal Enterprise Architecture
according to a collection of five reference models. These models are
intended to facilitate governmentwide improvement through cross-agency
analysis and the identification of duplicative investments, gaps, and
opportunities for collaboration, interoperability, and integration
within and across government agencies. According to OMB, the result
will be a more citizen-centered, customer-focused government that
maximizes technology investments to better achieve mission outcomes.
The Federal Enterprise Architecture reference models are summarized in
table 6.
Table 6: Federal Enterprise Architecture Reference Models:
Name: Business Reference Model;
Description: Describes the business operations of the federal
government independent of the agencies that perform them, including
defining the services provided to state and local governments.
Name: Service Component Reference Model;
Description: Identifies and classifies IT service (i.e., application)
components that support federal agencies and promotes the reuse of
components across agencies.
Name: Technical Reference Model;
Description: Describes how technology is supporting the delivery of
service components, including relevant standards for implementing the
technology.
Name: Performance Reference Model;
Description: Provides a common set of general performance outputs and
measures for agencies to use to achieve business goals and objectives.
Name: Data and Information Reference Model;
Description: Describes, at an aggregate level, the types of data and
information that support program and business line operations, and the
relationships among these types.
Source: GAO analysis.
[End of table]
In May 2005, the five reference models were combined into the
Consolidated Reference Model document to compose a framework for
describing important elements of the Federal Enterprise Architecture in
a common and consistent way. OMB views the Federal Enterprise
Architecture not as a static model, but as a program, built into the
annual budget process to repeatedly and consistently improve all
aspects of government service delivery. OMB officials acknowledged that
they are still mapping out the Federal Enterprise Architecture and
making it more robust and recognized that some lines of business have
fleshed out their areas in more detail than others. In prior testimony
on the Federal Enterprise Architecture,[Footnote 48] we recognized that
OMB and the CIO Council have made important progress, but that hard
work lies ahead to ensure that the Federal Enterprise Architecture is
appropriately described, matured, and used. The development of the
Federal Enterprise Architecture has continued to evolve and OMB has
been promoting the adoption of the Federal Enterprise Architecture. For
example, for the fiscal year 2007 budget submission, agencies will be
required[Footnote 49] to use predetermined codes to link their major IT
investments on Exhibit 53[Footnote 50] to the Federal Enterprise
Architecture. For fiscal year 2005, agencies were required to use the
Federal Enterprise Architecture Performance Reference Model to identify
performance measurements for each new major IT investment. As we have
previously testified,[Footnote 51] questions remain regarding the
nature of the Federal Enterprise Architecture, the relationship of
agency enterprise architectures to the Federal Enterprise Architecture,
and the security aspects of the Federal Enterprise Architecture.
Therefore, we will not be addressing these issues further from a
governmentwide perspective in this report.
Lines of Business:
Building upon the efforts of the Federal Enterprise Architecture
program, OMB and designated agency task forces have launched the lines
of business initiative. This initiative seeks to develop business-
driven common solutions for six lines of business[Footnote 52] that
span across the federal government. OMB and the lines of business task
forces plan to use enterprise architecture-based principles and best
practices to identify common solutions for business processes or
technology-based shared services to be made available to government
agencies. Driven from a business perspective rather than a technology
focus, the solutions are expected to address distinct business
improvements to enhance the government's performance and services for
citizens. The end results of the lines of business efforts are expected
to save taxpayer dollars, reduce administrative burden, and
significantly improve service delivery.
We have long supported and called for such initiatives to standardize
and streamline common systems, which can reduce costs and, if done
correctly, can also improve accountability. OMB officials from both
OFFM and the Electronic Government office told us that they worked
collaboratively to develop the financial management line of business
along with an interagency task force. The interagency task force
recommended the establishment of governmentwide service providers in
the areas of financial management and human resources management. The
financial management line of business raises a number of issues that
have far-reaching implications for the government and private sector
application service providers. This concept has commonly been used in
the private sector where application service providers provide services
such as payroll, sales force automation, and human resource
applications to many corporate clients. The interagency task force
analysis estimated that savings of more than $5 billion can be expected
over a 10-year time frame through consolidation of financial management
and human resources systems and the standardization and optimization of
associated business processes and functions. To help realize these
benefits, OMB evaluated agencies' business cases submitted as part of
the fiscal year 2006 budget process. On the basis of the review, the
following four agencies were designated as governmentwide financial
management application service providers, which OMB refers to as
centers of excellence.
* Department of the Interior (National Business Center);
* General Services Administration;
* Department of the Treasury (Bureau of the Public Debt's
Administrative Resource Center);
* Department of Transportation (Enterprise Services Center).
The National Business Center, the General Services Administration, and
the Bureau of the Public Debt have significant experience providing
financial management services to other federal entities. For a number
of years, these entities have provided financial management services--
primarily to smaller federal agencies such as the Nuclear Regulatory
Commission, the Office of Government Ethics, and the Panama Canal
Commission. The Department of Transportation plans to utilize its newly
implemented financial management system to provide services to other
agencies. OMB officials told us that, at a minimum, centers of
excellence must be able to support, or must use, core financial system
software that has passed the most recent qualification test of the
Financial Systems Integration Office, which is the current entity that
performs many of the roles and responsibilities of the former JFMIP
Program Management Office as we discuss below. Centers of excellence
may provide related maintenance, interfaces with feeder systems, and
transaction processing. Other services may also be offered, including
hosting[Footnote 53] and other financial applications such as payroll
and travel. OMB also indicated that it plans to explore using private
sector application service providers to serve as centers of excellence.
OMB expects to manage the migrations of agencies to centers of
excellence using the agencies' business cases submitted as part of the
annual budget process. According to OMB, agencies that submit business
cases with proposals to develop new financial systems or significantly
update or enhance current financial systems are prime candidates for
moving to a financial management center of excellence. The general
principle OMB plans to follow is that agencies should migrate to a
financial management center of excellence when it is cost effective to
do so and they have maximized the return on investment in the current
system, which averages about 5 to 7 years. OMB officials told us that
several major executive branch agencies are considering moving to a
financial management center of excellence.
In August 2005, OPM was the first large agency to announce its plans to
move to a designated center of excellence. At the time of our review,
OPM was still in the planning phase; although it had selected the
Bureau of the Public Debt as the provider, it did not yet have a
project plan. OPM officials recognized that moving to a center of
excellence at the beginning of a fiscal year and not converting mid-
year was a best practice they planned to follow. In addition, at the
time of our review, the Environmental Protection Agency was in the
planning and acquisition phase of its Financial System Modernization
Project. As part of its best-value determination, the Environmental
Protection Agency was considering the designated centers of excellence
as well as private sector providers for software, integration, and
hosting and had issued a draft request for quotations. Also, OMB
officials stated that they helped the National Gallery of Art in
preparing its solicitation for a new system, and the agency recently
selected a private sector firm as its application service provider. OMB
expects that most agencies will move to a center of excellence or
private sector firm within the next 7 to 8 years. In OMB Circular No. A-
11, for fiscal year 2007 OMB has asked agencies to provide an overview
of their current and future financial management systems framework,
including migration strategies for moving to a financial management
center of excellence.
JFMIP Realignment:
In an effort to eliminate duplicative roles and streamline financial
management improvement efforts, the four principals of JFMIP agreed to
realign JFMIP's responsibilities for financial management policy and
oversight as described in a December 2004 OMB memorandum.[Footnote 54]
Some of the former responsibilities of JFMIP, such as issuing systems
requirements, were to be placed under the authority of OFFM and a
renamed CFO Council committee--the Financial Systems Integration
Committee. As a result of the realignment, JFMIP ceased to exist as a
separate organization, although the principals will continue to meet at
their discretion consistent with the Budget and Accounting Procedures
Act of 1950 (codified, in part, at 31 U.S.C. §3511(d)).
Under the realignment announced in December 2004, the JFMIP Program
Management Office was to report to the chair of the CFO Council's
Financial Systems Integration Committee. This reporting relationship
subsequently changed. At the request of the OMB Controller, the CFO at
the Department of Labor now chairs the Financial Systems Integration
Committee and is the leading agency sponsor of the financial management
line of business. Two subcommittees were also established under the
announced realignment:
* Configuration Control Subcommittee--to focus on interface
requirements, and,
* Transaction Processing Standardization Subcommittee--to support
interagency development of functional requirements for the software
certification process.
OMB officials indicated that the roles and responsibilities of the two
subcommittees under the Financial Systems Integration Committee will
likely continue to evolve. However, the full committee will
periodically evaluate the subcommittees and whether they are well
aligned and still needed or if additional subcommittees are needed.
Other significant responsibilities of the former JFMIP Program
Management Office, which was previously managed by the JFMIP executive
director using funds provided by the CFO Council, were shifted to the
Financial Systems Integration Office (FSIO), which was established with
staff from the original JFMIP Program Management Office. The FSIO will
now report to the FSIO executive director, who will report to the OMB
Controller. Before the realignment, the JFMIP Program Management Office
was responsible for the testing and certification of commercial off-
the-shelf (COTS) core financial systems for use by federal agencies and
coordinating the development and publication of functional requirements
for financial management systems, among other things. OMB officials
expect that the FSIO will continue to focus on core financial systems
and still be responsible for certification and testing of core systems,
but they plan to evaluate the effectiveness of the certification and
testing function. In addition, OMB has recognized the need for
standardization and the inclusion of key stakeholders in developing
systems requirements and processes, but considers it a long-term goal.
The FSIO will develop systems requirements and the Financial Systems
Integration Committee will be responsible for advising OFFM on the
systems requirements. OFFM will now be responsible for issuing new
systems requirements.[Footnote 55]
According to OMB officials, the FSIO is reassessing the realignment
plan described in the December 2004 OMB memorandum and recently
developed foundational materials including the mission statement,
goals, objectives, performance indicators, scope of activities,
prioritization of work, budget, organizational chart, and communication
plan. According to OMB officials, resources at FSIO will be aligned
under the priorities identified and the office will be structured
according to the new priorities. The FSIO will identify its needs for
additional staff and determine how many are needed and what skill sets
are appropriate. The FSIO will continue defining its priorities and
evaluating the effectiveness of processes and its plans will continue
to evolve.[Footnote 56] While OMB has taken steps to accomplish the
Federal Enterprise Architecture, lines of business, and JFMIP
realignment initiatives, as discussed in the next section, it is
generally at the early stages of implementation and a firm foundation
has not yet been established to address the long-standing problems that
have impeded success.
Broad-Based Actions Needed to Implement Financial Management Systems
Governmentwide:
The key for federal agencies to avoid the long-standing problems that
have plagued financial management system improvement efforts is to
address the foremost causes of those problems and adopt solutions that
reduce the risks associated with these efforts to acceptable levels.
Although OMB has articulated an approach for reforming financial
management systems governmentwide under its financial management line
of business and JFMIP realignment initiatives, implementing these
initiatives will be complex and challenging. OMB has correctly
recognized that enhancing the government's ability to implement
financial management systems that are capable of providing accurate,
reliable, and timely information on the results of operations needs to
be addressed as a governmentwide solution, rather than as individual
agency stove-piped efforts designed to meet a given entity's needs.
However, OMB has not yet fully defined and implemented the processes
needed to successfully complete these initiatives. Specifically, based
on industry best practices, we identified four key concepts that are
not yet fully developed and integrated in OMB's initiatives and related
processes. While OMB has addressed certain elements of these best
practices in its initiatives, many specific steps are not yet
completed. Careful consideration of these four concepts, each one
building upon the next, will be integral to the success of OMB's
initiatives and will help break the cycle of failure in implementing
financial management systems. The four concepts are (1) developing a
concept of operations, (2) defining standard business processes, (3)
developing a strategy for ensuring that agencies are migrated to a
limited number of application service providers in accordance with
OMB's stated approach, and (4) defining and effectively implementing
disciplined processes necessary to properly manage the specific
projects. The following sections highlight the key issues to be
considered for each of the four areas.
Concept of Operations Provides Foundation:
Key Issues:
* What is considered a financial management system?
* Who will be responsible for developing a governmentwide concept of
operations and what process will be used to ensure that the resulting
document reflects the governmentwide solution rather than individual
agency stove-piped efforts?
* How will the concept of operations be linked to the Federal
Enterprise Architecture?
* How can the federal government obtain reliable information on the
costs of its financial management systems investments?
[End of table]
A concept of operations defines how an organization's day-to-day
operations are (or will be) carried out to meet mission needs. The
concept of operations includes high-level descriptions of information
systems, their interrelationships, and information flows. It also
describes the operations that must be performed, who must perform them,
and where and how the operations will be carried out. Further, it
provides the foundation on which requirements definitions and the rest
of the systems planning process are built. Normally, a concept of
operations document is one of the first documents to be produced during
a disciplined development effort and flows from both the vision
statement and the enterprise architecture. According to the IEEE
standards,[Footnote 57] a concept of operations is a user-oriented
document that describes the characteristics of a proposed system from
the users' viewpoint. The key elements that should be included in a
concept of operations are major system components, interfaces to
external systems, and performance characteristics such as speed and
volume.
In the case of federal financial management systems, another key
element for the concept of operations would be a clear definition and
scope of the financial management activities to be included. One
problem with the current OMB approach for reporting is that systems
that have historically been considered part of financial management,
such as payroll and inventory management, are not captured under the
financial management line of business when a particular agency reports
IT investments to OMB as part of the annual budget submission for
inclusion in the Budget of the United States Government. This is
because the Federal Enterprise Architecture coding structure for
agencies to use when transmitting IT investment information to OMB
calls for only IT investments that support certain financial system
functions to be identified as a financial management system. An
effective concept of operations would help identify these omissions.
Financial management systems are defined by OMB in Circulars No. A-11
and A-127 in similar terms to that found in statutes such as FFMIA.
This definition is also similar to that used by DOD to define a defense
business system as provided by the fiscal year 2005 Defense
Authorization Act.[Footnote 58] These various sources generally
consider financial management systems to be financial systems and the
financial portion of mixed systems that support the interrelationships
and interdependencies between budget, cost, and management functions,
and the information associated with business activities. A mixed system
is an information system that supports both financial and nonfinancial
functions of the federal government. At DOD, for example, an estimated
80 percent of the information needed to prepare annual financial
statements comes from mixed systems such as logistics, personnel, and
procurement systems that are outside of the responsibility of the DOD
CFO. In contrast, the Federal Enterprise Architecture's Business
Reference Model defines a financial management system as one that uses
financial information to measure, operate, and predict the
effectiveness and efficiency of an entity's activities in relation to
its objectives. These differences illustrate that a consistent
definition of financial management systems is not being used across the
federal government.
One of the key challenges faced by OMB when evaluating financial
management system implementation efforts is capturing all financial
management system investments and their related costs. The fiscal year
2006 budget requests for IT spending totaled about $65.2 billion. Our
analysis showed that, of this amount, only $3.9 billion, less than 6
percent, is reflected under the financial management mission as defined
by OMB using the definition of a financial management system in its
Federal Enterprise Architecture. A more comprehensive analysis of
financial management system investments using the definition in OMB
Circular No. A-127 that includes mixed systems such as payroll and
inventory and including those considered by DOD as business systems
brings the total to about $20 billion. Payroll and inventory management
systems clearly support financial management activities, but these
systems are not included in the financial management line of business
within the Federal Enterprise Architecture framework. The payroll and
inventory systems are reflected under the human resource management and
supply chain management lines of business, respectively.
Because of these differing definitions, the total number of systems and
the respective costs associated with financial management system
implementation efforts are difficult to capture. OMB officials stated
that they are currently revising OMB Circular No. A-127 and will
consider clarifying the definition to ensure that it is consistent with
FFMIA. In addition, an effective concept of operations would help
bridge this gap and facilitate the monitoring of the activity related
to financial management systems. Addressing this issue would be a key
factor in developing a foundation for the lines of business initiative
to consolidate federal financial management systems under a limited
number of application service providers.
An effective concept of operations would describe, at a high level (1)
how all of the various elements of federal financial systems and mixed
systems relate to each other, and (2) how information flows from and
through these systems. Further, a concept of operations would provide a
useful tool to explain how financial management systems at the agency
and governmentwide levels can operate cohesively. It would be geared to
a governmentwide solution rather than individual agency stove-piped
efforts. Further, it would provide a road map that can be used to (1)
measure progress and (2) focus future efforts. OMB officials told us
that they had developed a concept of operations, but did not know when
it would be released or if it meets the criteria in the IEEE standards.
Because the federal government has lacked such a document, a clear
understanding of the interrelationships among federal financial systems
and how the application service provider concept fits into this
framework has not yet been achieved.
While the Federal Enterprise Architecture, when fully populated, could
provide some of this perspective, a concept of operations document
presents these items from a user's viewpoint in nontechnical terms.
Such a document would be invaluable in getting various stakeholders,
including those at the agency and governmentwide levels, the software
vendors, and the three branches of the federal government, to
understand how the financial systems are expected to operate cohesively
and how they fit into "the big picture." A concept of operations from
this perspective would clarify which financial management systems
should be operated at an agency level and which ones would be handled
at a governmentwide level and how those two would integrate. In
addition, it could identify the nature and extent of skills needed to
effectively operate these systems. This would play a part in resolving
some of the human capital management problems discussed previously.
Another key element of a concept of operations is a transition strategy
that is useful for developing an understanding of how and when changes
will occur. Not only is this needed from an investment management point
of view, it is a key element in the human capital problems discussed
previously that revolved around change management strategies.
Describing how to implement OMB's approach for outsourcing financial
management systems and the process that will be used to deactivate
legacy systems that will be replaced or interfaced with a new financial
management system are key aspects that need to be addressed in a
transition strategy. This, in turn, allows the agencies to begin taking
the necessary actions to integrate this approach into their investment
management and change management processes.
Standard Business Processes Promote Consistency:
Key Issues:
* How can governmentwide standard business processes be developed to
meet the needs of federal agencies?
* How can agencies be encouraged to adopt new processes, rather than
selecting other methods that result in simply automating old ways of
doing business?
* How will the standard business processes be implemented by the
application service providers to provide consistency across government
agencies and among the application service providers?
* What process will be used to determine and validate the processes
needed for agencies that have unique needs?
[End of table]
Business process models provide a way of expressing the procedures,
activities, and behaviors needed to accomplish an organization's
mission and are helpful tools to document and understand complex
systems. Business processes are the various steps that must be followed
to perform a certain activity. For example, the procurement process
would start when the agency defines its needs, issues a solicitation
for goods or services and would continue through contract award,
receipt of goods and services, and would end when the vendor properly
receives payment. The identification of preferred business processes
would be critical for standardization of applications and training and
portability of staff, as well as for the software vendor community to
use for software design and implementation purposes. Without standard
processes, the federal government will continue to spend funds to
develop individual agency stove-piped efforts that may or may not meet
a given entity's needs.
To maximize the success of a new system acquisition, organizations need
to consider the redesign of current business processes. As we noted in
our Executive Guide: Creating Value Through World-class Financial
Management,[Footnote 59] leading finance organizations have found that
productivity gains typically result from more efficient processes, not
from simply automating old processes. Moreover, the Clinger-Cohen Act
of 1996 requires agencies to analyze the missions of the agency and,
based on the analysis, revise mission-related and administrative
processes, as appropriate, before making significant investments in
information technology used to support those missions.[Footnote 60]
Another benefit of what is often called business process modeling is
that it generates better system requirements, since the business
process models drive the creation of information systems that fit in
the organization and will be used by end users. Other benefits include
(1) providing a foundation for agency efforts to describe the business
processes needed for unique missions, or to develop subprocesses to
support those at the governmentwide level and (2) describing the
business processes of the federal government to the vendor community
for standardization. While in many cases, government business processes
will be identical or very similar to processes used by the private
sector, these standards should also describe processes unique to
federal accounting.
However, according to OMB officials, the lines of business initiative
is moving forward even though this important key issue has not yet been
addressed. OMB officials believed that for standardized processes, it
is important to get buy-in as the processes are developed, and not
force the process from the top. OMB officials we talked with recognized
that standardization of business processes is important, but they did
not want to wait to deploy the financial management line of business
initiative until standard business processes had been developed. OMB
planned to task the newly created CFO Council Transaction Processing
Standardization Subcommittee with the responsibility for developing
standard federal business processes.[Footnote 61] Because this key
issue has not been addressed, and the other key issues flow from it,
little has been done to address those important considerations. From
our perspective, adopting standardized processes is a fundamental step
needed for all financial system implementations, but especially for
making the financial management line of business initiative successful.
Otherwise, we believe that there is a much greater risk of the
continued proliferation of nonstandard business processes that would
not result in a marked improvement from the current environment.
Strategy for Implementing the Financial Management Line of Business
Initiative Will Be Key:
Key Issues:
* What guidance will be provided to assist agencies in adopting a
change management strategy that reduces the risks of moving to the
application service provider approach?
* What processes will be put in place to ensure that agency financial
management system investment decisions focus on the benefits of
standard processes and application service providers?
* What process will be used to facilitate the decision-making process
used by agencies to select a given provider?
* How will agencies incorporate strategic workforce planning in the
implementation of the application service provider approach?
[End of table]
Although OMB has a goal of migrating agencies to a limited number of
application service providers within the next 7 to 8 years to deliver
the standard business processes, rather than funding individual agency
efforts, it has not yet articulated a clear and measurable strategy for
achieving this goal. This is important because there has been a
historical tendency for agencies and units within agencies to view
their needs as urgent and resist standardization. Decisive action will
be needed to ensure that agencies adopt the application service
provider concept and that agencies do not continue to attempt to
develop and implement their own financial management systems. OMB has
been proactive since the beginning of the financial management line of
business initiative in describing the goals of the initiative by making
speeches, discussing the initiative with the media, including it in the
President's budget request, and highlighting it on its Web site.
However, there are limited tools and guidance available and OMB has not
provided centers of excellence with standard document templates needed
to minimize risk, provide assurance, and develop understandings with
customers on topics such as service level agreements and concept of
operations. A service level agreement is critical for both the
application service providers and the agencies to be held accountable
for their respective parts of the agreement. Much work remains to
develop a change management strategy that addresses key activities
needed to minimize the risk associated with the implementation of the
financial management line of business initiative.
Change management in the context of migrating federal agencies to an
application service provider will need to include activities such as
(1) developing specific criteria for requiring agencies to migrate to
an application service provider rather than attempting to develop and
implement their own stove-piped business systems; (2) providing the
necessary information for an agency to make a selection of an
application service provider; (3) defining and instilling new values,
norms, and behaviors within agencies that support new ways of doing
work and overcoming resistance to change; (4) building consensus among
customers and stakeholders on specific changes designed to better meet
their needs; and (5) planning, testing, and implementing all aspects of
the transition from one organizational structure and business process
to another.
According to leading IT organizations, organizational change management
is the process of preparing users for the business process changes that
will accompany implementation of a new system. An effective
organizational change management process includes project plans and
training that prepare users for impacts the new system might have on
their roles and responsibilities and a process to manage those changes.
We have reported on various problems with agencies' change management
including the failure to develop transition plans, reengineer business
processes, and limit customization.[Footnote 62] In addition, one CFO
Council member told us that from his perspective systems do not fail,
but there is an implementation failure because of (1) ineffective
coordination and communication between the CFO and CIO offices, (2)
excessive modification of COTS systems, (3) business processes not
being reengineered correctly, completely, or timely, and (4) a lack of
authority and leadership for the CFO and project management offices to
make the implementation work.
With regard to establishing criteria for transitioning agencies to an
application service provider, we note that providing governmentwide
financial services is not a new concept to the federal government. One
of the 24 Presidential Electronic Government initiatives is e-payroll,
which was intended to consolidate 22 federal payroll systems into 4
federal payroll providers[Footnote 63] to simplify and standardize
federal human resources/payroll policies and procedures to better
integrate payroll, human resources, and finance functions. Numerous
agencies had targeted their payroll operations for costly
modernizations, and according to OMB, by consolidating duplicative
payroll modernization efforts, an estimated $1.1 billion can be saved
over the next decade in future IT investments given the economies of
scale and cost avoidance. Federal agencies already have or will be
migrating to one of the four selected payroll providers to process
payroll and pay employees.
OMB officials told us they learned from the e-payroll initiative that
directing and forcing change as they had done with the e-payroll effort
was not palatable to federal agencies. The agencies preferred having
choices on timing the move and on having options for various providers.
As a result, for the financial management line of business initiative,
they do not plan to establish a migration path or time table. Further,
processes have not been put in place to facilitate agency decisions on
selecting a provider or focusing investment decisions on the benefits
of standard processes and application service providers. It is not
clear how this will impact the adoption of this initiative. Given the
pressures to reduce budgets, discipline with respect to following a
clear migration path will be essential. Without such a migration path,
while some agencies may readily migrate to a center of excellence or
application service provider to minimize the tremendous undertaking of
implementing or significantly upgrading a financial system, other
agencies will likely perpetuate the waste of taxpayer dollars
previously described related to failed system implementation efforts.
The need for clear criteria on migrating agencies to the financial
management line of business initiative is highlighted by the following
example.
In fiscal year 2004, the Department of Justice embarked on implementing
a new core financial system and is not planning to move to a center of
excellence. OMB officials stated that they were not requiring Justice
to move to a center of excellence because it had unique needs and was
already far enough along in its attempt to modernize and consolidate
the financial systems used throughout the agency. OMB officials also
speculated that Justice might eventually become a center of excellence
that focuses on law enforcement agencies and addresses the law
enforcement community's unique needs.[Footnote 64] According to a
supporting document of the Analytical Perspectives, Budget of the
United States Government, Fiscal Year 2006, Justice spent about $6.9
million on modernizing its core financial system in fiscal year 2004.
Further, Justice planned to spend $23.1 million for modernization
during fiscal year 2005, and expects fiscal year 2006 modernization
costs to more than triple to $72.5 million. In October 2004, the IG
reported that little progress had been made in implementing the new
system and continued to report financial management and systems as a
top management challenge.[Footnote 65] Thus, it is not clear why
Justice should continue with its financial systems development project
when the cost is expected to significantly escalate and significant
challenges remain.
Further, the application service provider concept will still require
that agencies address long-standing human capital problems by
incorporating elements of strategic workforce planning such as (1)
aligning an organization's human capital program with its current and
emerging mission and programmatic goals and (2) developing long-term
strategies for acquiring, developing, and retaining an organization's
total workforce to meet the needs of the future. This includes a range
of activities from identifying and defining roles and responsibilities,
to identifying team members, to developing individual competencies that
enhance performance. To maintain and enhance the capabilities of IT
staff, organizations should develop and implement a human capital
strategy that, among other things, includes assessing competencies and
skills needed to effectively perform IT operations to support agency
mission and goals, inventorying the competencies and skills of current
IT staff to identify gaps in needed capabilities, and developing and
implementing plans to fill the gap between requirements and current
staffing.
As we have testified,[Footnote 66] having sufficient numbers of people
on board with the right mix of knowledge and skills can make the
difference between success and failure. This is especially true in the
IT area, where widespread shortfalls in human capital have contributed
to demonstrable shortfalls in agency and program performance. According
to Building the Work Force Capacity to Successfully Implement Financial
Systems,[Footnote 67] the roles needed on an implementation team are
consistent across financial system implementation projects and include
a project manager, systems integrator, functional experts, information
technology manager, and IT analysts. Many of these roles require the
dedication of full-time staff for one or more of the project's phases.
Finally, sustained leadership will be key to a successful strategy for
moving federal agencies towards consolidated financial management
systems. In our Executive Guide: Creating Value Through World-class
Financial Management,[Footnote 68] we found that leading organizations
made financial management improvement an entitywide priority by, among
other things, providing clear, strong executive leadership. We also
reported that making financial management a priority throughout the
federal government involves changing the organizational culture of
federal agencies. Although the views about how an organization can
change its culture can vary considerably, leadership (executive
support) is often viewed as the most important factor in successfully
making cultural changes. Top management must be totally committed in
both words and actions to changing the culture, and this commitment
must be sustained and demonstrated to staff. In addition, a recent best
practice guide on shared services[Footnote 69] stated that it is not
enough for management to merely support the financial operations'
shared service implementation--top management must provide the
leadership structure to ensure that the transition is successful.
Because the tenure of political appointees is relatively short, the
current and future administrations must continue a strong emphasis on
top-notch financial management.
Disciplined Processes Will Help Ensure Successful Implementations:
Key Issues:
* How can existing industry standards and best practices be
incorporated into governmentwide guidance related to financial
management system implementation efforts, including migrating to an
application service provider?
* What actions will be taken to reduce the risks and costs associated
with data conversion and interface efforts?
* What oversight process will be used to ensure that modernization
efforts effectively implement the prescribed policies and procedures?
[End of table]
Once the concept of operations and standard business processes have
been defined and a migration strategy is in place, individual agencies
will have to work closely with the selected application service
provider or systems integrator to help ensure that the implementation
is successful. Although application service providers may provide a
COTS solution, effective implementation and testing processes are still
required to ensure that the system delivers the desired functionality
on time and within budget. As previously discussed, a partnership
between the CIO and CFO offices, as well as with those program
management offices responsible for financial or mixed systems such as
payroll and inventory, is critical for success. Agencies have
frequently struggled to implement key best practices when implementing
COTS financial management systems. The key to avoiding these long-
standing implementation problems is to provide specific guidance to
agencies for financial management system implementations, incorporating
the best practices identified by the SEI, the IEEE, the Project
Management Institute, and other experts that have been proven to reduce
risk in implementing systems. Such guidance should include the various
disciplined processes such as requirements management, testing, data
conversion and system interfaces, risk and project management, and
related activities, which have been problematic in the financial
systems implementation projects we reviewed.
Disciplined processes have been shown to reduce the risks associated
with software development and acquisition efforts to acceptable levels
and are fundamental to successful system implementations. The
principles of disciplined IT systems development and acquisition of
services apply to shared services implementation. A disciplined
software implementation process can maximize the likelihood of
achieving the intended results (performance) within established
resources (costs) on schedule. For example, disciplined processes
should be in place to address the areas of data conversion and
interfaces, two of the many critical elements necessary to successfully
implement a new system that have contributed to the failure of previous
agency efforts. The former JFMIP provided guidance on data conversion,
and the Configuration Control Subcommittee under the CFO Council's
Financial Systems Integration Committee was tasked with focusing on
interface requirements.[Footnote 70] However, a standard set of
practices will be needed to guide the migration from legacy systems to
new systems and application service providers. Further details on
disciplined processes needed can be found in appendix III.
In addition, oversight to help ensure that the disciplined processes
are in place and operating as intended will be a critical factor in the
success of the implementation of new and consolidated financial
management systems. Currently, OMB guidance[Footnote 71] requires
agencies to have qualified project managers and to use earned value
management tools for major IT investments. However, OMB only performs
limited reviews of agencies' financial management systems
implementations. OFFM officials told us that these reviews vary
considerably in scope and that one of their goals is to provide more
structure to the reviews. OMB's review depends on the agency and the
phase of the project, and generally does not focus on implementation of
the disciplined processes used. Industry experts agree that the best
indicator of whether risks have been reduced to an acceptable level is
an assessment of the disciplined processes in place. For example, in
the area of requirements management, disciplined processes would help
ensure (1) the requirements document contains all the requirements
identified by the customer, as well as those needed for the definition
of the system, (2) the requirements fully describe the software
functionality to be delivered, (3) the requirements are stated in clear
terms that allow for quantitative evaluation, and (4) traceability
among various documents is maintained. Proper oversight would entail
verification of these requirements-related disciplined processes.
In addition to problems with the structure and scope of OMB's current
system reviews, we noted that OFFM has a staff of only four employees
dedicated to reviewing federal executive branch agency projects to
implement financial management systems. These four staff also have
other time-consuming duties such as developing a coherent, coordinated
architecture and issuing federal financial system requirements. As a
result, the current level of detail in the existing system reviews is
necessarily limited. Moreover, there is limited follow-up by OMB on
suggested improvements they have made to agency officials, and there is
not any impetus for agencies to implement suggested improvements. For
example, OFFM officials told us that they advised an agency that there
were numerous disadvantages to deploying a new financial management
system mid-year. Nonetheless, the agency deployed the system at mid-
year and has faced problems by doing so. The FSIO also has a limited
number of staff to perform its numerous financial management policy and
oversight activities and is currently reassessing its priorities and
available resources. Given the range of OMB's leadership roles and its
relatively small size as part of the Executive Office of the President,
it is not realistic to expect OMB to be able to carry out a
comprehensive review function. Instead, agencies could be required to
have their financial management system projects undergo independent
verification and validation reviews to ensure that the projects
adequately implemented the disciplined processes needed to manage the
risks to acceptable levels. OMB could then review reports produced as a
result of the independent verification and validation process to
leverage its oversight efforts. Accordingly, OMB could then focus its
oversight efforts on the projects with the greatest risks.
Conclusions:
Because the federal government is one of the largest and most complex
organizations in the world, operating, maintaining, and modernizing its
financial management systems represent a monumental challenge--
technically and cost-wise. The past paradigm must be changed from one
in which each federal agency attempts to implement systems that, in
many cases, are to perform redundant functions and have all too often
resulted in failure, have been delayed, and cost too much. Thus, a more
holistic governmentwide approach as OMB has been advocating is
necessary to address the key causes of failure. OMB has recognized the
seriousness of the problems. Its primary initiative related to the use
of a limited number of application service providers is a step in the
right direction. This initiative is in the early stage and does not yet
include basic elements that are integral to its success. Based on
industry best practices, the following four concepts would help ensure
a sound foundation for developing and implementing a governmentwide
solution for long-standing financial management system implementation
failures: (1) developing a concept of operations that ties in other
systems, (2) defining standard business processes, (3) developing a
strategy for ensuring that agencies are migrated to a limited number of
application service providers, and (4) defining and effectively
implementing applicable disciplined processes. As pressure mounts to do
more with less, to increase accountability, and to reduce fraud, waste,
abuse, and mismanagement, and efforts to reduce federal spending
intensify, sustained and committed leadership will be a key factor in
the successful implementation of these governmentwide initiatives.
However, regardless of the approach taken, the adherence to disciplined
processes in systems development and acquisition will be at the core of
successfully addressing the key causes of financial management system
implementation failures.
Recommendations for Executive Action:
To help reduce the risks associated with financial management system
implementation efforts and facilitate the implementation of the
financial management line of business and JFMIP realignment initiatives
across the government, we recommend that the Director of OMB take the
following 18 actions. This would entail placing a high priority on
fully integrating into its approach the following concepts and
underlying key issues, all of which are related to the fundamental
disciplines in systems implementation:
Developing a concept of operations. This would include:
* identifying the interrelationships among federal financial systems
and how the application service provider concept fits into this
framework,
* prescribing which financial management systems should be operated at
an agency level and which should be operated at a governmentwide level
and how those would integrate, and:
* defining financial management systems in the Federal Enterprise
Architecture to be more consistent with the similar definitions used in
FFMIA and OMB Circulars No. A-11 and No. A-127.
Defining standard business processes. This would include:
* describing the standard business processes that are needed to meet
federal agencies' needs,
* developing a process to identify those that are needed to meet unique
agency needs,
* requiring application service providers to adopt standard business
processes to provide consistency, and:
* encouraging agencies to embrace new processes.
Developing a strategy for ensuring that agencies are migrated to a
limited number of application service providers in accordance with
OMB's stated approach. This would include:
* articulating a clear goal and criteria for ensuring agencies are
subject to the application service provider concept and cannot continue
developing and implementing their own stove-piped systems,
* establishing a migration path or time table for when agencies should
migrate to an application service provider,
* providing the necessary information for an agency to select an
application service provider, and:
* developing guidance to assist agencies in adopting a change
management strategy for moving to application service providers.
Defining and effectively implementing disciplined processes necessary
to properly manage the specific projects. This would include:
* providing specific guidance to agencies on disciplined processes for
financial system implementations,
* providing a standard set of practices to guide the migrations from
legacy systems to new systems and application service providers, and:
* developing processes to facilitate oversight and review that allow
for a more structured review and follow-up of agencies' financial
system implementation projects.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from the
Controller of OMB, which are reprinted in appendix IV. The Controller
agreed with our recommendations and described the approach and steps
that OMB is taking to improve financial management system modernization
efforts. As OMB moves forward to address the recommendations in our
report, it is important that it prioritize its efforts and focus on the
concepts and underlying key issues we discussed, such as adequately
defining and implementing disciplined processes. We are encouraged that
OMB plans to issue additional guidance outlining the fundamental risk-
reduction approaches that agencies can implement when acquiring and
implementing financial systems. It will be critical that the guidance
stresses the importance of this standard set of practices. We continue
to believe that careful consideration of all the building blocks and
key issues we identified will be integral to the success of OMB's
initiatives. OMB also provided additional oral comments which we
incorporated as appropriate.
We are sending copies of this report to the Chairman and Ranking
Minority Member, Senate Committee on Homeland Security and Governmental
Affairs, and other interested congressional committees. We are also
sending a copy to the Director of OMB. Copies will also be made
available to others upon request. The report will also be available at
no charge on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please
contact McCoy Williams, Director, Financial Management and Assurance,
who may be reached at (202) 512-9095 or by e-mail at
williamsm1@gao.gov, or Keith A. Rhodes, Chief Technologist, Applied
Research and Methods, who may be reached at (202) 512-6412 or by e-mail
at rhodesk@gao.gov. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made major contributions to this report are
listed in appendix V.
Signed by:
McCoy Williams:
Director:
Financial Management and Assurance:
Signed by:
Keith A. Rhodes:
Chief Technologist:
Applied Research and Methods:
Center for Engineering and Technology:
[End of section]
Appendix I: Scope and Methodology:
To determine the key causes for financial management system
implementation failures, we conducted database searches of GAO and
inspector general (IG) Web sites to identify reports issued by any GAO
teams or IGs that could be relevant. We summarized and analyzed prior
GAO reports on commercial off-the-shelf financial management system
implementations within the last 5 years. We performed a content
analysis of the GAO and IG reports to determine if causes for the
financial management system implementation problems were included. We
discussed the relevant GAO report findings and current status with the
key staff that worked on the reports. In addition, we identified other
potential data sources, such as key industry groups and well-known
national experts for information they had on this topic. We also
interviewed key Office of Management and Budget (OMB) officials and had
discussions with other interested parties such as Chief Financial
Officers (CFO) Council representatives.
To identify the significant governmentwide initiatives that are
currently under way that impact financial management systems
implementation failures, we interviewed key OMB officials and reviewed
relevant OMB policies, guidance, and memorandums related to the
initiatives. We also interviewed CFO Council representatives to discuss
the initiatives to reform federal financial management systems. In
addition, we interviewed Office of Personnel Management officials to
discuss their plans to migrate to a financial management center of
excellence. We also reviewed reports from various authors and
governmentwide forums where participants provided their perspectives on
governmentwide initiatives.
To provide our views on actions that can be taken to help improve the
management and control of agency financial management system
modernization efforts, we analyzed the GAO and IG reports we had
identified as relevant to the topic to highlight the actions called for
in those reports. Further, we reviewed material from key industry
groups and national experts to identify any potential solutions posed
by those groups, lessons learned, and relevant best practices. We took
into consideration those governmentwide initiatives that were currently
under way and the perspectives provided by authors and participants in
governmentwide forums. In addition, during our consultations with
various GAO stakeholders, and external groups such as OMB and the CFO
Council, we obtained their perspectives on the actions needed to
address the problems.
We conducted our work in Washington, D.C., from January 2005 through
October 2005, in accordance with U.S. generally accepted government
auditing standards. We did not evaluate the federal government's
overall IT strategy or whether a particular agency selected the most
appropriate financial management system. Because we have previously
provided agencies with specific recommendations in individual reports,
we are not making additional recommendations to them in this report. We
requested comments on a draft of this report from the Director of OMB
or his designee. Written comments from OMB are reprinted in appendix IV
and evaluated in the Agency Comments and Our Evaluation section.
[End of section]
Appendix II: IG Reports Reviewed:
Department of the Treasury Office of Inspector General. The
Modernization Program Is Establishing a Requirements Management Office
to Address Development and Management Problems. Reference No. 2005-20-
023. Washington, D.C.: January 19, 2005.
Department of Transportation Office of Inspector General. Consolidated
Financial Statements for Fiscal Years 2004 and 2003. Report FI-2005-
009. Washington, D.C.: November 15, 2004.
Department of Housing and Urban Development Office of Inspector
General. Fiscal Year 2004 Review of Information Systems Controls in
Support of the Financial Statements Audit. Report 2005-DP-0001.
Washington, D.C.: October 19, 2004.
Department of Justice Office of Inspector General. The Drug Enforcement
Administration's Management of Enterprise Architecture and Information
Technology Investments. Report 04-36. Washington, D.C.: September 2004.
Department of Veterans Affairs Office of Inspector General. Issues at
VA Medical Center Bay Pines, Florida and Procurement and Deployment of
the Core Financial and Logistics System. Report 04-01371-177.
Washington, D.C.: August 11, 2004.
Department of Energy Office of Inspector General. Management of the
Federal Energy Regulatory Commission's Information Technology Program.
Report DOE/IG-0652. Washington, D.C.: June 2004.
Department of Justice Office of Inspector General. The Federal Bureau
of Investigation's Implementation of Information Technology
Recommendations. Report 03-36. Washington, D.C.: September 2003.
Small Business Administration Office of Inspector General. Audit of
SBA's Acquisition, Development and Implementation of the Joint
Accounting and Administrative Management System. Report 3-32.
Washington, D.C.: June 30, 2003.
Department of Energy Office of Inspector General. Audit Report on
Business Management Information System. Report DOE/IG-0572. Washington,
D.C.: November 2002.
Department of the Interior Office of Inspector General. Developing the
Department of the Interior's Information Technology Capital Investment
Process: A Framework for Action. Report 2002-I-0038. Washington, D.C.:
August 2002.
Department of Defense Office of Inspector General. Development of the
Defense Finance and Accounting Service Corporate Database and other
Financial Management Systems. Report D-2002-014. Washington, D.C.:
November 7, 2001.
Department of Transportation Office of Inspector General. Implementing
a New Financial Management System. Report FI-2001-074. Washington,
D.C.: August 7, 2001.
[End of section]
Appendix III: Disciplined Processes:
Disciplined Processes Are Key to Successful Financial Management System
Implementation Efforts:
Disciplined processes have been shown to reduce the risks associated
with software development and acquisition efforts to acceptable levels
and are fundamental to successful system implementations. A disciplined
software implementation process can maximize the likelihood of
achieving the intended results (performance) within established
resources (costs) on schedule. Although a standard set of practices
that will guarantee success does not exist, several organizations, such
as the Software Engineering Institute (SEI) and the Institute of
Electrical and Electronic Engineers (IEEE), and individual experts,
have identified and developed the types of policies, procedures, and
practices that have been demonstrated to reduce development time and
enhance effectiveness. The key to having a disciplined system
development effort is to have disciplined processes in multiple areas,
including requirements management, testing, data conversion and system
interfaces, configuration management, risk management, project
management, and quality assurance.
Requirements Management:
Requirements are the specifications that system developers and program
managers use to design, develop, and acquire a system. They need to be
carefully defined, consistent with one another, verifiable, and
directly traceable to higher-level business or functional requirements.
It is critical that they flow directly from the organization's concept
of operations (how the organization's day-to-day operations are or will
be carried out to meet mission needs).[Footnote 72]
According to the IEEE, a leader in defining the best practices for such
efforts, good requirements have several characteristics, including the
following:[Footnote 73]
* The requirements fully describe the software functionality to be
delivered. Functionality is a defined objective or characteristic
action of a system or component. For example, for grants management, a
key functionality includes knowing (1) the funds obligated to a grantee
for a specific purpose, (2) the cost incurred by the grantee, and (3)
the funds provided in accordance with federal accounting standards.
* The requirements are stated in clear terms that allow for
quantitative evaluation. Specifically, all readers of a requirement
should arrive at a single, consistent interpretation of it.
* Traceability among various requirement documents is maintained.
Requirements for projects can be expressed at various levels depending
on user needs. They range from agencywide business requirements to
increasingly detailed functional requirements that eventually permit
the software project managers and other technicians to design and build
the required functionality in the new system. Adequate traceability
ensures that a requirement in one document is consistent with and
linked to applicable requirements in another document.
* The requirements document contains all of the requirements identified
by the customer, as well as those needed for the definition of the
system.
Studies have shown that problems associated with requirements
definition are key factors in software projects that do not meet their
cost, schedule, and performance goals. Examples include the following:
* A 1988 study found that getting a requirement right in the first
place costs 50 to 200 times less than waiting until after the system is
implemented to get it right.[Footnote 74]
* A 1994 survey of more than 8,000 software projects found that the top
three reasons that projects were delivered late, over budget, and with
less functionality than desired all had to do with requirements
management.[Footnote 75]
* A 1994 study found that, on average, there is about a 25-percent
increase in requirements over a project's lifetime, which translates
into at least a 25-percent increase in the schedule.[Footnote 76]
* A 1997 study noted that between 40 and 60 percent of all defects
found in a software project could be traced back to errors made during
the requirements development stage.[Footnote 77]
Testing:
Testing is the process of executing a program with the intent of
finding errors.[Footnote 78] Because requirements provide the
foundation for system testing, they must be complete, clear, and well
documented to design and implement an effective testing program. Absent
this, an organization is taking a significant risk that substantial
defects will not be detected until after the system is implemented. As
shown in figure 2, there is a direct relationship between requirements
and testing.
Figure 7: Relationship between Requirements Development and Testing:
[See PDF for image]
[End of figure]
Although the actual testing occurs late in the development cycle, test
planning can help disciplined activities reduce requirements- related
defects. For example, developing conceptual test cases based on the
requirements derived from the concept of operations and functional
requirements stages can identify errors, omissions, and ambiguities
long before any code is written or a system is configured. Disciplined
organizations also recognize that planning the testing activities in
coordination with the requirements development process has major
benefits.
Although well-defined requirements are critical for implementing a
successful testing program, disciplined testing efforts for projects
have several characteristics,[Footnote 79] which include the following:
* Testers who assume that the program has errors are likely to find a
greater percentage of the defects present in the system. This is
commonly called the testing mindset.
* Test plans and scripts that clearly define what the expected results
should be when the test case is properly executed and the program does
not have a defect that would be detected by the test case. This helps
to ensure that defects are not mistakenly accepted.
* Processes that ensure test results are thoroughly inspected.
* Test cases that include exposing the system to invalid and unexpected
conditions as well as the valid and expected conditions. This is
commonly referred to as boundary condition testing.
* Testing processes that determine if a program has unwanted side
effects. For example, a process should update the proper records
correctly but should not delete other records.
* Systematic gathering, tracking, and analyzing statistics on the
defects identified during testing.
Although these processes may appear obvious, they are often overlooked
in testing activities.[Footnote 80]
Data Conversion and System Interfaces:
Data conversion is defined as the modification of existing data to
enable them to operate with similar functional capability in a
different environment.[Footnote 81] It is one of the many critical
elements necessary to successfully implement a new system. Because of
the difficulty and complexity associated with financial systems data
conversion, highly skilled staff are needed. There are three primary
phases in a data conversion:
1. Pre-conversion activities prior to and leading up to the conversion,
such as determining the scope and approach or method, developing the
conversion plan, performing data cleanup and validation, ensuring data
integrity, and conducting necessary analysis and testing.
2. Cutover activities to convert the legacy data to the new system,
such as testing system process and data edits, testing system
interfaces (both incoming and outgoing), managing the critical path,
supervising workload completion, and reconciliation.
3. Post-installation activities such as verifying data integrity,
conducting final disposition of the legacy system data, and monitoring
the first reporting cycle.
There are also specific issues that apply uniquely to converting data
as part of the replacement of a financial system, including:
* identifying specific open transactions and balances to be
established,
* analyzing and reconciling transactions for validation purposes, and:
* establishing transactions and balances in the new system through an
automated or manual process.
Further, consideration of various data conversion approaches and
implications are important. Some considerations to be taken into
account for the system conversion are the timing of the conversion
(beginning-of-the-year, mid-year, or incremental) and other options
such as direct or flash conversions, parallel operations, and pilot
conversions. In addition, agencies should consider different data
conversion options for different categories of data when determining
the scope and time lines such as:
* opting not to conduct a data conversion,
* processing new transactions and activity only,
* establishing transaction balances in the new system for reporting
purposes,
* converting open transactions from the legacy system, and:
* recording new activity on closed prior year transactions.
Validation and adjustment of open transactions and data in the legacy
system are essential prerequisites to the conversion process and have
often been problematic. When data conversion is done right, the new
system can flourish. However, converting data incorrectly has lengthy
and long-term repercussions.
System interfaces operate on an ongoing basis linking various systems
and provide data that are critical to day-to-day operations, such as
obligations, disbursements, purchase orders, requisitions, and other
procurement activities. Testing the system interfaces in an end-to-end
manner is necessary so agencies can have reasonable assurance that the
system will be capable of providing the intended functionality. Systems
that lack appropriate system interfaces often rely on manual reentry of
data into multiple systems, convoluted systems, or both. According to
the SEI, a widely recognized model for evaluating the interoperability
of systems is the Levels of Information System Interoperability. This
model focuses on the increasing levels of sophistication of system
interoperability. Efforts at the highest level of this model--
enterprise-based interoperability--are systems that can provide
multiple users access to complex data simultaneously, data and
applications are fully shared and distributed, and data have a common
interpretation regardless of format. This is in contrast to the
traditional interface strategies that are more aligned with the lowest
level of the SEI model. Data exchanged at this level rely on electronic
links that result in a simple electronic exchange of data.
Configuration Management:
According to the SEI, configuration management is defined as a
discipline applying technical and administrative direction and
surveillance to (1) identify and document the functional and physical
characteristics of a configuration item, (2) control changes to those
characteristics, (3) record and report change processing and
implementation status, and (4) verify compliance with specified
requirements.[Footnote 82] The purpose of configuration management is
to establish and maintain the integrity of work products. Configuration
management involves the processes of:
* identifying the configuration of selected work products that compose
the baselines at given points in time,
* controlling changes to configuration items,
* building or providing specifications to build work products from the
configuration management system,
* maintaining the integrity of baselines, and:
* providing accurate status and current configuration data to
developers, integrators, and end users.
The work products placed under configuration management include the
products that are delivered to the customer, designated internal work
products, acquired products, tools, and other items that are used in
creating and describing these work products.
For COTS systems, configuration management focuses on ensuring that
changes to the requirements or components of a system are strictly
controlled to ensure the integrity and consistency of system
requirements or components. Two of the key activities for configuration
management include ensuring that (1) project plans explicitly provide
for evaluation, acquisition, and implementation of new, often frequent,
product releases[Footnote 83] and (2) modification or upgrades to
deployed versions of system components are centrally controlled, and
unilateral user release changes are precluded. Configuration management
recognizes that when using COTS products, it is the vendor, not the
acquisition or implementing organization, that controls the release of
new versions and that new versions are frequently released.
Risk Management:
Risk and opportunity are inextricably related. Although developing
software is a risky endeavor, risk management processes should be used
to manage the project's risks to acceptable levels by taking the
actions necessary to mitigate the adverse effects of significant risks
before they threaten the project's success. If a project does not
effectively manage its risks, then the risks will manage the project.
Risk management is a set of activities for identifying, analyzing,
planning, tracking, and controlling risks. Risk management starts with
identifying the risks before they can become problems. If this step is
not performed well, then the entire risk management process may become
a useless exercise since one cannot manage something that one does not
know anything about. As with the other disciplined processes, risk
management is designed to eliminate the effects of undesirable events
at the earliest possible stage to avoid the costly consequences of
rework.
After the risks are identified, they need to be analyzed so that they
can be better understood and decisions can be made about what actions,
if any, will be taken to address them. Basically, this step includes
activities such as evaluating the impact on the project if the risk
does occur, determining the probability of the event occurring, and
prioritizing the risk against the other risks. Once the risks are
analyzed, a risk management plan is developed that outlines the
information known about the risks and the actions, if any, which will
be taken to mitigate those risks. Risk monitoring is a continuous
process because both the risks and actions planned to address
identified risks need to be monitored to ensure that the risks are
being properly controlled and that new risks are identified as early as
possible. If the actions envisioned in the plan are not adequate, then
additional controls are needed to correct the deficiencies identified.
Project Management:
Effective project management is the process for planning and managing
all project-related activities, such as defining how components are
interrelated, defining tasks, estimating and obtaining resources, and
scheduling activities. Project management allows the performance, cost,
and schedule of the overall program to be continually measured,
compared with planned objectives, and controlled. Project management
activities include planning, monitoring, and controlling the project.
Project planning is the process used to establish reasonable plans for
carrying out and managing the software project. This includes (1)
developing estimates of the resources needed for the work to be
performed, (2) establishing the necessary commitments, and (3) defining
the plan necessary to perform the work. Effective planning is needed to
identify and resolve problems as soon as possible, when it is the
cheapest to fix them. According to one author, the average project
expends about 80 percent of the time on unplanned rework--fixing
mistakes that were made earlier in the project. Recognizing that
mistakes will be made in a project is an important part of planning.
According to this author, successful system development activities are
designed so that the project team makes a carefully planned series of
small mistakes to avoid making large, unplanned mistakes. For example,
spending the time to adequately analyze three design alternatives
before selecting one results in time spent analyzing two alternatives
that were not selected. However, discovering that a design is
inadequate after development can result in code that must be rewritten,
at a cost greater than analyzing the three alternatives in the first
place. This same author notes that a good rule of thumb is that each
hour a developer spends reviewing project requirements and architecture
saves 3 to 10 hours later in the project.[Footnote 84]
Project monitoring and control help to understand the progress of the
project and determine when corrective actions are needed based on the
project's performance. Best business practices indicate that a key
facet of project management and oversight is the ability to effectively
monitor and evaluate a project's actual performance, cost, and schedule
against what was planned.[Footnote 85] In order to perform this
critical task, the accumulation of quantitative data or metrics is
required and can be used to evaluate a project's performance. An
effective project management and oversight process uses quantitative
data or metrics to understand matters such as (1) whether the project
plan needs to be adjusted and (2) oversight actions that may be needed
to ensure that the project meets its stated goals and complies with
agency guidance. For example, an earned value management system is one
metric that can be employed to better manage and oversee a system
project.[Footnote 86] An earned value management system attempts to
compare the value of work accomplished during a given period with the
work scheduled for that period. With ineffective project oversight,
management can only respond to problems as they arise.
Agency management can also perform oversight functions, such as project
reviews and participation in key meetings, to help ensure that the
project will meet the agency needs. Management can use independent
verification and validation reviews to provide it with assessments of
the project's software deliverables and processes. Although independent
of the developer, verification and validation is an integral part of
the overall development program and helps management mitigate risks.
This core element involves having an independent third party--such as
an internal audit function or a contractor that is not involved with
any of the system implementation efforts--verify and validate that the
systems were implemented in accordance with the established business
processes and standards. Doing so provides agencies with needed
assurance about the quality of the system, which is discussed in more
detail in the following section.
Quality Assurance:
Quality assurance is defined as a set of procedures designed to ensure
that quality standards and processes are adhered to and that the final
product meets or exceeds the required technical and performance
requirements. Quality assurance is a widely used approach in the
software industry to improve upon product delivery and the meeting of
customer requirements and expectations. The SEI indicates that quality
assurance should begin in the early phases of a project to establish
plans, processes, standards, and procedures that will add value to the
project and satisfy the requirements of the project and the
organizational policies. Quality assurance provides independent
assessments, typically performed by an independent verification and
validation or internal audit team, of whether management process
requirements are being followed and whether product standards and
requirements are being satisfied. Some of the widely used quality
assurance activities include defect tracking, technical reviews, and
system testing.
* Defect tracking-keeping a record of each defect found, its source,
when it was detected, when it was resolved, how it was resolved (fixed
or not), and so on.
* Technical reviews-reviewing user interface prototypes, requirements
specifications, architecture, designs, and all other technical work
products.
* System testing-executing software for the purpose of finding defects,
typically performed by an independent test organization or quality
assurance group.
According to one author, quality assurance activities might seem to
result in a lot of overhead, but in actuality, exactly the opposite is
true.[Footnote 87] If defects can be prevented or removed early, a
significant schedule benefit can be realized. For example, studies have
shown that reworking defective requirements, design, and code typically
consumes 40 to 50 percent of the total costs of software development
projects.[Footnote 88] An effective quality assurance approach is to
detect as many defects as possible as early as possible to keep the
costs of corrections down. However, enormous amounts of time can be
saved by detecting defects earlier than during system testing.
[End of section]
Appendix IV: Comments from the Office of Management and Budget:
EXECUTIVE OFFICE OF THE PRESIDENT:
OFFICE OF MANAGEMENT AND BUDGET:
THE CONTROLLER:
WASHINGTON, D.C. 20503:
JAN 19 2006:
Mr. McCoy Williams:
Director, Financial Management and Assurance:
United States Government Accountability Office:
Washington, DC 20548:
Dear Mr. Williams:
Thank you for allowing us to comment on the Government Accountability
Office (GAO) draft audit report entitled "Financial Management Systems:
Additional Efforts Needed to Address Key Causes of Modernization
Failures."
In general, OMB agrees with your assessment that many agencies need to
produce better results when implementing financial management systems.
To improve performance in this important area, the Administration is
taking several important steps to mitigate the risks and costs
associated with financial system modernization efforts while improving
the quality of the final product. Steps that the Administration is
taking include: (1) implementing the Financial Management Line of
Business (FMLOB) initiative; (2) communicating lessons learned to the
financial management community; and, (3) emphasizing the President's
Management Agenda (PMA) accountability framework on financial
modernization.
As outlined in more detail in the enclosed Memorandum, the FMLOB
initiative is one of the President's top management priorities. When
the FMLOB is fully realized, agencies will consistently meet cost,
schedule, and performance goals as they implement new financial systems
that provide Federal managers with accurate, useful, and timely
information for decision-making. There are multiple aspects of the
FMLOB effort that together foster the necessary framework for improved
results. First, we are looking to standardize the business rules,
processes, interfaces, and data. Standardization will mitigate some of
the costs and risks in migrating to a modernized financial system.
Second, we are facilitating a more competitive environment for
financial system alternatives and leveraging the economies that shared
service solutions provide. In this environment a single entity provides
financial management services for multiple organizations.
In addition to the efforts being undertaken through the FMLOB
initiative, OMB is partnering with the Chief Financial Officers Council
(CFOC) to convey to senior agency leaders critical lessons that have
been learned from systems implementations in both the public and
private sectors. We are conducting recurrent forums in which senior
managers, either individually or as part of a panel, discuss diverse
approaches to reducing implementation risks. The forums are slated to
become a permanent feature of the CFOC committee structure. Also, OMB
will be issuing guidance outlining fundamental risk-reduction
approaches that agencies can implement when acquiring and implementing
financial systems.
As you know, the President and his financial management team are
committed to improving all aspects of financial management in the
government. The PMA provides an effective accountability framework for
ensuring that agencies take all the necessary steps to improve
financial performance, including financial systems. Thus, agency
efforts to improve the performance of financial system modernization
investments, as well as agency participation in the FMLOB initiative,
factor significantly into OMB's assessments under the PMA. Also, OMB's
revised Circular A-123 mandates that agencies evaluate their internal
controls and outline the financial data streams associated with
existing business processes. Identifying existing business processes -
and recording the control weaknesses existing therein - will be of
substantial value to agencies when preparing for any future systems
implementation.
Our common goal goes far beyond attaining unqualified opinions on
agency financial statements. We are both striving for the creation and
use - for both government managers and the taxpayer - of reliable,
useful and timely management information. Taken as a whole, we agree
with the recommendations in your report and are vigorously executing
many of them (and have been for some time). We believe that our
consensus-based approach - of which the major elements were described
above - presents a fully actionable strategy for improving financial
management in the government. We certainly value GAO's continued
support of our efforts.
We appreciate the opportunity to comment on the draft report and look
forward to working with GAO in improving Federal financial management
systems. If you have any questions please feel free to contact David
Alekson at 202.395.5642.
Sincerely,
Signed by:
Linda M. Combs:
Controller:
Enclosure:
EXECUTIVE OFFICE OF THE PRESIDENT:
OFFICE of MANAGEMENT AND BUDGET:
WASHINGTON. D.C. 20503:
DEC 16 2005
MEMORANDUM FOR CHIEF FINANCIAL OFFICERS:
FROM: Linda M. Combs, Controller:
SUBJECT: Update on the Financial Management Line of Business and the
Financial Systems Integration Office:
Achieving cost and quality improvements through the Financial
Management Line of Business (FMLOB) is one of the President's top
management priorities. To be successful, the Federal financial
management community must have a clear set of objectives for the FMLOB
initiative and an agreed upon roadmap for achieving them. The goal of
this memorandum is to clarify: (i) the vision and goals of the FMLOB;
(ii) the priority initiatives that will be undertaken in the near term
to enable success in the FMLOB; and (iii) the governance structure that
will be in place to ensure accountability for successful completion of
priority initiatives, including the enhanced role of the Financial
Systems Integration Office (FSIO).
The clarifications and decisions outlined below were developed in close
consultation with key stakeholders in the financial management
community. Specifically, the content of this memorandum is based on the
recommendations of a working group made up of staff from OMB (OFFM and
E-gov), FSIO, and the Financial Systems Integration Committee (FSIC) of
the CFO Council. In developing these recommendations, the working group
reviewed all relevant and historical materials on the FMLOB and JFMIP,
held a series of forums with vendors, shared service providers, and
GAO, and gathered information and feedback from the CFO and CIO
communities on draft deliverables and other work products. These
communications will continue on a regular basis so that the priority
initiatives described below can be adjusted to reflect practical
lessons learned as well as new insights into better solutions and
approaches.
1. What is the overall vision of the FMLOB?
The overall vision of the FMLOB is to improve the cost, quality, and
performance of financial management (FM) systems by leveraging shared
service solutions and implementing other government-wide reforms that
foster efficiencies in Federal financial operations.
2. What are the specific goals of the FMLOB?
The goals of the FMLOB are that Federal agencies are implementing
financial systems that:
- Provide timely and accurate data available for decision-making;
- Facilitate stronger internal controls that ensure integrity in
accounting and other stewardship activities;
- Reduce costs by providing a competitive alternative for agencies to
acquire, develop, implement, and operate financial management systems
through shared service solutions;
- Standardize systems, business processes and data elements; and:
- Provide for seamless data exchange between and among Federal agencies
by implementing a common language and structure for financial
information and system interfaces.
3. What are the critical milestones that must be accomplished in order
to achieve the vision and goals of the FMLOB?
Federal agencies have begun implementing the FMLOB initiative by
actively migrating to shared service providers and initiating solutions
to integrate financial data among and between agency business systems.
Nothing in this memorandum changes the expectation that agencies will
continue to take all the necessary steps (in the earliest possible
timeframes) to meet FMLOB objectives. The milestones described below,
therefore, are intended to facilitate, not delay, agency efforts.
As depicted in Attachment 1, the critical milestones of the FMLOB can
be broken down into three stages - (i) transparency and
standardization; (ii) competitive environment and seamless data
integration; and (iii) results.
Stage 1: Transparency and Standardization. In order to enable a
competitive environment where agencies have more options and leverage
in choosing a financial system, and in order to facilitate seamless
integration of financial data among agency business systems, additional
transparency and standardization is required.
Transparency: In determining the best options available when
modernizing financial systems, the Federal financial community must
have clarity on how to evaluate the performance and cost of shared
service alternatives (i.e., Centers of Excellence (COE)) as well as
clarity on what steps Federal agencies are expected to undertake in
order to migrate to a COE. As described in more detail below, a COE is
a shared service solution where a single entity provides financial
management services for multiple organizations. In order to achieve
additional transparency, two specific projects (with associated
milestones) will be undertaken:
* Establishment of Common Performance Measures - This project will
result in standard quality and cost measures for agencies to benchmark
and compare the performance of financial system alternatives.
* Development of Migration Planning Guidance - This project will result
in comprehensive guidance that helps Federal agencies describe, prepare
for, and manage an agency's migration to a COE. This guidance will also
include a definition of the full range of services to be provided by
all COEs and a description of the "rules of engagement," including
templates for service level agreements outlining provider and client
responsibilities.
Note: OMB is in the process of developing a "competition framework"
that will assist agencies in conducting competitions and selecting a
COE. This framework - expected to be issued in late December/early
January - will be incorporated into the Migration Planning Guidance
described above.
Standardization: In order to mitigate the cost and risk of migrations
to a COE and to ensure that financial data can be shared across agency
business systems, the Federal government must ensure greater
standardization of business processes, interfaces, and data. To this
end, two specific projects (with associated milestones) will be
undertaken:
Development of Standard Business Processes - This project will result
in government-wide common business rules, data components, and policies
for funds control, accounts payable, accounts receivable, and fixed
assets.
Creation of a Common Government-wide Accounting - This project will
result in a uniform accounting code structure, layout, and definitions.
Once established, all agencies will be expected to adopt these common
processes on a schedule agreed upon between the agency and OMB. See
Attachment 2 for additional details on the priority projects related to
the transparency and standardization initiatives described above.
Stage 2: Competitive Environment and Seamless Data Integration. In
order to enable improved performance of financial systems, the FMLOB
envisions more competitive alternatives for financial systems and an
environment where financial data can be more easily compared and
aggregated across agencies.
Competitive Environment: To enable improved cost, quality, and
performance of financial systems, Federal agencies must have
competitive options available for financial systems. The COE framework
is intended to help achieve these results. A COE is a shared service
solution where a single entity provides financial management services
for multiple organizations. When the FMLOB is successful, there will be
a limited number of stable and high performing COEs that provide
competitive alternatives for agencies investing in financial system
modernizations. The economies of scale and skill of a COE will allow it
to provide Federal agencies with a lower risk, lower cost, and
increased service quality alternative for financial system
modernization efforts.
Notably, a competitive environment is sustainable if Federal agencies
have the ability to migrate from one solution to a more competitive or
better performing alternative that is offered. The transparency and
standardization efforts described above will lay the foundation for
facilitating better portability of agency systems from one solution to
another.
Seamless Data Integration: The standardization efforts, associated with
Stage 1 of the FMLOB initiative, will enable financial data to be
easily compared and aggregated across agencies. For example, the
development of a common government-wide accounting code will assist in
the intra-governmental reconciliation process by requiring that all
common types of financial data be accounted for in a similar format. A
common structure will also enable easier transmission of financial
reports to OMB and Treasury and assist these central agencies with
aggregating similar-type data on a government-wide basis. Seamless and
standardized data exchange will enable the government to streamline
operations through more efficient information management and increased
data accuracy.
In addition, seamless data integration will reduce the costs and risks
of establishing interfaces between agency business systems. By
requiring standard core business processes, rules, data definitions,
and a common government-wide accounting code, interfacing systems, such
as travel, will not have to be specifically designed for each agency.
This will save agencies money and enable them to more easily migrate
between different system solutions.
Stage 3: Results. When the FMLOB is fully realized, agencies' data will
be more timely and accurate for decision-making and there will be
improved government-wide stewardship and accounting. More timely and
accurate data will result from the standardization and seamless data
integration efforts, including the implementation of centralized
interfaces between core financial systems and other systems. These
efforts will focus on promoting strong internal controls and ensuring
the integrity of accounting data. The easy exchange of data between
federal agencies will increase federal managers' stewardship abilities.
There will also be a reduction of government-wide information
technology costs and risks. These benefits will be the result of shared-
service solutions, also assisted by the standardization and seamless
data integration efforts. Shared-service solutions will enable
economies of scale by centrally locating, or consolidating, solution
assets and reusing Federal and commercial subject matter expertise
through common acquisitions, interface development, and application
management. The reduction in the number of agencies implementing their
own systems will reduce the risks, and associated costs, of systems
implementations.
4. What governance structure will be in place to ensure accountability
for successful completion of priority FMLOB initiatives?
As depicted in Attachment 3, FSIO will have direct responsibility for
completing priority projects under the FMLOB. OMB, in consultation with
the Financial Systems Integration Committee (FSIC) of the CFO Council,
will provide oversight and guidance to FSIO on priorities and expected
performance in meeting these priorities.
OMB will continue its role as Executive Sponsors of the FMLOB. The FSIC
chair will be the lead agency sponsor for the FMLOB. A liaison from the
CIO community and the Executive Director of FSIO will serve on the FSIC
and support the FSIC chair in his/her responsibilities as they relate
to the FMLOB. Going forward, FSIO will coordinate the collection and
expenditure of FMLOB funds.
The FSIC will assist OMB in evaluating and monitoring FSIO's progress
in completing FMLOB projects and provide feedback to OMB and FSIO. As
appropriate, members of the F SIC will participate in working groups to
assist FSIO with completing deliverables. The FSIC will evaluate its
current subcommittee structure to assess whether changes are needed to
best meet these objectives.
The updated governance structure ensures that the FSIO, FMLOB, and the
FSIC do not operate in separate stovepipes. In addition, responsibility
for work products will now rest with FSIO, where full time dedicated
staff will be held accountable for achieving FMLOB milestones.
5. What is the status of the realignment of JFMIP to FSIO?
In December of 2004, the JFMIP Principals voted to modify the roles and
responsibilities of the JFMIP Program Office, now FSIO. As a result,
OMB and the FSIC were given an increased management and oversight role
in the activities of FSIO. OMB and the FSIC have worked closely with
FSIO staff to update FSIO's mission statement and define FSIO's scope
of activities and priorities for FY 2006.
In terms of mission and scope, FSIO has three major areas of
responsibilities: (a) continuing its primary role of core financial
system requirements development, testing, and certification; (b)
providing support to the Federal financial community by taking on
special priority projects as determined by the OMB Controller, CFO
Council, and the FSIO Executive Director, and (c) conducting outreach
through the annual financial management conference and other related
activities.
Most importantly, the projects that FSIO undertakes will directly
reflect the priorities of the CFO Community and OMB. As noted above,
the priority projects to be undertaken in the near term will relate to
the transparency and standardization initiatives of the FMLOB. Other
projects that were previously under FSIO's purview - acquisition,
budget formulation, and property system requirements - have been
transitioned to the Chief Acquisition Council, the Budget Officers
Advisory Council, and the Federal Real Property Council, respectively,
for their consideration and completion.
Also, effective January 2006, the FSIO office will be transferred from
the General Service Administration's (GSA) Office of the Chief
Financial Officer to the Office of Government-wide Policy, Office of
Technology Strategy (OTS). There are several significant benefits of
this move
* lower administrative cost through shared resources (rent, supplies,
equipment, etc.)
* permanent SES in place to provide leadership to FSIO staff:
* access to immediate resources and expertise on IT, administrative
management, contract management, testing, etc.
* fits well with current mission and stakeholder focused model of OTS:
6. What specific actions are expected of Federal agencies?
As described above, a central goal of the FMLOB is that financial
system investments will be at lower risk and lower cost as agencies
leverage the economies offered by shared service solutions (i.e.,
COEs). To this end, OMB has instituted a policy that agencies seeking
to modernize their financial system must either be designated a public
COE or must migrate to a COE (public, private, or a combination of
both). Although exceptions to this policy will be made in limited
situations when an agency demonstrates compelling evidence of a best
value and lower risk alternative, it is OMB's intent to avoid
investments in "in-house" solutions wherever possible so that the
shared service framework can fully achieve potential and anticipated
returns.
To the extent we require any specific action on your part to carry out
the priority initiatives and milestones outlined above, we will
communicate such requests through subsequent memos from OMB or the
FSIC.
Thank you for your ongoing commitment to the FMLOB initiative. I look
forward to working with each of you to achieve specific and measurable
results in the immediate future.
cc: Chief Information Officers Council:
Attachment 1: FMLOB Vision/Framework:
[See PDF for image]
[End of figure]
Attachment 2: FMLOB - Workstreams:
[See PDF for image]
[End of figure]
Attachment 3: Governance Structure for Financial Management System
Initiatives:
[See PDF for image]
[End of figure]
[End of section]
Appendix V: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
McCoy Williams (202) 512-9095 or williamsm1@gao.gov;
Keith A. Rhodes (202) 512-6412 or rhodesk@gao.gov:
Staff Acknowledgments:
In addition to the contacts named above, Kay Daly, Assistant Director;
Chris Martin, Senior-Level Technologist; Francine DelVecchio; Mike
LaForge; and Chanetta Reed made key contributions to this report.
[End of section]
Related GAO Products:
DOD Business Systems Modernization: Navy ERP Adherence to Best Business
Practices Critical to Avoid Past Failures. GAO-05-858. Washington,
D.C.: September 29, 2005.
Financial Management: Achieving FFMIA Compliance Continues to Challenge
Agencies. GAO-05-881. Washington, D.C.: September 20, 2005.
Business Modernization: Some Progress Made toward Implementing GAO
Recommendations Related to NASA's Integrated Financial Management
Program. GAO-05-799R. Washington, D.C.: September 9, 2005.
Business Systems Modernization: Internal Revenue Service's Fiscal Year
2005 Expenditure Plan. GAO-05-774. Washington, D.C.: July 22, 2005.
DOD Business Systems Modernization: Long-standing Weaknesses in
Enterprise Architecture Development Need to Be Addressed. GAO-05-702.
Washington, D.C.: July 22, 2005.
Army Depot Maintenance: Ineffective Oversight of Depot Maintenance
Operations and System Implementation Efforts. GAO-05-441. Washington,
D.C.: June 30, 2005.
DOD Business Systems Modernization: Billions Being Invested without
Adequate Oversight. GAO-05-381. Washington, D.C.: April 29, 2005.
Internal Revenue Service: Assessment of the Fiscal Year 2006 Budget
Request. GAO-05-566. Washington, D.C.: April 27, 2005.
Information Technology: OMB Can Make More Effective Use of Its
Investment Reviews. GAO-05-276. Washington, D.C.: April 15, 2005.
Information Technology: Customs Automated Commercial Environment
Program Progressing, but Need for Management Improvements Continues.
GAO-05-267. Washington, D.C.: March 14, 2005.
Office of Personnel Management: Retirement Systems Modernization
Program Faces Numerous Challenges. GAO-05-237. Washington, D.C.:
February 28, 2005.
DOD Systems Modernization: Management of Integrated Military Human
Capital Program Needs Additional Improvements. GAO-05-189. Washington,
D.C.: February 11, 2005. Department of Defense: Further Actions Are
Needed to Effectively Address Business Management Problems and Overcome
Key Business Transformation Challenges. GAO-05-140T. Washington, D.C.:
November 18, 2004.
Business Systems Modernization: IRS's Fiscal Year 2004 Expenditure
Plan. GAO-05-46. Washington, D.C.: November 17, 2004.
Financial Management: Improved Financial Systems Are Key to FFMIA
Compliance. GAO-05-20. Washington, D.C.: October 1, 2004.
Financial Management Systems: HHS Faces Many Challenges in Implementing
Its Unified Financial Management System. GAO-04-1089T. Washington,
D.C.: September 30, 2004.
Financial Management Systems: Lack of Disciplined Processes Puts
Implementation of HHS' Financial System at Risk. GAO-04-1008.
Washington, D.C.: September 23, 2004.
Information Technology: DOD's Acquisition Policies and Guidance Need to
Incorporate Additional Best Practices and Controls. GAO-04-722.
Washington, D.C.: July 30, 2004.
Department of Defense: Financial and Business Management Transformation
Hindered by Long-standing Problems. GAO-04-941T. Washington, D.C.: July
8, 2004.
Information Security: Agencies Need to Implement Consistent Processes
in Authorizing Systems for Operation. GAO-04-376. Washington, D.C.:
June 28, 2004.
DOD Business Systems Modernization: Billions Continue to Be Invested
with Inadequate Management Oversight and Accountability. GAO-04-615.
Washington, D.C.: May 27, 2004.
Information Technology: The Federal Enterprise Architecture and
Agencies' Enterprise Architectures Are Still Maturing. GAO-04-798T.
Washington, D.C.: May 19, 2004.
National Aeronautics and Space Administration: Significant Actions
Needed to Address Long-standing Financial Management Problems. GAO-04-
754T. Washington, D.C.: May 19, 2004.
DOD Business Systems Modernization: Limited Progress in Development of
Business Enterprise Architecture and Oversight of Information
Technology Investments. GAO-04-731R. Washington, D.C.: May 17, 2004.
Information Technology: Early Releases of Customs Trade System
Operating, but Pattern of Cost and Schedule Problems Needs to Be
Addressed. GAO-04-719. Washington, D.C.: May 14, 2004.
Information Technology Investment Management: A Framework for Assessing
and Improving Process Maturity (Version 1.1). GAO-04-394G. Washington,
D.C.: March 2004.
Information Technology Management: Governmentwide Strategic Planning,
Performance Measurement, and Investment Management Can Be Further
Improved. GAO-04-49. Washington, D.C.: January 12, 2004.
Human Capital: Key Principles for Effective Strategic Workforce
Planning. GAO-04-39. Washington, D.C.: December 11, 2003.
Business Modernization: NASA's Challenges in Managing Its Integrated
Financial Management Program. GAO-04-255. Washington, D.C.: November
21, 2003.
Business Modernization: NASA's Integrated Financial Management Program
Does Not Fully Address Agency's External Reporting Issues. GAO- 04-151.
Washington, D.C.: November 21, 2003.
Business Modernization: Disciplined Processes Needed to Better Manage
NASA's Integrated Financial Management Program. GAO-04-118. Washington,
D.C.: November 21, 2003.
Information Technology: Architecture Needed to Guide NASA's Financial
Management Modernization. GAO-04-43. Washington, D.C.: November 21,
2003.
Information Technology: Leadership Remains Key to Agencies Making
Progress on Enterprise Architecture Efforts. GAO-04-40. Washington,
D.C.: November 17, 2003.
DOD Business Systems Modernization: Important Progress Made to Develop
Business Enterprise Architecture, but Much Work Remains. GAO- 03-1018.
Washington, D.C.: September 19, 2003. Business Systems Modernization:
IRS Has Made Significant Progress in Improving Its Management Controls,
but Risks Remain. GAO-03-768. Washington, D.C.: June 27, 2003.
Business Modernization: Improvements Needed in Management of NASA's
Integrated Financial Management Program. GAO-03-507. Washington, D.C.:
April 30, 2003.
Department of Housing and Urban Development: Status of Efforts to
Implement an Integrated Financial Management System. GAO-03-447R.
Washington, D.C.: April 9, 2003.
Information Technology: A Framework for Assessing and Improving
Enterprise Architecture Management (Version 1.1). GAO-03-584G.
Washington, D.C.: April 2003.
DOD Business Systems Modernization: Continued Investment in Key
Accounting Systems Needs to be Justified. GAO-03-465. Washington, D.C.:
March 28, 2003.
DOD Business Systems Modernization: Improvements to Enterprise
Architecture Development and Implementation Efforts Needed. GAO-03-
458. Washington, D.C.: February 28, 2003.
Customs Service Modernization: Automated Commercial Environment
Progressing, but Further Acquisition Management Improvements Needed.
GAO-03-406. Washington, D.C.: February 28, 2003.
Customs Service Modernization: Third Expenditure Plan Meets Legislative
Conditions, but Cost Estimating Improvements Needed. GAO- 02-908.
Washington, D.C.: August 9, 2002.
DOD Financial Management: Important Steps Underway but Reform Will
Require a Long-term Commitment. GAO-02-784T. Washington, D.C.: June 4,
2002.
Customs Service Modernization: Management Improvements Needed on High-
Risk Automated Commercial Environment Project. GAO-02-545. Washington,
D.C.: May 13, 2002.
Tax Administration: IRS Continues to Face Management Challenges in its
Business Practices and Modernization Efforts. GAO-02-619T. Washington,
D.C.: April 15, 2002.
Business Systems Modernization: IRS Needs to Better Balance Management
Capacity with Systems Acquisition Workload. GAO-02-356. Washington,
D.C.: February 28, 2002.
Human Capital: Building the Information Technology Workforce to Achieve
Results. GAO-01-1007T. Washington, D.C.: July 31, 2001.
Business Systems Modernization: Results of Review of IRS' March 2001
Expenditure Plan. GAO-01-716. Washington, D.C.: June 29, 2001.
Information Technology: DLA Should Strengthen Business Systems
Modernization Architecture and Investment Activities. GAO-01-631.
Washington, D.C.: June 29, 2001.
Customs Service Modernization: Results of Review of First Automated
Commercial Environment Expenditure Plan. GAO-01-696. Washington, D.C.:
June 5, 2001.
Information Technology: Architecture Needed to Guide Modernization of
DOD's Financial Operations. GAO-01-525. Washington, D.C.: May 17, 2001.
District of Columbia: Weaknesses in Financial Management System
Implementation. GAO-01-489. Washington, D.C.: April 30, 2001.
Tax Systems Modernization: Results of Review of IRS' Third Expenditure
Plan. GAO-01-227. Washington, D.C.: January 22, 2001.
Tax Systems Modernization: Results of Review of IRS' August 2000
Interim Spending Plan. GAO-01-91. Washington, D.C.: November 8, 2000.
Indian Trust Funds: Improvements Made in Acquisition of New Asset and
Accounting System But Significant Risks Remain. GAO/AIMD-00-259.
Washington, D.C.: September 15, 2000.
Tax Systems Modernization: Results of Review of IRS' March 7, 2000,
Expenditure Plan. GAO/AIMD-00-175. Washington, D.C.: May 24, 2000.
Executive Guide: Creating Value Through World-class Financial
Management. GAO/AIMD-00-134. Washington, D.C.: April 2000.
Indian Trust Funds: Interior Lacks Assurance That Trust Improvement
Plan Will Be Effective. GAO/AIMD-99-53. Washington, D.C.: April 28,
1999.
Federal Information System Controls Audit Manual, Volume I: Financial
Statement Audits. GAO/AIMD-12.19.6. Washington, D.C.: January 1999.
District of Columbia: Status of Efforts to Develop a New Financial
Management System. GAO/AIMD-97-101R. Washington, D.C.: July 9, 1997.
Information Security: Opportunities for Improved OMB Oversight of
Agency Practices. GAO/AIMD-96-110. Washington, D.C.: September 24,
1996.
[End of section]
(192162):
FOOTNOTES
[1] Pub. L. No. 101-576, 104 Stat. 2838 (Nov. 15, 1990).
[2] Pub. L. No. 104-208, div. A., § 101(f), title VIII, 110 Stat. 3009,
3009-389 (Sept. 30, 1996).
[3] According to systems requirements issued by the former Joint
Financial Management Improvement Program (JFMIP) which remain in
effect, core financial systems are the backbone of an agency's
integrated financial management system. They should provide common
processing routines, support common data for critical financial
management functions affecting the entire agency, and maintain the
required financial data integrity control over financial transactions,
resource balances, and other financial systems. A core financial system
should support an agency's general ledger, funds management, payments,
receivables, and basic cost management functions. Also, the system
should receive data from other financial-related systems, such as
inventory and property systems, and from direct user input. It should
also support financial statement preparation and financial performance
measurement and analysis.
[4] GAO, National Aeronautics and Space Administration: Significant
Actions Needed to Address Long-standing Financial Management Problems,
GAO-04-754T (Washington, D.C.: May 19, 2004).
[5] An ERP solution is an automated system using commercial off-the-
shelf software and consisting of multiple, integrated functional
modules that perform a variety of business-related tasks such as
accounts payable, general ledger accounting, and supply chain
management.
[6] GAO, DOD Business Systems Modernization: Navy ERP Adherence to Best
Business Practices Critical to Avoid Past Failures, GAO-05-858
(Washington, D.C.: Sept. 29, 2005).
[7] GAO, Army Depot Maintenance: Ineffective Oversight of Depot
Maintenance Operations and System Implementation Efforts, GAO-05-441
(Washington, D.C.: June 30, 2005).
[8] JFMIP was formed under the authority of the Budget and Accounting
Procedures Act of 1950, Pub. L. No. 81-784, § 111(f), 64 Stat. 832, 835
(Sept. 12, 1950) (codified at 31 U.S.C. § 3511), as a joint and
cooperative undertaking of GAO, the U.S. Department of the Treasury,
OMB, and Office of Personnel Management (OPM), working in cooperation
to improve financial management practices in the federal government.
Leadership and program guidance were provided by the four JFMIP
principals-the Comptroller General of the United States, the Secretary
of the Treasury, and the Directors of OMB and OPM.
[9] GAO, Information Technology: The Federal Enterprise Architecture
and Agencies' Enterprise Architectures Are Still Maturing, GAO-04-798T
(Washington, D.C.: May 19, 2004).
[10] An application service provider is a third-party entity that
manages and distributes software-based services and solutions to
customers across a wide area network from a central data center. In
essence, application service providers are a way for agencies to
outsource some or almost all aspects of their information technology
needs.
[11] OMB was required to establish this office under the E-Government
Act of 2002, Pub. L. No. 107-347, § 101(a), 116 Stat. 2899, 2902-05
(Dec. 17, 2002) (codified at 44 U.S.C. § 3602(a), (f)).
[12] Pub. L. No. 104-13, § 2, 109 Stat. 163, 166 (May 22, 1995)
(codified at 44 U.S.C. § 3503).
[13] GAO, Information Technology: DOD's Acquisition Policies and
Guidance Need to Incorporate Additional Best Practices and Controls,
GAO-04-722 (Washington, D.C.: July 30, 2004).
[14] The SEI is a federally funded research and development center
operated by Carnegie Mellon University and sponsored by DOD. The SEI
objective is to provide leadership in software engineering and in the
transition of new software engineering technology into practice.
[15] The IEEE is a nonprofit, technical professional association that
develops standards for a broad range of global industries, including
the IT and information assurance industries and is a leading source for
defining best practices.
[16] Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash.: Microsoft Press, 1996).
[17] Steve McConnell, Code Complete, Second Edition (Redmond, Wash.:
Microsoft Press, 2004).
[18] GAO, High-Risk Series: An Update, GAO-01-263 (Washington, D.C.:
Jan. 2001).
[19] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.:
Jan. 2005).
[20] GAO, Human Capital: Key Principles for Effective Strategic
Workforce Planning, GAO-04-39 (Washington, D.C.: Dec. 11, 2003).
[21] OMB, Information Technology Project Manager Qualification
Guidance, M-04-19 (Washington, D.C.: July 21, 2004).
[22] In July 2004, the CIO Council's Workforce and Human Capital for
Information Technology Committee released the Federal IT Project
Manager Guidance Matrix. The matrix identified the competencies,
experience, education, training, and development that managers should
possess for projects with three different levels of complexity.
[23] OMB Circular No. A-130, Management of Federal Information
Resources (Washington, D.C.: Nov. 28, 2000).
[24] Paperwork Reduction Act of 1995, Pub. L. No. 104-13, 109 Stat. 163
(May 22, 1995) (codified at 44 U.S.C. §§ 3501-3521).
[25] Clinger-Cohen Act of 1996, Pub. L. No. 104-106, div. E, § 5125,
110 Stat. 679, 684-85 (Feb. 10, 1996) (codified at 40 U.S.C. § 11315
(b)).
[26] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-584G
(Washington, D.C.: April 2003).
[27] For example, see GAO, Information Technology Investment
Management: A Framework for Assessing and Improving Process Maturity
(Version 1.1), GAO-04-394G (Washington, D.C.: March 2004); and OMB
Circular No. A-130.
[28] GAO-04-394G.
[29] Pub. L. No. 107-347, tit. III, § 301, 116 Stat. 2946, 2946-55
(Dec. 17, 2002) (codified at 44 U.S.C. §§ 3541-3549).
[30] GAO, Information Technology: Leadership Remains Key to Agencies
Making Progress on Enterprise Architecture Efforts, GAO-04-40
(Washington, D.C.: Nov. 17, 2003).
[31] GAO, Information Technology Management: Governmentwide Strategic
Planning, Performance Measurement, and Investment Management Can Be
Further Improved, GAO-04-49 (Washington, D.C.: Jan. 12, 2004).
[32] GAO, Information Security: Agencies Need to Implement Consistent
Processes in Authorizing Systems for Operation, GAO-04-376 (Washington,
D.C.: June 28, 2004).
[33] GAO, DOD Business Systems Modernization: Billions Continue to Be
Invested with Inadequate Management Oversight and Accountability, GAO-
04-615 (Washington, D.C.: May 27, 2004).
[34] GAO-05-441.
[35] GAO, Business Modernization: Some Progress Made toward
Implementing GAO Recommendations Related to NASA's Integrated Financial
Management Program, GAO-05-799R (Washington, D.C.: Sept. 9, 2005).
[36] Section 803 of FFMIA requires the major departments and agencies
covered by the CFO Act to implement and maintain financial management
systems that comply substantially with (1) federal financial management
systems requirements, (2) applicable federal accounting standards, and
(3) the U.S. Government Standard General Ledger at the transaction
level.
[37] Department of Veterans Affairs Office of Inspector General, Issues
at VA Medical Center Bay Pines, Florida and Procurement and Deployment
of the Core Financial and Logistics System, Report 04-01371-177
(Washington, D.C.: Aug. 11, 2004).
[38] GAO, Executive Guide: Creating Value Through World-class Financial
Management, GAO/AIMD-00-134 (Washington, D.C.: April 2000).
[39] GAO, Human Capital: Building the Information Technology Workforce
to Achieve Results, GAO-01-1007T (Washington, D.C.: July 31, 2001).
[40] GAO, Customs Service Modernization: Management Improvements Needed
on High-Risk Automated Commercial Environment Project, GAO-02-545
(Washington, D.C.: May 13, 2002).
[41] GAO, Information Technology: Customs Automated Commercial
Environment Program Progressing, but Need for Management Improvements
Continues, GAO-05-267 (Washington, D.C.: Mar. 14, 2005).
[42] GAO, Financial Management Systems: Lack of Disciplined Processes
Puts Implementation of HHS's Financial System at Risk, GAO-04-1008
(Washington, D.C.: Sept. 23, 2004).
[43] GAO-04-722.
[44] GAO, DOD Business Systems Modernization: Limited Progress in
Development of Business Enterprise Architecture and Oversight of
Information Technology Investments, GAO-04-731R (Washington, D.C.: May
17, 2004).
[45] GAO, DOD Business Systems Modernization: Billions Being Invested
without Adequate Oversight, GAO-05-381 (Washington, D.C.: Apr. 29,
2005).
[46] GAO, DOD Business Systems Modernization: Long-standing Weaknesses
in Enterprise Architecture Development Need to Be Addressed, GAO-05-702
(Washington, D.C.: July 22, 2005).
[47] GAO, Office of Personnel Management: Retirement Systems
Modernization Program Faces Numerous Challenges, GAO-05-237
(Washington, D.C.: Feb. 28, 2005).
[48] GAO-04-798T.
[49] OMB Circular No. A-11, Preparation, Submission, and Execution of
the Budget, Section 53 (Washington, D.C.: June 21, 2005).
[50] Exhibit 53 lists all of the IT projects and their associated costs
within a federal organization and are to be prepared each year as part
of the budget process in accordance with OMB Circular No. A-11.
[51] GAO-04-798T.
[52] In March 2004, OMB initiated a governmentwide analysis of five
lines of business--financial management, human resources management,
grants management, federal health architecture, and case management--
and in March 2005 started a task force to address a sixth line of
business on IT security.
[53] Hosting refers to a service provider who manages and provides
availability to a Web site or application, often bound by a service-
level agreement. The hosting entity generally maintains servers with
network support, power backup, fault tolerance, load balancing, and
storage backup.
[54] OMB, Realignment of Responsibilities for Federal Financial
Management Policy and Oversight, Memorandum (Washington, D.C.: Dec. 2,
2004).
[55] Subsequent to our review, OMB issued Update on the Financial
Management Line of Business and the Financial Systems Integration
Office, Memorandum (Washington, D.C.: Dec. 16, 2005) which updated the
status of the JFMIP realignment to FSIO. For example, responsibilities
for issuing certain system requirements that had been reassigned to OMB
were transitioned to the Chief Acquisition Council, the Budget Officers
Advisory Council, and the Federal Real Property Council.
[56] See OMB, Update on the Financial Management Line of Business and
the Financial Systems Integration Office, Memorandum (Washington, D.C.:
Dec. 16, 2005).
[57] IEEE Std. 1362-1998.
[58] Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1854 (Oct. 28, 2004)
(codified at 10 U.S.C. § 2222(j)(2)). The act defines a defense
business system as an information system, other than a national
security system, operated by, for, or on behalf of the department that
is used to support business activities, such as acquisition, financial
management, logistics, strategic planning and budgeting, installations
and environment, and human resources management. The act states that
such systems are to include financial systems, mixed systems, financial
data feeder systems, and IT and information assurance infrastructure.
[59] GAO/AIMD-00-134.
[60] See 40 U.S.C. § 11303(b)(2)(C).
[61] Subsequent to our review, the responsibility for developing
standard business processes was assigned to the FSIO according to the
December 16, 2005, OMB memorandum to CFOs.
[62] For example, see GAO, Indian Trust Funds: Improvements Made in
Acquisition of New Asset and Accounting System But Significant Risks
Remain, GAO/AIMD-00-259 (Washington, D.C.: Sept. 15, 2000); GAO-05-237;
and GAO, District of Columbia: Weaknesses in Financial Management
System Implementation, GAO-01-489 (Washington, D.C.: Apr. 30, 2001).
[63] The payroll providers selected are Defense Finance and Accounting
Service, the General Services Administration, the Department of
Agriculture's National Finance Center, and Interior's National Business
Center.
[64] Subsequent to our review, OMB officials told us that as part of
their oversight for the Justice project, Justice has agreed to consider
an application service provider solution and does not plan on applying
to be a designated center of excellence.
[65] Department of Justice Office of the Inspector General, Top
Management Challenges, Memorandum (Washington, D.C.: Oct. 13, 2004).
[66] GAO-01-1007T.
[67] JFMIP and the CFO Council issued this report in April 2002 that
reviewed human capital challenges related to implementing financial
management systems and identified strategies to meet the challenges.
[68] GAO/AIMD-00-134.
[69] Association of Government Accountants, Financial Management Shared
Services: A Guide for Federal Users (Alexandria, Va.: July 2005).
[70] Subsequent to our review, the December 16, 2005, OMB memorandum to
CFOs stated that the CFO Council's Financial Systems Integration
Committee was still evaluating its current subcommittee structure to
assess whether changes are needed to best meet its objectives.
[71] See OMB, Information Technology Project Manager Qualification
Guidance, M-04-19 (Washington, D.C.: July 21, 2004) and OMB Circular
No. A-11, Section 300.
[72] According to IEEE Std. 1362-1998, a concept of operations document
is normally one of the first documents produced during a disciplined
development effort since it describes system characteristics for a
proposed system from the user's viewpoint. This is important since a
good concept of operations document can be used to communicate overall
quantitative and qualitative system characteristics to the user,
developer, and other organizational elements. This allows the reader to
understand the user organizations, missions, and organizational
objectives from an integrated systems point of view.
[73] IEEE Std. 830-1998.
[74] Barry W. Boehm and Philip N. Papaccio, "Understanding and
Controlling Software Costs," IEEE Transactions on Software Engineering,
vol. 14, no. 10 (1988).
[75] The Standish Group, Charting the Seas of Information Technology
(Dennis, Mass.: The Standish Group, 1994).
[76] Caper Jones, Assessment and Control of Software Risks (Englewood
Cliffs, N.J.: Yourdon Press, 1994).
[77] Dean Leffingwell, "Calculating the Return on Investment from More
Effective Requirements Management," American Programmer (1997).
[78] Glenford J. Myers, The Art of Software Testing (N.Y.: John Wiley &
Sons, Inc., 1979).
[79] Testing covers a variety of activities. The discussion of the
testing processes in this appendix has been tailored to selected
aspects of system implementation efforts and is not intended to provide
a comprehensive discussion of all the processes that are required or
the techniques that can be used to accomplish a disciplined testing
process.
[80] Glendford J. Myers, The Art of Software Testing.
[81] Joint Financial Management Improvement Program, White Paper:
Financial Systems Data Conversion-Considerations (Washington, D.C.:
Dec. 20, 2002).
[82] IEEE Std. 610-1990.
[83] Donald J. Reifer, Victor R. Basili, Barry W. Boehm, and Betsy
Clark, "COTS-Based Systems--Twelve Lessons Learned about Maintenance."
(Presentation, 3rd International Conference on COTS-Based Software
Systems, Redondo Beach, Calif., Feb. 4, 2004.)
[84] Steve McConnell, Software Project Survival Guide (Redmond, Wash.:
Microsoft Press, 1998).
[85] GAO, Information Technology: DOD's Acquisition Policies and
Guidance Need to Incorporate Additional Best Practices and Controls,
GAO-04-722 (Washington, D.C.: July 30, 2004).
[86] According to Office of Management and Budget Circular No. A-11,
earned value management is a project (investment) management tool that
effectively integrates the investment scope of work with schedule and
cost elements for optimum investment planning and control. Agencies
must demonstrate use of an earned value management system that meets
American National Standards Institute/Electronic Industries Alliance
Standard 748, for both government and contractor costs, for those parts
of the total investment that require development efforts (e.g.,
prototypes and testing in the planning phase and development efforts in
the acquisition phase) and show how close the investment is to meeting
the approved cost, schedule, and performance goals. In addition,
agencies must provide an explanation for any cost or schedule variances
that are more than plus or minus 10 percent.
[87] Steve McConnell, Software Project Survival Guide.
[88] Steve McConnell, Rapid Development: Taming Wild Software Schedules
(Redmond, Wash.: Microsoft Press, 1996).
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
