Business Modernization
Disciplined Processes Needed to Better Manage NASA's Integrated Financial Management Program Gao ID: GAO-04-118 November 21, 2003The National Aeronautics and Space Administration (NASA) has struggled to implement a fully integrated financial management system. The lack of such a system has affected the agency's ability to control program costs, raising concerns about the management of its most costly programs, including the space shuttle program and the International Space Station. In April 2000 NASA initiated the Integrated Financial Management Program (IFMP)--its third effort to improve the agencywide management of its resources. Implementation is expected by fiscal year 2006 with an estimated life-cycle cost of nearly $1 billion. This report (1) assesses NASA's methodology for preparing the current life-cycle cost estimate for implementing IFMP, (2) determines whether NASA's current schedule is reasonable, and (3) evaluates NASA's processes for ensuring adequate cost contingencies.
The uncertain reliability of cost estimates, optimistic schedules, and insufficient processes for ensuring adequate funding reserves have put NASA's latest financial management modernization effort at risk. Over the past several years, IFMP's life-cycle cost estimates have fluctuated, and NASA's current estimate is 14 percent greater than the previous estimate. The reliability of these estimates is uncertain because disciplined costestimating processes required by NASA and recognized as best practices were not used in preparing them. For example, IFMP's current life-cycle cost estimate did not include the full cost likely to be incurred during the life of the program, including certain operations costs and costs to retire the system. In addition, NASA did not consistently use breakdowns of work in preparing the cost estimate, as recommended by NASA guidance. In cases where work breakdowns were used, the agency did not always show the connection between the work breakdown estimates and the official program cost estimate. This has been a weakness since the inception. Although more than half of the IFMP modules have been implemented--including the Core Financial module, which is considered the backbone of IFMP--the system may not be fully implemented by the end of fiscal year 2006 as planned. Efforts to complete the integrated system as quickly as possible might have resulted in schedule margins that are insufficient to manage program challenges--such as personnel shortages, uncertainties about software availability, and Office of Management and Budget (OMB) initiatives to implement electronic systems for agency business processes governmentwide. These OMB initiatives have put IFMP in a reactive mode and are already affecting planning for the payroll, procurement, and travel components of the integrated system, which could result in additional schedule delays and cost growth. Finally, reserve funding for IFMP contingencies may be insufficient, which is particularly problematic, given the program's unreliable cost estimates and optimistic schedule. One module--Budget Formulation--is already experiencing potential shortfalls in its reserves, and project officials expressed concerns that the module's functionality may have to be reduced. Yet the program continues to establish funding reserves based on reserve levels set by other high-risk NASA programs, such as NASA's space flight program--not on analyses of the potential cost impact of risks and unknowns specific to IFMP, as required by NASA guidance. Moreover, the program did not quantify the cost impact of high-criticality risks--also required by NASA--or link its risks to funding reserves to help IFMP develop realistic budget estimates.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-118, Business Modernization: Disciplined Processes Needed to Better Manage NASA's Integrated Financial Management Program
This is the accessible text file for GAO report number GAO-04-118
entitled 'Business Modernization: Disciplined Processes Needed to
Better Manage NASA's Integrated Financial Management Program' which was
released on December 22, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Committee on Commerce, Science, and Transportation,
U.S. Senate, and the Committee on Science, House of Representatives:
United States General Accounting Office:
GAO:
November 2003:
Business Modernization:
Disciplined Processes Needed to Better Manage NASA's Integrated
Financial Management Program:
Business Modernization:
GAO-04-118:
GAO Highlights:
Highlights of GAO-04-118, a report to the Senate Committee on
Commerce, Science, and Transportation, and the House Committee on
Science
Why GAO Did This Study:
The National Aeronautics and Space Administration (NASA) has struggled
to implement a fully integrated financial management system. The lack
of such a system has affected the agency‘s ability to control program
costs, raising concerns about the management of its most costly
programs, including the space shuttle program and the International
Space Station.
In April 2000 NASA initiated the Integrated Financial Management
Program (IFMP)”its third effort to improve the agencywide management
of its resources. Implementation is expected by fiscal year 2006 with
an estimated life-cycle cost of nearly $1 billion.
This report (1) assesses NASA‘s methodology for preparing the current
life-cycle cost estimate for implementing IFMP, (2) determines
whether NASA‘s current schedule is reasonable, and (3) evaluates
NASA‘s processes for ensuring adequate cost contingencies.
What GAO Found:
The uncertain reliability of cost estimates, optimistic schedules,
and insufficient processes for ensuring adequate funding reserves
have put NASA‘s latest financial management modernization effort at
risk. Over the past several years, IFMP‘s life-cycle cost estimates
have fluctuated, and NASA‘s current estimate is 14 percent greater
than the previous estimate. The reliability of these estimates is
uncertain because disciplined cost-estimating processes required by
NASA and recognized as best practices were not used in preparing
them. For example, IFMP‘s current life-cycle cost estimate did not
include the full cost likely to be incurred during the life of the
program, including certain operations costs and costs to retire the
system. In addition, NASA did not consistently use breakdowns of work
in preparing the cost estimate, as recommended by NASA guidance. In
cases where work breakdowns were used, the agency did not always show
the connection between the work breakdown estimates and the official
program cost estimate. This has been a weakness since the inception.
Although more than half of the IFMP modules have been implemented”
including the Core Financial module, which is considered the backbone
of IFMP”the system may not be fully implemented by the end of fiscal
year 2006 as planned. Efforts to complete the integrated system as
quickly as possible might have resulted in schedule margins that are
insufficient to manage program challenges”such as personnel
shortages, uncertainties about software availability, and Office of
Management and Budget (OMB) initiatives to implement electronic
systems for agency business processes governmentwide. These OMB
initiatives have put IFMP in a reactive mode and are already
affecting planning for the payroll, procurement, and travel
components of the integrated system, which could result in additional
schedule delays and cost growth.
Finally, reserve funding for IFMP contingencies may be insufficient,
which is particularly problematic, given the program‘s unreliable
cost estimates and optimistic schedule. One module”Budget Formulation”
is already experiencing potential shortfalls in its reserves, and
project officials expressed concerns that the module‘s functionality
may have to be reduced. Yet the program continues to establish
funding reserves based on reserve levels set by other high-risk NASA
programs, such as NASA‘s space flight program”not on analyses of the
potential cost impact of risks and unknowns specific to IFMP, as
required by NASA guidance. Moreover, the program did not quantify the
cost impact of high-criticality risks”also required by NASA”or link
its risks to funding reserves to help IFMP develop realistic budget
estimates.
What GAO Recommends:
GAO is recommending that IFMP follow best practices and NASA guidance
in preparing and updating the life-cycle cost estimate and establish
additional processes that would enable the agency to more accurately
estimate program cost and predict the impact of possible undesired
events. NASA concurred with GAO‘s recommendations for corrective
action.
www.gao.gov/cgi-bin/getrpt?GAO-04-118.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact Allen Li at (202)
512-4841 or lia@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Reliability of IFMP's Current Life-Cycle Cost Estimate Is Uncertain
Owing to a Lack of Disciplined Processes:
Program Schedule May Be Optimistic:
Processes Insufficient to Ensure Adequate Funding Set Aside
for Contingencies:
Conclusion:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Scope and Methodology:
Appendix I: Comments from the National Aeronautics and
Space Administration:
Appendix II: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: IFMP's System Modules and Their Status:
Table 2: IFMP's Cost Estimates for Life Cycle:
Table 3: IFMP's and NSSC's Share of Life-Cycle Costs in the May 2003
Budget for Fiscal Years 1999-2010:
Figure:
Figure 1: IFMP's Life-Cycle Cost Estimate Trend:
Abbreviations:
GAO: General Accounting Office:
IFMP: Integrated Financial Management Program:
NASA: National Aeronautics and Space Administration:
NSSC: NASA Shared Services Center:
OMB: Office of Management and Budget:
SAP: System Applications and Products:
SEI: Software Engineering Institute:
WBS: Work Breakdown Structure:
United States General Accounting Office:
Washington, DC 20548:
November 21, 2003:
The Honorable John McCain:
Chairman:
The Honorable Ernest F. Hollings:
Ranking Minority Member:
Committee on Commerce, Science, and Transportation:
United States Senate:
The Honorable Sherwood L. Boehlert:
Chairman:
The Honorable Ralph M. Hall:
Ranking Minority Member:
Committee on Science:
House of Representatives:
About 90 percent of the National Aeronautics and Space Administration's
(NASA) annual budget is spent on contractors. Yet since 1990 we have
identified NASA's contract management as a high-risk area--in part,
because the agency has failed to implement a modern, fully integrated
financial management system. As we reported in January 2003, the lack
of such a system has hampered NASA's ability to oversee contracts;
control program costs; and ensure an effective human capital management
strategy, raising serious concerns about NASA's management of its
largest and most costly programs, including the space shuttle program
and the International Space Station.[Footnote 1]
In April 2000 NASA initiated its third and most recent effort to
implement a modernized financial management system: the Integrated
Financial Management Program (IFMP).[Footnote 2] Through IFMP, NASA
plans to employ multiple software applications to improve the
agencywide management of its financial, physical, and human resources.
NASA expects to complete IFMP's implementation in fiscal year 2006 with
an estimated life-cycle cost totaling nearly $1 billion, including
operations and maintenance costs. However, several of the system's
modules remain to be developed, and the program faces significant
challenges in completing them as planned. Given its previous failures,
NASA is under tremendous pressure to ensure that IFMP is implemented
quickly and can achieve its goals.
In April 2003 we issued an interim report on IFMP,[Footnote 3] which
found that NASA was not following key best practices for acquiring and
implementing IFMP. As agreed, we continued our review in three areas:
(1) whether NASA is acquiring and implementing IFMP in the context of
an enterprise architecture, (2) the extent to which the Core Financial
module will address NASA's external reporting requirements, and
(3) NASA's life-cycle cost estimate and schedule for IFMP. We are
responding to the first two issues in separate reports.[Footnote 4]
This report addresses the third issue--IFMP's life-cycle cost estimate
and schedule. Specifically, you asked that we (1) assess the
reliability of NASA's methodology for preparing the current cost
estimate for implementing IFMP, (2) determine whether NASA's current
schedule is reasonable in terms of progress to date and available
resources, and (3) evaluate NASA's processes for ensuring the adequacy
of reserve funding for contingencies to mitigate the potential impact
of identified program risks and unknowns. In addition, we have
summarized our findings on the three areas previously cited in a
separate report.[Footnote 5]
Results in Brief:
The uncertain reliability of cost estimates, optimistic schedules, and
insufficient processes for ensuring the adequacy of funding reserves
have put NASA's latest financial management modernization effort at
risk of schedule delays and cost growth. The reliability of the current
estimate--which is 14 percent greater than the previous estimate
established in February 2002--is uncertain because disciplined cost-
estimating processes required by NASA and recognized as best practices
were not used in preparing the estimate. Specifically, IFMP's life-
cycle cost[Footnote 6] estimate did not include the full cost of all
resources likely to be incurred during the life of the program. In
addition, NASA did not consistently use breakdowns of the work to be
performed in preparing the cost estimate, as recommended by NASA
guidance. Without using the work breakdowns to prepare the cost
estimate, NASA cannot ensure that all costs are accounted for. In cases
where work breakdowns were used, the agency did not always provide a
clear audit trail between the work breakdown estimates and the
program's life-cycle cost estimate.
IFMP is scheduled for completion at the end of fiscal year 2006.
However, efforts to complete the integrated system as quickly as
possible might have resulted in schedule margins that are insufficient
to manage program challenges--such as personnel shortages and
uncertainties about software's availability. In addition, Office of
Management and Budget (OMB) initiatives to implement electronic systems
for agency business processes---which are expected to improve agency
operations governmentwide--are affecting planning for the payroll,
procurement, and travel modules of the integrated system, which could
result in additional schedule delays and cost growth.
Finally, reserve funding for IFMP contingencies may be insufficient--
which is particularly problematic, given the uncertain reliability of
cost estimates and optimistic schedule for the program--because the
program did not consistently perform in-depth analyses of the potential
cost impact of risks and unknowns specific to IFMP, as required by
NASA's guidance. Instead, the program established funding reserves on
the basis of reserve levels set by other high-risk NASA programs.
Moreover, the program did not quantify the cost impact of identified
risks or link its risks to funding reserves.
To help decision makers better assess all costs associated with
operating and implementing IFMP, we are recommending that the program
use current processes dictated by best practices and NASA guidance for
preparing and updating the life-cycle cost estimate as well as
establish additional disciplined processes to better ensure that the
agency more accurately estimate program cost, predict the impact of
possible undesired events, and plan accordingly.
In written comments on a draft of this report, NASA's Deputy
Administrator concurred with our recommendations for corrective action
but noted that all IFMP projects completed to date stayed within
budget, were completed ahead of schedule, and delivered the committed
scope. We do not agree that this is the case for the Core Financial
module, which NASA describes as the "backbone" of the program. When
NASA announced the implementation of the Core Financial module at all
of its centers, only about two-thirds of the financial events needed
for day-to-day financial operations and external reporting had been
implemented. In addition, we found that NASA deferred the
implementation of other key Core Financial module capabilities and
created new problems in recording certain financial
transactions.[Footnote 7] Thus, full functionality of the system has
been deferred, increasing the risk of additional costs and potentially
affecting the implementation of future modules. NASA's detailed
comments also noted that the program used a business case methodology
and professional cost estimators to perform cost and risk assessments.
As discussed in this report, the audit trail from these assessments to
the program's cost estimates required by NASA guidance and best
practices is not clear. NASA's detailed comments are included as
appendix I.
Background:
For more than a decade, we have reported that the lack of a modern
integrated financial management system to produce accurate and reliable
information has hampered NASA's ability to oversee contracts and
develop good cost estimates for NASA's programs. In 1990 NASA's lack of
effective systems and processes for overseeing contractor's activities
prompted us to identify NASA's contract management as a high-risk area.
In July 2002 we reported that the accuracy of NASA's $5 billion cost
growth estimate for the International Space Station was questionable
and that the agency might have difficulty preparing a reliable life-
cycle cost estimate because a modern integrated financial management
system was not available to track and maintain the data needed for
estimating and controlling costs. [Footnote 8] NASA's lack of a fully
integrated financial management system has also hurt the agency's
ability to collect, maintain, and report the full cost of its projects
and programs. For example, in March 2002 we testified that NASA was
unable to provide us with detailed support for the amounts that it
reported to the Congress as obligated against space station and related
shuttle program cost limits as required by the National Aeronautics and
Space Administration Authorization Act of 2000.[Footnote 9]
IFMP is designed as an integrated system to replace the separate
and incompatible financial management systems used by NASA's
10 centers.[Footnote 10] According to the IFMP Program Director, the
new system will provide better decision data, consistent information
across centers, and improved functionality.
Unlike NASA's previous efforts to modernize its financial management
system, IFMP does not rely on a single contractor. NASA selected System
Applications and Products (SAP) to provide its "best of suite"
software[Footnote 11] and contracted for implementation services under
a separate contract. NASA has also broken the project into modules that
will be implemented individually--instead of all at once--on the basis
of the availability of proven commercial-off-the-shelf software
products. IFMP initially segmented implementation into 14 modules but
has since reorganized the program into 9. Some of these modules may be
further broken out and others added, depending on the scope of OMB's e-
Government initiatives[Footnote 12] and other considerations. Table 1
describes the modules that currently comprise the system and their
status.
Table 1: IFMP's System Modules and Their Status:
Module: Position Description Management; Description: Web-based tool
that enables the generation and classification of job descriptions and
automatic generation of associated documents; NASA-reported status:
Implemented September 2002.
Module: Resume Management; Description: Web-based application that
allows applicants to apply for jobs online; NASA-reported status:
Implemented November 2001.
Module: Erasmus; Description: Web-based system providing financial
performance information on NASA's programs and projects in a
standardized format; NASA-reported status: Implemented November
2002.
Module: Travel Management; Description: Comprehensive system to
streamline and unify the NASA employee travel system and to improve
traveler and vendor reimbursement; NASA-reported status: Implemented
April 2003.
Module: Core Financial; Description: Accounting and financial reporting
system that serves as the "backbone" to the integrated system; NASA-
reported status: Implemented June 2003.
Module: Budget Formulation; Description: Web-based tool to formulate
project, program, institutional, enterprise, and agency-level budget
requirements; NASA-reported status: Planned completion of February
2004.
Module: Integrated Asset Management; Description: System to manage
NASA's physical assets through functions such as physical inventory and
financial reporting; NASA-reported status: To begin in late 2003.
Module: Contract Administration; Description: Comprehensive tool to
support procurement, receiving, invoicing, and payment of materials for
NASA; NASA-reported status: To begin in late 2004.
Module: Human Resources Management; Description: System allowing
managers to fill positions with staff that possess the appropriate
skill sets and career goals; NASA-reported status: To begin in late
2004.
Source: NASA.
[End of table]
When NASA announced in June 2003 that the Core Financial module had
been implemented at all of its centers, only about two-thirds of the
financial events needed for day-to-day financial operations and
external reporting had been implemented. In addition, we found that
NASA deferred implementation of other key core financial module
capabilities and created new problems in recording certain financial
transactions.[Footnote 13] Thus, full functionality of the system has
been deferred, increasing the risk of additional costs and potentially
affecting the implementation of future modules.
As we reported in April 2003, NASA is not following key best practices
for acquiring and implementing IFMP. For example, NASA has not analyzed
the interdependencies between selected and proposed IFMP components,
and it does not have a methodology for doing so. By acquiring IFMP
components without first understanding system component relationships,
NASA has increased its risk of implementing a system that will not
optimize mission performance and will cost more and take longer to
implement than necessary. In addition, in implementing the Core
Financial module, NASA faces risks in the areas of user needs and
requirements management because the agency did not consider the
information needs of key system users and is relying on a requirements
management process that does not require the documentation of detailed
system requirements prior to system implementation and testing.
Reliability of IFMP's Current Life-Cycle Cost Estimate Is Uncertain
Owing to a Lack of Disciplined Processes:
The reliability of the current life-cycle cost estimate--which has
fluctuated since the initial estimate and is 14 percent greater than
the previous estimate established in February 2002--is uncertain
because disciplined cost-estimating processes required by NASA and
recognized as best practices were not used in preparing the estimate.
Specifically, IFMP's life-cycle cost estimate did not include the full
cost likely to be incurred during the life of the program. In addition,
breakdowns of work to be performed--or Work Breakdown Structure
(WBS)[Footnote 14]--were not consistently used in preparing the cost
estimate. In cases where work breakdowns were used to prepare the
estimate, the agency did not always provide a clear audit trail. NASA
has made some improvements in the program's financial management, such
as hiring personnel to provide oversight and consistency for the cost-
estimating process. However, until NASA uses more disciplined processes
such as breakdowns of work in preparing the program's cost estimate,
the reliability of the life-cycle cost estimate will be uncertain and
the program will have difficulty with controlling costs.
IFMP's Life-Cycle Cost Estimates Show Overall Increase:
Since the program began, cost estimates for IFMP's 10-year life cycle-
-fiscal years 2001 through 2010--have fluctuated and increased overall,
as shown in figure 1.
Figure 1: IFMP's Life-Cycle Cost Estimate Trend:
[See PDF for image]
[End of figure]
NASA's current IFMP life-cycle cost estimate totals $982.7 million--an
increase of $121.8 million, or 14 percent, over the previous IFMP life-
cycle cost estimate. The estimate comprises IFMP direct program costs,
NASA's enterprise support,[Footnote 15] and civil service salaries/
benefits. (See table 2.):
Table 2: IFMP's Cost Estimates for Life Cycle:
Dollars in millions of then-year dollars.
Direct program; Feb. 2002 estimate: $644.8; May 2003 estimate: $635.3;
Change in dollars: $(9.5); Percent change: (1.5).
Enterprise support; Feb. 2002 estimate: 164.8; May 2003 estimate:
189.4; Change in dollars: 24.6; Percent change: 15.0.
Civil Service salaries/benefits; Feb. 2002 estimate: 51.3; May 2003
estimate: 158.0; Change in dollars: 106.7; Percent change: 208.0.
Total life-cycle cost; Feb. 2002 estimate: $860.9; May 2003 estimate:
$982.7; Change in dollars: $121.8; Percent change: 14.1.
Sources: NASA (data); GAO (analysis).
[End of table]
Although direct program costs decreased by $9.5 million, these costs
were shifted to the enterprise support component of the estimate with
the program's decision to fund only 1 year's worth of operations and
maintenance, rather than 2 years' worth from the direct program budget.
In addition, NASA anticipates that operations costs for fiscal years
2007 through 2010--estimated at $137.8 million--will be funded by the
NASA Shared Services Center (NSSC), a planned initiative to consolidate
various agency services such as purchasing and human resources. (See
table 3.) As a result, the fiscal year 2004 budget for the IFMP direct
program portion of implementing the system is $497.5 million.
Table 3: IFMP's and NSSC's Share of Life-Cycle Costs in the May 2003
Budget for Fiscal Years 1999-2010:
IFMP estimate component: Direct program; Development and implementation
(IFMP's share, FY 1999-2006): $497.5; Operations and maintenance
(NSSC's share, FY 2007-2010): $137.8; Total: $635.3.
IFMP estimate component: Enterprise support; Development and
implementation (IFMP's share, FY 1999-2006): 133.2; Operations and
maintenance (NSSC's share, FY 2007-2010): 56.2; Total: 189.4.
IFMP estimate component: Civil Service salaries/benefits; Development
and implementation (IFMP's share, FY 1999-2006): 125.3; Operations and
maintenance (NSSC's share, FY 2007-2010): 32.7[A]; Total: 158.0.
IFMP estimate component: Total life-cycle cost; Development and
implementation (IFMP's share, FY 1999-2006): $756.0; Operations and
maintenance (NSSC's share, FY 2007-2010): $226.7; Total: $982.7.
Source: NASA.
[A] This number is not final and is still being reviewed by NASA.
[End of table]
In March 2003 an independent cost estimate team concluded that there is
an 85 percent confidence level that the direct program portion can be
successfully completed with the available funding of $497.5 million.
However, the direct program portion represents only about half of the
total life-cycle cost estimate. In addition, the team's conclusion was
contingent on two optimistic assumptions: that there would be no
schedule disruptions and no increase in requirements.
Disciplined Processes Required by NASA Were Not Used in Preparing
IFMP's Cost Estimates:
Reflecting OMB guidance[Footnote 16] and the best practices of
government and industry leaders, NASA requires that life-cycle cost
estimates be prepared on a full-cost basis, that estimates be
summarized according to the current breakdown of work to be performed,
and that major changes be tracked to the life-cycle cost. OMB guidance
calls for a disciplined budget process to ensure that performance goals
are met with the least risk and the lowest life-cycle cost, which
includes direct and indirect costs, operations and maintenance, and
disposal. The Software Engineering Institute (SEI)[Footnote 17] echoes
the need for reliable cost-estimating processes in managing software
implementations--identifying tasks to be estimated, mapping the
estimates to the breakdown of work to be performed, and having a clear
audit trail are among SEI's requisites for producing reliable cost
estimates.
Despite NASA requirements and OMB and SEI guidance, IFMP did not
prepare a full life-cycle cost estimate--that is, all direct and
indirect costs for planning, procurement, operations and maintenance,
and disposal were not included. For example, the life-cycle cost
estimate does not include the following:
* the cost to operate and maintain the system beyond 2010;[Footnote 18]
* the cost of retiring the system;
* enterprise travel costs, which are provided monthly by the NASA
centers; and:
* the cost of nonleased NASA facilities for housing IFMP.
In addition, IFMP did not prepare WBS estimates for active modules--
that is, those currently being implemented. According to NASA guidance,
breaking down work into smaller units helps facilitate cost estimating
and project and contract management, and helps ensure that relevant
costs are not omitted. The guidance also states that the WBS should
encompass both in-house and contractor efforts. According to the IFMP
Deputy Program Director, WBS estimates are not prepared for active
modules because information such as contract task orders can be used to
prepare the cost estimates. However, there is not one overriding
contract where each module is considered a deliverable at a fixed
price. Rather, numerous contracts at both the project and center level
for a module's implementation--many of which can be awarded for a level
of effort at agreed-upon fixed rates at various phases in the
implementation. Without a WBS estimate for the project as a whole, NASA
cannot ensure that all relevant contractor costs are included in the
cost estimate. In addition, using contract task orders to prepare the
cost estimate would not ensure that government in-house costs are
included in the life-cycle cost estimate.
Finally, for modules in the planning phase, the program utilized NASA's
subject matter experts and professional cost estimators to prepare
business case analyses. However, although these analyses contained WBS
cost estimates, the audit trail from the WBS estimate to the program's
life-cycle cost estimate was not always clear. Without a clear audit
trail, it is difficult to determine whether the differences between the
detailed WBS estimates and the official program cost estimate are
appropriate. The lack of a clear audit trail has been a weakness since
the inception of the program. For example, IFMP was unable to provide
us with traceable support for its baseline cost estimate for direct
program costs.
NASA has made some improvements that should help the program prepare
better cost estimates. In May 2002 the NASA Administrator appointed an
executive to provide leadership and accountability in the direction and
operation of the system. The NASA headquarters program office also
hired a business manager to oversee and provide consistency for the
cost-estimating process and provide an analyst to review enterprise
support costs.
Program Schedule May Be Optimistic:
Although NASA guidance requires sufficient program schedule margins
to manage risks, efforts to complete the integrated system as quickly
as possible might have resulted in a schedule that is too compressed to
accommodate program challenges, such as personnel shortages and
uncertainties about software's availability. If the program schedule
margin is too compressed, the program could incur additional risks,
including added cost growth as well as failure to meet IFMP's schedule
objectives. OMB's e-Government initiatives--which aim to streamline
agency business processes and eliminate redundant systems
governmentwide--could also provide challenges for NASA's IFMP planning.
As a result, the program schedule may be optimistic.
While implementing the Core Financial module (see table 1), IFMP has
faced human resource challenges, and the program continues to face
these challenges with other modules. For example, personnel shortages
at Marshall Space Flight Center for several months affected the Core
Financial project and other projects. In this case, a schedule slip was
avoided, but during fiscal year 2002, the shortages resulted in nearly
$400,000 for extra hours worked by center employees. Human resource
challenges are also affecting the Budget Formulation module. The
simultaneous implementation of this module with the Core Financial
module--an action advised against by a contractor conducting a
lessons-learned study--placed heavy demand on already scarce resources
and added complexity to the program. As a result the schedule for
implementing the Budget Formulation module has already slipped.
Sometimes, relying more on contractor personnel can alleviate
shortfalls in civil service personnel, but a recent Budget Formulation
project status report indicated that the implementation contractor
might also have difficulties acquiring and/or retaining qualified
personnel. The implementation schedules for the remaining modules
overlap, putting the program at further risk of schedule slippages.
Uncertainty regarding software availability also puts the program at
risk for completing the integrated system on schedule. For example,
complete software solutions and requirements for IFMP's Contract
Administration module have not yet been determined. Although contract-
document-generation software is available and tailored to meet the
unique interface and reporting requirements of the federal
government,[Footnote 19] the "best of suite" software solution--SAP--
does not currently meet these requirements. NASA faces the same
challenge with IFMP's Human Resources Management module. NASA's monthly
status reports show that the program is working with SAP to develop a
software solution for the Human Resources Management module that will
meet federal government requirements, but the outcome is uncertain. In
addition, the program could adopt an e-Government solution for its
Human Resources Management module rather than the SAP solution.
Inserting e-Government solutions into IFMP planning--which calls
for using "best of suite" software--could create more difficult
interface development and a less-integrated system, thus interrupting
the program's cost and schedule. E-Government initiatives are already
affecting NASA's planning for the payroll, procurement, and travel
modules in the integrated system. For example, the payroll function,
which was once part of the Human Resources Management module, will
likely become a separate module under e-Government. Similarly, the
Contract Administration module has been split into two components: one
for procurement document generation, for which software is available
although requirements are not finalized, and one for the remainder of
NASA's Contract Administration requirements, for which requirements and
software are currently unknown. Furthermore, e-Travel could replace the
Travel Management module, which has already been implemented.
According to the program's fiscal year 2002 Independent Annual Review,
e-Government initiatives are forcing the program into a reactionary
mode, thus increasing risk to the program's success. The review
specifically noted that (1) the benefits of a fully integrated system
could be lost under e-Government, (2) the scope of IFMP and timing of
future projects' implementation have become uncertain, and (3) cost
increases and schedule slippage to accommodate directives may occur.
Processes Insufficient to Ensure Adequate Funding Set Aside
for Contingencies:
In addition to the uncertain reliability of IFMP's life-cycle cost
estimates and optimistic schedules, NASA cannot ensure that the funding
set aside for program contingencies is sufficient because the program
did not consistently perform in-depth analyses of the potential cost
impact of risks and unknowns specific to IFMP, as required by NASA
guidance. Moreover, the program did not quantify the cost impact of
identified risks, link its risks to funding reserves, or consistently
set aside cost contingencies for these risks.
In-Depth Analysis Not Performed in Establishing Cost Contingencies:
NASA guidance stipulates that programs incorporate financial reserves,
schedule margins, and technical performance margins to provide the
flexibility needed to manage risks. According to the guidance,
financial reserves are to be established and maintained commensurate
with programmatic, technical, cost, and schedule risks. In other words,
cost contingencies should be tailored to the specific risks associated
with a particular program or project. In addition, NASA guidance
suggests that tools such as Probabilistic Risk Assessment[Footnote 20]
can help in analyzing risk.
Although NASA's business case analyses include a risk assessment and
recommended reserve levels, we found no evidence that these recommended
levels were used in establishing the actual reserve levels for the IFMP
module projects. Regardless, the actual levels established did not
match the recommended levels in the business case analyses in most
cases. We found that reserves for some IFMP modules--both in the
planning and active phase--were based not on IFMP-specific risks but on
reserve levels for other high-risk NASA programs. For example, for a
number of IFMP modules, reserves were set at levels used for spacecraft
implementations--typically about 30 percent--because industry
experience showed that large cost overruns in system implementations
such as IFMP are common. Yet it is unclear whether this reserve margin
is adequate for IFMP because the effect of IFMP-specific risks and
assumptions--such as uncertainties relating to software, schedule, and
OMB's e-Government initiatives--were not analyzed. In addition, some
of the enterprises supporting the module projects described their
method of establishing funding reserves as a combination of rules of
thumb and guesswork.
The Budget Formulation module has already experienced shortfalls in
its reserves, and project officials expressed concerns that the
module's functionality may have to be reduced. As of April 2003, the
module had expended its baseline reserves, which were established at
about 20 percent on the basis of the level of risk for space flight
missions--not on the risks specific to the module. Although the project
was able to bring its budget back into balance by obtaining an
agreement with SAP to limit overtime pay to time in excess of 50 hours
per week, its remaining reserves total only $83,000 to cover all
contingencies--including those that could require changes to the Budget
Formulation module.
Cost Impact of Identified Risks Not Quantified or Linked to
Cost Contingencies:
NASA requires programs to quantify the cost impact of high-criticality
risks[Footnote 21] and to determine to what extent reserves may be
exhausted, should the risks become reality. According to SEI,
estimating the potential cost and schedule impact for all identified
risks is an element of good estimating practice. Quantifying the cost
impact of identified risks and clearly and consistently linking the
risk database to funding reserves helps programs develop realistic
budget estimates.
While IFMP identifies program risks, analyzes their severity, and plans
mitigation actions, the program typically does not prepare a cost
impact analysis for identified risks nor does it consistently link
identified risks to funding reserves to ensure that funds are
available, should the risk occur. For example, in February 2003, the
Travel Management Project found that some components of the Travel
Management module might not satisfy individual centers, be funded, or
be technically feasible. However, the cost impact of this risk, as well
as others, was not quantified. Similarly, in June 2003, the Budget
Formulation module did not quantify the cost impact of a number of
identified risks.[Footnote 22] Without estimating the potential cost
impact of these risks, NASA cannot determine whether it has sufficient
reserves to cover the risks--which is particularly problematic for
Budget Formulation, since virtually no reserves remain for this module.
Furthermore, in its July 2003 monthly status report, the IFMP
headquarters office identified three high-criticality risks that could
have a cost impact on the overall program; however, no liens[Footnote
23] were set aside against reserves for these risks:
* Reductions to out-year budgets could affect the implementation of
future integrated modules or the ongoing evolution of existing modules.
* An e-Government solution may be adopted for human resources
management rather than the IFMP solution, resulting in more difficult
interface development and a less-than-integrated solution.
* E-Government initiatives and policy decisions could disrupt IFMP
modules, resulting in delays or additional resource impacts.
An independent cost estimate team identified and quantified the impact
of two IFMP program risks, indicating that the cost and schedule impact
of a risk on a program or project can be sizeable. First, the team
identified a high-probability risk that NASA's "full cost requirement"-
-in which all direct and indirect agency costs, including civil service
personnel costs, are tied to individual programs and projects--could
affect the Budget Formulation module.[Footnote 24] The team estimated
this risk at $2 million to $3 million, with a potential schedule slip
of 3 to 6 months. The Budget Formulation Project is currently trying to
determine what impact it may have. The second risk identified by the
independent cost review team--that the Core Financial module may be
transitioned to operations before all integration points are addressed-
-could be more costly. The team estimated this risk at $10.5 million to
$20 million, also with a potential 3-to 6-month schedule slip. However,
the team considered this risk as having a low probability of
occurrence.
Conclusion:
NASA is at a critical juncture and faces major challenges in improving
contract management and controlling costs. These challenges seriously
affect the agency's ability to effectively manage its largest and most
costly programs. A modern integrated financial management system, as
envisioned in IFMP, is critical to ensuring that NASA has accurate and
reliable information to successfully meet these challenges. NASA has
made some improvements during the past year, such as hiring personnel
to provide the cost-estimating process with oversight and consistency.
However, if IFMP continues to ignore disciplined processes in
estimating program costs and impacts, it is unlikely that the program
will meet its goals.
Recommendations for Executive Action:
To ensure that IFMP's life-cycle cost estimate conforms to NASA
guidance and best practices, we recommend that the NASA Administrator
direct IFMP to do the following:
* Prepare cost estimates by the current Work Breakdown Structure for
the remaining modules.
* Provide a clear audit trail between detailed WBS estimates and the
program's cost estimate for the remaining modules.
* Prepare a full life-cycle cost estimate for the entire IFMP that
meets NASA's life-cycle cost and full cost guidance.
To ensure that contingencies are funded in accordance with NASA
guidance and best practices, we recommend that the NASA Administrator
direct IFMP to do the following:
* Utilize a systematic, logical, and comprehensive tool, such as
Probabilistic Risk Assessment, in establishing the level of financial
reserves for the remaining module projects and tailor the analysis to
risks specific to IFMP.
* Quantify the cost impact of at least all risks with a high likelihood
of occurrence and a high magnitude of impact to facilitate the
continuing analysis necessary to maintain adequate reserve levels.
* Establish a clear link between the program's risk database and
financial reserves.
Agency Comments and Our Evaluation:
Although NASA concurred with our recommendations for corrective action,
NASA indicated that its current processes are adequate for (1)
preparing WBS cost estimates, (2) estimating life-cycle costs, and (3)
establishing reserves on the basis of IFMP-specific risks. The agency
cited its business case analyses as the methodology through which it is
accomplishing these tasks.
We disagree that NASA's current processes are adequate, and our
recommendations are aimed at improving these processes. As discussed in
this report, while NASA prepares WBS cost estimates for IFMP modules in
the planning phases by using business case analyses, it does not
prepare WBS cost estimates for active modules. And although IFMP
indicates that preparing cost estimates by using contract task orders
is an appropriate methodology, this approach will not ensure that all
relevant costs, including both contractor and government in-house
costs, are included in the life-cycle cost estimate. Regarding contract
costs, there is not one overriding contract where each module is
considered a deliverable at a fixed price. Rather, there are numerous
contracts at both the project and center level for implementing
modules--many of which can be awarded for a level of effort at agreed-
upon fixed rates at various phases in the implementation. Without a WBS
estimate for the project as a whole, NASA cannot ensure that all
relevant contractor costs are included in the cost estimate. In
addition, using contract task orders to prepare the cost estimate would
not ensure that government in-house costs are included in the life-
cycle cost estimate.
According to NASA, IFMP will improve its business case analyses by
providing better estimates of operational costs through the expected
life of the module, retirement costs, and other full life-cycle costs.
However, as discussed in this report, an audit trail is needed between
the detailed estimates contained in the business case analyses and the
program's life-cycle cost estimate to ensure that these improvements
are reflected in the program's official cost estimate.
Finally, as discussed in this report, although NASA's business case
analyses include recommended reserve levels, we found no evidence that
these recommended levels were used in establishing the actual reserve
levels for the IFMP module projects. Regardless, the actual levels
established did not match the recommended levels in most cases. We
found that the program established funding reserves on the basis of
reserve levels set by other high-risk NASA programs, rather than on
IFMP-specific risks as required by NASA guidance.
Scope and Methodology:
To assess the reliability of NASA's methodology for preparing the
current cost estimate for IFMP, we reviewed program and project-level
documentation to obtain an understanding of NASA's current cost
estimate and its major components and the methodology used to develop
the estimate. We also interviewed program and project officials to
clarify our understanding of the cost estimate and how NASA derived it.
In addition, we compared the program's cost-estimating methodology with
SEI best practices, OMB requirements, and NASA's own procedures and
guidance. Finally, we reviewed internal and independent analyses of the
cost estimate. We did not attempt to validate NASA's estimate; rather,
we reviewed NASA's processes for preparing its estimate.
To determine whether NASA's current schedule is reasonable in terms
of progress to date and available resources, we reviewed the program's
schedule objectives and NASA's policies for managing program and
project schedules. We monitored the schedule and risks to the schedule
through our review of the program's monthly status reports and internal
NASA briefings. We interviewed program and project officials to
ascertain NASA's progress against the schedule.
To evaluate NASA's processes for ensuring the adequacy of cost
contingencies to mitigate the potential impact of identified program
risks and unknowns, we reviewed governmentwide and NASA policies and
SEI best practices for managing risk and establishing cost
contingencies. We also interviewed program officials at NASA
headquarters and project managers to obtain an understanding of how
reserve levels were established and maintained for the program. We then
compared IFMP's processes for ensuring adequate cost contingencies with
processes dictated by OMB and NASA guidance and by best practices.
To accomplish our work, we visited NASA headquarters, Washington, D.C;
Marshall Space Flight Center, Alabama; and Goddard Space Flight Center,
Maryland. We also contacted officials at Glenn Research Center, Ohio.
We performed our review from April through September 2003 in accordance
with generally accepted government auditing standards.
As agreed with your offices, unless you announce its contents earlier,
we will not distribute this report further until 30 days from its date.
At that time, we will send copies to interested congressional
committees; the NASA Administrator; and the Director, Office of
Management and Budget. We will make copies available to others upon
request. In addition, the report will be available at no charge on the
GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this
report are acknowledged in appendix I.
Allen Li:
Director:
Acquisition and Sourcing Management:
Signed by Allen Li:
[End of section]
Appendix I: Comments from the National Aeronautics and Space
Administration:
National Aeronautics and Space Administration:
Office of the Administrator Washington, DC 20546-0001:
October 31, 2003:
Mr. Allen Li:
Director, Acquisition and Sourcing Management Team:
United States General Accounting Office
Washington, DC 20548:
Dear Mr. Li:
Thank you for the opportunity to review and comment on the draft report
entitled, BUSINESS MODERNIZATION: Disciplined Processes Needed to
Better Manage NASA's Integrated Financial Management Program (GAO-04-
118). We appreciate the General Accounting Office's (GAO) continued
interest in this vital program and desire to see this undertaking
successfully completed.
Enclosed are NASA's responses to the draft report. NASA concurs with
all six of the GAO's recommendations for corrective action. We
acknowledge that the Integrated Financial Management Program is engaged
in a high-risk endeavor and that we will continue to improve our
processes. We respectfully request that the GAO, in producing its final
report on this matter, also indicate that all five IFMP projects
completed to date stayed within budget, were completed ahead of
schedule, and delivered the committed scope. We also ask that the
report recognize that the program did use a business case methodology
and professional cost estimators to perform cost and risk assessments
prior to starting each project. We believe that this process has been a
key component of our success to date.
My point-of-contact for this matter is Mr. Bobby German, Deputy Program
Director for NASA's Integrated Financial Management Program. He may be
contacted by e-mail at bobby. eg rman@nasa.gov, or by telephone at
(202) 358-2498.
Cordially,
Signed by:
Frederick D. Gregory:
Deputy Administrator:
Enclosure:
Enclosure:
NASA Response to Draft General Accounting Office (GAO) Report: BUSINESS
MODERNIZATION: Disciplined Processes Needed to Better Manage NASA's
Integrated Financial Management Program (GAO-04-118), dated October 1,
2003:
1. General Accounting Office (GAO) Recommendation: Prepare cost
estimates by the current Work Breakdown Structure (WBS) for the
remaining modules.
NASA Response to GAO Recommendation 1: Concur. The business cases
developed for every Integrated Financial Management Program (IFMP)
module have included full-cost estimates by Work Breakdown Structure
(WBS). NASA will continue to follow this practice. It should be noted,
however, that once an implementation contractor is selected, the module
project has the ability to update the work breakdown structure to match
the finalized implementer methodology. It should be further noted that
the majority of IFM costs are incurred through fixed-price contracts
with the module implementers (e.g., Accenture). The methodology for
cost reporting in a fixed-price contract environment is focused on
contract deliverable performance rather than work breakdown structure.
2. GAO Recommendation: Provide a clear audit trail between detailed WBS
estimates and the program cost estimate for the remaining modules.
NASA Response to GAO Recommendation 2: Concur. IFMP will take steps to
ensure that there is increased documented traceability linking the
estimates in the business cases to the initial project estimates.
3. GAO Recommendation: Prepare a full life-cycle cost estimate for the
entire IFM Program that meets NASA's established life-cycle cost and
full-cost guidance.
NASA Response to GAO Recommendation 3: Concur. For each module project,
the IFM Program has developed detailed business case analyses (BCA's)
that included full cost 10-year life-cycle estimates at low levels of
the work breakdown structure. Although the program did not estimate how
long each module would be operational, 10-year was option selected to
ensure that the decision processes for the projects appropriately
balanced development and operations costs. Nevertheless, the program
will ensure that future BCA's provide better estimated operational
costs through the expected life of the module, retirement costs, and
any other full-life cycle costs, as appropriate. The IFM Program will
also continue to submit to Office of Management and Budget (OMB) its
Exhibits 300 and 53 in compliance with Agency and OMB guidance.
4. GAO Recommendation: Use a systematic, logical, and comprehensive
tool, such as Probabilistic Risk Assessment, in establishing the level
of financial reserves for the remaining module projects and tailor the
analysis to risks specific to IFMP.
NASA Response to GAO Recommendation 4: Concur. The IFM Program will
enhance its risk evaluation methodology as part of each module
project's business case analysis (BCA). A comprehensive risk assessment
analysis, using eight different criteria, is currently performed in
each BCA. Those results are subsequently used to develop individual
project budgets, inclusive of reserves. The IFM Program will continue
to enhance this process, which is, nevertheless, currently fully
compliant with NASA policies in this area. Furthermore the program will
undertake a review of expenditures against reserves in past projects to
identify particular areas in which future projects should focus their
risk identification and quantification efforts.
5. GAO Recommendation: Quantify the cost impact of at least all risks
with a high likelihood of occurrence and a high magnitude of impact to
facilitate the continuing analysis necessary to maintain adequate
reserve levels.
NASA Response to GAO Recommendation 5: Concur. The IFM Program will
update its risk management policies to ensure that the program and its
individual projects analyze more consistently and better document the
cost impacts of high severity risks (which are classified as high
probability and high impact risks). Over the past 3 years, the program
has already identified and mitigated several hundred risks associated
with individual projects. Each risk carried a probability assessment
rating and an impact assessment rating, which were periodically updated
until the risk was either retired or mitigated. Additionally, detailed
individual costs assessments were computed for certain risks when it
became apparent that the mitigation efforts would not be successful in
fully retiring the risk. In most cases, reserves were used on those
tasks. Cost estimates are reflected as liens to the reserves or
application of reserves in the monthly status for the projects and
program.
6. Establish a clear link between the program's risk database and
financial reserves.
NASA Response to GAO Recommendation 6: Concur. The IFM Program will
provide more documentation and traceability on risks and other factors
that form the basis for cost reserves. Also, see above responses to
Recommendations 2 and 5.
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
Allen Li (202) 512-3600:
Acknowledgments:
Staff making key contributions to this report were Jerry Herley, Erin
Schoening, LaTonya Miller, and Karen Sloan.
FOOTNOTES
[1] See U.S. General Accounting Office, Major Management Challenges and
Program Risks: National Aeronautics and Space Administration,
GAO-03-114 (Washington, D.C.: January 2003).
[2] NASA abandoned two earlier efforts after spending about
$180 million over 12 years.
[3] See U.S. General Accounting Office, Business Modernization:
Improvements Needed in Management of NASA's Integrated Financial
Management Program, GAO-03-507 (Washington, D.C.: Apr. 30, 2003).
[4] See U.S. General Accounting Office, Business Modernization: NASA's
Integrated Financial Management Program Does Not Fully Address Agency's
External Reporting Issues, GAO-04-151 (Washington, D.C.: Nov. 21,
2003). Also, see U.S. General Accounting Office, Information
Technology: Architecture Needed to Guide NASA's Financial Management
Modernization, GAO-04-43 (Washington, D.C.: Nov. 21, 2003).
[5] See U.S. General Accounting Office, Business Modernization: NASA
Challenges in Managing Its Integrated Financial Management Program,
GAO-04-255 (Washington, D.C.: Nov. 21, 2003).
[6] Life-cycle cost is the total of the direct, indirect, recurring,
nonrecurring, and other related expenses incurred or estimated to be
incurred in the design, development, verification, production,
operation, maintenance, support, and retirement of a system over its
planned life.
[7] See GAO-04-151.
[8] See U.S. General Accounting Office, Space Station: Actions Under
Way to Manage Cost, but Significant Challenges Remain, GAO-02-735
(Washington, D.C.: July 17, 2002).
[9] See U.S. General Accounting Office, National Aeronautics and Space
Administration: Leadership and Systems Needed to Effect Financial
Management Improvements, GAO-02-551T (Washington, D.C.: Mar. 20, 2002).
[10] NASA is composed of headquarters offices; nine centers located
around the country; and the Jet Propulsion Laboratory, which is
operated by the California Institute of Technology. For the purpose of
this report, we treat the Jet Propulsion Laboratory as a center.
[11] When acquiring and implementing commercial hardware and software
solutions, organizations can generally pursue one of two basic
approaches: An organization can opt for a single package of already
integrated software components, which is referred to as the "best of
suite" approach, or it can opt for different software components from
different vendors, which is referred to as the "best of breed"
approach. "Best of suite" components are easier and less costly to
integrate.
[12] OMB's Electronic Government--or "e-Government"--initiatives
advocate the use of Internet-based technologies governmentwide for
agency business processes, such as payroll, travel management, and
recruiting. The goal of these initiatives is to eliminate redundant
systems and improve the government's quality of customer service.
[13] See GAO-04-151.
[14] A WBS is a method of organizing a program into logical
subdivisions at lower and lower levels of detail.
[15] NASA is organized into six strategic enterprises that function as
primary business areas for implementing NASA's mission. Each enterprise
draws on the capabilities of several NASA centers, while each center
contributes to multiple enterprises. For example, the Space Flight
Enterprise has oversight over NASA's human space flight program and
exercises management authority over the Johnson Space Center, Kennedy
Space Center, Marshall Space Flight Center, and Stennis Space Center.
[16] Planning, Budgeting, Acquisition, and Management of Capital
Assets, OMB Circular A-11, Part 7 (June 2002).
[17] SEI is a government-funded research organization that is widely
considered an authority on software implementations.
[18] NASA assumed a 10-year life cycle beginning in fiscal year 2001,
but the actual retirement date for the system is unknown, according to
the Deputy Program Director.
[19] For example, according to the current business case analysis,
NASA's document-generation system would have to meet several federal
requirements, including providing the General Services
Administration's Federal Procurement Data System and the National
Science Foundation's Federal Assistance Awards Data System with data,
along with reports to the Department of Labor and the Small Business
Administration.
[20] Probabilistic risk assessment is a method of systematically
examining complex technical systems to measure both the likelihood that
an undesired event will occur and the consequences that will result.
[21] Risk criticality is a function of the likelihood that an event
will occur and the severity of the consequences if the event does
occur. The criticality of each risk will be identified as low, medium,
or high. Risks with high criticality are also known as primary risks
and typically have a high likelihood of occurrence and a high magnitude
of impact.
[22] A mitigation plan for the Budget Formulation module indicated that
the project was in the process of assessing the potential cost impact
of four of its six high-criticality risks, which the project manager
confirmed.
[23] A lien is a potential cost to a project, direct or indirect, which
may or may not come to fruition, for which a portion of funding
reserves is set aside.
[24] NASA was to implement its full cost initiative October 1, 2003.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
