National Aeronautics and Space Administration
Significant Actions Needed to Address Long-standing Financial Management Problems Gao ID: GAO-04-754T May 19, 2004Congress asked GAO to testify on the status of the National Aeronautics and Space Administration's (NASA) financial management reform efforts. NASA faces major challenges that if not addressed, will weaken its ability to manage its highly complex programs. NASA has been on GAO's high-risk list since 1990 because of its failure to effectively oversee its contracts and contractors, due in part to the agency's lack of accurate and reliable information on contract spending. GAO's statement focused on (1) how NASA's history of clean audit opinions served to mask the true extent of the agency's financial management difficulties; (2) the results of NASA's fiscal year 2003 financial statement audit, which are a departure from the fiscal year 2002 results; (3) NASA's effort to implement an integrated financial management system; and (4) the challenges NASA faces in reforming its financial management organization. Although GAO does not make specific recommendations in this statement, GAO previously made several recommendations to improve NASA's acquisition and implementation strategy for its financial management system. While NASA ultimately agreed to implement all of the recommendations, it disagreed with most of the findings--stating that its acquisition and implementation strategy had already addressed GAO's concerns.
NASA faces major challenges in fundamentally reforming its financial management organization and practices. While some areas needing reform relate to automated systems, automation alone is not sufficient to transform NASA's financial management culture. Specifically, NASA needs to fully integrate its financial management operations with its program management decision-making process. Until that occurs, NASA risks addressing the symptoms of its problems without resolving the underlying causes. These causes include an agency culture that has not fully acknowledged the nature and extent of its financial management difficulties and does not link financial management to program implications. Historically, NASA management has downplayed the severity of its problems and has viewed the agency's financial operation as a function designed to produce clean financial audit opinions instead of viewing it as a tool that supports program managers in making decisions about program cost and performance. GAO's work has identified several areas of concern. Clean financial audit opinions masked serious financial management problems. Financial audits of NASA during the late 1990s did not provide an accurate picture of the agency's financial management operations, and instead masked serious problems that continue to exist today, including significant internal control weaknesses and systems that do not comply with federal standards. The new financial management system did not address all key stakeholder needs. GAO reported in April 2003 that NASA designed and implemented the new system's core financial module without involving key stakeholders, including program managers, cost estimators, and the Congress. NASA did not follow key best practices in implementing its new financial management system. GAO reported in April 2003 and again in November 2003 that the new system may do less and cost more than NASA expects because the agency did not follow key best practices for acquiring and implementing the system. For example, NASA acquired and deployed system components without an enterprise architecture and lacked discipline in its cost estimating processes. The new financial management system did not provide key external reporting capabilities. GAO reported in November 2003 that the system would not generate complete and accurate information necessary for external reporting of NASA property and budgetary data. Finally, if NASA is to reap significant benefits from its new financial management system, it must transform its financial management organization into a customer-focused partner in program results. This will require sustained top leadership attention combined with effective organizational alignment, strategic human capital management, and end-toend business process improvement.
GAO-04-754T, National Aeronautics and Space Administration: Significant Actions Needed to Address Long-standing Financial Management Problems
This is the accessible text file for GAO report number GAO-04-754T
entitled 'National Aeronautics and Space Administration: Significant
Actions Needed to Address Long-standing Financial Management Problems'
which was released on May 19, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Government Efficiency, Financial Management,
and Intergovernmental Relations, Committee on Government Reform, House
of Representatives:
For Release on Delivery Expected at 2:00 p.m. EDT Wednesday, May 19,
2004:
National Aeronautics and Space Administration:
Significant Actions Needed to Address Long-standing Financial
Management Problems:
Statement of Gregory D. Kutz,
Director, Financial Management and Assurance:
Allen Li,
Director, Acquisition and Sourcing Management:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-7541T]:
GAO Highlights:
Highlights of GAO-04-754T, a testimony before the Subcommittee on
Government Efficiency, Financial Management, and Intergovernmental
Relations, Committee on Government Reform, House of Representatives
Why GAO Did This Study:
The Subcommittee asked GAO to testify on the status of the National
Aeronautics and Space Administration‘s (NASA) financial management
reform efforts. NASA faces major challenges that if not addressed, will
weaken its ability to manage its highly complex programs. NASA has been
on GAO‘s high-risk list since 1990 because of its failure to
effectively oversee its contracts and contractors, due in part to the
agency‘s lack of accurate and reliable information on contract
spending. GAO‘s statement focused on (1) how NASA‘s history of clean
audit opinions served to mask the true extent of the agency‘s financial
management difficulties; (2) the results of NASA‘s fiscal year 2003
financial statement audit, which are a departure from the fiscal year
2002 results; (3) NASA‘s effort to implement an integrated financial
management system; and (4) the challenges NASA faces in reforming its
financial management organization.
Although GAO does not make specific recommendations in this statement,
GAO previously made several recommendations to improve NASA‘s
acquisition and implementation strategy for its financial management
system. While NASA ultimately agreed to implement all of the
recommendations, it disagreed with most of the findings”stating that
its acquisition and implementation strategy had already addressed GAO‘s
concerns.
What GAO Found:
NASA faces major challenges in fundamentally reforming its financial
management organization and practices. While some areas needing reform
relate to automated systems, automation alone is not sufficient to
transform NASA‘s financial management culture. Specifically, NASA needs
to fully integrate its financial management operations with its program
management decision-making process. Until that occurs, NASA risks
addressing the symptoms of its problems without resolving the
underlying causes. These causes include an agency culture that has not
fully acknowledged the nature and extent of its financial management
difficulties and does not link financial management to program
implications. Historically, NASA management has downplayed the severity
of its problems and has viewed the agency‘s financial operation as a
function designed to produce clean financial audit opinions instead of
viewing it as a tool that supports program managers in making decisions
about program cost and performance.
GAO‘s work has identified several areas of concern:
* Clean financial audit opinions masked serious financial management
problems. Financial audits of NASA during the late 1990s did not
provide an accurate picture of the agency‘s financial management
operations, and instead masked serious problems that continue to exist
today, including significant internal control weaknesses and systems
that do not comply with federal standards.
* The new financial management system did not address all key
stakeholder needs. GAO reported in April 2003 that NASA designed and
implemented the new system‘s core financial module without involving
key stakeholders, including program managers, cost estimators, and the
Congress.
* NASA did not follow key best practices in implementing its new
financial management system. GAO reported in April 2003 and again in
November 2003 that the new system may do less and cost more than NASA
expects because the agency did not follow key best practices for
acquiring and implementing the system. For example, NASA acquired and
deployed system components without an enterprise architecture and
lacked discipline in its cost estimating processes.
* The new financial management system did not provide key external
reporting capabilities. GAO reported in November 2003 that the system
would not generate complete and accurate information necessary for
external reporting of NASA property and budgetary data.
Finally, if NASA is to reap significant benefits from its new financial
management system, it must transform its financial management
organization into a customer-focused partner in program results. This
will require sustained top leadership attention combined with effective
organizational alignment, strategic human capital management, and
end-to-end business process improvement.
www.gao.gov/cgi-bin/getrpt? GAO-04-754T.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Gregory D. Kutz at (202)
512-9095 or kutzg@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
Thank you for the opportunity to discuss the financial management
challenges facing the National Aeronautics and Space Administration
(NASA). Since its inception in 1958, NASA has undertaken numerous
programs--involving earth and space science, aerospace technology,
human space flight, and biological and physical research--that have
resulted in significant scientific and technological advances,
enhancing the quality of life on earth. In recent years, NASA has
experienced a number of setbacks with its programs and operations,
including massive cost overruns associated with the International Space
Station and, with the Columbia tragedy, the need for the agency to
develop return-to-flight strategies and mitigate the impact of the loss
of the shuttle on the construction of the space station.
On January 14, 2004, President Bush outlined a bold new vision for U.S.
space exploration that will set a new course for NASA. However, to
successfully execute this new vision, NASA must address a number of
long-standing financial management challenges that threaten NASA's
ability to manage its programs, oversee its contractors, and
effectively allocate its budget across its numerous projects and
programs. In fact, since 1990 we have identified NASA's contract
management as an area of high risk, in part because the agency lacked
effective systems and processes for overseeing contract spending and
performance. NASA has begun taking action to address many of these
challenges through its effort to implement a new integrated financial
management system; however, many of NASA's financial management
problems are deeply rooted in an agency culture that has not fully
acknowledged the nature and extent of its financial management
difficulties and does not view finance as intrinsic to the agency's
program management decision process.
My testimony today will focus on the results of our recent work related
to NASA's financial management challenges and the agency's efforts to
implement an integrated financial management system. Specifically, I
will discuss (1) how NASA's history of clean audit opinions served to
mask the true extent of the agency's financial management difficulties;
(2) the results of NASA's fiscal year 2003 financial statement audit,
which are a departure from the fiscal year 2002 results; (3) NASA's
current effort to implement an integrated financial management system;
and (4) the challenges NASA faces in reforming its financial management
organization. We have performed work and issued several reports in
response to legislative mandates and at the request of other interested
committees. We also reviewed the reports of NASA's Office of Inspector
General and the independent public accounting firms that audited NASA's
financial statements for fiscal year 2003 and for several previous
years. With the exception of NASA's financial statements for fiscal
year 2002, in which we performed a limited-scope review of the
financial statement audit performed by NASA's contracted independent
public accountant (IPA), we did not review the IPA's underlying audit
work. We performed all work in accordance with generally accepted
government auditing standards. My statement today is drawn from the
findings and conclusions in GAO's, NASA's Office of Inspector
General's, and the independent auditors' reports.
Summary:
NASA has fundamental problems with its financial management operations
that not only affect its ability to externally report reliable
information, but more important, hamper its ability to effectively
manage and oversee its major programs, such as the space station and
the shuttle program. NASA's financial audits during the 1990s masked
serious problems with its financial management operations that continue
today. Specifically, from 1996 through 2000, NASA was one of the few
agencies to be judged by its independent auditor at the time, Arthur
Andersen, as meeting all of the federal financial reporting
requirements. However, our work at NASA during this same period told a
different story. During this period, we issued a wide range of reports
that detailed the agency's difficulties associated with (1) overseeing
its contractors and their financial and program performance, (2)
controlling program costs and producing credible cost estimates, and
(3) supporting the amounts that it had reported to the Congress as
obligated against statutory spending limits for the space station and
related space shuttle support. We also concluded, based on work we
performed related to a misstatement in NASA's fiscal year 1999
financial statements, that Arthur Andersen's work did not meet
professional standards, and we questioned NASA management's and its
auditor's determination that the agency's systems substantially
complied with federal standards.
The results of NASA's fiscal year 2003 financial statement audit
confirm that NASA's financial management problems continue today.
NASA's independent auditor, Pricewaterhouse Coopers (PwC), disclaimed
an opinion on NASA's fiscal year 2003 financial statements; reported
material weaknesses in internal controls; and for the third straight
year, concluded, just as we reported in November 2003,[Footnote 1] that
the agency's new financial management system did not comply with the
requirements of the Federal Financial Management Improvement Act of
1996 (FFMIA).[Footnote 2] Although NASA attributed the auditor's
disclaimer of opinion to the agency's implementation of a new financial
management system, many of the reported problems were long-standing
issues not related to implementation of the new system.
Recognizing the importance of successfully implementing an integrated
financial management system, in April 2000, NASA began an effort known
as the Integrated Financial Management Program (IFMP). Through IFMP,
NASA has committed to modernizing its business processes and systems in
a way that if implemented properly, will introduce interoperability and
thereby improve the efficiency and effectiveness of its operations as
well as bring the agency into compliance with federal system
requirements. NASA has also committed to implementing IFMP within
specific cost and schedule constraints. In 2003, we issued five
reports[Footnote 3] outlining the considerable challenges NASA faces in
meeting its IFMP commitments and providing NASA the necessary tools to
oversee its contracts and manage its program. For example, in April
2003, we reported that NASA had deferred addressing the needs of key
system stakeholders,[Footnote 4] including program managers and cost
estimators, and was not following key best practices for acquiring and
implementing the system. We also reported that NASA lacked the
disciplined requirements management and testing processes needed to
reduce the risk associated with its effort to acceptable levels.
Therefore, NASA did not have reasonable assurance that the program
would meet its cost, schedule, and performance objectives. Then, in
November 2003, we reported that NASA (1) acquired and deployed IFMP
system components without an enterprise architecture, or agencywide
modernization blueprint, to guide and constrain program investment
decisions; (2) did not use disciplined cost estimating processes or
recognized best practices in preparing its life cycle cost estimates;
and (3) had delayed implementation of many key external reporting
capabilities. We made a number of recommendations in these reports to
improve NASA's acquisition and implementation strategy for IFMP. While
NASA ultimately agreed to implement all of our recommendations, it
disagreed with most of our findings--stating that its acquisition and
implementation strategy had already addressed many of our concerns.
Finally, NASA faces significant challenges in overcoming its financial
management difficulties and reforming its financial management
operations. For example, NASA's independent auditor, PwC, attributed
many of the agency's financial management problems to a lack of
understanding by NASA's staff of federal reporting requirements. In
addition, over the past 4 years, we have issued numerous reports
highlighting NASA's financial management difficulties and making
recommendations for improvement. However, NASA management has been slow
to implement these recommendations and in many cases has denied the
existence of the problems we and others have identified--instead
attributing the agency's difficulties to the auditor's sampling
methodology or the auditor's lack of understanding of NASA's overall
operations. Until NASA fully acknowledges the nature and extent of its
financial management difficulties and better integrates the agency's
financial management operation with its program management decision
process, NASA will continue to face many of the same financial
management problems discussed in my testimony today.
Clean Financial Audit Opinions Masked Serious Financial Management
Problems:
NASA's financial audits during the 1990s masked serious problems with
its financial management operations that continue today. Specifically,
from 1996 through 2000, NASA was one of the few agencies to be judged
by its independent auditor, Arthur Andersen, as meeting all of the
federal financial reporting requirements. That is, NASA was one of the
few agencies to receive an unqualified, or "clean," opinion on its
financial statements, with no material internal control weaknesses
noted and with financial management systems that were reported to be in
substantial compliance with the requirements of FFMIA. FFMIA, building
on previous financial management reform legislation, stresses that
agencies need to have systems that provide managers with the reliable,
timely, and accurate financial information that they need to ensure
accountability on an ongoing basis, as well as to make informed
decisions on investing resources, managing costs, and overseeing
programs. Thus, the auditor's report implied that NASA could not only
generate reliable information once a year for external financial
reporting purposes but also could provide the kind of information
needed for day-to-day management decision making. However, as others
and we have reported, the independent auditor's reports did not provide
an accurate picture of NASA's financial management systems and failed
to disclose pervasive financial management problems that existed at
NASA then and continue today. Ultimately, these unqualified opinions
and positive reports on NASA's internal controls and systems served
only to mask the serious financial management problems that existed at
NASA throughout this period.
* First in 1990 and then in subsequent years, we identified contract
management as an area at high risk because of NASA's inability to (1)
oversee its contractors and their financial and program performance and
(2) implement a modern, integrated financial management system, which
is integral to producing accurate and reliable financial information
needed to support contract management.[Footnote 5] During this period,
we also issued a wide range of reports that detailed the agency's
difficulties associated with controlling program costs and producing
credible cost estimates.
* In 2000, congressional staff members found a $644 million
misstatement in NASA's fiscal year 1999 financial statements--an error
not previously detected by NASA or its independent auditor. As we
reported[Footnote 6] in March 2001, this error resulted because NASA's
systems could not produce the budgetary data required by federal
accounting standards. Instead, the agency was relying on an ad hoc,
year-end data call from its 10 reporting units and the aggregation of
data using a computer spreadsheet. We concluded that Arthur Andersen's
work did not meet professional standards, and we questioned NASA
management's and its auditor's determination that the agency's systems
substantially complied with the requirements of FFMIA.
* In 2001 and subsequent years, our work in response to a legislative
mandate revealed that NASA was unable to support the amounts that it
had reported to the Congress as obligated against statutory spending
limits for the space station and related space shuttle
support.[Footnote 7] Here again, NASA's inability to provide this
detailed obligation data was linked to its lack of a modern, integrated
financial management system.
* Finally, in February 2002, NASA's new independent auditor, PwC,
further confirmed NASA's financial management difficulties and
disclaimed an opinion on the agency's fiscal year 2001 financial
statements. The audit report also identified a number of material
internal control weaknesses and stated that contrary to previous
financial audit reports, NASA's financial management systems did not
substantially comply with FFMIA.
* Although NASA received an unqualified opinion on its fiscal year 2002
financial statements,[Footnote 8] NASA's auditor again report material
weaknesses in NASA's internal controls over its Property, Plant, and
Equipment (PP&E) and materials, which make up nearly $37 billion, or 85
percent, of NASA's assets, and over the agency's processes for
preparing its financial statements and performance and accountability
report. According to the auditor's report, various deficiencies
continued to exist within NASA's financial management operations,
including (1) insufficient resources to address the volume of
compilation work required to prepare NASA financial reports, (2) lack
of an integrated financial management system, and (3) lack of
understanding by NASA staff of federal reporting requirements. The
nature and extent of the reported material weaknesses highlighted the
agency's inability to generate reliable data for daily operations and
decision making. Thus, it is not surprising that the auditor again
concluded that NASA's financial management systems did not
substantially comply with the requirements of FFMIA.
NASA's Auditor Disclaims an Opinion on Fiscal Year 2003 Financial
Statements:
NASA's financial management problems and internal control weaknesses
continue to exist today. NASA's auditor, PwC, disclaimed an opinion on
NASA's fiscal year 2003 financial statements. According to the
auditor's report, NASA was unable to provide PwC sufficient evidence to
support the financial statements and complete the audit within the time
frames established by the Office of Management and Budget. In addition,
for the third straight year, NASA's independent auditor concluded, just
as we reported in November 2003,[Footnote 9] that the agency's new
financial management system did not comply with the requirements of
FFMIA. Although NASA attributed the auditor's disclaimer of opinion to
the agency's implementation of a new financial management system, many
of the reported problems were long-standing issues not related to
implementation of the new system. The auditor reported material
weaknesses that existed throughout NASA's financial management
operations.
* First, NASA was unable to provide reliable documentation and an audit
trail to support the financial statements. NASA's auditor reported that
in an effort to populate its new financial management system, NASA
summarized the previous 7 years of transaction-level detail from its
legacy systems and entered the cumulative amount into the new system as
if the transactions were current-year activity. As a result, many of
the accounts supporting the financial statements were overstated by
billions of dollars. In an effort to correct these errors and balance
the accounts to the general ledger, NASA made net adjustments totaling
$565 billion but was not able to provide documentation supporting the
adjustments.
* Second, NASA's internal controls over its reconciliation of fund
balance with Treasury accounts were ineffective. Specifically, NASA
failed to reconcile its fund balance with Treasury accounts during the
year and resolve all differences. At year-end, NASA's general ledger
account for fund balance with Treasury was materially overstated and
did not reconcile to the balance reported by Treasury at year-end. To
correct the overstatement, NASA made $2 billion in unsupported net
adjustments to its Fund Balance with Treasury account, which had the
effect of reducing NASA's recorded balance so it equaled Treasury's
reported balance. This type of adjustment is similar to forcing the
balance recorded in your checkbook at the end of the month to reconcile
with your bank statement. Instead of trying to determine the reason for
the error and resolve the difference you, simply "plug" the difference
to your checkbook balance. NASA's failure to perform reconciliation
procedures throughout the year is a fundamental breakdown in basic
internal controls and illustrates the human capital challenges NASA
faces in overcoming its financial management problems.
* Third, NASA's processes for preparing its financial statements
continue to be ineffective. The continued weaknesses in NASA's
financial statement preparation processes resulted in major delays and
errors in preparing fiscal year-end financial statements. For example,
NASA's auditor reported inconsistencies, such as the significant
differences between the agency's Fund Balance with Treasury and
Treasury's balance that should have been identified and corrected by
NASA as part of the agency's internal quality control review process.
In addition, NASA's financial statements were not prepared in
accordance with federal accounting standards. As we reported in
November 2003, the core financial module did not appropriately capture
accrued contract costs and accounts payable information in accordance
with federal accounting standards. Instead, in instances where costs
and the corresponding liabilities were greater than the associated
obligations, the differences were transferred outside of the general
ledger and held in suspense until additional funds were obligated, thus
understating NASA's reported program costs and liabilities. Although
NASA officials stated that as of October 1, 2003, they no longer post
costs in excess of obligations in a suspense account, their current
solution still does not appropriately capture accrued cost and accounts
payable in accordance with federal accounting standards.
* Finally, NASA continues to lack effective internal controls over PP&E
and materials. Although NASA reported that a corrective action plan had
been implemented to address the deficiencies identified in the previous
year's audit report, subsequent testing identified major errors in
contractor-held PP&E and materials.
NASA's Effort to Implement New Integrated Financial Management System:
NASA's new financial management system falls short in addressing the
long-standing financial management issues that have prevented the
agency from effectively monitoring over 90 percent of its annual budget
and managing costly and complex programs, such as the International
Space Station. For years, NASA has cited deficiencies within its
financial management systems as a primary reason for not having the
data required to oversee its contractors, accurately account for the
full cost of its operations, and efficiently produce accurate and
reliable information needed for both management decision-making and
external reporting purposes. Recognizing the importance of successfully
implementing an integrated financial management system, in April 2000,
NASA began its IFMP effort. When completed, IFMP is planned to consist
of nine modules[Footnote 10] that will support a range of financial,
administrative, and functional areas. This is NASA's third attempt at
modernizing its financial management systems and processes. The first
two efforts were eventually abandoned after a total of 12 years and a
reported $180 million. The schedule for implementing IFMP was
originally planned for fiscal year 2008, but after NASA's new
Administrator came on board in fiscal year 2002, the timeline was
accelerated to fiscal year 2006, with the core financial module to be
completed in fiscal year 2003. As of June 30, 2003, NASA reported that
it had fully implemented the core financial module at all of its 10
operating locations.
Through IFMP, NASA has committed to modernizing its business processes
and systems in a way that if implemented properly, will introduce
interoperability and thereby improve the efficiency and effectiveness
of its operations as well as bring the agency into compliance with
federal financial management systems requirements. NASA has also
committed to implementing IFMP within specific cost and schedule
constraints. In 2003, we issued five reports[Footnote 11] outlining the
considerable challenges NASA faces in meeting its IFMP commitments and
providing NASA the necessary tools to oversee its contracts and manage
its program. For example, in April 2003, we reported that NASA had
deferred addressing the needs of key system stakeholders,[Footnote 12]
including program managers and cost estimators, and was not following
key best practices for acquiring and implementing the system. Then, in
November 2003, we reported that NASA (1) acquired and deployed system
components of IFMP without an enterprise architecture, or agencywide
modernization blueprint, to guide and constrain program investment
decisions; (2) did not use disciplined cost estimating processes or
recognized best practices in preparing its life cycle cost estimates;
and (3) had delayed implementation of many key external reporting
capabilities.
IFMP Core Financial Module Will Not Fully Address the Needs of Key
Stakeholders:
Based on our review of NASA's three largest space flight programs--the
space station, the space shuttle, and the Space Launch
Initiative,[Footnote 13] in April 2003 we reported that the core
financial module, as currently implemented, did not fully address the
information requirements of stakeholders such as program managers, cost
estimators, or the Congress. While NASA considers these officials to be
the ultimate beneficiaries of the system's improvements, they were not
involved in defining or implementing the system requirements and will
not have a formal role in defining or measuring its success. As a
result, NASA has neither reengineered its core business processes nor
established adequate requirements for the system to address many of its
most significant management challenges, including improving contract
management; producing credible cost estimates; and providing the
Congress with appropriate visibility over NASA's large, complex
programs. Specific issues for key stakeholders include the following:
* Program managers. To adequately oversee NASA's largest contracts,
program managers need reliable contract cost data--both budgeted and
actual--and the ability to integrate these data with contract
schedule[Footnote 14] information to monitor progress on the contract.
However, because program managers were not involved in defining system
requirements or reengineering business processes, the core financial
module was not designed to integrate the cost and schedule data that
they need. As a result, program managers told us that they would not
use the core financial module to manage programs such as the space
station and space shuttle and instead would continue to rely on hard
copy reports, electronic spreadsheets, or other means to monitor
contractor performance.
* Cost estimators. In order to estimate the costs of programs, cost
estimators need reliable contract cost data at a level of detail
greater than what the core financial module maintains. Although this
module is technologically capable of maintaining the detail they need,
cost estimators were not involved in defining the system requirements
or reengineering business processes. Reengineering is critical here
because a driving factor in determining what information cost
estimators receive from contractors is what level of detail the
contractors are required to provide, based on the contracts that they
have negotiated with NASA. As a result, NASA has not determined the
most cost-effective way to satisfy the information needs of its cost
estimators. Because the core financial module will not contain the
sufficiently detailed historical cost data necessary for projecting
future costs, cost estimators will continue to rely on labor-intensive
data collection efforts after a program is completed.
* The Congress. Based on our discussions with congressional staffs from
NASA's authorizing committees, the agency did not consult with them
regarding their information needs. Consequently, NASA cannot be sure
that it is implementing a system that will provide the Congress with
the information it needs for oversight.
According to IFMP officials, they chose to forgo certain system
capabilities to expedite implementation of the core financial module.
Thus, while the core financial module software is technologically
capable of meeting key stakeholders' needs, it has not been configured
to do so. IFMP officials have stated that these capabilities can be
added at a later date. We made several recommendations related to
engaging stakeholders, including cost estimators and program managers,
in developing a complete and accurate set of requirements. Although
NASA officials concurred with our recommendations, they disagreed with
our finding--stating that they had already effectively engaged key
stakeholders.
NASA Was Not Following Key Best Practices for Acquiring and
Implementing IFMP:
We reported in April 2003 that NASA's approach to implementing its new
system did not optimize the system's performance and would likely cost
more and take longer to implement than necessary. Specifically, NASA
was not following key best practices for acquiring and implementing the
system, which may affect the agency's ability to fully benefit from the
new system's capabilities. First, NASA did not analyze the
relationships among selected and proposed IFMP components to understand
the logical and physical relationships among the components it
acquired. By acquiring these IFMP components without first
understanding system component relationships, NASA increased its risks
of implementing a system that will not optimize mission performance and
will cost more and take longer to implement than necessary. Second,
although industry best practices and NASA's own system planning
documents indicate that detailed requirements are needed as the basis
for effective system testing, NASA did not require documentation of
detailed system requirements prior to system implementation and
testing. NASA's approach instead relied on certain subject matter
experts' knowledge of the detailed requirements necessary to evaluate
the functionality actually provided.
We made several recommendations to focus near-term efforts on
stabilizing the operational effectiveness of deployed IFMP components.
While NASA officials concurred with our recommendations, they disagreed
with our findings--stating that they had already implemented effective
processes related to performing dependency analysis and requirements
and testing.
IFMP Components Deployed without an Enterprise Architecture:
We reported in November 2003 that NASA had acquired and deployed system
components of IFMP without an enterprise architecture, or agencywide
modernization blueprint, to guide and constrain program investment
decisions--actions that increased the chances that these system
components will require additional time and resources to be modified
and to operate effectively and efficiently. During the course of our
review of IFMP, NASA implemented some of these key architecture
management capabilities, such as having an enterprise architecture
program office; designating a chief architect; and using an
architecture development methodology, framework, and automated tools.
However, at the time, NASA had not yet established other key
architecture management capabilities, such as designating an
accountable corporate entity to lead the architecture effort, having an
approved policy for developing and maintaining the architecture, and
implementing an independent verification and validation function to
provide needed assurance that architecture products and architecture
management processes are effective.
As NASA proceeds with its enterprise architecture effort, it is
critical that it employs rigorous and disciplined management practices.
Such practices form the basis of our architecture management maturity
framework,[Footnote 15] which specifies by stages the key architecture
management controls that are embodied in federal guidance and best
practices, provides an explicit benchmark for gauging the effectiveness
of architecture management, and provides a road map for making
improvements. GAO made several recommendations to ensure that NASA had
the necessary agencywide context within which to make informed IFMP and
other systems modernization decisions. NASA agreed that improvements
were needed and reported that it had efforts under way, consistent with
our recommendations, to develop an architecture and ensure that IFMP
proceeded within the context of the architecture. We have not evaluated
NASA's progress on these commitments.
IFMP Further Challenged by Questionable Cost Estimates and an
Optimistic Schedule:
Questionable cost estimates, an optimistic schedule, and insufficient
processes for ensuring adequate funding reserves have put IFMP at an
even greater risk of not meeting program objectives. In preparing its
life cycle cost estimates for IFMP,[Footnote 16] NASA did not use
disciplined cost estimating processes as required by its standards and
recognized best practices. For example, NASA's current IFMP life cycle
cost estimate--which totals $982.7 million and is 14 percent, or $121.8
million, over the previous IFMP life cycle cost estimate--was not
prepared on a full-cost basis. The estimate included IFMP direct
program costs, NASA enterprise support, and civil service salaries and
benefits, but it did not include the cost of retiring the system,
enterprise travel costs, the cost of nonleased NASA facilities for
housing IFMP, and other direct and indirect costs likely to be incurred
during the life of the program. In addition, NASA did not consistently
use breakdowns of work in preparing the cost estimate, although NASA
guidance calls for breaking down work into smaller units to facilitate
cost estimating and project and contract management as well as to help
ensure that relevant costs are not omitted. In cases where work
breakdowns were used, the agency did not always show the connection
between the work breakdown estimates and the official program cost
estimate. This has been a weakness since the inception of the program.
Without a reliable life cycle cost estimate, NASA will have difficulty
controlling program costs.
In addition, NASA's schedule may not be sufficient to address program
challenges, such as personnel shortages. To address personnel shortages
during the implementation of the core financial module, NASA paid
nearly $400,000 for extra hours worked by center employees and avoided
a slip in IFMP's compressed schedule. However, the schedule for
implementing the budget formulation module has slipped because IFMP
implemented this module simultaneously with the core financial module-
-an action advised against by a contractor conducting a lessons-learned
study--placing heavy demand on already scarce resources.
Finally, the program did not consistently perform in-depth analyses of
the potential cost impact of risks and unknowns specific to IFMP, as
required by NASA guidance. Instead, the program established funding
reserves on the basis of reserve levels set by other high-risk NASA
programs. As a result, reserve funding for IFMP contingencies may be
insufficient--which is particularly problematic, given the program's
questionable cost estimates and optimistic schedule. As we were
completing our audit work, one module--budget formulation--was already
experiencing shortfalls in its reserves, and project officials
expressed concern that the module's functionality may have to be
reduced. Moreover, the program did not quantify the cost of high
criticality risks--risks that have a high likelihood of occurrence and
a high magnitude of impact--or link these risks to funding reserves to
help IFMP develop realistic budget estimates. We made recommendations
to provide NASA the necessary tools to accurately estimate program cost
and predict the impact of program challenges. Although NASA concurred
with our recommendations for corrective action, NASA indicated that its
current processes were adequate for preparing work breakdown structure
cost estimates, estimating life-cycle costs, and establishing reserves
based on IFMP-specific risks.
Core Financial Module Does Not Address Long-standing External Reporting
Issues:
The core financial module, as currently implemented, also does not
address many of the agency's most challenging external reporting
issues. Specifically, the core financial module does not address NASA's
past external reporting problems related to property accounting and
budgetary accounting. Such shortcomings limit the ability of the
Congress and other interested parties to evaluate NASA's performance on
an ongoing basis because NASA's financial management systems do not
provide a complete accounting of its assets and how funds were spent.
If these issues are not addressed, NASA will continue to face risks in
its ability to adequately oversee its programs, manage their costs, and
provide meaningful information to external parties, such as the
Congress.
* Property accounting. The core financial module has not addressed the
problems I discussed previously related to material weaknesses in
NASA's internal controls over PP&E and materials. NASA's PP&E and
materials are physically located throughout the world, at locations
including NASA centers, contractor facilities, other private or
government-run facilities, and in space. NASA's most significant
challenge, with respect to property accounting, stems from property
located at contractor facilities, which accounts for almost $11
billion, or about one-third, of NASA's reported $37 billion of PP&E and
materials and consists primarily of equipment being constructed for
NASA or items built or purchased for use in the construction process.
NASA has not reengineered the agency's processes for capturing contract
costs associated with PP&E and material, and therefore, does not record
these property costs in the general ledger at the transaction level.
Instead, according to NASA officials, the agency plans to continue to
(1) record the cost of PP&E and materials as expenses when initially
incurred, (2) periodically determine which of those costs should have
been capitalized, and (3) manually correct these records at a summary
level. Because NASA does not maintain transaction-level detail, the
agency is not able to link the money it spends on construction of its
property to discrete property items and therefore must instead rely
solely on its contractors to periodically report summary-level
information on these assets to NASA.
* Budgetary accounting. The software NASA selected, and is now using,
for its core financial module does not capture and report certain key
budgetary information needed to prepare its Statement of Budgetary
Resources. As a result, NASA continues to rely on manual compilations
and system queries to extract the data needed to prepare the Statement
of Budgetary Resources--just as it did using its legacy general ledger
system. According to NASA officials, a "patch" release or software
upgrade in October 2003 has addressed the issues we identified related
to budgetary accounting. However, we have not verified NASA's assertion
and previously reported that NASA had implemented similar "patch"
releases that did not fully address this issue. As we reported in March
2001, this cumbersome, labor-intensive effort to gather the information
needed at the end of each fiscal year was the underlying cause of a
$644 million misstatement in NASA's fiscal year 1999 Statement of
Budgetary Resources. Although the software that NASA purchased for the
core financial module was certified by the Joint Financial Management
Improvement Program (JFMIP) as meeting all mandatory system
requirements, NASA may have relied too heavily on the JFMIP
certification. JFMIP has made it clear that its certification, by
itself, does not automatically ensure compliance with the goals of
FFMIA. Other important factors that affect compliance with Federal
Financial Management System Requirements include how well the software
has been configured to work in the agency's environment and the quality
of transaction data in the agency's feeder systems. As I mentioned
previously, NASA did not use the disciplined requirements management
and testing processes necessary to reduce the risks associated with its
implementation efforts to acceptable levels. Therefore, it is not
surprising that NASA found that the system was not providing the
desired functionality or performing as expected.
Core Financial Module Does Not Comply with FFMIA:
As I mentioned previously, in November 2003,[Footnote 17] we reported
that NASA's new core financial module did not comply substantially with
the requirements of FFMIA. At the time, NASA disagreed with our
conclusions and recommendations regarding its financial management
systems and stated that many of the problems we identified as of June
30, 2003, had been resolved by September 30, 2003. However, in February
2004, after NASA's independent auditor also concluded that NASA's
financial management system, at September 30, 2003, did not
substantially comply with the requirements of FFMIA, NASA reversed its
position and concurred with all of our recommendations. Specifically,
NASA agreed to implement a corrective action plan that will engage key
stakeholders in developing a complete and accurate set of user
requirements, reengineering its acquisition management processes, and
bringing its systems into compliance with FFMIA.
FFMIA stresses the need for agencies to have systems that can generate
timely, accurate, and useful financial information with which to make
informed decisions, manage daily operations, and ensure accountability
on an ongoing basis. Compliance with FFMIA goes far beyond receiving a
"clean" opinion on financial statements. Instead, FFMIA provides
agencies with the building blocks needed to reform their financial
management organization and practices, and to support program managers
in making wise decisions about program cost and performance. However,
as we reported in April 2003 and in November 2003, NASA's core
financial module did not provide program managers, cost estimators, or
the Congress with managerially relevant cost information that they need
to effectively manage and oversee NASA's contracts and programs, such
as the International Space Station. NASA's continuing inability to
provide its managers with timely, relevant data on contract spending
and performance is a key reason that we continue to report NASA's
contract management as an area of high risk. Because this information
is not available through the core financial module, program managers
will continue to rely on hard copy reports, electronic spreadsheets, or
other means to monitor contractor performance. Consequently, NASA risks
operating with two sets of books--one that is used to report
information in the agency's general-purpose financial reports and
another that is used by program managers to run NASA's projects and
programs.
NASA Faces Significant Challenges in Reforming Its Financial Management
Operations:
Many of NASA's financial management problems are deeply rooted in an
agency culture that has not fully acknowledged the nature and extent of
its financial management difficulties and does not see finance as
intrinsic to the agency's program management decision process. Over the
past 4 years, we have issued numerous reports highlighting NASA's
financial management difficulties and making recommendations for
improvement. However, NASA management has been slow to implement these
recommendations and in many cases has denied the existence of the
problems we and others have identified--instead attributing the
agency's difficulties to the auditor's sampling methodology or the
auditor's lack of understanding of NASA's operations. For example:
* In response to our August 2001 and April 2002 reports on NASA's
compliance with the International Space Station and shuttle support
cost limits, NASA management disagreed with our finding that NASA was
unable to support the amounts that it had reported to the Congress as
obligated against the statutory spending limits for the space station
and related space shuttle support costs. At the time, NASA asserted
that the obligations were verifiable and that our audit methodology was
the problem. We planned to use statistical sampling, which is a
standard, widely used methodology that enables auditors to draw
conclusions about large populations of transactions by testing a
relatively small number of those transactions. In order for a
statistical sample to be valid, the complete population of items of
interest must be subject to selection and every transaction must have a
chance to be selected for testing. However, after nearly a year, NASA
was not able to provide us with a complete population of transactions
from which to draw our sample. Consequently, we were unable to verify
the accuracy of the amount NASA reported against the cost limits.
* In a March 20, 2002, statement before this subcommittee NASA
management attributed its failure to obtain an unqualified opinion on
the agency's fiscal year 2001 financial statements to its auditor's
newly required protocol for sampling. However, the only thing new about
the sampling protocol was that NASA's previous auditor, Arthur
Andersen, had not employed a similar approach. In fact, to test amounts
reported on NASA's fiscal year 2001 financial statements, NASA's new
financial statement auditor, PwC, attempted to use standard
transaction-based statistical sampling similar to the methods we had
attempted in our effort to audit the underlying support for amounts
charged to the spending limits. In its audit report, PwC noted that
successive summarization of data through NASA's various financial
systems impeded NASA's ability to maintain an audit trail down to the
detailed transaction-level source documentation. For this and other
reasons, PwC concluded that it was unable to audit NASA's financial
statements.
* In response to our April 2003 report on the status of NASA's
implementation of IFMP, NASA management disagreed with all of our
findings, including our concerns that NASA program managers and cost
estimators were not adequately involved in defining system requirements
and, therefore the system did not fully address their information
needs. In its written comments, NASA dismissed these concerns and
stated that the problem was a lack of understanding not a lack of
information, and that it was incumbent upon program managers and cost
estimators to learn and understand the capabilities of the new system
and take advantage of them for their specific purposes.
* Finally, in response to our November 2003 report on IFMP's external
reporting capabilities, NASA management disagreed with all of our
conclusions and recommendations, including our conclusion that the core
financial module, as implemented in June 2003, did not comply
substantially with FFMIA. In its written comments, dated October 31,
2003, NASA asserted that many of the problems we identified in June
2003 were resolved by September 30, 2003. However, NASA's assertions
did not prove to be accurate. In January 2004, NASA's independent
financial statement auditor confirmed that the problems we identified
in June 2003 related to NASA's accrued costs, budgetary accounting, and
property accounting still existed at September 30, 2003, and that the
system was not in compliance with FFMIA requirements. NASA reversed its
position in February 2004 and concurred with our recommendations that
it implement a corrective action plan that will engage key stakeholders
in developing a complete and accurate set of user requirements,
reengineering its acquisition management processes, and bringing its
systems into compliance with FFMIA.
The challenges that NASA faces in reforming its financial management
operations are significant, but not insurmountable. As our prior
work[Footnote 18] shows, clear, strong leadership will be critical for
ensuring that NASA's financial management organization delivers the
kind of analysis and forward-looking information needed to effectively
manage its many complex space programs. Further, in order to reap the
full benefit of a modern, integrated financial management system, NASA
must (1) routinely generate reliable cost and performance information
and analysis, (2) undertake other value-added activities that support
strategic decision making and mission performance, and (3) build a
finance team that supports the agency's mission and goals.
Conclusion:
Until NASA fully acknowledges the nature and extent of its financial
management difficulties and better integrates its financial management
operations with its program management decision process, it will
continue to face many of the same financial management problems I have
discussed today. While modernizing NASA's financial management system
is essential to enabling the agency to provide its managers with the
kind of timely, relevant, and reliable information that they need to
manage cost, measure performance, make program funding decisions, and
analyze outsourcing or privatization options, NASA cannot rely on
technology alone to solve its financial management problems. Rather,
transforming NASA's financial management organization will also require
sustained top leadership attention combined with effective
organizational alignment, strategic human capital management, and end-
to-end business process reengineering. This goes far beyond obtaining
an unqualified audit opinion and requires that agency financial
managers focus on their overall operations in a strategic way and not
be content with an automated system that helps the agency get a "clean"
audit opinion once a year without providing additional value to the
program managers and cost estimators who use its financial data.
Mr. Chairman, this concludes our prepared statement. We would be
pleased to respond to any questions that you or other members of the
Subcommittee may have.
Contacts and Acknowledgments:
For further information regarding this testimony, please contact
Gregory D. Kutz at (202) 512-9095 or [Hyperlink, kutzg@gao.gov] or
Allen Li at (202) 512-3600 or [Hyperlink, lia@gao.gov] or Diane
Handley at (404) 679-1986 or [Hyperlink, handley@gao.gov]. Individuals
making key contributions to this testimony included Fannie Bivins and
Francine DelVecchio.
[End of section]
Related GAO Products:
Business Modernization: Disciplined Processes Needed to Better Manage
NASA's Integrated Financial Management Program.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-118]
Washington, D.C.: November 21, 2003.
Business Modernization: NASA's Integrated Financial Management Program
Does Not Fully Address External Reporting Issues.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-151]
Washington, D.C.: November 21, 2003.
Information Technology: Architecture Needed to Guide NASA's Financial
Management Modernization.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-43]
Washington, D.C.: November 21, 2003.
Business Modernization: Improvements Needed in Management of NASA's
Integrated Financial Management Program.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-507]
Washington D.C.: April 30, 2003.
Major Management Challenges and Program Risks: National Aeronautics and
Space Administration.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-114]
Washington, D.C.: January 2003.
NASA: Compliance With Cost Limits Cannot Be Verified.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-504R]
Washington, D.C.: April 10, 2002.
NASA: Leadership and Systems Needed to Effect Financial Management
Improvements.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02- 551T]
Washington, D.C.: March 20, 2002.
NASA: International Space Station and Shuttle Support Cost Limits. GAO-
01-1000R.
Washington, D.C.: August 31, 2001.
Financial Management: Misstatement of NASA's Statement of Budgetary
Resources.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-438]
Washington, D.C.: March 30, 2001.
(192117):
:
FOOTNOTES
[1] U.S. General Accounting Office, Business Modernization: NASA's
Integrated Financial Management Program Does Not Fully Address Agency's
External Reporting Issues, GAO-04-151 (Washington, D.C.: Nov. 21,
2003).
[2] FFMIA requires auditors to report whether agencies' financial
management systems comply with federal financial management systems
requirements, applicable federal accounting standards (U.S. generally
accepted accounting principles), and the U.S. Government Standard
General Ledger at the transaction level.
[3] U.S. General Accounting Office, Business Modernization:
Improvements Needed in Management of NASA's Integrated Financial
Management Program, GAO-03-507 (Washington D.C.: Apr. 30, 2003);
Information Technology: Architecture Needed to Guide NASA's Financial
Management Modernization, GAO-04-43 (Washington, D.C.: Nov. 21, 2003);
Business Modernization: Disciplined Processes Needed to Better Manage
NASA's Integrated Financial Management Program, GAO-04-118
(Washington, D.C.: Nov. 21, 2003); Business Modernization: NASA's
Challenges in Managing Its Integrated Financial Management Program,
GAO-04-255 (Washington, D.C.: Nov. 21, 2003); and GAO-04-151.
[4] NASA defined those in the financial accounting arena as the
system's users who, under NASA's plan, would determine the system's
requirements, guide its implementation, and define and measure its
success. Those who would benefit from the system's new capabilities
were identified as stakeholders. Under NASA's plan, they would be the
ultimate beneficiaries of the system improvements, but would not have a
role in setting requirements or measuring and determining the success
of the system's implementation.
[5] At that time, we began a special effort to review and report on the
federal program areas that our work had identified as high risk because
of vulnerabilities to waste, fraud, abuse, and mismanagement. We first
issued our High-Risk Series in December 1992 and have continued to
include NASA's contract management as an area of high risk since. See
U.S. General Accounting Office, High-Risk Series: NASA Contract
Management, GAO/HR-93-11 (Washington, D.C.: December 1992) and Major
Management Challenges and Program Risks: National Aeronautics and Space
Administration, GAO-03-114 (Washington, D.C.: January 2003).
[6] U.S. General Accounting Office, Financial Management: Misstatement
of NASA's Statement of Budgetary Resources, GAO-01-438 (Washington,
D.C.: Mar. 30, 2001).
[7] U.S. General Accounting Office, NASA: International Space Station
and Shuttle Support Cost Limits, GAO-01-1000R (Washington, D.C.: Aug.
31, 2001), and NASA: Compliance with Cost Limits Cannot Be Verified,
GAO-02-504R (Washington, D.C.: Apr. 10, 2002).
[8] We conducted a limited scope review of NASA's fiscal year 2002
financial statement audit performed by NASA's IPA, PwC, to assist in
planning future audits of the U.S. government's consolidated financial
statements. Based on our review of PwC's supporting audit evidence, we
would not have been able to rely on its work for the purpose of
fulfilling our responsibilities related to the audit of the U.S.
government's consolidated financial statements. We reported in March of
2004 to the NASA Inspector General that our review of PwC's supporting
audit evidence revealed deficiencies in audit documentation, audit
planning, and testing. Specifically, adequate audit tests were not
performed for major balance sheet line items such as Fund Balance with
Treasury; property, plant, and equipment (PP&E); and materials. It was
not our intent to determine whether the audit opinion rendered was
appropriate or to reperform any of the auditor's work. Our procedures
consisted of an evaluation of evidence obtained from the auditor's
fiscal year 2002 audit documentation and discussions with audit
personnel. We did not independently test, reperform, or make
supplemental tests of any of the account balances.
[9] GAO-04-151.
[10] The nine modules are core financial, resume management, travel
management, position description management, human resource
management, payroll, budget formulation, contract administration, and
asset management.
[11] GAO-03-507, GAO-04-43, GAO-04-151, GAO-04-118, and GAO-04-255.
[12] NASA defined those in the financial accounting arena as the
system's users who, under NASA's plan, would determine the system's
requirements, guide its implementation, and define and measure its
success. Those who would benefit from the system's new capabilities
were identified as stakeholders. Under NASA's plan, they would be the
ultimate beneficiaries of the system improvements, but would not have a
role in setting requirements or measuring and determining the success
of the system's implementation.
[13] During the time of our review, NASA was pursuing a program--known
as the Space Launch Initiative--to build a new generation of space
vehicles to replace its aging space shuttle. This was part of NASA's
broader plan for the future of space travel--known as NASA's Integrated
Space Transportation Plan. On October 21, 2002, NASA postponed further
implementation of the program to focus on defining the Department of
Defense's role, determining future requirements of the International
Space Station, and establishing the agency's future space
transportation needs. In November 2002, the administration submitted to
the Congress an amendment to NASA's fiscal year 2003 budget request to
implement a new Integrated Space Transportation Plan. The plan made
investments to extend the space shuttle's operational life and
refocused the Space Launch Initiative program on developing an
orbital space plane--which provides crew transfer capability to and
from the space station--and next generation launch technology. The
President's vision on space exploration, announced in January 2004, may
alter that plan.
[14] The term "schedule" incorporates both the concept of status of
work and whether a project or task is being completed within planned
time frames. Depending on the nature of the work being performed, the
method of measuring work progress varies. Work is measured in terms of
tasks when a specific end product or result is produced. But when work
does not produce a specific end product or result, level-of-effort or a
more time-oriented method of measurement is used.
[15] U.S. General Accounting Office, Information Technology: A
Framework for Assessing and Improving Enterprise Architecture
Management (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003).
[16] Fiscal years 2001 through 2010.
[17] GAO-04-151.
[18] U.S. General Accounting Office, Executive Guide: Creating Value
Through World-class Financial Management, GAO/AIMD-00-134 (Washington,
D.C.: April 2000). Our executive guide was based on practices used by
nine leading organizations--Boeing; Chase Manhattan Bank; General
Electric; Pfizer; Hewlett-Packard; Owens Corning; and the states of
Massachusetts, Texas, and Virginia.
