NASA
Projects Need More Disciplined Oversight and Management to Address Key Challenges
Gao ID: GAO-09-436T March 5, 2009
This testimony discusses the National Aeronautics and Space Administration's (NASA) oversight and management of its major projects. As you know, in 1990, GAO designated NASA's contract management as high risk in view of persistent cost growth and schedule slippage in the majority of its major projects. Since that time, GAO's high-risk work has focused on identifying a number of causal factors, including antiquated financial management systems, poor cost estimating, and undefinitized contracts. Because cost growth and schedule delays persist, this area - now titled acquisition management because of the scope of issues that need to be resolved - remains high risk. To its credit, NASA has recently made a concerted effort to improve its acquisition management. In 2007, NASA developed a comprehensive plan to address systemic weaknesses related to how it manages its acquisitions. The plan specifically seeks to strengthen program/project management, increase accuracy in cost estimating, facilitate monitoring of contractor cost performance, improve agency wide business processes, and improve financial management. While we applaud these efforts our recent work has shown that NASA needs to pay more attention to effective project management. It needs to adopt best practices that focus on closing gaps in knowledge about requirements, technologies, funding, time and other resources before it makes commitments to large-scale programs. For instance, the Mars Science Laboratory, which was already over budget, recently announced a 2-year launch delay. Current estimates suggest that the price of this delay may be $400 million--which drives the current project life-cycle cost estimate to $2.3 billion; up from its initial confirmation estimate of $1.6 billion. Also, in just one year, the development costs of NASA's Glory mission increased by 54 percent, or almost $100 million, because of problems NASA's contractor is having developing a key sensor. Total project costs for another project, Kepler have increased almost another $100 million within 2 fiscal years because of similar issues. Taken together, these and other unanticipated cost increases hamper NASA's ability to fund new projects, continue existing ones, and pave the way to a post-shuttle space exploration environment. Given the constrained fiscal environment and pressure on discretionary spending it is critical that NASA get the most out of its investment dollars for its space systems. The agency is increasingly being asked to expand its portfolio to support important scientific missions including the study of climate change. Therefore, it is exceedingly important that these resources be managed as effectively and efficiently as possible for success. The recent launch failure of the Orbiting Carbon Observatory is an all-too-grim reminder of how much time, hard work, and resources can be for naught when a space project cannot execute its mission.
We assessed 18 projects in NASA's current portfolio. Four were in the "formulation" phase, a time when system concepts and technologies are still being explored and 14 were in the "implementation" phase, where system design is completed, scientific instruments are integrated, and a spacecraft is fabricated. When implementation begins, it is expected that project officials know enough about a project's requirements and what resources are necessary to meet those requirements that they can reliably predict the cost and schedule necessary to achieve its goals. Reaching this point requires investment. In some cases, projects that we reviewed spent 2 to 5 years and up to $100 million or more before being able to formally set cost and schedule estimates. Ten of the projects in our assessment for which we received data and that had entered the implementation phase experienced significant cost and/or schedule growth from their project baselines.3 Based on our analysis, development costs for projects in our review increased by an average of almost 13 percent from their baseline cost estimates--all in just two or three years--including one that went up more than 50 percent. It should be noted that a number of these projects had experienced considerably more cost growth before a baseline was established in response to statutory reporting requirements. Our analysis also shows that projects in our review had an average delay of 11 months to their launch dates. We found challenges in five areas that occurred throughout the various projects we reviewed that can contribute to project cost and schedule growth. These are not necessarily unique to NASA projects and many have been identified in other weapon and space systems that we have reviewed and have been prevalent in the agency for decades. (1) Technology maturity. Four of the 13 projects in our assessment for which we received data and that had entered the implementation phase did so without first maturing all critical technologies, that is they did not know that technologies central to the project's success could work as intended before beginning the process of fabricating the spacecraft. (2) Design stability. The majority of the projects in our assessment that held a critical design review did so without first achieving a stable design. If design stability is not achieved, but product development continues, costly re-designs to address changes to project requirements and unforeseen challenges can occur. (3) Complexity of heritage technology. More than half the projects in the implementation phase--eight of them--encountered challenges in integrating or modifying heritage technologies. (4) Contractor performance. Six of the seven projects that cited contractor performance as a challenge also experienced significant cost and/or schedule growth. (5) Development partner performance. Five of the thirteen projects we reviewed encountered challenges with a development partner. In these cases, the development partner could not meet its commitments to the project within planned timeframes. This may have been a result of problems within the specific development partner organization or as a result of problems faced by a contractor to that development partner.
GAO-09-436T, NASA: Projects Need More Disciplined Oversight and Management to Address Key Challenges
This is the accessible text file for GAO report number GAO-09-436T
entitled 'NASA: Projects Need More Disciplined Oversight and Management
to Address Key Challenges' which was released on March 5, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Space and Aeronautics, Committee on Science
and Technology, House of Representatives:
United States Government Accountability Office:
GAO:
For Release on Delivery:
Expected at 10:00 a.m. EST:
Thursday, March 5, 2009:
NASA:
Projects Need More Disciplined Oversight and Management to Address Key
Challenges:
Statement of Cristina Chaplain, Director:
Acquisition and Sourcing Management:
GAO-09-436T:
Madam Chairwoman and Members of the Subcommittee:
Thank you for inviting me to discuss the National Aeronautics and Space
Administration's (NASA) oversight and management of its major projects.
As you know, in 1990, GAO designated NASA's contract management as high
risk in view of persistent cost growth and schedule slippage in the
majority of its major projects. Since that time, GAO's high-risk work
has focused on identifying a number of causal factors, including
antiquated financial management systems, poor cost estimating, and
undefinitized contracts. Because cost growth and schedule delays
persist, this area - now titled acquisition management because of the
scope of issues that need to be resolved - remains high risk.
To its credit, NASA has recently made a concerted effort to improve its
acquisition management. In 2007, NASA developed a comprehensive plan to
address systemic weaknesses related to how it manages its acquisitions.
The plan specifically seeks to strengthen program/project management,
increase accuracy in cost estimating, facilitate monitoring of
contractor cost performance, improve agency wide business processes,
and improve financial management.
While we applaud these efforts our recent work has shown that NASA
needs to pay more attention to effective project management. It needs
to adopt best practices that focus on closing gaps in knowledge about
requirements, technologies, funding, time and other resources before it
makes commitments to large-scale programs. For instance, the Mars
Science Laboratory, which was already over budget, recently announced a
2-year launch delay. Current estimates suggest that the price of this
delay may be $400 million--which drives the current project life-cycle
cost estimate to $2.3 billion; up from its initial confirmation
estimate of $1.6 billion. Also, in just one year, the development costs
of NASA's Glory mission increased by 54 percent, or almost $100
million, because of problems NASA's contractor is having developing a
key sensor. Total project costs for another project, Kepler have
increased almost another $100 million within 2 fiscal years because of
similar issues. Taken together, these and other unanticipated cost
increases hamper NASA's ability to fund new projects, continue existing
ones, and pave the way to a post-shuttle space exploration environment.
Given the constrained fiscal environment and pressure on discretionary
spending it is critical that NASA get the most out of its investment
dollars for its space systems. The agency is increasingly being asked
to expand its portfolio to support important scientific missions
including the study of climate change. Therefore, it is exceedingly
important that these resources be managed as effectively and
efficiently as possible for success. The recent launch failure of the
Orbiting Carbon Observatory is an all-too-grim reminder of how much
time, hard work, and resources can be for naught when a space project
cannot execute its mission.
In response to congressional direction, we have prepared a
comprehensive report on the management and oversight of NASA's major
projects. It contains summaries of 18 projects with a combined life-
cycle cost exceeding $50 billion. It also contains an assessment of
issues affecting projects across-the-board. A copy of this report is
now available on GAO's Website [hyperlink, http://www.gao.gov].
[Footnote 1] In conducting this work, we compared projects against best
practice criteria for system development including attainment of
knowledge on technologies and design, as well as various aspects of
program management. We expect to continue this assessment on an annual
basis and to continually refine our examination so that our work can
inform your oversight and NASA's own efforts to improve in the high
risk area of acquisition management.
In responding to our report, NASA asserted that the unique nature of
its work and external factors beyond its control make it difficult to
apply the same criteria that we apply to other major government
acquisitions, particularly those with large production runs. We
disagree. The criteria we used to assess NASA's projects represent
commonly accepted, fundamental tenets of disciplined project
management, regardless of complexity or quantity. In fact, the concept
of the knowledge-based approach we use has been adopted in NASA's own
acquisition policy. Key criteria that we use have been developed by
NASA and/or incorporated into its engineering policy. Moreover, facing
long-standing cost and schedule growth and performance shortfalls, the
Department of Defense (DOD) acknowledges the need for a knowledge based
approach in the Air Force's "back to basics" policy for space systems.
Lastly, we remain open to discussions with NASA as to whether
additional criteria can and should be applied to its systems to ensure
that decisions to move forward in development are well-informed and
ultimately, that taxpayer dollars are well spent.
Today I will be highlighting the results of this work, the actions NASA
is taking to address the concerns raised in our high risk report and
better position its projects to meet their goals, and what we believe
is necessary to make these actions successful. Because we also have
responsibility for examining military space systems, we will also
highlight common challenges with space acquisitions within NASA and the
Department of Defense (DOD). This testimony is based on previously
issued GAO work, which was conducted in accordance with generally
accepted government auditing standards.
Acquisition Management Problems Persist:
We assessed 18 projects in NASA's current portfolio. Four were in the
"formulation" phase, a time when system concepts and technologies are
still being explored and 14 were in the "implementation" phase[Footnote
2], where system design is completed, scientific instruments are
integrated, and a spacecraft is fabricated. When implementation begins,
it is expected that project officials know enough about a project's
requirements and what resources are necessary to meet those
requirements that they can reliably predict the cost and schedule
necessary to achieve its goals. Reaching this point requires
investment. In some cases, projects that we reviewed spent 2 to 5 years
and up to $100 million or more before being able to formally set cost
and schedule estimates.
Ten of the projects in our assessment for which we received data and
that had entered the implementation phase experienced significant cost
and/or schedule growth from their project baselines.[Footnote 3] Based
on our analysis, development costs for projects in our review increased
by an average of almost 13 percent from their baseline cost estimates-
-all in just two or three years--including one that went up more than
50 percent. It should be noted that a number of these projects had
experienced considerably more cost growth before a baseline was
established in response to statutory reporting requirements. Our
analysis also shows that projects in our review had an average delay of
11 months to their launch dates.
We found challenges in five areas that occurred throughout the various
projects we reviewed that can contribute to project cost and schedule
growth. These are not necessarily unique to NASA projects and many have
been identified in other weapon and space systems that we have reviewed
and have been prevalent in the agency for decades.
* Technology maturity. Four of the 13 projects in our assessment for
which we received data and that had entered the implementation phase
did so without first maturing all critical technologies, that is they
did not know that technologies central to the project's success could
work as intended before beginning the process of fabricating the
spacecraft. This means that knowledge needed to make these technologies
work remained unknown well into development. Consequences accrue to
projects that are still working to mature technologies well into system
development, when they should be focusing on maturing system design and
preparing for production. Simply put, projects that start with mature
technologies experience less cost growth than those that start with
immature technologies.
* Design stability. The majority of the projects in our assessment that
held a critical design review did so without first achieving a stable
design. If design stability is not achieved, but product development
continues, costly re-designs to address changes to project requirements
and unforeseen challenges can occur. All of the projects in our
assessment that had reached their critical design review and that
provided data on engineering drawings experienced some growth in the
total number of design drawings after their critical design review.
Growth ranged from 8 percent to, in the case of two projects, well over
100 percent. Some of this increase can be attributed to change in
system design after critical design review.
* Complexity of heritage technology. More than half the projects in the
implementation phase--eight of them--encountered challenges in
integrating or modifying heritage technologies. Additionally, two
projects in formulation--Ares I and Orion--also encountered this
problem. We found that the projects that relied on heritage
technologies underestimated the effort required to modify them to the
necessary form, fit, or function.
* Contractor performance. Six of the seven projects that cited
contractor performance as a challenge also experienced significant cost
and/or schedule growth. For example, through our discussions with the
project offices, we were informed that contractors encountered
technical and design problems with hardware that disrupted development
progress.
* Development partner performance. Five of the thirteen projects we
reviewed encountered challenges with a development partner. In these
cases, the development partner could not meet its commitments to the
project within planned timeframes. This may have been a result of
problems within the specific development partner organization or as a
result of problems faced by a contractor to that development partner.
Common Acquisition Management Challenges Persist between NASA and DOD:
The challenges we identified in the NASA assessment are similar to ones
we have identified in other weapon systems, including Defense space
systems. We testified last year that DOD space system cost growth was
attributable to programs starting before they have assurance that
capabilities being pursued can be achieved within available resources
and time constraints. For example, DOD's National Polar-orbiting
Operational Environmental Satellite System (NPOESS) has increased in
cost from roughly $6 billion to over $11 billion because of challenges
with maturing key technologies. We have also tied acquisition problems
in space systems to inadequate contracting strategies and contract and
program management weaknesses. Further, we issued a report in 2006 that
found DOD space system cost estimates were consistently optimistic. For
example, DOD's Space-Based Infrared High program was originally
expected to cost about $4 billion and is now expected to cost more than
$10 billion.
We have found these problems are largely rooted in the failure to match
the customer's needs with the developer's resources--technical
knowledge, timing, and funding--when starting product development. In
other words, commitments were made to achieving certain capabilities
without knowing whether technologies and/or designs being pursued could
really work as intended. Time and costs were consistently
underestimated. As we have discussed in previous work on space systems
at both DOD and NASA, a knowledge-based approach to acquisitions,
regardless of the uniqueness or complexity of the system is beneficial
because it allows program managers the opportunity to gain enough
knowledge to identify potential challenges earlier in development and
make more realistic assumptions about what they can achieve.
NASA Is Making a Concerted Effort to Reduce High Risk in Acquisition
Management but More Needs to Be Done:
NASA has also taken significant steps to improve in the high risk area
of acquisition management. For example, NASA revised its acquisition
and engineering polices to incorporate elements of a knowledge-based
approach that should allow the agency to make informed decisions. The
agency is also instituting a new approach whereby senior leadership is
reviewing acquisition strategies earlier in the process and has
developed broad procurement tenets to guide the agency's procurement
practices. Further, NASA is working to improve management oversight of
project cost, schedule, and technical performance with the
establishment of a baseline performance review with senior management.
In order to improve it's contracting and procurement process, NASA has
instituted an agency wide standard contract-writing application
intended to ensure all contracts include the most up-to-date NASA
contract clauses and to improve the efficiency of the contracting
process. NASA is also requiring project managers to quantify the
program risks they identify and collect more consistent data on project
cost and technologies. It is taking other actions to enhance cost
estimating methodologies and to ensure that independent estimates are
used.
These changes brought the policy more in line with best practices for
product development. However, as we previously reported,[Footnote 4]
NASA lacks defined requirements across centers and mission directorates
for consistent metrics that demonstrate knowledge attainment through
the development cycle. In order for a disciplined approach to take
hold, we would expect project officials across the agency to be held
accountable for following the same required policies.
More steps also need to be taken to manage risk factors that NASA
believes are outside of its control. NASA asserts that contractor
deficiencies, launch manifest issues, partner performance, and funding
instability are to blame for the significant cost and schedule growth
on many of its projects that we reviewed. Such unforeseen events,
however, should be addressed in project-level, budgeting and resource
planning through the development of adequate levels of contingency
funds. NASA cannot be expected to predict unforeseen challenges, but
being disciplined while managing resources, conducting active oversight
of contractors, and working closely with partners can put projects in a
better position to mitigate these risks should they occur.
Realistically planning for and retiring technical or engineering risks
early in product development allows the project to target reserves to
issues NASA believes are outside of its control.
In conclusion, managing resources as effectively and efficiently as
possible is more important than ever for NASA. The agency is
undertaking a new multi-billion dollar program to develop the next
generation of spacecraft for human spaceflight and at a time when it is
faced with increasing demands to support important scientific missions,
including the study of climate change, and to increase aeronautics
research and development. By allowing major investment commitments to
continue to be made with unknowns about technology and design
readiness, contractor capabilities, requirements, and/or funding, NASA
will merely be exacerbating the inherent risks it already faces in
developing and delivering new space systems. Programs will likely
continue to experience problems that require more time and money to
address than anticipated. Over the long run, the extra investment
required to address these problems may well prevent NASA from pursuing
more critical science and space exploration missions. By contrast, by
continuing to implement its acquisition management reforms and ensuring
programs do not move forward with such unknowns, NASA can better align
customer expectations with resources, minimize problems that could hurt
programs, and maximize it ability to meet increased demands.
Madam Chairwoman, this concludes my statement. I will be happy to
answer any questions that you have.
GAO Contacts and Staff Acknowledgments:
For additional information, please contact Cristina Chaplain at 202-
512-4841 or chaplainc@gao.gov. Individuals making contributions to this
testimony include Jim Morrison, Assistant Director; Shelby S. Oakley,
Assistant Director; Greg Campbell; Richard A. Cederholm; Brendan S.
Culley; Deanna R. Laufer; Kenneth E. Patton; and Letisha T. Watson.
[End of section]
Footnotes:
[1] We only received data for 13 of the 14 projects in implementation.
NASA did not provide cost or schedule data for the James Webb Space
Telescope, which is in implementation.
[2] For purposes of our analysis, significant cost and schedule growth
occurs when a project's cost and/or its schedule growth exceeds the
thresholds established for Congressional reporting per the National
Aeronautics and Space Administration Authorization Act of 2005, Pub. L.
No. 109-161, §103; 42 U.S.C. §16613 (b), (f) (4).
[3] GAO, NASA: Sound Management and Oversight Key to Addressing Crew
Exploration Vehicle Project Risks. [hyperlink,
http://www.gao.gov/products/GAO-06-1127T] (Washington, D.C.: Sep. 28,
2006).
[4] GAO, NASA: Assessments of Selected Large-Scale Projects.
[hyperlink, http://www.gao.gov/products/GAO-09-306SP] (Washington,
D.C.: Mar. 2, 2009).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: