NASA
Long-Term Commitment to and Investment in Space Exploration Program Requires More Knowledge Gao ID: GAO-06-817R July 17, 2006The National Aeronautics and Space Administration (NASA) plans to spend nearly $230 billion over the next two decades implementing the Vision for Space Exploration. In January 2006, NASA publicly released its Exploration Systems Architecture Study (ESAS), which is an effort to identify the best architecture and strategy to implement the President's 2004 Vision for Space Exploration (Vision). The cost estimate for implementing the ESAS through fiscal year 2011 exceeds $31 billion. The estimate through fiscal year 2018 is over $122 billion, and the estimate through fiscal year 2025 is nearly $230 billion. These estimates include the architecture, robotic precursor missions, supporting technologies, and funding needed to service the International Space Station (ISS). NASA plans to implement this architecture through a "go as you can afford to pay" approach, wherein lower-priority efforts would be deferred, descoped, or discontinued to allow NASA to stay within its available budget profile. This approach assumes NASA's budget will increase moderately to keep pace with inflation. Given the long-term fiscal imbalances that will challenge the entire federal government now and in the future, it would be prudent for NASA to establish a program that reduces the risk that significant additional funding, beyond moderate increases for inflation, will be required to execute the program. Government leaders will have to make difficult decisions to resolve such challenges, and the debate over the potential cost and the federal government's role in implementing the Vision are emblematic of the challenges the nation will need to resolve in the years ahead. Because of the significance of this investment, competing demands on the federal discretionary budget, and the importance of the success of NASA's exploration program to the future of U.S. human spaceflight, Congress requested that GAO assess (1) the extent to which NASA has identified the architecture and costs necessary to implement the Vision, (2) whether NASA's exploration architecture cost estimates fit within the agency's projected available budgets, and (3) the risks associated with NASA's acquisition strategy for the Crew Exploration Vehicle (CEV) project.
Although NASA is continuing to refine its exploration architecture cost estimates, the agency cannot at this time provide a firm estimate of what it will take to implement the architecture. The absence of firm cost estimates is mainly due to the fact that the program is in the early stages of its life cycle. NASA will be challenged to implement the architecture recommended in the study within its projected budget. Whether using the architecture study estimates of funds available or NASA's Fiscal Year 2007 Budget Submission for ESMD that was based on the architecture study cost estimates, there are years when NASA does not have sufficient funding to implement the architecture. NASA's current acquisition strategy for the CEV places the project at risk of significant cost overruns, schedule delays, and performance shortfalls because it commits the government to a long-term product development effort before establishing a sound business case.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-817R, NASA: Long-Term Commitment to and Investment in Space Exploration Program Requires More Knowledge
This is the accessible text file for GAO report number GAO-06-817R
entitled 'NASA: Long-Term Commitment to and Investment in Space
Exploration Program Requires More Knowledge' which was released on July
26, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
July 17, 2006:
The Honorable Sherwood Boehlert:
Chairman:
The Honorable Bart Gordon:
Ranking Minority Member:
Committee on Science:
House of Representatives:
Subject: NASA: Long-Term Commitment to and Investment in Space
Exploration Program Requires More Knowledge:
The National Aeronautics and Space Administration (NASA) plans to spend
nearly $230 billion over the next two decades implementing the Vision
for Space Exploration. In January 2006, NASA publicly released its
Exploration Systems Architecture Study (ESAS), which is an effort to
identify the best architecture and strategy to implement the
President's 2004 Vision for Space Exploration (Vision)[Footnote 1]. The
cost estimate for implementing the ESAS through fiscal year 2011
exceeds $31 billion. The estimate through fiscal year 2018 is over $122
billion, and the estimate through fiscal year 2025 is nearly $230
billion[Footnote 2]. These estimates include the architecture, robotic
precursor missions, supporting technologies, and funding needed to
service the International Space Station (IS[Footnote 3]S). NASA plans
to implement this architecture through a "go as you can afford to pay"
approach, wherein lower-priority efforts would be deferred, descoped,
or discontinued to allow NASA to stay within its available budget
profile. This approach assumes NASA's budget will increase moderately
to keep pace with inflation. Given the long-term fiscal imbalances that
will challenge the entire federal government now and in the future, it
would be prudent for NASA to establish a program that reduces the risk
that significant additional funding, beyond moderate increases for
inflation, will be required to execute the program[Footnote 4].
Government leaders will have to make difficult decisions to resolve
such challenges, and the debate over the potential cost and the federal
government's role in implementing the Vision are emblematic of the
challenges the nation will need to resolve in the years ahead.
Because of the significance of this investment, competing demands on
the federal discretionary budget, and the importance of the success of
NASA's exploration program to the future of U.S. human spaceflight, you
requested that we assess (1) the extent to which NASA has identified
the architecture and costs necessary to implement the Vision, (2)
whether NASA's exploration architecture cost estimates fit within the
agency's projected available budgets, and (3) the risks associated with
NASA's acquisition strategy for the CEV project.
We presented our preliminary findings to your staff in May 2006.
Because of your committee's interest in how NASA is implementing the
Vision for Space Exploration, we are enclosing the full briefing that
supported that May presentation with this report (see encl. II), along
with a summary of our findings and conclusions. We are recommending
that the NASA Administrator modify the current CEV acquisition strategy
to ensure that the agency does not commit itself, and in turn the
federal government, to a long-term contractual obligation prior to
establishing a sound business case at the project's preliminary design
review. In written comments, NASA nonconcurred with our recommendation
and stated that it has the appropriate level of knowledge to proceed
with its current acquisition strategy. As a result of its
nonconcurrence, we are including as a matter for congressional
consideration that the Congress should consider restricting NASA's
appropriations and obligations for the CEV project to only the amount
of funding necessary to successfully complete the project's preliminary
design review.
Background:
The Vision includes plans to explore the moon, Mars, and
beyond.[Footnote 5] The first step in implementing the Vision is to
retire the space shuttle after completing assembly of the ISS by the
end of the decade. NASA currently plans to retire the space shuttle in
2010, creating a potential gap in U.S. human spaceflight of up to 4
years before development of the CEV and the CLV is complete. Congress
has voiced concern over the United States not having continuous access
to space, and NASA has made it a priority to minimize the gap by
accelerating the CEV project to have it in service as close to 2010 as
possible. NASA's Exploration Systems Mission Directorate's (ESMD)
Constellation program is responsible for the development of both the
CEV and the CLV. NASA awarded concept development contracts for the CEV
project to both Lockheed Martin and Northrop Grumman in July 2005 and
plans to award a contract for design, development, production and
sustainment in September 2006. That contract could extend through 2019.
For the CLV, NASA plans to award a sole-source contract for the first
stage of the CLV to ATK-Thiokol, the manufacturer of the Shuttle's
Reusable Solid Rocket Motor, in October 2006. Also, the agency plans to
award Pratt & Whitney Rocketdyne, the developer of the Space Shuttle
Main Engine (SSME) and J-2 engines, a sole-source contract for
development of the J-2X engine in November 2006. These contractors are
currently planning their respective efforts under interim contract
arrangements. NASA has started in-house preliminary design work on the
CLV upper stage structures and avionics and plans to begin awarding
competitive contracts for production of these items in May 2007.
Despite many successes in the exploration of space, such as landing the
Pathfinder and Exploration Rovers on Mars, the loss of life,
unsuccessful missions, and unforeseen cost overruns have recently
increased the level of concern over the benefits of such exploration,
particularly with regard to human spaceflight activities. NASA has had
difficulty bringing a number of projects to completion, including
several efforts to build a second generation of reusable human
spaceflight vehicle to replace the space shuttle. NASA has attempted
several expensive endeavors such as the National Aero-Space Plane, the
X-33 and X-34, and the Space Launch Initiative, among others. While
these endeavors have helped to advance scientific and technical
knowledge, none have completed their objective of fielding a new
reusable space vehicle. We estimate that these unsuccessful development
efforts have cost approximately $4.8 billion since the 1980s. The high
cost of these unsuccessful efforts and the potential costs of
implementing the Vision make it important that NASA achieve success in
its new exploration program.
Our past work has shown that developing a sound business case, based on
matching requirements to available and reasonably expected resources
before committing to a new product development effort, reduces risk and
increases the likelihood of successful outcomes.[Footnote 6] At the
heart of a business case is a knowledge-based approach to product
development that is a best practice among leading commercial firms and
successful government system developers. For a program to increase its
chances of delivering a successful product, high levels of knowledge
should be demonstrated before managers make significant program
commitments. In essence, knowledge supplants risk over time. This
building of knowledge can be described as three levels that should be
attained over the course of the program:
(1) At program start, the customer's needs should match the developer's
available resources in terms of availability of mature technologies,
time, human capital, and funding.
(2) Midway through development, the product's design should be stable
and demonstrate that it is capable of meeting performance requirements.
(3) By the time of the production decision, the product must be shown
to be producible within cost, schedule, and quality targets, and have
demonstrated its reliability.
Our work has shown that programs that have not attained the level of
knowledge needed to support a sound business case have been plagued by
cost overruns, schedule delays, decreased capability, and overall poor
performance. With regard to NASA, we have reported that in some cases
the agency's failure to define requirements adequately and develop
realistic cost estimates--two key elements of a business case--resulted
in projects costing more, taking longer, and achieving less than
originally planned.
Summary:
Although NASA is continuing to refine its exploration architecture cost
estimates, the agency cannot at this time provide a firm estimate of
what it will take to implement the architecture. The absence of firm
cost estimates is mainly due to the fact that the program is in the
early stages of its life cycle. According to NASA cost-estimating
guidance, early life cycle phase estimates are generally based upon
parametric models, which use data from projects with similar attributes
to predict cost because there are usually many unknowns and actual cost
or performance data are not available. NASA preliminarily identified
the resources needed to implement the architecture as outlined in the
architecture study primarily through the use of such models. NASA
conducted a cost risk analysis of its preliminary estimates through
fiscal year 2011. On the basis of this analysis and through the
addition of programmatic reserves (20 percent on all development and 10
percent on all production costs), NASA is 65 percent confident that the
actual cost of the program will either meet or be less than its
estimate of $31.2 billion through fiscal year 2011. For the cost
estimates for beyond 2011, when most of the cost risk for implementing
the architecture will be realized, NASA has not applied a confidence
level distinction. Since NASA released its preliminary estimates, the
agency has continued to make architecture changes. For example,
following the issuance of the architecture study, NASA conducted
several analysis cycles during which various aspects of the
architecture have evolved, such as the diameter of the CEV, the engine
used to support the upper stage of the CLV, and the size of the
Reusable Solid Rocket Booster on the CLV. While these changes, and
others, are appropriate for this phase of the program, when concepts
are still being developed, they leave the agency in the position of
being unable to firmly identify program requirements and needed
resources, which can also be expected at this phase of the program.
According to NASA officials, once they receive more detailed contractor
inputs, the agency will be able to produce higher-fidelity estimates of
program cost. NASA plans to commit to a firm cost estimate at the
preliminary design review (PDR) in 2008, when the program's
requirements, design, and schedule will all be baselined.
NASA will be challenged to implement the architecture recommended in
the study within its projected budget. Whether using the architecture
study estimates of funds available or NASA's Fiscal Year 2007 Budget
Submission for ESMD that was based on the architecture study cost
estimates, there are years when NASA does not have sufficient funding
to implement the architecture. Some yearly shortfalls exceed $1
billion, while in other years the funding available exceeds needed
resources. NASA maintains that the architecture could be implemented
within the projected available budgets through fiscal year 2011 when
funding is considered cumulatively. In addition, NASA preliminarily
projects multibillion-dollar shortfalls for ESMD in all fiscal years
from 2014 to 2020, with an overall deficit through 2025 of over $18
billion. In the short term, NASA is attempting to address this problem
within the Constellation program by redirecting funds to that program
from other ESMD activities to provide a significant surplus for fiscal
years 2006 and 2007 to cover projected shortfalls beginning in fiscal
year 2009. In addition, the Constellation program has requested more
funds than required for its projects in several early years to cover
shortfalls in later years. For example, the Exploration Communication
and Navigation Systems project within the Constellation program plans
to roll over $56.2 million from the fiscal year 2007 budget to make up
for budget shortfalls in fiscal years 2008, 2009, and 2010. NASA
officials stated the identified budget phasing problem could worsen
given that changes made to the exploration architecture following
issuance of the study will likely add to the near-term development
costs, where the funding is already constrained. In addition, NASA's
estimates beyond 2010 are based upon a surplus of well over $1 billion
in fiscal year 2011 due to the retirement of the space shuttle fleet in
2010. However, NASA officials said the costs for retiring the space
shuttle and transitioning to the new program are not fully understood,
and thus the expected surplus could be less than anticipated.
NASA's current acquisition strategy for the CEV places the project at
risk of significant cost overruns, schedule delays, and performance
shortfalls because it commits the government to a long-term product
development effort before establishing a sound business case. NASA
plans to award a contract for the design, development, production, and
sustainment of the CEV in September 2006--before it has developed key
elements of a sound business case, including well-defined requirements,
a preliminary design, mature technology, and firm cost estimates. The
period of performance for the contract scheduled for award in September
2006 will extend through at least 2014, with the possibility of
extending through 2019. This contract will comprise all design,
development, and test and evaluation activities, including production
of ground and flight test articles and at least four operational CEVs.
Although NASA is committing to a long-term contract, it will not have
the elements of a sound business case in place until the project level
PDR in fiscal year 2008. Awarding a contract for design, development,
production, and sustainment of the project as NASA has planned places
the CEV project at increased risk of cost growth, schedule delays, and
performance shortfalls. At PDR, NASA will likely (a) have the increased
knowledge necessary to develop a sound business case that includes high-
fidelity, engineering-based estimates of life cycle cost for the CEV
project, (b) be in a better position to commit the government to a long-
term effort, and (c) have more certainty in advising Congress on
required resources.
Implementing the Vision over the coming decades will require hundreds
of billions of dollars and a sustained commitment from multiple
Administrations and Congresses over the length of the program. The
realistic identification of the resources needed to achieve the
agency's short-term goals would provide support for such a sustained
commitment over the long term. With a range of federal commitments
binding the fiscal future of the United States, competition for
resources within the federal government will only increase over the
next several decades. Consequently, it is incumbent upon NASA to ensure
that it is wisely investing its existing resources. As NASA begins to
implement the Vision with several key acquisition decisions planned to
occur this fall, it will be essential that the agency ensure that the
investment decisions it is making are sound and are based upon high
levels of knowledge. NASA should make the prudent decision now to
ensure that it has attained the appropriate level of knowledge to
support a sound business case before it commits to the project.
However, under the current acquisition strategy for CEV, key knowledge-
-including well-defined requirements, a preliminary design, mature
technology, and firm cost estimates--will not be known until over a
year after the expected contract award date. Nevertheless, NASA plans
to commit the government to a long-term contract. This approach
increases the risk that the project will encounter significant cost
overruns, schedule delays, and decreased capability. Given the nation's
fiscal challenges and those that exist within NASA, the availability of
significant additional resources to address such issues, should they
occur, is unlikely. With the impending decisions pertaining to the CEV,
NASA has the opportunity to establish a firm foundation for the entire
Constellation program by ensuring that the appropriate level of
knowledge is available before proceeding with its acquisition strategy
and committing the government to a long-term design, development, and
production effort.
Recommendation for Executive Action:
Because of the importance of the CEV project to NASA's overall
implementation of the Vision, NASA should focus on ensuring that its
acquisition approach for the CEV project does not place the government
at risk by committing to a long-term design and development effort
without the knowledge needed to make wise investment decisions. We
therefore recommend that the NASA Administrator modify the current CEV
acquisition strategy to ensure that the agency does not commit itself,
and in turn the federal government, to a long-term contractual
obligation prior to demonstrating, through the establishment of a sound
business case at the project's preliminary design review, that the
project is affordable and executable.
Matter for Congressional Consideration:
Based on its response to our report, it appears that NASA plans to
proceed with its acquisition strategy for the CEV and award a long-term
contract for the project, although it continues to lack sufficient
knowledge and a sound business case for doing so. Congress is currently
being asked to approve NASA's fiscal year 2007 funding request and will
be asked to approve fiscal year 2008 and perhaps the fiscal year 2009
funding requests for the CEV project before NASA has demonstrated such
knowledge and has provided evidence, based on that knowledge, that the
project will be executable within existing and expected resources. In
light of the fact that NASA plans to award the contract for the CEV in
September 2006, Congress should consider restricting annual
appropriations and limiting NASA's obligations for the CEV project to
only the amount of funding necessary to support activities needed to
successfully complete the project's preliminary design review.
Agency Comments and Our Evaluation:
In written comments on a draft of this report (see encl. I), NASA
nonconcurred with our recommendation that it modify the current CEV
acquisition strategy to ensure that the agency does not commit itself,
and in turn the federal government, to a long-term contractual
commitment prior to establishing a sound business case at the project's
preliminary design review. NASA stated that it has the appropriate
level of knowledge to proceed with its acquisition plan to "down
select" to a single Crew Exploration Vehicle prime contractor in
September 2006. NASA added that it is maximizing competition by
soliciting from industry a development, production, and management
approach with an emphasis on life cycle cost. In the area of technology
maturity, NASA stated that it has a plan and process in place to
address the Thermal Protection and Landing subsystems technology risks
through in-house development work and collaboration with the prime
contractor. NASA also noted that during its design, development, and
test and evaluation effort, the agency will be using an end-item award
fee, which would make all award fees subject to a final evaluation to
determine how well the product met requirements, including cost and
schedule.
The CEV acquisition strategy is not knowledge-based in that it calls
for maturing technologies, designing systems, and preparing for initial
production concurrently--an approach that our work has shown carries
the increased risk of cost and schedule overruns and decreased
technical capability. Therefore, we disagree with NASA's statement that
it has the appropriate level of knowledge to proceed with its current
acquisition strategy and award a long-term contract for the project
prior to obtaining sufficient knowledge. Specifically:
² In its response, NASA suggests that there would be no benefit in
retaining two prime contractors for the CEV project through the
preliminary design review and that the best return on its investment
would be gained by down-selecting to one contractor and awarding the
contract in September 2006. Contrary to NASA's response, addressing our
recommendation would not preclude the agency from down-selecting to one
contractor. The thrust of our recommendation is that NASA should lessen
the government's obligation to the project at such an early stage when
realistic cost estimates have yet to be established and requirements
are not fully defined, and therefore limit the scope of the contract to
activities needed to successfully complete the preliminary design
review. At that point the project should have in place a sound business
case for proceeding and hence be in a better position to justify
continued investment. Implementation of the recommendation could be
accomplished through various means, including by retaining two
contractors through the preliminary design review and awarding a
contract at that time or by down-selecting as planned in September 2006
and limiting the scope of the contract as described above.
* NASA's suggestion that it is maximizing competition by soliciting
from industry its development, production, and management approach and
that it will receive firm competitive prices from industry for
completion of development and demonstration of two vehicles has little
basis. First, while the current structure will allow for competition in
the short term, the benefits of such competition will be short-lived.
Without well-defined requirements, mature technologies, an approved
preliminary design, and realistic cost estimates, NASA has insufficient
information to ensure that it is obtaining firm competitive prices for
the work conducted for the entirety of Schedule A--especially for
activities beyond the project's preliminary design review.
Because NASA continues to refine the project's requirements, as
demonstrated by the numerous changes to the exploration architecture as
discussed in our report, it cannot provide a firm estimate of project
cost. Without such information, it will likely be difficult for NASA to
establish realistic "not-to-exceed" prices for Schedule B activities.
Under the current strategy, NASA will not have high-fidelity,
engineering-based estimates of life cycle costs for the CEV until the
preliminary design review. As outlined in this report, projects with
cost estimates based on early, evolving designs and top-level
requirements are at increased risk of cost growth relative to estimates
based on mature designs and detailed requirements--which could be
achieved at the preliminary design review. According to NASA, it plans
to obtain this and further knowledge about program cost, schedule, and
risk elements following the contract award and in conjunction with the
contractor. In the absence of such information, it is not clear how
NASA can substantiate its statement that it has the knowledge necessary
to commit to activities beyond the project's preliminary design review.
Further, it cannot provide Congress with assurance of the
appropriateness of requested funding for the project.
² NASA stated that its current acquisition strategy for the CEV
minimizes the government's obligation during development by dividing
the CEV contract into three separate schedules. All three schedules,
however, will be awarded in September 2006 as part of one contract.
Although NASA plans to include language in the negotiated CEV contract
to state that the minimum quantity under Schedule B will not be
applicable until that schedule's period of performance begins in 2009-
-a step that would lessen the government's obligation during
production--it will continue to be responsible for all Schedule A
activities at the time of contract award. These activities include all
design, development, and test and evaluation activities, as well as the
production of two operational vehicles. Contractually obligating the
government to even these Schedule A activities, before it has
established a sound business case to support such a commitment, is not
in line with our knowledge-based approach and is ultimately not in the
best interest of the government.
* NASA's investment in identifying and maturing the Thermal Protection
and Landing Subsystems is a step in the right direction to ensure that
these technologies are mature and available when needed. NASA has no
guarantee, however, that these critical technologies will be mature by
the time of the project's preliminary design review--the point at which
our work has shown that technologies should be mature in order to
decrease the risk of cost and schedule growth. NASA's proposed
commitment to the project for activities beyond the preliminary design
review before retiring these technology risks increases the likelihood
that the project will experience schedule delays and cost overruns.
* NASA maintains that program risks have been marginalized and that the
agency will utilize incentives, including end-item award fees, to
ensure contractor performance. NASA suggests that the incentives it
plans to use in the form of end-item award fees will be a powerful tool
for meeting cost schedule, technical, and quality goals. The use of
these tools, however, does not compensate for proceeding with a risky
acquisition, nor do they lessen NASA's responsibility to implement an
executable program from the start. For them to function as intended,
NASA needs to address the more fundamental issues related to its
acquisition strategy, including its lack of a sound business case for
the CEV project.
* Finally, the use of cost-reimbursable contracting, while appropriate
for early development and design efforts, places most of the cost risk
for the project on the government. Given the nature of this effort, it
is likely that the project will change significantly as it moves
forward. Therefore, any scope changes or schedule slips could translate
into additional contract cost for NASA. Such cost impacts could be
minimized if NASA limited its contractual obligation to those
activities needed to achieve a successful preliminary design review, as
we recommended. In addition, limiting the scope of the CEV contract
would allow both NASA and Congress to assess the project's progress at
the preliminary design review and to decide if continued investment in
the project is prudent and in the best interest of the government.
It is important to note that Congress will continue to be asked to make
funding commitments in advance of CEV project events that would
demonstrate that the project has the knowledge necessary to support a
sound business case. Specifically, NASA's funding request for fiscal
years 2007 and 2008 are scheduled to be approved before the CEV holds
its preliminary design review. Since the preliminary design review is
currently scheduled for March 2008, this may also be the case for
fiscal year 2009. Congress should safeguard against a situation in
which contractual and budget decisions could hinder its ability to tie
further investments in the CEV project to demonstrated progress at the
preliminary design review. As such, we have included a matter for
congressional consideration.
We also received technical comments from NASA, which have been
addressed in the report, as appropriate.
Scope and Methodology:
To assess the extent to which NASA has identified the architecture and
costs necessary to implement the Vision and whether NASA's exploration
architecture cost estimates fit within the agency's projected available
budgets, we reviewed and analyzed NASA's Exploration Systems
Architecture Study, fiscal year 2007 budget request, ground rules and
assumptions provided from the Constellation program to project level
management estimators to perform the bottom up review, guidance for use
in preparing the fiscal year 2008 budget request, NASA cost-estimating
guidance in the NASA Cost Estimating Handbook, and congressional
hearings and testimonies pertaining to NASA and the Vision. We also
conducted interviews with NASA headquarters officials from the Cost
Analysis Division, the Exploration Systems Mission Directorate, and
Constellation program officials, Constellation program and CEV project
officials at Johnson Space Center; CLV project officials at Marshall
Space Flight Center; and cost analysts from the Kennedy Space Center.
During these interviews, we discussed the methodologies used in
preparing the ESAS and subsequent cost estimates, architecture changes
after the ESAS and the trades being considered, budgeting issues, and
procurement strategies and activities.
To assess the risks associated with NASA's acquisition strategy for the
CEV project, we reviewed and analyzed CEV project documentation,
including draft project plans, draft requirements documents, technology
development plans, documentation included in the contract request for
proposals, and past NASA human spaceflight acquisition programs. We
compared NASA's plans for the CEV with criteria contained in GAO best
practices work on systems acquisition. We also conducted interviews
with NASA headquarters officials from the Exploration Systems Mission
Directorate and Constellation Systems officials, Constellation program
and CEV project officials at Johnson Space Center, and CLV project
officials at Marshall Space Flight Center.
We conducted our work from January 2006 to May 2006 in accordance with
generally accepted government auditing standards.
As agreed with your offices, unless you announce its contents earlier,
we will not distribute this report further until 10 days from its date.
At that time, we will send copies of the report to NASA's Administrator
and interested congressional committees. We will also make copies
available to others upon request. In addition, the report will be
available at no charge on GAO's Web site at [Hyperlink,
http://www.gao.gov].
Should you or your staff have any questions on matters discussed in
this report, please contact me at (202) 512-4841 or lia@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. Principal
contributors to this report were James L. Morrison, Assistant Director;
Rick Cederholm; Shelby S. Oakley; Guisseli Reyes; Sylvia Schatz; and
John S. Warren, Jr.
Signed by:
Allen Li:
Director:
Acquisition and Sourcing Management:
Enclosures:
Enclosure I:
Comments from the National Aeronautics and Space Administration:
National Aeronautics and Space Administration:
Office of the Administrator:
Washington, DC 20546-0001:
July 6, 2006:
Mr. Allen Li:
Director Acquisition and Sourcing Management:
United States Government Accountability Office:
Washington, DC 20548:
Dear Mr. Li:
NASA has reviewed the Government Accountability Office (GAO) draft
report entitled "NASA: Long-Term Commitment to and Investment in Space
Exploration Program Requires More Knowledge (GAO Code 120515, Report
Number GAO-06-817R)." Thank you for the opportunity to provide comments
on the recommendation in the report.
NASA embraces GAO's recognition that a "knowledge-based" approach
reduces risks and increases the likelihood of successful outcomes. As
the primary steward for achieving the Vision for Space Exploration,
NASA fully recognizes the importance of investing its resources wisely
and maintaining stakeholder confidence in its performance. NASA has the
appropriate level of knowledge to proceed with its knowledge-and
performance-based acquisition plan to "down-select" a single Crew
Exploration Vehicle (CEV) prime contractor in September 2006. The
Agency's acquisition strategy and plans capitalize on the benefits of
competition, focus on performance, and address the inherent risk of
complex development projects. Accordingly, NASA nonconcurs with GAO's
recommendation that the NASA Administrator modify the current CEV
acquisition strategy.
NASA is maximizing competition by soliciting from industry their
development, production, and management approach with an emphasis on
Life Cycle Cost (LCC) for the CEV. While in this competitive
environment, NASA will receive firm competitive prices from industry to
complete development of the CEV and demonstrate one pressurized crew
vehicle and one pressurized unmanned vehicle. Under this competition,
NASA will also establish not-to-exceed prices for production of
required CEVs to support the current flight manifest through 2019. The
foundation of the CEV acquisition strategy is focused on gaining
commitment from industry for a design solution and controlling LCC
through competition and incentives.
NASA has diligently invested the time and resources in the formulation
phase of the CEV project in order to develop the knowledge necessary to
commit to a long-term design and development effort. In May 2005, the
Exploration Systems Architecture Study (ESAS) was initiated with one of
its tasks being to provide a complete assessment of the top-level CEV
requirements. As a result of the ESAS, the architecture and the top-
level requirements for the CEV were chosen. With the level of knowledge
gained through the Agency's investment in the ESAS and with its
acquisition strategy, NASA perceives no benefit in retaining two prime
contractors through Preliminary Design Review (PDR) at an estimated
cost of $1 billion each. Instead, NASA has determined that a better
return on its investment would be gained by competitively issuing two
Phase 1 CEV prime contracts for conceptual design and trade studies
against the ESAS architecture for an estimated cost of $46 million each
and including the CEV requirements as part of the CEV Phase 2 contract
competition. Additionally, NASA established an intra-agency CEV Smart
Buyer team which performed trade studies and design analyses that were
used by the CEV Project Office to understand and verify the
appropriateness of the requirements incorporated into the CEV Phase 2
solicitation and evaluation of proposals. With knowledge gained from
ESAS, the Smart Buyer team, and the CEV Phase 1 contracts, NASA is now
in a sound position to "down-select" a single prime contractor, thereby
base-lining an industry approach and commitment to meet the desired
outcomes of the CEV project.
NASA's business approach is consistent with GAO's knowledge-based
recommendation and recognition that knowledge replaces risk over time.
The CEV acquisition strategy contains separate contract schedules and
design reviews which are equivalent to GAO's defined:
Knowledge Points. NASA's acquisition strategy minimizes the
Government's obligation during development by dividing the CEV contract
into three different schedules:
* Schedule A for Design Development, Test, and Evaluation (DDT&E).
* Schedule B for production beyond Schedule A.
* Schedule C for sustaining activities during production and operation.
Schedule A is authorized at contract award and continues through the
first flight demonstration of each design variant of the CEV. Schedule
A executes the formulation phase of the project such that NASA can
utilize the contractor's knowledge to develop a set of validated
requirements, including component specifications and mature
technologies by the project's PDR. The current CEV strategy will allow
NASA and the contractor to attain further appreciation and knowledge
about the project and its required resources to provide firm cost,
schedule, and risk elements. At this point, the Non-Advocacy Review
(NAR) is typically scheduled immediately following the baseline of the
project's preliminary design.
Authorization of Schedule B is planned post PDR, NAR, and the Critical
Design Review (CDR) and is currently limited to a minimum production
quantity of two units. Authorization of Schedule C is planned at
approximately the same time as Schedule B. The CEV strategy does not
commit the Agency to any production until the NAR milestone is met.
Additionally, utilizing Delivery Orders (Indefinite Delivery Indefinite
Quantity) for Schedules B and C provides NASA with the ability to order
only the units and the sustaining engineering necessary, with
appropriate incentives, when the requirements and costs are better
understood by NASA and industry. NASA will not commit to Schedule B or
C activities until it is time to implement that portion of the
contract. To mitigate concerns about the minimum production quantity of
two units under the production contract (Schedule B), language will be
included in the final negotiated CEV contract that will make explicit
that the minimum quantity will not be applicable until the period of
performance of Schedule B begins. First production orders are planned
to be placed in the fall of 2009, nine months after the baseline of the
CEV's critical design or CDR (Knowledge Point 3).
NASA has a plan and process in place to address technology risks
through in-house development work and collaboration with the prime
contractor. For example, NASA has identified two areas where the
additional technology maturation is needed: the Thermal Protection
Subsystem and the Landing Subsystem. NASA has in-house advanced
development plans (ADP) to develop these technologies with Prime
Contractor participation through PDR. While the Prime Contractor will
participate with the ADP, it will not assume development responsibility
until after PDR.
Incentives are a critical element in the business case for the CEV
project. During DDT&E, NASA will use an end-item award fee. This makes
all award fees subject to final determination only after the product
has been demonstrated to meet requirements, including cost and
schedule. This is a powerful tool for the NASA project manager and
provides incentive to all elements of the project: cost (including life
cycle costs), schedule, technical, and most importantly, quality. There
will be inherent motivation toward schedule performance by means of
concluding each project milestone with an award fee determination.
Since no provisional payments will be made, industry will not receive
interim payments until the completion of an established project
milestone. A slip in schedule will be reflected both in a delay in
receipt of the interim payment and in the NASA evaluation that will
eventually follow.
In summary, NASA is confident that its acquisition strategy and plans
for selecting a CEV Prime Contractor are based on sound business case,
will establish a firm foundation for the Constellation Program, and are
in the Government's best interest.
Sincerely,
Signed by:
Shana Dale:
Deputy Administrator:
Enclosure II:
May 2006:
NASA: Long-Term Commitment to and Investment in Space Exploration
Program Requires More Knowledge:
Why GAO Did This Study:
In January 2006, the National Aeronautics and Space Administration
(NASA) publicly released its Exploration Systems Architecture Study
(ESAS), which aimed to identify the best architecture and strategy to
implement the President's 2004 Vision for Space Exploration (Vision).
The ESAS architecture supports development of a new Crew Exploration
Vehicle (CEV), Crew Launch Vehicle (CLV), Cargo Launch Vehicle (CaLV),
and other supporting systems, which are part of NASA's Exploration
Systems Mission Directorate's (ESMD) Constellation program. The
architecture also calls for various Research and Technology (R&T) and
Robotic Lunar Exploration Program (RLEP) projects.
The cost estimate for implementing the ESAS through fiscal year 2011
exceeds $31 billion. The estimate through fiscal year 2018 is $122
billion and the estimate through fiscal year 2025 is nearly $230
billion. These estimates include the architecture, robotic precursor
missions, supporting technologies, and funding needed to service the
International Space Station (ISS).
Because of the significance of this investment, competing demands on
the federal discretionary budget, and the importance of the success of
NASA's exploration program to the future of U.S. human spaceflight, we
assessed (1) the extent to which NASA has identified the architecture
and costs necessary to implement the Vision, (2) whether NASA's
exploration architecture cost estimates fit within the agency's
projected budgets, and (3) the risks associated with NASA's acquisition
strategy for the CEV.
Summary:
Although NASA is continuing to refine its exploration architecture cost
estimates, the agency cannot at this time provide a firm estimate of
what it will take to implement the architecture. The absence of firm
cost estimates is mainly due to the fact that the program is in its
early stages. NASA preliminarily identified the resources needed to
implement the architecture as outlined in the ESAS. However, since that
time, NASA has continued to make architecture changes. For example,
following the issuance of the ESAS, NASA undertook several analysis
cycles in which various aspects of the architecture have evolved, such
as the diameter of the CEV, the engine used to support the upper stage
of the CLV, and the size of the Reusable Solid Rocket Booster on the
CLV. These changes, and others, are appropriate for this phase of the
program, when concepts are being developed, but leave NASA in the
position of being unable to firmly identify program requirements and
needed resources. NASA plans to commit to a firm cost estimate at the
preliminary design review (PDR) in 2008, when the programs'
requirements, design, and schedule will all be baselined.
NASA will be challenged to implement the ESAS architecture with its
projected budget. Whether using the ESAS estimates of funds available
or NASA's fiscal year 2007 budget submission that was based upon the
ESAS estimates, there are years when NASA does not have sufficient
funding to implement the architecture. Some yearly shortfalls exceed $1
billion, while in other years the funding available exceeds needed
resources. NASA maintains that the architecture could be implemented
within its projected available budgets through fiscal year 2011 when
funding is considered cumulatively. In the short term, NASA has
redirected funds to the Constellation program from other ESMD
activities to provide a significant surplus for fiscal years 2006 and
2007 to cover projected shortfalls for the program beginning in fiscal
year 2009. The identified budget phasing problem in ESAS could worsen,
given that changes to the architecture following the ESAS will likely
add to the near term development costs, where funding is already
constrained. In addition, NASA anticipates a significant surplus in
fiscal year 2011 because of the retirement of the space shuttle fleet
in 2010. However, the transition costs are not fully understood.
NASA's acquisition strategy for the CEV places the project at risk of
cost overruns, schedule delays, and performance shortfalls because it
commits the government to a long-term product development effort before
establishing a sound business case. NASA plans to award a contract for
design, development, production, and sustainment of the CEV in
September 2006--before it has developed well-defined requirements, a
preliminary design, mature technology, and firm cost estimates. This
information is not expected until the project-level PDR in fiscal year
2008. At that point, NASA will likely (a) have the increased knowledge
necessary to develop a sound business case that includes high-fidelity,
engineering-based estimates of life cycle cost for the CEV project, (b)
be in a better position to commit the government to a long-term effort,
and (c) have more certainty in advising Congress on required resources.
Implementing the Vision:
NASA plans to bring the President's Vision to reality over the next
several decades by:
* conducting exploration activities in low-Earth orbit; for example,
flying the space shuttle to complete assembly of the ISS;
* exploring beyond low-Earth orbit; for example, establishing sustained
exploration of the moon and Mars;
* developing transportation that supports exploration; for example,
building crew exploration vehicles; and:
* pursuing opportunities for international and commercial
participation.
Exploration Contracts:
NASA awarded concept development contracts to both Lockheed Martin and
Northrop Grumman for the CEV project in July 2005. NASA plans to down-
select to one contractor and award a contract for development,
production, and sustainment of the CEV in September 2006. That contract
could extend through 2019.
NASA plans to award a sole-source contract for the first stage of the
CLV to ATK-Thiokol, the manufacturer of the Shuttle's Reusable Solid
Rocket Motor, in October 2006. Also, the agency plans to award Pratt &
Whitney Rocketdyne, the developer of the Space Shuttle Main Engine
(SSME) and J-2 engines, a sole-source contract for development of the J-
2X engine in November 2006. These contractors are currently planning
their respective efforts under interim contract arrangements. NASA has
started in-house preliminary design work on the CLV upper stage
structures and avionics and plans to begin awarding competitive
contracts for production of these items in May 2007.
Original Exploration Systems Architecture Study Overview:
The ESAS outlined the recommended architecture and strategy for
implementation of the Vision. The primary vehicles and elements of the
architecture include the CEV, the CLV, the CaLV that includes the Earth
Departure Stage (EDS), and the Lunar Surface Access Module (LSAM). The
diagram below outlines a launch mission for crew and cargo, utilizing
rendezvous locations in low-Earth and low-lunar orbits.
[See PDF for Image]
Source: NASA.
[End of Figure]
The original ESAS architecture is described below. Changes made to the
architecture since the release of ESAS are described in later sections.
CEV: The CEV is a reusable, Apollo-derived cone-shaped capsule launched
atop the CLV. The CEV consists of a Command Module (CM), a Service
Module (SM), and a Launch Abort System (LAS). The CEV is sized at 5.5
meter diameters for lunar polar missions carrying a crew of four, and
is also reconfigurable to accommodate up to six crew members for
missions to ISS. The vehicle uses a Low Impact Docking System (LIDS)
for ISS and lunar missions. The vehicle is reusable for up to 10
missions and will land on land with a water landing as a backup. The SM
utilizes a pressure-fed liquid oxygen (LOX)/methane propulsion system.
CLV: The CLV consists of a shuttle-derived four-segment Reusable Solid
Rocket Booster (RSRB) first stage and a newly designed upper stage with
one modified, and now expendable, SSME. It will launch 25 metric tons
to low-Earth orbit and serve as the long-term crew launch capability
for the United States.
CaLV: The CaLV will use a heritage shuttle external tank-derived LOX/
liquid hydrogen core stage propelled by five redesigned SSMEs. Attached
to this core stage are two newly developed five-segment RSRBs, allowing
over 100 metric tons to be launched to low-Earth orbit. The upper
stage, which also serves as the EDS, uses an external tank-derived LOX/
liquid hydrogen system and will employ two Saturn-derived J-2 engines.
LSAM: The LSAM is an expendable two-stage module launched atop the
CaLV. The descent stage will utilize a LOX/liquid hydrogen propulsion
system while the ascent stage will use a pressure-fed LOX-methane
propulsion system. A crew cabin will be located on the ascent stage and
will have an airlock to allow docking with the CEV. The LSAM will be
able to land at any location on the lunar surface and will house a four-
member crew for up to 7 days.
Cost Estimating:
Cost-Estimating Process:
NASA's Cost Estimating Handbook outlines cost-estimating processes in
relation to acquisition life cycle phases.
* In Pre-Phase A, there are many unknowns. At this point, the most
effective cost-estimating approach is a parametric or analogous
methodology, i.e., data from projects with similar attributes is used
to predict the cost.
* In Phase A, conceptual designs are better defined and a better
understanding of the system requirements and technical risks exists.
But, parametric or analogous cost-estimating techniques are still used,
because detailed data may still be unavailable.
* In Phase B, system designs are defined below the subsystem level. At
this point, estimating methodologies evolve to more detailed parametric
or engineering buildup estimates supported by technical experts. By the
end of Phase B, specific data are available to prepare a full life
cycle cost estimate.
* In Phases C and D, cost estimates are refined to include actual data.
At this point, the preferred cost methodology is an engineering buildup
based on the lowest level of detail available, including overhead,
labor, and material costs.
Firm Cost Estimates Cannot Be Developed at This Time:
NASA's cost estimates for implementing its exploration architecture are
preliminary--a fact that NASA has acknowledged since the ESAS was
publicly released. As part of the ESAS effort, NASA laid out the cost
estimates for implementing the recommended architecture. Because the
ESAS effort was an early life cycle activity, Pre-Phase A, the majority
of the individual estimates were based upon parametric models, with
little actual data.
The ESAS process evaluated the cost of various alternative exploration
architectures based upon high-level program requirements. The
recommended architecture costs totaled:
* over $31 billion dollars through fiscal year 2011,
* over $122 billion through fiscal year 2018, and:
* close to $230 billion through fiscal year 2025[Footnote 7]
NASA conducted a cost risk analysis of the estimates through fiscal
year 2011. This analysis provided a 65 percent confidence level for the
estimate (i.e., NASA is 65 percent certain that the actual cost of the
program will either meet or be less than the estimate). To obtain this
level of confidence in the estimates, NASA included programmatic
reserves--20 percent on all development and 10 percent on all
production costs. NASA only conducted the risk analysis through the
first flight date of the CEV at the time of ESAS--2011--leaving the
estimates through 2018 and 2025, when most of the cost risk for
implementing the architecture will be realized, with no confidence
level distinction. According to NASA officials, the cost risk analysis
lacked quality because of the evolving nature of the requirements for
the architecture and the compressed time frames with which they had to
conduct the analysis. According to NASA officials, once they receive
more detailed contractor inputs, the agency will be able to produce
higher-fidelity estimates of program cost. NASA has stated that it
would not commit to a cost estimate for implementing the exploration
architecture until the Constellation program's PDR, which will occur in
late fiscal year 2008. At that time, the requirements, design,
schedule, and cost will all be baselined.
NASA refined the architecture several times since ESAS. As a result of
these changes, the costs associated with the architecture have also
changed. As part of the fiscal year 2007 budget formulation process,
NASA made two major changes to plans laid out in the ESAS. First, the
requirement for use of a LOX/methane engine on the CEV service module-
-a high-risk development--was removed, and the approach for meeting the
propulsion requirement was left to the discretion of the contractor.
Second, the first flight of the CEV was delayed until no later than
2014.
NASA's Life Cycle for Flight Systems and Ground Support Projects
through Phase D:
[See PDF for Image]
Source: NASA.
SRR = System Requirements Review
SDR = System Definition Review
NAR = Non=Advocate Review
Pre-Nar = preliminary Non-Advocate Review
PDR = Preliminary Design Review
CDR = Critical Design Review
[End of Figure]
Subsequent to the submission of NASA's fiscal year 2007 budget, the
Constellation program conducted an internal bottom-up review (BUR) of
program costs. The goal of the BUR was to identify the funding it would
take to "get the job done," which, according to the BUR guidance, means
conducting the first flight of the CEV to the ISS by 2012 and first
lunar mission by 2017. This review attempted to determine the cost
impact of several major changes that were made to the architecture.
These changes included a reduction in CEV diameter from 5.5 to 5
meters, use of a five-segment RSRB and a Saturn-derived J-2x engine on
the upper stage of the CLV, deletion of the unpressurized cargo CEV,
the addition of an ISS docking system (Androgynous Peripheral
Attachment System), and the inclusion of a Ka Band for High Definition
Television on the CEV. Some of these architecture changes may help
lessen technology development risks in the future program due to the
planned commonality between the CLV and CaLV launch systems. While the
results of this review were an attempt to provide more fidelity to the
Constellation program's cost estimates, given the continued lack of a
firm program baseline for requirements, design, and schedule, along
with a continued lack of input from contractors, it is unlikely that
the program had the level of detail available to support a true
estimate of total costs this early in the program life cycle.
ESMD is conducting a follow-on review to the Constellation program's
BUR as NASA enters its fiscal year 2008 budget formulation cycle. As
part of this latest review, NASA has continued to evaluate changes to
the program architecture and schedule, such as the use of the RS-68
engine on the CaLV and the delay of the first lunar mission to either
fiscal year 2019 or fiscal year 2020.
The continued evolution of the exploration architecture serves to
highlight the preliminary nature of architecture itself and its
associated cost estimates. Although NASA is continuing to refine its
cost estimates for implementing the architecture to provide a more
reliable estimate of cost, history suggests that program costs could
increase significantly over estimates. In 2004, CBO reported that
fulfilling the Vision could require the addition of billions of dollars
to NASA's estimates of cost or extending the schedule for the first
lunar landing by several years. Applying NASA's average cost growth
figure of 45 percent to the ESAS cost estimates, assuming NASA business
as usual, would result in an increase of almost $14 billion over the
$31 billion it estimates it will need through 2011. With a significant
increase in NASA budgets unlikely, given the current national fiscal
imbalance, this level of cost growth could result in an unsustainable
long-term exploration program.
Cost Estimate Issues:
Historically, NASA has shown that it lacks a clear understanding of how
much its programs will cost and how long they will take to achieve
their objectives. NASA's cost estimates have often been unreasonable
when committing to programs because of several factors, including
inadequate requirements definition; changes in program content; and
inadequate processes to establish priorities, quantify risks, and make
informed investment decisions. GAO has reported on these issues for
several years in both its high-risk series and in specific reviews of
programs where NASA failed to apply discipline to its cost estimates to
ensure those estimates were reasonable. For example, in 2002, GAO
reported that since 1995, estimates for completion of the ISS had
increased by $13 billion and the scheduled completion date had slipped
4 years. Also, in 2004, GAO conducted a review of 27 other NASA
programs and reported that the initial baseline estimates for over half
of those programs were understated.
Costs for NASA programs have historically been greater, on average,
than initial estimates anticipated. A 2004 Congressional Budget Office
(CBO) examination of 72 NASA programs spanning the past 30 years found
that costs of NASA programs have increased, on average, 45 percent from
initial budget estimates.
Funding Shortfalls:
Expected Budget Challenges Architecture Implementation:
NASA will be challenged to implement the exploration architecture,
given the agency's expected budget profile. The ESAS effort defined the
recommended architecture and preliminary costs, which NASA contends
would allow the program to be accomplished within available budgets
through fiscal year 2011. However, phasing issues still needed to be
resolved. On an annual basis, NASA cannot afford to implement the
architecture, although, cumulatively, for fiscal years 2007-2011, the
agency says it has the money available. Beginning with fiscal year 2014
and for the remainder of the decade, where the anticipated available
budgets were adjusted for inflation, the ESAS cost projections show
yearly multibillion-dollar shortfalls with an overall deficit through
2025 of over $18 billion.
The projected ESMD available budget figures used in the ESAS were
developed well in advance of NASA's fiscal year 2007 President's budget
submission. However, using the updated budget estimates from the fiscal
year 2007 budget, the phasing issue becomes more pronounced when
compared to ESAS estimated costs. As shown in the chart below, ESAS
estimates could be accommodated within the ESMD available budget
through fiscal year 2007. From fiscal year 2008 through fiscal year
2010, however, NASA anticipates annual budget shortfalls for
implementing the architecture within ESMD to exceed $1 billion per
year. This shortfall could be partially offset, at least within the
Constellation program, by a carryover of approximately $1 billion in
both fiscal years 2006 and 2007 as a result of funds redirected from
R&T activities within ESMD to that program. In addition, NASA officials
stated the Constellation program has requested more funding than
required for its projects in several years to cover shortfalls in later
years. For example, the Exploration Communication and Navigation
Systems project within the Constellation program plans to roll over
$56.2 million from the fiscal year 2007 budget to make up for budget
shortfalls in fiscal years 2008, 2009, and 2010.
[See PDF for image]
Source: NASA (data) and GAO (analysis).
[End of figure]
NASA's approach, however, appears to be contrary the agency's stated
"go as you can afford to pay" approach to implement priority missions
within available resources. In addition, the surplus shown in fiscal
year 2011 is dependent upon dollars becoming available from the
retirement of the space shuttle fleet, even though NASA officials
stated the costs associated with retiring the space shuttle and
transitioning to new architecture are not fully understood and the
expected surplus could be less than anticipated. The shortfall
presented by the fiscal year 2007 budget would not allow NASA to
accomplish the stated program objectives within available resources
over the next 5 years.
In addition, changes to the architecture implementation schedule have
not been consistent within the Constellation program. As previously
stated, NASA moved the scheduled initial operational capability (IOC)
date of the CEV to no later than 2014 during the fiscal year 2007
budget formulation process. This change, along with modifications to
the architecture, allowed NASA's estimates to meet its overall budget
profile, despite continued year-to-year budget phasing issues. However,
because of NASA's focus on minimizing the gap between the retirement of
the space shuttle and the first flight of the CEV to the ISS, the
program continued to attempt to meet the earlier IOC date for the CEV
through its various analysis cycles. The earlier 2012 IOC date was
retained as the planning date during the bottom-up review process, the
Phase II request for proposal to the contractors involved CEV
development, and the recent announcement concerning its intention to
purchase the J-2x engine for the CLV from Pratt & Whitney Rocketdyne.
The 2012 date for CEV IOC, in addition to changes the Constellation
program made to the architecture during the BUR process, did not
alleviate issues with the short-term funding profile. According to
Constellation program officials, the net result of these changes will
add more cost to the early years of the program, when funding is
already constrained and phasing issues persist. Although the results of
the BUR will not be released, indications from Constellation program
officials are that the estimated costs of the program are higher than
the ESAS estimated costs and available funding per NASA's budget
profile.
In the meantime, NASA continues to look for ways to resolve its budget
phasing issues, such as by making additional changes to the exploration
architecture. As the Constellation program executes its budget
formulation process for the fiscal year 2008 budget cycle, it is
currently analyzing options to the current architecture in an attempt
to reduce development and production costs. For example, NASA recently
announced that it intends to use five RS-68 engines instead of five
SSMEs for the CaLV core stage, which would also require the CaLV core
stage diameter to be increased to approximately 33 feet to accommodate
the additional propellant needed by the RS-68 engines.
NASA Funding Approach:
The NASA Administrator recently testified that the agency is facing
challenges to ensuring adequate funding for the priorities of the
President and Congress within available budgetary resources. He stated
that NASA has adopted a "go as you can afford to pay" approach to
funding its exploration missions. This approach assumes NASA's top line
budget will grow at the moderate rate identified in the President's
fiscal year 2007 budget request.
Under this approach, NASA would implement its priority missions within
available resources and planned budgets through the redirection of
funding for longer-term and lower-priority R&T elements within ESMD. As
a result, several ESMD R&T programs and missions were discontinued,
descoped, or deferred. That funding, in turn, was shifted into the
Constellation Program to accelerate development of the CEV and the CLV.
Gap in Human Spaceflight:
The Vision called for retirement of the space shuttle fleet by the end
of this decade and that the CEV should be available no later than 2014,
creating a potential gap in human spaceflight of up to 4 years. The
NASA Administrator has stated that it is a priority of the agency to
close this gap and that the agency has taken steps to have the CEV in
service as close to 2010 as possible.
On the basis of lessons learned from the period between the end of the
Apollo Program and the first flight of the space shuttle, the
Administrator outlined several reasons why the CEV should not be
delayed. These reasons include the potential for:
* stagnation in the aerospace industry,
* loss of critical expertise,
* withering of the industrial base,
* higher overall program costs,
* program schedule delays, and:
* loss of leadership in space exploration.
Congress has also voiced its concern over the potential gap in human
spaceflight. In the National Aeronautics and Space Administration
Authorization Act of 2005, Congress stated it is the policy of the
United States to have the capability for human access to space on a
continuous basis.
CEV Project:
Best Practices:
GAO has frequently reported on the importance of developing a sound
business case before committing resources to a new product development
effort. The business case in its simplest form is demonstrated evidence
that (1) the need for the product is valid and that it can best be met
with the chosen concept, and (2) the chosen concept can be developed
and produced using existing and reasonably expected resources.
GAO has undertaken a best practices body of work on how leading
developers use a knowledge-based approach to develop products that
reduces risks and increases the likelihood of successful outcomes. This
type of approach is based on the premise of attaining knowledge about a
program and the resources available before making a contractual or
financial commitment. Knowledge that supports a sound business case
includes well-defined requirements, a preliminary design, mature
technology, and realistic cost estimates.
Use of this approach has enabled leading organizations to deliver high-
quality products on time and within budget. Conversely, GAO has also
reported that major systems that have not established a sound business
case have been plagued by cost overruns, schedule delays, decreased
capability, and overall poor performance. NASA's track record in
developing systems has not been good. GAO and others have reported that
NASA has had numerous problems with its programs and projects,
including underestimating program complexity and technology maturity,
and inadequate review and systems engineering processes.
Lack of Sound Business Case Puts CEV Acquisition at Risk:
NASA's acquisition strategy for the CEV places the project at risk of
cost overruns, schedule delays, and performance shortfalls because it
commits the government to a long-term product development effort before
establishing a sound business case. In September 2006, NASA plans to
award a contract for design, development, production, and sustainment
of the CEV--before it has developed well-defined requirements, a
preliminary design, mature technology, and firm cost estimates for the
project. The CEV project might not have all the elements of a sound
business case in place until the project-level PDR in March 2008. At
the completion of the PDR, NASA will approve the selected prime
contractor's preliminary design based on detailed, validated
requirements. Further, CEV project officials indicated that the CEV
project plans to retire all technology risks by the PDR. At that point,
NASA will likely have the increased knowledge necessary to develop a
sound business case that includes high-fidelity, engineering-based
estimates of life cycle cost for the CEV project. With this business
case in hand, NASA would be in a better position to commit the
government to a long-term design and development effort. NASA officials
disagree and have stated that it is appropriate for them to proceed
with the contract award because the agency is selecting "a designer,
not a design" for the CEV. In reality, by awarding a contract as
planned in September 2006, NASA is not only committing to an unknown
design but to production and long-term sustainment of the CEV as well.
The CEV contract scheduled for award in September 2006 will have three
schedules. At the time of contract award, NASA will be responsible for
fee earned and the reasonable, allowable, and allocable costs incurred
by the contractor in the performance of Schedule A and the minimum
quantities under Schedules B and C.
* Schedule A is for design, development, test and evaluation of the
CEV. Deliverables under Schedule A include all test articles and two
operational CEV vehicles--one human-rated variant and one pressurized
cargo variant.
* Schedule B is for production beyond the two operational vehicles
delivered under Schedule A. The CEV request for proposal states that
the "guaranteed minimum" quantity for Schedule B is "two CEV," the type
of which, according to NASA officials, is undetermined.
* Schedule C is for sustainment in support of operations and in support
of Schedule B activities.
CEV Timeline:
[See PDF for Image]
Source: NASA (data) and GAO (presentation).
Note: Contract award for all schedules is planned for September 2006.
Schedule B and C performance periods are from 2009 to 2014 with an
additional 5-year performance option to end in 2019.
[End of Figure]
An important step in developing a sound business case is defining
requirements. The acquisition strategy for the CEV lays out a series of
reviews to validate and approve CEV requirements. These reviews result
in approved system-level requirements at the October 2006 System
Requirements Review (SRR), and approved subsystem-level requirements at
the April 2007 System Definition Review (SDR) and culminate with
validated and approved component-level requirements at the March 2008
PDR. Under the current CEV strategy, NASA will select the winning
contractor about 1 month before the system level requirements are
approved at the SRR, over a year and a half before detailed component-
level requirements are approved at the PDR.
Another aspect of a sound business case is having mature technologies
before committing to product development. The CEV's acquisition
strategy is predicated upon using mature technologies as the basis for
system development. However, contractors will also be given discretion
to include immature technologies in areas where technology advancement
is critical to meeting requirements. NASA has independently identified
technology risks and implemented advanced technology development
projects to address risks in the areas of the thermal shielding needed
for reentry and the landing systems needed for ground landings. CEV
project officials also expect that each contractor's proposal will
include additional technology development risks. Under the current CEV
strategy, NASA is awarding a contract for product development and
production of the first two variants of the CEV before it has resolved
these technology development risks.
Past Development Attempts:
NASA has tried unsuccessfully to develop a number of vehicles to
replace the shuttle over the past three decades. In the 1980s NASA
initiated the National Aero-Space Plane (NASP) to build and test a
manned experimental flight vehicle for demonstrating single-stage-to-
orbit space launch and sustained hypersonic cruise capability. NASA
canceled the program as it was experiencing cost overruns, schedule
delays, and technology problems. GAO reported that from 1986 to 1993
NASA spent $398 million for the NASP program.
In the 1990s, NASA began the X-33 program to develop single-stage-to
orbit technology and the X-34 to demonstrate reusable two-stage-to -
orbit technologies. According to a 2006 Congressional Research Service
report, NASA terminated the X-33 and X-34 in March 2001--after spending
over $1.4 billion--because the cost to complete them was too high
relative to the benefits. In 1999, GAO reported that technical problems
and unrealistic cost estimates on the X-33 project alone led to cost
overruns of $75 million and over a year's delay.
In 2004, after the announcement of the Vision, NASA canceled the Space
Launch Initiative (SLI) program, which was to provide both launch
capabilities and an emergency crew return from the ISS. NASA's
Inspector General reported that NASA did not verify and validate basic
requirements for its second generation space transportation, while GAO
reported that key management controls could not be implemented until
such requirements were defined. GAO estimates that from 2001 to 2005
NASA provided the SLI program with about $3 billion in funding.
Appendix:
Scope and Methodology:
To assess the extent to which NASA has identified the architecture and
costs necessary to implement the Vision and whether NASA's exploration
architecture fits within the agency's projected available budgets, we
reviewed and analyzed NASA's Exploration Systems Architecture Study,
fiscal year 2007 budget request, ground rules and assumptions provided
from the Constellation program to project-level management estimators
to perform the BUR, guidance for use in preparing the fiscal year 2008
budget request, NASA cost-estimating guidance in the NASA Cost
Estimating Handbook, and congressional hearings and testimonies
pertaining to NASA and the Vision. We also conducted interviews with
NASA headquarters officials from the Cost Analysis Division, the
Exploration Systems Mission Directorate, and the Constellation Program;
Constellation program and CEV project officials at Johnson Space
Center; CLV project officials at Marshall Space Flight Center; and cost
analysts from Kennedy Space Center. During these interviews, we
discussed the methodologies used in preparing the ESAS and subsequent
cost estimates, architecture changes after ESAS and the trades being
considered, budgeting issues, and procurement strategies and
activities.
To assess the risks associated with NASA's acquisition strategy for the
CEV project, we reviewed and analyzed CEV project documentation,
including draft project plans, draft requirements documents, technology
development plans, documentation included in the contract request for
proposals, and documentation for past NASA human space flight
acquisition programs. We compared NASA's plans for the CEV with
criteria contained in GAO best practices work on systems acquisition.
We also conducted interviews with NASA headquarters officials from the
Exploration Systems Mission Directorate, Constellation Program and CEV
project officials at Johnson Space Center, and CLV project officials at
Marshall Space Flight Center.
Contributors:
If you have any questions concerning this briefing, please call Allen
Li at (202) 512-4841. Other key contributors to this briefing were
James L. Morrison, Assistant Director; Rick Cederholm; Shelby S.
Oakley; Guisseli Reyes; Sylvia Schatz; and John S. Warren, Jr.
FOOTNOTES
[1] The ESAS architecture supports the development of a new Crew
Exploration Vehicle (CEV), Crew Launch Vehicle (CLV), a Cargo Launch
Vehicle (CaLV), and other supporting systems. The architecture also
calls for various Research and Technology (R&T) and Robotic Lunar
Exploration Program (RLEP) projects.
[2] All cost estimates related to the Vision are reported as inflated
("real year") dollars.
[3] NASA's cost estimate through 2011--$31.2 billion--included the
costs of the R&T and RLEP projects needed to support the architecture.
Its estimate for the first lunar landing--$104 billion--did not include
$18 billion in funding for R&T and RLEP projects. To ensure
consistency, the estimates for 2018 and 2025 are presented with R&T and
RLEP funding included.
[4] GAO, 21st Century Challenges: Reexamining the Base of the Federal
Government, GAO-05-325SP (Washington, D.C.: Feb. 2005); 21st Century:
Addressing Long-Term Fiscal Challenges Must Include a Reexamination of
Mandatory Spending, GAO-06-456T (Washington, D.C.: Feb. 15, 2006); and
Highlights of a GAO Forum: The Long-Term Fiscal Challenge, GAO-05-282SP
(Washington, D.C.: Feb. 1, 2005).
[5] The Vision includes a return to the moon that is intended
ultimately to enable future exploration of Mars and other destinations.
To accomplish this, NASA initially plans to (1) complete its work on
the International Space Station by 2010, fulfilling its commitment to
15 international partner countries; (2) begin developing a new manned
exploration vehicle to replace the space shuttle; and (3) return to the
moon no later than 2020 in preparation for future, more ambitious
missions.
[6] Examples of our best practices reports include GAO, Best Practices:
Using a Knowledge-Based Approach to Improve Weapon Acquisition, GAO-04-
386SP (Washington, DC.: Jan. 2004); Space Acquisitions: Committing
Prematurely to the Transformational Satellite Program Elevates Risks
for Poor Cost, Schedule, and Performance Outcomes, GAO-04-71R
(Washington, D.C.: Dec. 4, 2003); Best Practices: Capturing Design and
Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-02-
701 (Washington, D.C.: Jul. 15, 2002); and Best Practices: Better
Matching of Needs and Resources Will Lead to Better Weapon System
Outcomes, GAO-01-288 (Washington, DC.: Mar. 8, 2001).
[7] NASA's cost estimate through 2011--$31 billion--included the costs
of the R&T and RLEP projects needed to support the architecture. Its
estimate for the first lunar landing--$104 billion--did not include $18
billion in funding for R&T and RLEP projects. To ensure consistency,
the estimates for 2018 and 2025 are presented with R&T and RLEP funding
included. The estimates include $20 billion to service the ISS.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
