Space Shuttle
Costs for Hubble Servicing Mission and Implementation of Safety Recommendations Not Yet Definitive Gao ID: GAO-05-34 November 19, 2004Hubble's continued operation has been dependent on manned servicing missions using the National Aeronautics and Space Administration's (NASA) shuttle fleet. The fleet was grounded in early 2003 following the loss of the Space Shuttle Columbia, as NASA focused its efforts on responding to recommendations made by the Columbia Accident Investigation Board (CAIB). In January 2004, NASA announced its decision to cancel the final planned Hubble servicing mission, primarily because of safety concerns. Without some type of servicing mission, NASA anticipates that Hubble will cease to support scientific investigations by the end of the decade. NASA's decision not to service the Hubble prompted debate about potential alternatives to prolong Hubble's mission and the respective costs of these alternatives. This report addresses the basis of NASA's cost estimates to (1) service Hubble using the shuttle and (2) implement recommendations made by the CAIB. GAO is continuing its work on the Congressional request that GAO examine the potential cost of a robotic servicing mission to the Hubble Telescope.
Although a shuttle servicing mission is one of the options for servicing the Hubble Space Telescope, to date, NASA does not have a definitive estimate of the potential cost. At our request, NASA prepared an estimate of the funding needed for a shuttle servicing mission to the Hubble. NASA estimates the cost at between $1.7 billion to $2.4 billion. However, documentary support for portions of the estimate is insufficient. For example, NASA officials told us that the Hubble project's sustaining engineering costs run $9 to 10 million per month, but they were unable to produce a calculation or documents to support the estimate because they do not track these costs by servicing mission. Additionally, the agency has acknowledged that many uncertainties, such as the lack of a design solution for autonomous inspection and repair of the shuttle, could change the estimate. A t the same time, NASA has yet to develop a definitive cost estimate for implementing all of the CAIB's recommendations but has developed a budget estimate for safely returning the shuttle to flight--a subset of activities recommended by the CAIB as needed to return the shuttle to full operations. NASA currently estimates return to flight costs will exceed $2 billion, but that estimate will likely be refined as the agency continues to define technical concepts. NASA provided support for portions of the estimate, but we found the support to be insufficient--either because key documents were missing or the estimates lacked sufficient detail. Further, NASA cautions that return to flight costs will remain uncertain until the first return to flight shuttle mission, which is scheduled to go to the International Space Station in spring 2005.
GAO-05-34, Space Shuttle: Costs for Hubble Servicing Mission and Implementation of Safety Recommendations Not Yet Definitive
This is the accessible text file for GAO report number GAO-05-34
entitled 'Space Shuttle: Costs for Hubble Servicing Mission and
Implementation of Safety Recommendations Not Yet Definitive' which was
released on December 17, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on VA/HUD-Independent Agencies, Committee on
Appropriations, U.S. Senate:
United States Government Accountability Office:
GAO:
November 2004:
Space Shuttle:
Costs for Hubble Servicing Mission and Implementation of Safety
Recommendations Not Yet Definitive:
GAO-05-34:
GAO Highlights:
Highlights of GAO-05-34, a report to the Subcommittee on VA/HUD-
Independent Agencies, Committee on Appropriations, U.S. Senate:
Why GAO Did This Study:
Hubble‘s continued operation has been dependent on manned servicing
missions using the National Aeronautics and Space Administration‘s
(NASA) shuttle fleet. The fleet was grounded in early 2003 following
the loss of the Space Shuttle Columbia, as NASA focused its efforts on
responding to recommendations made by the Columbia Accident
Investigation Board (CAIB). In January 2004, NASA announced its
decision to cancel the final planned Hubble servicing mission,
primarily because of safety concerns. Without some type of servicing
mission, NASA anticipates that Hubble will cease to support scientific
investigations by the end of the decade.
NASA‘s decision not to service the Hubble prompted debate about
potential alternatives to prolong Hubble‘s mission and the respective
costs of these alternatives. This report addresses the basis of NASA‘s
cost estimates to (1) service Hubble using the shuttle and (2)
implement recommendations made by the CAIB. GAO is continuing its work
on the Subcommittee‘s request that GAO examine the potential cost of a
robotic servicing mission to the Hubble Telescope.
What GAO Found:
Although a shuttle servicing mission is one of the options for
servicing the Hubble Space Telescope, to date, NASA does not have a
definitive estimate of the potential cost. At our request, NASA
prepared an estimate of the funding needed for a shuttle servicing
mission to the Hubble. NASA estimates the cost at between $1.7 billion
to $2.4 billion. However, documentary support for portions of the
estimate is insufficient. For example, NASA officials told us that the
Hubble project‘s sustaining engineering costs run $9 to 10 million per
month, but they were unable to produce a calculation or documents to
support the estimate because they do not track these costs by servicing
mission. Additionally, the agency has acknowledged that many
uncertainties, such as the lack of a design solution for autonomous
inspection and repair of the shuttle, could change the estimate.
At the same time, NASA has yet to develop a definitive cost estimate
for implementing all of the CAIB‘s recommendations but has developed a
budget estimate for safely returning the shuttle to flight”a subset of
activities recommended by the CAIB as needed to return the shuttle to
full operations. NASA currently estimates return to flight costs will
exceed $2 billion, but that estimate will likely be refined as the
agency continues to define technical concepts. NASA provided support
for portions of the estimate, but we found the support to be
insufficient”either because key documents were missing or the estimates
lacked sufficient detail. Further, NASA cautions that return to flight
costs will remain uncertain until the first return to flight shuttle
mission, which is scheduled to go to the International Space Station in
spring 2005.
Hubble Space Telescope:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO is not making recommendations in this report.
www.gao.gov/cgi-bin/getrpt?GAO-05-34.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Allen Li at (202)
512-4841 or lia@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
NASA's Cost Estimate for a Hubble Servicing Mission Using the Shuttle
Is Not Definitive:
Estimate for Implementing CAIB Recommendations Is Not Fully Developed:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the National Aeronautics and Space
Administration:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Budget Estimate for Shuttle Servicing of Hubble:
Table 2: Return to Flight Budget Estimates as of July 2004:
Table 3: Return to Flight Budget Estimate Review Status as of
July 2004:
Abbreviations:
CAIB: Columbia Accident Investigation Board:
NASA: National Aeronautics and Space Administration:
PRCB: Program Requirements Control Board:
UCA: undefinitized contract actions:
United States Government Accountability Office:
Washington, DC 20548:
November 19, 2004:
The Honorable Christopher S. Bond:
Chairman:
The Honorable Barbara A. Mikulski:
Ranking Member:
Subcommittee on VA/HUD-Independent Agencies:
Committee on Appropriations:
United States Senate:
For more than a decade, the National Aeronautics and Space
Administration's (NASA) Hubble Space Telescope has provided unique
images of the universe and has given scientists critical data needed to
help understand a number of space mysteries. Hubble's continued
operation has been dependent on NASA's space shuttle fleet, which
carried crewmembers to the telescope to perform periodic maintenance
and upgrades. The grounding of the shuttle fleet following the tragic
loss of Space Shuttle Columbia in February 2003 put these missions to
Hubble on hold, and in January 2004, NASA announced its decision to
cancel the fifth and final planned Hubble servicing mission. With no
future servicing missions, NASA anticipates that Hubble will cease to
support scientific investigations by the end of the decade.
NASA said that its decision to cancel the servicing mission was based
largely on concerns about shuttle safety--specifically, the need to
implement recommendations made by the Columbia Accident Investigation
Board (CAIB) for safely returning the space shuttle to flight.[Footnote
1] NASA also recognized that a Hubble servicing mission would
necessitate further changes both to shuttle hardware and operational
procedures. However, NASA's decision has prompted considerable debate.
Congressional members, scientists, and space policy experts have called
for an examination of alternatives to prolong Hubble's mission, such as
the possibility of servicing Hubble robotically. The CAIB Chairman
suggested that only a study of the benefits and risks of a shuttle
servicing mission to extend Hubble's life could determine whether it is
worth the risks.
This report addresses the basis for NASA's cost estimates to
(1) service the Hubble Telescope by using the shuttle and (2) implement
the CAIB's recommendations, including those recommendations directly
related to safely returning the shuttle to flight. We are continuing
our work on your request that we examine the potential cost of a
robotic servicing mission to the Hubble Telescope.
To assess the basis for NASA's Hubble servicing mission cost estimate,
we examined NASA's rationale for its decision to cancel the final
planned shuttle servicing mission, analyzed available funding
estimates, and requested analytical and documentary support for
selected high-dollar items to identify the sufficiency of support. In
reviewing the basis for NASA's cost estimate for implementing the
CAIB's recommendations, we analyzed available funding estimates and
requested analytical and documentary support for selected high dollar
portions of the estimate to determine the sufficiency of such support.
We performed our review from March through September 2004 in accordance
with generally accepted government auditing standards. For a complete
description of our scope and methodology, see appendix I.
Results in Brief:
Definitive cost estimates to facilitate decisions regarding options for
servicing the Hubble are critical. However, NASA has not yet developed
such an estimate for one of the options--a shuttle servicing mission.
At our request, NASA prepared an estimate of the funding needed for a
shuttle servicing mission to Hubble. However, NASA could not provide
documented support for key portions of the estimate, stating that there
are many uncertainties that could change the estimate, such as the lack
of a design solution for two safety-related requirements--autonomous
inspection and repair and crew rescue mission capabilities.
Similarly, NASA does not have a definitive cost estimate for
implementing all of the CAIB recommendations. The agency has been
focusing primarily on those recommendations it considers necessary to
return the shuttle fleet to flight. NASA's current estimate for
implementing those recommendations is more than $2 billion. NASA
provided us with documentary support for portions of the estimate, but
we found some of the support to be insufficient. In NASA's view, the
estimate for returning the shuttle fleet to flight will remain
uncertain until the first shuttle mission to the International Space
Station.
In written comments, which are reprinted in appendix II, NASA stated
that the agency believes that both the estimate and the methodology
used in calculating the costs of reinstating the servicing mission are
sound and accurate given the level of definition of the mission at this
point in time. However, the agency agreed that some portions of the
servicing mission activities lacked the design maturity required to
estimate the costs according to NASA accepted and established
procedures.
Background:
Since it was launched in 1990, the Hubble Space Telescope has sent
back images of space that have made a significant contribution to our
understanding of the universe. The telescope uses pointing precision,
powerful optics, and state-of-the-art instruments to explore the
visible, ultraviolet, and near-infrared regions of the electromagnetic
spectrum. To keep it at the forefront of astronomical research and
extend its operational life, Hubble's instruments have been upgraded
through a series of shuttle servicing missions. The fifth and final
planned servicing mission was intended to install new science
instruments, replace the telescope's insulation, and replace the
batteries and gyroscopes. According to NASA, the lifetime of the
observatory on orbit is ultimately limited by battery life, which may
extend into the 2007-2008 time frame, but scientific operations are
limited by the gyroscopes that stabilize the telescope--whose lifetimes
are more difficult to predict. NASA forecasts that the Hubble will
likely have fewer than three operating gyroscopes by mid-2006, and
fewer than two by mid-2007.
In response to congressional concerns about NASA's decision to cancel
the servicing mission, NASA requested that the National Research
Council[Footnote 2] conduct an independent assessment of options for
extending the life of the Hubble Space Telescope. In May 2004, the
Council established a committee to assess the viability of a shuttle
servicing mission, evaluate robotic and ground operations to extend the
life of the telescope as a valuable scientific tool, assess telescope
component failures and their impact, and provide an overall risk-
benefit assessment of servicing options. In an interim report issued in
July 2004, the committee urged NASA to commit to a Hubble servicing
mission that accomplishes the objectives of the canceled servicing
mission and to take no actions that would preclude using a space
shuttle to carry out this mission. According to a NASA official, the
agency is not actively pursuing the shuttle servicing option but is not
precluding it.
NASA is currently evaluating the feasibility of performing robotic
servicing of the Hubble Telescope. To facilitate the evaluation, the
agency has formulated a robotic mission concept, which includes a
vehicle comprised of a robotic servicing module and another module that
can be used to eventually de-orbit the telescope. The potential task
list of activities for robotic servicing includes replacing the
gyroscopes and batteries, installing new science instruments, and de-
orbiting the observatory at the end of its life. According to a NASA
official, contracts to facilitate the robotic mission were recently
awarded for work to begin on October 1, 2004.
The CAIB concluded that the Columbia accident was caused by
both physical and organizational failures. The Board's 15 return to
flight recommendations necessary to implement before the shuttle fleet
can return to flight primarily address the physical causes of the
accident and include eliminating external tank debris shedding and
developing a capability to inspect and make emergency repairs to the
orbiter's thermal protection system. NASA publishes periodic updates to
its plan for returning the shuttle to flight to demonstrate the
agency's progress in implementing the CAIB recommendations. The most
recent update is dated August 27, 2004. This update identifies the
first shuttle flight as occurring in spring 2005.
NASA's Cost Estimate for a Hubble Servicing Mission Using the Shuttle
Is Not Definitive:
NASA does not currently have a definitive cost estimate for servicing
the Hubble Telescope using the shuttle. The agency focused on safety
concerns related to a servicing mission by the space shuttle in
deciding not to proceed, and did not develop a cost estimate. At our
request NASA prepared an estimate of the funding needed for a Hubble
servicing mission by the space shuttle. NASA could not provide
documented support for its estimate. The agency recognizes that there
are many uncertainties that could change the estimate. NASA has now
begun to explore the costs and benefits of various servicing
alternatives, including robotic servicing, which should enable NASA to
make a more informed decision regarding Hubble's future.
At our request NASA began development of an estimate of the funding
needed for a shuttle servicing mission to the Hubble. The estimate
provided captures additional funds over and above NASA's fiscal year
2005 budget request that would be required to reinsert the mission in
the shuttle flight manifest for launch in March 2007. The estimate does
not include funding already expended to support the canceled servicing
mission and develop the science instruments. NASA has determined that
the additional funds needed to perform a shuttle servicing mission for
Hubble would be in the range of $1.7 billion to $2.4 billion. According
to NASA, this estimate is based on what it might cost, but it does not
take into account the technical, safety, and schedule risks that could
increase the cost and/or undermine the viability of the mission. For
example, NASA cites uncertainties related to two safety-related
requirements: inspection and repair and crew rescue mission
capabilities that would be autonomous of the International Space
Station and for which NASA currently has not formulated a design
solution.[Footnote 3] In addition, NASA cautions that it did not
examine whether design solutions could be accomplished in time to
service Hubble before it ceases operations. Table 1 shows NASA's budget
estimate phased by fiscal year (FY) for shuttle servicing of the Hubble
Space Telescope, including ranges for some of the estimates.
Table 1: Budget Estimate for Shuttle Servicing of Hubble:
Real year dollars in millions[A].
Sustaining engineering for Hubble project;
Fiscal year: 2005: $115;
Fiscal year: 2006: $141;
Fiscal year: 2007: $105;
Fiscal year: 2008: $26;
Fiscal year: 2009: $30;
Budget to Complete: $40;
Total[A]: $457.
Delay de-orbit mission until 2012;
Fiscal year: 2007: $(69);
Fiscal year: 2008: $(136);
Fiscal year: 2009: $79;
Budget to Complete: $166;
Total[A]: $40.
Extend Hubble operations to 2012;
Budget to Complete: $117;
Total[A]: $117.
Shuttle services unique to servicing mission and extravehicular
activity[B];
Fiscal year: $2006: $18;
Fiscal year: 2007: $15;
Total[A]: $33.
Payload processing;
Fiscal year: 2005: $7;
Fiscal year: 2006: $7;
Fiscal year: 2007: $13;
Fiscal year: 2008: $10;
Fiscal year: 2009: $7;
Total[A]: $43.
Autonomous inspection and repair capability;
Fiscal year: 2004: $116- 134;
Fiscal year: 2005: $74-86;
Fiscal year: 2006: $52-60;
Fiscal year: 2007: $18-21;
Total[A]: $260-300.
Autonomous rescue mission capability;
Fiscal year: 2004: $85-98;
Fiscal year: 2005: $116-134;
Fiscal year: 2006: $61-71;
Fiscal year: 2007: $31- 36;
Total[A]: $293-338.
Rescue mission unique requirements and ground operations;
Fiscal year: 2006: $1;
Fiscal year: 2007: $22;
Total[A]: $22.
International Space Station Program impact (3-month delay);
Fiscal year: 2006: $(5);
Fiscal year: 2007: $(15);
Fiscal year: 2008: $(13);
Fiscal year: 2009: $(4);
Budget to Complete: $50;
Total[A]: $13.
Delay shuttle phase-out 3 months;
Fiscal year: 2007: $9- 0;
Fiscal year: 2008: $17-72;
Fiscal year: 2009: $26-249;
Budget to Complete: $350-757;
Total[A]: $401-1,078.
Total[A];
Fiscal year: 2004: $200-231;
Fiscal year: 2005: $312-341;
Fiscal year: 2006: $275-293;
Fiscal year: 2007: $128-127;
Fiscal year: 2008: $(95)-(41);
Fiscal year: 2009: $137-361;
Budget to Complete: $723-1,130;
Total[A]: $1,679-2,441.
Source: NASA.
Note: Estimates are in full cost. The concept of full cost ties all
agency costs (including civil service personnel costs) to major
activities. This includes costs that are directly related to a specific
project, such as contractor-supplied hardware, and indirect costs such
as administrative costs, rents, utilities, and travel costs that cannot
be directly identified with a specific project but which can be
allocated based on direct labor hours, square footage, or other
methods.
[A] In some cases, totals do not add because of rounding.
[B] Extravehicular activity is a space walk.
[End of table]
While we did not independently verify each component of NASA's
estimate, we requested that NASA provide the analytical basis and
documentary support for selected portions of the estimate, primarily
those with large dollar values. NASA could not provide the requested
information. For example, NASA officials told us that the Hubble
project's sustaining engineering costs run $9 to 10 million per month,
but they were unable to produce a calculation or documents to support
the estimate because they do not track these costs by servicing
mission. We also requested the basis of estimate for the costs to delay
shuttle phase-out and for tools development for vehicle inspection and
repair without the International Space Station (a component of
extravehicular activity above). In response, NASA provided the
assumptions upon which the estimates were based and stated that the
estimates were based on information provided by Johnson Space Center
and Kennedy Space Center subject matter experts. NASA also added that
rigorous cost estimating techniques could not be applied to the tools
development estimate because a rescue mission currently is only a
concept. No analytical or documentary support was provided. In
estimating the cost for the autonomous inspection and repair and rescue
mission capabilities, NASA used a 30 to 50 percent uncertainty factor
because of the very high uncertainty in the cost of developing and
conducting a mission that is not adequately defined--i.e., NASA's
estimate of $425 million plus 50 percent equals the $638 million upper
range shown in the table above for these two items added together. As
with the other estimates for which we requested analytical and
documentary support, NASA was not able to provide it because the agency
could not do a risk analysis without a design solution, according to a
NASA official. The lack of documented support for portions of NASA's
estimate increases the risk of variation to the estimate. Further, NASA
recognizes that there are many uncertainties that could change the
current estimate.
The 2004 NASA Cost Estimating Handbook states that cost analysts should
document the results of cost estimates during the entire cost
estimating process and that the documentation should provide sufficient
information on how the estimate was developed so that independent cost
analysts could reproduce the estimate. According to the handbook, the
value of the documentation and analysis is in providing an
understanding of the cost elements so that decision-makers can make
informed decisions.
Recently, we also reported that dependable cost estimates are essential
for establishing priorities and making informed investment decisions in
the face of limited budgets.[Footnote 4] Without this knowledge, a
program's estimated cost could be understated and thereby subject to
underfunding and cost overruns, putting programs at risk of being
reduced in scope or requiring additional funding to meet their
objectives.
Since we began our review, attention has focused on alternatives to a
shuttle mission, such as robotic servicing of Hubble. NASA has formed a
team to evaluate Hubble servicing alternatives, including cost
information. This analysis should enable NASA to make a more informed
decision about Hubble's future and facilitate NASA's evaluation of the
feasibility of robotic servicing options.
Estimate for Implementing CAIB Recommendations Is Not Fully Developed:
Currently, NASA has developed budget estimates for implementing the
CAIB recommendations required to return the space shuttle to flight but
not for all of the CAIB recommendations. NASA provided us with
documentary support for portions of the return to flight estimate, but
we found it to be insufficient. According to NASA, the agency's cost
for returning the shuttle to flight, which is slightly over $2 billion,
will remain uncertain until the completion of the first shuttle
missions to the International Space Station in fiscal year 2005.
NASA's return to flight activities involve enhancing the shuttle's
external tank, thermal protection system, solid rocket boosters, and
imagery system to address the physical cause of the Columbia accident-
-a piece of insulating foam that separated from the external tank and
struck a reinforced carbon-carbon panel on the leading edge of the
orbiter's left wing. To address this cause, NASA is working to
eliminate all external tank debris shedding. Efforts are also in place
to improve the orbiter's thermal protection system, which includes heat
resistant tiles, blankets, and reinforced carbon-carbon panels on the
leading edge of the wing and nose cap of the shuttle, to increase the
orbiter's ability to sustain minor debris damage. NASA is also
redesigning the method for catching bolts that break apart when the
external tank and solid rocket boosters separate as well as providing
the capability to obtain and downlink images after the separation. NASA
and the United States Air Force are working to improve the use of
ground cameras for viewing launch activities. Table 2 shows NASA's
budget estimates for return-to-flight activities.
Table 2: Return to Flight Budget Estimates as of July 2004:
Real year dollars in millions[A].
Orbiter reinforced carbon-carbon inspections;
Fiscal year: 2003: $2;
Fiscal year: 2004: $38;
Fiscal year: 2005: $7.
On-orbit thermal protection system inspection and extravehicular
activity tile repair;
Fiscal year: 2003: $20;
Fiscal year: 2004: $68;
Fiscal year: 2005: $130.
Orbiter workforce;
Fiscal year: 2003: $0;
Fiscal year: 2004: $5;
Fiscal year: 2005: $37.
Orbiter thermal protection system hardening;
Fiscal year: 2003: $0;
Fiscal year: 2004: $28;
Fiscal year: 2005: $34.
Orbiter certification/verification;
Fiscal year: 2003: $0;
Fiscal year: 2004: $47;
Fiscal year: 2005: $26.
Orbiter other;
Fiscal year: 2003: $0;
Fiscal year: 2004: $15;
Fiscal year: 2005: $16.
External tank items (camera, bipod ramp, etc.);
Fiscal year: 2003: $11;
Fiscal year: 2004: $114;
Fiscal year: 2005: $94.
Solid rocket booster items (bolt catcher, camera, etc.);
Fiscal year: 2003: $1;
Fiscal year: 2004: $8;
Fiscal year: 2005: $26.
Ground camera ascent imagery upgrade;
Fiscal year: 2003: $8;
Fiscal year: 2004: $40;
Fiscal year: 2005: $58.
Kennedy Space Center ground operations workforce;
Fiscal year: 2003: $0;
Fiscal year: 2004: $32;
Fiscal year: 2005: $36.
Other (system integration, hardware processing and operations systems
verification, and space shuttle main engine technical assessment);
Fiscal year: 2003: $0;
Fiscal year: 2004: $67;
Fiscal year: 2005: $177.
Stafford-Covey team;
Fiscal year: 2004: $3;
Fiscal year: 2005: $1.
Total[B];
Fiscal year: 2003: $42;
Fiscal year: 2004: $465;
Fiscal year: 2005: $643;
Fiscal year: 2006-2009[A]: $1,067;
Total[B]: $2,217.
Source: NASA.
Note: According to NASA, not all elements of full cost have been
distributed to return to flight activities.
[A] Fiscal years 2006-2009 are NASA preliminary estimates, and a cost
breakout by activity is not available.
[B] In some cases, totals do not add because of rounding.
[End of table]
However, the majority of NASA's budget estimates for returning the
shuttle to flight are not fully developed--including those for fiscal
year 2005--as indicated by the agency's internal approval process. The
Program Requirements Control Board (PRCB) is responsible for directing
studies of identified problems, formulating alternative solutions,
selecting the best solution, and developing overall estimates.
[Footnote 5] According to NASA, actions approved with PRCB directives
have mature estimates, while those with control board actions in
process--that is currently under review but with no issued directives
yet--are less mature. Both the content and estimates for return to
flight work that have not yet been reviewed by the control board are
very preliminary and subject to considerable variation. Table 3 shows
the status of control board review of NASA return to flight budget
estimates and the percent of the total estimate at each level of
review.
Table 3: Return to Flight Budget Estimate Review Status as of July
2004:
Real year dollars in millions.
Control board review complete; directive issued;
Fiscal year: 2003: $31;
Fiscal year: 2004: $319;
Fiscal year: 2005: $117;
Fiscal year: 2006: $53;
Fiscal year: 2007: $47;
Fiscal year: 2008: $49;
Fiscal year: 2009: $39;
Total[A]: $655;
Percent of total[A]: 29%.
Been to control board; directive not yet issued;
Fiscal year: 2003: $11;
Fiscal year: 2004: $146;
Fiscal year: 2005: $217;
Fiscal year: 2006: $117;
Fiscal year: 2007: $125;
Fiscal year: 2008: $134;
Fiscal year: 2009: $109;
Total[A]: $859;
Percent of total[A]: 39%.
In review process;
Fiscal year: 2005: $309;
Fiscal year: 2006: $162;
Fiscal year: 2007: $84;
Fiscal year: 2008: $79;
Fiscal year: 2009: $70;
Total[A]: $704;
Percent of total[A]: 32%.
Total Return to Flight activities[A];
Fiscal year: 2003: $42;
Fiscal year: 2004: $465;
Fiscal year: 2005: $643;
Fiscal year: 2006: $331;
Fiscal year: 2007: $257;
Fiscal year: 2008: $261;
Fiscal year: 2009: $218;
Total[A]: $2,217;
Percent of total[A]: 100%.
Source: NASA.
Note: According to NASA, not all elements of full cost have been
distributed to return to flight activities.
[A] In some cases, totals do not add because of rounding.
[End of table]
NASA provided us with the PRCB directives and in some cases,
attachments which the agency believes support the estimate.[Footnote 6]
However, we did not find this support to be sufficient. According to
NASA's cost estimating handbook, estimates should be documented with
sufficient detail to be reproducible by an independent analyst.
Nevertheless, in many cases, there were no documents attached to the
directive, and in cases where documents were attached to the
directives, the documents generally provided high-level estimates with
little detail and no documentation to show how NASA arrived at the
estimate. For example, a request for $1.8 million to fund the network
to support external tank camera transmissions indicated that
$1.516 million of the amount would be needed for Goddard Space Flight
Center to provide the necessary equipment at receiving stations, labor,
subcontractor costs, and travel and that the remaining $290,000 would
be needed for improvements to the receiving antennas ($104,000) and
recurring costs ($62,000 per flight) for three trucks and the
associated transponder time. However, the documents did not show how
the requester for the $1.8 million arrived at the estimates. NASA
officials told us that the reason for this was that the managers
approving the directives trusted their employees to accurately
calculate the estimate and maintain the support. In addition, our
review of the documents indicated and NASA confirmed that quite a few
of the estimates were based on undefinitized contract actions (UCA)--
that is, unnegotiated contract changes. Under these actions, NASA
officials can authorize work to begin before NASA and the contractor
agree on a final estimated cost and fee. As we have stated in our high-
risk series, relying on unnegotiated changes is risky because it
increases the potential for unanticipated cost growth.[Footnote 7]
This, in turn, may force the agency to divert scarce budget resources
intended for other important programs. As of July 31, 2004, NASA
records showed 17 UCAs related to return to flight with not-to-exceed
amounts totaling $147.5 million. NASA's estimate for the entire effort
under these UCAs totals about $325 million, or 15 percent of NASA's
current $2.2 billion return to flight estimate.
In June 2004, NASA established additional requirements for funding
requests submitted to the PRCB. Under the new policy, an independent
cost estimate must be developed for requests greater than $25 million,
and a program-level cost evaluation must be completed for requests over
$1 million. The program-level evaluation consists of a set of standard
questions to document the rationale and background for cost-related
questions. The responses to the questions are initially assessed by a
cost analyst but are reviewed by the Space Shuttle Program Business
Manager before submission to the PRCB.
NASA provided us with two examples of requests falling under the new
requirements. Both of the examples had better support than those with
PRCB directives, but documentary support was still not apparent. For
example, the funding request for a debris radar indicated that the
estimate was based on a partnering agreement with the Navy and the
Navy's use of the technology. However, the program-level evaluation
pointed out that no detailed cost backup was provided. The other
example, which was a funding request to change the processes currently
in place for the Space Shuttle Program's problem reporting and
corrective action system, was very well supported in terms of analysis,
as the requester prepared detailed spreadsheets calculating the funding
requirements according to a breakdown of the work to be performed,
cited sources for labor rates, and provided assumptions underlying the
calculations. However, as pointed out in the program evaluation of the
request, there was no support provided for the estimate other than the
initiator's knowledge of the change. We believe that future compliance
with NASA's new policy establishing additional requirements for funding
requests and the inclusion of documentary support could potentially
result in more credible return to flight budget estimates.
According to NASA, estimates for fiscal year 2005 and beyond will be
refined as the Space Shuttle Program comes to closure on return to
flight technical solutions and the return to flight plan is finalized.
NASA expects that by late fall of 2004, a better understanding of the
fiscal year 2005 financial situation will be developed. However, NASA
cautions that the total cost of returning the shuttle to flight will
remain uncertain until completion of the first shuttle missions to the
space station, scheduled to begin in spring 2005.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, the NASA Deputy
Administrator stated that the agency believes that both the estimate
and methodology used in the calculation of costs for reinstating the
Hubble Space Telescope servicing mission are sound and accurate, given
the level of definition at this point in time. Notwithstanding that
belief, the agency agreed that portions of the servicing mission
activities lacked the design maturity required to estimate the costs
according to accepted and established NASA procedures.
Specifically, NASA agrees that the Hubble Space Telescope work
breakdown structure was not constructed to collect program costs. At
the same time, NASA believes it is erroneous to suggest that NASA has
no valid basis for the numbers provided, citing the "Servicing Mission
4 Resources Management Plan," which describes the effort required for
completion of a servicing mission. According to NASA, although the
program's accounting system does not capture sustaining engineering
costs in GAO's preferred format, the Servicing Mission 4 Resources
Management Plan details mission schedules and staffing, and applying
contractor and civil service rates to that staffing level can
accurately reflect the effort required to execute a servicing mission.
We requested this type of analysis and documentary support, but NASA
representatives did not offer such a calculation. Rather, the officials
stated that the sustaining engineering costs were based on management's
assessment of contractor financial data and in-house service pool
charges and that these activities could not be traced back to source
documentation. Without adequate supporting data, we cannot assess the
accuracy and reliability of such information.
NASA acknowledged that the agency does not have a technical design from
which to derive the cost for the on-orbit inspection and repair of the
shuttle independent of support from the International Space Station. In
the case of the unsupported cost estimate for delaying the phase out of
the space shuttle in order to complete a manned Hubble servicing
mission, NASA stated that it used approved budget projections for the
operating years affected by the insertion of the Hubble servicing
mission and prorated the extension of the service life. According to
NASA, a range was added to the estimate to account for uncertainties
and retention of critical skills. The estimates were presented as a
rough order of magnitude. NASA stated that it provided its assumptions
to demonstrate the reasonableness of the estimates. Nevertheless, in
spite of the uncertainties in the estimate, which we recognized in our
report, NASA guidance states that cost estimates should be documented
during the entire cost estimating process and that the documentation
should provide sufficient information on how an estimate was developed
to be reproducible by independent cost analysts. NASA did not provide
us with this type of documentation. Without adequate supporting data,
we cannot assess the accuracy and reliability of such information. We
do not agree that the use of approved budget projections is a reliable
cost estimating methodology, particularly given the long-term budget
implications of the extension of the space shuttle's service life.
NASA believes that the examples it provided of the actions to implement
several of the CAIB recommendations attest to the rigor of the process
and approved procedures NASA utilized to validate the costs. According
to NASA, the estimates will mature as the technical solutions mature,
but the estimates were not refined at the time of our review. The
agency believes the outstanding technical issues necessary to return to
flight are beginning to be resolved. However, the examples that NASA
provided were in support of estimates that the agency considers mature.
We requested support for high dollar portions of NASA's estimate, which
the agency did not provide. However, NASA selectively provided examples
of what it considered to be mature estimates. We reviewed the examples
but found that most of them contained insufficient documentation to
assess the reliability of the estimates. In many cases, there were no
documents in the approval packages to support the estimates, and in
cases where there were documents, they generally provided high-level
estimates with little detail and no documentation to show how NASA
arrived at the estimates. We believe that because of its difficulty
providing reliable cost estimates, NASA cannot provide the Congress
assurance that its budget request for the shuttle program for fiscal
year 2006 will be sufficient and that shortfalls would not need to be
met through reductions in other NASA programs.
NASA stated that it believes the use of UCAs is both reasonable and
necessary for return to flight activities. We agree that UCAs may be
justified to facilitate work outside the scope of existing contracts to
expedite the return to flight activities. However, the use of UCAs
appears to be a growing trend and is a risky contract management
practice because it increases the potential for unanticipated cost
growth. In the past, we cited the agency's use of UCAs as one of the
reasons we retained contract management as a high-risk designation for
NASA to focus management attention on problem areas that involve
substantial resources.[Footnote 8]
Finally, NASA agrees that cost estimates for significant development
activities should be appropriately documented. According to NASA,
additional requirements for cost estimates and internal controls
recently established by the program represent a step in ensuring the
appropriate documentation is developed as solutions are identified. As
stated in our report, we believe that future compliance with this new
policy could potentially result in more credible budget estimates.
In a broader context, reliable and supportable cost estimating
processes are important tools for managing programs. Without this
knowledge, a program's estimated cost could be understated and thereby
subject to underfunding and cost overruns, putting programs at risk of
being reduced in scope or requiring additional funding to meet their
objectives. Further, without adequate financial and nonfinancial data,
programs cannot easily track an acquisition's progress and assess
actions to be taken before it incurs significant cost increases and
schedule delays.
As agreed with your offices, unless you announce its contents earlier,
we will not distribute this report further until 30 days from its date.
At that time, we will send copies to the NASA Administrator and
interested congressional committees. We will make copies available to
others upon request. In addition, the report will be available at no
charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this
report are acknowledged in appendix III.
Signed by:
Allen Li:
Director:
Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
To assess the basis for NASA's Hubble servicing mission cost estimate,
we analyzed NASA's estimate of the funding needed for a shuttle
servicing mission and supporting documentation, and we reviewed NASA
documents explaining the rationale for the decision and identifying
alternatives to shuttle servicing. We interviewed program and project
officials to clarify our understanding of the available cost
information and NASA's rationale for the decision. To test the
sufficiency of the support for the estimates provided by NASA, we
requested the analytical basis and documentary support for selected
portions of the estimates, primarily those with large dollar values. In
addition, we compared NASA's decision-making process with relevant
Office of Management and Budget and NASA guidance on information and
analyses recommended to enable decision-makers to select the best
alternative.
To determine the basis for NASA's cost estimate for implementing all of
the CAIB recommendations, we reviewed the CAIB report (volume 1),
NASA's return to flight implementation plan and budget estimates, and
agency documentation discussing the return to flight budget estimate.
We interviewed program officials to obtain a better understanding of
NASA's plans for returning the space shuttle to flight, the status of
that effort, and the estimated cost. To test the sufficiency of the
support for NASA's return to flight estimate, we requested the
analytical basis and documentary support for selected high dollar
portions of the estimate.
To accomplish our work, we visited NASA Headquarters, Washington, D.C;
and Goddard Space Flight Center, Maryland.
We performed our review from March through September 2004 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the National Aeronautics and Space
Administration:
Department of Health & Human Services:
Office of Inspector General:
NOV 22 2004:
Ms. Leslie G. Aronovitz:
Director, Health Care-Program Administration and Integrity Issues:
United States Government Accountability Office:
Washington, D.C. 20548:
Dear Ms. Aronovitz:
Enclosed are the Department's comments on your draft report entitled,
"Medicare-Advisory Opinions As A Means of Clarifying Program
Requirements" (GAO-05-129). The comments represent the tentative
position of the Department and are subject to reevaluation when the
final version of this report is received.
The Department provided several technical comments directly to your
staff.
The Department appreciates the opportunity to comment on this draft
report before its publication.
Sincerely,
Signed by:
Daniel R. Levinson:
Acting Inspector General:
Enclosure:
The Office of Inspector General (OIG) is transmitting the Department's
response to this draft report in our capacity as the Department's
designated focal point and coordinator for Government Accountability
Office reports. OIG has not conducted an independent assessment of
these comments and therefore expresses no opinion on them.
COMMENTS BY THE DEPARTMENT OF HEALTH AND HUMAN SERVICES (HHS) ON THE
U.S. GOVERNMENT ACCOUNTABILITY OFFICE'S (GAO) DRAFT REPORT ENTITLED
"MEDICARE: ADVISORY OPINIONS AS A MEANS OF CLARIFYING PROGRAM
REQUIREMENTS (GAO-05-129):
The HHS appreciates the opportunity to review and comment on GAO's
draft report. We appreciate GAO's efforts to assess the role that a
broader advisory opinion process might play in clarifying Medicare
regulations.
In its concluding observations on page 21, the draft observes that the
Medicare program and its implementing regulations are "inherently
complex," and we agree. We do currently engage in numerous efforts to
assist stakeholders' understanding of the program's complexities, the
most important of which are described below. Included among these
efforts are targeted advisory opinion processes conducted by our Office
of Inspector General (OIG) and also within our Office of General
Counsel. As the draft also acknowledges, however, advisory opinion
processes are limited in their ability to provide quick answers to
pressing problems. We agree.
We believe that an enhanced and more formal process of developing
advisory opinions would not be a successful pursuit at this time. We
believe such a process would be costly and just as slow as the current
processes, which have been streamlined to the extent possible, given
the large volume of pertinent guidance, which is constantly (and
necessarily) in flux. Given the complexity and broad scope of the
Medicare program, an enhanced effort to provide advisory opinions would
require a far larger professional staff than is available under current
resource constraints. It is also important to note that current HHS
opinions specifically state that they are limited in scope to the
specific arrangement described in the request and have "no
applicability to other arrangements, even those which appear similar in
nature or scope" and "may not be introduced into evidence in any matter
involving an entity or individual that is not a requestor of this
opinion." We believe that the usefulness of a broader set of opinions
would be similarly constrained.
Additionally, the GAO draft states that the presence of user fees is
among the critical factors in making currently operational advisory-
opinion processes effective. It is important to note that fees
collected for our advisory opinions are not paid to or retained by HHS,
and thus do not offset the costs of the staff time allocated to this
work.
We cite the following as important existing sources of the guidance
needed by Medicare stakeholders:
The Center for Medicare and Medicaid Services (CMS)'s customized
provider web pages allow physicians, hospitals, ambulances, and other
providers quick access to relevant Medicare information. These web
pages, found on www.cms.hhs.gov/providers, have associated Listservs
that ensure providers will get new information as it becomes available.
CMS has a Medicare Coverage Database on the CMS website,
www.cms.hhs.gov, that contains all I national and local coverage policy
and articles produced by contractors that provide additional coverage
guidance. CMS has also implemented a new web page, allowing for easier
provider and public access to recent Administrator decisions regarding
the Provider Reimbursement Review Board and issued by the Office of the
Attorney Advisor.
CMS publishes a Quarterly Provider Update on the first business day of
each quarter on the CMS web site to inform the public about regulations
and major policies currently under development during this quarter;
regulations and major policies completed or cancelled; and new/revised
manual instructions. This Update, makes it easier for providers,
suppliers, and the general public to be aware of impending program
changes.
In fiscal year 2004, CMS implemented an On-Line Manual System, located
at www.cms.hhs.gov/manuals, to consolidate and update its manuals,
policy and billing instructions, eliminate duplicate policy across
manuals, and establish a single source to obtain information on the
Medicare and Medicaid programs. Additionally, a related Monthly
Bulletin is communicated to Medicare Contractors via email and is
posted on the CMS On-Line Manual System.
In 2001, CMS began an initiative to improve provider communications
when the agency required contractors to institute toll-free phone
service to answer inquiries from providers who bill for services under
fee-for-service Medicare. CMS also now issues nationally consistent
provider education materials to accompany contractor instructions that
implement new or revised policy. "Medlearn Matters ... Information for
Medicare Providers" contains educational articles, written in
consultation with clinicians, billing experts, and other medical
professionals, and tailored in content and language to the specific
provider types who are affected by the program change. These articles
explain in plain English content of the program instructions, and, more
importantly, the specific impact that the change has on the affected
providers. The articles are housed in one central, easily accessible
location (www.cms.hhs.gov/medlearn/matters).
CMS has held fourteen individual Open Door Forums and town hall
meetings for physicians to discuss new initiatives. CMS has also
established a provider partnership network with provider associations
and organizations, whereby providers give input on products and CMS
information tools, and assist in the dissemination of CMS information.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Allen Li (202) 512-4841:
Acknowledgments:
Staff making key contributions to this report were Jerry Herley, Erin
Schoening, Karen Sloan, and Jonathan Watkins.
FOOTNOTES
[1] The CAIB was created by NASA to investigate the February 1, 2003,
loss of the space shuttle Columbia.
[2] The National Research Council is part of the National Academies,
which also comprise the National Academy of Sciences, National Academy
of Engineering, and Institute of Medicine. They are private, nonprofit
institutions that provide science, technology, and health policy advice
under a congressional charter. The Research Council was organized by
the National Academy of Sciences in 1916 to associate the broad
community of science and technology with the Academy's purposes of
further knowledge and advising the federal government.
[3] Although the autonomous inspection and repair capability of the
shuttle's thermal protection system is a CAIB requirement for all
shuttle missions, the currently planned method depends on International
Space Station assets, such as the International Space Station's robotic
arm to stabilize a spacewalk crew making repairs to the shuttle. A
Hubble mission would not have this asset available; therefore, NASA
would have to develop an alternate method for stabilizing a
crewmember making repairs. The second requirement--NASA's requirement
for a crew rescue shuttle--would apply to all shuttle flights, but
according to NASA, the agency would need to dedicate two shuttles to a
Hubble servicing mission because of the shorter amount of time to react
to an emergency because the crew would not have the benefit of the
International Space Station as a safe haven. The agency contends that
the second (rescue) shuttle for the Hubble servicing mission would need
to be on the adjacent launch pad in countdown mode at the same time the
first shuttle is launched. The rescue shuttle capability would require
the development of a second generation boom to be used to transfer the
crew from the stricken orbiter and an enhanced camera to process
imagery in time to support a go-no go rescue mission decision. In
addition, a crew return kit would be needed to provide seats on the
rescue shuttle for the rescued crew, and a special crew would have to
be trained for the rescue mission.
[4] GAO, NASA: Lack of Disciplined Cost-Estimating Processes Hinders
Effective Program Management, GAO-04-642 (Washington, D.C.: May 28,
2004).
[5] The membership of the PRCB includes the Space Shuttle Program
Manager, Deputy Manager, all Project and Element Managers, Safety and
Mission Assurance personnel, and the Team Leader of the return to
flight Planning Team.
[6] We attempted to obtain analytical and documentary support for the
return to flight estimates on two occasions. First, in May 2004, we
requested support for three high-dollar items for which we were told
the estimates ranged from strong to weak to a mixture. In response to
this request, NASA told us that the detailed bases of estimate for the
activities were being developed and that NASA would report on return to
flight expenses to Congress and GAO in a few weeks. Then, several weeks
later when NASA released the July 28 return to flight update, we
requested support for the three largest estimates, two of which were
the same items for which we requested support in May. In response, NASA
provided the PRCB directives and backup documents underlying the
estimates that the agency considers to be mature. After we reviewed
these documents and concluded that the support was not adequate, NASA
offered to select some additional examples to show that the estimates
were credible.
[7] GAO, NASA: Major Management Challenges and Program Risks, GAO-03-
849T (Washington, D.C.: June 12, 2003); GAO, High-Risk Series: An
Update, GAO-03-119 (Washington, D.C.: January 2003); GAO, Major
Management Challenges and Program Risks: National Aeronautics and Space
Administration, GAO-03-114 (Washington, D.C.: January 2003); GAO, High-
Risk Series: An Update, GAO-01-263 (Washington, D.C.: January 2001);
and GAO, Major Management Challenges and Program Risks: National
Aeronautics and Space Administration, GAO-01-258 (Washington, D.C.:
January 2001).
[8] High-Risk Series: An Update (GAO-01-263, January 2001).
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
