Contract Reform
DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results Gao ID: GAO-02-798 September 13, 2002The Department of Energy (DOE), the largest civilian contracting agency in the federal government, relies primarily on contractors to operate its sites and carry out its diverse missions, such as maintaining the nuclear weapons stockpile, cleaning up radioactive and hazardous wastes, and performing research. Although federal law generally requires federal agencies to use competition in selecting a contractor, until the mid-1990s, DOE contracts for the management and operation of its sites generally fit within an exception that allowed for the use of noncompetitive procedures. Since 1996, DOE has made progress toward implementing contract reform initiative in three key areas--developing alternative contracting approaches, increasing competition, and using performance-based contracts. However, DOE continues to encounter challenges in implementing these initiatives. Although DOE has made strides in implementing contract reform initiatives, it is difficult to determine whether contractors' performance has improved because objective performance information is scarce. Over the past 8 years, DOE has primarily gauged progress by measuring its implementation of the reforms, such as the number of contracts competed each year, and by reviewing individual contract performance incentives. DOE faces a fundamental challenge to ensuring the effectiveness of its contract reform initiatives--developing an approach to managing its initiatives and sustaining improvements that would incorporate the best management practices of high-performing organizations. These practices include four key elements: (1) clearly defined goals; (2) an implementation strategy that sets milestones and establishes responsibility; (3) results-oriented outcome measures, established early in the process; and (4) systematic use of results-oriented data to evaluate the effectiveness of the initiative and make additional changes where warranted.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-02-798, Contract Reform: DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have Improved Results
This is the accessible text file for GAO report number GAO-02-798
entitled 'Contract Reform: DOE Has Made Progress, but Actions Needed to
Ensure Initiatives Have Improved Results' which was released on
September 13, 2002.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Secretary of Energy:
United States General Accounting Office:
September 2002:
Contract Reform:
DOE Has Made Progress, but Actions Needed to Ensure Initiatives Have
Improved Results:
DOE Contract Reform:
GAO-02-798:
Contents:
Letter:
Results in Brief:
Background:
DOE Has Made Progress in Implementing Contract Reforms, but Challenges
Remain:
Contract Reforms May Not Have Improved Contractor Performance:
DOE‘s Approach to Contract Reform Was Not Consistent with Best
Practices:
Conclusions:
Recommendation for Executive Action:
Agency Comments:
Appendix I: DOE‘s Decisions to either Compete or Extend
Non-Competitively Its Major Site Contracts:
Appendix II: Cost and Schedule Performance on DOE‘s Major
Projects as of December 2001:
Appendix III: Comments from the Department of Energy:
Appendix IV: Scope and Methodology:
Appendix V: GAO Staff and Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Tables:
Table 1: Extent to which DOE Competitively Awarded Its Major Site
Contracts, 1991-2001:
Table 2: Ten DOE Federally Funded Research and Development Centers That
Do Not Have Competitively Awarded Contracts:
Table 3: Comparison of Cost Overruns and Schedule Delays for Ongoing
Projects in 2001 with Ongoing Projects in 1996:
Table 4: DOE Management Improvement Initiatives That Could Enhance
Contract Reform:
Table 5: Original and Current Cost Estimates and Schedule for DOE
Projects with Estimated Costs Greater Than $200 Million as of December
2001:
Figure:
Figure 1: Total Aggregate Fee Available to Contractors on Major Site
Contracts, by Type of Fee, Fiscal Years 1996 and 2001:
Abbreviations:
DEAR: Department of Energy Acquisition Regulation
DOE: Department of Energy
OIG: Office of Inspector General:
United States General Accounting Office:
Washington, DC 20548:
September 13, 2002:
The Honorable Spencer Abraham
Secretary of Energy:
Dear Mr. Secretary:
The Department of Energy (DOE), the largest civilian-contracting agency
in the federal government, relies primarily on contractors to operate
its sites and carry out its diverse missions, such as maintaining the
nuclear weapons stockpile, cleaning up radioactive and hazardous
wastes, and performing research. To carry out these missions, DOE often
contracts for designing, constructing, and operating multimillion-
dollar, one-of-a-kind facilities. For fiscal year 2001, DOE spent about
90 percent of its total annual budget, or $18.2 billion, on contracts.
Of that amount, DOE spent about $16.2 billion on contracts to manage or
operate 28 DOE sites.
DOE‘s contracting activities are governed by federal law and
regulations, including the Federal Acquisition Regulation and the
Department of Energy Acquisition Regulation (DEAR). Although federal
law generally requires federal agencies to use competition in selecting
a contractor, until the mid-1990‘s, DOE contracts for the management
and operation of its sites generally fit within an exception that
allowed for the use of noncompetitive procedures. Furthermore, those
contracts were subject to rules in the DEAR that established
noncompetitive extensions of contracts with incumbent contractors as
the norm and permitted competition only when it appeared likely that
the competition would result in improved cost or contractor performance
and would not be contrary to the government‘s best interests. DOE has
traditionally used a cost-plus-fee type of contract, in which DOE
reimburses a contractor for all allowable costs under the contract and
also pays a fee over and above the contractor‘s costs. In some
situations, however, DOE has used a fixed-price contract, in which a
contractor accepts responsibility for completing a specified amount of
work for a fixed price. In that case, the contractor earns a profit if
its total costs are less than the contract price and loses money if its
total costs exceed the contract price.
For over a decade, DOE has been criticized by GAO, DOE‘s Office of
Inspector General, and others for its contracting practices,
particularly inadequate management and oversight, and for failure to
hold its contractors accountable for results. Under its long-standing
approach to its site contracts, DOE developed a broadly defined
statement of work, provided considerable direction to the contractor,
and reimbursed virtually all costs. This approach placed limited
emphasis on cost control or accountability for results. Furthermore,
the poor performance of DOE‘s contractors led to schedule delays and
cost increases on many of the department‘s major projects. Since 1990,
such problems have led us to designate DOE contract management--defined
broadly to include both contract administration and management of the
projects--as a high-risk area for fraud, waste, abuse, and
mismanagement.
In 1994, DOE began its contract reform initiative to improve
contractors‘ performance. This initiative recommended numerous
changes, including making greater use of alternatives to traditional
site management and operating contracts, such as the use of more fixed-
price contracts; increasing the use of competition as a basis for
selecting contractors to perform the work; and developing and using
performance-based contracting tools to reward good performance and
penalize poor performance. In an earlier review of DOE‘s contract
reform initiative, we reported, among other things, that DOE had issued
revised policy and guidance to implement the reforms but that it still
was not using competition for some of its major contracts and that its
early efforts at performance-based contracting were not linking a
contract‘s objectives to DOE‘s overall strategic goals.[Footnote 1]
In addition to its contract reform initiative, DOE has begun several
other initiatives that could enhance contract reform efforts. These
include DOE‘s efforts to strengthen its management of projects (project
management initiative); develop and use information systems for
oversight and control (management information systems initiative); and
improve training and expertise of the DOE staff overseeing contractor
activities (human capital initiative). None of those initiatives had
been fully implemented at the time of our review.
As part of our ongoing review of the high-risk area of DOE contracting
and project management, we (1) assessed the progress that DOE has made
since 1996 in implementing contract reform initiatives in key areas;
(2) determined the extent to which these initiatives have resulted in
improved contractor performance; and (3) identified the challenges, if
any, that DOE faces in ensuring the effectiveness of its contract
reform initiatives.
Results in Brief:
Since 1996, DOE has made progress toward implementing contract reform
initiatives in three key areas--developing alternative contracting
approaches, increasing competition, and using performance-based
contracts. However, DOE continues to encounter challenges in
implementing these initiatives. Regarding alternative contracting
approaches, DOE tested several different types of contracts aimed at
controlling costs while achieving specific results, but it did not
systematically determine an appropriate contracting approach for a
given project or activity. For example, the department used
competitively awarded, fixed-price contracts for both small, relatively
simple projects, such as laundry services, as well as for large,
complex cleanup projects. Under a variation of this approach, known as
’privatization,“ the contractor finances, designs, constructs, owns,
and operates treatment facilities for waste or other material and
receives payments per unit of treated material. DOE‘s experiences
demonstrated that privatization was not successful in controlling costs
on large, complex cleanup projects. But privatization worked
effectively on smaller, less complex projects that posed fewer risks.
To strengthen the process of selecting a contracting approach, DOE is
developing and implementing a formal acquisition strategy that
systematically evaluates contract and financing alternatives and the
risks associated with various approaches. In the second reform area--
increasing competition--the department changed its contracting rules to
set competition as the standard approach to awarding contracts. As of
December 2001, DOE had awarded competitively 14 of 25 contracts (56
percent) for its major sites, an increase from the 38 percent awarded
competitively as of 1996. All but one of the contracts that had not
been awarded competitively as of December 2001 were for research and
development centers, which, according to federal law, are exempt from
mandatory competition. Although the department now evaluates whether to
extend or compete these contracts, thus far it has decided on
extensions. Finally, DOE now requires performance-based contracts at
all of its major sites. Such contracts incorporate results-oriented
statements of work and set the performance objectives and measures that
DOE will use to evaluate the contractor‘s performance. In addition, DOE
has increased over time the proportion of contractors‘ fees tied to
achieving the performance objectives. Nevertheless, numerous studies
and reports have criticized DOE‘s performance-based contracts for
ineffective performance measures. DOE continues to modify and test its
performance measures, for example by improving the linkage between
performance measures and the department‘s strategic goals.
Although DOE has made strides in implementing contract reform
initiatives, it is difficult to determine whether contractors‘
performance has improved because objective performance information is
scarce. Over the past 8 years, DOE has primarily gauged progress by
measuring its implementation of the reforms, such as the number of
contracts competed each year, and by reviewing individual contract
performance incentives. While this information is important, it does
not help the department determine if, for example, competing more
contracts resulted in more favorable contract terms or better
performing contractors, or if performance-based contracts resulted in
shorter project schedules, reduced costs, or other improvements.
Nevertheless, DOE program managers and procurement officials at DOE
headquarters and several sites believe that contract reforms have made
a difference. These officials offered anecdotes as the primary evidence
for their view. Officials at DOE‘s Albuquerque operations office
pointed out, for example, that after awarding competitively the
contract at the Pantex site in Texas, the current contractor met
required production levels that were not achieved by the previous
contractor. However, anecdotal examples of poor performance are also
easily identified. For example, we reported in August 2000 that
management and oversight failures had caused major cost overruns and
schedule delays on the National Ignition Facility at the Lawrence
Livermore National Laboratory. These problems occurred despite
incorporating performance measures into the overall contract with the
University of California, which operates the laboratory and manages the
construction project. Furthermore, according to DOE‘s February 2002
review of its Environmental Management program, the program needed to
improve significantly its management of performance-based contracts and
focus on accomplishing measurable results. The Assistant Secretary for
Environmental Management reported that the program indicators ’measured
process, not progress, opinions, not results.“ As a potential indicator
of improved performance, we evaluated changes in cost and schedule for
16 of DOE‘s current major projects and compared those changes to
similar information we developed on DOE‘s major projects in 1996. We
found no indication of improved performance--in both groups of projects
over half of the ongoing projects were experiencing significant cost
increases, schedule delays, or both. Although the comparison provides
only a limited view of contractor performance, it does raise questions
about the overall impact of DOE‘s contract reform efforts.
DOE faces a fundamental challenge to ensuring the effectiveness of its
contract reform initiatives--developing an approach to managing its
initiatives and sustaining improvements that would incorporate the best
management practices of high-performing organizations. These practices
include four key elements: (1) clearly defined goals; (2) an
implementation strategy that sets milestones and establishes
responsibility; (3) results-oriented outcome measures, established
early in the process; and (4) systematic use of results-oriented data
to evaluate the effectiveness of the initiative and make additional
changes where warranted. DOE has an implementation strategy for its
contract reform initiatives, but it largely omitted the other three key
elements from its reform efforts. For example, by not setting specific
goals for its contract reform initiatives, DOE emphasized contract
reform itself as a goal rather than improved results. Similar
weaknesses in other management improvement initiatives, such as the
lack of results-oriented outcome measures to gauge progress in DOE‘s
project management initiative, could also limit DOE‘s ability to obtain
better performance from its contractors. However, if successful, the
project management initiative may help to improve contractor
performance by providing early indications of problems with cost and
schedule on projects. We are recommending that to ensure the
effectiveness of its contract reform initiative, as well as other
management improvement initiatives, DOE take steps to align its
management of the initiatives with current best practices.
We provided a copy of our draft report to DOE for review and comment.
DOE agreed with our recommendation that it adopt an approach to
implementing management improvement initiatives that is more consistent
with the best practices of high-performing organizations. However, DOE
also said that characterizing its contract reform initiative as a
fundamental management challenge does not fully capture the real issues
facing the department and creates a false sense that the procurement
system is capable of solving all of DOE‘s problems. Although we agree
that DOE faces many other challenges, we also believe that, within the
context of those challenges, strengthening contract management is an
important need. DOE also said that its contract reform initiative was
managed in a systematic manner and that our report implies otherwise.
We revised our report to clarify our concern that DOE‘s approach to
contract reform was not consistent with the best practices found in
high-performing organizations. Finally, DOE said that evidence of its
troubled projects has less to do with contract management issues and
more to do with program and project management issues. We believe that
the extended discussion in our report fairly describes the relationship
between contract reform and these other factors. DOE‘s comments are
presented in appendix III.
Background:
DOE‘s missions include developing, maintaining, and securing the
nation‘s nuclear weapons capability; cleaning up the environmental
legacy resulting from over 50 years of producing nuclear weapons; and
conducting basic energy and science research and development. The
department carries out these diverse missions at over 50 major
installations in 35 states. With a DOE workforce of about 16,000
employees and over 100,000 contractor staff, the department relies on
its contractors to manage and operate its facilities and accomplish its
missions. DOE manages these functions through its program offices at
DOE headquarters and its field offices. The three largest program
offices--Environmental Management, Defense Programs, and Science--
accounted for over 70 percent of DOE‘s budget for fiscal year 2001.
DOE‘s reliance on contractors to carry out its missions and the
department‘s history of both inadequate management and oversight and
failure to hold its contractors accountable for results led us to
designate DOE contract management as a high-risk area vulnerable to
fraud, waste, abuse, and mismanagement. In response to these and other
criticisms, DOE began evaluating its contracting practices and, in
February 1994, issued a report--Making Contracting Work Better and Cost
Less--that contained 48 recommendations. The recommendations included
three key areas: selecting alternatives to traditional contracting
arrangements used for management and operation of its sites, increasing
competition to improve performance, and developing and using
performance-based contracting tools. To facilitate and oversee the
implementation of the contract reform recommendations, in June 1994,
DOE established the Contract Reform Project Office, which became the
Office of Contract Reform and Privatization in 1997. This office, which
monitored and assessed the progress of DOE‘s contract reform
initiative, was disbanded in late 2001 as part of the department‘s
reorganization of its support offices. DOE‘s Office of Management,
Budget, and Evaluation/Chief Financial Officer is now responsible for
oversight of DOE‘s contract reform efforts.
DOE Has Made Progress in Implementing Contract Reforms, but Challenges
Remain:
Since 1996, the department has made progress in implementing three key
contract reform initiatives--developing alternative contracting
approaches, increasing competition, and converting to performance-
based contracts, although DOE continues to address challenges in
implementing these initiatives. Concerning alternative contracting
approaches, DOE encouraged the use of different types of contracts
aimed at improving contractor performance and results. However, DOE did
not use a systematic approach to determine the best contract type for a
given situation and experienced problems with implementation. To become
more systematic in making this contract selection decision, DOE has
been developing a formal strategy to evaluate contract and financing
alternatives and the risks associated with various approaches. In the
second reform area--increasing competition--DOE changed its
contracting rules to set competition as the standard approach to
awarding contracts. Under these rules, the percentage of major site
contracts awarded competitively (competed) increased to 56 percent as
of 2001, up from 38 percent as of 1996. All but one of the 11 contracts
that had not been competed were for managing research and development
centers exempted by statute from mandatory competition. The department
evaluates these contracts to determine whether they should be extended
or competed. DOE has thus far decided on non-competitive extensions for
these contracts, including some for contractors that have experienced
performance problems. DOE opted to address these performance problems
with specific contract provisions, but it remains to be seen whether
this approach will succeed. Finally, all of DOE‘s major site contracts
are now performance-based, incorporating results-oriented statements
of work and the performance objectives and measures used to evaluate
contractor performance. To further emphasize the importance of the
performance-based approach, DOE has increased the proportion of
contractor fees tied to achieving the performance objectives to 70
percent in fiscal year 2001 from 34 percent in fiscal year 1996.
However, development of good performance measures has continued to be a
challenge, and DOE acknowledges that it must make further progress in
this area.
DOE Implemented Alternative Contract Approaches, but Did Not Establish
an Acquisition Strategy:
One of the major focuses of DOE‘s contract reform initiative has been
developing alternatives to the traditional contracts used for the
management and operation of its major sites and facilities. Under these
’management and operating“ contracts, one primary contractor performed
almost all of the work at a site, the contractor had broadly defined
statements of work, and DOE reimbursed the contractor for virtually all
costs. As a result, work under these contracts focused more on annual
work plans and budgets rather than on specific schedule and cost
targets for accomplishing work.
In implementing alternatives to its traditional contracting
arrangements, DOE‘s intent was to use the best contracting alternative
given the required work and the objectives and risks associated with
that work. DOE implemented four main actions as alternatives to these
management and operating contracts, but has experienced problems with
implementation, in part due to difficulties in determining the most
appropriate approach for a given situation, as follows:
* Reducing the number of large, cost-reimbursement contracts that cover
virtually all of the activities at a DOE site. DOE has modified a total
of 20 site contracts since 1994, so that no single contractor manages
and operates those sites. Some of these management and operating
contracts were divided into smaller service contracts, such as for
guard services. Other management and operating contracts were changed
to integration contracts (commonly called management and integration
contracts). According to DOE officials, integration contracts were used
to better reflect the changing mission of the site and to better tailor
the contract scope to the program requirements. Under a management and
integration contract, one contractor is responsible for integrating the
work of a variety of subcontractors that carry out most of the actual
work at the sites. The integrating contractor is responsible for
selecting ’best-in-class“ subcontractors for specific work activities,
overseeing the work done by the subcontractors, and ensuring that
activities at the site are effectively coordinated. DOE has used this
integration contract approach at sites such as Oak Ridge in Tennessee
for environmental restoration work. However, DOE‘s Office of Inspector
General reported in March 2001 that the integrating contractor at Oak
Ridge has subcontracted out a third less work than originally proposed,
resulting in less cost savings to the government.[Footnote 2]
* Implementing a more disciplined approach to ’make-or-buy“ decisions
by site contractors. DOE revised its regulations in 1997 to require
that its major site contractors develop make-or-buy plans instead of
having most of the work at a site performed by the primary contractor.
Under these plans, the primary contractor must identify work functions
that could be performed at less cost or more effectively through
subcontracts. Although all of its major contractors have approved make-
or-buy plans, DOE acknowledges that it does not routinely gather
information on how much work is done by subcontractors, making it
difficult to determine the extent to which this approach was
implemented. In addition, DOE‘s Office of Inspector General reported in
February 2000 that three of the four contractors that it reviewed had
either not included all functions in their make-or-buy plans or had not
done the required cost-benefit analysis on work functions that could
have been subcontracted.[Footnote 3]
* Implementing an alternative contracting and financing approach called
privatization. DOE started its ’privatization initiative“ in 1995 as a
way to reduce the cost and speed the cleanup of its contaminated sites.
This initiative was primarily an alternative contracting and financing
strategy to foster open competition for fixed-price contracts; to
require the contractor to design, finance, build, own, and operate the
facilities necessary to meet waste treatment requirements; and to pay
the contractor for units of successfully treated waste. DOE‘s
experiences with this approach showed that privatization could achieve
cost savings on projects with a well-defined scope of work and few
uncertainties, such as laundry facilities for contaminated uniforms and
other items at the Hanford site. However, on complex cleanup projects
such as the effort at Idaho Falls to clean up Pit 9,[Footnote 4]
privatization had little success in achieving cost savings, keeping the
project moving forward on schedule, or getting improved contractor
performance.
* Establishing ’closure contracts“ that tie performance incentives to
contract completion, not to annual activities. DOE has used closure
contracts at several sites that are scheduled for cleanup and closure,
including the Rocky Flats site in Colorado and the Fernald site in
Ohio. These contracts emphasize completing all work at a site or a
portion of a site by a target date and at a target cost. Most of the
fee or profit to be earned by the contractor depends upon meeting the
schedule and cost targets. If the contractor can complete all work on
time or sooner and below the target cost, then the contractor can earn
additional fee. For example, under the Rocky Flats closure contract,
the amount of incentive fee that the contractor can earn ranges from
$130 million to $460 million, depending on cost and schedule
performance against the targets. Since the target closure date for this
contract is December 2006, it remains to be seen whether this approach
will be effective in completing the work on time and at lower costs to
the government.
These problems reflected the lack of a systematic approach to deciding
which contract type was best for a given situation. For example, we
reported in May of 1998 that DOE‘s use of fixed-price contracting was
appropriate when projects were well-defined, when uncertainties could
be allocated between DOE and the contractor, and when either adequate
cost information or multiple competing bidders were available to
determine a fair and reasonable price for the work.[Footnote 5]
However, when these conditions did not exist, cost overruns and
schedule delays could occur on these fixed-price contracts.
DOE has begun to develop a more systematic approach to determining the
best contract type for a given situation. For example, in October 2000,
DOE issued new policy and guidance for the acquisition of capital
assets such as waste treatment facilities. The guidance includes
developing an acquisition plan that considers the financial, technical,
and performance risks associated with a new project. This policy is
consistent with DOE‘s overall goal of tailoring the contract type to
the work to be performed and the business and technical risks
associated with that work. In addition, to strengthen oversight of
major acquisitions, in November 2001 DOE issued additional guidance
that requires approval of acquisition plans for projects of $5 million
and above at the assistant secretary level or higher. Despite these
initial steps, DOE is still developing and implementing its formal
acquisition strategy, and it is too soon to tell whether this new
strategy will help DOE make better decisions about how to acquire
capital assets.
Use of Competitively Awarded Contracts Increased, and Allowable
Exceptions Continue:
DOE has increased the proportion of major site contracts awarded
competitively, but still extends a number of these site contracts non-
competitively, as allowed by procurement law, including contracts for
some sites that have experienced contractor performance problems. DOE
competed 56 percent of its major site contracts that were up for award
or renewal from 1997 through 2001, a significant increase over the 38
percent it had competed from 1991 through 1996 (see table 1).
Table 1: Extent to which DOE Competitively Awarded Its Major Site
Contracts,
1991-2001:
Major site contract awards: Contracts awarded through competition[A];
1991-1996: 11; 1991-1996: (38%); 1997-2001: 14; 1997-2001: (56%).
Major site contract awards: Contracts extended or awarded without
competition; 1991-1996: ; 18; 1991-1996: ; (62%); 1997-2001: ; 11;
1997-2001: ; (44%).
Major site contract awards: Totals; 1991-1996: 29; 1991-1996: (100%);
1997-2001: 25; 1997-2001: (100%).
[A] To be classified as a contract awarded through competition, DOE
must have issued a request for proposals and a public announcement
inviting proposals. In addition to the above contracting actions, in
several cases DOE exercised an option to extend a competitively awarded
contract for a second period of up to 5 years. From 1991 to 1996, DOE
exercised an option to extend one such contract non-competitively. From
1997 to 2001, DOE exercised options to extend seven such contracts non-
competitively. We did not include those non-competitive extensions in
the table calculations.
Source: GAO analysis of DOE data.
[End of table]
During the 1997 through 2001 period, DOE selected new contractors for
10 of the 14 competitively awarded contracts, compared to 9 new
contractors for the 11 competitive awards from 1991 through 1996.
(Appendix I contains a listing of DOE‘s major site contracts in 2001
and the extent to which they have been competed). The growth in
competition at major DOE sites is largely a result of new regulations
the department issued under contract reform. The new rules generally
require competition for major site contracts and allow a contract
period consisting of an initial term of up to 5 years with options to
extend the contract provided that the total contract period does not
exceed 10 years.
Many of the contracts that DOE did not compete have been for its
federally funded research and development centers for which DOE may
extend contracts non-competitively under the Competition in Contracting
Act of 1984.[Footnote 6] By 2001, all but one of the 11 contracts
extended without competition fell under this exemption for research and
development centers. The exception was the major site contract for the
management of DOE‘s West Valley Demonstration Project in New York. DOE
extended the contract in 1998 and recently announced plans for another
extension. According to DOE procurement officials, this recent
extension was because of the limited amount of cleanup work remaining
at the site and the lack of interest by other contractors to compete
for the work.
As part of its overall effort to increase competition for site
contracts, DOE also reassessed which sites it should continue to
designate as federally funded research and development centers. As a
result of the reassessment, DOE has removed six of 22 sites from the
federally funded research and development center designation. The
department subsequently competed the contracts for two of these, the
Knolls and Bettis Atomic Power Laboratories in New York and
Pennsylvania. The department restructured the other four contracts and
no longer regards them as major site contracts. In six other instances,
although DOE has thus far decided the sites should remain designated as
federally funded research and development centers, the department has
competed the contracts even though federal law and regulations allow
DOE to extend the contracts non-competitively. These six competed
contracts included those for the Oak Ridge National Laboratory in
Tennessee and the Idaho National Engineering and Environmental
Laboratory.
In addition to its reassessment effort, in 1996 the department issued
guidance that it must follow to support any recommendation for a non-
competitive extension of any major site contract. Among other things,
the guidance called for DOE to provide:
* a certification that competition is not in the best interest of the
department,
* a description of the incumbent contractor‘s past performance,
* an outline of the principal issues and/or significant changes to be
negotiated in the contract extension, and:
* in the case of a federally funded research and development center, a
showing of the continued need for the research and development center.
Based on such documentation, the agency head can authorize a contract
extension of up to 5 years. Table 2 lists the ten federally funded
research and development centers for which DOE has awarded contracts
non-competitively since this guidance was issued.
Table 2: Ten DOE Federally Funded Research and Development Centers That
Do Not Have Competitively Awarded Contracts:
Site name: Ames National Laboratory; Site contractor: Iowa State
University; Year contractor began operating the site: 1943.
Site name: Argonne National Laboratory; Site contractor: University of
Chicago; Year contractor began operating the site: 1946.
Site name: Fermi National Laboratory; Site contractor: Universities
Research Association; Year contractor began operating the site: 1967.
Site name: Jefferson Lab; Site contractor: Southeastern Universities
Research Association; Year contractor began operating the site: 1984.
Site name: Lawrence Berkeley National Laboratory; Site contractor:
University of California; Year contractor began operating the site:
1947.
Site name: Lawrence Livermore National Laboratory; Site contractor:
University of California; Year contractor began operating the site:
1952.
Site name: Los Alamos National Laboratory; Site contractor: University
of California; Year contractor began operating the site: 1943.
Site name: Pacific Northwest National Laboratory; Site contractor:
Battelle Memorial Institute; Year contractor began operating the site:
1964.
Site name: Princeton Plasma Physics Laboratory; Site contractor:
Princeton University; Year contractor began operating the site: 1975.
Site name: Stanford Linear Accelerator Facility; Site contractor:
Stanford University; Year contractor began operating the site: 1976.
Source: GAO analysis of DOE data.
[End of table]
DOE‘s decision not to compete some of the federally funded research and
development center contracts has not been without controversy. For
example, in 2001, DOE extended the management and operating contracts
with the University of California for the Los Alamos and Lawrence
Livermore National Laboratories. The University of California has
operated these sites for 50 years or more and is the only contractor
ever to have operated them. In recent years, we and other organizations
have documented significant problems with laboratory operations and
management at these two laboratories--particularly in the areas of
safeguards, security, and project management.[Footnote 7]
Congressional committees and others have called for DOE to compete
these contracts. Even with these problems and concerns, however, DOE
chose not to compete these contracts. This decision was made at the
highest levels in the department and was based on national security
considerations. Rather than compete these contracts, DOE intends to
address these performance problems using contract mechanisms.
In the 2001 contract extension, DOE required the university to focus on
strengthening management performance in five areas, including
initiatives for safety and project management. For the first 2 years of
the 5-year contract period, the University of California must meet
specific requirements before it can earn any of the $17 million in
incentive fees available under the contract. DOE is to assess the
university‘s performance on these specific requirements on a pass/fail
basis. After the first 2 years of the contract, performance in these 5
areas will be assessed as part of the regular performance measures in
the contract. The department‘s first (2001) annual assessment found
that the contractor was meeting the required milestones for all of the
improvement initiatives. However, many of the milestones in the first
year involved evaluating existing systems or developing action plans.
For other objectives that focus on results, such as demonstrating
improved performance in nuclear facility operations, the final outcomes
will not be known for several years. Therefore, it remains to be seen
whether DOE will be successful in improving the University of
California‘s performance using these contracting tools. If the
University of California does not make significant improvements in its
performance, DOE may need to reconsider its decision not to compete the
contracts.
Contracts Are Performance-Based, but DOE Has Difficulty Developing
Effective Performance Objectives and Measures:
DOE has reported that all of its major site contracts incorporate
performance-based techniques to define requirements and measure
results. Before DOE initiated its contract reforms, major site
contracts generally had broad statements of work that focused more on
annual budgets and work plans rather than specific results to be
achieved. Fees[Footnote 8] under these contracts usually consisted of a
base fee that was guaranteed (fixed) plus an award fee that was paid if
the contractor met general performance expectations. In the mid-1990s,
DOE began restructuring its major site contracts to use results-
oriented statements of work and, for most of the major site contracts,
to incorporate performance incentive fees that were designed to reward
the contractor if it met or exceeded specific performance expectations
in priority areas. These fees may be tied to either subjective or
objective performance measures, but DOE regulations suggest the use of
specific and quantifiable measures whenever possible. In 1999, DOE
issued additional regulations that limited the use of base fee and
established a clear preference for contracts where all of the fee was
based on a contractor‘s performance.
Since DOE changed its policy in favor of using incentive fees, there
has been a substantial shift in the type of fees available on DOE
contracts. As shown in figure 1, between fiscal years 1996 and 2001,
DOE decreased the total aggregate amount of base and award fee
available to its contractors and substantially increased the amount of
fee that is based on performance incentives. For individual contracts,
the percentage of each fee type varied widely. For example, in fiscal
year 2001, the Sandia National Laboratories contract had 100 percent
base fee, and the Oak Ridge National Laboratory contract had 100
percent performance incentive fee.
Figure 1: Total Aggregate Fee Available to Contractors on Major Site
Contracts, by Type of Fee, Fiscal Years 1996 and 2001:
[See PDF for image]
Note: The 2001 fee data excludes two site contracts--Fernald and Rocky
Flats. These contracts included multi-year incentive fees that will not
be computed until site closure occurs. The contracts call for site
closure in 2010 and 2006, respectively.
Source: GAO analysis of DOE Data.
[End of figure]
In addition to shifting most of the fee available to incentive fee, in
1999, DOE also established a new contract clause making payment of fee
conditional on meeting certain safety requirements and other minimum
requirements in the contract. According to language in this clause, in
order to receive all of the earned fee, the contractor must meet, among
other requirements, minimum environment, safety, and health
requirements and avoid any ’catastrophic“ events such as a fatality or
serious workplace-related injury. Since 1999, DOE has withheld over $5
million in fees from six contractors under this conditional payment of
fee clause. The largest fee withheld--$2 million--was from CH2M Hill
Hanford Group, Inc., for ’failures to meet the contractually imposed
minimum environment, safety, and health performance requirements“ as
defined by the contractor‘s integrated safety management system.
Although these changes reflect a marked shift in DOE‘s approach, the
lack of good performance measures blunted their effect. Since 1997,
numerous studies and reports--both internal and external to the
department--criticized DOE‘s performance-based contracts for
ineffective performance measures. Examples include the following:
* DOE‘s Office of Inspector General has issued 11 reports since 1997
that found multiple problems with DOE‘s performance measures. In 2001,
the Inspector General reported, after reviewing the Office of River
Protection Tank Farm Management, Oak Ridge Y-12 Plant, and Kansas City
Plant contracts, that DOE was not focusing on high priority outcomes,
was loosening performance requirements over time without adequate
justification, and was failing to match appropriately challenging
contract requirements with fee amounts. The department disagreed with
this report, stating that it was not appropriate to evaluate the
overall success of performance-based contracts by looking at individual
performance measures.
* In 1999, reporting on a self-assessment of its performance-based
contracting practices, DOE concluded that while significant
improvements had been made in the management of performance-based
contracts, several issues had arisen. These issues included
difficulties with measuring the results of basic science activities,
establishing performance measures that were consistent with project
baselines, determining the appropriate use of incentive fees for non-
profit contractors, and balancing incentives that both challenge the
contractor and continue to reward performance that has been sustained
at an excellent level.
* In its 1999 review of project management at DOE, the National
Research Council found that DOE did not always take advantage of the
performance-based incentive approach and did not have standard methods
for measuring project performance. The council‘s 2001 follow-up
assessment stressed the importance of using methods such as
performance-based contracting to focus contractors on achieving desired
results. The council added that success would be determined by how well
these methods are followed and recommended that DOE strengthen its
performance-based contracting guidance and practices.
In response to these and other criticisms of its performance-based
incentives, DOE has taken several actions that include issuing criteria
for a performance incentive development process at the field office
level and focusing on developing performance incentives more directly
linked to a site‘s strategic objectives. For example, DOE officials
said that multi-year incentives in the Hanford contract and multi-site
incentives that tie together activities at four production sites--
Kansas City in Missouri, Savannah River in South Carolina, Pantex in
Texas, and Y-12 at Oak Ridge, Tennessee--strive to establish the
strategic focus that was absent from performance incentives in earlier
contracts. DOE officials pointed out that, with these new incentives,
greater progress was being made. For example, the Hanford site had
reached its cleanup goals for fiscal year 2001. However, it remains to
be seen if contractors will meet milestones throughout the contracts‘
full length and, if they do not, if DOE will require contractors to
forfeit the provisional fee payments as allowed under the contracts.
Contract Reforms May Not Have Improved Contractor Performance:
Although DOE has made strides in implementing its contract reform
initiatives and has reviewed the performance measures in many of its
contracts, the department has developed little objective information to
demonstrate whether the reforms have resulted in improved contractor
performance. In the early years of contract reform, DOE measured
progress in terms of developing and issuing new contracting policies
and guidance. As new policies were established, the department also
focused on assessing its progress in implementing these policies in key
areas of competition and performance-based contracting. More recently,
DOE has reviewed many of its site contracts to determine, among other
things, whether the performance incentives are working properly. While
these steps are useful, this information does not help DOE determine
outcomes--whether, for example, competing more contracts resulted in
more favorable contract terms for the government or better performance
from its contractors. DOE program managers and procurement officials at
DOE headquarters and several sites believe that contract reforms have
resulted in improved contractor performance, and they cite a number of
examples where they believe contractor performance has improved.
However, there are also numerous examples of contractors who performed
poorly. Furthermore, DOE‘s February 2002 review of its Environmental
Management program observed that significant progress in cleanup and
risk reduction had not been achieved despite the performance-based
contracting approach. Since DOE does not have measures to determine
whether the contract reform initiatives had resulted in improved
performance, we examined the extent of cost overruns and schedule
delays on a number of DOE‘s major projects as a partial indicator of
success. For these projects, cost and schedule data showed no
improvement when compared to similar data in 1996. While this
performance information provides only a limited view of department-wide
contractor performance, it does raise questions regarding the overall
effectiveness of the reform initiatives.
DOE‘s Measurement Focused on Implementation, Not Outcomes:
At the outset of contract reform, DOE established specific action steps
and related time frames for changing its contracting practices. For
example, DOE set a goal of developing guidance by August 1994 for
increasing competition in awarding contracts. Subsequently, DOE
proposed new regulations concerning contract reforms in the areas of
competition, performance-based contracting and fee policies. As the
department‘s contract reform activities shifted from issuing guidance
to restructuring actual contracts, officials began to monitor the
extent to which its contracting organizations adopted DOE‘s contracting
policy changes in key reform areas. Because the contract cycle for the
large site contracts was so long--typically contracts were renewed
about every 5 years--DOE encouraged early incorporation of contract
reform principles as each contract came up for renewal.
Over the 8 years since the contract reform initiative was introduced,
DOE has primarily gauged its progress by monitoring implementation of
the reforms and reviewing individual contracts rather than by
developing objective measures to determine whether the reforms have
resulted in improved contractor performance. In addition to tracking
the number of contracts that incorporated the new requirements to use
competition and performance-based features, the department reviewed the
implementation of performance-based contracting for many of its major
contracts. Some examples of DOE‘s monitoring activities include:
* DOE‘s annual performance reports required under the Government
Performance and Results Act contained measures for both competing major
site contracts and converting them to performance-based contracts. In
1999, DOE reported that it exceeded the goal of awarding at least 50
percent of the major site contracts using competitive procedures. In
the reports for the years 1999, 2000, and 2001, DOE met its performance
goals to convert all major site contracts awarded in each year to
performance-based contracts.
* DOE‘s Office of Procurement and Assistance Management monitored the
contracts awarded at major sites. For the years 1997 through 2000, the
office reported that DOE met its annual goal of awarding contracts that
were performance-based at all of the major sites.
* DOE maintains a Web site that provides information on the status of
its procurement goals. These goals include increasing the use of
competition in awarding contracts and of performance-based concepts in
those contracts. DOE‘s Web site reports that as of 2001, 26 of its
major site and facility contracts were competed and that 100 percent of
these major contracts are performance-based.
* In 1997, the department‘s self-assessment of contract reform
determined that progress had been made in implementing contract reforms
across the complex. However, the report noted difficulties in
identifying and quantifying contract reform data and recommended on-
going analysis of key reform areas such as the effectiveness of fixed-
price contracting.
* In both 1997 and 1999, the department reported on its use of
performance-based incentives in major site contracts. The department
documented considerable progress in developing guidance and in
incorporating performance-based incentives but also found that early
incorporation of performance-based concepts had resulted in some poorly
structured incentives. For example, performance incentives were
sometimes overly focused on process milestones rather than outcomes.
The 1997 report recommended issuing guidance on how to restructure
performance objectives, but not on how to assess the effectiveness of
the restructured incentives. The 1999 report concluded that the quality
of contractor performance incentives had improved and that the
performance incentives were incorporated into contracts in a more
timely manner. The report further stated that the best measure of the
effectiveness of the incentives was improvement in contractor
performance. The report discussed specific contracts but did not
present overall data on contractor performance.
Procurement and program officials in headquarters continue to be
actively involved in developing and reviewing performance measures in
major site contracts. DOE officials said this oversight is improving
the quality of performance incentives and providing valuable
information on lessons learned. They acknowledged, however, that DOE
has not developed objective information on the outcomes associated with
the reforms. Such results-oriented information is important to
determine the extent to which the contract changes have resulted in
improved contractor performance.
Anecdotal Evidence Provides No Overall Measure of Improved Performance:
Although objective performance information focusing on results is not
available, DOE program managers and procurement officials at both DOE
headquarters and field operations offices believe that contract reforms
have made a difference. In support of this view, DOE officials
generally provide examples that they believe demonstrate improved
contractor performance. For example, officials at DOE‘s Albuquerque
operations office pointed out that after competing the contract for the
Pantex site, the new contractor met required production levels that
were not achieved by the previous contractor. These officials also
mentioned that the poor performance by the previous contractor was one
of the deciding factors in competing the contract for the Pantex site.
In addition to the examples of improved performance provided by DOE
officials, DOE‘s 1999 review of its performance-based contracting
practices reported that ’anecdotal evidence supports that the proper
use of well-structured performance-based incentives is leading to
improvements in performance at some DOE sites.“[Footnote 9] One of the
examples cited in this internal review was improved performance at the
Rocky Flats site under a performance-based contract established in
1995. Under the previous contract with a broad statement of work, the
contractor was primarily safeguarding and maintaining facilities at the
site, and no buildings had been decontaminated, demolished, and
removed. When DOE competed the contract in 1995 and selected a new
contractor, DOE also incorporated performance measures into the
contract. Consistent with these measures, the new contractor
decontaminated, demolished, and removed six buildings during fiscal
year 1996 and 12 during fiscal year 1998.
Other examples demonstrate, however, that the instances DOE cites are
not necessarily representative of the overall performance of DOE‘s
contractors. Examples of poor performance by DOE‘s contractors include
the following:
* DOE has experienced major cost overruns and schedule delays on the
National Ignition Facility at Lawrence Livermore National Laboratory in
California. This facility, the size of a football stadium, is designed
to produce intense pressures and temperatures to simulate in a
laboratory the thermonuclear conditions created in nuclear explosions.
DOE considers the facility to be an essential component of the program
to ensure the safety and reliability of the nuclear weapons stockpile
in the absence of nuclear testing. Although DOE had incorporated
performance-based measures and incentives into the overall contract
with the University of California, which operates the laboratory and
manages the construction project, performance problems still occurred.
We reported in August 2000 that the estimated cost of this facility had
increased from $2.1 billion to $3.3 billion and that the scheduled
completion date had been extended by 6 years to 2008. We attributed
these major cost and schedule changes to inadequate management by the
contractor and DOE oversight failures. [Footnote 10] We also found that
the performance-based contract placed little emphasis on the National
Ignition Facility project even though it dominated the laboratory‘s
budget and mission. DOE withheld $2 million of the fiscal year 1999
performance fee in recognition of the ’significant mission disruption“
caused by problems with this project. DOE officials said that the
department has since modified the performance-based contract to
increase the emphasis on this project and has taken additional steps to
improve both contractor management and DOE oversight.
* DOE has had problems with cost and schedule performance on its
contract for the Mound site in Ohio. In August 1997, DOE awarded a
cost-plus-award fee performance-based contract for the accelerated
cleanup of the Mound site. This contract called for cleaning up the
site and transferring facilities to the local community by no later
than September 2005 at a total estimated cost of $427 million. In May
2001, DOE‘s Office of Inspector General reported that the department
and the contractor had committed to that schedule without knowing
whether the date was achievable and that the cost and schedule had been
established with limited knowledge of the soil and building
contamination.[Footnote 11] The report added that completion of this
work was estimated for December 2009 at a cost of over $1 billion.
DOE is becoming aware of the problems with relying heavily on anecdotal
information when trying to assess outcomes. Officials in one of DOE‘s
largest program offices--Environmental Management, representing almost
a third of the department‘s overall budget--recently reported
fundamental problems with their program, and with the department‘s
ability to manage for results. In a February 2002 review, the office
stated that although the Environmental Management program had spent
over $60 billion since 1989, little progress had been made toward
cleaning up radioactive and hazardous wastes resulting from over 50
years of producing nuclear weapons, or toward reducing risks to the
public and the environment.[Footnote 12] During fiscal years 2000 and
2001, however, most of the contractors at Environmental Management
sites had earned more than 90 percent of their available performance
incentive fee, indicating that the contractors were successfully
achieving the performance goals established in their contracts. The
Assistant Secretary for Environmental Management reported that if such
’successes“ can take place without significant progress in cleanup and
risk reduction, the program has been using the wrong set of indicators
to measure success. She added that Environmental Management program
indicators ’measured process, not progress, opinions, not results.“
Among the conclusions in the report was that the Environmental
Management program needed to significantly improve its management of
performance-based contracts, focus on accomplishing measurable
results, and align contractors‘ performance fees with end points rather
than intermediate milestones.
Status of Major DOE Projects Does Not Indicate Improved Performance:
Based on our review of the performance of selected projects, it does
not appear that DOE‘s contractors have significantly improved their
performance since 1996. Because we could not determine whether DOE‘s
contract reform initiatives had resulted in improved performance using
the department‘s measures, we reviewed DOE‘s ongoing projects to assess
whether they were experiencing cost overruns or schedule delays. We
compared current ongoing DOE projects with estimated total costs
exceeding $200 million with similar information we developed in 1996 on
projects[Footnote 13] with estimated total costs exceeding $100
million.[Footnote 14]
In both 1996 and 2001, over half of the projects we reviewed had both
schedule delays and cost increases. Furthermore, as shown in table 3,
the proportion of projects experiencing cost increases of more than
double the initial cost estimates or schedule delays of 5 years or more
increased during the 6-year period. For example, the initial cost
estimate in 1998 for the spent nuclear fuels dry storage project at
Idaho Falls, Idaho, was $123.8 million with a completion date of 2001.
Currently, the cost estimate for this project is $273 million with a
completion date of 2006. Appendix II contains additional information on
DOE‘s ongoing major projects as of December 2001.
Table 3: Comparison of Cost Overruns and Schedule Delays for Ongoing
Projects in 2001 with Ongoing Projects in 1996:
Number of projects reviewed; Number of projects: 1996: 25[A]; Number
of projects: 2001: 16[B];
Projects with a cost estimate of more than double the initial cost
estimate; Number of projects: 1996: 7; Number of projects: 1996: (28%);
Number of projects: 2001: 6; Number of projects: 2001: (38%).
Projects with schedule delays of 5 years or more; Number of projects:
1996: 8; Number of projects: 1996: (32%); Number of projects: 2001: 6;
Number of projects: 2001: (38%).
[A] We evaluated 34 projects in 1996 with estimated costs greater than
$100 million. However, nine of the projects were environmental
restoration projects, and DOE‘s original and/or current cost estimates
did not estimate costs through project completion. In 1998, DOE divided
these environmental restoration projects into multiple projects at each
site. Therefore, we excluded these projects from our current analysis.
[B] There are 10 additional projects with total project costs greater
than $200 million, but those projects have either recently started or
have been suspended.
Source: GAO analysis of DOE data.
[End of table]
The projects we reviewed--with estimated costs ranging from $270
million to $8.4 billion--may not be representative of all DOE
projects.[Footnote 15] Although this comparison provides only a limited
measure of contractor performance, it does raise questions about the
overall impact of DOE‘s contract reform initiative on improving
contractor performance.
DOE‘s Approach to Contract Reform Was Not Consistent with Best
Practices:
The problems with DOE‘s ability to track the results of contract reform
reflect a broader need to develop an approach to managing its
initiatives that is more consistent with best practices. As part of our
review, we looked at best practices for managing improvement
initiatives. We found that high-performing organizations use a
systematic results-oriented management approach that includes defining
goals for the initiative and gauging progress towards those goals. They
also use information on results to continuously adjust the
implementation of the initiative and sustain improvements. DOE‘s
approach to contract reform did not incorporate these best practices,
and its emphasis on measuring progress in terms of implementation
indicated a focus primarily on contract reform itself as a goal rather
than improved performance. Furthermore, DOE faces the same fundamental
challenge--lack of a results-oriented approach--in several other
management improvement initiatives that, if successful, could enhance
its contract reform efforts.
DOE Has Not Followed Best Practices in Implementing Its Contract Reform
Initiative:
DOE‘s approach to implementing its contract reform initiatives has not
followed best management practices. In our review of authoritative
literature[Footnote 16] we found that leading organizations were able
to sustain such management improvement initiatives by using a
systematic, results-oriented approach that incorporated a rigorous
measurement of progress. Such an approach typically included the
following steps: (1) define clear goals for the initiative, (2) develop
an implementation strategy that sets milestones and establishes
responsibility, (3) establish results-oriented outcome measures to
gauge progress toward the goals, and (4) use results-oriented data to
evaluate the effectiveness of the initiative and make additional
changes where warranted. While DOE followed an implementation strategy
for its contract reform initiatives, it implemented those initiatives
largely without clearly defining goals, gauging progress toward those
goals with results-oriented measures, or using results-oriented data to
evaluate the effectiveness of its reforms.
Although DOE had set general, overarching goals for its contract reform
efforts, the department did not further define those goals. As stated
in the 1994 report of the Contract Reform Team, the overall goal of
contract reform was to make the department‘s contracting process ’—work
better and cost less.“ The secretary‘s preface to the report presented
the fundamental problem: ’DOE is not adequately in control of its
contractors. As a result, the contractors are not sufficiently
accountable to the department, and we are not in a position to ensure
prudent expenditure of taxpayer dollars in pursuit of our principle
missions.“ However, DOE did not further align those broad goals in
relation to the specific contract reform efforts. For example, the
department did not frame its contract reform initiatives to increase
competition in terms of improved contractor accountability, better
performance, or reduced costs. While increasing the number of
competitively awarded contracts is a positive development, it does not
by itself indicate that the department‘s contracting processes work
better or cost less.
DOE was effective at establishing an implementation strategy that set
milestones and assigned responsibility for carrying it out. For
example, DOE‘s February 1994 report by its contract reform team
contained 48 specific reform actions, each containing a required
action, establishing a deadline, and assigning a specific DOE office
with responsibility for developing the reform action. These reform
actions, for the most part, involved developing policies, procedures,
guidance, and plans to implement reforms such as competitive
procurements and performance incentives. Our 1996 assessment[Footnote
17] of DOE progress toward implementing those goals found that DOE had
completed 47 of 48 reform actions. Since that time, DOE has continued
to set milestones and assign responsibility for its reform initiatives.
For example, following an internal review in 1997, the department
developed another series of actions to improve its implementation of
reform initiatives pertaining to performance-based incentives. Those
actions also had milestones for completion and assigned responsibility
for carrying them out.
DOE did not establish results-oriented outcome measures for its
contract reform initiatives. Instead, as discussed earlier, DOE
generally focused on measuring the progress of implementing its reform
initiatives and reviewing individual contracts, but did not develop
ways to gauge progress towards its overarching reform goals of making
contracting work better and cost less. A shortcoming of goals defined
so generally is the lack of objective ways in which to measure progress
in meeting those goals. Translating the general goal of ’working
better“ into a more specific objective, such as having contractors
complete a greater number of their projects on time and within budget,
would have helped the department to identify ways it could measure
results and, therefore, gauge progress towards the goals of contract
reform.
Finally, DOE does not have the results-oriented data to evaluate the
effectiveness of its contract reform initiatives. Because the
department did not develop clear goals and results-oriented measures,
it does not have the results-oriented data necessary to systematically
review progress, take corrective action, and reinforce success.
Although DOE has received feedback on its reform efforts from internal
reviews such as self-assessment reports and external reports by the DOE
Inspector General, GAO, and others, these outside reviews are not a
substitute for a systematic feedback process.
Despite not following best practices for reform initiatives, DOE has
taken steps to strengthen the management and oversight of its
activities. For example, DOE has recently taken steps to integrate
contract, project, and financial management functions under a single
office--the Office of Management, Budget, and Evaluation/Chief
Financial Officer. DOE officials believe that this action will improve
the coordination, oversight, and control of these important activities.
Other Management Improvement Initiatives That Could Enhance Contract
Reform Have Similar Problems:
Although DOE‘s contract reform initiative has focused on increasing
competition and holding contractors more accountable for results, DOE
recognizes that contract reform by itself is not enough to ensure that
improved contractor performance actually occurs. DOE has begun several
other initiatives that, if successfully implemented, could enhance its
contract reform efforts. These initiatives include efforts to
strengthen its management of projects, develop and use information
systems for oversight and control, and improve the training and
expertise of the DOE staff overseeing contractor activities. We
conducted only a limited review of these initiatives and did not fully
assess DOE‘s implementation against all four steps in a ’best
practices“ approach. Nevertheless, we identified instances where, as
with the contract reform initiative, DOE‘s management of the initiative
fell short of best management practices in one or more areas. Table 4
below outlines these initiatives, how they could enhance the contract
reform efforts, and the potential management weakness that could limit
their effectiveness.
Table 4: DOE Management Improvement Initiatives That Could Enhance
Contract Reform:
Improvement initiative: Project management; Description/status: Began
in June 1999 to implement recommendations in National Research Council
review[A] on how to improve cost and schedule performance on major
projects; In 2000, DOE issued new policy, order, and guidance on
managing and controlling projects; In 2001, DOE established a
project tracking system and required monthly status reporting on all
projects with total costs over $5 million; In 2002, DOE established
a goal that 85 percent of major projects have less than a 10 percent
variance in cost and schedule; How the initiative could enhance
contract reform efforts: Provide early indications of problems with
projects‘ cost and schedule; Provide DOE managers with information
necessary to hold contractors accountable for results; Potential
problem: National Research Council follow-up report[B] found that DOE
had not clearly defined its goals for the initiative nor developed the
results-oriented outcome measures to gauge progress.
Improvement initiative: Management information systems; Description/
status: In 2001, DOE began developing a unified planning, programming,
budgeting and evaluation process to integrate budget and program
results information; System expected to be available for fiscal year
2004 budget cycle; How the initiative could enhance contract reform
efforts: Provide timely feedback on results to DOE managers; Provide
better management information for both budget requests to the Congress
and internal management use; Potential problem: DOE has not established
performance measures and indicators to evaluate progress towards
meeting
program goals.
Improvement initiative: Human capital; Description/status: In 1998, DOE
began developing a training and certification program for acquisition
management; In 2001, DOE began developing a training and certification
program for federal project management; In 2001, DOE began efforts to
address an aging workforce, the need for succession planning, and skill
gaps; DOE‘s September 2001 Five-Year Workforce Restructuring Plan
included strategies to address acquisition and project management skill
gaps; How the initiative could enhance contract reform efforts: Provide
DOE staff with technical and managerial skills necessary for effective
oversight of contractor activities; Provide DOE staff with knowledge
and
experience to assess technical and performance risks and determine
optimum contracting approach; Potential problem: DOE has not developed
performance measures to indicate whether the human capital initiatives
will result in improved performance.
[A] National Research Council, Improving Project Management in the
Department of Energy (Washington, D.C.: June 1999).
[B] National Research Council, Progress in Improving Project Management
at the Department of Energy--2001 Assessment (Washington, D.C.: Nov.
2001).
Source: GAO analysis of DOE and National Research Council data.
[End of table]
Although none of these initiatives have been fully implemented, their
effectiveness may be limited by the same lack of a results-oriented
approach to managing the initiative and sustaining improvement as does
the department‘s contract reform efforts.
Conclusions:
Poor performance by DOE contractors and inadequate DOE management and
oversight of those contractors led us to conclude in 1990 that DOE‘s
contracting practices were at high risk for fraud, waste, abuse, and
mismanagement. Subsequently, DOE began its contract reform initiative
to improve the performance and accountability of its contractors.
Although DOE has undertaken a number of reforms over the years and has
monitored its progress in implementing those reforms, it has no good
measure of the results of the reforms. Aside from individual examples
of good or poor performance on specific projects, DOE cannot tell, for
example, if the contract reforms have resulted in better performance by
its contractors or more favorable contract terms for the government.
Limited evidence we developed suggests that contractors managing DOE‘s
major projects are performing no better in 2001 than on similar
projects in 1996.
DOE faces a fundamental challenge to ensuring the effectiveness of its
contract reform initiative--developing an approach to managing the
initiative that is more consistent with the best practices of high-
performing organizations. DOE‘s practices in managing its contract
reform initiative, as well as its other initiatives such as project
management, that could also help to improve contractor performance,
fall short of the best practices followed by high-performing
organizations. Unless DOE strengthens the way in which it manages
initiatives such as contract reform, DOE may not be able to fully
realize the benefits of these initiatives and ensure that its programs
are adequately protected from fraud, waste, abuse, and mismanagement.
Recommendation for Executive Action:
To improve the effectiveness of DOE‘s contract reform initiative, as
well as other management improvement initiatives, we recommend that the
department develop an approach to implementing its initiatives that
incorporates best practices including the key elements of (1) clearly
defined goals, (2) an implementation strategy that sets milestones and
establishes responsibility, (3) results-oriented outcome measures, and
(4) a mechanism that uses results-oriented data to evaluate the
effectiveness of the department‘s initiatives and to take corrective
actions as needed.
Agency Comments:
We provided a draft of this report to the Department of Energy for its
review and comment. DOE‘s Director, Office of Management, Budget, and
Evaluation/Chief Financial Officer responded that DOE had three main
concerns about our report but agreed with our recommendation that DOE
develop an approach to its management improvement initiatives, such as
contract reform, that is more consistent with the practices of high-
performing organizations. DOE‘s first concern was that the report
characterizes contract reform as DOE‘s fundamental management challenge
but the report also discusses program and project management issues.
DOE believes this creates the misperception that the procurement system
can be used to address the myriad of issues facing the department. We
believe that our report fairly and accurately describes the context of
contract management in DOE. Our report identifies contract management
as a major management challenge for DOE, and one that we have reported
on for over 10 years. The report does not suggest that contract
management is DOE‘s primary or most fundamental management challenge.
In fact, we have issued other reports such as our December 2001 report
on DOE‘s major mission, structure, and accountability problems[Footnote
18] that discuss more fundamental management issues. However, within
the context of those more fundamental management challenges, DOE can
and should strive to effectively manage its contracts. Our report does
not imply that effective contract management will solve the other
problems facing the department. In fact, the report discusses
initiatives other than contract reform that are under way at DOE,
including the project management initiative, because those initiatives
could also have an impact on the results of the contract reform
initiative.
DOE‘s second concern was that our report concluded that its contract
reform initiative was not managed in a systematic manner. DOE said that
its 1994 contract reform initiative was managed systematically and
included top management oversight, a matrixed implementation team,
clearly defined goals and objectives, an implementation strategy, and
identified outcomes. DOE also said it used internal assessments of the
effectiveness of specific reform initiatives. Our analysis involved
comparing DOE‘s approach to contract reform with the best practices for
managing improvement initiatives followed by high-performing
organizations. That comparison showed that DOE‘s approach to contract
reform, and to several other management improvement initiatives, was
not consistent with those best practices, particularly in the areas of
defining measurable goals, establishing results-oriented outcome
measures, or developing results-oriented data with which to measure the
effectiveness of the initiatives. We revised our report to clarify this
point. DOE also questioned how we could criticize its approach to
contract reform when we had recommended in earlier reports that it
pursue contract reform. Our report does not question the need for
contract reform in DOE or the components of DOE‘s reform initiative,
such as increasing competition and the use of performance-based
contracts. Rather, our report assesses what progress DOE has made in
implementing the initiatives, whether the initiatives have resulted in
improved contractor performance, and any challenges DOE faces in
ensuring that its contract reform initiatives are effective.
DOE‘s third concern was that the report identifies a limited number of
projects to support a conclusion that DOE‘s contract management system
is in trouble. DOE believes the problems are more likely due to program
and project management issues and the risks generally associated with
unique, technically complex projects and DOE‘s funding and political
environment. We believe that our report fairly characterizes DOE‘s
contract management system. Our report clearly states that DOE has
developed little objective information to demonstrate whether its
contract reforms have improved contractor performance. We pointed out
that anecdotal examples can be used to illustrate both improved
contractor performance and continued poor contractor performance. And
we identify other evidence to suggest that contractor performance may
not have improved. We also acknowledged that other factors, such as
DOE‘s approach to managing projects, could also affect the outcome of
DOE‘s contract reform efforts.
Regarding our recommendation that DOE develop an approach to
implementing its management improvement initiatives that includes the
key elements found in the best practices of high-performing
organizations, DOE agreed with the recommendation and said that it
would incorporate our observations and recommendation into its future
improvement efforts.
DOE also provided technical corrections, which we incorporated as
appropriate. DOE‘s written comments on our draft report are included in
appendix III.
We conducted our review from October 2001 through August 2002, in
accordance with generally accepted government auditing standards.
Appendix IV provides details on our scope and methodology.
This report contains a recommendation to you. As you know, 31 U.S.C.
720 requires the head of a federal agency to submit a written statement
of the actions taken on our recommendations to the Senate Committee on
Governmental Affairs and to the House Committee on Government Reform
not later than 60 days from the date of this letter and to the House
and Senate Committees on Appropriations with the agency‘s first request
for appropriations made more than 60 days after the date of this
letter.
Copies of this report are available on request. In addition, the report
will be available at no charge on the GAO Web site at http://
www.gao.gov. If you or your staff has any questions on this report,
please call me at (202) 512-3841. Key contributors to this report are
listed in appendix V.
Sincerely yours,
(Ms.) Gary L. Jones
Director, Natural Resources and Environment:
[Signed by Gary L. Jones]
[End of section]
Appendix I: DOE‘s Decisions to either Compete or Extend Non-
Competitively Its Major Site Contracts:
[End of section]
Major site or facility: Ames Laboratory; Contractor in 2001: Iowa State
University; Year of decision to compete or extend: 1999; Contract
action[A]: Extended; Budget estimate for fiscal year 2001: $24.8.
Major site or facility: Argonne National Laboratory; Contractor in
2001: University of Chicago; Year of decision to compete or extend:
1999; Contract action[A]: Extended; Budget estimate for fiscal year
2001: 498.3.
Major site or facility: Bettis Laboratory; Contractor in 2001: Bechtel
Bettis Inc; Year of decision to compete or extend: 1998; Contract
action[A]: Competed; Budget estimate for fiscal year 2001: 342.0.
Major site or facility: Brookhaven National Laboratory; Contractor in
2001: Brookhaven Science Associates; Year of decision to compete or
extend: 1997; Contract action[A]: Competed; Budget estimate for fiscal
year 2001: 453.2.
Major site or facility: Fermi National Accelerator Laboratory;
Contractor in 2001: Universities Research Association; Year of decision
to compete or extend: 2000; Contract action[A]: Extended; Budget
estimate for fiscal year 2001: 308.0.
Major site or facility: Fernald Environmental Management Project;
Contractor in 2001: Fluor Fernald, Inc; Year of decision to compete or
extend: 2000; Contract action[A]: Competed; Budget estimate for fiscal
year 2001: 290.0.
Major site or facility: Hanford Environmental Restoration; Contractor
in 2001: Bechtel Hanford Inc; Year of decision to compete or extend:
1998; Contract action[A]: Extended[B]; Budget estimate for fiscal year
2001: 177.0.
Major site or facility: Hanford Site; Contractor in 2001: Fluor Hanford
Inc; Year of decision to compete or extend: 2000; Contract action[A]:
Extended[B]; Budget estimate for fiscal year 2001: 665.6.
Major site or facility: Idaho National Engineering and Environmental
Laboratory; Contractor in 2001: Bechtel BWXT Idaho, LLC; Year of
decision to compete or extend: 1999; Contract action[A]: Competed;
Budget estimate for fiscal year 2001: 683.0.
Major site or facility: Jefferson Lab; Contractor in 2001: Southeastern
Universities Research Association; Year of decision to compete or
extend: 1999; Contract action[A]: Extended; Budget estimate for fiscal
year 2001: 76.0.
Major site or facility: Kansas City Plant; Contractor in 2001:
Honeywell Federal Manufacturing and Technologies; Year of decision to
compete or extend: 2000; Contract action[A]: Competed; Budget estimate
for fiscal year 2001: 383.5.
Major site or facility: Knolls Atomic Power Laboratory; Contractor in
2001: KAPL, Inc.; Year of decision to compete or extend: 2000; Contract
action[A]: Competed; Budget estimate for fiscal year 2001: 269.0.
Major site or facility: Lawrence Berkeley National Laboratory;
Contractor in 2001: University of California; Year of decision to
compete or extend: 1997; Contract action[A]: Extended; Budget estimate
for fiscal year 2001: 320.0.
Major site or facility: Lawrence Livermore National Laboratory;
Contractor in 2001: University of California; Year of decision to
compete or extend: 2000; Contract action[A]: Extended; Budget estimate
for fiscal year 2001: 1,389.1.
Major site or facility: Los Alamos National Laboratory; Contractor in
2001: University of California; Year of decision to compete or extend:
2000; Contract action[A]: Extended; Budget estimate for fiscal year
2001: 2,000.0.
Major site or facility: Mound; Contractor in 2001: BWXT of Ohio; Year
of decision to compete or extend: 1997; Contract action[A]: Competed;
Budget estimate for fiscal year 2001: 98.9.
Major site or facility: National Renewable Energy Laboratory;
Contractor in 2001: Midwest Research Institute; Year of decision to
compete or extend: 1998; Contract action[A]: Competed; Budget estimate
for fiscal year 2001: 218.1.
Major site or facility: Nevada Test Site; Contractor in 2001: Bechtel
Nevada Corp.; Year of decision to compete or extend: 2000; Contract
action[A]: Extended[B]; Budget estimate for fiscal year 2001: 320.0.
Major site or facility: Oak Ridge Environmental Management; Contractor
in 2001: Bechtel Jacobs Company, LLC; Year of decision to compete or
extend: 1997; Contract action[A]: Competed; Budget estimate for fiscal
year 2001: 546.5.
Major site or facility: Oak Ridge National Laboratory; Contractor in
2001: UT-Battelle, LLC; Year of decision to compete or extend: 1999;
Contract action[A]: Competed; Budget estimate for fiscal year 2001:
769.4.
Major site or facility: Pacific Northwest National Laboratory;
Contractor in 2001: Battelle Memorial Institute; Year of decision to
compete or extend: 1997; Contract action[A]: Extended; Budget estimate
for fiscal year 2001: 457.0.
Major site or facility: Pantex Plant; Contractor in 2001: BWXT Pantex,
LLC; Year of decision to compete or extend: 2000; Contract action[A]:
Competed; Budget estimate for fiscal year 2001: 354.7.
Major site or facility: Princeton Plasma Physics Laboratory; Contractor
in 2001: Princeton University; Year of decision to compete or extend:
2001; Contract action[A]: Extended; Budget estimate for fiscal year
2001: 74.1.
Major site or facility: River Protection Project Tank Farm Management;
Contractor in 2001: CH2M Hill Hanford Group; Year of decision to
compete or extend: 2001; Contract action[A]: Extended[B]; Budget
estimate for fiscal year 2001: 402.7.
Major site or facility: Rocky Flats Environmental Technology Site;
Contractor in 2001: Kaiser-Hill Co. LLC; Year of decision to compete or
extend: 2000; Contract action[A]: Extended[C]; Budget estimate for
fiscal year 2001: 661.0.
Major site or facility: Sandia National Laboratories; Contractor in
2001: Sandia Corporation; Year of decision to compete or extend: 1998;
Contract action[A]: Extended[B]; Budget estimate for fiscal year 2001:
1,596.5.
Major site or facility: Savannah River Site; Contractor in 2001:
Westinghouse Savannah
River Co.; Year of decision to compete or extend: 2001; Contract
action[A]: Extended[B]; Budget estimate for fiscal year 2001: 1,431.0.
Major site or facility: Stanford Linear Accelerator Center; Contractor
in 2001: Stanford University; Year of decision to compete or extend:
1998; Contract action[A]: Extended; Budget estimate for fiscal year
2001: 190.0.
Major site or facility: Strategic Petroleum Reserve; Contractor in
2001: Dyn McDermott Petroleum Operations Company; Year of decision to
compete or extend: 1998; Contract action[A]: Extended[B]; Budget
estimate for fiscal year 2001: 101.8.
Major site or facility: Waste Isolation Pilot Plant; Contractor in
2001: Westinghouse TRU Solutions LLC; Year of decision to compete or
extend: 2000; Contract action[A]: Competed; Budget estimate for fiscal
year 2001: 101.3.
Major site or facility: West Valley Demonstration Project; Contractor
in 2001: West Valley Nuclear Services; Year of decision to compete or
extend: 1998/2002; Contract action[A]: Extended; Budget estimate for
fiscal year 2001: 107.4.
Major site or facility: Y-12 National Security Complex; Contractor in
2001: BWXT Y-12, LLC; Year of decision to compete or extend: 2000;
Contract action[A]: Competed; Budget estimate for fiscal year 2001:
567.4.
Major site or facility: Yucca Mountain Site; Contractor in 2001:
Bechtel SAIC; Year of decision to compete or extend: 2000; Contract
action[A]: Competed; Budget estimate for fiscal year 2001: 294.5.
[A] To be classified as a competitively awarded contract, DOE must
have issued a request for proposals and a public announcement inviting
proposals.
[B] DOE competitively awarded the first 5-year contract and
subsequently exercised its option to extend the contracts for up to 5
more years, in accordance with its regulations.
[C] The 1995 contract for the Rocky Flats site did not include an
option for an extension. When the closure of the site was accelerated
to 2006, DOE decided to extend the contract and renegotiate it as a
closure contract.
Source: GAO presentation of DOE data.
[End of table]
[End of section]
Appendix II: Cost and Schedule Performance on DOE‘s Major Projects as
of
December 2001:
The following table shows the original and current cost estimates and
completion dates for ongoing DOE projects with estimated costs greater
than $200 million. The table does not include 10 additional DOE
projects with estimated costs greater than $200 million because the
projects were suspended or only recently started as of December 2001.
Table 5: Original and Current Cost Estimates and Schedule for DOE
Projects with Estimated Costs Greater than $200 Million as of December
2001:
Dollars in millions.
Advanced Mixed Waste Treatment Project (97-PVT-2)[C]; Cost: Original
cost estimate[B]: $1,078.9; Cost: Current cost estimate: $1,087.7[D];
Cost: [Empty]; Schedule: Original completion date: December 2002;
Schedule: Current completion date: December 2002.
Civilian Radioactive Waste Management Program[E]; Cost: Original cost
estimate[B]: 6,300.0[F]; Cost: Current cost estimate: 8,394.6; Cost:
[Empty]; Schedule: Original completion date: October 2001[F]; Schedule:
Current completion date: December 2004.
Dual-Axis Radiographic Hydrodynamic Test Facility (97-D-102)[G]; Cost:
Original cost estimate[B]: 30.0[H]; Cost: Current cost estimate: 269.7;
Cost: [Empty]; Schedule: Original completion date: September 1990;
Schedule: Current completion date: December 2002.
East Tennessee Technology Park Three-Building Decontamination and
Decommissioning and Recycle Project (OR-493); Cost: Original cost
estimate[B]: 283.9; Cost: Current cost estimate: 348.1; Cost: [Empty];
Schedule: Original completion date: December 2003; Schedule: Current
completion date: March 2004.
Facilities Capability Assurance Program (88-D-122)[I]; Cost: Original
cost estimate[B]: N/A[J]; Cost: Current cost estimate: 445.6; Cost:
[Empty]; Schedule: Original completion date: N/A[J]; Schedule: Current
completion date: June 2000.
Hanford Tank Waste Treatment and Immobilization Plant (01-D-416); Cost:
Original cost estimate[B]: 12,488.0[K]; Cost: Current cost estimate:
4,350.0; Cost: [Empty]; Schedule: Original completion date: 2007;
Schedule: Current completion date: 2007.
High-Level Waste Removal from Filled Waste Tanks (93-D-187)[L]; Cost:
Original cost estimate[B]: 88.6[M]; Cost: Current cost estimate:
1,550.5; Cost: [Empty]; Schedule: Original completion date: September
1999[M]; Schedule: Current completion date: September 2028.
Initial Tank Retrieval Systems (94-D-407); Cost: Original cost
estimate[B]: 245.0[N]; Cost: Current cost estimate: 274.9; Cost:
[Empty]; Schedule: Original completion date: March 2000[N]; Schedule:
Current completion date: December 2015.
National Ignition Facility (96-D-111); Cost: Original cost estimate[B]:
1,073.6; Cost: Current cost estimate: 2,248.1; Cost: [Empty]; Schedule:
Original completion date: June 2002; Schedule: Current completion date:
September 2008.
Silos; Cost: Original cost estimate[B]: N/A; Cost: Current cost
estimate: 338.1; Cost: [Empty]; Schedule: Original completion date: N/
A; Schedule: Current completion date: December 2006.
Spallation Neutron Source (99-E-334); Cost: Original cost estimate[B]:
1,332.8; Cost: Current cost estimate: 1,411.7; Cost: [Empty]; Schedule:
Original completion date: September 2005; Schedule: Current completion
date: June 2006.
Spent Nuclear Fuel Dry Storage (98-PVT-2)[O]; Cost: Original cost
estimate[B]: 123.8; Cost: Current cost estimate: 273.0; Cost: [Empty];
Schedule: Original completion date: June 2001; Schedule: Current
completion date: December 2005.
Hanford Spent Nuclear Fuels; Cost: Original cost estimate[B]: 714.8;
Cost: Current cost estimate: 1,600.0; Cost: [Empty]; Schedule: Original
completion date: 2001; Schedule: Current completion date: September
2006.
Tank Farm Restoration and Safe Operations (97-D-402); Cost: Original
cost estimate[B]: 289.2; Cost: Current cost estimate: 285.3; Cost:
[Empty]; Schedule: Original completion date: June 2005; Schedule:
Current completion date: June 2005.
Tritium Extraction Facility (98-D-125)[P]; Cost: Original cost
estimate[B]: 390.7; Cost: Current cost estimate: 401.0; Cost: [Empty];
Schedule: Original completion date: June 2005; Schedule: Current
completion date: March 2006.
Weldon Springs Site Remedial Action Project; Cost: Original cost
estimate[B]: 357.7[Q]; Cost: Current cost estimate: 905.2; Cost:
[Empty]; Schedule: Original completion date: September 1995[Q];
Schedule: Current completion date: September 2002.
[A] Projects that are not funded as construction line items do not have
project numbers. All costs, unless otherwise specified, are ’total
project costs.“ The cost data were obtained from DOE Congressional
budget requests and other DOE-provided data. The term N/A means cost or
schedule not available or not yet developed.
[B] For consistency we used, when available, preliminary budget
estimates submitted to Congress as the basis for original cost
estimates.
[C] Total project cost for construction projects typically includes
only the design, construction, and startup costs that precede
production operations. Total project cost for this project also
includes estimated costs for over 10 years of production operations and
other associated costs. The current completion date refers to
completion of the construction phase.
[D] The contractor has submitted a ’Request for Equitable Adjustment“
of over $48 million due to a six-month schedule slip the project
experienced as a result of a delay in the issuance of environmental
permits. Because the Request for Equitable Adjustment is still under
review, the $48 million is not included in the current cost estimate.
[E] The original baseline for this program included construction of the
exploratory studies facility and, if suitable, a site recommendation
and a license application. The current scope of the program was
broadened in 1997 to include all elements of the Civilian Radioactive
Waste Management Program, which now includes development of license
application, design and construction of Yucca Mountain Repository,
licensing interactions with the Nuclear Regulatory Commission, and
development of a transportation system. The current completion date is
only for the license application.
[F] We reported in 1996 that the current cost and completion date for
the Yucca Mountain Site Characterization Project were $4,300 million
and March 2002, respectively. In 1997, DOE expanded the project to
include the entire Civilian Radioactive Waste Management Program.
[G] The original scope of this project at initial authorization in 1988
included two buildings and two single pulse flash x-ray machines. The
project has since undergone several changes in scope, which now
includes three buildings, a containment vessel to reduce emissions to
the environment, a single pulse machine, and a multiphase machine.
[H] This amount is a total estimated cost from the fiscal year 1988
Budget Request, which does not include other project costs. Other
project costs include supporting research and development and plant
support costs during construction, activation, and startup. There was
no requirement for a total project cost estimate in 1988.
[I] This project has a few subprojects completing closeout activities
and two still underway. DOE anticipates additional funding needs and a
schedule extension to complete the final two subprojects.
[J] We reported in 1996 that the current cost for the Facilities
Capability Assurance Program was $447 million and the completion date
was not available. No cost estimate was available when the project was
originally proposed.
[K] This original cost estimate from the fiscal year 2001 Budget
Request was based upon the privatization concept and included plant
operations through fiscal year 2018.
[L] DOE expanded the original scope of this project in fiscal year 1994
to incorporate three ongoing projects, which increased the total
project cost from $88.6 million to $828 million and the project
completion date from 1999 to 2008 in the fiscal year 1996 budget. The
cost and schedule were revised again in fiscal year 2000 to include,
among other projects, the equipment and infrastructure required to
remove the high level waste inventory from nine additional tanks.
[M] We reported in 1996 that the current cost and completion date for
the High Level Waste Removal project were $828.2 million and September
2008, respectively. DOE expanded the scope of this project in 1994.
[N] We reported in 1996 that the current cost and completion date for
the Initial Tank Retrieval System project were $358.2 million and March
2010, respectively.
[End of table]
[O] The original and current estimated costs include design,
construction, startup, and operating costs. The current completion date
refers to completion of the construction and startup phase.
[PI] n June 2002 DOE‘s Office of Inspector General reported that the
total project cost for the Tritium Extraction Facility could increase
to as much as $500 million and that the facility may not be completed
until December 2006.
[Q] We reported in 1996 that the current cost and completion date for
the Weldon Springs Remedial Action Project were $865.0 million and
2001, respectively.
[End of section]
Source: GAO analysis of DOE and National Research Council data.
[End of section]
Appendix III: Comments from the Department of Energy:
Department of Energy Washington, DC 20585:
AUG 27 2002:
Ms. Gary L. Jones:
Director, Natural Resources and Environment, U.S. General Accounting
Office:
441 G Street, NW Room 2964 Washington, DC 20548:
Dear Ms. Jones:
We have reviewed the draft General Accounting Office (GAO) Report
entitled ’Contract Reform: DOE Has Made Progress, but Actions Needed to
Ensure Initiatives Have Improved Results“ (GAO-02-798). Specific
technical comments to the draft report are attached. We appreciate the
opportunity to discuss these matters with you and are pleased that the
report recognizes the Department of Energy‘s (DOE) positive
achievements in a number of areas related to contract and project
management, particularly in GAO‘s recognition of DOE‘s progress in
implementing its 1994 Contract Reform initiative. That early initiative
was an important first step in what is an important and continuing area
of management attention in DOE. Your past advice has assisted us in
defining parallel, as well as, subsequent improvement initiatives. The
report also identifies DOE‘s progress in such areas as: use of
competition for its major site and facility management contracts; the
use of performance-based contracting approaches, including new contract
types that shift more performance and financial risk to the contractor;
and improvements in project management.
We continue to have a basic concern with GAO‘s formally stated opinion
that contract reform is DOE‘s fundamental management challenge. We note
that the original audit focus was contract reform, but this focus was
broadened to include program and project management. Having broadened
its scope of inquiry, however, the GAO‘s report title and overall
summation remains the same. That is, the report continues to underscore
the misperception that the terms and conditions of the contract, and
procurement regulations and guidance, can fully address the myriad of
issues facing DOE.
We also disagree with the report‘s apparent conclusion that the 1994
Contract Reform initiative was not managed in a systematic manner. To
the contrary, the initiative had the clear support of the Secretary, a
top-level board of directors, a project manager, a matrixed
implementation team, clearly defined goals and objectives, an
implementation strategy and schedule, identified roles and
responsibilities, identified outcomes and tracking mechanisms.
Furthermore, DOE used a number of mechanisms, most notably internal
assessments of the effectiveness of specific contract reform
initiatives, to determine whether the reforms were accomplished and to
identify their impact. Where problems were identified, corrective
measures have been taken; where changes did not result in anticipated
outcomes, adjustments have been made. Parallel and follow-on reform
initiatives not discussed in this report have been integrated into the
Department‘s strategic planning and performance measurement systems.
Although GAO has conducted numerous past assessments of the 1994
contract reform effort and was very familiar with the Department‘s
methodology, it has not critiqued DOE‘s management approach.
Furthermore, we note that GAO questions certain of the original
recommendations/objectives of contract reform which were in direct
response to earlier criticisms and recommendations made by the GAO and
Congressional Committees.
Regarding the issues related to performance-based contracting,
measures, and incentives, we agree that the concept of performance-
based contracts presents a difficult challenge and that progress is
evolutionary. In this regard, a recent study by outside consultants
compared DOE facility management/remediation contracts to similar
contracts awarded by other federal agencies. The study found that DOE
major site and facility contracts, when compared to other agencies‘
contracts, generally evidenced significantly higher risk factors (for
the contractor) and technical complexity and somewhat lower available
fees. The study team also found that the other-agency contracts, while
characterized as performance-based, relied on largely subjective
performance objectives and measures not unlike the cost-plus-award fee
contracts used by DOE prior to 1994. On the other hand, DOE has
aggressively pursued the use of performance-based contracting
approaches and is one of the few Federal agencies where performance
management has been reflected in its contracts.
We are also concerned that the report anecdotally identifies a limited
number of troubled projects to support its contention that DOE‘s
contract management system is in trouble. A review of the projects at
issue would indicate the difficulties have less to do with contract
management issues than with programmatic and project management issues
and the significant risks generally associated with one-of-a-kind
projects involving advanced technologies with extensive complexity,
uncertain funding, and difficult political environments.
Notwithstanding the foregoing, we agree with the report‘s conceptional
recommendation that DOE adopt systematic approaches to developing,
implementing and managing various management initiatives. Finally, we
note that the GAO report presents a historical snapshot of the
Department‘s implementation of performance-based contracts. The
Department continues to strive toward putting effective and efficient
contracts in place that will result in achieving or exceeding the
Department‘s objectives at its various sites. Commencing in 2001, the
Department put in place numerous contracts with either multi-year
incentives or which were awarded on a cost-plus-incentive-fee
completion basis. These contracts not only provide incentives to the
contractor to achieve the Department‘s objectives, but if possible,
exceed them.
Even with the placement of these contracts, the Department continues to
review its approach to performance-based contracting as evidenced by
the numerous internal reviews/reports examining the Department‘s
implementation of these contracts. The Department continues to seek
ways to expand the implementation of what works and correct what does
not.
However, this is a learning process, and requires time to fully
implement. We appreciate the GAO‘s observations and recommendations and
will incorporate them into our future improvement efforts.
Bruce M. Carnes, Director, Office of Management, Budget and Evaluation/
Chief
Financial Officer:
[Signed by Bruce M. Carnes]
Attachment:
[End of section]
Appendix IV: Scope and Methodology:
To assess the progress that DOE has made since 1996 in implementing
contract reform initiatives in the key areas of developing alternative
contracting approaches, increasing competition, and using performance-
based contracts, we reviewed DOE‘s three self-assessment reports on
contract reform efforts and GAO and DOE Office of Inspector General
reports on DOE contract and project management since 1996. We also
interviewed officials from DOE‘s Offices of Contract Management and
Procurement and Assistance Policy, and procurement officials with the
National Nuclear Security Administration. The National Nuclear Security
Administration, a semi-autonomous agency within DOE, has its own
procurement organization. However, since both entities follow the same
policies, regulations, and guidance, we have not made a distinction in
this report between contracts and projects of the two organizations. To
assess the extent to which DOE had incorporated the key contract
reforms into its major facility contracts, we obtained information on
33 contracts that DOE‘s headquarters procurement office identified as
site or facility management contracts. We reviewed the contract award
history of these major facility contracts, to determine which contracts
had been competed as of 1996 and as of 2001. To qualify as a
competitively awarded contract, DOE must have issued a request for
proposals and a public announcement inviting proposals. We also
obtained data on annual budgets and fees available and earned for these
same contractors for fiscal years 1996 through 2001. We did not attempt
to validate this information provided by DOE. In addition, we reviewed
documentation for major facility contracts obtained from DOE‘s
Albuquerque Operations Office, Richland Operations Office, and the
Office of River Protection.
To determine the extent to which these initiatives have resulted in
improved contractor performance, we interviewed DOE officials from the
Office of Contract Management and the three largest program offices--
Environmental Management, Defense Programs, and Science. In addition,
we interviewed procurement and program office officials at DOE‘s
Albuquerque Operations Office, Richland Operations Office, and the
Office of River Protection. We reviewed documents they provided,
including the procurement organization‘s balanced scorecard. In
addition, we reviewed DOE‘s February 2002 review of the Environmental
Management program, and numerous GAO and Inspector General reports.
Because DOE did not have objective results-oriented measures of
contractor performance, as a potential indicator of that performance,
we developed information as of December 2001 on the cost and schedule
performance of DOE‘s ongoing projects and compared that information
with similar information we developed in 1996 on DOE major system
acquisitions. In 1996, DOE categorized a ’major system acquisition“ as
a project with a total project cost greater than $100 million. When we
began our review in January 2002, we learned that DOE had since raised
the threshold of ’major project“ to $400 million. Since our compilation
of DOE reported data revealed only 19 ongoing projects that meet the
current $400 million threshold (nine of which had recently started or
were on hold), we expanded our scope to projects with total project
costs greater than $200 million, in order to compare results on a
similar number of projects. Those projects were under the management
and oversight of DOE‘s site contractors or under privatization projects
under DOE‘s oversight. There may be other projects with total project
costs greater than $200 million, but they were not identified by DOE
during our review. Because DOE does not maintain centralized data on
its projects, we obtained information from project management offices
within DOE and its National Nuclear Security Administration. We did not
verify the data obtained from DOE, but we did examine the
reasonableness of these data based on information in prior GAO reports
and audits. For consistency, we used, when available, preliminary
budget estimates submitted to the Congress as the basis for original
cost estimates and completion dates, comparing those to current cost
estimates and completions dates as of December 2001. For this report,
we used, wherever possible, the projects‘ ’total project cost,“ which
includes construction and operating funds. Where these costs are not
available, we used the ’total estimated cost,“ which includes
construction costs. We have footnoted the latter. (See appendix II.):
To identify the challenges, if any, that DOE faces in ensuring the
effectiveness of its contract reform initiatives, we reviewed the
reports of the National Research Council on improving DOE project
management. In addition, we reviewed reports and other documentation
from the National Academy of Public Administration, the Project
Management Institute, and prior GAO work to develop best practices
criteria for managing improvement initiatives. We compared DOE‘s
implementation of its contract reform initiative to these best
practices criteria to determine areas of concern. To identify the other
management improvement initiatives that could impact contract reform,
we reviewed the reports of the National Research Council, GAO and
Inspector General; the President‘s Management Agenda for fiscal year
2002; and DOE‘s 5-year workforce restructuring plans. We also
interviewed DOE officials in the Office of Engineering and Construction
Management and the Office of Program Analysis and Evaluation.
We conducted our review from October 2001 through August 2002 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix V: GAO Staff and Acknowledgments:
GAO Contacts:
William Swick (206) 287-4800:
Staff Acknowledgments:
In addition to those named above, Carole Blackwell, Robert Crystal,
Doreen Feldman, Molly Laster, Patricia Rennie, Carol Shulman, Stan
Stenersen, and Arvin Wu made key contributions to this report.
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site (www.gao.gov) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
U.S. General Accounting Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
FOOTNOTES
[1] U.S. General Accounting Office, Department of Energy: Contract
Reform Is Progressing, but Full Implementation Will Take Years,
GAO-RCED-97-18 (Washington, D.C.: Dec. 10, 1996).
[2] U.S. Department of Energy, Bechtel Jacobs Company LLC‘s Management
and Integration Contract at Oak Ridge, DOE/IG-0498 (Washington, D.C.:
Mar. 21, 2001).
[3] U.S. Department of Energy, The Department‘s Management and
Operating Contactor Make-or-Buy Program, DOE/IG-0460 (Washington,
D.C.: Feb. 17, 2000).
[4] The Pit 9 project involved an effort to clean up 250,000 cubic feet
of buried radioactive and hazardous wastes. The waste treatment
operations were to start in August 1996 and be completed by February
1999. At the time the contract was terminated because of problems with
contractor performance, the project was at least 26 months behind
schedule, had been assessed $940,000 by state regulators for failing to
meet deadlines, and the contractor had requested an additional $257
million over the contract price.
[5] U.S. General Accounting Office, Department of Energy: Alternative
Financing and Contracting Strategies for Cleanup Projects,
GAO-RCED-98-169 (Washington, D.C.:
May 29, 1998).
[6] Under the Competition in Contracting Act of 1984, an agency may
award new contracts or extend existing ones with educational or
nonprofit institutions or federally funded research and development
centers non-competitively in order to maintain an essential research
and development capability. See 41 U.S.C. 253 (c)(3).
[7] For example, see U.S. General Accounting Office, Department of
Energy: Key Factors Underlying Security Problems at DOE Facilities,
GAO/T-RCED-99-159 (Washington, D.C.: Apr. 20, 1999); U.S. General
Accounting Office, Nuclear Security: Improvements Needed in DOE‘s
Safeguards and Security Oversight, GAO/RCED-00-62 (Washington, D.C.:
Feb. 24, 2000); and A Special Investigative Panel, President‘s Foreign
Intelligence Advisory Board, Science at its Best, Security at its
Worst: A Report on Security Problems of the U.S. Department of Energy
(Washington, D.C.: June 1999).
[8] The contract fee, or profit, is the amount DOE pays to the
contractor over the allowable costs.
[9] U.S. Department of Energy, Follow-up Assessment of the
Effectiveness of Actions Taken to Improve Performance-Based Incentives
in Performance-Based Management and Management and Integration
Contracts (Washington, D.C.: Mar. 31, 1999).
[10] U.S. General Accounting Office, National Ignition Facility:
Management and Oversight Failures Caused Major Cost Overruns and
Schedule Delays, GAO/RCED-00-271 (Washington, D.C.: Aug. 8, 2000).
[11] U.S. Department of Energy, Remediation and Closure of the
Miamisburg Environmental Management Project, DOE/IG-0501 (Washington,
D.C.: May 2, 2001).
[12] U.S. Department of Energy, A Review of the Environmental
Management Program, (Washington, D.C.: Feb. 4, 2002).
[13] U.S. General Accounting Office, Department of Energy: Opportunity
to Improve Management of Major System Acquisitions, GAO/RCED-97-17
(Washington, D.C.: Nov. 26, 1996).
[14] We selected projects with estimated total costs exceeding $200
million in 2001 to ensure a roughly equivalent sample size of projects
compared to the number of projects we reviewed in 1996, while limiting
the sample size to a manageable number of projects.
[15] As of January 2002, DOE records indicated at least 42 ongoing
projects with estimated costs greater than $100 million. We did not
review all of DOE‘s capital projects with costs over $100 million
because of the level of effort that would have been required, since DOE
does not maintain centralized information on those projects.
Furthermore, five of the ongoing projects we reviewed in 2001 began
before the advent of DOE‘s contract reform initiatives.
[16] This literature included publications from the National Academy of
Public Administration, the Project Management Institute, the Six Sigma
model for process improvement, and past GAO studies on implementation
of the Government Performance Results Act and other management
initiatives.
[17] Department of Energy, Contract Reform Is Progressing, but Full
Implementation Will Take Years, RCED-97-18 (Washington, D.C.: Dec. 10,
1996).
[18] U.S. General Accounting Office, Department of Energy: Fundamental
Reassessment Needed to Address Major Mission, Structure, and
Accountability Problems, GAO-02-51 (Washington, D.C.: Dec. 21, 2001).
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
