Defense Contracting
Questions for the Record Gao ID: GAO-07-217R December 21, 2006On September 7, 2006, GAO testified before the Subcommittee on Defense, Committee on Appropriations, on recent trends in Department of Defense (DOD) contracting. Specifically, GAO testified about practices that undermine DOD's ability to establish sound business arrangements, particularly those involving the selection and oversight of DOD's contractors and their performance. This correspondence responds to a number of post-hearing questions relating to various issues, including measures that DOD can employ to ensure better contracting outcomes.
GAO-07-217R, Defense Contracting: Questions for the Record
This is the accessible text file for GAO report number GAO-07-217R
entitled 'Defense Contracting--Questions for the Record' which was
released on December 22, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
December 21, 2006:
The Honorable C. W. Bill Young:
Chairman:
The Honorable John P. Murtha:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
Subject: Defense Contracting--Questions for the Record:
On September 7, 2006, I testified before the Subcommittee on recent
trends in Department of Defense (DOD) contracting. Specifically, I
testified about practices that undermine DOD's ability to establish
sound business arrangements, particularly those involving the selection
and oversight of DOD's contractors and their performance. The need for
prudence with taxpayer funds and long-range fiscal challenges demand
that DOD ensure that its funds are spent wisely, and that it is buying
the right things, the right way.
The Subcommittee requested that I respond to a number of post-hearing
questions relating to various issues, including measures that DOD can
employ to ensure better contracting outcomes. The specific questions
and my responses are attached in appendix I. The responses are based on
previously issued GAO products on DOD acquisitions, all of which were
conducted in accordance with generally accepted government auditing
standards. Because the responses are based on prior work, we did not
obtain comments from DOD.
We will make copies of this letter available to others upon request,
and it will be available at no charge on the GAO Web site at
http://www.gao.gov. If you have any questions about this letter or need
additional information please contact Katherine V. Schinasi at (202)
512-4841 or schinasik@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this letter. Key contributors to this letter were Carol Dawn
Petersen, Chris Kunitz, and Bill Woods.
Sincerely,
Signed by:
David M. Walker:
Comptroller General of the United States:
Attachment:
Appendix I: GAO Responses to Questions for the Record:
Award and Incentive Fees:
1.Could you address how these award and incentive fees work, in
practical terms?
When we talk about fees, we are really talking about a government
expense and a form of profit for contractors. Depending on how a
contract is structured, award and incentive fees can account for all or
a portion of a contractor's potential fee or profit. Unlike other
contracts, award-and incentive-fee contracts allow an agency to adjust
the amount of fee paid to contractors based on the contractors'
performance. In practical terms, an award-fee contract sets up a system
where the government periodically assesses the contractor's performance
in areas such as quality, timeliness, technical ingenuity, and cost-
effective management. These criteria are generally subjective and can
include inputs as well as outcomes. Incentive fees, however, are more
objective and outcome-based. The contract identifies specific cost or
delivery targets, and/or performance goals, and the amount of fee
earned by the contractor is to be directly related to achievement of
these goals.
2.Do you believe they are an effective way to create incentives with
contractors? In other words, do they improve contractor performance?
Award and incentive fees can be an effective tool if properly designed
and effectively administered. However, for these fees to be effective,
DOD must address the underlying problems in its acquisition system and
more directly link the fees to the outcomes it wants. Because the
weapon system programs that result from this system are in many cases
unexecutable and/or subject to changing "requirements" or funding
levels, DOD has been unwilling to hold its programs or its contractors
accountable for achieving the very acquisition outcomes it has
identified. As a result, fees are paid even when outcomes do not meet
expectations. Addressing these broader acquisition issues and
strengthening the link between fees and acquisition outcomes can
increase the accountability of DOD programs for fees paid, of
contractors for results achieved, and the likelihood that these fees
will motivate the contractors and be an effective tool for the
government.
3.Do you believe we should discontinue the use of award and incentive
fees?
No, we do not believe they should be discontinued. Award and incentive
fees can be useful if they are used in the appropriate setting. Each
contract type has a use based on the level of risk involved. The
problem occurs when you proceed into programs without realistic
requirements and sufficiently mature technologies on which to base
realistic cost and schedule estimates and attempt to offset that
increased risk by offering award fees to motivate the contractor to
overcome that risk. The important question is: Have you adequately
defined and established appropriate criteria that enable you to measure
outcomes? And finally, how do you apply those criteria in determining
the level of fee that can be justified? We have made several
recommendations to this effect, and DOD has responded favorably with
new guidance to link award fees to acquisition outcomes. However, as
with other recommendations we have made related to DOD weapon system
acquisitions, the key will be how this new policy is implemented.
Acquisition Policy:
1.Given that DOD's acquisition policy seems to embrace practices
endorsed by the Defense Acquisition Performance Assessment, the Defense
Science Board Summer Study 2005, and GAO, what is preventing the
services from consistently executing these policies?
The services are not consistently executing these policies because DOD
is not enforcing a knowledge-based approach, discipline is lacking, and
business cases do not measure up. This is occurring, in part, because
there are no consequences for actions that run counter to the intent of
DOD acquisition polices--officials responsible for approving program
starts are no longer in their positions by the time the consequences of
their actions become evident. The department routinely accepts high
levels of technology risk at the start of major acquisition programs.
Mature technologies are pivotal to developing new products. Without
mature technologies at the outset, a program will almost certainly
incur cost and schedule problems. However, DOD's acquisition community
moves forward on programs with technologies before they are mature and
takes on responsibility for technology development and product
development concurrently. Our work has also shown that DOD allows
programs to begin without establishing a sound business case that
matches requirements with technology, acquisition strategy, time, and
funding. And once these programs begin, their requirements and funding
change over time. In fact, program managers consider shifting
requirements--which can result in added program complexity and costs--
and funding instabilities--which occur throughout the program--to be
their biggest obstacles to success. Fundamentally, DOD will need to
reexamine the entirety of its acquisition process and how it is
affected by requirements and funding processes. This includes making
significant changes to program requirements setting, funding, and
execution.
2.What else is needed to ensure that cost, schedule, and performance
outcomes for new weapons are predictable and achievable when these
programs seek approval from Congress?
DOD needs to take additional steps to achieve outcomes on par with best
practices. These include:
* developing and implementing an acquisition investment strategy;
* ensuring that individual programs are executable; and:
* clearly delineating responsibilities and holding government employees
and contractors accountable for achieving desired results.
While DOD has incorporated into policy a framework that supports a
knowledge-based acquisition process similar to that used by leading
organizations, it must establish stronger controls to ensure that
decisions on individual programs are informed by demonstrated
knowledge. Moreover, Congressional approval of programs that have not
taken these steps encourages DOD's subsequent requests for additional
funding.
3.What are the services doing now to ensure that the technologies in
these programs are achieving the desired level of maturity before they
proceed into system development?
DOD is now required by law to certify to the Congress that technology
is demonstrated to a specific maturity level before being approved for
system development. However, this is a recent statutory requirement and
it is too early to tell whether this requirement will ensure
achievement of the desired technology maturity level before programs
are authorized to proceed into system development. Prior to this new
law, some of the criteria upon which this certification is based were
reflected in DOD policy. However, our prior work showed that DOD
written polices are not always observed in practice.
4.GAO believes a sound business case is needed before a weapon system
begins system development and demonstration yet finds many of the
existing weapons' business cases to be broken. What do you consider key
elements of a business case for starting a development program and how
do you ensure these are followed in each planned development program?
The following are the essential elements of a sound business case:
* A requirement exists that warrants a solution consistent with
national security priorities.
* The developer has the mature technologies and technical knowledge
necessary to meet the requirement.
* The developer has a knowledge-based product development plan that
will attain high levels of design and production maturity at the right
times.
* Reasonable estimates have been developed to execute the product
development and production plan.
* Current and future funding will be available to fully resource the
product development and production plan.
* The agency has the capacity, either in-house or by contractual
arrangement, to manage its programs, including ensuring that
contractors are subject to appropriate independent oversight as to
cost, schedule, and quality.
To ensure that these elements are in place at the individual program
level, DOD needs to (1) begin programs in the context of an overall
funding-constrained, weapon system portfolio, and (2) say "no" when
individual programs do not measure up. DOD will need the support of the
Congress to succeed.
Management of Technological Risk:
1.It has often been demonstrated that concurrent technology development
and production in programs lead to wasted funds and undelivered
capability. Do you share in this concern? Please elaborate.
I do share in this concern. It is essential that technology be
developed separately--and before--the development phase of a product or
weapon system. This is a best practice that results in predictable
program outcomes. Our work shows programs that proceed with mature
technology average about 5 percent cost growth; programs with immature
technology experience about 35 percent cost growth. Moreover, immature
technology delays design and production maturity.
2. Your most recent annual report on DOD weapon systems has many cases
in which programs are entering production before TRL Level 7 has been
reached. Can you offer some examples of programs with unnecessary
concurrency?
As we noted in that report (GAO-06-391), allowing technology
development to spill over into product development puts an extra burden
on decision makers and provides a weak foundation for making product
development estimates. We found that programs that began with immature
technologies have experienced average research and development cost
growth of 34.9 percent; programs that began with mature technologies
have only experienced cost growth of 4.8 percent. Examples of programs
with concurrency include the F-22A, the Joint Strike Fighter, the
Future Combat Systems, the VH-71 Presidential Helicopter, the Global
Hawk, the CVN-21, the DDG-1000, the Multi-mission Maritime Aircraft,
and the National Polar-orbiting Operational Environmental Satellite
System, among others.
3.What prevents service acquisition executives from demanding that
acquisition programs not go forward until technology is mature?
First, the budgeting process requires that funding for a program is put
on the table 2 to:
3 years in advance--this creates pressure to proceed with the program
despite how mature the technology is. Second, the requirements process
tends to settle on ultimate performance, which puts pressure on
programs to reach for exotic technology. Third, it is easier to say
"yes" than "no". There can always be a compelling case made that a
particular program is an exception; the problem is that there are way
too many exceptions. And finally, this is occurring, in part, because
there are no consequences for actions that run counter to the intent of
DOD acquisition polices--officials responsible for approving programs
are no longer in their positions by the time the consequences of their
actions become evident.
4.Should concurrent technology development and system production be an
exception rather than the rule?
Definitely. Sequential phasing of technology development and system
development is DOD policy, best practice, and essential to delivering
capability faster to the warfighter and more economically to the
taxpayer.
5.What concrete proposals should be put in place to ensure programs are
not allowed to proceed to production without demonstrated maturity in
technology?
DOD needs to make sure each new program is executable before the
development begins, and well before production is considered. For
example,
* Science and technology organizations need the resources, authority,
and tools to mature technology and transition it to acquisition
programs.
* Requirements must be clearly defined and achievable given available
resources, including beginning product development with mature
technologies. Once begun, requirements should not change without
assessing their potential to disrupt the program and without being
fully resourced.
* DOD should say no to programs that do not measure up, including
canceling programs when appropriate.
* If a decision is made to proceed with immature technologies, the cost
and schedule estimates should be increased commensurate with the
additional risk.
In addition, the Congress can help by not funding programs that do not
measure up.
Role of Contractors:
1.Would you agree that DOD is relying more heavily on contractors to
manage and deliver weapon systems?
It is clear that DOD continues to increase its reliance on contractors
to provide both products and services. DOD's obligations on service
contracts rose from $82.3 billion in fiscal year 1996 to $141.2 billion
in fiscal year 2005. Based on our work looking at various major
systems, we have observed how DOD is relying on contractors in new ways
to manage and deliver weapon systems. For example, the Army's Future
Combat Systems is using a lead system integrator approach in which the
prime contractor has greater than usual responsibilities in areas such
as requirements definition, system design, and selection of major
subsystem contractors. This could increase taxpayer risk, especially if
the government does not provide adequate oversight.
2.What factors are contributing to this increased dependence on
contractors?
A variety of factors have contributed to the increased dependence on
contractors. In the case of the Future Combat Systems program, for
example, the Army chose to use a lead systems integrator because it did
not believe it had the in-house resources or flexibility to field such
a complex system in the time required. More generally, spending on
service contracting has increased in part to compensate for a declining
civilian workforce. Increased security requirements since September 11
have also led to more contract spending, particularly in areas such as
guard services.
3.What risks does it pose to protecting the government's interests?
As reliance on contractors has increased, so has the need to ensure
that the government's interests are protected. Organizational conflicts
of interest present one type of risk to the government and these must
be identified and managed, which requires careful government oversight.
When the government is acquiring services, challenges arise when
attempting to define requirements, establish measurable and performance-
based outcomes, and assess contractor performance.
4.Do you think this increased dependence has been a conscious, managed
trend, or has it more or less happened to us?
In the area of service acquisitions in particular, which currently
exceeds the value of major weapon systems, the growth in spending has
not been a managed outcome. For example, when DOD reduced the size of
its civilian workforce in the 1990s, it did so without proactively
shaping the workforce to ensure that it had the specific skills and
competencies needed to accomplish its mission. DOD has acknowledged
that it faces significant workforce challenges that, if not effectively
addressed, could impair the responsiveness and quality of acquisition
outcomes. In a recent report, Defense Acquisitions: Tailored Approach
Needed to Improve Service Acquisition Outcomes, GAO-07-20, Nov. 2006,
we recommended a number of steps DOD should take to make service
acquisitions a managed outcome.
5.What steps do we need to take to protect the government's interests
(that is, on behalf of both the warfighter and the taxpayer) in an
environment of increasing dependence on contractors?
We are currently reviewing a number of aspects of this issue, including
conflict of interest laws and ethical conduct standards for contractor
employees. In addition, the Congress can play an important role through
its oversight function in making sure that taxpayer interests are
protected. On November 17, 2006, we offered a set of issues, based on
GAO's work, for consideration for the agenda of the 110th Congress
(Suggested Areas for Oversight for the 110th Congress, GAO-07-235R).
The first set suggested targets for near-term oversight, including the
need to address governmentwide acquisition and contracting issues. In
this regard, we noted that the work of the government is increasingly
being performed by contractors and many agencies rely on contractors to
carry out their basic missions. At the same time, GAO's list of
government high-risk areas includes acquisition and contract management
issues that collectively expose hundreds of billions of taxpayer
dollars to potential waste and misuse. Consequently, we suggested that
Congress continue to monitor agencies' efforts to address existing
problems, while facilitating a re-examination of the rules and
regulations that govern the government-contractor relationship in an
increasingly blended workforce. Actions and topics that need continued
congressional oversight include:
* requiring agencies to report on mechanisms in place to ensure that
contractors are playing appropriate roles and that agencies have
retained sufficient workforce capacity to monitor contractor cost,
quality, and performance, particularly in such critical operations as
responding to Hurricane Katrina and rebuilding Iraq;
* assessing agencies' efforts to ensure that acquisitions are
performance-and outcome-based, with appropriate risk-sharing contracts
in place.
* requiring agencies with significant acquisition budgets, such as DOD,
to better align requirements, budget, and acquisition processes to
reconcile the differences between wants, needs, affordability, and
sustainability, given current and future demands and resources; and:
* monitoring the implementation of agency action plans to address the
GAO high-risk areas related to acquisition and contract management.
Incremental Funding:
1.Is it a good idea to expand the use of incremental funding for major
weapon system programs? If not, why not?
GAO has advocated full funding for capital asset acquisitions as a way
to increase recognition of implied commitments embodied in budgetary
decisions as compared to the incremental funding approach. That said, I
am not aware of any plans by DOD to expand the use of incremental
funding for major weapon system programs. Nevertheless, in testimony
earlier this year before the Airland Subcommittee of the Senate Armed
Services Committee, I expressed concern about an Air Force plan to use
incremental funding for a multiyear contract for the F-22A program
(Tactical Aircraft: Questions Concerning the F-22A's Business Case, GAO-
06-991T, July 25, 2006). Subsequent to that hearing, the Congress
authorized multiyear procurement for the F-22A, but expressly
prohibited the use of incremental funding for the contract.
2.Is it more dangerous to expand the use of incremental funding when
the overall budget is projected to fall or become more competitive? If
so, why?
Inherent in the concept of incremental funding of a program or project
is the expectation that additional funds will be available in
subsequent years to bring the program or project to completion.
Declining budgets or increased competition for available resources in
subsequent years would certainly make it more difficult for that
expectation to be realized.
Navy: DD(X) Destroyer:
1.What factors have contributed to driving DD(X) costs up and
quantities down?
DD(X) (now DDG 1000) lead ship procurement costs have increased to $3.3
billion because the Navy underestimated the resources that would be
required to construct (as well as design) a multi-mission ship offering
significant new capabilities related to air defense, land attack, and
undersea warfare. In addition, the Navy did not complete a
comprehensive, program life cycle cost estimate until March 2005. Prior
to this estimate, the Navy relied on program and contractor estimates
and the life cycle cost estimate completed in 1997 for the former DD 21
program.
2.Is this symptomatic of DOD's requirements, budgeting, and acquisition
processes in general?
Yes. Our work has shown that DOD programs often cost more and take
longer to develop than estimated when program managers are forced to
make cost and schedule predictions based on unproven technologies.
Alternatively, initiating product development with a high level of
knowledge about program technologies can help mitigate such risks.
As knowledge has grown in the DD(X) program, the Navy has frequently
discovered that its desired capabilities did not align with the funding
it had available. To compensate, the Navy has steadily reduced planned
ship quantities and land attack capabilities to enable continued
funding of other technology demonstration efforts on the ship.
We have also pointed out that the Navy generally does not hold a
milestone B, which approves entry into the System Development and
Demonstration phase, for ship programs until it is authorizing ship
construction. Other DOD programs are required to complete milestone B
before entering System Development and Demonstration. Completion of an
independent cost estimate is tied to the milestone. As a result the
Navy invested billions in DD(X) research and development without an in-
depth analysis of cost. This practice of delaying the milestone
continues today on such programs as Littoral Combat Ship and amphibious
assault ship (LHA 6).
3.What are the key risks that could jeopardize delivering the DD(X) on
time and within cost?
The Navy's strategy to continue maturing DD(X) technologies while
refining the ship's design and beginning construction of lead ships
places the program at significant risk for additional cost growth and
schedule delays.
Key development risks include remaining work associated with maturing
technologies, especially the volume search radar; completing design of
the integrated deckhouse; and maintaining software release schedules.
Because the Navy plans to begin construction of the lead DD(X) ships
with an incomplete understanding of the ship's critical technologies,
design changes may occur that could require costly rework throughout
the construction phase for the lead ships.
4.What steps need to be taken on DD(X) to ensure that it is delivered
on time and within cost?
The Navy's cost estimate for DD(X) assessed a 45 percent probability of
the lead ship delivering at or below a cost of $3.3 billion.
Consequently, the Navy may be forced to revisit planned capabilities
for the ship in the event that its $3.3 billion cost target becomes
unattainable. Moreover, before construction begins in fiscal year 2008,
the Navy should ensure that all technologies, such as the volume search
radar, have been fully demonstrated.
Navy/marine Corps: Expeditionary Fighting Vehicle (EFV):
1.What went wrong on the EFV program after it had such a good start?
Program difficulties occurred in part because not enough time was
allowed to demonstrate maturity of the EFV design during the System
Development and Demonstration (SDD) phase. Best practices and current
DOD acquisition policy call for system integration work to be conducted
before critical design review is held. This review represents the
commitment to building full-scale SDD prototypes that are
representative of the production vehicle. In the case of the EFV,
however, the SDD critical design review was held before the system
integration work had been completed. While testing of early prototypes
began one year before the SDD critical design review, it continued for
three more years after the decision to begin building the SDD
prototypes. Test schedules for demonstrating design maturity proved
optimistic and success-oriented, and were extended, and major problems
were discovered in testing of the prototypes. Specifically, the
original schedule did not allow adequate time for testing, evaluating
the results, fixing the problems, and retesting to make certain that
problems are fixed before moving forward. The SDD schedule of about
three years proved too short to conduct all necessary planning and to
incorporate the results into design changes, resulting in schedule
delays and cost increases.
2.Could these problems have been avoided?
Yes. The program was positioned for a good start in that it had mature
technologies fairly early. Subsequent problems could have been avoided
if the program's System Development and Demonstration phase followed
DOD policy preferences and best practices as they related to design and
production maturity.
3.Was DOD policy in error, or was the program not following policy?
DOD policy was not in error. In fact, the EFV was following a knowledge-
based approach and other best practices, which had been incorporated
into DOD's guidelines in 2000 for major acquisition and expanded into
its May 2003 policy. In addition to having reached a relatively high
level of technology maturity before starting SDD, the EFV program had
earlier adopted best practices in its implementation of Integrated
Product Teams and had trained its program office staff on this
acquisition improvement initiative. As noted above, the EFV departed
from the tenets of DOD acquisition policy and best practices shortly
after SDD began, when it attempted to do system integration and system
demonstration concurrently.
4.How do strategies that go against sound policy get approved?
Our work on multiple weapon systems shows that most programs proceed
with strategies that do not comport with the knowledge-based approach
embodied in DOD written policies. For example, in our March 2006
assessment of weapon systems, only 10 percent of the weapon systems
reviewed had begun SDD with mature technologies. There are several
reasons for this. First, although DOD acquisition policy allows
individual programs to tailor their own acquisition approaches, it does
not include sufficient controls to ensure that key aspects of the
policy are implemented. Second, programs proceed with optimistic
assumptions about what they can accomplish for their estimated cost and
schedules. Third, as weapon system programs proceed through
development, the desire to protect the program against disruption
intensifies.
What lesson from the EFV experience can be drawn and applied to DOD
acquisitions in general?
Using the lens of a knowledge-based business case, the EFV at the start
of SDD was sound on requirements and technology maturity (knowledge
point 1). While design stability was judged to be attained at the
critical design review (knowledge point 2) immediately after entering
SDD, it appears that holding critical design review so soon was
premature. This particular acquisition strategy did not provide the
resources (time and money) necessary to demonstrate design maturity and
production maturity (knowledge point 3). A key lesson learned is that
while it is necessary to demonstrate one knowledge point before a
subsequent one can be demonstrated, this alone is not sufficient.
Attaining one knowledge point does not guarantee the attainment of the
next one. Rather, the acquisition strategy for any program must
adequately provide for the attainment of each knowledge point even in
programs, such as the EFV, which were in a favorable position at the
start of SDD. If the acquisition strategy does not adequately provide
for the attainment of all knowledge points, the estimates for cost and
schedule will not have a sound basis.
Navy: Cost Growth in Shipbuilding Programs:
1.What did the GAO's work reveal as the major causes of continued
overruns in the Navy's shipbuilding program?
Our February 2005 report (GAO-05-183) on cost growth in shipbuilding
programs cited unrealistic estimates of costs as a key source of cost
growth. Unrealistic estimates in turn led to unrealistic budgets and
contract prices. Increases in labor hours and material costs were
common, with low technology maturity at program start and immature
design before beginning production contributing to these. We also
identified deficiencies in cost reporting as a problem.
2.The Navy has pledged over the past year or so to address this
problem. Have you seen any concrete proposals, and if so, what do you
think of them?
The Navy has cited several actions it has taken to improve cost
performance. These include using risk analysis in developing cost
estimates, budgeting at the Office of the Secretary of Defense Cost
Analysis Improvement Group estimate, using realistic inflation rates in
developing cost estimates, withholding fees if the contractor does not
provide useable and timely cost reports, and introducing more
aggressive contract incentives.
We believe these are positive steps but it remains to be seen whether
the Navy will follow through with these efforts. For example, while the
Navy has said that it will use contract incentives in the form of more
demanding share lines, lower ceiling prices, and lower minimum fees
(for cost plus incentive fee contracts), a key issue is whether these
policies will be sustained over time.
We also believe that further actions are needed. To reduce the risk of
cost growth, the Navy needs to insure that technology risk is not
carried forward into construction. Technology risk, and with it the
potential for design changes, needs to be addressed before construction
begins. This has not been the case with the DD(X) program, where key
technologies--such as the dual band radar--have not been fully
demonstrated.
Moreover, we believe the Navy could better ensure realism of budgets
and contracts. We believe funding and contracting for a detailed design
of the ship should be separated from the funding and contracting for
construction. A budget request for ship construction would be informed
by the knowledge gained from the first year or two of detail design.
3.Should ship programs be put on firm fixed price contracts?
Use of fixed price contracts is dependent on how much uncertainty
exists. It is important to reduce uncertainty early in a program so
that technology and design risks are not carried forward into
construction. Following a knowledge-based approach will likely enable
follow-on ships to use fixed price contracts. Use of such contracts
would provide the strongest incentive for improving cost performance--
under a fixed price contract the cost risk of performance generally
shifts to the contractor.
4.What specific ideas would you offer to deal with this problem?
In addition to the suggestions discussed above, we believe the Navy
should increase its use of cost-performance trade studies--including
design-to-cost and cost as an independent variable (CAIV) techniques--
to identify cost reduction opportunities in its new ship designs and
position the Navy to achieve "best value" outcomes in its shipbuilding
programs. Establishing design-to-cost goals early in a program
encourages consideration of cost as a key design parameter when
evaluating other parameters such as schedule, performance, and
operational capability. Goals can be set in each phase of the
shipbuilding acquisition process and tracked until desired cost targets
have been achieved. In addition, iterative CAIV analyses can aid Navy
efforts to properly balance cost with performance, design, and system
requirements in its shipbuilding programs. Such analyses can help the
Navy quantify and evaluate the degree to which its planned investments
will result in increased utility to the warfighter and subsequently
position the Navy to tailor future ship capabilities in ways that
maximize value.
Army: Future Combat Systems (FCS):
1.Was the Army's decision to begin System Development and Demonstration
phase with over 75 percent of FCS critical technology elements immature
consistent with DOD policy?
The decision to enter the System Development and Demonstration phase
with 75 percent of technologies immature does not satisfy the basic
tenets of DOD policy and best practices. DOD policy and best practices
are very clear that technologies should be mature before system
development begins. FCS, in particular, is worrisome because it is now
three years past the beginning of development, and most of the
technologies remain immature by best practice standards.
2.In retrospect, what should have been done to bring the program in
line with DOD policy and best practices?
The program should not have entered System Development and
Demonstration with undefined requirements and immature technologies.
Rather, it should have remained in the science and technology community
until the enabling technologies were mature, or requirements should
have been traded down to match the available technologies.
3.What are the implications for other acquisition programs--will they
adhere to DOD policies for awarding System Development and
Demonstration contracts for only those weapons whose critical
technologies have achieved desired maturity levels?
Programs like FCS underscore the challenge in getting better program
outcomes. Despite being out of alignment with the knowledge-based
approach outlined in DOD policy and advocated by best practices, the
program was approved to begin the System Development and Demonstration
phase and funded at high levels. The example of success then becomes
not programs that follow the intent of the policy, but those that do
not and still win approval and funding.
4.What can be done to mitigate risks in FCS, given where it is today?
We have recommended that the Secretary of Defense limit DOD's
commitment to the FCS product development phase and eventual production
until a sound business case that is consistent with DOD acquisition
policy and best practices can be clearly demonstrated. Further, we have
recommended a go/no-go decision be made in 2008 after the preliminary
design is done. This would in essence be the time that the program is
ready for a system development decision, as technologies should have
reached a basic level of maturity and system level requirements should
be finalized by this time. In addition, the Army's fiscal outlook is
extremely challenging. Outside of the Army, competition for production
dollars will be great as both the Air Force and Navy have significant
production programs planned for that time. Outside DOD, there will be
an increasing squeeze on discretionary funds (weapons are
discretionary) from other federal sectors.
5.What does the reliance on a lead systems integrator (LSI) say about
the Army's ability to manage a program of this magnitude?
The Army's reliance on a LSI suggests that the FCS is so ambitious in
scope, complexity and schedule, that it exceeded the Army's own ability
to effectively manage such a complex program.
6.What risks does the use of an LSI pose for protecting the interests
of the government?
The use of an LSI can pose a risk with respect to organizational
conflicts of interest. For example, an LSI might favor its own
solutions over other competing solutions. Further, if the LSI stands to
benefit from the continuation of a program into production, it has a
financial stake in the outcome that could compromise judgment. Over
time, the government's ability to oversee and act independently from
the LSI could weaken as experienced government managers retire and
dependence on the LSI increases.
Army: Warfighter Information Network-Tactical (WIN-T):
1.Was WIN-T another program that was approved with unrealistic
expectations for requirements, technology, cost, and schedule? Did the
program's original strategy follow DOD policy and best practices?
At the time DOD approved the start of system development in August
2003, only three of WIN-T's 12 critical technologies were close to full
maturity. Although DOD policy calls for technology maturity by the
start of system development, DOD did not require that the other nine
critical technologies reach a level close to full maturity until the
start of low-rate production.[Footnote 1] The technical and program
risks of moving forward with immature technologies are exacerbated by
the significant interrelationships among WIN-T's critical technologies.
For example, mobile communications technologies--a critical component
of WIN-T's operational concept--rely on antennas and other technologies
to achieve their performance objectives, and a lag in the development
of any of these technologies would result in a lag in the overall
development of mobile communications.
With respect to requirements, DOD had approved an operational
requirements document for WIN-T shortly before the start of system
development. However, WIN-T's interdependencies with other programs
further heighten the technical challenges and risks associated with the
program. In particular, at the time WIN-T system development began, the
Army's networking requirements for FCS--which WIN-T is expected to
heavily support--were not well understood. As a result, the networking
needs of FCS and the capabilities WIN-T planned to deliver to this
program were not well synchronized. In addition, WIN-T's tightly
compressed schedule assumed nearly flawless execution and did not allow
sufficient time for correcting problems if they arose.
2.What steps did the Army have to take to buy JNN? Why was this
approach necessary?
Rather than undergoing the normal acquisition processes, the Army
resorted to extraordinary measures outside of the normal requirements,
acquisition, and budgeting processes--including the supplemental budget
process--to rapidly acquire and field the Joint Network Node- Network
(JNN-N). Specifically, instead of initiating a formal acquisition
program, the Army utilized existing contracts, originally established
for modifying legacy networking systems, to procure JNN-N equipment in
approximately 5 months. For example, a purchase agreement from the WIN-
T program was used to procure satellite trailers; a General Services
Administration contract was used to arrange training; and existing Army
contracts were used to procure JNN-N in seven spirals, with each spiral
equating to approximately one division's worth of equipment.
Supplemental funding totaling $763.6 million for fiscal years 2004 and
2005 was used to purchase the first seven spirals of JNN-N. Additional
supplemental funding totaling $818.7 million for fiscal year 2006 is
being used to acquire JNN-N lots 8 through 10.[Footnote 2]
The Army's approach for acquiring JNN-N was intended to address an
urgent need identified during Operation Iraqi Freedom and Operation
Enduring Freedom to provide better communications capabilities between
soldiers fighting out of sight of one another. DOD's processes for
approving formal programs are extensive and have not been conducive to
rapid acquisition.[Footnote 3] However, these processes exist to ensure
that programs are managed and overseen, properly funded, and adequately
tested prior to fielding. The Army's decision to acquire the capability
through an approach that was outside these processes did not address
the legal and policy requirements for managing an acquisition program
of this magnitude. In particular, JNN-N procurement did not complete
initial operational testing before going beyond low-rate production, as
required by law and DOD policies.
The acquisition of JNN-N through extraordinary measures was made
necessary, in part, by DOD's decision to acquire a revolutionary WIN-T
capability. When the Army opted to pursue large technology advances in
networking capabilities to support the future force through WIN-T,
rather than pursuing a more incremental approach, it accepted a gap in
providing tactical networking capabilities to the warfighter. Moreover,
the Army's optimistic acquisition approach for WIN-T created the
impression that this capability gap was far smaller than it really was.
As the Army pursued an advanced WIN-T capability, its existing
communications equipment became increasingly obsolete and the need for
near-term improvements became more urgent with the onset of the war in
Iraq. If the Army had followed DOD's acquisition policy preferences,
which emphasize achieving capabilities in increments based on mature
technologies to get capability into the hands of the user more quickly,
it might have been able to get needed communications capabilities to
the warfighter sooner.
3.What lessons do the WIN-T and JNN acquisitions have for DOD's
requirements, budgeting, and acquisition processes?
While JNN-N's acquisition was a pragmatic response to an urgent need,
the urgency was brought about, in part, by the Army's decision to use a
revolutionary, rather than evolutionary, approach to acquiring WIN-T.
This approach appeared less risky at the time because optimistic
projections created the impression that the capability gap was narrower
than it really was. In hindsight, focusing on meeting the needs of the
future force at the expense of providing capabilities to the current
force proved detrimental once operations commenced in Iraq. To field
capabilities quickly, the Army did not use DOD's formal requirements
setting, budgeting, and acquisition processes, and over a billion
dollars was spent on JNN-N without it being established as a formal
program. This is not a good model for providing needed capabilities.
The WIN-T/JNN-N experience has implications beyond these two
communications systems. The Army's experience thus far creates
opportunities to capture lessons learned and apply them not only to the
Army's two programs but also to DOD's larger ongoing efforts to reform
its acquisition processes. We recently recommended that the Secretary
of Defense examine ways to apply the lessons learned from the Army's
experience with the acquisition of WIN-T and JNN-N to ongoing DOD-wide
efforts to reform its key decision support processes for setting
requirements, allocating resources, and acquiring weapon systems. DOD
concurred with this recommendation.
4.What challenges are presented by the merging of aspects of the WIN-T
and JNN programs?
The Army ultimately intends to create a tactical communications network
that functions as a single tactical backbone. To achieve this, the Army
must now reconcile two separate acquisition efforts intended to meet
one overarching need. Without a sound strategy for meeting near-and
long-term needs for communications and networking capabilities,
significant risks remain. Affordability continues to be an overriding
concern, particularly as the Army and DOD as a whole struggle to meet
the needs of current military operations while making prudent
investments in the future in a difficult fiscal environment. We
recently recommended that the Secretary of Defense direct the Secretary
of the Army to develop a sound strategy for transitioning the Army's
tactical communications from JNN-N to WIN-T. Such a strategy needs to
ensure that requirements for both of these efforts are clearly defined
and well integrated, that the fielding strategy is executable, and that
sufficient funding has been committed. DOD concurred with this
recommendation and noted that the Office of the Secretary of Defense
has directed that an approved JNN-N to WIN-T transition strategy be in
place before acquiring additional JNN-N systems.
Air Force/Navy/Marine Corps: Multi-Service F-35 Joint Strike Fighter
(JSF):
1.Were JSF technologies mature at the time of the decision to enter
System Development and Demonstration?
JSF entered development without its eight critical technologies being
mature. Information from the program office indicates that maturity has
progressed. However, seven technologies were not mature at the recent
design review and two of the technologies needed for mission
effectiveness are not expected to be fully mature until 3 years after
production begins. We reported in October 2001 that failure to mature
these critical technologies could result in increases in the production
and long-term ownership costs, schedule delays, and compromised
performance as problems arise in product development.
2.Is the program ready for a low-rate initial procurement decision in
FY 2007? What risks does the program face at the time of this decision?
The JSF program is not well positioned to enter production at this
time. At the time of the low-rate production decision, only one early
prototype will have flown and less than one percent of the flight
testing will be complete. In addition, none of the three variants will
have a production-representative aircraft built and in flight testing.
The program intends to invest $26 billion in initial production before
a fully integrated, production-representative aircraft will fly in
2011--4 years after the production decision. The cost of discovering
design problems during production could be significant if testing shows
that large, structural components of the aircraft require
modifications. Furthermore, the length and scope of the remaining
effort make it difficult to accurately estimate cost and completion
schedules. The program has about 7 years to complete the remaining
activities of the System Development and Demonstration phase, such as
developing 22 million lines of software code and completing a 7-year,
12,000-hour flight test program. By investing heavily in procuring JSF
aircraft before flight testing proves it will perform as expected, DOD
has significantly increased its risk of adding more cost and delaying
the delivery of critical capabilities to the warfighter.
3.What criteria must the program meet to demonstrate readiness for low-
rate production?
Key indicators of a program's readiness for entering production and
making significant investments in tooling, facilities, and materials
include demonstrating that (1) the aircraft's flying qualities function
within the parameters of the flight envelope, (2) the aircraft design
is reliable, or (3) a fully integrated and capable aircraft system can
perform as intended. In addition to a stable design, production
processes must be mature. This point is achieved when the program can
manufacture the aircraft within cost, schedule, and quality targets.
Because the program will lack key design and testing knowledge, DOD
plans to use cost reimbursement contracts for early production
aircraft. This type of contract places significantly greater cost risk
on DOD and the taxpayer. Confidence that investment decisions will
deliver expected capability within the cost and schedule goals is
likely to increase when flight testing proves the JSF will work as
expected.
4.When will the first production-representative, fully integrated
prototype aircraft be demonstrated? How many production aircraft will
be on contract by that time?
Under the current schedule, the first production representative, fully
integrated aircraft will not begin flight testing until the 2011
timeframe. This occurs 4 years after production has begun. At that
time, DOD will have committed to buy about 190 aircraft costing $26
billion. By the time testing is planned to be completed, in 2013, DOD
will have procured more than double that amount--424 aircraft--at an
estimated cost of about $49 billion.
5.What are the cost implications if that prototype indicates the need
for design changes at that stage of the program?
The costs of discovering design changes could be significant if testing
shows that large, structural components of the aircraft require
modifications or mission systems require additional development. For
example, problems discovered late in the development of the F-22A
increased a 4-year flight test program to about 8 years and affected
the program's ability to conduct operational testing and move into
production. In the JSF program, design changes needed in one variant
could also ripple through the other two variants, reducing efficiencies
necessary to lower production costs with common parts and manufacturing
processes for the three variants. Some industry officials have
indicated that the cost of design changes such as these could be 10 to
1,000 times greater, depending on how far the product has progressed
into production.
6.Under a cost-reimbursable production contract, will the government be
responsible for assuming the risk and cost of any needed design
changes?
Cost reimbursement contracts provide for payment of allowable incurred
costs, to the extent prescribed by the contract. These contracts are
generally used when uncertainties involved in contract performance do
not permit costs to be estimated with sufficient accuracy to use any
type of fixed price contract. Under a cost reimbursement contract, the
government assumes most of the risks of performance--because it agrees
to pay the contractor its allowable incurred costs to the extent
prescribed in the contract.
Air Force: F-22 Aircraft:
1.What fundamental causes led to the F-22's cost, schedule, and
performance problems?
We have reported in the past on the inefficiencies of the F-22
acquisition program that led to numerous schedule delays and cost
increases. This program provides an excellent example of what can
happen when a major acquisition program is not guided by the principles
of evolutionary, knowledge-based acquisition. The F-22 program failed
to match requirements with resources and make early trade-offs and took
on a number of new and unproven technologies. As a result, the program
failed to capture the appropriate technology, design, and manufacturing
knowledge of the F-22 aircraft at the right times during the
development process. This created a challenging and risky acquisition
environment that ultimately delayed by about 10 years to the warfighter
the capabilities expected from this new aircraft. It additionally led
to rising development costs, affordability issues, and changing
requirements over the 19 years in development.
2.Has the F-22 followed DOD policy and best practices?
No, the F-22 did not follow a best practices approach to its
development and procurement. As to whether the F-22 followed DOD
policy, we have not specifically assessed the F-22 program's compliance
with DOD policy. However, we reported in 2003 (GAO-04-53) and 2006 (GAO-
06-368) that DOD policy includes a good framework for weapons system
development and acquisition, but it does not include sufficient
controls to ensure key elements of the policy are implemented in each
phase of a weapons system program. As a result, our 2006 report found
that weapons system development programs continue to experience cost
and schedule growth.
3.What concerns do you have over the multiyear procurement proposal for
buying more F-22s?
In testimony earlier this year before the Airland Subcommittee of the
Senate Armed Services Committee, we expressed a number of concerns
about an Air Force proposal for multiyear procurement of the F-22A
(TACTICAL AIRCRAFT: Questions Concerning the F-22's Business Case, GAO-
06-991T, July 25, 2006). These concerns related to the savings expected
to be realized, the use of incremental funding, and the cost risk that
could arise if quantities to be ordered continued to decline. Although
Congress subsequently authorized multiyear procurement of the F-22A, it
listed a number of requirements the Air Force would have to address
related to the concerns we had expressed. In addition, the Congress
expressly prohibited the use of incremental funding.
Questions from Representative Kaptur:
In addition to the questions listed above, the Subcommittee also
submitted to GAO and DOD a list of eleven supplemental questions for
the record from Representative Marcy Kaptur, some of which deal with
the use of contractors in Iraq. These questions and others touch on an
emerging issue of critical importance, that is, the increasing role
that contractors play in conducting the business of government. We
appreciate and share the concern underlying the questions. Although DOD
is in a better position than GAO to provide the information that would
be needed to respond to each of the eleven questions, we have issued
reports regarding the use of private security contractors in Iraq and
DOD's use of logistic support contractors, which we believe are
relevant to the issues raised by Representative Kaptur. The reports
are:
Rebuilding Iraq: Actions Needed to Improve Use of Private Security
Providers, GAO-05-737, July 2005.
Military Operations: DOD's Extensive Use of Logistics Support Contracts
Requires Strengthened Oversight, GAO-04-854, July 2004.
We would be pleased to provide copies of these reports upon request.
They are also available on our Web site, http://www.gao.gov.
(120598):
FOOTNOTES
[1] In mid-2003, the Army developed risk mitigation plans for the nine
immature technologies, but a program review sponsored by the Army in
July 2004 concluded that these plans lacked sufficient detail.
[2] In keeping with its transition from a division-based to a modular
brigade-based force, the Army is now procuring JNN-N in batches of
brigade sets. It terms these batches lots, rather than spirals.
[3] The Army explored using special rapid acquisition authority to
respond to combat emergencies to formally waive acquisition laws and
policies for JNN-N, but the dollar limit for use of this authority--
$100 million per year for the entire Department of Defense--was far too
low to accommodate the Army's need for JNN-N. See section 806 of the
Bob Stump National Defense Authorization Act for Fiscal Year 2003 (P.L.
107-314), as amended by section 811 of the Ronald W. Reagan National
Defense Authorization Act for Fiscal Year 2005 (P.L. 108-375).
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
