Best Practices
Setting Requirements Differently Could Reduce Weapon Systems' Total Ownership Costs Gao ID: GAO-03-57 February 11, 2003For fiscal year 2003, the Department of Defense (DOD) asked for about $185 billion to develop, procure, operate, and maintain its weapon systems. This request represents an increase of 18 percent since 2001 for the total ownership costs of DOD weapon systems. Often, DOD systems need expensive spare parts and support systems after they are fielded to meet required readiness levels. DOD has been increasingly concerned that the high cost of maintaining systems has limited its ability to modernize and invest in new weapons. This report examines the best practices of leading commercial firms to manage a product's total ownership costs and determines if those practices can be applied to DOD.
Even though DOD has implemented several initiatives to reduce total ownership costs, some systems, such as the Apache helicopter or the Abrams tank, have experienced costly maintenance problems and low readiness rates, which persisted even after the systems were fielded. We found several reasons for these problems. First, DOD based requirements for weapon systems in product development almost exclusively on technical performance, with little attention to operating and support costs and readiness at the beginning of development when there is the greatest chance of affecting those costs positively. Second, using immature technologies to meet performance goals weakened DOD's ability to design weapon systems with high reliability. Finally, DOD's organizational structure is linear and limits collaboration and feedback among organizations charged with requirements setting, product development, and maintenance. In contrast, commercial companies that we visited considered operating and support costs to be integral to their new product development decisions. Studies have shown that by the time a product is ready for development, over 90 percent of the operating and support costs have been determined. As a result, these companies required their equipment be easy to maintain, ready when needed, and reliable at a low cost. These requirements were of equal importance to other performance characteristics. After setting requirements, product developers then designed products to meet established reliability rates, using technologies that were proven through past use or testing. At all of the companies we visited, customers and product developers alike, had very collaborative processes and practices that draw extensively on data from past operations to influence the design of new products.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-57, Best Practices: Setting Requirements Differently Could Reduce Weapon Systems' Total Ownership Costs
This is the accessible text file for GAO report number GAO-03-57
entitled 'Best Practices: Setting Requirements Differently Could Reduce
Weapon Systems' Total Ownership Costs' which was released on February
11, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Subcommittee on Readiness and Management Support,
Committee on Armed Services, U.S. Senate:
United States General Accounting Office:
GAO:
February 2003:
BEST PRACTICES:
Setting Requirements Differently Could Reduce Weapon Systems‘ Total
Ownership Costs:
GAO-03-57:
GAO Highlights:
Highlights of GAO-03-57, a report to the Subcommittee on Readiness and
Management Support, Committee on Armed Services, U.S. Senate:
February 2003:
BEST PRACTICES:
Setting Requirements Differently Could Reduce Weapon Systems‘ Total
Ownership Costs:
Why GAO Did This Study:
For fiscal year 2003, the Department of Defense (DOD) asked for about
$185 billion to develop, procure, operate, and maintain its weapon
systems.
This request represents an increase of 18 percent since 2001 for the
total
ownership costs of DOD weapon systems. Often, DOD systems need
expensive
spare parts and support systems after they are fielded to meet required
readiness levels. DOD has been increasingly concerned that the high
cost
of maintaining systems has limited its ability to modernize and invest
in
new weapons. This report examines the best practices of leading
commercial
firms to manage a product‘s total ownership costs and determines if
those
practices can be applied to DOD.
What GAO Found:
Even though DOD has implemented several initiatives to reduce total
ownership costs, some systems, such as the Apache helicopter or the
Abrams
tank, have experienced costly maintenance problems and low readiness
rates,
which persisted even after the systems were fielded. We found several
reasons for these problems. First, DOD based requirements for weapon
systems
in product development almost exclusively on technical performance,
with
little attention to operating and support costs and readiness at the
beginning
of development when there is the greatest chance of affecting those
costs
positively. Second, using immature technologies to meet performance
goals
weakened DOD‘s ability to design weapon systems with high reliability.
Finally,
DOD‘s organizational structure is linear and limits collaboration and
feedback
among organizations charged with requirements setting, product
development,
and maintenance.
In contrast, commercial companies that we visited considered operating
and
support costs to be integral to their new product development
decisions.
Studies have shown that by the time a product is ready for development,
over
90 percent of the operating and support costs have been determined. As
a
result, these companies required their equipment be easy to maintain,
ready
when needed, and reliable at a low cost. These requirements were of
equal
importance to other performance characteristics. After setting
requirements,
product developers then designed products to meet established
reliability
rates, using technologies that were proven through past use or testing.
At
all of the companies we visited, customers and product developers
alike, had
very collaborative processes and practices that draw extensively on
data from
past operations to influence the design of new products.
Figure: Percent of Life Cycle Costs Determined at Various Points in the
Acquisition Process:
[See PDF for image]
[End of figure]
What GAO Recommends:
GAO recommends DOD (1) revise its guidance for setting requirements to
include
total ownership cost goals and readiness rates for any major weapons
system as
performance parameters equal to any others; (2) revise acquisition
regulations
to require a firm estimate of component and subsystem reliability by
the systems
integration phase and an estimate of system reliability at the
production
decision; and (3) structure contracts to ensure proper trade-offs
between
reliability and performance.
www.gao.gov/cgi-bin/getrpt?GAO-03-57.
To view the full report, including the scope and methodology, click on
the
link above. For more information, contact Katherine Schinasi at (202)
512-4841.
Contents:
Letter:
Executive Summary:
Purpose:
Background:
Results in Brief:
Principal Findings:
Recommendations for Executive Action:
Agency Comments:
Chapter 1: Introduction:
Total Ownership Cost Is the Cost to Ensure Readiness:
Commercial Best Practices:
Objectives, Scope, and Methodology:
Chapter 2: DOD‘S Requirements-Setting and Development Practices Yield
Higher Total Ownership Costs:
DOD‘s Weapon System Programs Encounter Cost Growth in Achieving
Readiness Rates:
DOD‘s Linear Acquisition Approach Makes It Difficult to Control
Operations and Support Costs:
Chapter 3: Commercial Companies Deliberately Manage Ownership Costs
through Product Requirements and Design:
A Best Practices Model:
Leading Companies Treat Readiness and Operating and Support Cost as
Critical Product Requirements:
Knowledge-Based Product Development Is Critical to Achieving Desired
Reliability and Managing Operating and Support Costs:
Leading Commercial Firms Use Feedback from Operations to Better
Understand Customer Needs, Product Deficiencies, and Operating and
Support Costs:
Chapter 4: Stressing Operating and Support Cost at the Outset of an
Acquisition Could Help DOD Reduce Total Ownership Costs:
Differences in Practices Explain Different Outcomes for Commercial
Companies and DOD in Controlling Total Ownership Costs:
Several DOD Efforts Underway to Reduce Total Ownership Costs:
DOD‘s Current Environment Does Not Provide Incentives to Reduce Total
Ownership Cost Early:
Chapter 5: Conclusions and Recommendations:
Recommendations for Executive Action:
Agency Comments and Our Response:
Related GAO Products:
Appendix I: Comments from the Department of Defense:
Appendix II: GAO Staff Acknowledgments:
Tables:
Table 1: Readiness and Operating and Support Costs for Selected
Weapons:
Table 2: DOD and Commercial Practices for Controlling Operating and
Support Costs:
Figures:
Figure 1: Nominal Life-Cycle Cost of Typical DOD Acquisition Program
with a 30-Year Service Life:
Figure 2: Percent of Operating and Support Costs Determined at Various
Points in the Acquisition Process:
Figure 3: Readiness, Reliability, and Operating and Support Costs:
Figure 4: DOD‘s Linear Acquisition Process:
Figure 5: System Readiness Comes at High Operating and Support Costs
When Reliability Is Not Ensured:
Figure 6: Apache Helicopter:
Figure 7: Abrams Tank:
Figure 8: Commercial Model for Reducing Operating and Support Costs:
Figure 9: Benefits of Ensuring High Reliability Rates During Product
Development:
Figure 10: Polar Tanker‘s Polar Endeavor:
Figure 11: United Airlines/Boeing 777:
Figure 12: FedEx Express Delivery Van:
Figure 13: Joint Strike Fighter:
Figure 14: Advanced Amphibious Assault Vehicle:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO‘s product.
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images, or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO‘s product.
Letter:
February 11, 2003:
The Honorable John Ensign
Chairman
The Honorable Daniel Akaka
Ranking Minority Member
Subcommittee on Readiness and Management Support
Committee on Armed Services
United States Senate:
As you requested, this report examines how best practices offer
improvements to the way the Department of Defense develops new weapon
systems to reduce their total ownership cost, especially the operating
and support costs, during design. It examines how the department
currently designs for operating and support costs and how best
practices could improve outcomes and reduce costs. We make
recommendations to the Secretary of Defense for improvements to weapon
system requirements policy, including establishing operating and
support cost and readiness goals as performance parameters equal to any
other performance parameters; revising acquisition policies to require
a firm estimate of reliability during product development; and
providing contract incentives for product developers to make
appropriate trades between reliability and performance before
production.
We are sending copies of this report to the Secretary of Defense; the
Secretary of the Army; the Secretary of the Navy; the Secretary of the
Air Force; the Director of the Office of Management and Budget; the
Director, Missile Defense Agency; and interested congressional
committees. We will also make copies available to others upon request.
In addition, the report will be available at no charge on the GAO Web
site at http://www.gao.gov.
If you have any questions regarding this report, please call me at
(202) 512-4841. Other contacts are listed in appendix II.
Katherine V. Schinasi
Director
Acquisition and Sourcing Management:
Signed by Katherine V. Schinasi:
[End of section]
Executive Summary:
Purpose:
For fiscal year 2003, the Department of Defense (DOD) asked for about
$184.9 billion to develop, procure, operate, and maintain weapon
systems. With this funding, DOD will have received increases totaling
about 18 percent since 2001 for what it defines as total ownership
costs of its equipment. DOD‘s budget for operations and maintenance
increased by about 5.6 percent during the same period--from about $59
billion to about $62.3 billion. DOD has been increasingly concerned
that the high cost of maintaining weapon systems to meet required
readiness levels is depleting modernization accounts and denying the
department the flexibility to invest in new weapons. In recognition of
this concern, DOD has established goals to reduce operating and support
costs of weapon systems already in the field as well as those currently
in development. In order to provide another perspective on this
problem, GAO has continued a body of work to identify best practices
used by leading commercial companies to manage new products‘ total
ownership costs to see if these practices can be applied to DOD‘s
weapon system acquisitions.
This report addresses how DOD can use best practices from commercial
companies during its acquisition process to reduce total ownership
costs of its major weapon systems. It presents a model of the process
commercial companies use to incorporate reasonable and manageable
operating and support cost into their product development process. In
response to a request from the Chairman and Ranking Minority Member,
Subcommittee on Readiness and Management Support, Senate Committee on
Armed Services, GAO (1) determined the practices, processes and metrics
DOD has historically used to manage and control operating and support
costs of its major weapon systems; (2) determined the practices,
processes and metrics commercial companies use to manage and control
operating and support costs; and (3) analyzed the extent to which
opportunities exist to apply best practices to reduce operating and
support costs during product development.
Background:
Readiness is a critical parameter of all DOD‘s weapon systems. If a
system is not ready, its performance characteristics are of no use.
Each weapon system has an expected readiness rate, usually expressed in
some percentage of available units, that it is expected to maintain for
our national security. Readiness can be achieved by building highly
reliable weapon systems or, if the systems are not highly reliable,
supporting them with an extensive logistics system that can ensure
spare parts and other support items are available when needed. In
essence, the cost of a product‘s readiness is the cost to develop,
produce, and operate and maintain that system.
DOD recognizes that the total cost of a weapon system includes more
than development and procurement costs. Traditionally, development and
procurement have accounted for about 28 percent of a weapon‘s total
ownership cost, while costs to operate, maintain, and dispose of the
weapon system account for about 72 percent of the total. For a number
of years, the department‘s goal has been to spend less on supporting
systems and to devote more funds to development and procurement in
order to modernize weapon systems. But, in fact, growth in operating
and support costs has limited the department‘s buying power. DOD
officials have cited shortages of spare parts and unreliable equipment
as reasons for low mission-capable rates for some weapons. As a result,
some modernization has been postponed in order to pay high and
unexpected operating and maintenance costs.
GAO has issued a series of reports on best practices that commercial
firms use to manage and control the acquisition cost of their products.
Commercial firms attain knowledge early in the development process
about technology that they plan to incorporate; they make sure the
design is mature before production; and they have production processes
under control before production begins. The building of knowledge in
these areas can also contribute to the reduction of a product‘s
operating costs over its life cycle, thereby reducing a major portion
of its total ownership costs. While those reports focused on best
practices for reducing the cost and cycle time for acquiring weapon
systems, this report will focus on what DOD can do during the
acquisition of a weapon system, prior to and during product
development, to ensure that products are available when needed, more
cost effective to operate and support and more reliable once fielded,
thereby reducing their total ownership cost.
Results in Brief:
DOD weapons acquisition processes do not consider operations and
support costs and readiness as key performance requirements for new
weapon systems, and DOD places less emphasis on establishing operations
and support cost and readiness as key nontradable goals early in
product development. Generally, the department settles for lower
reliability in its new weapon systems‘ designs. In our review of data
for five fielded weapon systems, we found none had established
operating and support cost or readiness as key requirements. Although
recent readiness levels were acceptable to the services for the most
part, the five systems had experienced growth in their operating and
support cost estimates of between 16 and 48 percent within the last 12
years and problems with reliability once in the field.
We found several reasons for cost growth. DOD‘s acquisition process is
linear, progressing from one organization to the next with little
interaction among the groups. Requirements-setting by the war-fighting
community focuses on system performance. DOD policy does not require
inclusion of readiness or operating and support cost goals as key
performance requirements equal in importance to other performance
requirements. None of the systems we reviewed had a readiness or an
operating and support cost goal as a key requirement. Further, during
product development, the use of immature technologies and components to
meet performance goals worked against designing weapon systems with
high reliability. Using immature technologies also acts as a barrier to
manufacturing techniques such as open systems[Footnote 1] or designing
for fewer parts, practices that typically help reduce maintenance costs
of the system and increase its reliability. DOD‘s systems for
accumulating data to analyze operating and maintenance actions on
weapon systems already in the field do not provide adequate or reliable
information, thus making it difficult for DOD to understand the total
cost of operations and support. The outcome of these practices at DOD
has been an inability to stem the continuous growth in total ownership
cost, with actual operating costs continuing to exceed initial
estimates. As a result, DOD continues to request more operating and
support funding to sustain its systems or to reprogram funds from other
accounts to pay the bills.
In contrast, commercial companies consider reasonable operating and
support costs and the readiness or availability of their equipment as
requirements equal in importance to other performance characteristics,
thereby ensuring that the developer places proper emphasis on achieving
reliability and operating and support cost goals during product
development. Commercial companies have a collaborative process for
setting requirements, developing the product, and collecting and
sharing data on maintaining and supporting the product once it is
delivered. Both the customer and the developer have a voice in the
process. During product development, especially during the design
process, the maintainer has an active voice and is armed with
information about operating and support cost drivers in the previous
product. Commercial product developers maintain high standards for
reliability, using proven technologies to achieve critical performance
requirements. They find an evolutionary development process is critical
to reducing operating and support costs and achieving high readiness.
Emphasis is placed on reducing the number of parts in a design so there
is less to maintain, using standardized parts that are readily
available in the industrial base and using open systems to maintain
competition. Once the product is delivered to the customer, maintainers
keep detailed records on its reliability and the cost of its
maintenance and support. Importantly, information on the product‘s
performance is communicated back to the developer to be used to improve
the product.
DOD has implemented initiatives to reduce the total ownership cost of
its weapon systems. It has modified acquisition policies, established
programs to reduce operating and maintenance costs in existing systems,
and selected several acquisition programs to test different approaches
to reduce life-cycle costs during development. However, these steps do
not incorporate many of the practices used by commercial companies
during requirements determination, product development, and fielding.
In comparing DOD‘s practices to those found at leading commercial
firms, we have identified several differences. Because companies
operate in an environment where operating costs and readiness are
critical to their survival, commercial customers establish low
operating and support cost and high readiness requirements when
purchasing a product. This forces product developers to design reliable
systems that are easy and relatively cheap to operate and maintain. The
collaborative relationship between the customer and the product
developer is essential to driving down operating and support costs.
Further, companies understand that accurate operating and support cost
data from current products are also necessary to make good decisions
related to the purchase of a product, facilitate cost/performance
trade-offs, and provide feedback to the manufacturer for continuous
improvement.
GAO is making recommendations to the Secretary of Defense on ways to
improve DOD‘s management of operating and support costs. We are
recommending that DOD revise its requirements generation process to
include total ownership cost, especially operating and support cost,
and weapon system readiness rates as performance parameters equal to
any others. We also recommend that any revision of the current policy
governing acquisition processes require a firm estimate of the systems‘
reliability based on demonstrated reliability rates at component and
subsystem levels no later than the end of the system integration phase
and at the system level no later than the production decision. Finally,
we recommend that DOD structure contracts by Milestone B, the start of
the system development and demonstration phase, to ensure that proper
trades are made between reliability and performance before the
production decision.
Principal Findings:
DOD‘S Current Practices for Setting Requirements and Developing New
Weapon Systems Continue to Yield Higher Total Ownership Costs:
DOD is spending more on operating and support costs for its weapon
systems than it planned. We found three primary reasons for the high
cost of operating and supporting DOD‘s fielded weapon systems. These
were (1) little or no attention to the trade-offs between readiness
goals and the cost of achieving them when setting the key parameters
for weapon systems;[Footnote 2] (2) the use of immature technologies
during product development and delays in acquiring knowledge about the
design and its reliability until late in development, or in some cases,
production; and (3) insufficient data on the operations and maintenance
costs and actions for fielded systems that would allow improvements in
products currently in development. The outcome of these practices in
DOD has been an inability to stem continuous growth in total ownership
cost, with actual operating costs continuing to exceed initial
estimates. As a result, DOD continues to request more operating and
support funding to sustain its systems or reprogram funds from other
accounts to pay the bills.
Even though operating and support costs are the largest factor in a
weapon system‘s total ownership cost, they do not receive the same
attention when requirements are set for a weapon system as other
performance characteristics. In our review of data for five fielded
weapon systems, we found that none had established an operating and
support cost or a readiness goal as a key requirement prior to product
development. In fact, operating and support cost estimates were not
available in the Selected Acquisition Reports until at least 5 years
into product development on these programs. Most of the fielded systems
we reviewed were near or achieving readiness goals, but had experienced
significant cost growth in operations and support cost estimates within
the last 12 years to do so. Two of these systems, the Apache and the
Abrams, were designated as the Army‘s most expensive weapons to
support. The C-17 reported a cost increase of almost 25 percent, and
program officials stated that they would not have a firm estimate of
operating and support cost until 2010--more than 25 years after the
start of development.
We found practices in three areas--requirements-setting, product
development, and operations and maintenance--that contributed to this
condition. DOD‘ s acquisition process is linear and serially involves
several organizations whose responsibilities in the process have
differing objectives. Communication among the different organizations
is fragmented. Requirements focused on the weapon system‘s performance
characteristics. Once the weapon system‘s requirements were set and the
development of the system began, product development focused on
achieving the program‘s acquisition cost, schedule and performance
goals, rather than on increasing reliability in order to reduce its
total life-cycle costs. We found that the maintainers had limited
involvement in making design trades for lower operating and support
costs during development and that best practices such as designing for
open systems or ease of maintenance were not used by the developer. We
also found that once a system is fielded, the services‘ systems for
tracking operating and support costs were suspect, providing inadequate
feedback to suppliers and requirements setters.
Commercial Companies Deliberately Manage Ownership Costs through
Product Requirements and Design Process:
We found that commercial companies that use capital equipment
considered operating and support costs integral to their new product
development decisions. Companies such as United Airlines, FedEx
Express, and Polar Tanker employ practices to maintain the readiness of
their fleets at as low an operating cost as possible. Reducing these
costs translates into revenues, profits, and market growth. Increases
to these costs can mean market failure. As customers, they have
established operating and support costs and product readiness as key
system requirements before development begins for a new product that
are equal in importance to requirements for its performance and the
cost to develop and produce the product. For example, United Airlines
requires that new aircraft maintain a readiness rate of 98.5 percent or
the manufacturer must reimburse it for lost revenue. Polar Tanker
established a requirement that its Endeavor Class tanker operate at
least 330 days a year at a reduced operating cost per tanker. These
dual requirements drove trades during design, sometimes increasing
development costs to achieve lower operating costs. Before FedEx
Express agreed to a new design for its delivery trucks, it required
that the new design last for at least 300,000 miles over a specified
number of years and at a specific cost per mile. In gaining agreement
with product developers on these requirements prior to product
development, these companies sometimes had to trade performance or
spend more in development, but they received more reliable products,
reduced total ownership costs, and made those costs more predictable.
To meet their customers‘ supportability requirements, we found that
commercial product developers focused on designing a product that was
easy to maintain, would be ready when needed, and reliable at low cost.
They used an evolutionary development process. Consequently, they did
not allow components or subsystems into a product‘s design unless the
technology had been proven reliable through past use or testing. For
example, Boeing told us that it defers use of immature technology to
later evolutions of design and makes the reliability ratings of its
components available to the airlines before it begins product
development. Maytag completed reliability testing on every new product
prior to going to production. These companies also emphasized product
designs with fewer parts and open systems. Maytag has established a
parts reduction program as part of its development process, and Boeing
built its 777 so that any of three engines--GE, Pratt & Whitney, or
Rolls Royce--would fit. Developers also gained insight into design
features their customers valued through regular communication with
them. For example, the design for Boeing‘s latest generation 737--
geared toward reducing operating costs--was inspired by the airlines.
Boeing emphasized open systems, standardized parts, and reduced parts
from one generation to the next to satisfy the airlines‘ need for
reduced operating costs.
All of the companies we visited, customers and product developers
alike, had very collaborative processes and practices for drawing
extensively on data from past operations to influence the design of new
products. This information was used as a baseline for new product
designs and was used to estimate the operating costs of new products.
United Airlines officials told us that the airlines and the
manufacturers both keep meticulous reliability and cost records at all
levels of an airplane--components, subsystems, and at the system level.
Major operating costs drivers are tracked on a daily basis by the
airlines, and the manufacturers usually have personnel residing with
the airlines‘ maintenance crew to help solve problems on the spot and,
just as importantly, to feed information back to the manufacturer so
that the next product can be improved. FedEx Express and Polar Tanker
both emphasized extensive data collection from current operations. In
fact, FedEx Express sets annual targets for operating and support cost
reductions based on data gathered on the road. Polar Tanker gathered
maintenance data from past operations and established a team made up of
its own maintenance personnel and outside consultants to determine
areas that could result in higher reliability and lower maintenance
costs in designing the new Endeavor Class tanker.
Greater Emphasis on Operating and Support Cost at the Outset and during
an Acquisition Program Could Help DOD Reduce Total Ownership Costs:
DOD and the commercial companies we visited have policy goals of
developing products that will meet customers‘ needs at the lowest
possible cost to build and operate. The difference between them is in
how each implements its policies. Leading commercial companies follow
an integrated, collaborative process of setting requirements,
developing the product, and ensuring that the product can be supported
at an acceptable cost. DOD‘s process is composed of disparate practices
carried out by separate organizations with differing objectives and
little communication among them about how to support fielded systems.
While commercial firms focus on total ownership costs at the outset,
DOD focuses mostly on technical performance. One cause of this is that
in DOD the accountability and responsibility for total ownership costs
are spread across many organizations with separate goals. Another cause
lies in motivation for low costs. The commercial companies we visited
are driven by the need to be as profitable as possible to survive, and
low total ownership costs translate to higher profitability. DOD‘s
environment does not provide such incentives, and the organizations
charged with acquiring and operating weapon systems are unconstrained
by a need to lower costs since they can request additional operations
and maintenance funding to keep systems working.
Some of the practices used by commercial companies to reduce a new
product‘s operating costs during its development may be helpful to DOD.
In setting requirements, commercial customers make readiness and
operating cost requirements and collaborate directly with the product
developer. Product developers establish sound cost estimates early;
designs are simplified by using open systems and reducing parts;
reliability testing is done early; and a reliability growth curve is
established before production begins. Once a product is fielded,
operating costs are managed to established targets; operating cost data
is collected, analyzed, and used by the developer and the customer to
develop more reliable products in the future; and continuous
improvements are made to future products. The commercial practice of
establishing readiness and operating cost as key requirements for a new
product necessitates substantive input from operators and developers
before and during product development. The commercial practices used
during product development to design reliable systems that are easy and
less costly to operate and maintain depend on the use of good product
development practices including the use of mature technologies to meet
requirements. Commercial firms use incremental product development
processes and depend on a strong relationship between the manufacturer
and the customer‘s operators and maintainers to continue throughout
product development.
DOD does not focus on operating and support costs to the degree
commercial companies do. In setting requirements, DOD does not make
readiness and operating cost key parameters, performance is rarely
reduced in favor of reliability or reduced operating cost, and there is
no direct relationship between the requirement setters and the product
developer. During product development, firm estimates of operating
costs are not required, little attention is paid to reliability rates,
and open systems or design for manufacturing techniques are rarely
used. Once a weapon system is fielded, there is a lack of complete and
reliable data available from the field, and there is little
collaboration between maintainers and product developers to improve new
systems. DOD‘s acquisitions usually begin with critical technologies
that are immature, with unproven reliability. This makes it difficult
to implement best practices such as design for manufacturing during
product development. Accurate operating and support cost data are not
available for helping management make good decisions, facilitating
cost/performance trade-off decisions, and providing feedback to the
manufacturer for continuous improvement. On the weapon systems we
reviewed, we found that the programs had poor initial estimates of
operating and support costs for weapon systems, partly because they do
not have reliable systems in place to track those costs per weapon
system.
DOD has taken some steps to lower its weapon systems‘ total ownership
costs. Those actions include concurring with and implementing
recommendations concerning the use of technology readiness levels,
indicators of design maturity, and controlled production processes.
Further, the department initiated pilot programs with 30 acquisition
programs to develop methods for reducing total ownership costs.
However, DOD‘s current environment--both culturally and
organizationally--is not presently conducive to applying them.
Currently, its acquisition policies do not provide specific guidance
for controlling total ownership cost and its requirements-generation
policies provide no guidance for establishing readiness or cost goals
for weapon systems once they are fielded.
Recommendations for Executive Action:
GAO recommends that the Secretary of Defense:
* revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01B
on the requirements generation process to include total ownership cost,
especially operating and support cost, and weapon system readiness
rates as performance parameters equal in priority to any other
performance parameters for any major weapon system prior to beginning
the acquisition program;
* revise the current policy governing the operation of the defense
acquisition system (currently under revision) to require that the
product developer establish a firm estimate of a weapon system‘s
reliability based on demonstrated reliability rates at the component
and subsystem level no later than the end of the system integration
phase, coinciding with the system-level critical design review, before
proceeding into the system demonstration phase of product development;
and at the system level no later than the full-rate production
decision; and:
* structure DOD contracts for major systems acquisitions so that at
Milestone B the product developer has incentives to ensure that proper
trades are made between reliability and performance prior to the
production decision. One option is to provide specific clauses in the
development contract to address reliability growth.
Agency Comments:
DOD partially concurred with all of our recommendations but, for the
most part, found no further action was needed to lower total ownership
cost. We disagree. We believe that if DOD takes no further action in
implementing these recommendations, it would ignore significant
opportunities to improve readiness by lowering total ownership cost in
a budget environment that demands more effort to reduce these costs.
A detailed discussion of DOD‘s comments appears in Chapter 5 and the
full text of DOD‘s comments is in appendix I.
[End of section]
Chapter 1: Introduction:
For fiscal year 2003, DOD asked for $184.9 billion to fund research and
development, procurement and direct operations and maintenance costs of
its weapon systems. These elements along with disposal costs are
defined as the total ownership cost of a weapon system. The budget has
increased by about 18 percent for these activities since 2001--with
direct cost for operations and maintenance of weapon systems increasing
by about 5.6 percent from $59 billion to about $62.3 billion. Since the
late 1990‘s, DOD has been increasingly concerned that the cost of
operating and supporting weapon systems to meet required readiness
levels is depleting its modernization accounts and denying the
department the flexibility to invest in new weapons.
Total Ownership Cost Is the Cost to Ensure Readiness:
Commercial companies and DOD both use readiness as a key indicator of a
product‘s success. Readiness is usually expressed as the percentage of
total units available and capable of performing a mission at any given
time. If a weapon system is not ready when it is needed, its
performance characteristics are of no use. In general, readiness can be
achieved either by building highly reliable weapon systems or, if the
systems are not highly reliable, supporting them with an extensive
logistics system that can ensure spare parts and other support items
are available when needed. In essence, the cost to ensure a product‘s
readiness is the cost to develop, produce, operate, and maintain that
product through its life cycle.
The development and production cost of a weapon system, also known
as acquisition cost, usually represents about 28 percent of the weapon
system‘s total ownership costs. The acquisition cost is funded through
DOD‘s research, development, test and evaluation, and procurement
accounts. These funds are used to mature new technology and design and
manufacture new weapon systems. Operating and support costs[Footnote 3]
are typically the highest portion of a weapon system‘s total ownership
cost because they represent the cost to operate the system and keep it
ready for action over many years, sometimes more than 30 years. These
costs are about 72 percent of the total ownership cost of a weapon
system and are funded through DOD‘s operations and maintenance account.
Operating and support costs reflect the purchases of fuel, lubricants,
and repair parts and their associated maintenance as well as
modification kit procurement and installation. Figure 1 depicts the
typical distribution of total ownership costs of DOD weapon systems
over a 30-year life cycle.
Figure 1: Nominal Life-Cycle Cost of Typical DOD Acquisition Program
with a 30-Year Service Life:
[See PDF for image]
[End of figure]
Figure 1 shows that the greater part of a weapon system‘s total
ownership cost is made up of its operating and support cost. While 72
percent of the life-cycle cost of a weapon system is realized only
after it is fielded, the decisions made during its acquisition--when
its performance requirements are being established and its design is
being matured--will dictate operating and support costs very early. In
fact, studies show that about 85 percent of the operating and support
costs of a weapon system will be determined as soon as requirements are
set, while less than 10 percent of the life-cycle cost have been spent.
By the time a product is ready for production, over 90 percent of the
operating and support costs have been determined, and about 28 percent
of the total life-cycle costs have been spent. Figure 2 illustrates
this phenomenon.
Figure 2: Percent of Operating and Support Costs Determined at Various
Points in the Acquisition Process:
[See PDF for image]
[End of figure]
Because so much of the eventual cost to support and maintain a weapon
system is decided very early, it makes sense that more attention should
be paid to supportability when the product‘s requirements are being set
and its design is being finalized. World-class commercial companies
that either use or develop high-performing products know this and set
requirements and designs accordingly.
Commercial Best Practices:
GAO has undertaken an extensive body of work that examines DOD‘s
acquisition issues from a different, more cross-cutting perspective--
one which draws upon the lessons learned from the best commercial
practices to see if they are applicable for DOD‘s acquisition
processes. Previous GAO best practices reports focused on what DOD
could do to control product development costs that represent about 28
percent of total ownership costs. This report will focus on best
practices for reducing the largest segment of total ownership costs--
the operating and support costs. The concepts discussed build on our
previous reports that looked at earlier phases of an acquisition,
including matching customer needs with available resources and
designing and manufacturing products within cost, schedule, and
performance goals. A complete list of best practices reports is at the
end of this report:
Leading commercial companies expect to obtain high-quality products
that meet their expectations in terms of performance, price, and
reliability. To ensure they make prudent buying decisions, they use a
structured product development process that ensures a high level of
knowledge exists about a product at key junctures during its
development. Such a knowledge-based process enables decision makers to
be reasonably certain that product quality, reliability, and timeliness
are assured.
* Knowledge point 1 occurs when a match is made between the customer‘s
needs and the available resources--technology, design, time, and
funding. Commercial companies use this knowledge to meet essential
product requirements, such as low operating and support costs. To
ensure that the knowledge is attained, private companies require the
product be demonstrated in its intended environment. In addition, the
product developer must complete a preliminary product design, using
systems engineering to balance customer desires with available
resources.
* Knowledge point 2 occurs when the product‘s design demonstrates its
ability to meet performance requirements. Program officials are
confident that the design is stable and will perform acceptably when at
least 90 percent of engineering drawings are complete. To obtain this
knowledge, commercial companies use simulations and testing to fully
understand how the product should be built.
* Knowledge point 3 occurs when the product can be manufactured within
cost, schedule, and quality targets and is reliable. Important
indicators of this are when critical manufacturing processes are in
control and consistently producing items within quality standards and
tolerances. Private companies demand these critical manufacturing
processes be in control because they could affect the product
reliability.
Objectives, Scope, and Methodology:
The Chairman and the Ranking Minority Member, Subcommittee on Readiness
and Management Support, Senate Committee on Armed Services, requested
that we examine best practices for reducing total ownership cost of
DOD‘s weapon systems. This report primarily covers the operating and
support cost portion of total ownership costs. Our overall objective
was to determine whether commercial best practices for reducing total
ownership costs, particularly operating and support costs, prior to and
during the acquisition of weapon systems offer opportunities to improve
outcomes in DOD‘s acquisitions and its efforts to manage and reduce
total ownership costs. Specifically, we (1) determined the practices,
processes, and metrics DOD uses to manage and control total ownership
costs of its major weapon systems; (2) determined the practices,
processes, and metrics commercial companies use to manage and control
total ownership cost; and (3) analyzed the extent to which
opportunities exist for DOD to apply best practices to reduce operating
and maintenance costs during product development.
We used case studies of the following six DOD weapons, chosen to
reflect all of the services across time, to examine DOD‘s practices,
processes and metrics:
The Advanced Amphibious Assault Vehicle: This vehicle is the U.S.
Marine Corps‘ replacement for its presently fielded amphibious assault
vehicle. The new development vehicle is equipped with a 30mm automatic
cannon and provides the capability to transport a Marine rifle squad at
a speed of 20-25 knots in the water, and cross country with the agility
and mobility equal to or greater than that of the M-1 tank. The
contract for the Product Definition/Risk Reduction Phase was awarded in
1996. The Marine Corps expects to buy 1,013 of these vehicles.
Acquisition costs total $9.6 billion; operating and support costs,
$16.0 billion.
The Joint Strike Fighter program: This aircraft is the next-generation
aircraft for the Navy, Air Force, Marine Corps, and Allies. There are
three variants: a carrier variant will provide the Navy a multi-role,
stealthy aircraft to complement the F/A-18E/F. The Air Force variant
will be a multi-role aircraft, but primarily used in an air-to-ground
role to replace the F-16 and the A-10 and complement the F-22. The
Marine Corps variant will be a short-takeoff and vertical-landing
aircraft to replace the Sea Harrier and GR-7 for the United Kingdom
Royal Navy and Royal Air Force. The program is currently in system
development and demonstration. Acquisition costs total $226.5 billion;
operating and support costs, $387.6 billion.
The Landing Platform Dock 17 ship program: These ships are used for
transporting amphibious assault vehicles and other cargo. They
incorporate both a flight deck for helicopters and a well deck to
support landing craft. The contract for the detail design, integration
and construction of the ship was awarded in 1996. The Navy expects to
buy 12 ships to replace 27 older amphibious ships. Acquisition costs
total $15.4 billion; operating and support costs, $56.5 billion.
The Apache helicopter program: This Army helicopter‘s mission is to
find tanks and other armored targets and destroy them with its laser-
guided Hellfire missiles, its 2.75-inch rockets, or its 30-millimeter
gun. Apache development began in 1973 and the helicopter entered
production in 1982 and was fielded in 1984. The Longbow Apache is a
remanufactured and upgraded version of the Apache, which includes
improved radar, engine and Hellfire missiles. The Army currently fields
741 Apache and Longbow Apache helicopters. Acquisition costs total $ 19
billion; operating and support costs for the Longbow Apache are $11.1
billion.[Footnote 4]
The C-17 cargo aircraft: The C-17 is a multi-engine, cargo aircraft
expected to improve Air Force capability to rapidly deploy, reinforce,
and sustain combat forces worldwide. The C-17 is capable of carrying
outsized cargo over extended distances into unimproved airfields. The
C-17 introduces a direct deployment capability that significantly
improves airlift responsiveness. Development began in 1982 and the
aircraft entered production in 1988 and was fielded in 1993. As of
December 2002, the Air Force fielded 96 of 180 aircraft. Acquisition
costs total $58.2 billion; operating and support costs, $144.9 billion.
The M-1 series Abrams tank program: This tank is the Army‘s main battle
tank. The M-1‘s development began in November 1972, entered production
in 1979, and was fielded in 1986. The M1A2 is an improved version of
the M-1, with improved armor, digital electronics, and an improved
commander‘s weapon station. The Army has fielded over 8,800 M-1 and
M1A1 tanks, with about 1,000 upgraded to the M1A2 versions. Acquisition
costs total $29 billion; operating and support costs, $16
billion.[Footnote 5]
For each of the six programs, we interviewed key managers and logistics
representatives to discuss how operating and support costs were being
managed and controlled during design. For the Joint Strike Fighter
program we also visited the prime contractor, Lockheed Martin Aircraft
Company, in Ft. Worth, Texas, and interviewed key designers and
engineers. We analyzed operating and support cost data contained in
Selected Acquisition Reports for the Apache Longbow helicopter, the
C-17 cargo aircraft, the Abrams tank, the Advanced Amphibious Assault
vehicle, Joint Strike Fighter and Landing Platform Dock 17. Information
obtained from the Selected Acquisition Reports was not always
consistent because three of the systems--the Advanced Amphibious
Assault Vehicle, Joint Strike Fighter and Landing Platform Dock 17--are
still in development. We supplemented information for the development
systems with other program cost estimates. We also conducted limited
analysis of the B-1, B-2, and F-22 operating and support cost and
readiness data, based on information provided by their respective
program offices and previous GAO reports.
In addition to the case studies, we reviewed DOD policy, describing the
roles and responsibilities of various organizations in the requirements
development process. We discussed the implementation of these policies,
particularly the role of the logistics community in the requirements
determination process, with officials from each of the six program
offices listed above, as well as appropriate officials from the:
Under Secretary of Defense (Acquisition, Technology, and Logistics);
Air Mobility Command, Scott Air Force Base, Ill.;
437th Airlift Wing, Charleston Air Force Base, S.C.;
Office of the Assistant Secretary of the Army, Washington, D.C.; and
U.S. Army Aviation Center, Ft. Rucker, Ala.
To determine the best practices, processes, and metrics commercial
companies use to manage and control operating and support costs, we
used a case study methodology by judgmentally selecting companies based
upon general literature searches and discussions with experts. On this
basis, we identified a number of commercial companies that have a
structured and defined process for managing and controlling their
operating and support costs. The following are descriptions of the six
commercial companies and one quasi-governmental agency we visited:
Boeing Commercial Aircraft designs and manufactures commercial
airplanes. In 2001, it reported revenues totaling $58.2 billion. We
visited its offices in Seattle, Washington, and discussed the
development of the 737, the 767, and the 777 aircraft.
Chicago Transit Authority is a quasi-governmental agency that operates
the nation‘s second largest public transportation system and covers the
City of Chicago and 38 surrounding suburbs. In 2002, it reported an
operating budget totaling $915 million. We visited its offices in
Chicago, Illinois, and discussed the requirements determination
process for acquiring buses.
Allison Transmission, a division of General Motors, designs and
manufactures transmissions for medium and large vehicles as well as
military vehicles. In 2001, General Motors reported sales and revenues
totaling $177.3 billion. We visited its offices in Indianapolis,
Indiana, and discussed the development process for new transmissions.
FedEx Express delivers packages, freight, and information to its
customers worldwide. In 2001, it reported sales and revenues totaling
$15.5 billion. We visited its offices in Memphis, Tennessee and
discussed their process for setting requirements for a new package
delivery vehicle.
Maytag designs and manufactures major home appliances. In 2001, it
reported sales and revenues totaling $4.1 billion. We visited its
offices in Newton, Iowa, and discussed the development process for new
appliances, particularly the Neptune washer and the Wide-By-Side
refrigerator.
Polar Tanker, a shipping division of ConocoPhillips Marine, manages the
marine transportation of ConocoPhillips‘ Alaska North Slope crude oil
production. In 2001, it reported sales and revenues totaling $26.9
billion. We visited its offices in Avondale, Louisiana, and discussed
the development process for the new Endeavor Class tanker, the Polar
Endeavor.
United Airlines, a commercial airline division for the UAL Corporation,
is a major commercial air transportation company, engaged in the
transportation of persons, property, and mail throughout the U.S. and
abroad. In 2001, it reported sales and revenues totaling $16.1 billion.
We visited its offices in Seattle, Washington; and San Francisco,
California; and discussed the product development process and
maintenance activities for the Boeing 777 aircraft.
At each of these organizations, we conducted structured interviews with
representatives to gather uniform and consistent information about
their processes, practices, and metrics for controlling operating and
support costs. During these meetings, we obtained a detailed
description of the practices and processes they believe are necessary
and vital to control and reduce operating and support costs. We met
with managers of reliability, maintainability, and new aircraft
development; general directors of operations; controllers; directors of
configuration and integration; and principal engineers. We did not use
examples from Chicago Transit or from Allison Transmissions, but
discussions with those firms helped to refine the commercial model and
inform our analysis of commercial best practices.
During the past 5 years, we have also gathered information on operating
and support costs from such companies as 3M, Chrysler Corporation,
Caterpillar, Bombardier Aerospace, Ford Motor Company, Hughes Space and
Communications, and Motorola Corporation. This information enabled us
to develop an overall model to describe the general practices,
processes, and metrics leading commercial companies take to control
operating and support costs.
This report highlights several best practices in controlling operating
and support costs based on our fieldwork. As such, they are not
intended to describe all practices or suggest that commercial companies
are without flaws. Representatives from the commercial companies
visited told us that the practices and processes, which we considered
best practices, evolved over many years and that the practices continue
to be improved based on lessons learned and new ideas and information.
They admit that the application and the use of these practices have not
always been consistent or without error. However, they strongly
suggested that the probability of success in controlling operating and
support costs is greatly enhanced by the use of these practices and
processes.
We conducted our review between August 2001 and February 2003 in
accordance with generally accepted government auditing standards.
[End of section]
Chapter 2: DOD‘S Requirements-Setting and Development Practices Yield
Higher Total Ownership Costs:
DOD is spending more on operating and support costs for its weapon
systems than it planned. We found three primary reasons for the high
cost of operating and supporting DOD‘s fielded weapon systems. These
were (1) little or no attention to the trade-offs between readiness
goals and the cost of achieving them when setting the key parameters
for weapon systems; (2) the use of immature technologies during product
development and delays in acquiring knowledge about the design and its
reliability until late in development, or in some cases, production;
and (3) insufficient data on the operations and maintenance costs and
actions for fielded systems that would allow improvements in products
currently in development. The outcome of these practices in DOD has
been an inability to stem continuous growth in total ownership cost,
with actual operating costs continuing to exceed initial estimates. As
a result, DOD continues to request more operating and support funding
to sustain its systems or reprogram funds from other accounts to pay
the bills.
Three key groups are involved in DOD‘s process to get a weapon system
to the war fighter. First, the war fighter‘s service-based requirements
community establishes requirements for a new system. Second, the
service-based acquisition organizations design and produce a product.
Finally, after the product is developed and produced, it is turned over
to the war fighter‘s operating and maintenance communities, who have
the responsibility to operate and maintain it. Decisions made in
setting requirements very early in product development have the most
impact on the subsequent costs of supporting a system. Trade-offs
during the design process can also be significant. In DOD, the focus in
the requirements and development process is to establish and meet
technical war-fighting performance capabilities, and when trade-offs
are made, they are usually to optimize those capabilities. The
maintainers often come into this process very late and have little
influence. At the end, DOD has no alternative but to pay the operating
and support bills that accrue in order to maintain readiness.
DOD‘s Weapon System Programs Encounter Cost Growth in Achieving
Readiness Rates:
A weapon system‘s operating and support cost will depend to a great
extent on its performance characteristics, expected readiness rate, and
the overall reliability of its design. If a weapon system has a very
high expected-readiness rate but its design is not reliable, its
operating and support costs will be high and unpredictable. Conversely,
if the design has been thoroughly tested for reliability and is robust,
the cost to operate and support it will be lower and more predictable.
Ideally, there is a balance that customers and product developers can
strike between readiness and operating cost. Figure 3 illustrates this
balance.
Figure 3: Readiness, Reliability, and Operating and Support Costs:
[See PDF for image]
[End of figure]
A product developer can opt to drive higher reliability into the
product during its development by reducing technical requirements,
using highly reliable and proven components, or investing more in early
testing. Those decisions would increase the product‘s reliability and
consequently improve prospects for readiness and reduce operating costs
across the life cycle of the product.
We reviewed five weapon system programs currently in the field and
found that most had experienced significant growth in their estimated
operating and support cost. We also found that none of the programs
established goals for readiness or operating and support cost as key
parameters for the weapon system prior to launching the acquisition
program. In addition, we found that once fielded, some systems were not
achieving the readiness rates that program officials thought were
possible during development. Table 1 shows whether systems had
specified readiness and operating and support cost goals as key
requirements as well as the growth in operating and support costs that
the systems have experienced within the last 12 years.
Table 1: Readiness and Operating and Support Costs for Selected
Weapons:
Weapon: Apache; System readiness
as a key parameter: No; Operating and support cost
goals as a key parameter: No; Readiness: planned/actual: 62/73 %;
Percentage growth for
operating and support cost: 48 %.
Weapon: Abrams; System readiness
as a key parameter: No; Operating and support cost
goals as a key parameter: No; Readiness: planned/actual: 90/93 ;
Percentage growth for
operating and support cost: 24 .
Weapon: C-17; System readiness
as a key parameter: No; Operating and support cost
goals as a key parameter: No; Readiness: planned/actual: 92/84 ;
Percentage growth for
operating and support cost: 25 .
Weapon: B-2; System readiness
as a key parameter: No; Operating and support cost
goals as a key parameter: No; Readiness: planned/actual: 70/42 ;
Percentage growth for
operating and support cost: 35 .
Weapon: B-1; System readiness
as a key parameter: No; Operating and support cost
goals as a key parameter: No; Readiness: planned/actual: 67/64 ;
Percentage growth for
operating and support cost: 16 .
Source: DOD (data), GAO (analysis).
[End of table]
While some systems have maintained their expected readiness rates,
they have experienced between 16 and 48 percent growth in estimated
operating and support cost. It is reasonable to conclude that the
systems have not achieved the reliability rates that were needed to
meet their expected readiness goals and, as a result, had to expend
more funds on parts and labor in the field than were planned for
maintenance. Two of the systems, the Apache and the Abrams, were
designated as the Army‘s most expensive weapons to support. The C-17
has already experienced a cost growth of 25 percent in its operating
costs, but program officials stated that they would not have a firm
estimate of operating and support costs until 2010--more than 25 years
after the start of development.
DOD‘s Linear Acquisition Approach Makes It Difficult to Control
Operations and Support Costs:
Traditionally, DOD has used a linear approach to setting requirements
and developing a product. It focuses attention during product
development on achieving revolutionary performance goals while trying
to keep acquisition costs for a program as low as possible. Often, it
is not until the system is fielded and responsibility shifts to other
agencies or the services that the operating and support costs become an
overriding concern. By this time, there is no alternative but to pay
the bills that accrue to maintain readiness, no matter the cost.
Three key groups are involved in DOD‘s process to get a weapon system
to the war fighter. First, requirements representatives from the war-
fighting community establish a need for a new system, and the combat
developers formulate a set of operating performance requirements to
address the need. Requirements concerning how available the system must
be and the cost to operate and support it are not considered key
performance parameters. Second, DOD establishes an acquisition program
office to begin product development and coordinate design development
and production with the defense contractor. Often, the technologies and
components needed to achieve tough performance requirements are new and
unreliable; however, the program manager is responsible for developing
and producing the weapon system within certain acquisition costs and
schedule guidelines. Third, after the product is developed and
produced, the war fighter has the responsibility to operate and
maintain it. Although decisions made in setting requirements very early
in product development have the most impact on the cost to support the
system, the personnel who maintain it have less influence on the
product development process because the focus is on achieving difficult
performance requirements.
Figure 4 briefly describes DOD‘s process for managing a new weapon
system‘s requirements development, acquisition, and fielding. The most
notable aspect of the model is that there is little communication or
input from the maintenance community early in the process--during
requirement setting and product development--when decisions will be
made that will significantly influence the cost to maintain and support
the weapon system. The model shows that, traditionally, DOD‘s processes
are separate and independent of each other. First, requirements are set
independently of the maintainers and the product developers. Second,
once the product development process begins, the focus is on maturing
technologies and achieving a producible design that will meet the
technical performance requirements for the weapon system. Finally, the
operator is tasked to use and maintain the system that has been
developed and produced.
Figure 4: DOD‘s Linear Acquisition Process:
[See PDF for image]
[End of figure]
The goal of this process is to field a high-performing weapon system
that will satisfy the needs of the war fighter better than any other
weapon system available. We previously reported that competition for
limited funding both within and among the services leads to performance
requirements that will make the particular weapon system stand out from
existing or alternative systems. [Footnote 6] Those that provide the
greatest leap forward in promised performance stand the best chance of
winning the funding. As a result, the design for the weapon system is
usually based on undemonstrated components and subsystems that, when
integrated into a weapon system, have low reliability and, ultimately,
high operating and maintenance costs over their lifetime. Figure 5
illustrates this phenomenon. The demonstrated reliability of the new
system is lower, causing an imbalance in the relationship between
readiness and operating cost toward the need for higher costs to
maintain readiness.
Figure 5: System Readiness Comes at High Operating and Support Costs
When Reliability Is Not Ensured:
[See PDF for image]
[End of figure]
Logistics officials at Air Mobility Command told us that even though
they are represented in the requirements determination process for new
weapon systems, they view their role as providing input on how the
logistics community could support performance requirements of a new
system, not on forcing trade-offs that would reduce operating and
support costs. The officials also said that the logistics community
leads many reliability improvement and cost reduction initiatives once
aircraft are fielded. However, this approach is harder and more costly
to implement than if reliability and operating and support costs had
been considered when requirements were set and product development
began. Further, the program must compete against other programs for
operating and support funds to implement the upgrades and improvement
initiatives.
Operating and Support Cost Estimates Are Provided Later in DOD‘s
Process:
Supportability aspects of system performance, such as operating the
system at the lowest possible cost and the percentage of time the
system has to be available for operations, are not given the same high
priority as enhanced performance. Operating and support costs are not
estimated until much later in development. For example, the B-2 bomber,
Apache helicopter, and Abrams tank programs did not publish an initial
estimate for the operating and support costs for those weapon systems
in the Selected Acquisition Reports until more than 15 years after the
start of development. The C-17 program did not publish an estimate
until more than 7 years after development began. These estimates were
not in the form of goals or key performance parameters. Throughout
product development, then, design goals for reliability are not
supported by the war fighters‘ need to meet operating cost and
availability goals.
Maturing the Technology Is the Focus of Development:
In DOD, requirements for new weapon systems are usually based on the
enhanced performance deemed necessary to achieve a certain war-fighting
mission with little hesitancy in using new technology or what the cost
to support it may be. The user representatives define system
performance with limited input from the product developers and
maintainers.
A case in point is the B-2 bomber. The low-observable requirement for
the B-2 bomber could only be met with technology that was immature at
program launch, and this technology continued to cause problems
throughout development, production, and fielding. The B-2 program began
full-scale development in 1981, and the Air Force began low-rate
initial production concurrently with development in November 1987. By
1991, problems with the B-2‘s low-observable material were still being
reported, causing delays in delivery and cost increases because the
material was difficult to manufacture. Once fielded, the low-observable
materials required very high maintenance. The processes to repair them
were time consuming and required an environmentally controlled repair
facility. Poor durability and extensive maintenance kept the aircraft
from achieving its planned availability. All of these factors are due
to decisions to proceed with product development without understanding
this technology.
A more recent example is the F-22 fighter. The requirements for the
F-22 were very demanding. Performance characteristics included low
observability, super cruise speed, and fused avionics. These
requirements caused the product design to include many new and unproven
technologies. During product development, the program planned to
achieve a system-level reliability rate for the F-22 of 3 flying hours
mean time between maintenance actions when fully mature. The Air Force
had estimated that in late 2001, when the F-22 entered limited
production, it should have been able to demonstrate almost 2 flying
hours between maintenance actions. However, when the F-22 actually
began limited production, it could only fly an average of .44 hours
between maintenance actions. In other words, the F-22 is requiring
significantly more maintenance actions than planned. To date, the
program has identified about 260 types of failures, such as main
landing gears wearing out more quickly than planned, fasteners being
damaged, and canopy delaminating, all leading to unanticipated
operating and support costs.
Product Development in DOD Goes Forward in Spite of Poor Prospects for
Reliability:
Our previous work identifying best practices during product development
concluded that during this phase, the tasks are to ensure the stability
of the design and to ensure that the product can be produced.[Footnote
7] In DOD, the product developer frequently is trying to catch up to
design and production tasks because product developments begin with
immature technology. Schedule concerns override the need to capture
knowledge about the design and production processes, and programs often
proceed through development and into production before the reliability
of the subsystems and systems has been demonstrated. Design features
such as open architectures that allow systems to receive upgrades as
technology advances or reductions to the number of parts in a design
that reduce the need for spare parts and maintenance time are not given
due consideration, even though they could lower operating and support
costs of the system.
As the schedule tightens, the lack of knowledge becomes more
acceptable, even preferred. Reliability testing is often pushed closer
to fielding, resulting in supportability problems being identified
during operational testing. The Office of the Director, Operational
Test and Evaluation has commented that operational testers identified
reliability as a problem in almost every program, because the product
developer places more emphasis on performance requirements than
suitability. According to the operational testers, many systems enter
operational test and evaluation with known, but unresolved reliability
problems.
Once a system is fielded, unless the reliability and ease of
maintenance have been incorporated into the design already, there are
limited opportunities to improve these metrics without costly redesign
or retrofit. As the product‘s design becomes firm and the system is
produced and fielded, the opportunities to influence these costs
diminish. For example, the Army is currently attempting to reduce
operating and support costs on its Apache helicopter and its Abrams
tank. These efforts should have a favorable impact on the systems‘
operating and support costs. However, the Army is retrofitting and
replacing components that were identified as problems much earlier in
the programs. Examples from the Apache and Abrams development
illustrate the problems.
Figure 6: Apache Helicopter:
[See PDF for image]
[End of figure]
The Apache program focused on acquisition costs, schedule, and
performance during product development, even when problems were
identified that would impact reliability and maintainability. Today,
the Apache helicopter is the Army‘s most costly system in terms of
operating and support costs with those costs rising over the years. For
example, the target acquisition and designation system enables the
Apache to find targets and guide its weapons. It is the helicopter‘s
most sophisticated system, involving 26 major electrical, optical and
mechanical components. However, the sight requires frequent
maintenance, and its complexity reduces its reliability. The pilot‘s
night vision system is also a highly complex system that experienced
problems in development. Because its target acquisition and designation
system and the pilot‘s night vision system proved unreliable, the
Apache incurred higher than expected costs to maintain the helicopter‘s
availability.
During development, problems identified with the Apache foreshadowed
future support problems. These problems included excessive vibration
and excessive aircraft weight in addition to difficulty in achieving
reliability rates for the aircraft‘s target acquisition and designation
system. During development, problems with the Apache‘s fault detection
system led to questions concerning whether it could operate safely and
reliably during operations. Further, Army test and evaluation agencies
warned that these problems could cause serious supportability issues
since they would result in frequent need for maintenance and repair. By
1990, the majority of the Apache helicopters had been produced and
fielded, but Apache could achieve only 50 percent availability rates,
well short of their 70 percent goal. Tests showed that the Apache
required maintenance actions to correct significant problems every 2.5
flying hours while the Army‘s goal was 4 hours between failures.
Maintenance units were physically unable to handle the repairs required
to keep the helicopter flying. Subsequently, the Army added 18
maintainers to the Apache battalion, constructed more contractor repair
facilities, and hired more contractors. During Operation Desert Storm,
although the system was effective in destroying tanks and other
targets, it continued to experience reliability and logistical support
problems that grounded some aircraft.
Figure 7: Abrams Tank:
[See PDF for image]
[End of figure]
Likewise, the Army produced the Abrams tank without first resolving
reliability problems. The tank provided a major improvement in speed,
agility and lethality over the older M-60 tank and was also supposed to
decrease the operating and support cost burden. However, during
development, the tank experienced serious failures with the track, the
engine, fuel filters, the fuel and water separator, and the fuel pump.
In particular, the durability and reliability of the Army‘s turbine
engine was a major concern. Tests performed up to the time of the
initial production decision showed that the tank generally met its
performance requirements. However, frequent breakdowns and component
failures raised serious questions about approving the tank‘s
production. In a 1993 operational test report from the Office of the
Director, Operational Test and Evaluation, the reviewer concluded that
the tank was operationally effective, but not operationally suitable
because of its many support problems such as those mentioned above.
Today, the Abrams is the second most costly Army system in terms of
operating and support costs and accounts for half the repair parts
costs by the entire ground combat fleet. The tank engine is a major
contributor to the tank‘s high support costs. In recent years, the
operating and support costs for the Abrams have remained steady at
about $2 billion per year, although the Army has reduced the number of
tanks to be supported by more than 300.
Data on Operating and Support Costs Is Not Reliable:
When estimating operating and support costs for a new weapon system or
trying to establish maintenance trends, maintenance data on the current
system are an important source of information. However, the three
services‘ operating and support data collection systems do not provide
accurate and reliable information about the cost to operate and
maintain the systems because they do not collect and maintain data on
all elements of the weapon system‘s operating and support cost. Without
reliable information, the services cannot assess trends or identify top
cost drivers and take corrective action.
The Army‘s Operating and Support Management Information System provides
historical cost data on Army weapon systems and is the primary source
of operating and support data used by the program managers to project
costs of new systems, forecast spare parts budgets, and generally
manage their programs. The Army uses this data system to develop its
operating and support cost budget for weapon systems for consumable
items such as repair parts, petroleum, oil, lubricant, fuel, and
ammunition as well as for some maintenance. However, the system does
not provide a complete and reliable basis for developing and reporting
the costs of weapon system support. The data are often incomplete and
12 to 18 months out-of-date. Several important cost elements used to
establish the Army‘s operating and support budget estimates--such as
contractor logistics support, supply maintenance and software support-
-are not included in the database.
The Navy‘s central tracking system for operating and support costs,
like the Army data collection system just described, provides
historical cost data on weapon systems. Navy program officials told us
that this data is often inaccurate and cannot be relied upon to
pinpoint causes of failure or maintenance actions.
Until recently, tracking Air Force operating and support costs was
difficult because the cost data were unavailable in a usable format or
were of poor quality. Poor cost data weakened operating and support
cost estimates. In 1998, the Air Force set up a total ownership cost
database, with the objective of providing accurate and reliable data to
identify cost drivers and support decision makers in making
improvements to fielded aircraft. However, the new system is not
available at all aircraft maintenance locations, and therefore data may
not be complete. In any case, the Air Force will need several years of
cost data before it can evaluate the effectiveness of the system.
[End of section]
Chapter 3: Commercial Companies Deliberately Manage Ownership Costs
through Product Requirements and Design:
Leading commercial companies consider the total ownership cost of a
new product, including its operating and support costs, integral to
their new product development decisions. Companies that use airplanes,
trucks, and ships to deliver goods and people such as United Airlines,
FedEx Express, and Polar Tanker understand the importance of
maintaining the readiness of their fleets at as low an operating cost
as possible. Reducing these costs results in increased revenues,
profits, and market growth. Increases in these costs, on the other
hand, can result in failure. The companies also understand that unless
reliability is designed into a product, there are limited opportunities
to improve readiness and reduce cost without costly retrofit or
redesign. They have been successful in reducing these costs because
they developed a collaborative process with companies that develop
those products, such as Boeing, for setting the product‘s requirements,
developing the product with operating and support costs in mind, and
capturing accurate operations and support data once it is delivered.
The companies we visited that bought products all established the
product‘s operating cost and its availability as requirements equal to
its performance characteristics prior to product development, thereby
ensuring that the developer placed priority on those goals during
product development. They were amenable to reducing the product‘s
performance features to reduce its operating cost, as long as
performance was within acceptable limits for achieving market
objectives. They also considered bearing additional cost for the
product‘s design if it resulted in a net benefit from reduced operating
costs. Once product development began, maintainers had a strong voice
in the product‘s design and leading product developers set high
reliability standards for components they chose, using proven
technologies to achieve performance requirements. Companies, such as
Boeing and Maytag, chose an evolutionary approach to product
development to achieve goals for life-cycle costs, testing extensively
and early for reliability. These companies also emphasized other
practices to reduce operating costs such as reducing the number of
parts in a product‘s design, using standardized parts, and using open
systems to ease maintenance and maintain competition. Once the product
is delivered, maintainers keep detailed records on its reliability and
maintenance and provide that information to the developer to improve
future products.
A Best Practices Model:
Figure 8 represents a model of the best practices that were most
helpful to the companies we visited in achieving high reliability and
reducing a product‘s operating and support costs. Notably, the most
critical events--those that have the most impact on a product‘s
operating and support costs--take place very early, either prior to
product development when the product‘s key requirements are
established, or very early in product development before the design is
finalized. Another notable aspect of the model is that each of the
activities--requirements-setting, product development, and operations-
-depend on clear and constant communication and collaboration among the
customer, the product developer, and the maintainer from the time a
product is conceived until the operator disposes of it.
Figure 8: Commercial Model for Reducing Operating and Support Costs:
[See PDF for image]
[End of figure]
The goal of this process is to develop and field a product that will
perform in accordance with the customer‘s needs and will be ready when
needed within cost targets. As illustrated above, decisions about the
product‘s performance and cost are finalized prior to beginning the
product‘s development, and the costs to operate and support the product
are key considerations. Once the requirements are set, the product
developer can focus as much on achieving a reliable and robust design
for the product as on achieving its performance goals. Finally, once
the product is fielded, those responsible for its operation and
maintenance continue to feed information back to the developer to
improve future designs. Figure 9 illustrates what happens when the
commercial firms we visited set requirements that force the product
developer to consider reliability rates during design. When both
operating cost and readiness are key requirements for the product
developer, the developer focuses on using as many components and
subsystems as possible that have demonstrated reliability rates. The
customer, by demanding readiness at a certain cost as a hard
requirement upfront has raised the importance of achieving it from the
very start.
Figure 9: Benefits of Ensuring High Reliability Rates During Product
Development:
[See PDF for image]
[End of figure]
Leading Companies Treat Readiness and Operating and Support Cost as
Critical Product Requirements:
We visited three companies--Polar Tanker, an operator of large oil
tankers; United Airlines; and FedEx Express--to determine their
practices for ensuring low operating and support costs from the
equipment they purchased from product developers. All three companies
believe that understanding and controlling the cost to operate and
support a product while it is being designed is essential to driving
down the total ownership cost of a product. To do this, they set
requirements for a new product‘s availability and its operating and
support costs equal in importance to requirements for its performance
and acquisition cost. We found these companies set requirements before
development begins that their products be ready almost 100 percent of
the time at the lowest operating cost possible. They typically set
maintenance goals that drive operating and support cost decisions, such
as maintenance cost per mile over a product‘s lifetime. The following
summaries illustrate the commercial processes.
Figure 10: Polar Tanker‘s Polar Endeavor:
[See PDF for image]
[End of figure]
Polar Tanker:
Polar Tanker is a commercial oil-transporting firm that recently
decided that a new oil tanker was necessary to haul oil between Prince
William Sound and Puget Sound. The company‘s critical requirements for
the new Endeavor tanker, which Polar Tanker believed would reduce the
cost of delivering oil, were:
* less expensive operations and maintenance over a 30-year life cycle
(versus the industry standard of a 20-year life cycle); and:
* availability for operations at least 330 days a year.
Polar Tanker teamed its maintenance engineers with industry consultants
to ensure these requirements were met. The procurement team relied on
its archived maintenance data from previous Alaskan operations to
develop its double-hulled tankers. It documented locations, lengths and
types of fractures, and stresses in the structures of its existing
inventory of ships. This documentation from past operations was used to
determine structural changes required to reduce maintenance on the new
Endeavor class. As a result of their record keeping and the constant
communication with the product developer, Northrop Grumman‘s Avondale
Shipyard, Polar Tanker‘s owners were successful in redesigning their
ships and meeting their requirements for improved performance,
reliability, and lower operations and maintenance costs.
The dual requirement of reduced operating and support cost coupled with
high readiness rates drove design trades that increased development
costs but improved reliability. For example, once Polar Tanker‘s
procurement team identified ballast tank maintenance as one of the most
significant maintenance burdens, it directed Northrop Grumman to use
the best and most expensive epoxy coatings and specialized paints to
protect the tanks from corrosion. Another design trade--adding
additional structure to the ship‘s hull to reduce the impacts of
fatigue cracking--increased the acquisition cost of the tanker but
improved reliability and reduced the need for maintenance. Polar Tanker
accumulated data from its current fleet and conducted extensive
modeling of the hull design to understand where the most critical
cracking occurred and to identify operating and support cost drivers.
It hired an engineering consulting company to conduct further analysis.
As a result, Polar Tanker and Northrop Grumman utilized the most
current design tools to optimize the ship‘s structure. Polar Tanker
also developed a list of equipment and suppliers based on reliability
analysis and incorporated that list into the design contract with
Northrop Grumman. Polar Tanker estimated that these design trades cost
about $25 million in additional design costs, but they believe the
changes will ensure its tankers will be able to operate more reliably
over 30 years.
Polar Tanker‘s procurement team also required that the new Endeavor
Class design use open systems when possible. It worked closely with
Northrop Grumman, its contractor, to ensure requirements for an
integrated bridge system that consisted of commercial-off-the shelf
components with open systems to provide the capability to modernize the
system much less expensively as technology improved. For example,
systems in the tanker‘s wheelhouse including the autopilot, marine
radars, bridge control console, and satellite communication equipment
used the open systems concept. The design also included easily
accessible decks to minimize delays and disruption during maintenance.
Polar Tanker has already upgraded its electronics since putting its
first ship, the Polar Endeavor, into service and experienced minimal
disruption in operations.
Figure 11: United Airlines/Boeing 777:
[See PDF for image]
[End of figure]
United Airlines:
As the launch customer for the new Boeing 777, United Airlines
established stringent requirements for aircraft readiness and operating
costs, thereby ensuring that reliability would be an important design
element. When United and Boeing negotiated the requirements, United
stated that it wanted a twin-engine airplane that could fly extended
ranges from any airport in the United States. In addition, United
required that the 777 be available at the gate for departure within 15
minutes of scheduled departure 98.5 percent of the time. Boeing
guaranteed United that the 777 would meet the departure requirement by
the third year of operation or Boeing would pay a financial penalty.
According to United officials, the 98.5 percent rate was achieved by
the third year. United also specified that operating and support costs
for the 777 be no higher than on past airplanes. The agreement reached
with Boeing was that Boeing would reimburse United for revenues lost as
a result of airplanes being unavailable. By setting requirements for
operating cost and readiness, United ensured that Boeing would build
reliability into the design of the 777.
Boeing brought together a working group of customers--the leading
commercial airlines--to discuss requirements for the new design. Boeing
officials told us that during those requirements meetings the
participating airlines defined major design initiatives for the Boeing
777 based on estimates of life-cycle costs. Initially there were
differences among the airlines as to what exactly was needed on the new
aircraft--from wider fuselages to additional electronics--but from the
very first meeting, the airlines were all equally concerned with
operating and maintenance costs. They were focused on designing an
aircraft that would be easy for mechanics to repair. The airlines
unanimously agreed that an airline maintenance representative was
needed to adequately address operations and maintenance requirements.
Boeing named a chief mechanic who had previously worked for United to
the working group that was influential in defining the maintenance
requirements. Although Boeing provided its customers engineering
estimates for the operating cost of its new aircraft in comparison to
the older model, United officials said they developed their own
estimates based on its historical experience with Boeing aircraft.
United officials said that having two perspectives from which to
consider its purchase was helpful.
Figure 12: FedEx Express Delivery Van:
[See PDF for image]
[End of figure]
FedEx Express:
The FedEx Express mission is to provide global air and ground
transportation of high-priority goods and documents that require rapid,
time-certain delivery. This mission demands high availability and
reliability from its delivery equipment. In purchasing its newly
designed fleet of delivery trucks, FedEx Express considered
reliability, maintainability, and low operating and support costs to be
critical measures of a successful acquisition. For example, improving
availability and reliability were the key drivers in its acquisition of
the new 700 cubic foot truck. FedEx Express collaborated with a product
developer, Freightliner, to set the requirements for a newly designed
vehicle with high reliability and endurance to withstand frequent
stops, short travel distances between stops, and demanding use of the
brakes. During the requirements-setting process, they also established
cost and reliability requirements for the new truck that estimated an
assumed number of stops per day, a certain number of miles per year at
an assumed cost per mile.
To make sure all costs, particularly operating and support costs were
considered during product development, FedEx Express had a logistics
manager lead their discussions with Freightliner for the development of
the new truck. The FedEx Express logistics managers told us that they
are required to reduce the company‘s operating and support costs, and
the company believes that it is essential to give these managers an
integral role during these discussions. The new delivery truck has been
successful in meeting its reliability and maintenance goals. The truck
currently averages 70,000 miles between breakdowns and is operating
within the cost per mile of service that was set as a requirement by
FedEx Express at the beginning of development.
Knowledge-Based Product Development Is Critical to Achieving Desired
Reliability and Managing Operating and Support Costs:
To meet the readiness and operating and support requirements that their
customers demanded, we found that product developers focused on
designing a product that was durable, easy to maintain, ready when
needed, and reliable at low cost. They used an evolutionary development
process to meet these requirements and did not allow components or
subsystems into a product‘s design unless their reliability had been
proven through past use or testing. Boeing told us that it makes the
reliability ratings of its components available to the airlines before
it begins product development, and Maytag did extensive reliability
testing on every new product prior to going to production. The
companies also emphasized product designs with fewer parts and used
open systems architectures as much as possible. Developers also gained
valuable insight into design features their customers valued that drove
down operating and support costs. For example, the design for Boeing‘s
latest generation of the 737--geared toward reducing operating costs--
was inspired by the airlines‘ request for more affordable operations
and maintenance.
Once defined, the functional and operating and support requirements are
tightly managed and controlled to minimize scope increase during
product development. Companies work within a common framework to
provide management oversight and control. To move a product
successfully from concept to operations, the companies we visited used
a ’gated“ product development process and firm criteria to dictate when
a product is ready to exit a stage. The design reviews address all
operating and support cost requirements of the products. Senior
management review teams grant product approval at each gate only after
determining that business cases adequately address major drivers of
operating and support cost as well as reliability goals.
Boeing:
Boeing uses a structured, gated product development process to define,
evaluate, and approve projects and to integrate new technology into its
aircraft. This process separates technology development from product
development programs. In fact, Boeing keeps immature or untested
technologies in a research and development phase until they have been
tested for reliability in a realistic environment. The process forces
the company to obtain purchase agreements from customers and build a
business case that shows the expected profitability of a product line
before detailed designs are developed and a large dollar investment is
made in manufacturing. In order to get these purchase agreements,
Boeing product teams work collaboratively with potential customers to
set requirements for readiness, reliability, performance, acquisition
costs, and operating and support costs. Because aircraft are large
capital investments, customers are working with other developers to
make sure that the competitive forces of the market will help them get
the best aircraft at the lowest cost.
Boeing officials told us that to remain competitive in this
environment, they focus on meeting customer requirements at the highest
reliability leading to the lowest operating and support cost possible.
For example, when it was designing the new 737, Boeing used maintenance
records to prove to the airline that they could redesign the 737 with
high reliability and reduced operating and support costs, a key market
requirement for that aircraft. According to Boeing officials, they
prepared a comparative analysis of the acquisition and operating and
support costs for older 737 aircraft with the estimated costs for the
737 Next Generation. This analysis showed that operating and support
costs were significantly lower for the proposed next generation 737 and
resulted in big savings to the customer. Boeing was able to develop
good operating and support cost estimates for potential customers
because it used an evolutionary approach for developing new aircraft.
This approach allowed Boeing to improve performance and to insert new,
reliable, and mature technology.
Once a new product line is approved, Boeing continues to work with its
customers to identify improvements that can be made to an aircraft in
terms of parts reduction, parts standardization, and ease of
maintenance. For example, Boeing formed four airline working groups
with representatives from 21 airlines around the world to focus on
maintainability, interiors, power plant, auxiliary power units, and
common display system issues on the 737. Their objective was to collect
feedback from operators on design changes that would reduce maintenance
costs. Examples of airline-driven design changes include:
* simplifying the wing flaps by eliminating one third of the parts,
designing a simpler flap mechanism, and making parts removable with
common grip length fasteners;
* reducing engine removal and installation time by increasing the on-
wing life of the engine by 40 percent and reducing the predicted change
time from 12 hours to 6 hours; and:
* improving the reliability and standardizing parts of the fuel system
by using a fuel shutoff valve that is common with other aircraft.
To facilitate and improve communication with its customers, Boeing
oftentimes collocates customer representatives at its production
facilities. Boeing has found that involving the customer in early
design decisions improves their ability to design a reliable aircraft
that is easy and less costly to maintain. For example, when it began
developing the 777 aircraft, airline maintenance workers offered over
5,000 suggestions for changes to the design based upon their
experiences with other Boeing aircraft. These suggestions helped reduce
parts and improve reliability of the 777, resulting in increased time
between maintenance actions. Boeing also utilized the concept of open
systems to reduce the total ownership cost of the 777 by allowing
customers to choose from three different types of engines--GE, Pratt &
Whitney, or Rolls Royce--depending on their needs.
Maytag:
Maytag follows a similar product development process. For Maytag, the
most critical phase is when design specifications for product
reliability and manufacturing feasibility are fully defined and
understood. Early in its product development, an integrated product
team takes all new product features through rigorous reliability growth
analysis to determine what can be expected from the design and whether
it will be able to meet the requirements. Maytag uses failure analyses,
mock-ups, and other simulation tools to focus testing on the most
critical elements of the new design and reduce the number of design/
build/test iterations of the product. Even though these analyses
require more upfront planning, Maytag officials stated that they still
cut testing times in half and yet improved reliability and lower costs.
During this phase, Maytag employs two project leaders, both a
marketing manager and technical systems engineer, to conduct cost
performance trades and co-chair subsystem design reviews. As a result
of this co-leadership, most design issues are resolved immediately, and
80 percent of all reliability testing and cost reductions take place
early in the process. Also, during this phase the project leaders
present their strongest business case, fully disclosing feasibility
data, product definitions, and estimates of life-cycle costs for team
approval. The team uses this information to make the critical product
development decision to commit further to product development or kill a
product idea. Maytag officials stated that their company has a quality
image to uphold and that consumers expect the highest reliability and
quality from their products.
Maytag officials were also conservative in their use of new
technologies during product development. Decisions to incorporate new
technology were made in the earliest phases of product development and
were based on assessments of the adaptability and maturity of the
technology and associated risks to achieving established reliability
targets. Even though its Neptune washer incorporated over 90 percent
new technologies in its development, Maytag officials stated that they
spent time with suppliers and developers, maturing technologies to an
acceptable level of reliability before launching the new development.
Leading Commercial Firms Use Feedback from Operations to Better
Understand Customer Needs, Product Deficiencies, and Operating and
Support Costs:
The collection and analysis of the operating and support costs for
delivered products was considered essential by leading commercial
companies. Once a product is fielded, leading companies track actual
operating cost, reliability of parts, and readiness of the product
against what was estimated during product development to make sure the
company is getting what it paid for. We found companies are always
identifying the top drivers of operating and support cost and working
with the manufacturers to reduce these costs. When there are part
failures, the companies can quickly identify whether or not they are
under warranty and get the part replaced.
All of the companies we visited, customers and product developers
alike, had very collaborative processes and practices for drawing
extensively on data from current and past operations to improve the
reliability of existing products or influence the design of new
products. United Airlines officials told us that the airlines and
Boeing both keep meticulous reliability and cost records at all levels
of the 777: components, subsystems, and the system level. Major
operating costs drivers are tracked on a daily basis by the airlines.
The manufacturers usually have personnel residing with the airlines‘
maintenance crews to help solve problems on the spot and, just as
importantly, feed information back to the manufacturer so that the next
product can be improved. FedEx Express and Polar Tanker both emphasized
extensive collection of data from current operations. In fact, FedEx
Express sets annual targets for operating and support cost reductions
based on reliability data gathered on the road. Polar Tanker gathered
maintenance data from past operations to determine areas that could
result in higher reliability and lower maintenance costs in designing
the new tanker.
United conducts quarterly meetings for each of its fleets to discuss
open issues and short-term and long-term solutions to current problems
with operational aircraft. Attendees at the meeting include maintenance
mangers, financial representatives, representatives of the
manufacturers, and executives with authority to resolve issues.
United‘s practice is to resolve problems as expeditiously as possible,
no matter how small. United also monitors flight movements on a real
time basis through a computer system that tracks each aircraft by tail
number. The monitoring system provides the maintenance history of the
aircraft, reports problems on a current flight that require maintenance
upon landing, and alerts for other required maintenance based upon the
number of flying hours on the aircraft. If a specific part is broken,
the system also indicates whether or not it is under warranty. United
archives information on parts that break, when they break, and whether
they are still under warranties.
Manufacturers like Boeing and other major suppliers like this type of
feedback because it provides useful information to them on how to
improve the product for future iterations. They also believe that quick
responses to customer problems will help them get repeat business. They
use this feedback to develop preventive maintenance schedules, better
estimate operating and support costs, and refine reliability
requirements to be used in preparing budgets and cost estimates for
future products. Feedback mechanisms also accumulated operations and
maintenance data and ’lessons learned“ that highlighted reliability
problems and other maintenance issues. United officials stated that
taking lessons learned from data gathered from current products is an
effective tool for improving product reliability and maintainability or
developing requirements for new products.
FedEx Express takes any failure to meet its on-time delivery goal very
seriously and holds daily failure analysis meetings every morning to
analyze and review each delivery failure from the night before. This
constant feedback allows them to take immediate corrective action on
individual vehicles and to identify trends that may result in larger
problems and costlier maintenance requirements. FedEx Express
maintained a metric for vehicle miles between road calls and
maintenance costs per asset. In addition to tracking these costs per
asset, FedEx Express has established a performance goal with its
managers to reduce the overall costs of maintenance each year. FedEx
Express managers set their annual targets for reductions in operating
and support cost based on reliability data gathered on the road.
[End of section]
Chapter 4: Stressing Operating and Support Cost at the Outset of an
Acquisition Could Help DOD Reduce Total Ownership Costs:
DOD and the commercial companies we visited have policy goals of
developing products that will meet customers‘ needs at the lowest
possible cost to build and operate. The difference between them is in
how each implements its policies. Leading commercial companies follow
an integrated, collaborative process of setting requirements,
developing the product, and ensuring that the product can be supported
at an acceptable cost. DOD‘s process is composed of disparate practices
carried out by separate organizations with differing objectives and
little communication between them about how to support fielded systems.
While commercial firms focus on total ownership costs at the outset,
DOD focuses mostly on technical performance. One cause of this is that
in DOD the accountability and responsibility for total ownership costs
are spread across many organizations with separate goals. Another cause
lies in motivation for low costs. The commercial companies we visited
are driven by the need to be as profitable as possible to survive, and
low total ownership costs translate to higher profitability. DOD‘s
environment does not provide such incentives. The organizations charged
with acquiring and operating weapon systems are unconstrained by a need
to lower costs since they can request additional operations and
maintenance funding to keep systems working.
From time to time, DOD stated the need to lower its total ownership
costs in policy documents and in annual budget statements; however, it
has not been successful because it does not have an environment that
demands collaboration and accountability in setting requirements and
developing products with operating costs in mind. DOD has some efforts
underway to improve. First, it has rewritten its acquisition and
requirements generation policies. Second, a few programs now in
development established an early estimate of operating and support
costs and are working to gain knowledge of the impacts of requirements
and design on those costs. Third, information from pilots on fielded
systems, if disseminated throughout the acquisition community, could be
used to lower costs. Results are pending.
What DOD‘s efforts do not do is provide incentives to make investments
for more reliable, less-costly-to-maintain systems at the beginning of
an acquisition. Instead, DOD provides incentives to field systems with
unknown reliability by allowing whatever funding necessary to operate
and maintain the systems once they are fielded.
Differences in Practices Explain Different Outcomes for Commercial
Companies and DOD in Controlling Total Ownership Costs:
In Chapter 3, we discussed our findings that leading commercial
companies set specific requirements for readiness and operating and
support costs prior to initiating product development that forced
developers to design products with a high degree of reliability. In
Chapter 2, we noted that DOD, on the other hand, typically focused its
requirements on revolutionary performance that often forced developers
to mature technologies at the same time they were completing detailed
design work. As a consequence, system reliability often suffered,
forcing the department to spend a great deal of money to maintain and
repair fielded systems in an effort to achieve desired readiness
levels. The following table provides a comparison of the specific
practices used by commercial companies we visited and DOD programs we
reviewed to address operating and support costs early in a new
product‘s life cycle.
Table 2: DOD and Commercial Practices for Controlling Operating and
Support Costs:
Commercial prevailing practice: Practices used to set initial product
requirements.
Commercial prevailing practice: Operating and support cost goals as a
key requirement.; DOD prevailing practice: Practices used to set
initial product requirements: Operating and support cost goals are not
established as key parameters..
Commercial prevailing practice: Readiness a key requirement.; DOD
prevailing practice: Practices used to set initial product
requirements: Readiness is not a key parameter..
Commercial prevailing practice: Trade performance for reduced operating
and support costs, if appropriate; sometimes results in increased
costs.; DOD prevailing practice: Practices used to set initial product
requirements: Technical performance is sometimes traded using cost as
an independent variable, but cost is usually production cost or
development cost, and the trades occur during the design phase..
Commercial prevailing practice: Direct relationship during
requirements-setting between the user and the product developer.; DOD
prevailing practice: Practices used to set initial product
requirements: User and product developer separated by user
representative and government program office..
Commercial prevailing practice: Practices used during product
development.
Commercial prevailing practice: Provide detailed operating and support
cost estimates early in product development.; DOD prevailing practice:
Practices used to set initial product requirements: Operating and
support cost estimates not required until product development launch..
Commercial prevailing practice: User and developer focus on ways to
reduce product parts and standardize parts across product lines.; DOD
prevailing practice: Practices used to set initial product
requirements: Product developer has responsibility of focusing on ways
to reduce parts counts or use standardized parts with little input from
the user (operators or maintainers)..
Commercial prevailing practice: Use open systems architecture approach
to improve the cost effectiveness and installation efficiency of future
upgrades to the product.; DOD prevailing practice: Practices used to
set initial product requirements: Open systems approach is mandated but
implementation is limited..
Commercial prevailing practice: Set realistic reliability growth goals
for the product.; DOD prevailing practice: Practices used to set
initial product requirements: Reliability goals set, but they are
tradable or not met..
Commercial prevailing practice: Conduct reliability testing early.; DOD
prevailing practice: Practices used to set initial product
requirements: Reliability testing sporadically performed..
Commercial prevailing practice: Practices used during operations.
Commercial prevailing practice: Collect and analyze operations and
support data.; DOD prevailing practice: Practices used to set initial
product requirements: Data is often incomplete or unreliable..
Commercial prevailing practice: Manage operations and support costs to
targets.; DOD prevailing practice: Practices used to set initial
product requirements: Do not manage to operations and support targets..
Commercial prevailing practice: Identify areas for continuous
improvement.; DOD prevailing practice: Practices used to set initial
product requirements: Lack of complete and reliable data makes
identifying areas for improvement difficult; some areas that are
identified are not funded for improvement..
Commercial prevailing practice: Feedback to developer on product
performance.; DOD prevailing practice: Practices used to set initial
product requirements: Limited feedback to the developer. The maintainer
does not have a direct relationship with the product developer..
[End of table]
Source: GAO.
Clearly, the practices used by the commercial companies we visited
before product development when requirements are set, early in product
development when the design is finalized, and during the new product‘s
operating life focus as much on providing a reliable product as on
providing a high-performance product. The companies make operating cost
and readiness key requirements, they perform extensive reliability
testing, and they aim toward continuous improvement once the product is
in the field. In DOD, because performance is the overriding concern of
the requirement setters, none of these practices are in place.
Several DOD Efforts Underway to Reduce Total Ownership Costs:
The changes to policies and the investment in improving systems‘
reliability are encouraging indicators that DOD has focused its
attention on reducing costs to support weapons. DOD has revised
acquisition policies, tested new approaches for reducing costs in a few
systems, explored differing approaches, and created initiatives to
reduce costs of legacy systems. Each of these efforts had some initial
success, but most are aimed at reducing costs after fielding when over
90 percent of the costs have been determined.
DOD‘s Requirements Generation and Acquisition Policies Could Be More
Specific in Addressing Supportability Issues:
DOD has revised its 5000 series acquisition policies several times over
the past 10 years with the intent of defining an acquisition
environment that makes DOD a smart and responsive buyer. During this
time, the policy has not substantively changed with regard to how
acquisition programs can best control total ownership costs. The
department is striving for an integrated acquisition and logistics
process that is characterized by, among other things, a stronger focus
on using supportability as a key design and performance factor.
However, rules for total ownership cost goals at the outset of an
acquisition program are defined by the Chairman of the Joint Chiefs of
Staff‘s Instruction 3170.01B[Footnote 8] on requirements generation.
This guidance states that cost should be addressed in the operational
requirements document for a new weapon system, if an estimate is
available at that time. However, policy does not require the services
to set requirements for operating and support costs or readiness.
Instead, it allows them to identify system capabilities or
characteristics they consider essential for successfully completing the
mission. It states that the DOD sponsor may make cost a key requirement
if it desires and identify the cost it wishes to evaluate.
We previously reported that DOD officials believe they must promise
new, revolutionary weapon systems with significantly better performance
capabilities than the ones they are replacing in order to obtain
funding.[Footnote 9] Therefore, key parameters are usually focused on
performance rather than supportability.
In order to effectively minimize total ownership costs of its systems,
the Department of the Navy recently issued its own guidance that
establishes specific supportability and affordability thresholds and
objectives[Footnote 10] for all requirements documents. The Navy
believes that by establishing readiness and operating and support cost
as required parameters, there is assurance that major drivers of total
ownership costs will be addressed and minimized throughout the
acquisition process. Specifically, the new guidance states that
requirements documents must include goals for operating and support
costs. It also states that operational availability be included as a
key performance parameter, except when logistics delays are not an
issue or if the requirements are for a major aircraft or ship platform.
In those cases, mission capable rates or full mission capable rates
focused on the platform‘s primary mission areas will be used as key
requirements. The Army is discussing a similar change in its guidance.
Three New Acquisition Programs Are Placing Greater Emphasis on
Readiness and Operating Cost Goals:
We found three DOD programs still in development--the Joint Strike
Fighter, the Advanced Amphibious Assault Vehicle, and the Landing
Platform Dock 17--that appear to be using good practices to reduce
operating and support costs during product development. Each, in its
own way, has had a powerful internal incentive to establish more
collaborative practices or to focus attention on operating and support
costs and product reliability.
Figure 13: Joint Strike Fighter:
[See PDF for image]
[End of figure]
The Joint Strike Fighter program is intended to produce an affordable
next-generation aircraft to replace DOD‘s aging aircraft inventory. The
program is structured to use a common production line to produce three
aircraft variants that meet conventional flight requirements for the
Air Force, short take-off and vertical landing characteristics for the
Marine Corps, and carrier operation suitability needs for the Navy. The
program will also provide aircraft to the British Royal Navy and Air
Force. A key objective of the acquisition strategy is affordability--
reducing the development, production, and operating costs of the
program relative to prior fighter aircraft it will replace. The
program‘s latest stated estimate for operating and support cost savings
compared to legacy systems is $135 billion, or a 56 percent reduction
in cost.
To achieve this affordability objective, the program office has
incorporated various DOD and commercial initiatives into the
acquisition strategy. For example, two key provisions in its
operational requirements document--mission reliability and logistics
footprint--will have a direct impact on operating and support costs.
Specifically, all variants of the fighter are expected to achieve a
mission reliability rate of over 90 percent and meet numeric goals of
cargo aircraft or ship space needed to support a 30-day self-sustained
deployment. These two requirements, along with other reliability and
maintainability goals, demonstrate DOD‘s desire to reduce total
ownership costs. The product developer currently estimates the Joint
Strike Fighter will be able to reduce operating and support cost
primarily through efforts to improve:
* reliability and durability of materials,
* accessibility of parts or systems that need to be inspected or
replaced,
* supportability of low observable materials,
* ability of on-board systems to predict impending flight critical
failures, and:
* training materials and systems.
For example, DOD expects to save about $39 billion over the life of the
fighter through reduced maintenance on low observable materials. The
developer estimates that 99 percent of the maintenance actions will
require no low observable restoration because they are using high
durability materials, parts, or systems that are easier to access and
harder to damage. In order to reach this level of savings, the
developer spent a great deal of time evaluating previous DOD
maintenance experience with the B-2A bomber and the F-117 fighter
aircraft and used an evolutionary approach for upgrading these
materials. Operating and support costs for the B-2A bomber, for
example, were significantly increased by the decision to use an
immature technology for low observability.
However, the Joint Strike Fighter program must be careful not to
overestimate the total ownership cost savings it can achieve over
legacy systems it will be replacing because the program is also
depending on new technology for on-board systems to predict failure--
prognostics and health management technology--that is not yet ready for
product development. In October 2001, GAO reported that this technology
was not at an acceptable readiness level for inclusion in product
development, but DOD and the contractor decided to include it in order
to meet total ownership cost objectives.[Footnote 11] Program officials
stated that about $16 billion--12 percent of the estimated $135 billion
in total ownership cost savings--is expected to come from that
technology. Since then, the officials have allowed for the possibility
that the technology may not be included on initial production lots for
the Joint Strike Fighter if it is not ready.
Figure 14: Advanced Amphibious Assault Vehicle:
[See PDF for image]
[End of figure]
The Advanced Amphibious Assault Vehicle is a Marine Corps program to
improve its amphibious landing vehicle. The new development promises
faster sea and land speeds, better protection, and more lethality. The
development program has focused on maturing technology and paying
attention to operating and support costs early in development. The
program has used some of the best practices of commercial companies
during development. Some of those include collocating the program
office at the contractor‘s facility and making extensive use of Marine
Corps war fighters and maintainers to provide a ’hands on“ assessment
of how effective the vehicle would be in operations as well as how
supportable it would be during operations. The Marines developed an
early estimate of total ownership costs and included a reliability
metric as one of its key requirements. The Marines developed three
vehicle prototypes to mature the design and have conducted extensive
reliability testing. The vehicle will have parts that are common with
other weapons such as a gun that will be common with the Landing
Platform Dock 17. Advanced Amphibious Assault Vehicle officials
estimated that they will save $29 million in operating and support
costs.
The Navy program office for the Landing Platform Dock 17 has adopted a
total ownership cost approach. The program office established a process
for suggesting and evaluating design trades that could reduce operating
and support costs. Some of the design changes that the Navy made
include enclosing the mast to reduce exposure to weather and salt
water, and investing in high performance covering for the deck and well
deck to mitigate corrosion. Other practices include involving users to
complete tasks using the virtual software to test special design
elements in the ship, making greater use of sensors and automated
processes to reduce maintenance and to reduce crew requirements.
Both the Landing Platform Dock -17‘s and the Advanced Amphibious
Assault Vehicle‘s estimate for operating and support costs have
recently increased. The Navy raised its estimate of Landing Platform
Dock 17‘s requirements for spare parts, fuel, and software maintenance.
The Advanced Amphibious Assault Vehicle program office attributed its
increase in the cost estimate to funding additional prototypes to
improve reliability. Information like this allows options for decision
makers, while the system is still in development, to accept the costs
or re-examine the performance characteristics to see if they can be
relaxed in order to improve reliability and, thereby, reduce operating
and support costs.
DOD Pilot Programs Attempt to Reduce Total Ownership Costs:
In 1999, the Defense Systems Affordability Council implemented a
program to explore ways to reduce the total ownership cost of its
weapon systems. The Council--chaired by the Undersecretary of Defense,
Acquisition, Technology, and Logistics--set a goal of reducing
logistics costs for selected fielded weapon systems by 20 percent by
fiscal year 2005. The Council also set a goal for selected systems
still in development to achieve total ownership costs targets that are
20 percent to 50 percent below historical norms.
DOD selected 30 programs (10 from each military department) to test
various approaches for reducing total ownership costs, such as using
commercial items or technology to reduce costs of legacy systems and
using industry standards when developing systems to make upgrades
easier and less costly to complete.
The Air Force‘s C-17 program is one of the pilot programs. It is a
fielded system whose operating and support costs increased by about 25
percent between 1995 and 1999. One C-17 initiative is an engine upgrade
to extend the time between removals, reduce unexpected shop visits and
spares purchases, and reduce the number of engine overhauls. The Air
Force believes the C-17 could avoid $724.5 million in support costs if
the upgrade is completed.
The Apache recapitalization program, another of DOD‘s pilots,
integrates a number of selected upgrades that taken together are
expected to achieve a 30 percent reduction in operating and support
costs by 2010. Most of the Apache helicopters will be refurbished and
modified to the Apache Longbow configuration. The target acquisition
and designation system, the top cost driver, is a focus of these
improvements along with improvements in the drive train, the rotor, and
the propulsion system. The current average cost per flying hour for the
Apache fleet is $3,348. The Army‘s projected cost per flight hour after
the modifications is $2,230.
The Abrams tank is also undergoing a major upgrade estimated to cost
about $5 billion. The top cost driver on the tank is the power pack,
which includes the engine and transmission, followed by the auxiliary
automotive system, hull and frame, fire control system, armament, and
track. Army officials believe upgrading and replacing the engine is the
most effective way to reduce operating and support costs for the tank.
The current cost per mile for the Abrams fleet is $181 per mile,
including repair parts and fuel but excluding most personnel cost. The
recapitalization program‘s goal is to reduce the cost to $107 per mile
and to improve reliability.
Other Initiatives Show Promise, but Implementation Is Slow:
We found three other initiatives--the change in acquisition policy
toward evolutionary acquisition, an open systems approach for weapon
systems and the Commercial Operations and Support Savings Initiative--
that could help DOD reduce total ownership cost. However,
implementation has been limited in the latter two initiatives because
consistent high-level support is lacking.
DOD defines evolutionary acquisition as an approach for delivering
capability in increments, recognizing the need for future capability
improvements. DOD allows two processes to achieve an evolutionary
acquisition, both of which include requirements for collaboration
between the user, the tester, and the developer. The first process is
referred to as incremental development. In an incremental development
process, a desired capability is identified, an end-state requirement
is established, and the requirement is met over time by the development
of several increments of the product, each dependent on available,
mature technology. The second process is referred to as spiral
development. In a spiral development process, the end-state requirement
is not known, and each increment of the product is based on feedback
from the user. Each increment yields the best possible capability for
the user. The movement toward evolutionary acquisition and time-phased
requirements bodes well for the potential to understand reliability,
readiness, and implications for total ownership cost early, because an
evolutionary process allows an acquisition program to design a weapon
system to requirements based only on demonstrated technologies. This is
very similar to commercial practices.
DOD chartered an open-systems joint task force to implement an open
systems approach in weapon systems acquisitions. Open systems can
reduce cost through use of widely accepted standard products from
multiple suppliers, allowing DOD to benefit from the commercial market
place and take advantage of the competitive pressures that motivate
commercial companies to improve products and reduce prices. DOD
expected open systems to reduce the cost of ownership of weapon
systems, delay system obsolescence, and allow fielding superior war-
fighting capability more quickly. The DOD Inspector General recently
reported that the DOD acquisition community has not fully applied the
use of open systems objectives in the acquisition planning and review
process.[Footnote 12] The report recommended DOD enforce the use of an
open systems approach as part of the acquisition milestone review
process.
Another initiative that showed promise but lacks high level support is
the Commercial Operations and Support Savings Initiative introduced to
improve weapon system readiness and reduce operating and support costs
by inserting existing commercial items or technology into military
legacy systems. It emphasizes the rapid development of prototypes and
fielding of production items based on current commercial technology.
According to a 2001 report by an independent assessment team, the
initiative‘s three objectives of reducing operations and support costs
for legacy systems, simplifying prototype development, and attracting
commercial firms to the defense marketplace are being met. But, the
initiative lacks high-level support. The Under Secretary of Defense
(Acquisition, Technology, and Logistics) recently directed that funding
for the program be terminated.
DOD‘s Current Environment Does Not Provide Incentives to Reduce Total
Ownership Cost Early:
While initiatives for acquisition programs and potential reductions for
the fielded systems are welcomed, the department has not
institutionalized the practices used in the initiatives by demanding
them on all acquisition programs. As we discussed in Chapter 1, 90
percent of the operating and support costs are determined before
fielding, and these initiatives do not attack the causes of higher
operating and support costs. Those are: the division of responsibility
among the requirements community, the acquisition community, and the
maintenance community for controlling costs; the lack of focused
attention on reliability early in development; and the lack of
accountability for total ownership cost when setting requirements that
is caused by the division of responsibilities across these communities.
Companies we visited have incentives to make operating cost and product
readiness equal to technical performance when setting requirements for
new products because these factors largely determine their
profitability and, therefore, survival in the market place. Lower
operating costs translate to higher profits and increased sales.
Customers cannot afford to have large amounts of capital tied up in
extra equipment, spare parts, or personnel to ensure their equipment is
ready to perform when needed. They cannot afford to have equipment fail
during operations, because failure precludes accomplishment of the
company‘s mission and loss of revenue. These companies are constrained
by a finite amount of funding to acquire and operate their equipment,
and, therefore, they hold the people setting the product requirements
accountable for total ownership cost. Many of the companies we visited
use one integrated product team to identify needs, set requirements,
and monitor product development. Most importantly, the organization
that will be responsible for supporting the equipment in the field sets
requirements for new products. There is also a direct relationship
between the requirements-setting team and the product developer while
the product‘s requirements are being set, during development, and after
products are put into service. Information flows throughout the
integrated process, with each new phase in the process being informed
by knowledge from the phase just ending.
DOD‘s current acquisition environment does not provide the same
incentives or practices. Traditionally, DOD does not constrain its
requirement setters in the same way. Requirement setters in DOD have
demanded weapon systems that, due to their performance features,
consistently cost more to operate and support than anticipated to
achieve necessary readiness levels. This has been accepted because a
large logistics organization--separate from the requirement setting
organization--is charged with supporting these weapon systems and uses
monies from a different budget to do so. In essence, DOD‘s environment
frees the requirements community to insist on technical requirements
that cannot be made into reliable products, are costly to support, and
cannot be maintained cost effectively. Accountability for operating and
support costs does not rest with the requirement setters, or, for that
matter, with the acquisition community. Eventually, maintenance
organizations have no choice but to request sufficient funding to keep
weapon systems operating once they are fielded. DOD has identified this
division of responsibility as a key cause of higher weapon system
operating and support costs. In this current environment, there is no
incentive for collaboration and accountability in setting requirements
and developing products with operating costs in mind. Instead, it
provides incentives to field systems with unknown reliability by
allowing whatever funding necessary to operate and maintain the systems
once they are fielded.
[End of section]
Chapter 5: Conclusions and Recommendations:
Acceptable readiness levels are a function of having platforms
available when required. Such levels can be achieved by having highly
reliable platforms, by spending whatever is necessary on ongoing
maintenance, or by having excess capacity. The high cost of maintaining
weapon systems to meet required readiness levels is depleting DOD‘s
modernization accounts and denying DOD the flexibility to invest in new
weapons. DOD must find ways to reduce total ownership cost while
maintaining needed readiness rates. Readiness is a critical component
of all DOD weapons systems. If a system is not ready, its performance
capabilities are of no use. The decision on whether readiness will be
achieved by spending additional funds on operations or by designing
high reliability into the weapon system must be made while requirements
are being set and early in product development.
DOD‘s prevailing practices run counter to achieving high reliability.
Often, DOD does not make readiness or operating cost performance
parameters equal in importance to others when it establishes
requirements for weapons systems. Further, reliability growth during
product development is hampered by immature technologies and delays in
gaining knowledge about the product‘s design. Finally, DOD does not
have sufficient knowledge about its fielded systems to inform its
product development process for new systems. DOD is at a crossroad in
this regard. It has made positive changes to acquisition policy in
order to change its environment. Requiring higher readiness at lower
cost will enable DOD to take the next step, ensuring lower total
ownership cost.
In contrast, commercial companies that are in the market for new
capital equipment understand that they must specify and control the
readiness and total ownership cost of a product, especially the
operating and support costs early in development. Therefore, they
specify how available or ready the products must be in order to carry
out the company‘s mission. Further, they set goals for operating costs
when acquiring new equipment; they make sure they understand their own
operating costs from data they have collected and analyzed on equipment
they are now using. Those two goals--how available the product must be
and how much the customer wants to spend per operating unit to support
the equipment--are key requirements equal in importance to other
performance characteristics that the commercial customer demands from
the companies that develop the products. Bounded by the twin
requirements of specific operating costs and availability, the product
developer sets reliability goals for the components, subsystems, and
the full system once it is integrated into a product that will satisfy
the customer‘s requirements. Product developers remain focused on good
product development practices with mature technologies, stable designs,
and production processes that are in control. During operations they
collect data from their customers on reliability and performance and
use that data to predict operating and support costs for subsequent
developments or upgrades.
DOD has initiatives underway that partially address the issue of
controlling operating and support costs. However, without significant
emphasis on providing a better framework for decision-making, these
initiatives will not yield sufficient improvements. The department has
encouraged the services to include key performance parameters in its
newer developments such as the Joint Strike Fighter that indicate how
long a system must perform between maintenance actions. It has moved to
follow best practices for reducing risk from technology and achieving
more stable designs in the Advanced Amphibious Assault Vehicle and the
Joint Strike Fighter. However, these programs are early in development
and it will take some time to see how reliably they perform. We believe
that practices found at the commercial companies we visited to make
operating and support costs and product readiness requirements equal in
priority to other performance characteristics forces developers to
focus on achieving high reliability and that adopting these practices
will help DOD achieve high readiness and control total ownership cost.
Recommendations for Executive Action:
DOD should take steps to make the cost to operate and support weapon
systems at required readiness rates a priority when setting weapon
system requirements for an affordable weapon system and finalizing the
design of the selected system. To do this, its requirements and
acquisition communities must collaborate to fully understand and
control the costs to operate and support a weapon system prior to and
early in product development, when it is possible to have significant
impact on those costs. In establishing requirements for a weapon
system, the requirements community should include the costs to operate
and support the weapon system over its life cycle and the readiness
rate for the weapon system. To establish an affordable design for the
weapon system, the acquisition community and acquisition programs
should be required to accurately estimate--based on demonstrated
component and subsystem reliability testing--that portion of the costs
that DOD plans to spend on operations and support of the weapon system
throughout its life cycle before the design is finalized.
With this in mind, to ensure that the user‘s requirements for a weapon
system can be met with a reliable design, we recommend that the
Secretary of Defense:
* revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01B
on the requirements generation process to include total ownership cost,
especially operating and support cost, and weapon system readiness
rates as performance parameters equal in priority to any other
performance parameters for any major weapon system before beginning the
acquisition program;
* revise the current policy governing the operation of the defense
acquisition system (currently under revision) to require that the
product developer establish a firm estimate of a weapon system‘s
reliability based on demonstrated reliability rates at the component
and subsystem level no later than the end of the system integration
phase, coinciding with the system-level critical design review, before
proceeding into the system demonstration phase of product development;
and at the system level no later than the full-rate production
decision; and:
* structure its contracts for major systems acquisitions so that at
Milestone B the product developer has incentives to ensure that proper
trades are made between reliability and performance prior to the
production decision. One option is to provide specific clauses in the
development contract to address reliability growth.
Agency Comments and Our Response:
DOD partially concurred with all of our recommendations; however, for
the most part, it found no further action was needed to lower total
ownership cost. We disagree. We believe that if DOD takes no further
action in implementing these recommendations, it ignores significant
opportunities to improve readiness and lower the total ownership cost
of its major weapon systems. The current budget environment demands
more effort in reducing these costs.
The performance of weapon systems as described in this report is
evidence that they demand much more money than planned to remain ready.
DOD should consider each of these recommendations as parts of a whole
solution for its ’death spiral“--that is, the inability to modernize
its forces because the cost to operate and maintain unreliable weapon
systems at needed readiness rates constantly impinges on its
modernization budget. Taken as a whole, our recommendations encourage
DOD‘s requirement setters to demand readiness at an affordable cost as
a part of a system‘s performance, provide a mechanism to hold the
product developer accountable for determining the reliability needed to
satisfy DOD‘s requirements, and provide contractual incentives for the
product developer to build reliability into a weapon system very early
in its development. The details of DOD‘s response to each
recommendation are summarized below along with our rebuttal.
In a response prepared by the Office of the Joint Chiefs of Staff, DOD
partially concurred with our recommendation to include readiness and
total ownership cost as performance parameters equal in priority to any
others before beginning an acquisition program, commenting that they
are currently equal in priority to ’non-key performance parameters.“ We
are concerned that DOD does not recognize the importance of requiring
targets for a system‘s readiness and its total ownership cost before
beginning product development. These targets are critical to providing
a realistic goal for the product developer to deliver reliable, cost-
efficient weapon systems.
We examined five deployed weapon systems for this report. None had key
requirements for readiness rates or operating costs. All had
significant problems with reliability and, therefore, readiness and
total ownership cost. We also reviewed several commercial products that
were developed with readiness and total ownership cost as critical
requirements. In each case, the products were ready to perform when
needed at affordable and predictable cost. Unless these requirements
are equal in importance to any others, they will not withstand the
pressures of an acquisition program. The Joint Chiefs stated that its
requirements generation policy is currently under revision and that no
decision has been made about the priority readiness and cost should
have as requirements. We believe DOD has an excellent opportunity to
finally lower the total ownership cost of current and future weapon
acquisitions, thereby freeing significant funds for modernization if it
implements this recommendation.
In a response prepared by the Office of the Undersecretary of Defense
for Acquisition, Technology, and Logistics, DOD partially concurred
with our second recommendation to establish estimates of a weapon
system‘s reliability--first, based on demonstrated reliability rates at
the component and subsystem level by the end of system integration,
coincident with the critical design review and next, at the system
level at the time the production decision is made--however, DOD found
no need to revise the policy governing the defense acquisition system
to achieve this. We disagree. Demonstrating reliability during product
development has not received the priority it requires if DOD is to have
a realistic opportunity to reduce the total ownership cost of its
weapons while maintaining required readiness levels. The current policy
does not provide a mechanism to ensure consistent application of
reliability estimates based on demonstrated performance. We believe
that the disparity between the actual costs to operate and maintain
weapon systems and what DOD had estimated those costs to be during the
weapon systems‘ development, as described in the report, provides
strong evidence that product developers do not understand reliability
under DOD‘s current process. Implementation of this recommendation will
assist DOD in requiring reliability estimates on a consistent basis.
In a response prepared by Undersecretary of Defense, Acquisition,
Technology, and Logistics to our recommendation that DOD structure its
development contracts to include requirements to provide incentives for
product developers to trade performance for reliability when it makes
sense, DOD found no need for additional incentives to contractors
beyond giving them total system performance responsibility. We have yet
to see evidence that total system performance responsibility has
provided an incentive for any product developer to trade performance
for reliability in order to reduce total ownership cost. Further, we
believe a contractual agreement similar to those we found in commercial
cases--such as financial penalties for readiness below certain
specified rates--would provide an excellent incentive for product
developers to gain the knowledge required to meet reliability rates
early in a weapon system‘s design before committing to production.
DOD provided some technical comments in attachment 2. We have addressed
those in the report as necessary. The full text of the department‘s
response to the recommendations is provided in appendix I.
[End of section]
Related GAO Products:
Best Practices: Capturing Design and Manufacturing Knowledge
Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.:
July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead
to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.:
March 8, 2001.
Defense Acquisitions: Higher Priority Needed for Army Operating and
Support Cost Reduction Efforts. GAO/NSIAD-00-197. Washington, D. C.:
September 29, 2000.
Defense Acquisitions: Air Force Operating and Support Costs Reductions
Need Higher Priority. GAO/NSIAD-00-165. Washington, D. C.: August 29,
2000.
Best Practices: A More Constructive Test Approach Is Key to
Better Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.:
July 31, 2000.
Defense Logistics: Actions Needed to Enhance Success of Reengineering
Initiatives. GAO/NSIAD-00-89. Washington, D. C.: June 23, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 26,
2000.
Best Practices: DOD Training Can Do More to Help Weapon System Programs
Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.:
August16, 1999.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. GAO/NSIAD-99-162. Washington, D.C.: July 30,
1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible. GAO/
T-NSIAD-98-123. Washington, D.C.: March 18, 1998.
Best Practices: DOD Can Help Suppliers Contribute More to Weapon System
Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD‘s Environment. GAO/NSIAD-98-56. Washington, D.C.:
February 24, 1998.
Major Acquisitions: Significant Changes Underway in DOD‘s Earned Value
Management Process. GAO/NSIAD-97-108. Washington, D.C.: May 5, 1997.
Best Practices: Commercial Quality Assurance Practices Offer
Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.:
August 26, 1996.
[End of section]
Appendix I: Comments from the Department of Defense:
ACQUISITION, TECHNOLOGY AND LOGISTICS:
OFFICE OF THE UNDER SECRETARY OF DEFENSE:
3000 DEFENSE PENTAGON WASHINGTON, DC 20301-3000:
DEC 2002:
Ms. Katherine V. Schinasi:
Director, Acqusition and Sourcing Management U.S. General Accounting
Office:
441 G Street, N. W. Washington, D.C. 20548:
Dear Ms. Schinasi:
This is the Department of Defense (DoD) response to the GAO draft
report, ’BEST PRACTICES: Setting Requirements for Cost and Readiness
Could Reduce Weapon Systems‘ Total Ownership Costs,“ dated November 26,
2002 (GAO Code 120092/GAO-03-057). There are two enclosures. The first
contains the DoD detailed comments to the recommendations. The second
are additional comments on the report. The DoD response to
Recommendation 1 was provided by the Joint Chiefs of Staff.
Sincerely,
Spiros G. Pallas, Principal Deputy, Strategic and Tactical Systems:
Signed by Spiros G. Pallas:
Enclosure(s): As stated:
ENCLOSURE (1):
GAO DRAFT REPORT - DATED NOVEMBER 26, 2002 GAO CODE 120092/GAO-03-057:
’BEST PRACTICES: Setting Requirements for Cost and Readiness Could
Reduce Weapon Systems‘ Total Ownership Costs“:
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
revise the Chairman of the Joint Chiefs of Staff Instruction 3170.01 on
the requirements generation process to include total ownership cost,
especially operating and support cost, and weapon system readiness
rates as performance parameters equal in priority to any other
performance parameters for any major weapon system prior to beginning
the acquisition program. (p. 66/GAO Draft Report):
DoD RESPONSE: Partially Concur. The Operational Requirements Document
currently addresses program affordability, stated in terms of a
performance parameter threshold and objective, and system performance
parameters, including mission reliability; they are equal in priority
to any other ’non-KPP“ performance parameter. The supporting guidance
in CJCSI 3170.01 is currently under revision-the Department has not
decided whether to clarify it with a more specific description towards
weapon system readiness rates.
RECOMMENDATION 2: The GAO recommended that the Secretary of Defense
revise the current policy governing the operation of the defense
acquisition system to require that the product developer establish a
firm estimate of a weapon system‘s reliability based on demonstrated
reliability rates at the component and subsystem level no later than
the end of the system integration phase, coinciding with the system-
level critical design review, before proceeding into the system
demonstration phase of product development; and at the system level no
later than the full-rate production decision. This policy is currently
under revision, to be finalized in early 2003. (p. 66/GAO Draft
Report):
DoD RESPONSE: Partially Concur: While there is no formal direction to
provide a firm estimate of reliability during critical design review,
program system performance is reviewed for the final design
configuration, including compliance to any reliability requirement or
specification included in the contract. Because this is already
accomplished, there is no need to further direct the recommendation.
RECOMMENDATION 3: The GAO recommended that the Secretary of Defense
structure its contracts for major systems acquisitions at Milestone B
to provide incentives for the product developer to ensure that proper
trades are made between reliability and performance prior to the
production decision. One option is to provide specific clauses in the
development contract to address reliability growth. (p. 66/GAO Draft
Report):
DoD RESPONSE: Partially Concur: DoD views reliability as part of system
performance. Also, DoD already encourages system design trades
throughout development. Incentivizing reliability growth is one of many
possible approaches already used in some cases. There is no need for
additional emphasis by DoD. Other approaches used by DoD include total
system support responsibility (TSSR) which automatically incentivizes
contractors to provide reliable systems while increasing their profits.
[End of section]
Appendix II GAO Staff Acknowledgments:
Acknowledgments:
Cheryl Andrew, Beverly Breen, Belinda LaValle, Carol Mebane, Gary
Middleton, Michael Sullivan, Adam Vodraska, and Earl C. Woodard:
FOOTNOTES
[1] An open system is one that is designed with interfaces to accept
upgrades easily without redesign of the total unit. Replacements in an
open system only have to meet interface requirements to be accepted.
[2] A key performance parameter represents a capability that is so
significant that failure to meet the minimum value could be a reason
for DOD or the services to reevaluate the concept or system or
terminate the program.
[3] Operations and support of weapons systems is a part of the
Operations and Maintenance budget, which also includes amounts for
health care, base and facilities support, and other activities for the
well-being and operations of the military forces. Costs for operations
and support of weapon systems were about 48 percent of the Operations
and Maintenance budget in fiscal year 2002.
[4] The operating and support cost for the Apache is not available
before 1993.
[5] The operating and support cost for the M-1 series Abram is not
available before 1993.
[6] U.S. General Accounting Office, Best Practices: Better Matching of
Needs and Resources Will Lead to Better Weapon System Outcomes, GAO-01-
288 (Washington, D.C.: Mar. 8, 2001).
[7] U.S. General Accounting Office, Best Practices: Capturing Design
and Manufacturing Knowledge Early Improves Acquisition Outcomes, GAO-
02-701 (Washington, D.C.: July 15, 2002).
[8] Chairman of the Joint Chiefs of Staff Instruction, Requirements
Generation System, (Washington, D.C.: Apr. 15, 2001).
[9] GAO-01-288.
[10] A threshold is the minimum acceptable operational combat
capability
required to meet war-fighter minimum requirements. An objective is the
capability desired of the system beyond minimum requirements.
[11] U.S. General Accounting Office, Joint Strike Fighter Acquisition:
Mature Critical Technologies Needed to Reduce Risks, GAO-02-39
(Washington, D.C.: Oct. 19, 2001).
[12] Department of Defense, Office of the Inspector General, Use of An
Open Systems Approach for Weapon Systems, Report No. D-2000-149
(Washington, D.C.: June 14, 2000).
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
