Defense Management
Opportunities to Enhance the Implementation of Performance-Based Logistics Gao ID: GAO-04-715 August 16, 2004The Department of Defense (DOD) is pursuing a policy that promotes performance-based logistics at the platform level as the preferred product support strategy for its weapon systems, based in part on DOD's perception that this is an industry best practice. GAO was asked to compare industry practices for activities using complex and costly equipment with life-cycle management issues similar to those of military systems to identify lessons learned that can be useful to DOD. This is the first of two reports addressing DOD's implementation of performance-based logistics and is intended to facilitate DOD's development of new guidance on the use of this approach.
DOD's current policy for implementing performance-based logistics as a preferred support approach at the weapon system platform level does not reflect the practices of private-sector companies that support expensive and complex equipment with life-cycle management issues. The companies GAO interviewed use performance-based contracting as a tool rather than as a preferred support concept at the weapon system platform level. While 7 of the 14 companies GAO interviewed use some type of performance-based contracting, they use it at the subsystem or component level--for commodities such as engines, wheels, and brakes--when it is cost-effective and reduces risk in a noncompetitive environment. DOD's proposed policy of pursuing performance-based logistics as the preferred support approach at the platform level results in contracting out the program-integration function--a core process the private-sector firms consider integral to successful business operations. Further, this proposed policy could limit opportunities to take advantage of competition when it is available for subsystems or components as well as limit opportunities to gain purchasing power from volume discounts on components across an entire fleet and avoid the administrative costs charged by a prime integrator. While DOD is proposing the aggressive use of performance-based logistics on both older and new weapon system platforms, the companies GAO interviewed use performance-based contracting at the subsystem or component level when it is cost-effective--often in a noncompetitive environment when the manufacturer controls expensive repair parts, such as engines. In general company officials said they rely more widely on other contracting vehicles, such as time and material contracts, particularly for new systems. Company officials noted that in the absence of accurate and reliable information on system performance to establish a baseline for evaluating the cost-effectiveness of a performance-based contract for new systems, the risk of the negotiated price's being excessive is increased. The companies GAO interviewed also emphasized the importance of having rights to the technical data--such as maintenance drawings, specifications, and tolerances--needed to support the management of all logistics contracts and, should the service provider arrangements fail, to support competition among alternate providers. In contrast, DOD program managers often opt to spend limited acquisition dollars on increased weapon system capability rather than on rights to the technical data--thus limiting their flexibility to perform work in-house or to support alternate source development should contractual arrangements fail.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-715, Defense Management: Opportunities to Enhance the Implementation of Performance-Based Logistics
This is the accessible text file for GAO report number GAO-04-715
entitled 'Defense Management: Opportunities to Enhance the
Implementation of Performance-Based Logistics' which was released on
August 16, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Subcommittee on Readiness and Management Support,
Committee on Armed Services, U.S. Senate:
United States Government Accountability Office:
GAO:
August 2004:
Defense Management:
Opportunities to Enhance the Implementation of Performance-Based
Logistics:
GAO-04-715:
GAO Highlights:
Highlights of GAO-04-715, a report to the Subcommittee on Readiness and
Management Support, Committee on Armed Services, U.S. Senate
Why GAO Did This Study:
The Department of Defense (DOD) is pursuing a policy that promotes
performance-based logistics at the platform level as the preferred
product support strategy for its weapon systems, based in part on DOD‘s
perception that this is an industry best practice. GAO was asked to
compare industry practices for activities using complex and costly
equipment with life-cycle management issues similar to those of
military systems to identify lessons learned that can be useful to DOD.
This is the first of two reports addressing DOD‘s implementation of
performance-based logistics and is intended to facilitate DOD‘s
development of new guidance on the use of this approach.
What GAO Found:
DOD‘s current policy for implementing performance-based logistics as a
preferred support approach at the weapon system platform level does not
reflect the practices of private-sector companies that support
expensive and complex equipment with life-cycle management issues. The
companies GAO interviewed use performance-based contracting as a tool
rather than as a preferred support concept at the weapon system
platform level. While 7 of the 14 companies GAO interviewed use some
type of performance-based contracting, they use it at the subsystem or
component level”for commodities such as engines, wheels, and brakes”
when it is cost-effective and reduces risk in a noncompetitive
environment. DOD‘s proposed policy of pursuing performance-based
logistics as the preferred support approach at the platform level
results in contracting out the program-integration function”a core
process the private-sector firms consider integral to successful
business operations. Further, this proposed policy could limit
opportunities to take advantage of competition when it is available for
subsystems or components as well as limit opportunities to gain
purchasing power from volume discounts on components across an entire
fleet and avoid the administrative costs charged by a prime
integrator.
While DOD is proposing the aggressive use of performance-based
logistics on both older and new weapon system platforms, the companies
GAO interviewed use performance-based contracting at the subsystem or
component level when it is cost-effective”often in a noncompetitive
environment when the manufacturer controls expensive repair parts, such
as engines. In general company officials said they rely more widely on
other contracting vehicles, such as time and material contracts,
particularly for new systems. Company officials noted that in the
absence of accurate and reliable information on system performance to
establish a baseline for evaluating the cost-effectiveness of a
performance-based contract for new systems, the risk of the negotiated
price‘s being excessive is increased.
The companies GAO interviewed also emphasized the importance of having
rights to the technical data”such as maintenance drawings,
specifications, and tolerances”needed to support the management of all
logistics contracts and, should the service provider arrangements fail,
to support competition among alternate providers. In contrast, DOD
program managers often opt to spend limited acquisition dollars on
increased weapon system capability rather than on rights to the
technical data”thus limiting their flexibility to perform work in-house
or to support alternate source development should contractual
arrangements fail.
What GAO Recommends:
GAO recommends that DOD (1) revise its policy and guidance to the
services to reflect the industry practice of using performance-based
logistics as a tool to achieve economies at the subsystem or component
level, rather than at the platform-level, and (2) provide for
sufficient technical data to support alternative support options using
either the public or the private sector. DOD concurred with our
recommendations, noting that it would re-emphasize via policy and
training the use of performance-based logistics at the subsystem level
and take steps to update acquisition policy to include guidance on
purchasing rights or long-term access to technical data.
www.gao.gov/cgi-bin/getrpt?GAO-04-715.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact William M. Solis at (202)
512-8365 or solisw@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DOD's Current Preferred Approach to Performance-Based Logistics May
Limit Opportunities for Competition and Savings:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Scope and Methodology:
Appendix I: Comments from the Department of Defense:
Tables:
Table 1: Company Use of Contracting Tools to Outsource Subsystem and
Component Support:
Table 2: Percentage of Maintenance Dollars for Outsourced Subsystems
and Components Managed Using Performance-Based Contracts:
Abbreviations:
APU: auxiliary power unit:
DOD: Department of Defense:
United States Government Accountability Office:
Washington, DC 20548:
August 16, 2004:
The Honorable John Ensign:
Chairman:
The Honorable Daniel Akaka:
Ranking Minority Member:
Subcommittee on Readiness and Management Support:
Committee on Armed Services:
United States Senate:
In the past 4 to 5 years, the Department of Defense (DOD) has taken
steps to manage the total life-cycle support costs of its weapon
systems and to improve logistics support to the warfighter by
reengineering its processes for both acquiring and sustaining weapon
systems. As part of these reengineered processes, DOD has directed
weapon system program managers to develop acquisition strategies that
maximize competition, innovation, and interoperability and to
capitalize on commercial technologies to reduce costs. Within the area
of weapon system sustainment, DOD is pursuing a policy to implement a
concept it calls performance-based logistics as the preferred support
strategy for DOD weapon systems. This concept is a variation on other
contractor logistics support strategies calling for long-term support
of military systems by the systems' manufacturers.[Footnote 1] The
concept involves defining a level of performance for a weapon system
already fielded or about to be fielded that is to be achieved over a
fixed period of time for a fixed level of annual funding. More
recently, in February 2004, the Deputy Secretary of Defense issued a
memorandum promoting a more aggressive implementation of performance-
based logistics that was in part based on the perception that this is
an industry best practice.
As requested, we are reviewing DOD's process of implementing
performance-based logistics as the preferred support strategy for its
weapon systems. As a part of this review we determined what types of
contractor logistics support arrangements the private sector uses for
activities that have complex and costly equipment with life-cycle
management issues similar to those of military systems, and what
potential lessons can be learned from a comparison between private-
sector contractor support practices and the contractor logistics
support practices DOD is urging the services to implement. While
conducting our work, we learned that DOD soon will be issuing
additional policy guidance on its use of performance-based logistics.
This is the first of two GAO reports addressing DOD's implementation of
performance-based logistics and is intended to provide the Secretary of
Defense with recommendations that should facilitate DOD's development
of new guidance. Our follow-on report in early 2005 will determine
similarities and differences in the way the identified DOD programs are
structured and managed, identify approaches that appear to offer the
greatest opportunities to achieve cost effectiveness, and evaluate the
demonstrated cost savings or improved responsiveness of the new
DOD concept.
As a part of this review, we examined Office of the Secretary of
Defense and service policies and guidance; collected data on
performance-based logistics programs identified by the services; and
conducted case studies on a limited number of the programs. We also
reviewed the logistics-contracting practices of 14 private-sector
companies from the air carrier, maritime shipping, energy exploration,
mining, and entertainment industries--companies that use complex and
costly equipment with life-cycle issues similar to those of military
weapon systems and that are motivated by the desire to minimize costs
and maximize profits to choose the most cost-effective option.[Footnote
2] We held group discussions covering standard questions about the
industries' contractor logistics support practices, and we compared the
results of these interviews with the preliminary information obtained
from our analyses of DOD policies and programs. We reviewed the
reliability of the projected cost and savings data used in this report
and determined that it was sufficient for our purposes. We performed
our work from September 2003 through June 2004 in accordance with
generally accepted government auditing standards. The scope and
methodology section contains more detailed information about the work
we performed.
Results in Brief:
DOD's current policy for implementing performance-based logistics as a
preferred support approach at the weapon system platform level does not
reflect the practices of private-sector companies that support
expensive and complex equipment with life-cycle management issues. A
Deputy Secretary of Defense memorandum to the services cites the
private sector's use of performance-based logistics as the basis for
aggressively pursuing this concept. Private-sector companies we
interviewed having complex and expensive assets with life-cycle issues
similar to those of military weapon systems do not use performance-
based contracting this way. Although 7 of the 14 companies we
interviewed use some type of performance-based contracting, they use it
at the subsystem or component level, when it is cost-effective and
reduces risk in a noncompetitive environment. The companies rely more
widely on other contracting methods to benefit from competition for
those subsystems or components where it is practicable. DOD's approach
supports aggressive implementation of performance-based logistics at
the weapon system platform level and for new as well as older systems.
As a result, DOD's proposed approach to implementing performance-based
logistics could limit opportunities for achieving cost savings from
competition, volume discounts, and reduced administrative costs.
Further, it could result in the contracting out of the program-
integration function--a core process that the private-sector firms
consider integral to successful business operations. Private-sector
company officials we interviewed reported that their firms use the
following approaches:
* Use performance-based contracting selectively when it is cost-
effective--often in a noncompetitive environment when the manufacturer
controls expensive repair parts, such as engines. In general, company
officials said they rely more widely on other contracting vehicles,
such as time and material contracts, particularly for newer systems
that don't have a performance history. DOD's approach, in contrast,
proposes aggressive implementation of performance-based contracts on
both older and newer weapon systems. Company representatives emphasized
that in the absence of accurate and reliable information on system
performance to establish a baseline for evaluating the cost-
effectiveness of a performance-based contract for new systems, the risk
of the negotiated price being excessive is increased.
* Use performance-based logistics at the subsystem or component level,
such as for engines; DOD's approach, in contrast, proposes to support
implementation at the weapon system platform level, such as was tried
for the T-45 trainer aircraft. We found no private-sector performance-
based contracts being used at the platform level. The company
representatives preferred to retain the program integration function
that they consider a core function essential to the success of their
business operations. Additionally, they prefer to (1) take advantage of
competition when it is available for subsystems or components, (2) gain
purchasing power from volume discounts on subsystems or components
across their entire fleet, and (3) avoid the administrative costs that
would be charged by a prime integrator. Indeed, Navy officials told us
that the T-45 platform level performance-based logistics contract
resulted in their paying the contractor for hours that the Navy did not
fly and that the contract was not cost-effective. But by dividing the
airframe and engine into separate contracts, adding a sortie completion
metric, and competing the airframe workload, the Navy projects that
savings of $144 million ($118 for the airframe and $26 million for the
engine) can be achieved over 5 years.
* Emphasize the importance of having rights to the technical data
needed to support the management of all logistics contracts--such as
detailed maintenance drawings, specifications, and tolerances--and,
should the companies' service provider arrangements fail, to support
competition among alternate providers. DOD program managers, in
contrast, often opt to spend limited acquisition dollars on increased
weapon system capability rather than on the rights to technical data.
This trade-off limits DOD's flexibility, because although DOD may be
obtaining access to technical data needed to manage performance-based
contracts, it may not be developing product-support strategies that
provide for the future delivery of technical data when required to
support competition or alternate source development if performance-
based logistics arrangements were to fail.
We are making recommendations that, if followed and included in the
soon to be issued guidance, should improve the implementation of
performance-based logistics in the department. In commenting on a draft
of this report, DOD concurred with our findings and recommendations.
DOD's response is included as appendix I.
Background:
Performance-based logistics is the DOD term for the process of
(1) identifying a level of performance required by the warfighter and
(2) negotiating a performance-based contract between the government and
the product support integrator--that is generally the original
equipment manufacturer of the total system--to provide long-term total
system support for a weapon system at a fixed level of annual funding.
Instead of buying spares, repairs, tools, and data in individual
transactions, the method in a performance-based logistics arrangement
is to buy a predetermined level of availability that meets the
warfighter's objectives. To implement performance-based logistics, DOD
selects a product support integrator to serve as the single point of
accountability, integrating support from all sources to achieve the
performance outcome metrics specified in the performance-based support
agreement. The metrics used include operational availability (a measure
of the degree to which an item is in an operable state and can be
committed at the start of a mission when the mission is called for at
an unknown point in time); mission capability (the material condition,
indicating that it can perform at least one and potentially all of its
designated missions); and customer wait time (the total elapsed time
between issuance of a customer order and fulfillment of that order).
For example, the Navy now uses two metrics for its performance-based
contract for the T-45 aircraft system[Footnote 3]--"ready for
training," which requires that the contractor have a minimum number of
aircraft ready for training at 7:00 AM each business day in order to
achieve a 57 percent aircraft availability; and "sortie completion,"
which requires that the contractor meet 98 percent of the requirements
for the scheduled training flights. As an incentive, the contract pays
a performance bonus (maximum of $5 million annually) if the contractor
exceeds the performance metrics. If the contractor only meets--or fails
to meet--the minimum metrics, the contractor then receives none of the
annual performance bonus.
DOD Directive 5000.1, the Defense Acquisition System, highlights the
department's preference for using performance-based logistics at the
platform level, stating, "Program Managers shall develop and implement
performance-based logistics strategies that optimize total system
availability while minimizing cost and logistics footprint." As part of
its implementation of this strategy, in 2003 DOD proposed that the
Congress adopt legislative changes that would allow the services to
increase the appropriations allocation flexibility within a weapon
system program, allowing the program manager to use funds from
different accounts (such as operation and maintenance; research,
development, test, and evaluation; and procurement) to pay for system
support costs. Although this proposal was not adopted, DOD continues to
pursue various avenues that would support the overall objective of
having greater flexibility by using a single line of support funding
managed by the program office for total system operation and
maintenance costs. Most recently, on February 4, 2004, the Deputy
Secretary of Defense (1) directed the Under Secretary of Defense
(Acquisition, Technology, and Logistics) in conjunction with the Under
Secretary of Defense (Comptroller) to issue clear guidance on
purchasing using performance criteria; and (2) directed each service to
provide a plan to aggressively implement performance-based logistics,
including transferring appropriate funding as needed,[Footnote 4] on
current and planned weapon system platforms for fiscal years 2006-2009.
While this directive does not preclude the services from using
performance-based logistics contracts below the platform level, it does
express DOD's intent to apply the concept at the platform level as a
preferred practice. As we discuss in the next section, DOD has
established separate goals for implementing performance-based service
contracts, and the services have identified many contracts as
performance-based logistics arrangements that are, in fact, below the
platform level. However, according to Office of Secretary of Defense
officials, DOD would like to implement performance-based logistics at
the platform level to move from contracting for material availability
to weapon system availability. DOD considers that the platform level
offers the metrics needed to implement a true performance-based
logistics arrangement.
DOD Performance-Based Logistics Evolving from Its Use of Performance-
Based Service Contracting:
The Office of Management and Budget indicates that performance-based
service contracting, from which performance-based logistics has
evolved, has been referenced in regulation, guidance, and policy for
more than two decades, and federal agencies have used performance-based
contracting to varying degrees for acquiring a range of services. In
1991 the Office of Management and Budget issued a policy letter
establishing the use of a performance-based approach for service
contracting, and in 1994 it initiated a governmentwide pilot project to
encourage the use of performance-based service contracts in federal
agencies, including DOD. The use of performance-based service contracts
to acquire services offers a number of potential benefits, particularly
when services are acquired by means of a fixed price agreement.
Performance-based contracts can encourage contractors to be innovative
and to find cost-effective ways of delivering services for a fixed
level of funding. By shifting the focus from process to results, these
contracts can potentially produce better outcomes and reduced costs.
In view of the potential benefits, Congress has been encouraging
greater use of performance-based service contracting.[Footnote 5] In an
August 2003 memorandum to the military departments, the Under Secretary
of Defense (Acquisition, Technology and Logistics) stated that DOD
should continue to increase its use of performance-based service
acquisitions. He noted that DOD has a goal to award 50 percent of
contract actions and dollars using performance-based specifications by
fiscal year 2005.
The more specific concept of performance-based logistics as an approach
for supporting military systems emerged from DOD's 1999 study, Product
Support for the 21st Century, which identified 30 pilot programs (10 in
each military department) to test logistics support reengineering
concepts that placed greater reliance on the private sector. Many of
the pilots involved various types of contractor logistics support,
prime vendor support, or performance-based type arrangements. Others
focused on including reduced operation and support costs and improved
readiness as performance requirements for new system
development.[Footnote 6] The September 30, 2001, Quadrennial Defense
Review advanced DOD's move toward this concept by advocating the
implementation of performance-based logistics with appropriate metrics
that would be designed to compress the supply chain and improve the
readiness of major weapon systems and commodities. [Footnote 7] A
November 2001 Office of the Deputy Under Secretary of Defense document,
Product Support for the 21st Century: A Program Manager's Guide to
Buying Performance, intended as a guide for program managers, stated
that program managers will implement performance-based logistics on all
new systems and on acquisition category I and II fielded systems
selected on the basis of a sound business case.
It is unclear how many performance-based logistics programs the
services have implemented. In response to our inquiries, the Army
identified 74 performance-based logistics programs, the Navy identified
106, the Air Force 4, and the Marine Corps 1. We noted a broad range of
contract arrangements is identified under the performance-based
logistics umbrella, with many of them initiated under a different name,
such as contractor logistics support or total systems support
responsibility and later identified as performance-based logistics
arrangements. Most of the DOD performance-based logistics arrangements
currently identified by the services are used for subsystems or
components rather than for weapon system platforms.
Fiscal years 2003 to 2007 Defense Planning Guidance required the
services to submit plans that identified their implementation schedules
for performance-based logistics to all new weapon systems and
acquisition category I and II fielded systems. Similarly, a February
13, 2002, letter from the Under Secretary of Defense (Acquisition,
Technology, and Logistics) to the services emphasized the need for the
plans required by the Defense Planning Guidance and directed that the
plans be issued by May 1, 2002.
But although the services issued plans, they did not take an aggressive
approach toward adopting this concept, according to Office of Secretary
of Defense logistics officials. An October 2003 Defense Business Board
report encouraged the department to move more quickly in adopting the
performance-based logistics, stating, "Performance-based logistics is
an industry best practice and a DOD best practice. DOD should consider
using it for all its weapon systems, new and legacy, provided it is
supported by a business-case analysis."[Footnote 8] This task force was
chartered by the Under Secretary of Defense (Comptroller) and Chief
Financial Officer to describe private-sector best practices used in
managing supply chain partnering arrangements and to propose how to
apply such practices to the supply chain processes used by DOD. Citing
this task force report, the aforementioned February 2004 Deputy
Secretary of Defense memorandum to the military departments stated,
"Delay in implementing this practice complicates our funding, limits
industry flexibility, and increases DOD inventory. We must streamline
our contracting and financing mechanisms aggressively to buy
availability and readiness measured by performance criteria."[Footnote
9]
DOD's Current Preferred Approach to Performance-Based Logistics May
Limit Opportunities for Competition and Savings:
Because DOD proposes using performance-based logistics at the platform
level as the predominant support strategy for its military systems, it
may limit opportunities for savings from competition, volume discounts,
and reduced administrative costs. Also, by often not contracting for
long-term access to technical data, programs officials are further
limiting their support options.
In the private sector, performance-based contracting is a tool used
according to the applicability of subsystem or component and
circumstance, when it is cost-effective and reduces risk in a
noncompetitive environment. DOD, by contrast, proposes using it as the
predominant product support strategy for its military systems. Further,
when private-sector companies use performance-based contracting, they
use it at the subsystem or component level, retaining the program
integration function themselves as a core business function essential
to successful business operations. Conversely, DOD policy memoranda
support using performance-based contracting at the platform level and
using the contractor as the support integrator. Moreover, private
sector companies emphasize the importance of having the rights to
contracts and competition. DOD, in contrast, is frequently not
acquiring the same level of technical data in its acquisition of new
programs.
Performance-Based Contracting for Logistics Used as a Tool in Industry
Rather Than a Preferred Support Concept as Proposed by DOD:
While our review of private sector companies did find that half of
those we interviewed are using performance-based contracting, the
industry approach is much different from DOD's preferred approach for
performance-based logistics. As previously discussed, Office of the
Secretary of Defense guidance has over the past several years
encouraged the services to use performance-based logistics at the
weapon system level as the preferred approach for life-cycle management
of military systems. DOD officials have stated that this is an industry
best practice and should be adopted more aggressively, but in 7 of 14
companies we interviewed that used some type of performance-based
contracting, this agreement was used at the subsystem or component
level--that is, for engines, auxiliary power units, wheels, or brakes-
-and it was generally used for older systems.
The following chart characterizes the companies we interviewed by
industry type, by the extent to which they outsource logistics support
activities, by the predominant contracting practices used, and by the
types of subsystems or components outsourced using performance-based
contracting. Pseudonyms are used rather than the actual company names.
These companies generate annual revenue generally exceeding $1 billion,
and they use complex and expensive equipment for which they require
high levels of availability and reliability as well as efficiency in
managing lifecycle costs. The life-cycle management issues are
comparable to those of DOD in managing its weapon system sustainment
programs.
Table 1: Company Use of Contracting Tools to Outsource Subsystem and
Component Support:
Industry/company: Air carrier industry: Company A;
Percent outsourced: 65%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Airframes.
Industry/company: Air carrier industry: Company A;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, auxiliary power units
(APUs), avionics, wheels and brakes.
Industry/company: Air carrier industry: Company B;
Percent outsourced: 20%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Airframe, engines, avionics.
Industry/company: Air carrier industry: Company B;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: APUs, avionics, wheels and brakes.
Industry/company: Air carrier industry: Company C;
Percent outsourced: 38%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Engines.
Industry/company: Air carrier industry: Company C;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, APUs.
Industry/company: Air carrier industry: Company D;
Percent outsourced: 90%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Airframes, engines, APUs, avionics.
Industry/company: Air carrier industry: Company D;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, avionics, wheels and
brakes.
Industry/company: Air carrier industry: Company E;
Percent outsourced: 76%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Airframes, engines, avionics.
Industry/company: Air carrier industry: Company E;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, APUs, wheels and brakes.
Industry/company: Air carrier industry: Company F;
Percent outsourced: 33%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Airframes, engines, APUs, avionics.
Industry/company: Air carrier industry: Company F;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, APUs, brakes.
Industry/company: Energy exploration and mining industries: Company G;
Percent outsourced: 75%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Engines.
Industry/company: Energy exploration and mining industries: Company G;
Predominate type of contracting tool: Performance-based;
Subsystem or component outsourced: Engines, transmission, torque
converters, wheels and brakes.
Industry/company: Energy exploration and mining industries: Company H;
Percent outsourced: 5%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: All components.
Industry/company: Energy exploration and mining industries: Company I;
Percent outsourced: 65%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Engines, pumps.
Industry/company: Energy exploration and mining industries: Company J;
Percent outsourced: 65%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Engines, hydraulics, pumps,
transmissions.
Industry/company: Maritime and entertainment industries: Company K;
Percent outsourced: 75%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Subsystems/components.
Industry/company: Maritime and entertainment industries: Company L;
Percent outsourced: 75%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Subsystems/components.
Industry/company: Maritime and entertainment industries: Company M;
Percent outsourced: 35%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Subsystems/components.
Industry/company: Maritime and entertainment industries: Company N;
Percent outsourced: 20%;
Predominate type of contracting tool: Fixed price--time and material;
Subsystem or component outsourced: Subsystems/components.
Source: GAO analysis of company data.
[End of table]
As shown above, performance-based contracting in the companies we
interviewed is most widely used in the air carrier industry, and it
also has limited use in the energy exploration and mining industry.
According to air carrier officials, time and material contracts are
more prevalent than performance contracts, because industry prefers to
use short-term (2 to 3 years) competitive contracts when possible. In a
sole-source environment companies sometimes use longer-term (10 to12
years) performance-based contracts for supporting some subsystems such
as engines, if they have sufficient historical data to establish an
accurate baseline. For example, all but one of the air carrier industry
companies had performance-based contracts for one or more engines. The
amount of engine workload managed by performance-based contracts varied
from company to company. For example, Company C, which outsourced
38 percent of its total maintenance workload, used performance-based
contracts for one-fifth of its outsourced engine work; while Company A,
which outsourced 65 percent of its maintenance workload, used
performance-based contracts for all of its engine work. The air carrier
companies did not use performance-based contracts for contracted work
on airframes, work that generally comprises about 30 percent of the
commercial aviation maintenance and repair market.[Footnote 10] Table 2
provides information regarding the percentage of dollars spent on the
repair of each type of subsystem or component managed using
performance-based contracts by the air carrier companies and the one
non-air carrier company that used performance-based contracts. The
subsystems or components for which the companies used performance-based
contracts most widely were auxiliary power units and wheels and brakes.
Table 2: Percentage of Maintenance Dollars for Outsourced Subsystems
and Components Managed Using Performance-Based Contracts:
Out sourced subsystems and components: Engines;
Companies: A: 100%;
Companies: B: 0%;
Companies: C: 20%;
Companies: D: 90%;
Companies: E: 57%;
Companies: F: 28%;
Companies: G: 45%.
Out sourced subsystems and components: Auxiliary power units;
Companies: A: 90%;
Companies: B: 100%;
Companies: C: 100%;
Companies: D: 0%;
Companies: E: 100%;
Companies: F: 41%;
Companies: G: N/A.
Out sourced subsystems and components: Avionics;
Companies: A: 36%;
Companies: B: 35%;
Companies: C: 0%;
Companies: D: 5%;
Companies: E: 0%;
Companies: F: 0%;
Companies: G: N/A.
Out sourced subsystems and components: Wheels and brakes;
Companies: A: 100%;
Companies: B: 100%;
Companies: C: 0%;
Companies: D: 100%;
Companies: E: 100%;
Companies: F: 80%[A];
Companies: G: 25%.
Out sourced subsystems and components: Transmission and torque
converters;
Companies: A: N/A;
Companies: B: N/A;
Companies: C: N/A;
Companies: D: N/A;
Companies: E: N/A;
Companies: F: N/A;
Companies: G: 100%.
Source: GAO analysis of company data.
Note: N/A = Not applicable because some subsystems and components are
not applicable to all industries.
[A] Includes only brakes. Wheels are maintained in-house.
[End of table]
Company officials noted that performance-based contracts are a tool
most often used selectively in a noncompetitive environment in an
effort to control cost and reduce risk. Additionally, they said that
performance-based contracting works better for subsystems and
components where available cost and performance data are sufficient to
establish a good business case analysis, noting that this is more
difficult to accomplish for new systems, where performance data are
uncertain. Performance-based contracts differ from traditional
logistics contracts by focusing on the purchase of weapon system
sustainment as an integrated package based on output measures--such as
a predetermined level of system availability. In contrast, traditional
transaction-based time and material contracts are used to purchase
logistics inputs--such as quantities of spare parts, specific repair
tasks, and engineering studies. Under transaction-based contracts, the
government pays for each transaction as a separate deliverable; whereas
under a performance-based contract, the contractor is being paid for
achieving an outcome performance metric, regardless of what he does to
achieve that performance.
In concept, performance-based contracts encourage the contractor to
achieve a high level of performance at a fixed cost. However, air
carrier industry officials we interviewed said that entering into a
performance-based contract without good baseline data introduces a
higher level of risk that the arrangement may not be cost-effective.
For example, officials from one company said they used a performance-
based contract for the older of the two types of engines in the
company's inventory. Officials said they would wait to collect
sufficient performance data on the newer engine before considering a
performance-based contract. The officials noted that they had
originally used a performance-based contract on the newer engine, but
found that, because the reliability of the engine was greater than
expected, the contract arrangement was not cost-effective. The company
was able to change the contract to a time and material contract, to
allow time to collect sufficient performance data to support a fact-
based business case analysis to determine the company's "should" cost
amount for a performance-based contract.
Performance-based contracting offers DOD opportunities to
provide contractors incentives to achieve desired levels of operational
performance at a fixed cost when the department has historical
performance information. But in the absence of reliable and
complete performance data as a baseline, the adoption of this approach
as the preferred support strategy for new weapon systems could
undermine DOD's ability to negotiate cost-effective terms--
particularly since the performance-based contracts at the weapon system
level have cost-reimbursement elements, while the private-sector
companies generally used fixed-price agreements. Private-sector
officials noted that it is important to use fixed prices for materials,
since the high price of materials is a key factor driving the companies
to use performance-based contracts.
Private Sector Applies Performance-Based Logistics at the Subsystem or
Component Level Rather Than at the Platform Level and Retains System
Integrator Function:
DOD policy promotes using performance-based contracting differently
from the way private-sector firms use it in supporting complex and
expensive systems. The companies we reviewed generally used
performance-based contracting at the subsystem level for engines and
certain other components rather than at the platform level, as proposed
by DOD. Furthermore, when using performance-based contracting, these
companies do not contract out the program integration function, as the
military services are doing.
We found no performance-based contracts for maintenance of airframes or
maintenance of any equipment platform among the private-sector
companies we reviewed. Industry officials cited three reasons why they
believe the use of performance contracts is more advantageous at the
subsystem or component level. First, they prefer to take advantage of
competition whenever it is available and to manage support contracts
through the use of competitive procedures. For example, because
airframe maintenance support is available from a competitive market,
the companies generally use a combination of fixed price and time and
material contracts for this category of service. Conversely,
performance-based contracts are often used for engine repair because of
the high cost of spare and repair parts that are available only from
the original equipment manufacturer. Officials said that there are too
few third-party repair vendors to foster competition. Second, company
officials emphasized the importance of gaining purchasing power from
volume discounts on subsystems or components across their entire fleet
of systems as a reason for not implementing performance contracting at
the platform level. Finally, by having contracts at the subsystem or
component level, companies can avoid the administrative costs that
would be charged by a prime integrator.
Similar to the approach used by the companies we reviewed, we noted
that the Navy has used performance-based contracts primarily at the
subsystem or component level. Navy officials said that implementation
at this level is easier because the service could implement this
concept more readily under DOD's current funding structure. The funding
is handled through the working capital fund, with reimbursement to the
fund coming from the sale of subsystems or components to the fleet.
Navy officials also noted that by implementing performance-based
logistics at this level, they can save money by competing subsystems or
components where a competitive market exists, consolidating the
requirements of multiple programs and leveraging their buying power to
obtain a pricing advantage, and reducing administrative costs--
advantages also recognized by the private sector.
The Navy's history of using a performance-based contract for logistics
support of the T-45 trainer aircraft illustrates how savings may be
achieved by implementing the concept at the subsystem level rather than
the weapon system level. The program office originally had a
performance-based contract for the entire weapon system with the
original equipment manufacturer. The contract was a 5-year firm-fixed
price with an option for a sixth year period. Program office officials
said the sole metric used, ready-for-training aircraft, resulted in
there being an insufficient number of aircraft available to fly
scheduled training sorties. Additionally, because actual flying hours
were fewer than forecasted, the Navy was paying for flying hours it was
not flying. Concluding that benefits weren't as expected, that the
costs were too high, and that savings were achievable by negotiating
separate contracts for the airframe and engine, the program office
chose not to exercise the option. The new engine contract is a
performance-based contract awarded on a sole-source basis to the engine
manufacturer, and the airframe performance-based contract was awarded
competitively. According to Navy program office officials, the revised
approach resulted in a projected savings of $37 million in the first
year and projected savings of $144 million at the end of a 5-year
period.[Footnote 11] The savings are being achieved through elimination
of the administrative costs charged by the prime contractor for the
engine work and through competition for the aircraft system.
Another potential adverse effect of awarding a performance-based
contract at the weapon system level is the loss of management control
and expertise over the system that private-sector firms said was
essential to the success of their business operations. Industry
officials said that managing their supplier base and ensuring the
availability of their equipment to generate revenue is too critical to
entrust to a second party. Further, they believe that contracting out
support at a platform level by using a system integrator limits the
potential to optimize savings through competition and volume discounts
and adds administrative costs charged by the prime integrator for
managing subcontractors.
Not Obtaining Sufficient Rights to Technical Data Could Limit Long-Term
Support Options and Could Increase Long-Term Costs:
The spokespersons for every company we visited told us that when they
purchase equipment they make sure to acquire the technical
data[Footnote 12] necessary to support it, regardless of whether the
company intends to support the equipment in-house or outsource some of
its support operations. Company officials said that this data was
essential to their own management and oversight functions. For example,
officials from a company that outsources most of its repairs pointed
out that its engineers use the data to perform such tasks as
establishing reliability metrics, evaluating performance, and revising
repair standards. Additionally, officials stated that owning the
technical data afforded their companies the flexibility that enabled
them either to perform the work in-house or to offer the work up for
competition. Several company officials said that it is best to obtain
the technical data at the time the equipment is purchased, when the
buyer has the most leverage in its negotiations with the manufacturer.
Trying to obtain the technical data at a later time is difficult to
negotiate and more expensive. These companies do not price their
technical data items separately. DOD program offices, however,
negotiate a price for maintenance-and-repair technical data separately
from the price of the military hardware systems. According to service
competition-advocate officials, program managers faced with limited
acquisition dollars often make trade-off decisions to buy increased
weapon system capability in lieu of technical data.
We reported in 2002 that DOD program offices have often failed to put
adequate emphasis on obtaining needed technical data during the
acquisition process.[Footnote 13] We recommended that DOD emphasize the
importance of obtaining technical data and consider including a priced
option for the purchase of technical data when considering proposals
for new weapon systems or modifications to existing systems. DOD
concurred with our recommendation, noting that there was a requirement
in DOD 5000.2R for program offices to provide long-term access to data
required for the competitive sourcing of systems support throughout the
life cycle. Additionally, by implementing total life-cycle systems
management, DOD would strengthen its emphasis on acquiring technical
data when negotiating support agreements with logistics providers.
Nonetheless, the DOD has further diminished the emphasis it places on
the need to acquire rights to technical data. For example, in May 2003,
DOD replaced its acquisition regulation with a streamlined
instruction,[Footnote 14] which eliminated the prior regulation's
requirement for the program manager to provide for long-term access to
data required for the competitive sourcing of weapon system support
throughout the life cycle of the system. This language is now provided
as guidance in the Interim Defense Acquisition Guidebook, but it is not
mandatory that this guidance be followed.
According to DOD and service logistics officials, program managers
should develop strategies that provide the government with sufficient
and affordable technical data rights to enable them to put the work out
for competition or develop alternate public or private sources for
weapon system support if performance-based logistics arrangements fail
or become too expensive. Logistics officials recognize that program
managers who implement performance-based logistics contracts on new
weapon systems may wish to delay taking delivery of technical data
early in the life of the system, because unlike the stable designs of
commercial equipment purchased in the private sector, the data for
cutting edge technology lacks maturity and is frequently changed.
Alternatively, program managers sometimes pay the original equipment
manufacturers both to maintain the technical and weapon system
configuration data and to provide the program managers with sufficient
access to enable them to manage and oversee the performance-based
logistics contract. However, logistics officials agree that the product
support strategy should clearly provide for the future delivery of the
technical data when required to support competition or alternative
source development.
Service logistics and competition-advocate officials said that it is
critical that this strategy be developed during the weapon system
acquisition phase, when the program office has its greatest leverage in
negotiating the price of the technical data and the conditions under
which the manufacturer must deliver the data. For example, in the
course of the acquisition of the V-22 aircraft engine, the Navy program
office obtained a technical data license agreement, according to which
the manufacturer agreed to deliver a complete data package if it failed
to perform in compliance with the statement of work at the agreed-to
price and schedule. Conversely, when the program office does not obtain
the technical data at the time of purchase, the future costs for
obtaining these data are not knowable and, without the leverage of the
original package purchase, could be prohibitively expensive. In our
review of data collected from DOD's performance-based logistics program
offices, we noted that DOD had not negotiated for the maintenance
drawing packages for the Javelin missile, F-117 aircraft, and TOW
missile improved target acquisition system, and DOD would have to
purchase them at a later date at a price to be negotiated.
In April 2004, the Logistics Management Institute reported in a review
of performance-based logistics arrangements that it found no evidence
to indicate either the quantity or the quality of logistics management
data--including technical data--available to the government was
compromised by the use of performance-based logistics
arrangements.[Footnote 15] This report also noted, however, that the
acquisition guidance published by the Office of the Secretary of
Defense does not address strategies for terminating interim contractor
support or performance-based logistics contracts. The Logistics
Management Institute report recommended that the Defense Acquisition
Working Group include performance-based logistics "exit strategy"
guidance in the defense acquisition guidebook. Nonetheless, as we have
previously noted, guidance in this handbook is not mandatory.
Conclusions:
The use of performance-based contracting for the support of complex
and costly military systems offers opportunities for military program
managers to incentivize contractors to achieve desired levels of weapon
system performance. However, our review of the use of the practice in
private-sector firms indicates that DOD's proposed guidance to adopt
performance-based logistics aggressively at the platform level could
limit competition, and such guidance might not be the most cost-
effective approach for using this concept. Additionally, although DOD
based its rationale for using performance-based logistics at least
partially on the perception that this is an industry best practice, it
appears that perception is not the case. DOD's approach toward
implementing the concept appears inconsistent with the way private-
sector companies we interviewed use performance-based contracting in
acquiring support for their equipment, and DOD's approach has risks
that should be addressed as it develops its guidance for using
performance-based logistics.
Using performance-based logistics as the preferred approach for
managing the support of major weapon system programs--even though
private-sector company officials use performance-based contracting
selectively, when appropriate and cost-effective--carries the risk of
increasing life-cycle cost. Both private-and public-sector experiences
with performance-based contracting illuminate the challenges involved
in developing a meaningful baseline for establishing a performance-
based arrangement for new systems, because not enough is known early in
the program about performance characteristics and because there is risk
to both the program office and the contractor that may translate into
high cost. Additionally, the use of performance-based logistics can
limit the competition that would be available for providing logistics
support when support decisions are made at the subsystem or component
level rather than at the platform level. Using performance-based
logistics at the platform level also creates risk by contracting out
the program integration function--a core function that private
contractors consider essential for the cost-effective management of
costly and complex systems over their life cycle.
Finally, adopting performance-based logistics at the weapon system
platform level may be influencing program offices to obtain access only
to technical data necessary to manage the performance-based contract
during the acquisition phase--and not to provide a strategy for the
future delivery of technical data in case the performance-based
arrangement fails. In such a case, the program manager would have
limited flexibility in choosing whether to perform maintenance in-
house, select an alternative vendor, or offer the work for competition.
Recommendations for Executive Action:
In order for the department to improve the implementation of
performance-based logistics, we recommend that the Secretary of Defense
direct that the Under Secretary of Defense (Acquisition, Technology and
Logistics) and the Under Secretary of Defense (Comptroller) implement
the following two recommendations:
1. Incorporate in DOD's guidance to the services the private sector's
practice of using performance-based logistics as a tool to achieve
economies at the subsystem or component level, rather than as a
preferred practice at the platform level. Also, incorporate the private
sector's practice of using it when sufficient performance data are
available to establish a meaningful cost baseline and:
2. Consider requiring program offices, during weapon system
acquisition, to develop acquisition strategies that provide for the
future delivery of sufficient technical data to enable the program
office to select an alternate source--public or private--or to offer
the work out for competition if the performance-based arrangement fails
or becomes prohibitively expensive.
Agency Comments and Our Evaluation:
In commenting on a draft of this report, DOD concurred with our
recommendations to enhance the implementation of performance-based
logistics.
Regarding our recommendation to incorporate in its performance-based
logistics guidance to the services the private sector's practice of
using performance-based logistics as a tool to achieve economies at the
subsystem or component level, DOD's response stated that the department
recognizes the need to re-emphasize the use of performance-based
logistics for subsystems and components in its policy memorandum and
guide books. Nonetheless, the response noted that the department
believes that it is still prudent to pursue performance-based logistics
strategies at the platform level where supported by a business case
analysis. The private sector companies we interviewed noted that the
more cost effective alternative is to use competitive procedures where
practicable at the subsystem or component level supported by a cost
analysis using reliable performance data.
Regarding our comment that DOD also incorporate in its guidance the
private sector practice of using performance-based logistics when
sufficient performance data are available to establish a meaningful
cost baseline, DOD stated that its policy is that a business case
analysis should be performed to help make the determination to use
performance-based logistics or traditional logistics support
arrangement, and that the business case analysis incorporate the use of
performance data, if available, in establishing a meaningful cost
baseline. DOD stated that it will emphasize the use of performance data
in a revised policy memorandum on performance-based logistics. However,
based on information we obtained from the private sector companies we
interviewed, developing reliable cost and performance data to support a
valid cost analysis at the platform level for a new system will be a
challenge and may not be reliable in identifying the most cost-
effective support option over the life cycle of the system. As we noted
in our report, one company tried a performance-based contract for a new
engine but found that because the reliability of the engine was greater
than expected, this contract management was not cost-effective. Company
officials said they preferred to collect reliable performance data over
a period of time to support negotiations for a performance-based
contract.
In response to our recommendation to consider requiring program offices
to develop acquisition strategies that provide for the future delivery
of sufficient technical data to select an alternate source--public or
private--or to offer the work out for competition if the performance-
based arrangement fails or becomes prohibitively expensive, DOD stated
that it will take steps to address this issue in the next iteration of
the DOD Directive 5000.1 and DOD Instruction 5000.2 acquisition
regulation policy. According to the response, the new policy will
require the program manager to establish a data management strategy
that requires access to the minimum data necessary to sustain the
fielded system, recompete or reconstitute sustainment if necessary,
promote real time access vice delivery of the data, and provide for the
availability of quality data at the point of need for the intended
user. According to DOD, for performance-based logistics arrangements,
these actions will include acquiring the appropriate technical data to
support an exit strategy should the arrangement fail or become too
expensive.
Scope and Methodology:
The objectives of our review were to determine (1) what types of
contractor support practices the private sector used to support complex
and costly equipment that have life-cycle management issues similar to
military weapons systems and (2) what potential lessons could be
learned through a comparison of private sector contractor logistics
support practices that DOD currently uses, or plans to use, under its
implementation of performance-based logistics.
To identify commercial industries that use complex and costly equipment
with life-cycle management issues similar to military weapon systems,
we interviewed DOD depot maintenance and logistics policy officials. We
also conducted a literature search to identify appropriate industry
groups and interviewed officials from the Industrial College of the
Armed Forces, the Aerospace Industries Association, the American
Association of Port Authorities, International Council of Cruise Lines,
the Society for Mining Metallurgy and Exploration, the Construction
Industry Institute, and the Council of Logistics Management to validate
and refine the identified industries and to identify appropriate
candidate companies within the industry groups. Within the air carrier,
maritime shipping, energy exploration, mining, and entertainment
industries, we identified over 250 companies and selected 67 companies
based on sales/revenues, production rankings, and management awards
that might be good candidates for our study. We eliminated three
companies that did not outsource significant amounts of logistics
support, and 50 companies either did not respond to our initial
inquires or declined to participate in the study. Fourteen companies
agreed to participate and completed our interviews and follow-up
questions. Thirteen of the 14 companies we interviewed agreed to be
identified and are listed below by industry group: Air carriers
(Continental Airlines, Houston, Texas; Delta Air Lines, Atlanta,
Georgia; FedEx Corp., Memphis, Tennessee; Southwest Airlines, Dallas,
Texas; and United Airlines, San Francisco, California); Energy
Exploration and Mining (British Petroleum, Houston, Texas; Diamond
Offshore, Houston, Texas; Phelps Dodge, Phoenix, Arizona; and Vulcan
Material, Birmingham, Alabama); Maritime (Carnival Cruises, Miami,
Florida; Conoco Philips Polar Tanker, Long Beach, California; and
Disney Cruise, Orlando, Florida); and Entertainment (Disney World,
Orlando, Florida).
To identify private sector support practices, including performance-
based logistics, we conducted group discussions with respective company
officials responsible for maintenance and support operations,
budgeting, and contracting. To collect consistent information among the
companies, we developed standard group discussion questions based on
our literature search and discussions with industry experts. We also
included questions to determine how the companies addressed logistics
and contracting issues similar to those that DOD faced in implementing
performance-based logistics. We analyzed the responses to identify the
prevailing industry practices in supporting complex and costly
equipment, especially focusing on the contracting approaches and
practices used to outsource support functions and activities.
We reviewed and discussed with Office of the Secretary of Defense and
military department officials at the headquarters and major acquisition
commands the department's plans, policies, and procedures for using
performance-based logistics. We also collected policy and guidance
(published and under development) by the Office of the Secretary of
Defense as well as the military departments' policies and
implementation plans.
To assess what lessons could be drawn from the private sector
companies' experiences to guide DOD's logistics support efforts, we
interviewed DOD officials and reviewed ongoing logistics programs. We
assessed the reliability of the projected cost and savings data we used
in this report by reviewing supporting documentation and interviewing
knowledgeable personnel; and we determined that it was sufficient for
our purposes. We compared and contrasted the contract logistic
approaches and practices used by private sector activities with those
currently used by DOD and envisioned under its plans for implementing
performance-based logistics. This comparison included such elements as
the (1) use of performance-based contracting and the extent of its
application, (2) assigning a single integrator for equipment or weapons
system maintenance and logistics support on a platform level,
(3) management and oversight including the importance of technical
data, and (4) the degree of competitive sourcing and the importance of
leveraging purchasing power. As part of our continuing review we are
also conducting case studies on DOD performance-based logistics weapon
systems to further compare the new DOD approach and practices with
those of the private sector. This work is continuing and we expect to
complete our final report early in 2005.
We performed our work from September 2003 through June 2004 in
accordance with generally accepted government auditing standards.
We are sending copies of this report to the appropriate congressional
committees, and it will be available at no charge on GAO's Web site at
http://www.gao.gov. We are continuing with our review of performance-
based logistics in the private sector and in DOD and plan to report the
results early in 2005.
If you or your staff have any questions on the matters discussed in
this letter, please contact me at (202) 512-8412 or solisw@gao.gov or
my assistant director, Julia Denman, at (202) 512-4290 or
denmanj@gao.gov. Larry Junek, Thom Barger, Pamela Valentine, Judith
Collins, and Cheryl Weissman were major contributors to this report.
Signed by:
William Solis, Director:
Defense Capabilities and Management:
[End of section]
Appendix I: Comments from the Department of Defense:
Note: A GAO comment supplementing those in the report text appears at
the end of this appendix.
Note: Page numbers in the draft report may differ from those in this
report.
DEPUTY UNDER SECRETARY OF DEFENSE FOR LOGISTICS AND MATERIEL READINESS:
3500 DEFENSE PENTAGON:
WASHINGTON, DC 20301-3500:
July 30, 2004:
Mr. William Solis:
Director, Defense Capabilities and Management:
U.S. General Accounting Office:
441 G Street, N.W.:
Washington, DC 20548:
Dear Mr. Solis,
This is the Department of Defense (DoD) response to the GAO draft
report, "DEFENSE MANAGEMENT: Opportunities to Enhance the
Implementation of Performance-Based Logistics," dated July 6, 2004 (GAO
Code 350424/GAO-04-715).
The Department concurs with the findings and recommendations presented
in the report. The findings indicate that industry on the whole uses
Performance-Based Logistics (PBL)-type arrangements on the subsystem or
component level, as opposed to an entire platform. The DoD will re-
emphasize via policy and training the use of PBL at the subsystem and
component level as well as the platform level where viable. Regarding
the finding on technical data, the Department concurs, and will take
steps to update Acquisition Policy documents such as the 5000 series of
DoD regulations to include guidance to Program Managers on purchasing
rights or long term access to technical data.
Detailed DoD comments on the draft GAO recommendations are provided in
the enclosure. The DoD appreciates the opportunity to comment on the
draft report.
Sincerely,
Signed for:
Bradley Berkson
Acting:
Attachment As stated:
GAO DRAFT REPORT - DATED JULY 6, 2004 GAO CODE 350424/GAO-04-715:
"DEFENSE MANAGEMENT: Opportunities to Enhance the Implementation of
Performance-Based Logistics":
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) and the Under Secretary of Defense (Comptroller) incorporate
in DoD's guidance to the Services the private sector's practice of
using performance-based logistics as a tool to achieve economies at the
subsystem or component level, rather than as a preferred practice at
the platform level. Also, incorporate the private sector's practice of
using it when sufficient performance data are available to establish a
meaningful cost baseline. (Page 22/GAO Draft Report):
DOD RESPONSE: Concur. The DoD believes it is prudent to pursue PBL
strategies at the platform level where supported by a business case
analysis, but we also recognize the need to re-emphasize the use of PBL
for subsystems and components. The vast majority of current DoD PBLs
are at the component or subsystem level. The Department will continue
to focus on that aspect in future policy memoranda and guidebooks.
Regarding the private sector practice of using PBLs when there is
sufficient performance data to establish a meaningful cost baseline,
the Department has stated in policy that a Business Case Analysis
should be performed to help make the determination to use PBL, and the
BCA should incorporate use of performance data, if available, in
establishing a meaningful cost baseline. The Department will emphasize
this in our upcoming revised policy memoranda on PBL.
RECOMMENDATION 2: The GAO recommended that the Secretary of Defense
direct the Under Secretary of Defense (Acquisition, Technology and
Logistics) and the Under Secretary of Defense (Comptroller) to consider
requiring program offices, during weapon system acquisition, to develop
acquisition strategies that provide for future delivery of sufficient
technical data to enable the program office to select an alternate
source-public or private-or to offer the work out for competition if
the performance-based arrangement fails or becomes prohibitively
expensive. (Page 22/GAO Draft Report):
DOD RESPONSE: Concur. As stated in the draft report on pages 18-19, the
DoD had previously required Program Managers to ensure long term access
to technical data in the previous edition of the DoD Regulation 5000.2-
R, but this was rescinded in 2002. The Department concurs that
technical, product, and logistics data should be acquired by the PM to
support the development, production, operation, sustainment,
improvement, demilitarization
and disposal of a system. Data management guidance has been added to
the Defense Acquisition Guidebook and the draft updated "Performance
Based Logistics: A Program Manager's Product Support Guide" which is
currently in coordination. Both of these documents convey the
importance of having a Data Management strategy that considers life
cycle requirements in the decision to acquire data for use throughout
the product life cycle. However, since these documents are guidance in
nature and not considered "mandatory" by the Program Managers, the
Department will take steps to include, in the next iteration of the
DoDD 5000.1 and DoDl 5000.2 acquisition regulations, policy that will
require the program manager to establish a data management strategy
that requires access to the minimum data necessary to sustain the
fielded system, recompete or reconstitute sustainment if necessary,
promote real time access vice delivery of data, and provide for the
availability of quality data at the point of need for the intended
user. For a PBL arrangement, this will include acquiring the
appropriate technical data needed to support an exit strategy should
the PBL fail or become too expensive.
GAO's Comment:
1. Only 7 of the 14 companies we interviewed use some type
of performance-based contracting arrangements. None of the
performance-based arrangements in the seven companies using them were
at the platform level.
[End of section]
FOOTNOTES
[1] There is a performance-based logistics agreement between the
program office and the Tobyhanna Army Depot to support the Common
Ground Station.
[2] This equipment includes airline and air cargo aircraft, cruise
ships and oil tankers, heavy mining equipment, offshore drilling and
production platforms, and unique electronic equipment.
[3] Initially the Navy had one contract for the entire T-45 aircraft
system and had only one metric, aircraft availability, for evaluating
the contract. Under this approach the Navy was paying the contractor
based on its forecasted flying hours rather than actual hours. Because
forecasted hours were more than actual hours, the Navy paid for hours
it was not flying. The Navy added the second metric and a fixed labor
rate for over-and-above work when it revised the T-45 performance
approach and negotiated separate contracts for the aircraft and engine
systems.
[4] DOD officials suggest this might involve establishing a single line
of accounting for all operations and maintenance funding for a program
that would be managed by the system program office. Today, operation
and maintenance funds are managed by different parties including the
operational commands, the weapon system managers, and the program
offices. We have previously reported that warfighters have expressed a
concern about the loss of flexibility of operational commanders when
system operation and maintenance funding is fenced and controlled by
the program manager. See U.S. General Accounting Office, Defense
Logistics: Air Force Lacks Data to Assess Contractor Logistics Support
Approaches, GAO-01-618 (Washington, D.C.: Sept. 7, 2001) and Defense
Logistics: Opportunities to Improve the Army's and Navy's Decision-
making Process for Weapons Systems Support, GAO-02-306 (Washington,
D.C.: Feb. 28, 2002).
[5] In October 2000, Congress passed section 821 (b) of the National
Defense Authorization Act for Fiscal Year 2001, which allows DOD to
treat performance-based service contracts or task orders as contracts
for the procurement of commercial items under certain conditions
(Public Law No. 106-398).
[6] U.S. General Accounting Office, Best Practices: Setting
Requirements Differently Could Reduce Total Ownership Costs, GAO-03-57
(Washington, D.C.: Feb. 11, 2003), emphasized the need to include total
ownership cost goals and readiness rates as performance metrics equal
to any others in the development of major weapon systems.
[7] Supply chain management refers to all of the inter-related
components and processes required to ensure that the correct amount of
product is in the correct location at the right time and at the lowest
cost.
[8] The Supply Chain/Performance-Based Logistics Task Group's October
15, 2003, report to the Senior Executive Council provided this group's
perspective regarding what needed to be done to implement performance-
based logistics in the Department of Defense.
[9] Memorandum from the Deputy Secretary of Defense to the Secretaries
of the Military Departments and the Under Secretary of Defense
(Comptroller), February 4, 2004.
[10] Additionally, 29 percent of the total is for engines, 23 percent
for line maintenance, and 19 percent for components.
[11] The savings estimates were calculated independent of the program
office by the Naval Air System Command's cost estimators within the
Research and Engineering Competency, and they follow a standard
methodology called the "Maintenance and Trade Analysis." A baseline was
established by updating the original baseline cost analysis supporting
the initial contract with actual costs from the 2001 through 2004. The
calculation also quantified the pass-through administrative cost
charged by the prime contractor for the engine work. This analysis
identified that the Navy would save $118 million on the new
competitively awarded airframe contract and $26 million on the new
engine contract.
[12] Technical data includes detail maintenance drawings and repair
publications containing specifications and tolerances.
[13] U.S. General Accounting Office, Defense Logistics: Opportunities
to Improve the Army's and Navy's Decision-making Process for Weapons
Systems Support, GAO-02-306 (Washington, D.C.: Feb. 28, 2002).
[14] DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense
Acquisition Programs and Major Automated Information System Acquisition
Programs was replaced by DOD instruction 5000.2, Operation of the
Defense Acquisition System, to create a simplified and flexible
management framework for acquisition.
[15] Logistics Management Institute, Visibility of Maintenance Data in
Performance-Based Logistics Arrangements, LG301L4 (McLean, Va.: Apr.
2004.)
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
