Defense Contracting
Enhanced Training Could Strengthen DOD's Best Value Tradeoff Decisions Gao ID: GAO-11-8 October 28, 2010The Department of Defense (DOD) obligated about $380 billion in fiscal year 2009 to acquire products and services. One approach DOD can take to evaluate offerors' proposals is the best value tradeoff process in which the relative importance of price varies compared to non-cost factors. The National Defense Authorization Act for Fiscal Year 2010 required GAO to review DOD's use of the best value tradeoff process, specifically when non-cost factors were more important than price. In response, GAO determined (1) how often and for what types of contracts DOD used the best value tradeoff process; (2) why and how DOD used such an approach; and (3) challenges, if any, DOD faces in using the best value tradeoff process. GAO identified a probability sample of new, competitively awarded fiscal year 2009 contracts in which DOD obligated $25 million or more. GAO reviewed guidance, solicitations, source selection decisions, and other documents for 129 contracts and interviewed DOD contracting and program staff about the use of the best value tradeoff process.
In fiscal year 2009, DOD used best value processes for approximately 95 percent of its new, competitively awarded contracts in which $25 million or more was obligated. Almost half of DOD's contracts--47 percent--were awarded using a tradeoff process in which non-cost evaluation factors, when combined, were more important than price. DOD used best value tradeoffs principally to acquire services, such as construction of troop housing, as well as for professional management services. DOD used the best value tradeoff process in 88 of the 129 contracts GAO reviewed. For 60 of the 88 contracts, DOD weighted non-cost factors as more important than price. In these cases, DOD was willing to pay more for a contractor that demonstrated it understood complex technical issues more thoroughly, could provide a needed good or service to meet deadlines, or had a proven track record in successfully delivering products or services of a similar nature. In making tradeoff decisions, GAO found that DOD selected a lower priced proposal nearly as often as it selected a higher technically rated, but more costly proposal. Overall, GAO found that DOD paid a combined total of more than $230 million in price differentials--the difference in price between the awardee and the offeror next in line for award--on 21 contracts, but chose not to pay more than $800 million in proposed costs by selecting a lower priced offer over a higher technically rated offer in 18 contracts. DOD does not track whether the use of best value tradeoff processes correlates with the contractor successfully meeting the terms of the contract and noted that many factors ultimately contribute to an acquisition's success or failure. DOD officials identified several challenges in using the best value tradeoff process, including the difficulty in determining meaningful evaluation factors and the business judgment of acquisition staff required. DOD officials also noted that the complexity of the tradeoff process increases the risk of bid protests. For example, GAO found that 15 of the 88 contracts awarded using a best value tradeoff process reviewed were protested to GAO, resulting in 4 cases in which DOD terminated the contract or made a new source selection decision when DOD determined that it failed to adhere to the solicitations' requirements. Such concerns are heightened given the expected influx of more than 6,400 new contracting personnel over the next few years. According to DOD officials, making sound tradeoff decisions, and in particular, deciding whether or not a price differential is warranted, is one of the most difficult aspects of using a best value tradeoff process. DOD is developing a new departmentwide source selection guide and intends to subsequently revise its training curriculum, but neither the guide nor DOD's current training curriculum provides agency personnel with information on assessing price differentials when performing tradeoff analyses. GAO recommends that to help DOD effectively employ best value tradeoff processes, DOD develop training elements, such as case studies, that focus on reaching tradeoff decisions, as it updates its training curriculum. DOD concurred with this recommendation.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: John P. Hutton Team: Government Accountability Office: Acquisition and Sourcing Management Phone: (202) 512-7773GAO-11-8, Defense Contracting: Enhanced Training Could Strengthen DOD's Best Value Tradeoff Decisions
This is the accessible text file for GAO report number GAO-11-8
entitled 'Defense Contracting: Enhanced Training Could Strengthen DOD‘s
Best Value Tradeoff Decisions' which was released on October 28, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
October 2010:
Defense Contracting:
Enhanced Training Could Strengthen DOD's Best Value Tradeoff Decisions:
Best Value Contracting:
GAO-11-8:
GAO Highlights:
Highlights of GAO-11-8, a report to congressional committees.
Why GAO Did This Study:
The Department of Defense (DOD) obligated about $380 billion in fiscal
year 2009 to acquire products and services. One approach DOD can take
to evaluate offerors‘ proposals is the best value tradeoff process in
which the relative importance of price varies compared to non-cost
factors. The National Defense Authorization Act for Fiscal Year 2010
required GAO to review DOD‘s use of the best value tradeoff process,
specifically when non-cost factors were more important than price. In
response, GAO determined (1) how often and for what types of contracts
DOD used the best value tradeoff process; (2) why and how DOD used such
an approach; and (3) challenges, if any, DOD faces in using the best
value tradeoff process. GAO identified a probability sample of new,
competitively awarded fiscal year 2009 contracts in which DOD obligated
$25 million or more. GAO reviewed guidance, solicitations, source
selection decisions, and other documents for 129 contracts and
interviewed DOD contracting and program staff about the use of the best
value tradeoff process.
What GAO Found:
In fiscal year 2009, DOD used best value processes for approximately 95
percent of its new, competitively awarded contracts in which $25
million or more was obligated. Almost half of DOD‘s contracts”47
percent”were awarded using a tradeoff process in which non-cost
evaluation factors, when combined, were more important than price. DOD
used best value tradeoffs principally to acquire services, such as
construction of troop housing, as well as for professional management
services.
DOD used the best value tradeoff process in 88 of the 129 contracts GAO
reviewed. For 60 of the 88 contracts, DOD weighted non-cost factors as
more important than price. In these cases, DOD was willing to pay more
for a contractor that demonstrated it understood complex technical
issues more thoroughly, could provide a needed good or service to meet
deadlines, or had a proven track record in successfully delivering
products or services of a similar nature. In making tradeoff decisions,
GAO found that DOD selected a lower priced proposal nearly as often as
it selected a higher technically rated, but more costly proposal.
Overall, GAO found that DOD paid a combined total of more than $230
million in price differentials”the difference in price between the
awardee and the offeror next in line for award”on 21 contracts, but
chose not to pay more than $800 million in proposed costs by selecting
a lower priced offer over a higher technically rated offer in 18
contracts. DOD does not track whether the use of best value tradeoff
processes correlates with the contractor successfully meeting the terms
of the contract and noted that many factors ultimately contribute to an
acquisition‘s success or failure.
DOD officials identified several challenges in using the best value
tradeoff process, including the difficulty in determining meaningful
evaluation factors and the business judgment of acquisition staff
required. DOD officials also noted that the complexity of the tradeoff
process increases the risk of bid protests. For example, GAO found that
15 of the 88 contracts awarded using a best value tradeoff process
reviewed were protested to GAO, resulting in 4 cases in which DOD
terminated the contract or made a new source selection decision when
DOD determined that it failed to adhere to the solicitations‘
requirements. Such concerns are heightened given the expected influx of
more than 6,400 new contracting personnel over the next few years.
According to DOD officials, making sound tradeoff decisions, and in
particular, deciding whether or not a price differential is warranted,
is one of the most difficult aspects of using a best value tradeoff
process. DOD is developing a new departmentwide source selection guide
and intends to subsequently revise its training curriculum, but neither
the guide nor DOD‘s current training curriculum provides agency
personnel with information on assessing price differentials when
performing tradeoff analyses.
What GAO Recommends:
GAO recommends that to help DOD effectively employ best value tradeoff
processes, DOD develop training elements, such as case studies, that
focus on reaching tradeoff decisions, as it updates its training
curriculum. DOD concurred with this recommendation.
View [hyperlink, http://www.gao.gov/products/GAO-11-8] or key
components. For more information, contact John Hutton at (202) 512-4841
or huttonj@gao.gov.
[End of section]
Contents:
Letter:
Background:
DOD Relies Heavily on Best Value Processes to Evaluate Contractor
Offers:
DOD Used a Best Value Tradeoff Process to Address Complex or Time
Sensitive Needs, but Paid Relatively Few Price Differentials:
DOD Faces Several Challenges in Using the Best Value Tradeoff Process:
Conclusions:
Recommendation for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Staff Acknowledgments:
Table:
Table 1: Estimated Percentages of Source Selection Approaches Reviewed:
Figures:
Figure 1: DOD Contract Obligations in Fiscal Year 2009 (Dollars in
billions):
Figure 2: Estimated Frequencies of Source Selection Approaches Used in
Fiscal Year 2009 for New, Competitively Awarded DOD Contracts
Obligating over $25 million:
Figure 3: Estimated Frequencies of Selected Characteristics of
Contracts or Agreements Awarded using a Best Value Tradeoff Process:
Figure 4: Most Frequently Selected Non-cost Evaluation Factors and
Their Relative Importance in 88 Contracts Using a Tradeoff Process:
Figure 5: Price Differentials in the 68 DOD Contracts Reviewed in Which
a Tradeoff Analysis Was Conducted:
Abbreviations:
ACE: Acquisition Center of Excellence:
DAU: Defense Acquisition University:
DFARS: Defense Federal Acquisition Regulation Supplement:
DLA: Defense Logistics Agency:
DOD: Department of Defense:
FAR: Federal Acquisition Regulation:
FPDS-NG: Federal Procurement Data System-Next Generation:
IDIQ: Indefinite Delivery Indefinite Quantity:
LPTA: Lowest Price Technically Acceptable:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
October 28, 2010:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Department of Defense (DOD) obligated about $380 billion in fiscal
year 2009[Footnote 1] to acquire products and services needed to
support its missions. DOD has several approaches it can use to evaluate
offerors' proposals. For example, DOD can use a best value tradeoff
process, in which it can vary the relative importance of cost or price
to other factors, such as technical approach or past performance in its
solicitations to offerors. In these cases, DOD may award a contract to
other than the lowest-priced offeror if DOD determines that a higher-
priced proposal provides a greater benefit to DOD, and this greater
benefit is worth paying an additional cost, or price differential.
Proper execution of the tradeoff process is essential, however, to
ensure that DOD lays the foundation for successful acquisition
outcomes. We have identified DOD contract management as a long-standing
high-risk area, due in part to DOD's use of ill-suited business
arrangements that have contributed to unmet expectations and placed the
department at risk of potentially paying more than necessary.[Footnote
2]
Section 845 of the National Defense Authorization Act for Fiscal Year
2010 directed GAO to report on DOD's use of the best value tradeoff
process, and specifically for cases in which DOD determined that it
would evaluate contractors' proposals on factors other than cost or
price, if these non-cost factors, when combined, were to be considered
more important than cost or price.[Footnote 3] To respond to the
mandate, we determined (1) how often and for what types of contracts
DOD used the best value tradeoff process; (2) why and how DOD used the
best value tradeoff process; and (3) what challenges, if any, DOD faces
in using the best value tradeoff process.
To conduct our work we used the Federal Procurement Data System-Next
Generation (FPDS-NG)[Footnote 4] to identify new, competitively awarded
contracts in which DOD obligated $25 million or more in fiscal year
2009. We selected the $25 million threshold based on a Defense Federal
Acquisition Regulation Supplement (DFARS) requirement that contracts
for production or services with $25 million or more in estimated total
costs for any fiscal year have written acquisition plans, which contain
information on the anticipated source selection approach.[Footnote 5]
This analysis identified 363 contracts on which DOD had obligated a
total of $39.2 billion, or about 10 percent of the total amount DOD
obligated on contracts in fiscal year 2009.
From this population, we selected a random sample of 160 contracts,
including 60 indefinite delivery contracts.[Footnote 6] We identified
errors in the information provided by FPDS-NG on 31 contracts,
including contracts that were incorrectly coded as competitively
awarded or had incorrect award amounts.[Footnote 7] We excluded these
contracts from our sample and determined that FPDS-NG was sufficiently
reliable for the purposes of our review after adjusting for these
errors. Based on our analysis of the remaining 129 sample contracts, we
produced estimates of source selection approaches used, contract type,
and whether a product or service was acquired. To determine the type of
and rationale for the source selection process used for the contracts,
we reviewed the associated acquisition plans, solicitations, source
selection decision memorandums, and other relevant documents for each
of the contracts in our sample. In particular, we reviewed source
selection decision documents for contracts that used a tradeoff process
to determine whether DOD paid a price differential. For the purposes of
this report, we defined a price differential as the difference in price
between the awardee's price and the price of the offeror next in line
for award.
From our sample, we also judgmentally selected buying activities from
each military department and one defense agency based on such factors
as the number of best value contracts, the type of contract, and the
type of product or service acquired. Results based on these selected
buying activities are not generalizable to a larger population. At each
activity, we reviewed contract files and interviewed program and
contracting officials to discuss their rationale for selecting a
tradeoff process including the selection of non-cost evaluation factors
for 34 contracts and 23 task orders as illustrative case studies. We
also reviewed contract documents, DOD and military department source
selection guidance, and interviewed program and contracting officials
from DOD, the military departments, and one defense agency to identify
what challenges, if any, DOD faces in using the best value tradeoff
process. A more detailed description of our scope and methodology is
included in appendix I.
We conducted this performance audit from March 2010 through October
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
The Federal Acquisition Regulation (FAR) Part 15 allows the use of
several best value competitive source selection techniques to meet
agency needs. Within the best value continuum, DOD may choose an
approach that it considers the most advantageous to the government,
including the lowest price technically acceptable (LPTA) process and
the tradeoff process.
DOD may elect to use the LPTA process in acquisitions where the
requirement is clearly definable and the risk of unsuccessful contract
performance is minimal. In such cases, DOD may determine that cost or
price should play a dominant role in source selection. When using the
LPTA process, DOD specifies its minimal technical requirements in the
solicitation. Once DOD determines that the contractors meet or exceed
the technical requirements, no tradeoffs between cost or price and non-
cost factors are permitted and the award is made based on the lowest
price offered to the government.
By contrast, DOD may elect to use a tradeoff process in acquisitions
where the requirement is less definitive, more development work is
required, or the acquisition has greater performance risk. In these
instances, non-cost evaluation factors, such as technical capabilities
or past performance, may play a dominant role in the source selection
and tradeoffs among price and non-cost factors allow DOD to accept
other than the lowest priced proposal. This report focuses on DOD's use
of the tradeoff process, and specifically, in which non-cost factors,
when combined, were considered more important than cost or price.
When using a tradeoff process, the FAR requires that evaluation factors
and significant subfactors that affect contract award and their
relative importance be clearly stated in the solicitation; and the
solicitation must provide whether all evaluation factors other than
cost or price, when combined, are significantly more important than,
approximately equal to, or significantly less important than cost or
price[Footnote 8]. Additionally, the FAR requires that each factor
represent key areas of importance and emphasis to be considered in the
source selection decision and that support meaningful comparison and
discrimination between and among competing proposals[Footnote 9]. The
FAR also requires the source selection authority document the perceived
benefits of the higher priced proposal and the rationale for tradeoffs
in the contract file[Footnote 10]. The resulting source selection
decision should be based on a comparative assessment of proposals
against all source selection criteria in the solicitation. The decision
must also include the rationale for any business judgments and
tradeoffs made or relied on by the source selection official, including
benefits associated with additional costs. Although the rationale for
the source selection decision must be documented, the documentation
need not quantify the tradeoffs that led to the decision.
In fiscal year 2009, DOD obligated about $380 billion on contracts for
goods and services. Our analysis of data reported by DOD to FPDS-NG
indicates that $69.9 billion or 18 percent of DOD's obligations were
made on new contracts competitively awarded in fiscal year 2009 (see
figure 1). By contrast, about $176 billion were modifications to or
orders issued under contracts that were awarded prior to fiscal year
2009 and $133 billion were awarded non-competitively,[Footnote 11]
which in combination totaled to nearly 82 percent of DOD's reported
contract obligations in fiscal year 2009.
Figure 1: DOD Contract Obligations in Fiscal Year 2009[A] (Dollars in
billions):
[Refer to PDF for image: pie graph and bar graph]
Pie graph:
Non-competed contracts[B]: $133.2;
Completed Contracts: $264.4.
Bar graph:
New contracts: $39.2;
More than $25 million obligated.
New contracts: $30.7;
Less than $25 million obligated.
Orders and modifications on contracts awarded prior to FY 2009: $125.1;
More than $25 million obligated.
Orders and modifications on contracts awarded prior to FY 2009: $51.3;
Less than $25 million obligated.
Source: GAO analysis of FPDS-NG obligations data.
[A] Dollar amounts may not sum due to rounding.
[B] The non-competed contracts category consists of those contracts
awarded under other than full and open competition as defined in FAR
subpart 6.3.
[End of figure]
Properly managing the acquisition of goods and services requires an
acquisition workforce with the right skills and capabilities. In March
2009, however, we reported that DOD lacked complete information on the
skill sets of the current acquisition workforce and whether these skill
sets were sufficient to accomplish its missions.[Footnote 12] In April
2009, the Secretary of Defense announced his intent to grow the
acquisition workforce by 15 percent by fiscal year 2015. As part of
this strategy, DOD indicated that it intends to grow its contracting
career field by more than 6,400 personnel, an increase of more than 28
percent from fiscal year 2008 staffing levels.
DOD Relies Heavily on Best Value Processes to Evaluate Contractor
Offers:
DOD relies heavily on the use of the best value process to evaluate
offers from potential contractors. DOD chose a best value process for
approximately 95 percent of its new, competitively awarded contracts on
which it had obligated $25 million or more in fiscal year
2009.[Footnote 13] Almost half of DOD's contracts--47 percent--were
awarded using a tradeoff process in which non-cost evaluation factors,
when combined, were more important than price. Figure 2 shows how often
DOD used the different best value processes and other source selection
approaches.
Figure 2: Estimated Frequencies of Source Selection Approaches Used in
Fiscal Year 2009 for New, Competitively Awarded DOD Contracts
Obligating over $25 million[A]:
[Refer to PDF for image: pie graph]
Best value processes (95%): Lowest price technically acceptable: 26%;
Best value tradeoff prices (69%): Non-cost factors less important than
price: 2%;
Best value tradeoff prices (69%): Non-cost factors equal to price: 20%;
Best value tradeoff prices (69%): Non-cost factors more important than
price: 47%;
Sealed bid[B]: 5%.
Source: GAO analysis of DOD contract documents.
[A] The 95 percent confidence intervals for estimates in this graph are
within +/-8 percentage points of the estimates themselves.
[B] In sealed bidding, award is made to the responsible bidder whose
bid conforms to the invitation for bid and is the most advantageous for
the government considering only price and price-related factors
included in the invitation.
[End of figure]
In 69 percent of the contract awards, DOD used the best value tradeoff
process. When doing so, it acquired services approximately four times
as often as it acquired products. Over half of these procurements were
for building or civil engineering construction services, including
projects for troop housing, administrative facilities, and hurricane
protection systems. Other services procured using the tradeoff process
were equipment maintenance and professional management services. For
example, in fiscal year 2009, the Army Corps of Engineers awarded a
contract worth more than $963 million to construct one of the largest
pumping stations in the world, along with floodgates and floodwalls for
hurricane protection. Similarly, the Air Force awarded the Contract
Field Team program multiple-award contract, with an estimated base
value of $2.6 billion for modification, maintenance and repair of
systems including aircraft and missile defense for the departments of
the Army, Navy, Air Force and several federal agencies. Small arms and
electronic countermeasure equipment were among the products most
frequently procured using a tradeoff process, including the contracts
for the Squad Automatic Weapons--lightweight, automatic rifles issued
to each Army and Marine rifle squad--and an Army contract to procure
devices that counteract radio-controlled improvised explosives. Our
analysis of selected characteristics of contracts awarded using a best
value tradeoff process in fiscal year 2009 is shown in figure 3.
Figure 3: Estimated Frequencies of Selected Characteristics of
Contracts or Agreements Awarded using a Best Value Tradeoff Process[A]:
[Refer to PDF for image: circular chart]
Contract pricing[B];
Firm fixed price: 78%;
Cost reimbursement: 2%;
Other fixed price: 2%;
Combination: 17%[C].
Contract structure;
Indefinite delivery: 31%;
Stand-alone[D]: 66%;
Other[E]: 3%.
Contract purpose;
Product: 17%;
Product/Service: 5%;
Service: 78.
Source: GAO analysis of DOD contract documents and data obtained from
FPDS-NG.
[A] Note: The 95 percent confidence intervals for estimates in this
graph are within +/-8 percentage points of the estimates themselves.
[B] Percentages may not sum to 100 percent due to rounding.
[C] Combination refers to contracts that allow for orders to be placed
using more than one pricing arrangement.
[D] Stand-alone refers to contracts that do not allow for individual
orders to be placed against the contract.
[E] Other includes basic ordering agreements and blanket purchase
agreements.
[End of figure]
As part of our work, we reviewed 10 IDIQ contracts that had been
awarded using a best value tradeoff process and 23 task and delivery
orders under these contracts which had obligations ranging from $11
million to over $319 million. In most cases, DOD did not issue the task
or delivery orders we reviewed using a tradeoff process. For example,
an Air Force official explained that the initial task orders for the
Contract Field Team program, including 13 orders in our sample, were
issued under an LPTA process because data needed to assess contractor
performance and timeliness were not yet available to use given the
short time between award of the base contract and issuance of the first
orders. An Air Force official indicated that once they had obtained
sufficient performance data, they intended to issue task orders using a
tradeoff process when possible. DOD officials issued six other task
orders on the basis of negotiating with a contractor who had been
awarded a single award IDIQ contract. The four remaining orders were
awarded using a tradeoff process. For example, the Army wanted infrared
vision enhancement equipment for nighttime and battlefield use in Iraq
and Afghanistan to be delivered as quickly as possible. Consequently,
the Army used a contractor's ability to meet delivery requirements as
the principal evaluation factor in selecting the contractor for
delivery order award.
Some DOD officials noted that the use of various source selection
evaluation methods can change over time. For example:
* Defense Logistics Agency (DLA) officials noted that they have
recently transitioned from principally using the tradeoff process to
using the LPTA process for most fuel purchases because the majority of
their procurements were for a commercial product in relatively stable
domestic and international markets. They noted, however, that they
still use the tradeoff process in less stable areas, such as Iraq and
Afghanistan, where they require more information about vendors' past
performance and technical capability when operating in war zones.
* Conversely, Army Corps officials in New Orleans reported that they
have been using the tradeoff process more frequently since the increase
in civil works construction projects following Hurricane Katrina. While
they typically used sealed bids in the past, they told us that use of
the tradeoff process enabled them to better assess contractors' ability
to meet safety and schedule requirements.
DOD Used a Best Value Tradeoff Process to Address Complex or Time
Sensitive Needs, but Paid Relatively Few Price Differentials:
DOD officials tended to use a best value tradeoff process with non-cost
factors weighted more important than price when they were willing to
accept a higher price if a contractor could demonstrate certain
advantages, such as meet a deadline, demonstrate that it understood
complex technical issues, or propose an innovative approach. DOD often
indicated in tradeoff solicitations that non-cost factors would be
significantly more important than price in making award decisions, but
our analysis indicated that DOD selected a lower priced proposal among
those offerors remaining in the final competition almost as often as it
selected a higher technically rated, but more costly, proposal.
Overall, DOD paid a price differential--the difference in the price of
the offeror awarded the contract and the price of the offeror next in
line for award--in 21 of the 68 contracts in which a price differential
was considered. Most differentials were less than 5 percent. While DOD
officials told us that the tradeoff process provides an essential tool
to obtain desired capabilities, they rely on the case-by-case judgment
of contracting and program officials to determine the best acquisition
approach suited to program requirements and do not specifically track
whether use of the tradeoff process is in DOD's interest.
Contract Requirements Drove DOD's Selection of Technical and Past
Performance Evaluation Factors:
The FAR and DOD guidance generally provide acquisition staff
flexibility to develop evaluation factors that meet their procurement
needs and does not indicate which evaluation factors should be most
important. The FAR requires that DOD officials consider, among other
things, past performance on all negotiated competitive acquisitions
exceeding $100,000, but DOD officials have broad discretion in
selecting other non-cost factors and their relative importance. The
factors are intended to provide meaningful discriminators to evaluate
proposals. Army, Navy, and Air Force officials told us that they formed
interdisciplinary teams that developed evaluation factors and the
factors' relative importance by consensus.
We found that 88 of the 129 contracts we reviewed used a best value
tradeoff process. Our analysis shows that DOD considered past
performance and technical evaluation factors as the most important
among the non-cost factors. Figure 4 shows the five most frequently
used non-cost evaluation factors for the 88 contracts in our review in
which a tradeoff was conducted and how often the technical and past
performance factors were most important among the non-cost factors.
Figure 4: Most Frequently Selected Non-cost Evaluation Factors and
Their Relative Importance in 88 Contracts Using a Tradeoff Process:
[Refer to PDF for image: chart]
Army;
Past performance: Occurrence of non-cost factor: 19;
Past performance: Occurrence of past performance or technical factor as
most important: 25;
Technical: Occurrence of non-cost factor: 5;
Technical: Occurrence of past performance or technical factor as most
important: 36;
Small business: Occurrence of non-cost factor: 33;
Small business: Occurrence of past performance or technical factor as
most important: 0;
Experience: Occurrence of non-cost factor: 26;
Experience: Occurrence of past performance or technical factor as most
important: 0;
Management: Occurrence of non-cost factor: 27;
: Occurrence of past performance or technical factor as most important:
0.
Navy;
Past performance: Occurrence of non-cost factor: 8;
Past performance: Occurrence of past performance or technical factor as
most important: 13;
Technical: Occurrence of non-cost factor: 0;
Technical: Occurrence of past performance or technical factor as most
important: 14;
Small business: Occurrence of non-cost factor: 13;
Small business: Occurrence of past performance or technical factor as
most important: 0;
Experience: Occurrence of non-cost factor: 15;
Experience: Occurrence of past performance or technical factor as most
important: 0;
Management: Occurrence of non-cost factor: 9;
Management: Occurrence of past performance or technical factor as most
important: 0.
Air Force;
Past performance: Occurrence of non-cost factor: 0;
Past performance: Occurrence of past performance or technical factor as
most important: 5;
Technical: Occurrence of non-cost factor: 0;
Technical: Occurrence of past performance or technical factor as most
important: 0;
Small business: Occurrence of non-cost factor: 3;
Small business: Occurrence of past performance or technical factor as
most important: 0;
Experience: Occurrence of non-cost factor: 0;
Experience: Occurrence of past performance or technical factor as most
important: 0;
Management: Occurrence of non-cost factor: 3;
Management: Occurrence of past performance or technical factor as most
important: 0.
DLA;
Past performance: Occurrence of non-cost factor: 0;
Past performance: Occurrence of past performance or technical factor as
most important: 1;
Technical: Occurrence of non-cost factor: 0;
Technical: Occurrence of past performance or technical factor as most
important: 1;
Small business: Occurrence of non-cost factor: 0;
Small business: Occurrence of past performance or technical factor as
most important: 0;
Experience: Occurrence of non-cost factor: 0;
Experience: Occurrence of past performance or technical factor as most
important: 0;
Management: Occurrence of non-cost factor: 0;
Management: Occurrence of past performance or technical factor as most
important: 0.
Other;
Past performance: Occurrence of non-cost factor: 1;
Past performance: Occurrence of past performance or technical factor as
most important: 0;
Technical: Occurrence of non-cost factor: 0;
Technical: Occurrence of past performance or technical factor as most
important: 1;
Small business: Occurrence of non-cost factor: 0;
Small business: Occurrence of past performance or technical factor as
most important: 0;
Experience: Occurrence of non-cost factor: 0;
Experience: Occurrence of past performance or technical factor as most
important: 0;
Management: Occurrence of non-cost factor: 0;
Management: Occurrence of past performance or technical factor as most
important: 0.
Percentage of contracts using this factor: Past performance: 82%;
Percentage of contracts using this factor: Technical: 65%;
Percentage of contracts using this factor: Small business: 56%;
Percentage of contracts using this factor: Experience: 47%;
Percentage of contracts using this factor: Management: 44%.
Source: GAO analysis of DOD contract documents.
Note: For the purposes of this report, in cases where multiple non-cost
factors were considered to be of equal importance, GAO considered all
of these factors to be most important. The small business factors
typically consisted of evaluating the offerors' proposed use of small
businesses.
[End of figure]
DOD officials told us that the selection of these evaluation factors
and their relative importance was based on specific acquisition
requirements, such as the ability to meet production deadlines, ensure
compatibility with existing ship and aircraft systems, or provide
needed security for delivery of goods in war zones. Our analysis found
that DOD considered non-cost factors more important than price in 60 of
the 88 contracts awarded using a tradeoff process. The following
illustrate instances where DOD's acquisition needs led them to make non-
cost factors the principle criteria for source selection.
* Army officials had to quickly meet surge requirements based on a
Joint Urgent Operational Needs Statement[Footnote 14] for roadside bomb
detectors as well as services to provide training and support the
system once fielded, and accordingly, made technical capability the
most important evaluation factor. The acquisition plan specified that
deliveries of the critical technology and support services needed to be
made within 6 months of contract award.
* Army officials sought contractors with innovative approaches and a
superior understanding on how to counter the threat of roadside bombs
in awarding a professional services contract for a range of training
programs to be used within the Military Service Combat Training
Centers. According to the Army, the selected contractor provided a
proposal that was superior in nearly all the technical categories
sought by the Army.
* The Navy considered contractors' proposed technical approach the most
important evaluation factor for a helicopter upgrade kit procurement
because the design had to be compatible with existing helicopters. In
this case, the timing of fleet deployment was also critical and the
Navy sought a contractor that could meet their schedule.
* DLA used a tradeoff process primarily for commercial fuel contracts
in dangerous areas, such as Iraq and Afghanistan, due to the heightened
need for contractor reliability in these war zones. In these
situations, DLA officials explained the tradeoff process allowed them
to emphasize security and past performance in their evaluations to
mitigate acquisition risks, especially since they do not know the
vendors well.
* Army Corps of Engineers officials that needed to procure construction
services for barracks for wounded soldiers made the technical and
performance capability factors most important because they needed to be
responsive to new schedule and price targets. These officials used the
tradeoff process to incentivize timeliness and price reductions, and
they were also able to obtain better features, such as more durable
materials.
DOD officials also told us they used these non-cost factors to
encourage contractors to provide innovative solutions to meet DOD's
needs. For example, Army officials expressed a need for technological
innovation in a solicitation for equipment, field support services, and
associated maintenance needed to intercept enemy communications. The
Army encouraged the contractors to develop a system that would enable
them to upgrade the equipment frequently over the life of the contract.
The statement of work clarified that these upgrades would be essential
to maintain relevancy in the battlefield and keep pace with technology
advancements. Similarly, Marine Corps officials we spoke to about an
urban warfare training system told us that they used the tradeoff
process to seek innovative designs when awarding a 5-year, $1 billion
dollar contract. Marine Corps officials indicated that they had a
system that worked, but wanted to push industry to come up with a
solution that allowed the Marines to reconfigure building structures
more quickly and to provide more realistic and current combat scenarios
prior to deployment. In the winning design, the offeror proposed using
modular building sets that Marines could assemble more quickly to
maximize the training opportunities available in the field.
In contrast, our analysis found that the 28 cases in which DOD
officials considered non-cost factors as equal to or less important
than price were nearly all related to construction projects. For
example, in 15 cases we reviewed, the Army Corps of Engineers
considered non-cost factors such as management and technical,
experience, and past performance as equal to price to address less
complex project requirements such as building a new runway for aircraft
and constructing a maintenance facility. In these instances, the
contracting and program officials were able to request and review
information from potential contractors and conduct a tradeoff process
that would not be available through an LPTA approach, but still
considered price of equal importance to non-cost factors in the award
decision.
DOD Paid Relatively Few Price Differentials Despite Best Value Tradeoff
Solicitations That Emphasized Non-Cost Factors:
For the 88 contracts awarded using a best value tradeoff process, DOD
considered whether to pay a price differential in 68
contracts.[Footnote 15] Our analysis indicated that DOD selected the
lower priced option nearly as often as it selected the highest rated,
but more costly, proposal. In the 18 cases in which DOD officials
decided not to pay a price differential, they determined that the lower
price outweighed the advantages of the offeror with the higher
technical rating. In doing so, DOD officials decided not to pay over
$800 million in price differentials. In 29 other cases, DOD awarded
contracts to the offerors that had both the lowest price and the
highest non-cost factor rating.
DOD accepted a higher price in 21 of the 68 contracts in which a price
differential was considered, for a combined difference of more than
$230 million. Most differentials paid were less than 5 percent above
the price submitted by the offeror next in line for award. The largest
price differential from the contracts in our sample was 48 percent
higher, or roughly $13.6 million more, than the next in line offeror's
price. In this case, Marine Corps officials determined that the
product--burn resistant clothing for use by soldiers in Iraq--was worth
the price difference because it provided substantially greater 2nd and
3rd degree burn protection than the product proposed by the other
offeror. Figure 5 shows the frequency with which DOD elected to pay or
not pay a price differential for the 68 contracts in which a price
differential was considered, as well as the value of the price
differentials either paid or not paid.
Figure 5: Price Differentials in the 68 DOD Contracts Reviewed in Which
a Tradeoff Analysis Was Conducted:
[Refer to PDF for image: bar graph]
Contracts with a price differential paid: 21;
Contracts with no price differential paid: Proposals selected had
lowest price and highest non-cost ratings: 29;
Contracts with no price differential paid: Lower priced proposals
selected outweighed advantages of higher non-cost ratings: 18;
Total: 68.
Contracts with a price differential paid: 21;
Value of price differentials as a percentage of the contract value:
Less than or equal to 5%: 12;
Value of price differentials as a percentage of the contract value: 5%
to 20%: 7;
Value of price differentials as a percentage of the contract value:
Greater than 20%: 2.
Contracts with no price differential paid: 18;
Value of price differentials as a percentage of the contract value:
Less than or equal to 5%: 1;
Value of price differentials as a percentage of the contract value: 5%
to 20%: 6;
Value of price differentials as a percentage of the contract value:
Greater than 20%: 11.
Source: GAO analysis of DOD contract documents.
[End of figure]
DOD Relies on the Judgment of Contracting and Program Officials to
Determine Whether Use of Best Value Tradeoffs Meet DOD's Needs:
DOD contracting and program officials believed that the use of best
value tradeoffs provide DOD an essential tool, which allows them to
obtain better insights into the contractors' capabilities and their
understanding of the government's needs, and the reasonableness of the
contractor's approach. DOD and military department officials stated
that they do not specifically track use of the tradeoff process to
determine if DOD's interests are met. Instead, they rely on the
judgment of contracting and program officials to select the best
acquisition approach suited to program requirements on a case by case
basis. Further, DOD officials stated that they do not track whether the
solicitation approach used correlated with whether the contractor
successfully met the terms of the contract and noted that many factors
ultimately contribute to the success or failure of an individual
acquisition that may not have been foreseeable when awarding the
contract. For example, DOD officials noted that DOD would often use a
best value tradeoff process to award a contract to develop a major
weapons system. As our work has found, DOD often encounters cost
increases, schedule delays, and performance shortfalls on its major
systems.[Footnote 16]
DOD Faces Several Challenges in Using the Best Value Tradeoff Process:
DOD officials acknowledged several challenges in using the best value
tradeoff process such as the difficulties in developing meaningful
evaluation factors, the additional time investment needed to conduct
best value procurements, and the business judgment required of
acquisition staff when compared to other acquisition approaches. DOD
officials also noted that the complexity of the tradeoff process
increases the risk of bid protests. To help address source selection
challenges, DOD is drafting a source selection guide to improve
consistency and standardize source selection procedures for
competitively awarded negotiated procurements.
DOD officials told us that developing non-cost factors that
meaningfully discriminate between offers is a challenging part of the
tradeoff process. They noted that as the complexity of the acquisition
increases, so does the need for individuals with the expertise to help
develop the evaluation factors. For example, Army Corps of Engineers
officials told us that the contract for one of the world's largest
flood pump stations required experts with experience in issues ranging
from water flow management to real estate to develop evaluation
factors. Further, Navy officials explained that while they often use
past performance as a non-cost discriminator, it can be difficult to
identify differences between contractor proposals because contractors
often provide their best performance examples and the government often
lacks data to evaluate additional contractor projects. Our past work
has also identified governmentwide challenges in obtaining needed past
performance information to support contract award decisions.[Footnote
17] Further, the absence of meaningful non-cost discriminators can
result in offerors receiving equal scores on the factors that were
identified as being significantly more important than price. As such,
the decision may default simply to a consideration of price alone. For
example, Air Force officials noted that they are considering updating
factors used to award task orders under the Contract Field Team
contract because contractors tend to receive the highest ratings for
each non-cost factor reviewed, so price is typically the only
discriminator.
DOD officials also noted that using the best value tradeoff process is
often far more time-consuming than other approaches. Navy officials
told us that the tradeoff process is administratively burdensome and
requires a large time investment from program staff, which can make it
challenging to keep the same acquisition team together for an entire
procurement. During our site visits, many contract and program staff
told us that the tradeoff process often takes between 18 and 24 months.
In addition, in Afghanistan and Iraq, the challenges of conducting a
tradeoff process have contributed to decisions by the CENTCOM Joint
Theater Support Contracting Command and Army Corps of Engineers to
discourage its use. For example, recent Army Corps of Engineer projects
in Afghanistan have emphasized using simpler, less complex designs or
requirements that are more suitable for the use of a lowest price
technically acceptable approach.
The complex nature of the best value tradeoff process, including
decisions on whether to pay a price differential, requires much greater
business judgment when compared to other acquisition approaches. DOD
officials stated that making tradeoff decisions, particularly when to
pay a price differential, is among the most difficult aspects of the
tradeoff process, which will become more challenging with less
experienced staff coming into the acquisition workforce. DOD officials
indicated that DOD intends to increase the size of its contracting
career field by more than 6,400 personnel through fiscal year 2015.
With the influx of new staff, many of the contracting officers we met
with noted challenges in preparing staff to conduct the tradeoff
process. For example, a Navy contracting officer told us that guidance
and training only go so far to prepare acquisition staff to conduct
best value tradeoff procurements. Instead, acquisition staff need to be
involved in a number of best value tradeoff procurements to develop the
business judgment necessary to conduct a successful acquisition.
DOD officials stated that the complexity of the tradeoff process also
increases the risk of bid protests. Of the 88 contracts we reviewed
that used a tradeoff process, 15 were the subject of a bid protest to
GAO. While most of the protests were denied, DOD took corrective
actions in 5 cases, including 4 cases in which DOD terminated the
contract or made a new source selection decision when it determined
that it failed to adhere to the solicitations' requirements.
Some of the services have developed initiatives to address these
challenges. For example, the Air Force set up an Acquisition Center of
Excellence (ACE) at Tinker Air Force Base, which provides pre-award
source selection assistance to contract and program staff. Air Force
officials stated that ACE reviews the evaluation factors within
individual source selection plans, serves in an advisory capacity on
source selection teams and holds workshops for contracting officers.
Similarly, Army officials at Ft. Monmouth's Communications--
Electronics Command have developed an online business tool--the ASSIST
tool--that shepherds contracting officers through the solicitation
process. For example, the tool provided a list of steps that must be
completed for best value tradeoff procurements and automatically routes
documents through source selection evaluation boards and other
participating officials for review, as required.
DOD is also drafting a departmentwide source selection guide to improve
consistency and standardize source selection procedures for
competitively awarded negotiated procurements. Given the influx of new
acquisition staff, DOD officials stated they wanted to develop a more
prescriptive guide for best value procurements. While the DOD draft
source selection guidance contains information on various aspects of
the best value process, such as the source selection decision document,
DOD officials told us it does not address price differentials. Numerous
DOD officials underscored the importance of training in the use of the
best value process, particularly training that addresses the tradeoff
decision that acquisition staff must make. For example, one Army Corps
of Engineers official told us that source selection officials would
benefit by training that contains real life lessons on how other
officials have made price differential decisions during the tradeoff
process. Similarly, Marine Corps and Army officials told us that while
decisions are made on a case-by-case basis, informal rules of thumb
regarding price differentials can come into play and indicated that
additional guidance or training, especially case studies or scenarios,
would be helpful.
The Defense Acquisition University (DAU) is responsible for providing
training to the DOD acquisition workforce. According to DAU officials,
they offer more than 10 courses that contain elements of the best value
tradeoff process, but none of the current courses provide case studies
or scenarios that focus on reaching price differential decisions during
source selection. They noted that once the new source selection
guidance is implemented, which is anticipated for January 2011, they
plan to augment existing contracting courses to reflect the new
guidance.
Conclusions:
The best value tradeoff process underlies the vast majority of DOD
competitively awarded contracts, and effective use of this process
hinges on making sound tradeoffs between price and non-cost factors. By
focusing on non-cost factors, DOD anticipates that it will obtain
technical solutions that are innovative and address complex and time-
sensitive program requirements. Applying a tradeoff process, however,
does not guarantee successful acquisitions, nor is it without other
challenges. In particular, using a tradeoff process can be more complex
and take more time than other source selection methods, and requires
that acquisition staff have proper guidance, needed skills, and sound
business judgment. With the anticipated influx of more than 6,400 DOD
contracting personnel over the next few years, providing a firm
foundation for use of the tradeoff process is essential. While DOD and
the military departments have taken steps to improve source selection
procedures, acquisition personnel noted a lack of training to assist
them in deciding whether or not a price differential is warranted when
making tradeoff decisions. For example, while DOD's new source
selection guide provides insights on the source selection process, it
is silent on how to reach decisions on when to pay a price
differential, as is DOD's current training curriculum. DOD has an
opportunity as it updates its training curriculum to provide
acquisition staff with better insights using real life examples on
reaching tradeoff decisions. Taking this step can help DOD minimize the
risk of paying a price differential when not warranted or losing the
benefit of a technically superior solution.
Recommendation for Executive Action:
To help DOD effectively employ the best value tradeoff process, we
recommend that the Secretary of Defense direct the Director of Defense
Procurement and Acquisition Policy to work with the Defense Acquisition
University to develop training elements, such as case studies or
scenarios that focus on reaching tradeoff decisions, including
consideration of price differentials, as it updates the source
selection curriculum.
Agency Comments and Our Evaluation:
DOD provided written comments on a draft of this report. DOD concurred
with our recommendation and intends to request the Panel on Contracting
Integrity--comprised of senior DOD leaders tasked, in part, to help
improve DOD's performance--to assist the Defense Acquisition University
in developing training case studies and scenarios that focus on
reaching tradeoff decisions. DOD's letter is reprinted in appendix II.
We are sending copies of this report to interested congressional
committees and the Secretary of Defense. In addition, this report will
be available at no charge on GAO's Web site at [hyperlink,
http://www.gao.gov].
Should you or your staff have any questions on the matters covered in
this report, please contact me at (202) 512-4841 or huttonj@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix III.
Signed by:
John P. Hutton, Director:
Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
Section 845 of the National Defense Authorization Act for Fiscal Year
2010 directed GAO to report on the Department of Defense's (DOD) use of
the best value tradeoff process, and specifically for cases in which
DOD evaluated contractors' proposals on factors other than cost or
price, if these non-cost factors, when combined, were considered more
important than cost or price.[Footnote 18] To respond to the mandate,
we determined (1) how often and for what types of contracts DOD used
the best value tradeoff process; (2) why and how DOD used the best
value tradeoff process; and (3) what challenges, if any, DOD faces in
using the best value tradeoff process.
To determine how often and for what types of contracts DOD used the
best value tradeoff process, we used data from the Federal Procurement
Data System-Next Generation (FPDS-NG) as of January 2010 to identify a
population of contracts based on the following criteria: (1) newly
awarded by DOD in fiscal year 2009; (2) competitively awarded, and (3)
had obligations of $25 million or more in fiscal year 2009.
We established the $25 million threshold because the Defense Federal
Acquisition Regulation Supplement (DFARS) requires contracts with total
estimated costs of $25 million or more in any fiscal year to prepare
written acquisition plans, which contain information about the source
selection approach. This analysis identified 363 contracts. From this
population, we selected a probability sample of 160 contracts,
including 60 indefinite delivery contracts, and reviewed associated
solicitations, source selection decision documents, and other contract
documents to determine the solicitation approach DOD used.
We verified the obligations and contract award fields in FPDS-NG with
contract data to ensure that the contracts within our sample were
within scope. Thirty-one contracts from our initial sample of 160
contracts were outside the scope of our review because they were
incorrectly coded in key parameters, such as being coded as
competitively awarded when they were not or had misreported the amount
of obligations made on the contract or task order. We excluded these
contracts from our sample and determined that FPDS-NG was sufficiently
reliable for the purposes of our review after adjusting for these
errors.
Because we followed a probability procedure based on random selections,
our sample is only one of a large number of samples that we might have
drawn. Since each sample could have provided different estimates, we
express our confidence in the precision of our particular sample's
results as a 95 percent confidence interval (e.g., plus or minus 8
percentage points). This is the interval that would contain the actual
population value for 95 percent of the samples we could have drawn.
Unless otherwise noted, percentage estimates based on our sample have
95 percent confidence intervals that are within plus or minus 8
percentage points of the estimate itself. Confidence intervals for
other numeric estimates are reported along with the estimate itself.
Table 1 summarizes the estimated percentage of contracts of various
source selection approaches reviewed.
Table 1: Estimated Percentages of Source Selection Approaches Reviewed:
Approach: Best value: Tradeoff process;
Estimated percent representation: 55%.
Approach: Best value: Lowest price technically acceptable;
Estimated percent representation: 21%.
Approach: Best value: Sealed bid;
Estimated percent representation: 4%.
Approach: Best value: Erroneously reported as being within our scope of
review;
Estimated percent representation: 19%.
Source: GAO analysis of DOD contract information.
Note: Percentages do not sum to 100 percent due to rounding.
[End of table]
Based on our analysis of the remaining 129 sample contracts, we
estimate that the total number of best value tradeoff, lowest price
technically acceptable, or sealed bid award decisions (in-scope
contracts) in the full population of interest was about 293.[Footnote
19] For contracts that utilized a best value tradeoff process, we
categorized them based on the relative importance placed on price. In
addition, we determined contract type, the type of procurement (product
versus services), and the type of product or service for our sample
contracts using FPDS-NG data and verified this information with the
contract documents.
To determine why and how DOD used the best value tradeoff process and,
in particular, when non-cost factors were considered more important
than price, we obtained and reviewed DOD and service level acquisition
guidance related to source selection policies and procedures that
describe how and when the tradeoff process may be used, including those
used for issuing task orders. In addition, for each of the contracts
within our sample, we obtained contract documentation including the
acquisition plan, solicitation, and source selection decision
memorandum and reviewed them in preparation for interviews with DOD
officials. In several cases, the solicitation was unclear as to which
type of tradeoff process was used. In these cases, we relied on the
source selection decision document to categorize the tradeoff process
used.
We judgmentally selected buying activities to visit based on factors
including the number of contracts awarded on a best value tradeoff
basis, contract type, and goods or services procured. Buying activities
included at least one command from each military department as well as
a defense agency. We reviewed 27 contracts and 23 task orders through
our site visits. Specifically, we judgmentally selected 23 of 48 task
orders for review by compiling all task orders issued on indefinite
delivery/indefinite quantity contracts obligating over $10 million that
were administered by the officials at the sites we visited. We chose
this dollar threshold to exceed the FAR requirement to provide fair
opportunity notices for task orders valued at $5 million or more.
Results from these selected contracts or task orders cannot be
generalized beyond the specific contract contracts or task orders
selected.
During the course of our review, we interviewed officials from the
following commands:
* Department of the Army, U.S. Army Corps of Engineers, New Orleans
District Office, Louisiana, and Afghanistan Engineering District, Kabul
and Kandahar, Afghanistan;
* Department of the Army, Armament Research, Development and
Engineering Center, Picatinny Arsenal, New Jersey;
* Department of the Army, Communications-Electronics Command, Fort
Monmouth, New Jersey;
* Department of the Navy, Marine Corps Combat Development Command,
Quantico, Virginia;
* Department of the Navy, Naval Air Systems Command, Patuxent River,
Maryland and Lakehurst, New Jersey;
* Department of the Air Force, Air Force Materiel Command, Tinker Air
Force Base, Oklahoma;
* Defense Logistics Agency Energy, Ft. Belvoir, Virginia; and:
* Joint Theater Support Contracting Command, U.S. Central Command,
Kabul and Kandahar, Afghanistan, and Baghdad, Iraq.
We interviewed DOD acquisition and contracting officials to identify
their rationale for the selected source selection approach (e.g., the
thought process behind why a best value approach was chosen over other
approaches). For award decisions that used a best value tradeoff
process, we discussed why the evaluation factors were chosen and how
their relative weights were assigned. We also interviewed officials
about the process used and the underlying rationale when issuing
selected task orders. We also interviewed officials to determine what
the expected outcomes were from using the best value tradeoff process.
We reviewed applicable DOD source selection decision documents and
related memoranda to determine how often DOD paid a price differential,
the amount of the price differential, and the reasons that were given
underlying the decision to pay a higher price. We defined a price
differential as a positive difference in price between the offeror who
received the award and the offeror next in line for award.
To determine what challenges if any, DOD faces in using the best value
tradeoff process, we reviewed DOD guidance and interviewed officials
from Defense Procurement and Acquisition Policy, the military
departments and defense agencies.
We conducted this performance audit from March 2010 through October
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Defense:
Office of the Under Secretary of Defense:
3000 Defense Pentagon:
Washington, DC:
20301-3000:
Acquisition, Technology, and Logistics:
October 20, 2010:
Mr. John Hutton:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G. Street, N.W.:
Washington, DC 20548:
October 20, 2010:
Dear Mr, Hutton:
This is the Department of Defense (DoD) response to the GAO draft
report 11-8 "Defense Contracting" Enhanced Training Could Strengthen
DoD's Best Value Tradeoff Decisions," dated October 2010, GAO Code
120890.
The Department has reviewed the draft report and concurs with its
recommendation. I intend to request the Panel on Contracting Integrity,
Subcommittee 5, Appropriate Contracting Approaches and Techniques, to
assist the Defense Acquisition University in developing the training
case studies and scenarios that you recommended. I thank you for the
opportunity to comment on the draft report.
Sincerely,
Signed by:
Shay D. Assad:
Director, Defense and Procurement Policy:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
John P. Hutton, (202) 512-4841 or huttonj@gao.gov:
Acknowledgments:
In addition to the contact named above, Timothy DiNapoli, Assistant
Director; William Russell, Katheryn Hubbell, Paige Muegenburg, Jodi
Munson, Anna Russell, Sylvia Schatz, Roxanna Sun, and Bob Swierczek
made key contributions to this report.
[End of section]
Footnotes:
[1] According to the Federal Procurement Data System-Next Generation,
DOD's total obligations in fiscal year 2009 were about $366 billion.
However, this figure reflects an approximately $13.9 billion downward
adjustment made by DOD to correct an administrative error made in
fiscal year 2008. As this adjustment significantly affected DOD's
reported obligations in fiscal year 2009, the $380 billion figure we
report reflects what DOD's total obligations would have been had the
error not occurred.
[2] GAO, High-Risk Series: An Update, [http://www.gao.gov/products/GAO-
09-271] (Washington, D.C.: January 2009).
[3] Pub. L. No. 111-84 (2009).
[4] The Federal Procurement Data System-Next Generation is the federal
government's system for tracking information on contracting actions.
[5] DFARS 207.103(d)(i)(B).
[6] Indefinite delivery contracts can be used when the government needs
flexibility in the timing of orders within a specified period of time.
One type of indefinite delivery contract is an indefinite delivery/
indefinite quantity (IDIQ), which allows the government to order
unspecified quantities, within stated limits, of products or services
during a fixed period when they cannot predetermine their needs. After
award of the base IDIQ contract, products and services are procured
through individual delivery or task orders during the contract period
based on governmental needs. IDIQ contracts may be issued as a single
award to one contractor, or preferably on a multiple award basis to
several contractors, in which case the FAR requires that each awardee
be given a fair opportunity to compete for subsequent orders.
[7] We have previously reported on data reliability issues with FPDS-
NG. See, e.g., GAO, Federal Contracting: Observations on the
Government's Contracting Data Systems, [http://www.gao.gov/products/GAO-
09-1032T] (Washington, D.C.: Sept. 29, 2009); and Contract Management:
Minimal Compliance with New Safeguards for Time-and-Materials Contracts
for Commercial Services and Safeguards Have Not Been Applied to GSA
Schedules Program, [http://www.gao.gov/products/GAO-09-579]
(Washington, D.C.: June 24, 2009).
[8] Hereafter for the purposes of this report, we use the term "non-
cost factors" for both non-cost and non-price evaluation factors and
the term "price" for both cost and price, despite the different ways
that they are evaluated under the FAR.
[9] FAR 15.304(b).
[10] FAR 15.308.
[11] For additional information see GAO, Federal Contracting:
Opportunities Exist to Increase Competition and Assess Reasons When
Only One Offer is Received, [http://www.gao.gov/products/GAO-10-833]
(Washington D.C.: July 26, 2010).
[12] GAO, Department of Defense: Additional Actions and Data Are Needed
to Effectively Manage and Oversee DOD's Acquisition Workforce,
[http://www.gao.gov/products/GAO-09-342] (Washington D.C.: March 25,
2009).
[13] All percentage and fraction estimates reported within this section
have 95 percent confidence intervals within plus or minus 8 percentage
points of the estimates themselves.
[14] For additional information on joint urgent operational needs, see
GAO, Warfighter Support: Improvements to DOD's Urgent Needs Processes
Would Enhance Oversight and Expedite Efforts to Meet Critical
Warfighter Needs, [http://www.gao.gov/products/GAO-10-460] (Washington,
D.C.: Apr. 30, 2010).
[15] In the other 20 contracts, DOD did not consider a price
differential because only one vendor was considered for award or these
involved a multiple award contract.
[16] GAO, Defense Acquisitions: Assessment of Selected Weapon Systems,
[http://www.gao.gov/products/GAO-10-388SP] (Washington D.C.: March 30,
2010).
[17] See GAO, Federal Contractors: Better Performance Information
Needed to Support Agency Contract Award Decisions,
[http://www.gao.gov/products/GAO-09-374] (Washington D.C.: April 23,
2009).
[18] For the purposes of this report, we use the term "non-cost
factors" for both non-cost and non-price evaluation factors and the
term "price" for both cost and price, despite the different ways that
they are evaluated under the FAR.
[19] The 95 percent confidence interval for this estimate is from 276
to 309 contracts.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation, and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site ([hyperlink, http://www.gao.gov]). Each
weekday afternoon, GAO posts on its Web site newly released reports,
testimony, and correspondence. To have GAO e-mail you a list of newly
posted products, go to [hyperlink, http://www.gao.gov] and select "E-
mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO's actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black
and white. Pricing and ordering information is posted on GAO's Web
site, [hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S.
Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548:
