Defense Acquisitions
Status of DOD's Implementation of Independent Management Reviews for Services Acquisitions Gao ID: GAO-10-284 January 28, 2010The Department of Defense (DOD) is the federal government's largest purchaser of contractor-provided services, obligating more than $207 billion on services contracts in fiscal year 2009. DOD contract management has been on GAO's high-risk list since 1992, in part because of continued weaknesses in DOD's management and oversight of contracts for services. The National Defense Authorization Act for Fiscal Year 2008 directed DOD to issue guidance providing for independent management reviews for services acquisitions. The Act required that the guidance provide a means to evaluate specific contracting issues and to address other issues, including identifying procedures for tracking recommendations and disseminating lessons learned. The Act also directed GAO to report on DOD's implementation of its guidance. GAO (1) assessed the extent to which DOD's guidance addressed the Act's requirements and how the guidance was implemented and (2) determined the status of actions taken by the military departments pursuant to DOD's guidance. GAO compared DOD's guidance with the Act's requirements; obtained data on the number of reviews conducted as of September 2009; and analyzed memoranda of 29 acquisitions valued at over $1 billion. In its written comments, DOD noted it planned to refine its processes to better share the lessons learned and best practices identified during peer reviews.
To meet the legislative requirement regarding independent management reviews, DOD issued guidance in September 2008 and February 2009 providing for a peer review process for services acquisitions. DOD's guidance generally addresses requirements in the Act to issue guidance designed to evaluate specified contracting issues, but according to officials, DOD has not yet determined how it plans to disseminate lessons learned or track recommendations that result from the newly instituted reviews. Under this guidance, the Office of Defense Procurement and Acquisition Policy (DPAP) is responsible for conducting pre- and post-award peer reviews for services acquisitions with an estimated value of over $1 billion. Peer review teams include senior contracting officials from the military departments and defense agencies as well as legal advisors. As of September 30, 2009, DPAP had conducted 29 reviews of 18 services acquisitions, including 3 post-award reviews. DOD has also conducted peer reviews on two task orders but has not yet determined if it will do so on individual task orders in the future. The peer review teams made a number of recommendations and identified some best practices. DOD officials expect to refine their processes, including developing a more formal means for disseminating lessons learned and tracking recommendations, as DOD assesses its initial experiences with peer reviews. Each of the military departments has issued guidance establishing peer review processes for services acquisitions valued at less than $1 billion although the guidance is still evolving. The departments' guidance identifies the offices or commands tasked with conducting peer reviews based on various dollar thresholds. The military departments reported conducting hundreds of peer reviews for services acquisitions as of September 30, 2009, but could not provide exact numbers because of the lack of comprehensive reporting processes. Further, as peer review processes evolve, the military departments are considering ways to disseminate lessons learned and track recommendations.
GAO-10-284, Defense Acquisitions: Status of DOD's Implementation of Independent Management Reviews for Services Acquisitions
This is the accessible text file for GAO report number GAO-10-284
entitled 'Defense Acquisitions: Status of DOD's Implementation of
Independent Management Reviews for Services Acquisitions' which was
released on January 28, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
January 2010:
Defense Acquisitions:
Status of DOD's Implementation of Independent Management Reviews for
Services Acquisitions:
GAO-10-284:
GAO Highlights:
Highlights of GAO-10-284, a report to congressional committees.
Why GAO Did This Study:
The Department of Defense (DOD) is the federal government‘s largest
purchaser of contractor-provided services, obligating more than $207
billion on services contracts in fiscal year 2009. DOD contract
management has been on GAO‘s high-risk list since 1992, in part
because of continued weaknesses in DOD‘s management and oversight of
contracts for services.
The National Defense Authorization Act for Fiscal Year 2008 directed
DOD to issue guidance providing for independent management reviews for
services acquisitions. The Act required that the guidance provide a
means to evaluate specific contracting issues and to address other
issues, including identifying procedures for tracking recommendations
and disseminating lessons learned.
The Act also directed GAO to report on DOD‘s implementation of its
guidance. GAO (1) assessed the extent to which DOD‘s guidance
addressed the Act‘s requirements and how the guidance was implemented
and (2) determined the status of actions taken by the military
departments pursuant to DOD‘s guidance. GAO compared DOD‘s guidance
with the Act‘s requirements; obtained data on the number of reviews
conducted as of September 2009; and analyzed memoranda of 29
acquisitions valued at over $1 billion.
In its written comments, DOD noted it planned to refine its processes
to better share the lessons learned and best practices identified
during peer reviews.
What GAO Found:
To meet the legislative requirement regarding independent management
reviews, DOD issued guidance in September 2008 and February 2009
providing for a peer review process for services acquisitions. DOD‘s
guidance generally addresses requirements in the Act to issue guidance
designed to evaluate specified contracting issues, but according to
officials, DOD has not yet determined how it plans to disseminate
lessons learned or track recommendations that result from the newly
instituted reviews. Under this guidance, the Office of Defense
Procurement and Acquisition Policy (DPAP) is responsible for
conducting pre- and post-award peer reviews for services acquisitions
with an estimated value of over $1 billion. Peer review teams include
senior contracting officials from the military departments and defense
agencies as well as legal advisors. As of September 30, 2009, DPAP had
conducted 29 reviews of 18 services acquisitions, including 3 post-
award reviews. DOD has also conducted peer reviews on two task orders
but has not yet determined if it will do so on individual task orders
in the future. The peer review teams made a number of recommendations
and identified some best practices. DOD officials expect to refine
their processes, including developing a more formal means for
disseminating lessons learned and tracking recommendations, as DOD
assesses its initial experiences with peer reviews.
Each of the military departments has issued guidance establishing peer
review processes for services acquisitions valued at less than $1
billion although the guidance is still evolving. The departments‘
guidance identifies the offices or commands tasked with conducting
peer reviews based on various dollar thresholds. The military
departments reported conducting hundreds of peer reviews for services
acquisitions as of September 30, 2009, but could not provide exact
numbers because of the lack of comprehensive reporting processes.
Further, as peer review processes evolve, the military departments are
considering ways to disseminate lessons learned and track
recommendations.
Figure: Comparison of Peer Review Processes Established by DPAP and
the Military Departments:
[Refer to PDF for image: illustrated table]
Department: DPAP;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal: review is to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Over $1 billion.
Department: Air Force;
Review cycle: Pre-award;
Milestone: Issue solicitation: 2 reviews to occur;
Milestone: Request final proposal: 2 reviews to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Under $1 billion.
Department: Army;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option; Threshold for review:
Under $1 billion.
Department: Navy;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal: review is to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Under $1 billion.
Source: GAO analysis of DOD guidance.
[End of figure]
View [hyperlink, http://www.gao.gov/products/GAO-10-284] or key
components. For more information, contact John Hutton at (202) 512-
4841 or huttonj@gao.gov.
[End of section]
Contents:
Letter:
DOD Has Implemented Guidance That Generally Addresses Legislative
Requirements:
Concluding Observations:
Agency Comments:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Staff Acknowledgments:
Figures:
Figure 1: Comparison of DPAP's Peer Reviews and Management Reviews for
Services Acquisitions Estimated to Be Valued at $1 Billion or More:
Figure 2: Peer Reviews of Major Services Acquisitions Conducted by
DPAP as of September 30, 2009:
Figure 3: Comparison of Peer Review Processes Established by DPAP and
the Military Departments:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
January 28, 2010:
[Refer to PDF for image]
[End of figure]
Congressional Committees:
The Department of Defense (DOD) is the federal government's largest
purchaser of contractor-provided services, including professional and
management support, information technology support, weapon system and
base operations support, and intelligence support. DOD obligated more
than $207 billion on services contracts in fiscal year 2009, more than
double the amount it obligated in fiscal year 2001, when measured in
real terms.[Footnote 1] DOD's reliance on contracted services makes
effective management and oversight of these contracts essential;
however, DOD contract management has been on our high-risk list since
1992.[Footnote 2]
Our recent work continues to identify weaknesses in DOD's management
and oversight of services contracts. In 2006, we found that DOD's
approach to managing services acquisitions did not allow the
department to determine whether its investments in services achieved
the desired outcomes. For example, we found that the department often
focused its efforts on awarding contracts without determining the
areas of risk that needed greater attention or capturing the knowledge
to enable more informed strategic decisions when contracting for such
services.[Footnote 3] In November 2009, we reported that DOD needed to
take further actions to improve its management of professional and
management support services, particularly in assessing the risks
associated with contractors performing tasks that closely support
inherently governmental functions and in providing guidance for
enhancing oversight when contracting for such services.[Footnote 4]
To improve DOD's services acquisition process, Section 808 of the
National Defense Authorization Act for Fiscal Year 2008 (the Act)
directed the Secretary of Defense to issue guidance and implementing
instructions providing for periodic independent management reviews of
contracts for services.[Footnote 5] The Act specified that the
guidance and instructions were to be designed to evaluate certain
issues (hereafter referred to as contracting issues), including:
* contract performance in terms of cost, schedule, and requirements;
* the use of contracting mechanisms, including the use of competition,
the contract structure and type, the definition of contract
requirements, cost or pricing methods, the award and negotiation of
task orders,[Footnote 6] and management and oversight mechanisms;
* the contractor's use, management, and oversight of subcontractors;
* the staffing of contract management and oversight functions; and:
* the extent of any pass-through or excessive pass-through charges by
the contractor.[Footnote 7]
Further, when one contractor provides oversight for services performed
by other contractors, the Act required the DOD guidance to provide
procedures for the periodic review of such contracts to include the
evaluation of:
* the extent of DOD's reliance on the contractor performing
acquisition functions closely associated with inherently governmental
functions and:
* the financial interest of any prime contractor performing
acquisition functions closely associated with inherently governmental
functions in any contract or subcontract in which the prime contractor
provided advice or recommendations to the agency.
In addition to the contracting issues above, the Act required the DOD
guidance to address several elements related to the review process.
These elements include the contracts subject to review, the frequency
of reviews, procedures for tracking the implementation of
recommendations made during reviews, and procedures for identifying
and disseminating lessons learned from reviews.
The Act also directed that we report on DOD's implementation of the
guidance and instructions for independent management reviews.
Specifically, we (1) assessed the extent to which DOD's guidance
addressed the Act's requirements at the department level and how the
guidance was implemented and (2) determined the status of actions
taken by the military departments pursuant to DOD's guidance. To do
so, we reviewed memoranda issued by DOD in September 2008 and February
2009 and compared these documents to the requirements of the Act. We
obtained information on the number of reviews that the Office of the
Under Secretary of Defense for Acquisition, Technology and Logistics'
Office of Defense Procurement and Acquisition Policy (DPAP) and the
military departments conducted as of September 30, 2009. We reviewed
summary memoranda for DPAP-led reviews of services contracts and
compared the topics discussed in the summary memoranda to DOD policy
and guidance, focusing on the contracting issues specified in the Act.
We also interviewed officials from DPAP and the military departments
to discuss how the guidance and instructions were developed and
implemented for the review process. A more detailed description of our
scope and methodology is included in appendix I.
We conducted this performance audit from October 2009 through January
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
DOD Has Implemented Guidance That Generally Addresses Legislative
Requirements:
To meet the legislative requirements regarding independent management
reviews, DOD issued guidance and instructions providing for a peer
review process for services acquisitions. DOD's guidance generally
addresses requirements prescribed in the Act to develop a process to
evaluate the specified contracting issues, but according to DOD
officials, DOD has not yet determined how the department plans to
disseminate lessons learned or track recommendations that result from
the newly instituted reviews. DOD officials expect to further refine
their processes, including developing a more formal means for
disseminating lessons learned and tracking recommendations as DOD
assesses its initial experiences with peer reviews. Through the first
year of implementation, DPAP, which is responsible for conducting
reviews of acquisitions over $1 billion, had conducted 29 peer reviews
on 18 services acquisitions. Similarly, the military departments,
which are responsible for conducting reviews of their acquisitions
under $1 billion, issued guidance that provides for peer reviews at
various levels within the departments based on dollar values. The
military departments could not, however, determine the exact number of
peer reviews conducted because of the absence of comprehensive
reporting processes. Further, as peer review processes evolve, the
military departments are considering ways to disseminate lessons
learned and track recommendations.
DPAP Has Established a Peer Review Process for Services Acquisitions:
DPAP issued a memorandum in September 2008 establishing a peer review
process to fulfill the requirement for an independent management
review of contracts for services.[Footnote 8] The requirement for a
peer review process was subsequently incorporated into DOD Instruction
5000.02, Operation of the Defense Acquisition System, in December
2008. The guidance states that these reviews are intended to ensure
consistent and appropriate implementation of policy and regulations,
improve the quality of contracting processes, and facilitate sharing
best practices and lessons learned. According to DOD officials, peer
reviews by design are a means of improving individual acquisitions and
not necessarily a tool for strategically managing DOD's services
portfolio.
Under DPAP's guidance, peer reviews supplement its existing process to
review and approve services acquisitions. Pursuant to congressional
direction, DOD had previously established a management review process
that was intended to ensure that DOD services acquisitions are based
on clear, performance-based requirements with measurable outcomes and
that acquisitions are planned and administered to achieve intended
results.[Footnote 9] In these management reviews, DPAP assesses and
approves the acquisition strategies submitted by the military
departments or defense agencies for obtaining contractor-provided
services estimated to be valued at $1 billion or more. Once the
acquisition strategies are approved, DOD contracting offices may
continue the acquisition process, including soliciting bids for
proposed work and subsequently awarding contracts. DOD may award
different contract types to acquire products and services, or issue
task orders under existing contracts. In November 2009, we reported
that the number of contracts and task orders issued after the
acquisition strategies were approved was significant. For example, we
reported that nearly 1,900 task orders were issued under the seven
professional and management support services acquisitions we
reviewed.[Footnote 10]
DOD generally conducts peer reviews at three key points in the
acquisition process prior to contract award--prior to issuance of the
solicitation (phase 1), prior to request for final proposal revisions
(phase 2), and prior to contract award (phase 3)--and is to conduct
periodic post-award reviews (phase 4) (see figure 1).[Footnote 11]
Figure 1: Comparison of DPAP's Peer Reviews and Management Reviews for
Services Acquisitions Estimated to Be Valued at $1 Billion or More:
[Refer to PDF for image: illustration]
Peer review:
Phase 1: Prior to issuance of the solicitation: Documents typically
reviewed: performance work statement, quality assurance surveillance
plan, request for proposal, and source selection plan;
Milestones for services acquisition: Development of acquisition
strategy and supporting documentation begins;
Management review: Acquisition strategy review and approval: Documents
typically reviewed: draft acquisition strategy, instructions for
proposals, and proposal evaluation criteria.
Phase 2: Prior to request for final proposal revisions: Documents
typically reviewed: instructions for proposals and proposal evaluation
criteria, source selection evaluation guide, source selection plan,
and evaluations of contractor proposals;
Milestones for services acquisition: Solicitation issued.
Phase 3: Prior to contract award: Documents typically reviewed:
proposal analysis report and selection decision document;
Milestones for services acquisition: Final request for proposals
issued.
Phase 4: Post-award review: Documents typically reviewed: any
documentation related to the program, such as task orders, award fee
plan, and performance assessments;
Milestones for services acquisition: Contract awarded; Midpoint of
performance period or exercise of option.
Source: GAO analysis of DOD policy and guidance.
[End of figure]
In February 2009, DOD issued guidance that clarified the relationship
between the management reviews and the peer reviews. For example, the
guidance identifies specific issues to assess and the criteria for the
reviewers to use during the management reviews or pre-award peer
reviews. According to the guidance, some contracting issues identified
in the Act, such as contract type and competition, are to be assessed
during the management reviews. Conversely, other contracting issues
identified in the Act, including requirements definition and the
extent of the agency's reliance on contractors to perform functions
closely associated with inherently governmental functions, are to be
assessed during pre-award peer reviews. The pre-award peer reviews
also are to evaluate several elements of the source selection process
that are not specified in the Act, such as the clarity and consistency
of the documentation. Further, the guidance established review
criteria for post-award reviews that address each of the contracting
issues identified in the Act. For example, during post-award reviews,
reviewers are to assess the extent to which the contracting office was
able to achieve competition for orders and whether it was using
appropriate contract types, well-defined requirements, and appropriate
cost/pricing methods.
According to DOD officials, in conducting these reviews, DPAP convenes
a peer review team consisting of three to five members. Officials said
that the teams are generally chaired by a deputy director within DPAP
and include participation from senior contracting officials from the
military departments and defense agencies as well as legal advisors
from the Office of the Secretary of Defense's General Counsel. The
teams review acquisition documents prior to an on-site review and hold
discussions with contracting officers over multiple days. Upon
completion of the on-site review, peer review teams develop summary
memoranda that include observations and recommendations.
The February 2009 guidance indicated that DPAP is to review services
acquisitions with an expected value of over $1 billion. In addition,
DPAP may review acquisitions under that threshold that it has
designated as special interest because of the nature or sensitivity of
the services to be acquired. According to DOD officials, DPAP does not
have a capability to independently identify acquisitions that will
require its review, but rather relies on the military departments and
defense agencies to notify DPAP of acquisitions that will exceed the
threshold. DPAP officials noted that some reviews were not conducted
because the military departments did not notify DPAP that a peer
review was necessary.
DPAP officials stated that they are currently focusing on the pre-
award peer reviews and are phasing in post-award peer reviews. As of
September 30, 2009, DPAP had conducted 29 peer reviews for 18 services
acquisitions.[Footnote 12] Because the peer review process was only
implemented in September 2008, no single acquisition has been subject
to all phases of the peer review process and no acquisition has been
peer reviewed in both the pre-and post-award phases. While most of the
reviews have focused on proposed acquisitions for which the initial
contract had not yet been awarded, DPAP has also conducted two phase 3
peer reviews for proposed task orders valued at over $1 billion that
were to be issued under an existing contract that had previously been
reviewed. DPAP has not yet determined if it will establish a policy
for conducting peer reviews for all individual task orders over this
amount in the future. For the 29 peer reviews of services acquisitions
that DPAP conducted, figure 2 shows when each review occurred and the
corresponding milestone. For example, DPAP conducted a phase 1 peer
review prior to the issuance of the solicitation for 12 of the 18
services acquisitions.
Figure 2: Peer Reviews of Major Services Acquisitions Conducted by
DPAP as of September 30, 2009:
[Refer to PDF for image: illustrated timeline]
Pre-award review, conducted prior to issuance of solicitation:
November 2008;
Pre-award review, conducted prior to request for final proposals:
April 2009;
Pre-award review, conducted prior to contract award: May 2009.
Pre-award review, conducted prior to issuance of solicitation:
December 2008;
Pre-award review, conducted prior to request for final proposals: June
2009;
Pre-award review, conducted prior to contract award: September 2009.
Pre-award review, conducted prior to issuance of solicitation:
November 2008;
Pre-award review, conducted prior to request for final proposals:
August 2009.
Pre-award review, conducted prior to issuance of solicitation:
December 2008;
Pre-award review, conducted prior to request for final proposals: May
2009;
Pre-award review, conducted prior to contract award: August 2009.
Pre-award review, conducted prior to issuance of solicitation: January
2009;
Pre-award review, conducted prior to request for final proposals:
April 2009;
Pre-award review, conducted prior to contract award: May 2009.
Pre-award review, conducted prior to issuance of solicitation: January
2009;
Pre-award review, conducted prior to request for final proposals:
August 2009.
Post-award review, Review conducted midway through performance period
or prior to exercising an option: March 2009;
Pre-award review, conducted prior to contract award: June 2009;
Pre-award review, conducted prior to contract award: July 2009.
Post-award review, Review conducted midway through performance period
or prior to exercising an option: March 2009.
Pre-award review, conducted prior to request for final proposals:
March 2009;
Pre-award review, conducted prior to contract award: May 2009.
Pre-award review, conducted prior to contract award: May 2009.
Pre-award review, conducted prior to request for final proposals:
April 2009.
Pre-award review, conducted prior to issuance of solicitation: April
2009.
Pre-award review, conducted prior to issuance of solicitation: May
2009.
Post-award review, Review conducted midway through performance period
or prior to exercising an option: July 2009.
Pre-award review, conducted prior to issuance of solicitation: August
2009.
Pre-award review, conducted prior to issuance of solicitation:
September 2009.
Pre-award review, conducted prior to issuance of solicitation:
September 2009.
Source: GAO.
[End of figure]
Our review of the summary memoranda of the pre-award peer reviews that
DPAP conducted as of September 30, 2009, found that review teams
generally documented the evaluation of the use of contracting
mechanisms and, to a lesser extent, the use, management, and oversight
of subcontractors. DPAP officials noted that other contracting issues
may have been discussed during pre-award site visits and not included
in the summary memorandum because the peer review team did not
identify any concerns that warranted inclusion. Further, we found that
review teams made several related recommendations, as illustrated in
the following examples:
* One pre-award peer review team recommended that the contracting
office reconsider the number of contracts that it had proposed be
awarded under an acquisition. In this case, the contracting office had
proposed limiting the number of contracts to three prior to knowing
what proposals and business arrangements would be submitted by
industry. The peer review team noted that this may unduly restrict
flexibility of the military department. Further, the team was unsure
if documentation to support the limitation on contract number would be
sufficient to withstand a bid protest from an unsuccessful offeror.
* Another pre-award peer review recommended that the contracting
office increase its use of subcontractors and encourage the prime
contractors to establish mentor-protégé relationships with their
subcontractors to bring more qualified contractors into an industry.
Our review of the summary memoranda for the three post-award peer
reviews conducted by DPAP found that consistent with guidance, the
review teams evaluated all the contracting issues identified in the
Act. All three summary memoranda listed the required contracting
issues and then reported the peer review teams' observations and
recommendations for the contracting offices to consider for the
acquisition, as illustrated by the following examples:
* One post-award peer review team recommended that the contracting
officer modify the contract to include provisions requiring the
contractor to provide information on pass-through charges for all
future task orders issued. At the time of the peer review, the
contract did not contain a clause requiring the contractor to provide
such information, and therefore the government was unable to determine
the extent of pass-through charges and whether they were excessive.
* Another post-award team recommended that the contracting office
reduce the use of time-and-materials task orders. In this case, the
acquisition strategy envisioned that most of the work would be
performed through fixed-priced task orders; however, time-and-
materials task orders accounted for 62 percent of the value of orders
issued under the contract in the first 2 years of performance.
While DPAP's guidance noted that the recommendations made during peer
reviews are advisory in nature, it also states that contracting
offices are to document in the contract file the disposition of all
pre-award peer review recommendations prior to contract award. The
guidance does not address recommendations made during post-award
reviews. According to DOD officials, contracting offices generally
accept recommendations provided by the peer review teams. DPAP
officials said that if the contracting office decides not to accept a
peer review team's recommendation, the contracting officer is expected
to document the reason in the contract file and provide a copy to DPAP.
In addition to providing recommendations to address potential issues
in proposed acquisitions, the peer review teams have also identified
some best practices. For example, in one summary memorandum the team
called attention to the contracting office's post-award performance
plan for the acquisition, which specified how the office intended to
evaluate and assess contract performance to maintain effective
contract surveillance procedures. The team noted that the plan allowed
real-time access to detailed cost performance data when combined with
regular surveillance. According to officials, DOD, however, has not
yet issued guidance establishing procedures to systematically track
the recommendations made by peer review teams or disseminate best
practices as required by the Act. DOD officials noted that to date,
sharing lessons learned from peer reviews has largely occurred through
word of mouth or through conferences. For example, at a December 2009
conference for senior DOD contracting officials, DPAP presented an
update on its peer review process that included a discussion of
lessons learned. To identify methods to better disseminate trends,
lessons learned, and best practices identified during peer reviews, in
August 2009 DPAP established a subcommittee within the Panel on
Contracting Integrity.[Footnote 13] DPAP officials expect that the
subcommittee will report on its findings in 2010. Further, an official
stated that DPAP plans to consider ways to track the implementation of
recommendations made during peer reviews.
Military Departments' Peer Review Processes Are Evolving:
The September 2008 DPAP guidance required the military departments to
establish their own procedures for conducting pre-and post-award peer
reviews on acquisitions under $1 billion, but provided the flexibility
to the services to tailor the process to best meet their needs. In
response, the Air Force issued its guidance in January 2009,[Footnote
14] the Navy in March 2009,[Footnote 15] and the Army in April
2009.[Footnote 16] The military departments' policies varied in such
areas as the frequency and timing of the reviews and the
organizational levels delegated responsibility for conducting the
reviews. For example, the Air Force conducts up to five pre-award pre-
award peer reviews whereas the Army conducts two (see figure 3). The
military departments plan to refine their policies as they gain
experience with the peer review process.
Figure 3: Comparison of Peer Review Processes Established by DPAP and
the Military Departments:
[Refer to PDF for image: illustrated table]
Department: DPAP;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal: review is to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Over $1 billion.
Department: Air Force;
Review cycle: Pre-award;
Milestone: Issue solicitation: 2 reviews to occur;
Milestone: Request final proposal: 2 reviews to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Under $1 billion.
Department: Army;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option; Threshold for review:
Under $1 billion.
Department: Navy;
Review cycle: Pre-award;
Milestone: Issue solicitation: review is to occur;
Milestone: Request final proposal: review is to occur;
Milestone: Award contract: review is to occur;
Review cycle: Post-award:
Milestone: End contract or exercise option: review is to occur;
Threshold for review: Under $1 billion.
Source: GAO analysis of DOD guidance.
Note: While DPAP is to conduct peer reviews for acquisitions over $1
billion, Air Force and Navy officials noted that they also review
these acquisitions prior to submitting them to DPAP.
[End of figure]
According to officials, both the Air Force and Army modified existing
pre-award reviews to incorporate the peer review requirements. The
existing reviews were mandatory steps in each department's contract
award process and, as such, focused on the proposed acquisition's
contracting approach, source selection process, and readiness to issue
a contract solicitation. Air Force officials stated that the
department previously had a post-award review process that focused on
cost, schedule, and performance metrics, which was revised to
incorporate peer review requirements. Army officials noted that the
Army has focused its attention on implementing pre-award peer reviews,
but has not yet established a post-award peer review process. These
officials noted that the Army plans to issue guidance on conducting
post-award reviews in 2010.
In contrast, the Navy developed a new process, modeled on DPAP's
process, to review proposed services acquisitions. Navy officials are
considering making some refinements to this process. For example, at
the time of our review the Navy had not yet determined the optimal
timing of its post-award peer reviews. The department was trying to
determine a point at which there had been enough contract performance
to evaluate the contractor while still allowing the contracting
officers sufficient time to implement any peer review team
recommendations prior to exercising an option year.
While DPAP was not required to approve the military departments'
guidance, DPAP officials reported that the guidance issued by the
military departments was consistent with the intent of the September
2008 guidance. There are differences, however, in how the military
departments addressed certain issues. For example, each of the
military departments delegated responsibility for conducting peer
reviews to commands and organizational units within their departments
based on expected acquisition value. In that regard:
* The Air Force delegated responsibility for conducting peer reviews
to its major commands for proposed services acquisitions valued from
$50 million to $1 billion.
* The Army delegated responsibility to the head of the contracting
activity within each of its commands for conducting peer reviews for
services acquisitions valued from $250 million to $1 billion.
Similarly, it identified the principal assistant responsible for
contracting as being responsible for conducting peer reviews for
acquisitions from $50 million to $250 million.
* The Navy delegated responsibility to the Deputy Assistant Secretary
of the Navy - Acquisition and Logistics Management (DASN-A&LM) for
conducting peer reviews for acquisitions valued from $250 million to
$1 billion, while individual commands are responsible for conducting
reviews of acquisitions valued from $50 million to $250 million.
Further, the Air Force does not require peer reviews on noncompetitive
acquisitions--in other words, on contracts awarded using other than
full-and-open competition. Air Force officials explained that such
contracts are already reviewed under a separate process and therefore
believed that an additional peer review would be unnecessary.
Similarly, both the Air Force and Army allow the offices responsible
for conducting reviews to waive peer reviews under certain
circumstances, whereas the Navy does not provide for a waiver process.
Air Force guidance allows peer reviews to be waived based on
acquisition/source selection history, such as for recurring
acquisitions and where there is no history of bid protests. The Army
also allows peer reviews to be waived but did not specify in its
guidance which acquisitions could be waived.
As of September 2009, the military departments reported conducting
hundreds of peer reviews for services acquisitions, but the
departments do not have comprehensive processes for determining the
exact number of reviews conducted. Specifically:
* The Navy reported that it had conducted 257 peer reviews for
services acquisitions, including 5 post-award reviews. The Navy could
not identify how many of the reviews conducted by the commands
occurred by September 30, 2009. DASN-A&LM conducted its first 4 peer
reviews on September 22, 2009.
* Though the Air Force did not know the specific number of peer
reviews conducted, officials noted that it had conducted up to five
pre-award reviews on approximately 85 services acquisitions as of
September 30, 2009.
* Army officials stated that though commands had conducted pre-award
peer reviews, an exact number of reviews could not be identified
because the Army does not have a reporting process. The Army also
acknowledged that it did not conduct any post-award reviews because it
has not yet established a post-award peer review process.
As peer review processes evolve, the military departments are
considering ways to disseminate lessons learned and track
recommendations. For example, Navy officials said the department is
waiting to see the results of initial reviews and will then develop
additional guidance to address lessons learned made during peer
reviews. Army officials stated that the department plans to address
recommendations and lessons learned in 2010 when it issues guidance on
post-award reviews. Finally, Air Force policy requires commands to
submit annual reports to the Secretary of the Air Force - Acquisition
and Contracting Policy that are to include major issues identified
during pre-award peer reviews and the resolutions taken.
Concluding Observations:
DOD's guidance implementing a peer review process for major services
acquisitions at the departmental level generally addresses the
requirements prescribed by the Act. While DOD has derived benefits
from these initial reviews, it has also recognized that there are
issues that still need to be addressed, such as how to track
recommendations and disseminate lessons learned. Further, at this
stage, DOD's focus has been on evaluating acquisition strategies and
proposed contracts at the pre-award stage. DOD has conducted
relatively few post-award reviews, in which DOD assesses how well it
is managing the contractor's actual performance. A key issue is
whether and how to apply the peer review process to task orders
through which DOD obtains much of its contractor-provided services.
Few of these are large enough to reach the $1 billion DOD review
threshold, but below the threshold they could be so numerous as to
overtax the departments' peer review processes. Addressing these
issues, as well as those at the military department level, is
important if DOD is to achieve its stated objectives for peer reviews--
ensuring consistent and appropriate implementation of policy and
regulations, improving the quality of contracting processes, and
facilitating sharing best practices and lessons learned--on a more
strategic or enterprisewide basis rather than limiting the peer
reviews' benefits to the individual acquisitions being reviewed.
Although we are not making any recommendations because DOD plans to
address these issues, resolving these concerns in a timely manner is
essential if DOD is to maximize the benefits of the peer review
process.
Agency Comments:
DOD provided written comments on a draft of this report. In its
comments, DOD stated that peer reviews had improved the quality of its
significant business arrangements. DOD indicated that it will continue
to refine its peer review process to better disseminate trends,
lessons learned, and best practices that are identified during peer
reviews. DOD provided a technical comment, which was incorporated into
the report. DOD's comments are reprinted in appendix II.
We are sending copies of this report to the Secretary of Defense; the
Secretaries of the Air Force, Army, and Navy; and interested
congressional committees. The report also is available at no charge on
GAO's Web site at [hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made major contributions to this
report are listed in appendix III.
Signed by:
John P. Hutton:
Director:
Acquisition and Sourcing Management:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable John Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
Section 808 of the National Defense Authorization Act for Fiscal Year
2008 (the Act) directs GAO to report on the Department of Defense's
(DOD) implementation of its guidance and implementing instructions
providing for periodic management reviews of contracts for services.
In response to this mandate, we (1) assessed the extent to which DOD's
guidance addressed the Act's requirements at the department level and
how the guidance was implemented and (2) determined the status of
actions taken by the military departments pursuant to DOD's guidance.
To do so, we reviewed DOD's September 2008 and February 2009 guidance
issued by the Under Secretary of Defense for Acquisition, Technology
and Logistics' Office of Defense Procurement and Acquisition Policy
(DPAP). We compared the guidance and instructions to the requirements
stipulated in Section 808 of the Act. The September 2008 guidance
indicated that peer reviews were to be conducted for both supplies and
services. As the Act's requirements were specific to services
acquisitions, we limited our analysis to services. We also obtained
guidance and implementing instructions issued by the Departments of
the Air Force, Army, and Navy. We interviewed officials from DPAP and
the Departments of the Army, Navy, and Air Force to gain further
insight into how each organization developed its guidance and
instructions. DOD's September 2008 memorandum also indicated that
defense agencies were required to develop their own guidance. While
these were outside the scope of our review, DPAP officials indicated
that 13 of 17 defense agencies that DPAP believed would be required to
develop guidance had done so at the time of this review.
We obtained information on the number of peer reviews on services
acquisitions that DPAP and the military departments reported they had
conducted as of September 30, 2009. DPAP was able to identify the
number of reviews that it had conducted. We determined this
information to be sufficiently reliable for the purposes of our
review. The Air Force provided an approximate number of acquisitions
that had been reviewed but could not identify the number of individual
peer reviews conducted. The Army did not provide any information on
the specific number of reviews conducted. The Navy provided
information on the number of reviews it had conducted but could not
specify how many had been conducted as of September 30, 2009. We could
not independently verify the information provided by the military
departments because of the lack of available documentation.
To determine the nature of the discussions and the issues addressed
during peer reviews, we obtained the summary memoranda from each of
the 29 peer reviews conducted by DPAP as of September 30, 2009. These
29 memoranda represented 18 unique acquisitions, as DPAP had reviewed
some acquisitions more than once. Twenty-six of the memoranda were for
pre-award peer reviews and 3 were for post-award reviews. We analyzed
summary memoranda from each of the 29 peer reviews to determine the
topics discussed in the memoranda, focusing specifically on the
contracting issues identified in the Act. We also interviewed DPAP
officials who chaired or participated in these reviews to obtain their
views on the peer review process.
We conducted this performance audit from October 2009 through January
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition, Technology And Logistics:
3000 Defense Pentagon:
Washington, DC 20301-3000:
January 21, 2010:
Mr. John Hutton:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Hutton:
This is the Department of Defense ODD) response to the GAO Draft
Report, GAO-10-284, "Defense Contracting: Status of Implementation of
Independent Management Reviews for Services Acquisitions," dated
December 23, 2009 (GAO Code 120858).
We appreciate GAO's assessment of the Department's peer review
initiative and our approach to implement the statutory requirements
for Independent Management Reviews. Looking back at our first year of
execution, there is no question the peer review initiative has yielded
innumerable benefits to improve the quality of the Department's
significant business arrangements. This year, as indicated in the
report, we will evolve the process to implement methods to better
disseminate trends, lessons learned, and best practices that are
identified during peer reviews. To this end, the Peer Review
subcommittee of the Department's Panel on Contracting Integrity will
guide our efforts as we seek to adopt any refinements that are
necessary to make this initiative even more effective.
Sincerely,
[Signed by]
Shay D. Assad:
Director, Defense Procurement and Acquisition Policy:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
John P. Hutton, (202) 512-4841 or huttonj@gao.gov:
Acknowledgments:
In addition to the contact named above, Timothy DiNapoli, Assistant
Director; E. Brandon Booth; Morgan Delaney Ramaker; Christopher
Mulkins; Thomas Twambly; and Alyssa Weir made key contributions to
this report.
[End of section]
Footnotes:
[1] According to the Federal Procurement Data System-Next Generation,
DOD's total obligations for services in fiscal year 2009 were about
$193 billion. However, this figure reflects an approximately $13.9
billion downward adjustment made by DOD to correct an administrative
error made in fiscal year 2008. As this adjustment significantly
affected DOD's reported obligations in fiscal year 2009, the $207
billion figure we report reflects what DOD's total obligations for
services would have been had the error not occurred.
[2] GAO, High-Risk Series: An Update, [hyperlink,
http://www.gao.gov/products/GAO-09-271] (Washington, D.C.: January
2009).
[3] GAO, Defense Acquisitions: Tailored Approach Needed to Improve
Service Acquisition Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-07-20] (Washington, D.C.: Nov. 9,
2006).
[4] GAO, Defense Acquisitions: Further Actions Needed to Address
Weaknesses in DOD's Management of Professional and Management Support
Contracts, [hyperlink, http://www.gao.gov/products/GAO-10-39]
(Washington, D.C.: Nov. 20, 2009).
[5] Pub. L. No. 110-181, § 808 (2008).
[6] The Federal Acquisition Regulation defines a task order as an
order for services placed against an established contract or
government sources.
[7] Pass-through charges are contractor charges for the costs
associated with subcontracting work. Section 852 of the John Warner
National Defense Authorization Act for Fiscal Year 2007, Pub. L. No.
109-364, states that an "excessive pass-through charge" with respect
to a contractor or subcontractor that adds no, or negligible, value to
a contract or subcontract, means a charge to the government by the
contractor or subcontractor that is for overhead or profit on work
performed by a lower-tier contractor or subcontractor (other than
charges for the direct costs of managing lower-tier contracts and
subcontracts and overhead and profit based on such direct costs).
[8] Office of the Under Secretary of Defense for Acquisition,
Technology and Logistics, Peer Reviews of Contracts for Supplies and
Services, September 29, 2008.
[9] We use the term management review to collectively refer to the
procedures established by DOD for the review and approval of
acquisition strategies. For additional information on DOD's management
review processes, see [hyperlink,
http://www.gao.gov/products/GAO-10-39].
[10] [hyperlink, http://www.gao.gov/products/GAO-10-39].
[11] According to DPAP guidance, acquisitions using other than full-
and-open competition are to have two pre-award peer reviews instead of
three.
[12] While the Act only requires DOD to issue guidance on services
contracts, DOD's policy is to review both services and products,
including weapon systems. In addition to 29 reviews for services
acquisitions, DPAP also conducted 22 pre-award peer reviews for
acquisitions of supplies and weapon systems as of September 30, 2009.
[13] DOD established the Panel on Contracting Integrity in response to
Section 813 of the John Warner National Defense Authorization Act for
Fiscal Year 2007, Pub. L. No. 109-364 (2006). This act directed DOD to
establish a panel consisting of senior leaders representing a cross-
section of the department to conduct reviews of progress made to
eliminate areas of vulnerability within the defense contracting system
that allow fraud, waste, and abuse to occur and recommend changes in
law, regulations, and policy that it determined necessary to eliminate
such areas of vulnerability. The Under Secretary of Defense for
Acquisition, Technology and Logistics subsequently tasked the panel
with taking a holistic view of all ongoing efforts and initiatives to
improve performance in identified areas of weakness.
[14] Office of the Assistant Secretary of the Air Force, Peer Reviews
of Contracts for Supplies and Services (Jan. 5, 2009). The Air Force
refers to its pre-award reviews as multifunctional independent review
teams. It refers to its post-award reviews as annual execution reviews.
[15] Office of the Assistant Secretary of the Department of Navy,
Department of the Navy Peer Review Program (Mar. 26, 2009).
[16] Office of the Assistant Secretary of the Army, Peer Reviews of
Contracts for Supplies and Services (Apr. 23, 2009). The Army refers
to its pre-award reviews as solicitation review boards and contract
review boards.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
