Defense Management
DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative Gao ID: GAO-10-566R May 27, 2010The Department of Defense (DOD) has sought improved efficiencies and cost reductions in its delivery of services that could be provided by the private sector, using both competitions with private companies and processes to create high performing organizations (HPO). The Office of Management and Budget's (OMB) Circular A-76 establishes federal policy for the competition of commercial activities. According to the circular, the longstanding policy of the federal government has been to rely on the private sector for needed commercial services. To ensure that the American people receive maximum value for their tax dollars, it is the federal government's policy that commercial activities should be subject to the forces of competition. As the largest federal agency, DOD has conducted more A-76 competitions than any other federal agency. However, the A-76 process has drawn criticism from both the public and private sectors. These criticisms largely center on the costs and length of time required to conduct competitions and the manner in which long-term savings are calculated. In light of these concerns, a panel of public and private sector experts convened in 2001 to identify ways in which the federal government could improve the A-76 process and included an option that focused on improving efficiencies in-house through the creation of HPOs rather than seeking improved efficiencies through public-private competitions. Although the A-76 process has been DOD's preferred method for ensuring the most efficient operation of a function, it is currently subject to a number of recently enacted statutory limitations. Given these relatively new developments and the potential for HPOs to serve as an alternative to the A-76 process, you asked us to examine the extent to which the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (OSD) has made progress in implementing and evaluating DOD's HPO initiative.
OSD has made some progress in implementing DOD's HPO initiative by providing guidance to organizations that it selected to participate, but is not always collecting reliable data and has no clear plan to evaluate the effectiveness of the HPO initiative. Pilot initiatives such as this are typically used to evaluate alternative approaches or test new ideas. Gathering reliable data--data that are complete, accurate, and meet intended purposes--and measuring performance are critical to assessing the effectiveness of new ways of doing business. To implement DOD's HPO initiative, OSD selected nine organizations and issued guidance providing procedures for implementing and monitoring the performance of, and costs for, the HPOs. OSD's guidance provides performance measures that somewhat mirror performance measures set out in the National Defense Authorization Act (NDAA) for Fiscal Year 2004. The 2004 NDAA contains a list of performance measures that, according to the law, should be included among performance measures used in the pilot program. Our review of the annual reports submitted to OSD revealed that while all HPOs generally used the broad performance measures set out in the 2004 NDAA to report on performance, several reports did not include information on all of the performance measures. We found a number of instances in which the HPOs failed to include information on workforce expertise and customer satisfaction. Further, for the performance measures for which information was provided, we found instances of incomplete and inaccurate reporting. For example, in our review of annual reports covering the 2008 reporting cycle, one annual report did not clearly capture the costs associated with hiring temporary employees and the overtime required to augment the much smaller organization in response to unexpected workload increases. Moreover, although most HPOs have submitted at least one annual report, OSD has not used the information to evaluate the HPO initiative. Although OSD initially asserted a role in implementing the HPO initiative, according to OSD officials, their interest in overseeing the HPOs has waned because with the ongoing moratorium on new public-private competitions, they believe that organizations will not have any incentive to participate in the HPO process. Moreover, they believe that with the A-76 moratorium in place, existing HPOs might be less willing to continue working toward fulfilling the performance commitments they have made. Therefore, OSD officials stated that while they do encourage organizations to become more efficient, their position is that selecting new organizations for HPO designation is of questionable value due to the current moratorium on public-private competitions. However, unless OSD assesses the reliability of the information provided by the HPOs and uses reliable performance data to make decisions, DOD may miss opportunities to sustain efficiencies gained by existing HPOs, and will be unable to make management decisions and inform policymakers on the effectiveness of the HPO initiative.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: John H. Pendleton Team: Government Accountability Office: Defense Capabilities and Management Phone: (404) 679-1816GAO-10-566R, Defense Management: DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative
This is the accessible text file for GAO report number GAO-10-566R
entitled 'Defense Management: DOD Needs to Assess Effectiveness and
Determine Future Direction for Its High Performing Organizations
Initiative' which was released on May 27, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-10-566R:
United States Government Accountability Office:
Washington, DC 20548:
May 27, 2010:
The Honorable Ike Skelton:
Chairman:
The Honorable Howard P. McKeon:
Ranking Member:
Committee on Armed Services:
United States House of Representatives:
Subject: Defense Management: DOD Needs to Assess Effectiveness and
Determine Future Direction for Its High Performing Organizations
Initiative:
The Department of Defense (DOD) has sought improved efficiencies and
cost reductions in its delivery of services that could be provided by
the private sector, using both competitions with private companies and
processes to create high performing organizations (HPO). The Office of
Management and Budget's (OMB) Circular A-76 establishes federal policy
for the competition of commercial activities. According to the
circular, the longstanding policy of the federal government has been
to rely on the private sector for needed commercial services. To
ensure that the American people receive maximum value for their tax
dollars, it is the federal government's policy that commercial
activities should be subject to the forces of competition. As the
largest federal agency, DOD has conducted more A-76 competitions than
any other federal agency. However, the A-76 process has drawn
criticism from both the public and private sectors. These criticisms
largely center on the costs and length of time required to conduct
competitions and the manner in which long-term savings are calculated.
In light of these concerns, a panel of public and private sector
experts convened in 2001 to identify ways in which the federal
government could improve the A-76 process and included an option that
focused on improving efficiencies in-house through the creation of
HPOs rather than seeking improved efficiencies through public-private
competitions.
In 2003, Congress passed legislation that directed the Secretary of
Defense to establish a pilot program for the creation or continued
implementation of HPOs:
through the conduct of a business process re-engineering effort.
[Footnote 1] Defense organizations seeking participation under this
pilot program are required to re-engineer their business processes to
improve efficiencies and cost effectiveness. A 2003 revision to OMB's
Circular A-76 also allowed entities that were designated as Most
Efficient Organizations (MEO) as part of the A-76 process to seek HPO
designation, thereby avoiding future public-private competitions for a
3-to 5-year period. On July 26, 2005, the Office of the Under
Secretary of Defense for Acquisition, Technology, and Logistics (OSD)
solicited nominations from defense organizations seeking HPO
designation as part of its HPO initiative and has since designated
nine organizations from across DOD as HPOs.[Footnote 2]
Although the A-76 process has been DOD's preferred method for ensuring
the most efficient operation of a function, it is currently subject to
a number of recently enacted statutory limitations. For example, the
Omnibus Appropriations Act for 2009[Footnote 3] contains a provision
that prohibited the use of funds for beginning or announcing a study
or public-private competition under A-76, and the same language was
included in the Consolidated Appropriations Act for 2010[Footnote 4]
extending that prohibition. Additionally, section 325 of the National
Defense Authorization Act for Fiscal Year 2010[Footnote 5] temporarily
suspends public-private competition for conversion of DOD functions to
performance by contractors until the Secretary of Defense satisfies a
number of requirements, including, among other things, the completion
of a comprehensive review of the department's policies with respect to
the conduct of public-private competitions and submission of a report
on that review to the congressional defense committees. According to
OSD and HPO officials, these legislative actions have effectively
imposed a moratorium on public-private competitions. Given these
relatively new developments and the potential for HPOs to serve as an
alternative to the A-76 process, you asked us to examine the extent to
which OSD has made progress in implementing and evaluating DOD's HPO
initiative.
Scope and Methodology:
To examine the extent of OSD's progress in implementing and evaluating
its HPO initiative, we focused on the nine organizations designated by
OSD as HPOs. The nine organizations included in our review were:
* Defense Contract Management Agency: Procurement Technicians,
* Defense Finance and Accounting Service: Accounting Services,
* Defense Logistics Agency: Document Automation Production and Service,
* Defense Logistics Agency: Human Resources,
* Washington Headquarters Services: Federal Facilities Division,
* U.S. Air Force: Edwards Air Force Base Aircraft Maintenance,
* U.S. Army: Corps of Engineers Logistics Management,
* U.S. Army, Fort Hood: Directorate of Aviation Operations, and:
* U.S. Army: Fort Huachuca Installation Personnel Management.
We obtained and analyzed relevant statutory provisions, regulations
related to the HPO process, and DOD and OMB guidance related to the
HPO initiative. We also reviewed OSD's criteria for selecting
organizations, letters of obligation[Footnote 6] between OSD and
participating organizations, and the annual reports documenting the
performance of each of the HPOs. We met with OSD officials to discuss
their efforts to implement and assess the effectiveness of DOD's HPO
initiative. During site visits to HPO locations, we interviewed
officials and conducted discussion groups with employees at eight of
the nine HPOs.[Footnote 7] Following the discussion groups, we asked
employees to anonymously complete a self-administered questionnaire to
give them an opportunity to summarize their thoughts on the discussion
and provide input on any topics discussed that may have been
considered sensitive. Although we did not visit the U.S. Army: Fort
Hood Directorate of Aviation Operations, we reviewed the documentation
associated with OSD's approval of the HPO.[Footnote 8] In an effort to
protect the anonymity of individuals with whom we spoke, we did not
specify the names of the HPOs in the examples presented in our report.
We conducted this performance audit from April 2009 through May 2010
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Summary:
OSD has made some progress in implementing DOD's HPO initiative by
providing guidance to organizations that it selected to participate,
but is not always collecting reliable data and has no clear plan to
evaluate the effectiveness of the HPO initiative. Pilot initiatives
such as this are typically used to evaluate alternative approaches or
test new ideas. Gathering reliable data--data that are complete,
accurate, and meet intended purposes--and measuring performance are
critical to assessing the effectiveness of new ways of doing
business.To implement DOD's HPO initiative, OSD selected nine
organizations and issued guidance providing procedures for
implementing and monitoring the performance of, and costs for, the
HPOs.[Footnote 9] OSD's guidance provides performance measures that
somewhat mirror performance measures set out in the National Defense
Authorization Act (NDAA) for Fiscal Year 2004.[Footnote 10] The 2004
NDAA contains a list of performance measures that, according to the
law, should be included among performance measures used in the pilot
program.[Footnote 11] Our review of the annual reports submitted to
OSD revealed that while all HPOs generally used the broad performance
measures set out in the 2004 NDAA to report on performance, several
reports did not include information on all of the performance
measures. We found a number of instances in which the HPOs failed to
include information on workforce expertise and customer satisfaction.
Further, for the performance measures for which information was
provided, we found instances of incomplete and inaccurate reporting.
For example, in our review of annual reports covering the 2008
reporting cycle, one annual report did not clearly capture the costs
associated with hiring temporary employees and the overtime required
to augment the much smaller organization in response to unexpected
workload increases. Moreover, although most HPOs have submitted at
least one annual report, OSD has not used the information to evaluate
the HPO initiative. Although OSD initially asserted a role in
implementing the HPO initiative, according to OSD officials, their
interest in overseeing the HPOs has waned because with the ongoing
moratorium on new public-private competitions, they believe that
organizations will not have any incentive to participate in the HPO
process. Moreover, they believe that with the A-76 moratorium in
place, existing HPOs might be less willing to continue working toward
fulfilling the performance commitments they have made. Therefore, OSD
officials stated that while they do encourage organizations to become
more efficient, their position is that selecting new organizations for
HPO designation is of questionable value due to the current moratorium
on public-private competitions. However, unless OSD assesses the
reliability of the information provided by the HPOs and uses reliable
performance data to make decisions, DOD may miss opportunities to
sustain efficiencies gained by existing HPOs, and will be unable to
make management decisions and inform policymakers on the effectiveness
of the HPO initiative.
We are making two recommendations to provide future direction for
DOD's HPO initiative that include assessing the reliability of the
data provided to date by the HPOs and evaluating the effectiveness of
the HPO initiative against the performance measures set out in the
2004 NDAA. In written comments on a draft of this report, DOD
partially concurred with our recommendations. In response to our
recommendation to assess the reliability of the data provided by the
HPOs, DOD stated that the DOD components are accountable for
collecting and reporting data for their respective HPOs. As we
reported, DOD has required the components to submit data on HPO
performance, but has not assessed the reliability of the data. We
continue to believe that gathering reliable data is essential to
assessing the effectiveness of alternative approaches and, without
reliable data, DOD's ability to evaluate the HPO program will be
limited. In response to our recommendation to use performance data to
fully evaluate the effectiveness of DOD's HPO initiative, DOD stated
that it would conduct an evaluation of the HPOs' performance using
data already submitted by each of the components. However, we remain
concerned that the data reliability problems we identified could limit
the usefulness of this assessment and that DOD should work with the
components to obtain reliable data. In addition, the department
provided technical comments, which we incorporated in the report where
appropriate. DOD's comments are reprinted in enclosure II.
Background:
OMB's Circular A-76 is a governmentwide executive branch policy that
subjects commercial activities and functions performed by federal
government employees to public-private competition. According to the
circular, the longstanding policy of the federal government has been
to rely on the private sector for needed commercial services. To
ensure that the American people receive maximum value for their tax
dollars, it is the federal government's policy that commercial
activities should be subject to the forces of competition.
The HPO option is a departure from the A-76 process because it
involves improving efficiencies in-house rather than seeking improved
efficiencies through public-private competitions. To become an HPO,
any federal organization could conduct an analysis and a re-
engineering of mission and support functions and processes to achieve
improvements in performance. The incentives for an organization to use
this approach were twofold: first, an organization could achieve
improvements in-house and avoid entering into a public-private
competition, and second, the number of positions were counted toward
meeting public-private competition goals established for each
component.
Responsibility for preparing the overarching management policy that
was to be used to develop and implement DOD's HPO initiative was
delegated to the Office of the Under Secretary of Defense for
Acquisition, Technology, and Logistics-Installations and Environment
(OSD). Defense organizations seeking the HPO designation had two
avenues that they could pursue:
* Revised Circular A-76. As part of the Circular A-76 process, federal
agencies create a staffing plan to be submitted with the agency's
tender offer in response to a solicitation for a standard competition.
This staffing plan represents the agency's Most Efficient Organization
(MEO).[Footnote 12] A 2003 revision to OMB Circular A-76 added a
provision that encourages agencies to use a formalized process for
deviating from the requirements of A-76 to explore innovative
alternatives to standard or streamlined competitions, including
HPOs.[Footnote 13] Following the revision to Circular A-76, several
existing MEOs were converted to HPOs, which enabled the agencies to
avoid recompeting their function at the end of their initial MEO
performance period.
* DOD's HPO Pilot Program. The 2004 NDAA included a provision that
directed the Secretary of Defense to establish a pilot program for the
creation or continued implementation of HPOs through the conduct of a
business process re-engineering effort.[Footnote 14] To be eligible
for selection to participate, organizations are required to, among
other things, complete a total organizational assessment that would
result in enhanced performance measures at least comparable to those
that might be achieved through public-private competitions. After an
organization selected to participate in the pilot program implements
the business process reengineering initiative, the secretary concerned
is required to determine whether the organization has achieved initial
progress toward designation as an HPO. In the absence of such
progress, the organization's participation in the HPO pilot is to be
terminated.
OSD Has Made Some Progress Implementing DOD's HPO Initiative, but Has
No Clear Plans to Evaluate Its Effectiveness:
OSD Selected Organizations and Provided Guidance for Implementing and
Evaluating HPOs:
To implement DOD's HPO initiative, which includes participants in the
HPO pilot program and MEOs created for the A-76 process that converted
to HPOs, OSD approved nine organizations for participation. (See table
1 for a list of approved organizations and enclosure I for a brief
description of each HPO.)
Table 1: DOD's High-Performing Organizations:
Organization: U.S. Air Force, Edwards Air Force Base, 412[TH]
Maintenance Group;
HPO type: Pilot;
Function: Aircraft Maintenance;
Approval date: 02-16-2006;
Implementation date: 10-01-2006.
Organization: Defense Logistics Agency;
HPO type: Pilot;
Function: Human Resources;
Approval date: 02-16-2006;
Implementation date: 04-01-2007.
Organization: Defense Contract Management Agency;
HPO type: Pilot;
Function: Procurement Technicians;
Approval date: 02-16-2006;
Implementation date: 11-07-2006.
Organization: U.S. Army Corps of Engineers;
HPO type: Pilot;
Function: Logistics Management;
Approval date: 02-16-2006;
Implementation date: 07-23-2007.
Organization: U.S. Army, Fort Huachuca;
HPO type: A-76 MEO;
Function: Personnel Management;
Approval date: 10-01-2007;
Implementation date: 10-01-2007.
Organization: Defense Finance and Accounting Service;
HPO type: Pilot;
Function: Accounting Services;
Approval date: 01-11-2008;
Implementation date: 06-01-2008.
Organization: Defense Logistics Agency;
HPO type: A-76 MEO;
Function: Document Automation Production Services;
Approval date: 05-13-2008;
Implementation date: 10-01-2009.
Organization: Washington Headquarters Services;
HPO type: Pilot;
Function: Facilities Operations and Maintenance;
Approval date: 12-08-2008;
Implementation date: 10-08-2008.
Organization: U.S. Army, Fort Hood;
HPO type: A-76 MEO;
Function: Fort Hood Directorate of Aviation Operations;
Approval date: 07-28-2009;
Implementation date: 08-21-2009.
Source: DOD Office of the Under Secretary for Acquisition, Technology
and Logistics (Installations and Environment) and each of the above-
listed HPOs.
[End of table]
OSD also issued a memorandum providing procedures and guidance for
implementing and monitoring the performance of, and costs for, the
pilot HPOs.[Footnote 15] OSD's guidance provides performance measures
that somewhat mirror performance measures set out in the 2004 NDAA,
and requires HPOs to develop quality assurance surveillance plans that
include annual reports on four of the five measures.[Footnote 16] The
2004 NDAA contains a list of performance measures that, according to
the law, should be included among performance measures used in the
pilot program. These performance measures include the costs, savings,
and overall financial performance of the organization; organic
knowledge, skills, or expertise; efficiency and effectiveness of key
functions or processes; efficiency and effectiveness of the overall
organization; and general customer satisfaction.
Performance Data Reported to OSD Is Not Fully Reliable:
OSD is collecting performance data from HPOs; however, evaluating the
effectiveness of the HPO initiative is hindered because some data
reported annually by some HPOs are unreliable. Pilot initiatives such
as DOD's HPO initiative typically are used to evaluate alternative
approaches or test new ideas,and gathering reliable data is critical
to assessing the effectiveness of new ways of doing business.[Footnote
17] To be reliable, data must be complete, accurate, meet intended
purposes, and incorporate quality control checks.[Footnote 18]
Ensuring that performance information is reliable requires
documentation that is appropriately detailed and organized, contains
sufficient information to support management's assertion, and includes
appropriate representation from officials and personnel responsible
for monitoring, improving, and assessing internal controls.
Our review of annual reports submitted to OSD over the last 3 years
revealed that 7 of the 13 reports submitted as of March 2010 did not
include information about all of the performance measures set out in
OSD's guidance, which mirror the performance measures set out in the
2004 legislation. The most frequently missed items involved workforce
skills or experience and general customer satisfaction. For example,
we found that 6 of the reports did not contain information on
workforce skills or experience and 5 reports did not contain
information on general customer satisfaction.
Our analysis of the performance information that was provided in the
annual reports also yielded instances in which reported information
was not reliable. For example, in our discussions with managers from
one HPO, they acknowledged that the two annual reports they submitted
were incomplete, presenting a misleading and inaccurate picture of
performance. They explained that the latest annual report identified
cost savings that were achieved by the organization functioning with a
smaller workforce than authorized. However, the smaller workforce was
not the result of a re-engineering effort, but due to the
organization's lengthy hiring process and a high turnover of
employees. As a result, the 300 vacant positions were claimed as cost
savings. At another HPO location, managers claimed over $6 million in
savings in their annual report; however, the report did not fully
explain the extra costs associated with hiring several hundred
temporary workers to perform a workaround operation when a project to
automate a portion of the work processes failed. In addition to these
problems, we found workforce impacts that were not reflected in some
annual reports, which could affect the continued efficiency of the
HPOs. For example, HPO employees and supervisors at several locations
we visited described the impact that vacancies and high turnover rates
were having on the workforce. In some cases, they noted that employees
were leaving their organizations because the employees were overworked
and felt challenged in meeting mission requirements. Without complete
and accurate performance information, OSD does not have reliable data
to evaluate the effectiveness of the overall HPO initiative.
With a Moratorium in Place on the Use of Public-Private Competitions,
the Future of DOD's HPO Initiative Is Unclear:
Although OSD initially asserted a role in implementing the HPO
initiative, according to OSD officials, their interest in establishing
oversight over the HPOs has waned because they believe that incentives
for organizations to participate have been lost. First, in their view,
the moratorium on the use of A-76 public-private competitions has
removed the primary incentive for new organizations to pursue HPO
status. OSD officials contend that the primary incentive to seek HPO
status was to avoid an A-76 public-private competition and for the
time being, the moratorium on the A-76 process is believed to have
removed that incentive. Second, OSD officials believe that the
moratorium could have a potential impact on existing HPOs in
maintaining efficiency gains achieved to date. OSD officials expressed
concerns that if they no longer have the option to subject
organizations to A-76 public-private competitions, current HPOs might
be less willing to continue working toward fulfilling the performance
commitments they have made.
OSD officials told us that as long as there is a moratorium on the use
of OMB's A-76 process, they do not plan to select any additional
organizations for participation under the HPO initiative or take
further action to assess the reliability of information submitted by
the HPOs. However, officials stated that they do plan to review the
annual reports submitted to date to identify lessons learned. In
assessing performance, they intend to focus on reviewing the
performance of some of the reportedly more successful HPOs to gain
insights into potentially effective approaches to improving
organizational performance. Officials explained that the selection of
the HPOs will be subjective and based on information contained in past
HPO annual performance reports. OSD officials stated that they have
not established milestones for this review nor have they developed an
official assessment plan. Officials in the OSD office responsible
acknowledge the need for oversight, but expressed concerns about their
ability to conduct such an assessment with current staffing levels.
Further, they stated that as a result of staff reductions, OSD's
ability to conduct such an evaluation is restricted and thus will have
a very limited scope.
Conclusions:
In light of the current moratorium placed on DOD's use of public-
private competitions--combined with OSD's related reluctance to assert
further oversight over the HPO initiative--the future of defense-
related HPOs is unclear. Although OSD has collected some performance
information, it does not appear to have a plan to ensure that all of
its HPOs report on all performance measures set out in the 2004 NDAA
and provide performance data that could show whether HPOs are making
progress toward the high-performance outcomes established in the
formal agreements with OSD. Also, since some data collected to date
are of questionable reliability, the ability to assess the HPOs is
limited. Facing little oversight and no longer subject to public-
private competitions under the A-76 process, HPOs may not meet their
high-performance objectives. Moreover, new organizations have little
or no incentive to seek HPO status. Looking forward, it is unknown
whether the moratorium on public-private competitions will be extended
beyond the current fiscal year; however, the recent trend seems to be
toward seeking efficiencies in-house rather than through competitions
with the private sector. OSD's limited oversight of the HPO initiative
to date curtails the opportunity to assess whether HPOs offer a viable
option to achieve better value without a private-sector competition.
However, given DOD's investment in the HPO initiative and its
potential to serve as an alternative to the A-76 process, it is our
view that DOD should make it a priority to assess the HPO initiative
thoroughly to determine how it has performed and whether it has
resulted in improved organizational performance.
Recommendations for Executive Action:
To determine the effectiveness of the HPO initiative and provide
future direction for the programs, we recommend that the Secretary of
Defense direct the Under Secretary of Defense for Acquisition,
Technology and Logistics to take the following two actions:
* Assess the reliability of the data provided by the HPOs and take
steps to ensure reporting and collection of reliable data; and:
* Use the performance data to fully evaluate the effectiveness of the
DOD's HPO initiative using all performance measures set out in the
National Defense Authorization Act for Fiscal Year 2004.
Agency Comments and Our Evaluation:
We provided DOD a draft of this report for review and comment. DOD's
written comments are reprinted as enclosure II. DOD partially
concurred with our recommendations and provided technical comments,
which we have incorporated where appropriate.
DOD partially concurred with our first recommendation to assess the
reliability of the data provided by the HPOs and take steps to ensure
the reporting and collection of reliable data. DOD stated that DOD
components are accountable for collecting and reporting data for their
respective HPOs and cited staff reductions in the office responsible
for overseeing the HPO initiative. DOD did not address whether it
would assess the reliability of the data or take steps to ensure that
reliable data are reported and collected. As we point out in this
report, DOD has required the DOD components to submit data on HPO
performance and has been collecting this data from the HPOs since the
onset of the program. We found instances in which unreliable data were
reported to DOD and found no mechanisms in place to ensure that the
data that were being reported and collected were reliable. We continue
to believe additional steps are needed to ensure that data collected
on HPOs are reliable. Without complete and accurate performance data,
DOD's ability to evaluate the effectiveness of the overall HPO
initiative will be limited.
DOD also partially concurred with our second recommendation to fully
evaluate the effectiveness of DOD's HPO initiative by using all
performance measures set out in the National Defense Authorization Act
for Fiscal Year 2004. DOD stated that it would conduct an evaluation
of the HPOs' performance using data already submitted by each of the
components. However, we remain concerned that the data reliability
problems we identified could limit the usefulness of this assessment
and that DOD should work with the components to obtain reliable data.
The department also noted that the effectiveness of an HPO is directly
influenced by the incentive for organizations to avoid public-private
competitions and added that past experiences with HPO-like business
process re-engineering efforts suggested that savings would be
minimal. Although DOD stated that it will continue to encourage
efficiency initiatives and will promulgate lessons learned to help
organizations achieve that goal, DOD's response provides little detail
on how it will address the data reliability problems that we have
identified. Until DOD fully evaluates the effectiveness of its HPO
initiative, DOD will not be able to determine whether the
organizations designated as HPOs have met their goals of greater cost
savings and increased efficiencies, nor will it be able to determine
whether the HPO option is a viable option for other organizations to
achieve these goals.
We are sending copies of this report to other congressional committees
and interested parties. We are also sending copies to the Secretaries
of Defense, Army, and Air Force, and to the directors of defense
agencies with approved HPOs. In addition, this report will be
available at no charge on our Web site at [hyperlink,
http://www.gao.gov]. If you or your staff have any questions about
this report, please contact me at (404) 679-1816 or
pendletonj@gao.gov. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made key contributions to this report are listed
in enclosure III.
Signed by:
John Pendleton, Director:
Defense Capabilities and Management:
[End of section]
Enclosure I:
Summary of DOD's High Performing Organizations:
U.S. Air Force: Edwards Air Force Base, 412th Maintenance Group:
The Air Force 412th Maintenance Group, located at Edwards Air Force
Base, Lancaster, California, was approved as a High Performing
Organization (HPO) on February 16, 2006. This HPO is responsible for
maintaining numerous Air Force developmental and legacy aircraft of
varying design and subsystems and provides logistics test and
evaluation for aircraft assigned or attached to the Air Force Flight
Test Center.[Footnote 19] The HPO also supports new acquisitions, such
as the Joint Strike Fighter (JSF), and system modification programs,
such as those employed in the F-16 Fighting Falcon.
Defense Contract Management Agency: 1106 Procurement Technician Job
Series:
The Defense Contract Management Agency's (DCMA) Procurement Technician
Job series was approved as an HPO on February 16, 2006. DCMA
Procurement Technicians are responsible for various data input tasks
involving several Department of Defense (DOD) procurement databases.
The Procurement Technician workforce is dispersed among DCMA's six
divisions and contract management offices. As of December 2009, DCMA
had 47 field offices worldwide, 42 of which are located in the United
States.
Defense Finance and Accounting Service: Limestone Maine Accounting
Function:
The Defense Finance and Accounting Service (DFAS) Limestone Accounting
operation was approved as an HPO on January 11, 2008. DFAS Limestone
provides service to all of the operational Air Force, including Air
Combat Command, Air Force Special Operations Command, U.S. Air Forces
Europe, and Air Education and Training Command. Services include
appropriated funds accounting and reporting, commercial pay, working
capital funds accounting and reporting, travel accounting, and
accounts receivable. This function is centralized at the Limestone,
Maine location.
Defense Logistics Agency: Document Automation and Production Service:
The Defense Logistics Agency Document Automation and Production
Service (DLA-DAPS) was approved for conversion from an OMB Circular A-
76 most efficient organization to an HPO on May 13, 2008. The mission
of DLA-DAPS HPO is to actively promote document automation by
identifying and proposing technology enhancements and other innovative
solutions to facilitate and expedite DOD's desired transition to a
paperless environment. This HPO operates from 143 locations across the
United States.
Defense Logistics Agency: Human Resources Center:
The Defense Logistics Agency Human Resources Center (DLA-HRC) was
among the first HPOs approved by OSD on February 12, 2006. The DLA-HR
function provides Human Resources services to DLA employees and other
DOD Components at locations worldwide. DLA's human resources functions
were consolidated into a single Human Resources center at Fort
Belvoir, Virginia and two customer support offices located in
Columbus, Ohio and New Cumberland, Pennsylvania.
U.S. Army Corps of Engineers Logistics Management:
The U.S. Army Corps of Engineers (USACE) Logistics Management function
was approved as an HPO on February 12, 2006. The HPO supports USACE's
activities by providing direction, coordination, technical guidance,
and services for logistics functions and business processes;
integrated logistics support; management controls; and strategic
planning. The HPO also supports 53 USACE offices located throughout
the United States.
U.S. Army Installation Management Command: Fort Huachuca, Adjutant
General Directorate:
The Fort Huachuca Personnel Function was approved for conversion from
an OMB Circular A-76 most efficient organization to an HPO on October
25, 2007. Among other things, the Fort Huachuca Adjutant General (AG)
HPO provides military personnel services, such as issuance of
identification cards, pre-and post-transition services, and human
capital system automation services.
U.S. Army Installation Management Command: Fort Hood, Directorate of
Aviation Operations:
On July 28, 2009, the U.S. Army Installation Management Command's
Director of Aviation Operations at Fort Hood, Texas was approved for
conversion from an OMB Circular A-76 most efficient organization to an
HPO. The organization is responsible for managing Airfield and Flight
Simulator Operations at the Fort Hood Army Installation.
Washington Headquarters Services: Federal Facilities Division:
The Washington Headquarters Services: Federal Facilities Division (WHS-
FFD) was approved as an HPO on December 8, 2008. This organization
provides facilities maintenance, repair, construction, and logistics
support services to Department of Defense activities in the National
Capital Region, and operates primarily on the Pentagon Reservation and
in delegated facilities, such as U.S. Court of Military Appeals in
Washington, D.C. and the Defense Logistics agency in Arlington,
Virginia.
[End of Enclosure I]
Enclosure II: Comments from the Department of Defense:
Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition, Technology And Logistics:
3010 Defense Pentagon:
Washington, DC 20301-3010:
May 7, 2010:
Mr. John Pendleton:
Director:
Defense Capabilities and Management:
U.S. General Accounting Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Pendleton:
This is the Department of Defense (DoD) response to the General
Accountability Office (GAO) Draft Report GAO-10-566R, "Defense
Management: DoD Needs to Assess Effectiveness and Determine Future
Direction for Its High Performing Organizations Initiative," April 7,
2010 (GAO Code 351337). The Department of Defense (DoD) partially
concurs with the draft report.
Public Law 108-136 H.R. 1588 SEC. 337 defines "Secretary concerned" as
"the Secretary of a military department and the Secretary of Defense
with respect to matters concerning the Defense Agencies." The
Secretary of Defense was designated as the lead to establish the High
Performing Organization (HPO) pilot program and completed the task as
required.
We partially concur with the recommendation that the Under Secretary
of Defense for Acquisition, Technology and Logistics should assess the
reliability of the data provided by the HPOs and take steps to ensure
reporting and collection of reliable data. The DoD Components are
accountable for the collection and reporting data on their own high-
performing organization (HPO). The Secretary concerned manages its own
resources, namely funding and manpower. The Secretary concerned must
budget for his or her own organization and ultimately must ensure they
are in compliance with their individual budget plans submitted to the
Office of the Secretary of Defense (OSD). The Secretary concerned
assesses whether the HPOs will receive continued exemption from
public/private competition as authorized by the statute. The current
moratorium on public-private competition, however, obviates the need
for assessment for purposes of exemption. Since OSD has significantly
reduced the size of its competitive sourcing office, such assessment
is no longer feasible.
We partially concur with the recommendation of "using performance data
to fully evaluate the effectiveness of the DoD's HPO initiative using
all performance measures set out in the National Defense Authorization
Act for FY2004." The effectiveness of an HPO is directly influenced by
the incentive for organizations to avoid public-private competition.
Absent public-private competition, HPOs have little incentive to make
difficult management decisions. While evaluation of HPO performance
data as submitted by the Components will be performed, experience with
earlier reengineering efforts suggest savings will be minimal.
Nonetheless, we continue to encourage efficiency initiatives and will
promulgate lessons learned to help organizations achieve that goal.
I appreciate the opportunity to comment on the draft report. If you
have any questions about the technical corrections that have been
provided or this response, please contact my point of contact, Ms.
Laura Montoya at 703.602.4469.
Sincerely,
Signed by:
Joseph K. Sikes:
Director, Housing and Competitive Sourcing:
Enclosure:
Department of Defense (DoD) Technical Corrections To the General
Accountability Office (GAO) Draft Report GAO-10-566R, "Defense
Management: DoD Needs to Assess Effectiveness and Determine Future
Direction for Its High Performing Organizations Initiative," May 2010
(GAO Code 351337).
[End of Enclosure II]
Enclosure III: GAO Contact and Staff Acknowledgments:
GAO Contact:
John Pendleton, (404) 679-1816 or pendletonj@gao.gov:
Staff Acknowledgments:
In addition to the person named above, Elizabeth McNally, Assistant
Director; Owen D. Bruce; Kurt A. Burgeson; Grace A. Coleman; Janida
Grima; Pamela (Nicole) Harris; Karen (Nicole) Harms; Richard Powelson;
Steven R. Putansu; Jerome T. Sandau; and William T. Woods made key
contributions to this report.
[End of Enclosure III]
Footnotes:
[1] National Defense Authorization Act for Fiscal Year 2004, Pub. L.
No. 108-136 §337 (2003). Specifically, the law requires the Secretary
of Defense to establish a pilot program under which the secretary
concerned shall create, or continue the implementation of, high-
performing organizations through the conduct of a Business Process Re-
engineering initiative at selected military installations and
facilities under the jurisdiction of the secretary concerned. The law
defines "secretary concerned," for purposes of section 337 as the
secretary of a military department and the Secretary of Defense, with
respect to matters concerning the defense agencies.
[2] DOD's HPO initiative includes six components that were selected as
part of the HPO pilot program and three that transitioned from MEO to
HPO status.
[3] Omnibus Appropriations Act, 2009, Pub. L. No. 111-8, §737 (2009).
[4] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, §735
(2009).
[5] National Defense Authorization Act for Fiscal Year 2010, Pub. L.
No. 111-84, §325 (2009).
[6] A formal agreement that an agency implements when a standard or
streamlined competition results in agency performance.
[7] HPO sites visited included: (1) Edwards Air Force Base (Lancaster,
Calif.); (2) U.S. Army Corps of Engineers (Millington, Tenn.; Seattle,
Wash.; Portland, Ore.; and San Francisco, Calif.); (3) Defense Finance
and Accounting Service (Limestone, Maine); (4) Defense Logistics
Agency Human Resources Activity (New Cumberland, Pa.); (5) Defense
Logistics Agency Document Automation and Production Service
(Philadelphia, Pa.); (6) Defense Contract Management Agency
(Baltimore, Md.); (7) Washington Headquarters Services (Arlington,
Va.); and (8) Fort Huachuca Personnel Management (Fort Huachuca,
Ariz.).
[8] We did not conduct a site visit to the U.S. Army, Fort Hood HPO
because we were notified of Fort Hood's HPO status after completing
our site visits. However, we obtained and analyzed the HPO's letter of
obligation with OSD.
[9] Office of the Under Secretary of Defense, Acquisition, Technology,
and Logistics Memorandum, Procedures for Implementing, Monitoring, and
Reporting High Performing Organizations (HPO) (Oct. 4, 2006).
[10] Pub. L. No. 108-136 §337 (h) (2003).
[11] These measures include (1) costs, savings, and overall financial
performance of the organization; (2) organic knowledge, skills, or
expertise; (3) efficiency and effectiveness of key functions or
processes; (4) efficiency and effectiveness of the overall
organization; and (5) general customer satisfaction.
[12] OMB defines a most efficient organization (MEO) as the staffing
plan of the agency developed to represent the agency's most efficient
and cost-effective organization.
[13] OMB Circular A-76 (Revised), Performance of Commercial Activities
¶ 5. c. (May 29, 2003).
[14] Pub. L. No. 108-136 §337 (2003).
[15] Office of the Under Secretary of Defense, Acquisition,
Technology, and Logistics Memorandum, Procedures for Implementing,
Monitoring, and Reporting High Performing Organizations (HPO) (Oct. 4,
2006).
[16] The memorandum requires the quality assurance surveillance plans
developed during business process re-engineering to include an annual
report on, at a minimum, organic knowledge, skills, or experience;
efficiency and effectiveness of key functions or processes; efficiency
and effectiveness of the overall organization; and general customer
satisfaction.
[17] GAO, Highlights of a GAO Forum on High-Performing Organizations:
Metrics, Means, and Mechanisms for Achieving High Performance in the
21st Century Public Management Environment, [hyperlink,
http://www.gao.gov/products/GAO-04-343SP] (Washington, D.C.: Feb. 13,
2004).
[18] GAO, Assessing the Reliability of Computer-Processed Data,
[hyperlink, http://www.gao.gov/products/GAO-09-365G] (Washington D.C.,
Feb. 2, 2009).
[19] Developmental aircraft that are maintained at Edwards Air Force
Base include the F/A-22 Raptor, F-117 Nighthawk, F-35 Joint Strike
Fighter, and the CV-22 Osprey, among other aircraft. Legacy aircraft
at Edwards Air Force Base include the F-16 Fighting Falcon; T-38
Talon; KC-135 Stratotanker; as well as the B-1, B-2, and B-52 Bombers.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
