Defense Management

DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative Gao ID: GAO-10-566R May 27, 2010

The Department of Defense (DOD) has sought improved efficiencies and cost reductions in its delivery of services that could be provided by the private sector, using both competitions with private companies and processes to create high performing organizations (HPO). The Office of Management and Budget's (OMB) Circular A-76 establishes federal policy for the competition of commercial activities. According to the circular, the longstanding policy of the federal government has been to rely on the private sector for needed commercial services. To ensure that the American people receive maximum value for their tax dollars, it is the federal government's policy that commercial activities should be subject to the forces of competition. As the largest federal agency, DOD has conducted more A-76 competitions than any other federal agency. However, the A-76 process has drawn criticism from both the public and private sectors. These criticisms largely center on the costs and length of time required to conduct competitions and the manner in which long-term savings are calculated. In light of these concerns, a panel of public and private sector experts convened in 2001 to identify ways in which the federal government could improve the A-76 process and included an option that focused on improving efficiencies in-house through the creation of HPOs rather than seeking improved efficiencies through public-private competitions. Although the A-76 process has been DOD's preferred method for ensuring the most efficient operation of a function, it is currently subject to a number of recently enacted statutory limitations. Given these relatively new developments and the potential for HPOs to serve as an alternative to the A-76 process, you asked us to examine the extent to which the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (OSD) has made progress in implementing and evaluating DOD's HPO initiative.

OSD has made some progress in implementing DOD's HPO initiative by providing guidance to organizations that it selected to participate, but is not always collecting reliable data and has no clear plan to evaluate the effectiveness of the HPO initiative. Pilot initiatives such as this are typically used to evaluate alternative approaches or test new ideas. Gathering reliable data--data that are complete, accurate, and meet intended purposes--and measuring performance are critical to assessing the effectiveness of new ways of doing business. To implement DOD's HPO initiative, OSD selected nine organizations and issued guidance providing procedures for implementing and monitoring the performance of, and costs for, the HPOs. OSD's guidance provides performance measures that somewhat mirror performance measures set out in the National Defense Authorization Act (NDAA) for Fiscal Year 2004. The 2004 NDAA contains a list of performance measures that, according to the law, should be included among performance measures used in the pilot program. Our review of the annual reports submitted to OSD revealed that while all HPOs generally used the broad performance measures set out in the 2004 NDAA to report on performance, several reports did not include information on all of the performance measures. We found a number of instances in which the HPOs failed to include information on workforce expertise and customer satisfaction. Further, for the performance measures for which information was provided, we found instances of incomplete and inaccurate reporting. For example, in our review of annual reports covering the 2008 reporting cycle, one annual report did not clearly capture the costs associated with hiring temporary employees and the overtime required to augment the much smaller organization in response to unexpected workload increases. Moreover, although most HPOs have submitted at least one annual report, OSD has not used the information to evaluate the HPO initiative. Although OSD initially asserted a role in implementing the HPO initiative, according to OSD officials, their interest in overseeing the HPOs has waned because with the ongoing moratorium on new public-private competitions, they believe that organizations will not have any incentive to participate in the HPO process. Moreover, they believe that with the A-76 moratorium in place, existing HPOs might be less willing to continue working toward fulfilling the performance commitments they have made. Therefore, OSD officials stated that while they do encourage organizations to become more efficient, their position is that selecting new organizations for HPO designation is of questionable value due to the current moratorium on public-private competitions. However, unless OSD assesses the reliability of the information provided by the HPOs and uses reliable performance data to make decisions, DOD may miss opportunities to sustain efficiencies gained by existing HPOs, and will be unable to make management decisions and inform policymakers on the effectiveness of the HPO initiative.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: John H. Pendleton Team: Government Accountability Office: Defense Capabilities and Management Phone: (404) 679-1816


GAO-10-566R, Defense Management: DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative This is the accessible text file for GAO report number GAO-10-566R entitled 'Defense Management: DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative' which was released on May 27, 2010. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. GAO-10-566R: United States Government Accountability Office: Washington, DC 20548: May 27, 2010: The Honorable Ike Skelton: Chairman: The Honorable Howard P. McKeon: Ranking Member: Committee on Armed Services: United States House of Representatives: Subject: Defense Management: DOD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative: The Department of Defense (DOD) has sought improved efficiencies and cost reductions in its delivery of services that could be provided by the private sector, using both competitions with private companies and processes to create high performing organizations (HPO). The Office of Management and Budget's (OMB) Circular A-76 establishes federal policy for the competition of commercial activities. According to the circular, the longstanding policy of the federal government has been to rely on the private sector for needed commercial services. To ensure that the American people receive maximum value for their tax dollars, it is the federal government's policy that commercial activities should be subject to the forces of competition. As the largest federal agency, DOD has conducted more A-76 competitions than any other federal agency. However, the A-76 process has drawn criticism from both the public and private sectors. These criticisms largely center on the costs and length of time required to conduct competitions and the manner in which long-term savings are calculated. In light of these concerns, a panel of public and private sector experts convened in 2001 to identify ways in which the federal government could improve the A-76 process and included an option that focused on improving efficiencies in-house through the creation of HPOs rather than seeking improved efficiencies through public-private competitions. In 2003, Congress passed legislation that directed the Secretary of Defense to establish a pilot program for the creation or continued implementation of HPOs: through the conduct of a business process re-engineering effort. [Footnote 1] Defense organizations seeking participation under this pilot program are required to re-engineer their business processes to improve efficiencies and cost effectiveness. A 2003 revision to OMB's Circular A-76 also allowed entities that were designated as Most Efficient Organizations (MEO) as part of the A-76 process to seek HPO designation, thereby avoiding future public-private competitions for a 3-to 5-year period. On July 26, 2005, the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (OSD) solicited nominations from defense organizations seeking HPO designation as part of its HPO initiative and has since designated nine organizations from across DOD as HPOs.[Footnote 2] Although the A-76 process has been DOD's preferred method for ensuring the most efficient operation of a function, it is currently subject to a number of recently enacted statutory limitations. For example, the Omnibus Appropriations Act for 2009[Footnote 3] contains a provision that prohibited the use of funds for beginning or announcing a study or public-private competition under A-76, and the same language was included in the Consolidated Appropriations Act for 2010[Footnote 4] extending that prohibition. Additionally, section 325 of the National Defense Authorization Act for Fiscal Year 2010[Footnote 5] temporarily suspends public-private competition for conversion of DOD functions to performance by contractors until the Secretary of Defense satisfies a number of requirements, including, among other things, the completion of a comprehensive review of the department's policies with respect to the conduct of public-private competitions and submission of a report on that review to the congressional defense committees. According to OSD and HPO officials, these legislative actions have effectively imposed a moratorium on public-private competitions. Given these relatively new developments and the potential for HPOs to serve as an alternative to the A-76 process, you asked us to examine the extent to which OSD has made progress in implementing and evaluating DOD's HPO initiative. Scope and Methodology: To examine the extent of OSD's progress in implementing and evaluating its HPO initiative, we focused on the nine organizations designated by OSD as HPOs. The nine organizations included in our review were: * Defense Contract Management Agency: Procurement Technicians, * Defense Finance and Accounting Service: Accounting Services, * Defense Logistics Agency: Document Automation Production and Service, * Defense Logistics Agency: Human Resources, * Washington Headquarters Services: Federal Facilities Division, * U.S. Air Force: Edwards Air Force Base Aircraft Maintenance, * U.S. Army: Corps of Engineers Logistics Management, * U.S. Army, Fort Hood: Directorate of Aviation Operations, and: * U.S. Army: Fort Huachuca Installation Personnel Management. We obtained and analyzed relevant statutory provisions, regulations related to the HPO process, and DOD and OMB guidance related to the HPO initiative. We also reviewed OSD's criteria for selecting organizations, letters of obligation[Footnote 6] between OSD and participating organizations, and the annual reports documenting the performance of each of the HPOs. We met with OSD officials to discuss their efforts to implement and assess the effectiveness of DOD's HPO initiative. During site visits to HPO locations, we interviewed officials and conducted discussion groups with employees at eight of the nine HPOs.[Footnote 7] Following the discussion groups, we asked employees to anonymously complete a self-administered questionnaire to give them an opportunity to summarize their thoughts on the discussion and provide input on any topics discussed that may have been considered sensitive. Although we did not visit the U.S. Army: Fort Hood Directorate of Aviation Operations, we reviewed the documentation associated with OSD's approval of the HPO.[Footnote 8] In an effort to protect the anonymity of individuals with whom we spoke, we did not specify the names of the HPOs in the examples presented in our report. We conducted this performance audit from April 2009 through May 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Summary: OSD has made some progress in implementing DOD's HPO initiative by providing guidance to organizations that it selected to participate, but is not always collecting reliable data and has no clear plan to evaluate the effectiveness of the HPO initiative. Pilot initiatives such as this are typically used to evaluate alternative approaches or test new ideas. Gathering reliable data--data that are complete, accurate, and meet intended purposes--and measuring performance are critical to assessing the effectiveness of new ways of doing business.To implement DOD's HPO initiative, OSD selected nine organizations and issued guidance providing procedures for implementing and monitoring the performance of, and costs for, the HPOs.[Footnote 9] OSD's guidance provides performance measures that somewhat mirror performance measures set out in the National Defense Authorization Act (NDAA) for Fiscal Year 2004.[Footnote 10] The 2004 NDAA contains a list of performance measures that, according to the law, should be included among performance measures used in the pilot program.[Footnote 11] Our review of the annual reports submitted to OSD revealed that while all HPOs generally used the broad performance measures set out in the 2004 NDAA to report on performance, several reports did not include information on all of the performance measures. We found a number of instances in which the HPOs failed to include information on workforce expertise and customer satisfaction. Further, for the performance measures for which information was provided, we found instances of incomplete and inaccurate reporting. For example, in our review of annual reports covering the 2008 reporting cycle, one annual report did not clearly capture the costs associated with hiring temporary employees and the overtime required to augment the much smaller organization in response to unexpected workload increases. Moreover, although most HPOs have submitted at least one annual report, OSD has not used the information to evaluate the HPO initiative. Although OSD initially asserted a role in implementing the HPO initiative, according to OSD officials, their interest in overseeing the HPOs has waned because with the ongoing moratorium on new public-private competitions, they believe that organizations will not have any incentive to participate in the HPO process. Moreover, they believe that with the A-76 moratorium in place, existing HPOs might be less willing to continue working toward fulfilling the performance commitments they have made. Therefore, OSD officials stated that while they do encourage organizations to become more efficient, their position is that selecting new organizations for HPO designation is of questionable value due to the current moratorium on public-private competitions. However, unless OSD assesses the reliability of the information provided by the HPOs and uses reliable performance data to make decisions, DOD may miss opportunities to sustain efficiencies gained by existing HPOs, and will be unable to make management decisions and inform policymakers on the effectiveness of the HPO initiative. We are making two recommendations to provide future direction for DOD's HPO initiative that include assessing the reliability of the data provided to date by the HPOs and evaluating the effectiveness of the HPO initiative against the performance measures set out in the 2004 NDAA. In written comments on a draft of this report, DOD partially concurred with our recommendations. In response to our recommendation to assess the reliability of the data provided by the HPOs, DOD stated that the DOD components are accountable for collecting and reporting data for their respective HPOs. As we reported, DOD has required the components to submit data on HPO performance, but has not assessed the reliability of the data. We continue to believe that gathering reliable data is essential to assessing the effectiveness of alternative approaches and, without reliable data, DOD's ability to evaluate the HPO program will be limited. In response to our recommendation to use performance data to fully evaluate the effectiveness of DOD's HPO initiative, DOD stated that it would conduct an evaluation of the HPOs' performance using data already submitted by each of the components. However, we remain concerned that the data reliability problems we identified could limit the usefulness of this assessment and that DOD should work with the components to obtain reliable data. In addition, the department provided technical comments, which we incorporated in the report where appropriate. DOD's comments are reprinted in enclosure II. Background: OMB's Circular A-76 is a governmentwide executive branch policy that subjects commercial activities and functions performed by federal government employees to public-private competition. According to the circular, the longstanding policy of the federal government has been to rely on the private sector for needed commercial services. To ensure that the American people receive maximum value for their tax dollars, it is the federal government's policy that commercial activities should be subject to the forces of competition. The HPO option is a departure from the A-76 process because it involves improving efficiencies in-house rather than seeking improved efficiencies through public-private competitions. To become an HPO, any federal organization could conduct an analysis and a re- engineering of mission and support functions and processes to achieve improvements in performance. The incentives for an organization to use this approach were twofold: first, an organization could achieve improvements in-house and avoid entering into a public-private competition, and second, the number of positions were counted toward meeting public-private competition goals established for each component. Responsibility for preparing the overarching management policy that was to be used to develop and implement DOD's HPO initiative was delegated to the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics-Installations and Environment (OSD). Defense organizations seeking the HPO designation had two avenues that they could pursue: * Revised Circular A-76. As part of the Circular A-76 process, federal agencies create a staffing plan to be submitted with the agency's tender offer in response to a solicitation for a standard competition. This staffing plan represents the agency's Most Efficient Organization (MEO).[Footnote 12] A 2003 revision to OMB Circular A-76 added a provision that encourages agencies to use a formalized process for deviating from the requirements of A-76 to explore innovative alternatives to standard or streamlined competitions, including HPOs.[Footnote 13] Following the revision to Circular A-76, several existing MEOs were converted to HPOs, which enabled the agencies to avoid recompeting their function at the end of their initial MEO performance period. * DOD's HPO Pilot Program. The 2004 NDAA included a provision that directed the Secretary of Defense to establish a pilot program for the creation or continued implementation of HPOs through the conduct of a business process re-engineering effort.[Footnote 14] To be eligible for selection to participate, organizations are required to, among other things, complete a total organizational assessment that would result in enhanced performance measures at least comparable to those that might be achieved through public-private competitions. After an organization selected to participate in the pilot program implements the business process reengineering initiative, the secretary concerned is required to determine whether the organization has achieved initial progress toward designation as an HPO. In the absence of such progress, the organization's participation in the HPO pilot is to be terminated. OSD Has Made Some Progress Implementing DOD's HPO Initiative, but Has No Clear Plans to Evaluate Its Effectiveness: OSD Selected Organizations and Provided Guidance for Implementing and Evaluating HPOs: To implement DOD's HPO initiative, which includes participants in the HPO pilot program and MEOs created for the A-76 process that converted to HPOs, OSD approved nine organizations for participation. (See table 1 for a list of approved organizations and enclosure I for a brief description of each HPO.) Table 1: DOD's High-Performing Organizations: Organization: U.S. Air Force, Edwards Air Force Base, 412[TH] Maintenance Group; HPO type: Pilot; Function: Aircraft Maintenance; Approval date: 02-16-2006; Implementation date: 10-01-2006. Organization: Defense Logistics Agency; HPO type: Pilot; Function: Human Resources; Approval date: 02-16-2006; Implementation date: 04-01-2007. Organization: Defense Contract Management Agency; HPO type: Pilot; Function: Procurement Technicians; Approval date: 02-16-2006; Implementation date: 11-07-2006. Organization: U.S. Army Corps of Engineers; HPO type: Pilot; Function: Logistics Management; Approval date: 02-16-2006; Implementation date: 07-23-2007. Organization: U.S. Army, Fort Huachuca; HPO type: A-76 MEO; Function: Personnel Management; Approval date: 10-01-2007; Implementation date: 10-01-2007. Organization: Defense Finance and Accounting Service; HPO type: Pilot; Function: Accounting Services; Approval date: 01-11-2008; Implementation date: 06-01-2008. Organization: Defense Logistics Agency; HPO type: A-76 MEO; Function: Document Automation Production Services; Approval date: 05-13-2008; Implementation date: 10-01-2009. Organization: Washington Headquarters Services; HPO type: Pilot; Function: Facilities Operations and Maintenance; Approval date: 12-08-2008; Implementation date: 10-08-2008. Organization: U.S. Army, Fort Hood; HPO type: A-76 MEO; Function: Fort Hood Directorate of Aviation Operations; Approval date: 07-28-2009; Implementation date: 08-21-2009. Source: DOD Office of the Under Secretary for Acquisition, Technology and Logistics (Installations and Environment) and each of the above- listed HPOs. [End of table] OSD also issued a memorandum providing procedures and guidance for implementing and monitoring the performance of, and costs for, the pilot HPOs.[Footnote 15] OSD's guidance provides performance measures that somewhat mirror performance measures set out in the 2004 NDAA, and requires HPOs to develop quality assurance surveillance plans that include annual reports on four of the five measures.[Footnote 16] The 2004 NDAA contains a list of performance measures that, according to the law, should be included among performance measures used in the pilot program. These performance measures include the costs, savings, and overall financial performance of the organization; organic knowledge, skills, or expertise; efficiency and effectiveness of key functions or processes; efficiency and effectiveness of the overall organization; and general customer satisfaction. Performance Data Reported to OSD Is Not Fully Reliable: OSD is collecting performance data from HPOs; however, evaluating the effectiveness of the HPO initiative is hindered because some data reported annually by some HPOs are unreliable. Pilot initiatives such as DOD's HPO initiative typically are used to evaluate alternative approaches or test new ideas,and gathering reliable data is critical to assessing the effectiveness of new ways of doing business.[Footnote 17] To be reliable, data must be complete, accurate, meet intended purposes, and incorporate quality control checks.[Footnote 18] Ensuring that performance information is reliable requires documentation that is appropriately detailed and organized, contains sufficient information to support management's assertion, and includes appropriate representation from officials and personnel responsible for monitoring, improving, and assessing internal controls. Our review of annual reports submitted to OSD over the last 3 years revealed that 7 of the 13 reports submitted as of March 2010 did not include information about all of the performance measures set out in OSD's guidance, which mirror the performance measures set out in the 2004 legislation. The most frequently missed items involved workforce skills or experience and general customer satisfaction. For example, we found that 6 of the reports did not contain information on workforce skills or experience and 5 reports did not contain information on general customer satisfaction. Our analysis of the performance information that was provided in the annual reports also yielded instances in which reported information was not reliable. For example, in our discussions with managers from one HPO, they acknowledged that the two annual reports they submitted were incomplete, presenting a misleading and inaccurate picture of performance. They explained that the latest annual report identified cost savings that were achieved by the organization functioning with a smaller workforce than authorized. However, the smaller workforce was not the result of a re-engineering effort, but due to the organization's lengthy hiring process and a high turnover of employees. As a result, the 300 vacant positions were claimed as cost savings. At another HPO location, managers claimed over $6 million in savings in their annual report; however, the report did not fully explain the extra costs associated with hiring several hundred temporary workers to perform a workaround operation when a project to automate a portion of the work processes failed. In addition to these problems, we found workforce impacts that were not reflected in some annual reports, which could affect the continued efficiency of the HPOs. For example, HPO employees and supervisors at several locations we visited described the impact that vacancies and high turnover rates were having on the workforce. In some cases, they noted that employees were leaving their organizations because the employees were overworked and felt challenged in meeting mission requirements. Without complete and accurate performance information, OSD does not have reliable data to evaluate the effectiveness of the overall HPO initiative. With a Moratorium in Place on the Use of Public-Private Competitions, the Future of DOD's HPO Initiative Is Unclear: Although OSD initially asserted a role in implementing the HPO initiative, according to OSD officials, their interest in establishing oversight over the HPOs has waned because they believe that incentives for organizations to participate have been lost. First, in their view, the moratorium on the use of A-76 public-private competitions has removed the primary incentive for new organizations to pursue HPO status. OSD officials contend that the primary incentive to seek HPO status was to avoid an A-76 public-private competition and for the time being, the moratorium on the A-76 process is believed to have removed that incentive. Second, OSD officials believe that the moratorium could have a potential impact on existing HPOs in maintaining efficiency gains achieved to date. OSD officials expressed concerns that if they no longer have the option to subject organizations to A-76 public-private competitions, current HPOs might be less willing to continue working toward fulfilling the performance commitments they have made. OSD officials told us that as long as there is a moratorium on the use of OMB's A-76 process, they do not plan to select any additional organizations for participation under the HPO initiative or take further action to assess the reliability of information submitted by the HPOs. However, officials stated that they do plan to review the annual reports submitted to date to identify lessons learned. In assessing performance, they intend to focus on reviewing the performance of some of the reportedly more successful HPOs to gain insights into potentially effective approaches to improving organizational performance. Officials explained that the selection of the HPOs will be subjective and based on information contained in past HPO annual performance reports. OSD officials stated that they have not established milestones for this review nor have they developed an official assessment plan. Officials in the OSD office responsible acknowledge the need for oversight, but expressed concerns about their ability to conduct such an assessment with current staffing levels. Further, they stated that as a result of staff reductions, OSD's ability to conduct such an evaluation is restricted and thus will have a very limited scope. Conclusions: In light of the current moratorium placed on DOD's use of public- private competitions--combined with OSD's related reluctance to assert further oversight over the HPO initiative--the future of defense- related HPOs is unclear. Although OSD has collected some performance information, it does not appear to have a plan to ensure that all of its HPOs report on all performance measures set out in the 2004 NDAA and provide performance data that could show whether HPOs are making progress toward the high-performance outcomes established in the formal agreements with OSD. Also, since some data collected to date are of questionable reliability, the ability to assess the HPOs is limited. Facing little oversight and no longer subject to public- private competitions under the A-76 process, HPOs may not meet their high-performance objectives. Moreover, new organizations have little or no incentive to seek HPO status. Looking forward, it is unknown whether the moratorium on public-private competitions will be extended beyond the current fiscal year; however, the recent trend seems to be toward seeking efficiencies in-house rather than through competitions with the private sector. OSD's limited oversight of the HPO initiative to date curtails the opportunity to assess whether HPOs offer a viable option to achieve better value without a private-sector competition. However, given DOD's investment in the HPO initiative and its potential to serve as an alternative to the A-76 process, it is our view that DOD should make it a priority to assess the HPO initiative thoroughly to determine how it has performed and whether it has resulted in improved organizational performance. Recommendations for Executive Action: To determine the effectiveness of the HPO initiative and provide future direction for the programs, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology and Logistics to take the following two actions: * Assess the reliability of the data provided by the HPOs and take steps to ensure reporting and collection of reliable data; and: * Use the performance data to fully evaluate the effectiveness of the DOD's HPO initiative using all performance measures set out in the National Defense Authorization Act for Fiscal Year 2004. Agency Comments and Our Evaluation: We provided DOD a draft of this report for review and comment. DOD's written comments are reprinted as enclosure II. DOD partially concurred with our recommendations and provided technical comments, which we have incorporated where appropriate. DOD partially concurred with our first recommendation to assess the reliability of the data provided by the HPOs and take steps to ensure the reporting and collection of reliable data. DOD stated that DOD components are accountable for collecting and reporting data for their respective HPOs and cited staff reductions in the office responsible for overseeing the HPO initiative. DOD did not address whether it would assess the reliability of the data or take steps to ensure that reliable data are reported and collected. As we point out in this report, DOD has required the DOD components to submit data on HPO performance and has been collecting this data from the HPOs since the onset of the program. We found instances in which unreliable data were reported to DOD and found no mechanisms in place to ensure that the data that were being reported and collected were reliable. We continue to believe additional steps are needed to ensure that data collected on HPOs are reliable. Without complete and accurate performance data, DOD's ability to evaluate the effectiveness of the overall HPO initiative will be limited. DOD also partially concurred with our second recommendation to fully evaluate the effectiveness of DOD's HPO initiative by using all performance measures set out in the National Defense Authorization Act for Fiscal Year 2004. DOD stated that it would conduct an evaluation of the HPOs' performance using data already submitted by each of the components. However, we remain concerned that the data reliability problems we identified could limit the usefulness of this assessment and that DOD should work with the components to obtain reliable data. The department also noted that the effectiveness of an HPO is directly influenced by the incentive for organizations to avoid public-private competitions and added that past experiences with HPO-like business process re-engineering efforts suggested that savings would be minimal. Although DOD stated that it will continue to encourage efficiency initiatives and will promulgate lessons learned to help organizations achieve that goal, DOD's response provides little detail on how it will address the data reliability problems that we have identified. Until DOD fully evaluates the effectiveness of its HPO initiative, DOD will not be able to determine whether the organizations designated as HPOs have met their goals of greater cost savings and increased efficiencies, nor will it be able to determine whether the HPO option is a viable option for other organizations to achieve these goals. We are sending copies of this report to other congressional committees and interested parties. We are also sending copies to the Secretaries of Defense, Army, and Air Force, and to the directors of defense agencies with approved HPOs. In addition, this report will be available at no charge on our Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (404) 679-1816 or pendletonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in enclosure III. Signed by: John Pendleton, Director: Defense Capabilities and Management: [End of section] Enclosure I: Summary of DOD's High Performing Organizations: U.S. Air Force: Edwards Air Force Base, 412th Maintenance Group: The Air Force 412th Maintenance Group, located at Edwards Air Force Base, Lancaster, California, was approved as a High Performing Organization (HPO) on February 16, 2006. This HPO is responsible for maintaining numerous Air Force developmental and legacy aircraft of varying design and subsystems and provides logistics test and evaluation for aircraft assigned or attached to the Air Force Flight Test Center.[Footnote 19] The HPO also supports new acquisitions, such as the Joint Strike Fighter (JSF), and system modification programs, such as those employed in the F-16 Fighting Falcon. Defense Contract Management Agency: 1106 Procurement Technician Job Series: The Defense Contract Management Agency's (DCMA) Procurement Technician Job series was approved as an HPO on February 16, 2006. DCMA Procurement Technicians are responsible for various data input tasks involving several Department of Defense (DOD) procurement databases. The Procurement Technician workforce is dispersed among DCMA's six divisions and contract management offices. As of December 2009, DCMA had 47 field offices worldwide, 42 of which are located in the United States. Defense Finance and Accounting Service: Limestone Maine Accounting Function: The Defense Finance and Accounting Service (DFAS) Limestone Accounting operation was approved as an HPO on January 11, 2008. DFAS Limestone provides service to all of the operational Air Force, including Air Combat Command, Air Force Special Operations Command, U.S. Air Forces Europe, and Air Education and Training Command. Services include appropriated funds accounting and reporting, commercial pay, working capital funds accounting and reporting, travel accounting, and accounts receivable. This function is centralized at the Limestone, Maine location. Defense Logistics Agency: Document Automation and Production Service: The Defense Logistics Agency Document Automation and Production Service (DLA-DAPS) was approved for conversion from an OMB Circular A- 76 most efficient organization to an HPO on May 13, 2008. The mission of DLA-DAPS HPO is to actively promote document automation by identifying and proposing technology enhancements and other innovative solutions to facilitate and expedite DOD's desired transition to a paperless environment. This HPO operates from 143 locations across the United States. Defense Logistics Agency: Human Resources Center: The Defense Logistics Agency Human Resources Center (DLA-HRC) was among the first HPOs approved by OSD on February 12, 2006. The DLA-HR function provides Human Resources services to DLA employees and other DOD Components at locations worldwide. DLA's human resources functions were consolidated into a single Human Resources center at Fort Belvoir, Virginia and two customer support offices located in Columbus, Ohio and New Cumberland, Pennsylvania. U.S. Army Corps of Engineers Logistics Management: The U.S. Army Corps of Engineers (USACE) Logistics Management function was approved as an HPO on February 12, 2006. The HPO supports USACE's activities by providing direction, coordination, technical guidance, and services for logistics functions and business processes; integrated logistics support; management controls; and strategic planning. The HPO also supports 53 USACE offices located throughout the United States. U.S. Army Installation Management Command: Fort Huachuca, Adjutant General Directorate: The Fort Huachuca Personnel Function was approved for conversion from an OMB Circular A-76 most efficient organization to an HPO on October 25, 2007. Among other things, the Fort Huachuca Adjutant General (AG) HPO provides military personnel services, such as issuance of identification cards, pre-and post-transition services, and human capital system automation services. U.S. Army Installation Management Command: Fort Hood, Directorate of Aviation Operations: On July 28, 2009, the U.S. Army Installation Management Command's Director of Aviation Operations at Fort Hood, Texas was approved for conversion from an OMB Circular A-76 most efficient organization to an HPO. The organization is responsible for managing Airfield and Flight Simulator Operations at the Fort Hood Army Installation. Washington Headquarters Services: Federal Facilities Division: The Washington Headquarters Services: Federal Facilities Division (WHS- FFD) was approved as an HPO on December 8, 2008. This organization provides facilities maintenance, repair, construction, and logistics support services to Department of Defense activities in the National Capital Region, and operates primarily on the Pentagon Reservation and in delegated facilities, such as U.S. Court of Military Appeals in Washington, D.C. and the Defense Logistics agency in Arlington, Virginia. [End of Enclosure I] Enclosure II: Comments from the Department of Defense: Department of Defense: Office Of The Under Secretary Of Defense: Acquisition, Technology And Logistics: 3010 Defense Pentagon: Washington, DC 20301-3010: May 7, 2010: Mr. John Pendleton: Director: Defense Capabilities and Management: U.S. General Accounting Office: 441 G Street, N.W. Washington, DC 20548: Dear Mr. Pendleton: This is the Department of Defense (DoD) response to the General Accountability Office (GAO) Draft Report GAO-10-566R, "Defense Management: DoD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative," April 7, 2010 (GAO Code 351337). The Department of Defense (DoD) partially concurs with the draft report. Public Law 108-136 H.R. 1588 SEC. 337 defines "Secretary concerned" as "the Secretary of a military department and the Secretary of Defense with respect to matters concerning the Defense Agencies." The Secretary of Defense was designated as the lead to establish the High Performing Organization (HPO) pilot program and completed the task as required. We partially concur with the recommendation that the Under Secretary of Defense for Acquisition, Technology and Logistics should assess the reliability of the data provided by the HPOs and take steps to ensure reporting and collection of reliable data. The DoD Components are accountable for the collection and reporting data on their own high- performing organization (HPO). The Secretary concerned manages its own resources, namely funding and manpower. The Secretary concerned must budget for his or her own organization and ultimately must ensure they are in compliance with their individual budget plans submitted to the Office of the Secretary of Defense (OSD). The Secretary concerned assesses whether the HPOs will receive continued exemption from public/private competition as authorized by the statute. The current moratorium on public-private competition, however, obviates the need for assessment for purposes of exemption. Since OSD has significantly reduced the size of its competitive sourcing office, such assessment is no longer feasible. We partially concur with the recommendation of "using performance data to fully evaluate the effectiveness of the DoD's HPO initiative using all performance measures set out in the National Defense Authorization Act for FY2004." The effectiveness of an HPO is directly influenced by the incentive for organizations to avoid public-private competition. Absent public-private competition, HPOs have little incentive to make difficult management decisions. While evaluation of HPO performance data as submitted by the Components will be performed, experience with earlier reengineering efforts suggest savings will be minimal. Nonetheless, we continue to encourage efficiency initiatives and will promulgate lessons learned to help organizations achieve that goal. I appreciate the opportunity to comment on the draft report. If you have any questions about the technical corrections that have been provided or this response, please contact my point of contact, Ms. Laura Montoya at 703.602.4469. Sincerely, Signed by: Joseph K. Sikes: Director, Housing and Competitive Sourcing: Enclosure: Department of Defense (DoD) Technical Corrections To the General Accountability Office (GAO) Draft Report GAO-10-566R, "Defense Management: DoD Needs to Assess Effectiveness and Determine Future Direction for Its High Performing Organizations Initiative," May 2010 (GAO Code 351337). [End of Enclosure II] Enclosure III: GAO Contact and Staff Acknowledgments: GAO Contact: John Pendleton, (404) 679-1816 or pendletonj@gao.gov: Staff Acknowledgments: In addition to the person named above, Elizabeth McNally, Assistant Director; Owen D. Bruce; Kurt A. Burgeson; Grace A. Coleman; Janida Grima; Pamela (Nicole) Harris; Karen (Nicole) Harms; Richard Powelson; Steven R. Putansu; Jerome T. Sandau; and William T. Woods made key contributions to this report. [End of Enclosure III] Footnotes: [1] National Defense Authorization Act for Fiscal Year 2004, Pub. L. No. 108-136 §337 (2003). Specifically, the law requires the Secretary of Defense to establish a pilot program under which the secretary concerned shall create, or continue the implementation of, high- performing organizations through the conduct of a Business Process Re- engineering initiative at selected military installations and facilities under the jurisdiction of the secretary concerned. The law defines "secretary concerned," for purposes of section 337 as the secretary of a military department and the Secretary of Defense, with respect to matters concerning the defense agencies. [2] DOD's HPO initiative includes six components that were selected as part of the HPO pilot program and three that transitioned from MEO to HPO status. [3] Omnibus Appropriations Act, 2009, Pub. L. No. 111-8, §737 (2009). [4] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, §735 (2009). [5] National Defense Authorization Act for Fiscal Year 2010, Pub. L. No. 111-84, §325 (2009). [6] A formal agreement that an agency implements when a standard or streamlined competition results in agency performance. [7] HPO sites visited included: (1) Edwards Air Force Base (Lancaster, Calif.); (2) U.S. Army Corps of Engineers (Millington, Tenn.; Seattle, Wash.; Portland, Ore.; and San Francisco, Calif.); (3) Defense Finance and Accounting Service (Limestone, Maine); (4) Defense Logistics Agency Human Resources Activity (New Cumberland, Pa.); (5) Defense Logistics Agency Document Automation and Production Service (Philadelphia, Pa.); (6) Defense Contract Management Agency (Baltimore, Md.); (7) Washington Headquarters Services (Arlington, Va.); and (8) Fort Huachuca Personnel Management (Fort Huachuca, Ariz.). [8] We did not conduct a site visit to the U.S. Army, Fort Hood HPO because we were notified of Fort Hood's HPO status after completing our site visits. However, we obtained and analyzed the HPO's letter of obligation with OSD. [9] Office of the Under Secretary of Defense, Acquisition, Technology, and Logistics Memorandum, Procedures for Implementing, Monitoring, and Reporting High Performing Organizations (HPO) (Oct. 4, 2006). [10] Pub. L. No. 108-136 §337 (h) (2003). [11] These measures include (1) costs, savings, and overall financial performance of the organization; (2) organic knowledge, skills, or expertise; (3) efficiency and effectiveness of key functions or processes; (4) efficiency and effectiveness of the overall organization; and (5) general customer satisfaction. [12] OMB defines a most efficient organization (MEO) as the staffing plan of the agency developed to represent the agency's most efficient and cost-effective organization. [13] OMB Circular A-76 (Revised), Performance of Commercial Activities ¶ 5. c. (May 29, 2003). [14] Pub. L. No. 108-136 §337 (2003). [15] Office of the Under Secretary of Defense, Acquisition, Technology, and Logistics Memorandum, Procedures for Implementing, Monitoring, and Reporting High Performing Organizations (HPO) (Oct. 4, 2006). [16] The memorandum requires the quality assurance surveillance plans developed during business process re-engineering to include an annual report on, at a minimum, organic knowledge, skills, or experience; efficiency and effectiveness of key functions or processes; efficiency and effectiveness of the overall organization; and general customer satisfaction. [17] GAO, Highlights of a GAO Forum on High-Performing Organizations: Metrics, Means, and Mechanisms for Achieving High Performance in the 21st Century Public Management Environment, [hyperlink, http://www.gao.gov/products/GAO-04-343SP] (Washington, D.C.: Feb. 13, 2004). [18] GAO, Assessing the Reliability of Computer-Processed Data, [hyperlink, http://www.gao.gov/products/GAO-09-365G] (Washington D.C., Feb. 2, 2009). [19] Developmental aircraft that are maintained at Edwards Air Force Base include the F/A-22 Raptor, F-117 Nighthawk, F-35 Joint Strike Fighter, and the CV-22 Osprey, among other aircraft. Legacy aircraft at Edwards Air Force Base include the F-16 Fighting Falcon; T-38 Talon; KC-135 Stratotanker; as well as the B-1, B-2, and B-52 Bombers. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.