Human Capital

Monitoring of Safeguards and Addressing Employee Perceptions Are Key to Implementing a Civilian Performance Management System in DOD Gao ID: GAO-10-102 October 28, 2009

In 2004, the Department of Defense (DOD) began implementing the National Security Personnel System (NSPS)--a human capital system for DOD civilians. NSPS significantly redesigned the way DOD civilians are hired, compensated, and promoted. Pub. L. No. 110-181 mandated that GAO conduct reviews of the NSPS performance management system in calendar years 2008, 2009, and 2010. In this report, GAO assessed (1) the extent to which DOD has implemented certain internal safeguards to ensure the fairness, effectiveness, and credibility of NSPS, and monitored their implementation, and (2) how DOD civilian personnel perceive NSPS, and the actions DOD has taken to address those perceptions. GAO analyzed relevant documents and employee survey results, interviewed DOD officials, and conducted discussion groups with DOD employees at eight locations outside of the continental United States. Toward the end of GAO's review, both Houses of Congress passed proposed legislation that, if enacted, would terminate NSPS and require any future performance management system for DOD civilians to include certain internal safeguards

DOD continues to take steps to implement internal safeguards as part of NSPS, but implementation of some safeguards could still be improved, and continued monitoring of all safeguards' implementation is needed. In general, DOD has taken some steps to meet the intent of each of the safeguards, and it has implemented some of the recommendations from GAO's September 2008 report. However, opportunities exist for DOD to improve implementation of some safeguards. For example, DOD has not yet evaluated the effectiveness of the training employees receive, although doing so could help DOD measure the impact of its training and its progress toward achieving agency goals. In addition, DOD has not specified in its guidance what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. Consequently, the components may not follow a consistent approach when investigating potential barriers, which could hinder their efforts to eliminate them. Further, GAO previously noted that continued monitoring of the safeguards was needed to ensure that DOD's actions were effective. While DOD monitors some aspects of the system's implementation, it does not monitor how or the extent to which the safeguards specifically are implemented across the department. As a result, decision makers lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. DOD civilian personnel have mixed perceptions about NSPS, and while the department has taken some steps toward addressing employee concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system, as GAO recommended in 2008. DOD's survey data from 2008 revealed that overall, NSPS employees responded positively about some aspects of performance management, such as connecting pay to performance, and negatively about others, such as the performance appraisal process. According to the most recent survey data, the negative perceptions of employees who worked under NSPS the longest remain largely unchanged from what was reported by GAO in 2008. Further, as GAO reported in 2008, employees and supervisors continue to express negative perceptions in discussion groups about NSPS--for example, voicing concerns about the negative impact of NSPS on employees' motivation and morale, and about the excessive amount of time spent navigating the performance management process. Such negative perceptions are not surprising given that large-scale organizational transformations often require an adjustment period to gain employees' trust and acceptance. DOD has taken some steps to address employees' perceptions of NSPS--for example, by issuing a memorandum with suggested actions the components could take to address employee concerns. However, DOD has not yet developed and implemented an action plan that fully meets the intent of GAO's 2008 recommendation. Specifically, DOD has not yet specified such things as its intended actions, who will be responsible, and the time frames for these actions. GAO continues to believe that implementing such a plan has merit.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-10-102, Human Capital: Monitoring of Safeguards and Addressing Employee Perceptions Are Key to Implementing a Civilian Performance Management System in DOD This is the accessible text file for GAO report number GAO-10-102 entitled 'Human Capital: Monitoring Safeguards and Addressing Employee Perceptions Are Key to Implementing a Civilian Performance Management System in DOD' which was released on October 28, 2009. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: October 2009: Human Capital: Monitoring of Safeguards and Addressing Employee Perceptions Are Key to Implementing a Civilian Performance Management System in DOD: Human Capital: GAO-10-102: GAO Highlights: Highlights of [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-10- 102], a report to congressional committees. Why GAO Did This Study: In 2004, the Department of Defense (DOD) began implementing the National Security Personnel System (NSPS)”a human capital system for DOD civilians. NSPS significantly redesigned the way DOD civilians are hired, compensated, and promoted. Pub. L. No. 110-181 mandated that GAO conduct reviews of the NSPS performance management system in calendar years 2008, 2009, and 2010. In this report, GAO assessed (1) the extent to which DOD has implemented certain internal safeguards to ensure the fairness, effectiveness, and credibility of NSPS, and monitored their implementation, and (2) how DOD civilian personnel perceive NSPS, and the actions DOD has taken to address those perceptions. GAO analyzed relevant documents and employee survey results, interviewed DOD officials, and conducted discussion groups with DOD employees at eight locations outside of the continental United States. Toward the end of GAO‘s review, both Houses of Congress passed proposed legislation that, if enacted, would terminate NSPS and require any future performance management system for DOD civilians to include certain internal safeguards. What GAO Found: DOD continues to take steps to implement internal safeguards as part of NSPS, but implementation of some safeguards could still be improved, and continued monitoring of all safeguards‘ implementation is needed. In general, DOD has taken some steps to meet the intent of each of the safeguards, and it has implemented some of the recommendations from GAO‘ s September 2008 report. However, opportunities exist for DOD to improve implementation of some safeguards. For example, DOD has not yet evaluated the effectiveness of the training employees receive, although doing so could help DOD measure the impact of its training and its progress toward achieving agency goals. In addition, DOD has not specified in its guidance what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. Consequently, the components may not follow a consistent approach when investigating potential barriers, which could hinder their efforts to eliminate them. Further, GAO previously noted that continued monitoring of the safeguards was needed to ensure that DOD‘s actions were effective. While DOD monitors some aspects of the system‘s implementation, it does not monitor how or the extent to which the safeguards specifically are implemented across the department. As a result, decision makers lack information that could be used to determine whether the department‘s actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. DOD civilian personnel have mixed perceptions about NSPS, and while the department has taken some steps toward addressing employee concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system, as GAO recommended in 2008. DOD‘s survey data from 2008 revealed that overall, NSPS employees responded positively about some aspects of performance management, such as connecting pay to performance, and negatively about others, such as the performance appraisal process. According to the most recent survey data, the negative perceptions of employees who worked under NSPS the longest remain largely unchanged from what was reported by GAO in 2008. Further, as GAO reported in 2008, employees and supervisors continue to express negative perceptions in discussion groups about NSPS”for example, voicing concerns about the negative impact of NSPS on employees‘ motivation and morale, and about the excessive amount of time spent navigating the performance management process. Such negative perceptions are not surprising given that large- scale organizational transformations often require an adjustment period to gain employees‘ trust and acceptance. DOD has taken some steps to address employees‘ perceptions of NSPS”for example, by issuing a memorandum with suggested actions the components could take to address employee concerns. However, DOD has not yet developed and implemented an action plan that fully meets the intent of GAO‘s 2008 recommendation. Specifically, DOD has not yet specified such things as its intended actions, who will be responsible, and the time frames for these actions. GAO continues to believe that implementing such a plan has merit. What GAO Recommends: GAO is recommending that DOD evaluate its training, revise its guidance for analyzing ratings, and monitor the implementation of safeguards for NSPS or any future performance management system. DOD generally concurred with GAO‘s recommendations. View [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-10-102] or key components. For more information, contact Brenda S. Farrell at (202) 512-3604 or farrellb@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: DOD Continues to Make Progress, but Implementation of Some Safeguards Could Be Improved, and Continued Monitoring of the System's Implementation, Including the Safeguards, Is Needed: Although DOD Civilian Employees under NSPS Have Mixed Views about the System, DOD Has Not Yet Developed and Implemented a Plan to Address Employees' Negative Perceptions of Some Aspects of the System: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: Reconsideration Process: Appendix III: Comments from the Department of Defense: Appendix IV: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: Number of DOD Civilian Employees Phased into NSPS, as of September 2009: Table 2: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about Performance Management in the 2008 Status of Forces Survey of Civilian Employees: Table 3: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about Performance Appraisals in the 2008 Status of Forces Survey of Civilian Employees: Table 4: Estimated Percentage of Responses from Spiral 1.1 Employees to Selected Questions about Performance Management from the May 2006, November 2006, May 2007, and February 2008 Administrations of the Status of Forces Survey of Civilian Employees: Table 5: Estimated Percentage of Responses from Spiral 1.1 Employees about the Overall Impact of the National Security Personnel System from the May 2006, November 2006, May 2007, and February 2008 Administrations the Status of Forces Survey of Civilian Employees: Table 6: Composition of Discussion Groups: Table 7: Composition of Discussion Groups by Demographic Category per Component: Figures: Figure 1: Example of NSPS Pay Pool Organization: Figure 2: Sample Screen Shots from the NSPS 101 and iSuccess Training Modules: Abbreviations: DMDC: Defense Manpower Data Center: DOD: Department of Defense: GS: General Schedule: NSPS: National Security Personnel System: OPM: Office of Personnel Management: PEO: Program Executive Office: [End of section] United States Government Accountability Office: Washington, DC 20548: October 28, 2009: Congressional Committees: In 2004, the Department of Defense (DOD) began implementing the National Security Personnel System (NSPS)--a human capital system for DOD civilian employees. NSPS significantly redesigned the rules, regulations, and processes that govern the way in which civilian employees are hired, compensated, and promoted within the department. DOD began converting employees to NSPS in 2006 and, according to the department, by early 2009 had converted more than 200,000 employees to the system. However, a series of events transpired in 2009 leaving the system's future uncertain. Specifically, in February 2009, the Chairman of the House Committee on Armed Services and the Chairman of the committee's Subcommittee on Readiness urged DOD to halt conversions to NSPS, highlighting concerns over a lack of transparency and widespread discontent with the system. Subsequently, in March 2009, DOD and the Office of Personnel Management (OPM) announced that they would suspend any further conversions of organizations to NSPS pending the outcome of a review of the system led by the Defense Business Board.[Footnote 1] In July 2009, the Defense Business Board presented the Secretary of Defense with the report of its review of NSPS, which included recommendations to initiate a reconstruction of the system and to continue the moratorium on conversions to NSPS until reconstruction is complete. Most recently, in October 2009, the proposed National Defense Authorization Act for Fiscal Year 2010, which passed both Houses of Congress,[Footnote 2] contains provisions that would terminate NSPS and convert DOD civilian employees currently under the system to previously existing civilian personnel systems no later than January 1, 2012. Before the enactment of the legislation that authorized NSPS, we raised a number of critical issues about the proposed regulations for NSPS in a series of testimonies in 2003.[Footnote 3] Since then, we have provided congressional decision makers with insight into DOD's processes to design and implement NSPS and the most significant challenges the department faces in implementing the system.[Footnote 4] While GAO supports human capital reform in the federal government, a key challenge is ensuring sustained and committed leadership for such a major transformation effort. Further, moving too quickly or prematurely to implement such systems, whether at DOD or elsewhere, can significantly raise the risk of problems implementing them.[Footnote 5] We have also reported that DOD and other federal agencies must ensure that their performance management systems contain appropriate accountability mechanisms and internal safeguards to ensure that the systems are fair, effective, and credible.[Footnote 6] To help inform congressional deliberations on the implementation of NSPS, the National Defense Authorization Act for Fiscal Year 2008 mandated that GAO conduct reviews of NSPS in calendar years 2008, 2009, and 2010 to determine the extent to which DOD has effectively incorporated certain accountability mechanisms and internal safeguards into the system and to assess employee attitudes toward the system.[Footnote 7] In September 2008, we issued our first review of NSPS, and in April 2009 we testified on that assessment before the House Committee on Armed Services' Subcommittee on Readiness.[Footnote 8] We used the following safeguards to assess NSPS:[Footnote 9] * Involve employees, their representatives, and other stakeholders in the design of the system, to include employees directly involved in validating any related implementation of the system. * Assure that the agency's performance management system links employee objectives to the agency's strategic plan, related goals, and desired outcomes. * Implement a pay-for-performance evaluation system to better link individual pay to performance, and provide an equitable method for appraising and compensating employees. * Provide adequate training and retraining for supervisors, managers, and employees in the implementation and operation of the performance management system. * Institute a process for ensuring ongoing performance feedback and dialogue between supervisors, managers, and employees throughout the appraisal period, and setting timetables for review. * Assure that certain predecisional internal safeguards exist to help achieve consistency, equity, nondiscrimination, and nonpoliticization of the performance management process (e.g., independent reasonableness reviews by a third party or reviews of performance rating decisions, pay determinations, and promotions before they are finalized to ensure that they are merit-based, as well as pay panels who consider the results of the performance appraisal process and other information in connection with final pay decisions). * Assure that there are reasonable transparency and appropriate accountability mechanisms in connection with the results of the performance management process, including periodic reports on internal assessments and employee survey results relating to performance management and individual pay decisions while protecting individual confidentiality. * Assure that the agency's performance management system results in meaningful distinctions in individual employee performance. * Provide a means for ensuring that adequate agency resources are allocated for the design, implementation, and administration of the performance management system. We initially developed a list of safeguards based on our extensive body of work looking at the performance management practices used by leading public sector organizations both in the United States and in other countries. These safeguards were then merged with the list of safeguards specified in section 9902(b)(7) of Title 5, U.S. Code, to develop the aforementioned safeguards. It is important to keep in mind that the list of safeguards we are using to assess DOD's efforts is not intended to cover all attributes of a modern, results-oriented performance management system; therefore, other safeguards may exist. Rather, as we previously reported, the items on the list cover possible safeguards that help ensure those systems are fair, effective, and credible.[Footnote 10] In responding to the congressional mandate for the 2008 review of NSPS, we reported, in September 2008, that while DOD has taken some steps to ensure that NSPS is fair, effective, and credible, implementation of certain safeguards could be improved. We further reported that although DOD civilian employees under NSPS responded positively regarding some aspects of performance management, the department does not have an action plan to address employees' negative perceptions of NSPS. To address our findings, we recommended that DOD improve the implementation of certain safeguards and develop and implement an action plan to address employees' concerns about NSPS. DOD generally concurred with our recommendations, with the exception of one-- requiring a predecisional analysis of its rating results. We discuss the steps DOD has taken to address our previous recommendations later in this report. To address the congressional mandate for the 2009 review of NSPS, we established the following objectives: (1) To what extent has DOD implemented safeguards to ensure the fairness, effectiveness, and credibility of the NSPS performance management system and monitored the implementation of the safeguards? (2) How do DOD civilian personnel perceive NSPS, and what actions has DOD taken to address these perceptions? For our first objective, we reviewed the safeguards specified in section 9902(b)(7) of Title 5, U.S. Code, for the NSPS performance management system, as well as other safeguards we have previously identified as key for performance management systems. We also obtained and analyzed regulations, guidance, and other documentation provided by officials in DOD and the NSPS program offices of the four components-- the Army, the Air Force, the Navy, and the Fourth Estate[Footnote 11]- -and interviewed knowledgeable officials in all of these offices to obtain a comprehensive understanding of their efforts to implement the NSPS performance management system and the safeguards as well as the processes, procedures, and controls used for monitoring and overseeing implementation of the system. In addition, we visited eight DOD organizations--two for each component--located outside the continental United States to assess the extent to which the department has implemented safeguards for NSPS. Because our 2008 review focused on assessing implementation of NSPS and the safeguards at locations that were geographically distributed throughout the United States, we focused our efforts for this review on visiting locations outside the continental United States.[Footnote 12] During our site visits we reviewed documents, such as pay pool business rules,[Footnote 13] and met with the performance review authority, pay pool managers, pay pool panel members, rating officials, the NSPS program manager or transition team, and DOD civilian employees under NSPS to obtain their perspectives on the extent to which the safeguards had been implemented. The organizations we visited were selected based on a number of factors, such as the presence of a large number or concentrated group of civilian employees under NSPS. For our second objective, we obtained and analyzed the results of DOD's May 2006, November 2006, May 2007, and February 2008 Status of Forces Survey of Civilian Employees, which gauge employee attitudes toward NSPS, among other things. We used these data to identify indications of movement or trends in employee perceptions. As we reported in our first assessment of NSPS, we reviewed the results of prior administrations of DOD's Status of Forces Survey of Civilian Employees and found the results to be sufficiently reliable for the purposes of our report. For our current assessment, we found no substantial changes in the methodology for the 2008 Status of Forces Survey of Civilian Employees. In addition, we obtained the perspectives of DOD civilian personnel at the eight locations we visited by conducting small group discussions with nonsupervisory and supervisory employees under NSPS and administering a short questionnaire. Because the locations we visited are not representative of all DOD locations, the results from our discussions and the comments provided are not generalizable to the entire NSPS civilian population. However, the composition of our discussion groups was designed to ensure that we spoke with employees from each of the four DOD components at locations outside the continental United States. Further details about our scope and methodology can be found in appendix I. We conducted this performance audit from November 2008 through September 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Results in Brief: DOD continues to take steps to implement safeguards as part of the NSPS performance management system, but implementation of some safeguards could be improved, and continued monitoring of the safeguards' implementation is needed. In general, DOD has taken some steps to meet the intent of each of the safeguards, and it has implemented some of the recommendations from our 2008 report, including requiring commands to publish the final overall rating results and providing guidance to pay pools encouraging managers to rate employees appropriately. Nonetheless, DOD's implementation of some of the safeguards could be improved. First, for example, DOD has not evaluated the effectiveness of the training that it provides to employees on the system. Under NSPS, the components are responsible for training their employees, while DOD's NSPS Program Executive Office (PEO) supports the components' efforts by offering a variety of departmentwide training courses and other materials. However, officials[Footnote 14] at each of the eight organizations we visited expressed concerns over the effectiveness of the training provided, noting that additional training was needed, that training was not always helpful, and that some training was not timely or was outdated. We previously reported that it is increasingly important for agencies to measure the real impact of training and thus evaluate the training efforts to ascertain progress toward achieving agency goals[Footnote 15]. Second, while, in 2008, DOD did not agree with our recommendation to require a predecisional analysis of ratings--stating that postdecisional analysis of rating results is useful to identify barriers and corrective actions--we found during the course of this review that the department's postdecisional analyses following the 2007 and 2008 NSPS performance management cycles lacked consistency and did not generally include demographic information. However, in May 2009, the PEO issued guidance to promote a degree of standardization in the components' postdecisional analyses. While not predecisional, we recognize that DOD's approach represents a noteworthy step and does provide some benefits, some of which are similar to those of a predecisional analysis. For example, like predecisional analysis, postdecisional analysis is a mechanism to ensure that employees receive fair and equitable treatment in all aspects of personnel management[Footnote 16]. However, the guidance does not specify what process the components should follow to investigate potential barriers to fair and equitable ratings and their causes, nor a process for eliminating barriers that are found. Because DOD's guidance does not specify these steps, the components may not follow a consistent approach when investigating potential barriers, which could hinder their efforts to eliminate them. In our 2008 report, we also noted that continued monitoring of the safeguards was needed to ensure that DOD's actions were effective as implementation of NSPS proceeded. We found that DOD monitors some aspects of the implementation of NSPS, such as whether the system is on track to achieve certain goals it established, but does not monitor how or the extent to which the safeguards specifically are implemented across the department. Because DOD does not monitor the implementation of the safeguards, decision makers in DOD and the Congress lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. Accordingly, until DOD effectively implements the safeguards and monitors their implementation across the department, employees will not have assurance that NSPS is a fair, equitable, and credible system and decision makers in DOD and the Congress will not have the information that could be used to determine if the department's actions are effective. DOD civilian personnel have mixed perceptions about NSPS, and while the department has taken some steps toward addressing their concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system, as we recommended in 2008. DOD's survey data from 2008 revealed that overall, NSPS employees responded positively about some aspects of performance management, such as connecting pay to performance, and negatively about other aspects, including the performance appraisal process. For example, DOD's survey data for 2008 indicate that an estimated 42 percent of NSPS employees agree that pay raises depend on how well employees perform their jobs, as compared with an estimated 25 percent of non-NSPS employees. However, when asked about their performance appraisal system, an estimated 29 percent of NSPS employees, as compared with an estimated 34 percent of non-NSPS employees, agreed with the statement that their performance appraisal system improves organizational performance. Additionally, DOD's most recent survey data indicated that employees who worked under NSPS the longest--spiral 1.1 employees[Footnote 17]--expressed negative perceptions of the system consistent with those we reported on in 2008. For example, between the November 2006 and February 2008 administrations of DOD's survey, the percentage of spiral 1.1 employees that agreed that they understood what they had to do to be rated at a different performance level declined from an estimated 59 percent in November 2006 to an estimated 53 percent in May 2007, then remained consistent with an estimated 54 percent responding that they agree in February 2008. Further, the results of our discussion groups indicated that employees and supervisors continue to have consistent concerns and negative perceptions of NSPS. These included the following: (1) NSPS's negative impact on employee motivation and morale; (2) the excessive amount of time spent navigating the performance management process; (3) challenges with job objectives; (4) factors undermining employee confidence in the system, such as the subjectivity of the pay pool panel process; and (5) factors unrelated to job performance affecting employees' final performance ratings, such as supervisors' and employees' writing skills. As we have previously reported, these negative perceptions are not surprising given that large-scale organizational transformations--for example, the adoption of a new performance management system--often entail fundamental and radical changes that require an adjustment period to gain employees' trust and acceptance. With regard to DOD's efforts to address employee perceptions of the system, DOD issued a memorandum, in June 2009, highlighting actions the department had taken in this area, as well as suggested future actions that the components could take to address identified negative perceptions. While we believe that issuing this memorandum is an important first step in addressing employee concerns, it does not fully meet the intent of our 2008 recommendation that DOD develop and implement a specific action plan to address employee perceptions of NSPS because it does not specify such things as the actions DOD intends to take, who will be responsible for taking the action, and timelines for doing so--items stipulated in OPM guidance on action planning.[Footnote 18] We continue to believe that implementing such an action plan is important, and note that doing so would be a step that DOD could take to involve employees in the system's implementation--which is one of the safeguards we discuss. In September 2009, we provided DOD with a draft of this report that included three recommendations to better address the safeguards and improve implementation of the NSPS performance management system. Subsequently, as we were preparing to issue this report, the proposed National Defense Authorization Act for Fiscal Year 2010 passed both Houses of Congress.[Footnote 19] That legislation contained provisions that, if enacted, would terminate NSPS while also requiring that DOD include certain safeguards as part of any future performance management system. Citing the uncertainty regarding the system's future, DOD partially concurred with our three recommendations specific to the NSPS performance management system. DOD also stated that it would consider acting on our recommendations to the extent they are relevant as the department moves forward with any future performance management system. We believe that this is a reasonable approach. Accordingly, we have revised our recommendations to apply to any future performance management system for DOD's civilian employees--whether NSPS or another system. Specifically, we are recommending that DOD (1) review and evaluate the effectiveness of the department's performance management training; (2) ensure that guidance is in place for conducting a postdecisional analysis that specifies what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings; and (3) include, as part of the department's monitoring of the implementation of its system, efforts to monitor and evaluate how the safeguards specifically are being implemented by lower- level organizations across the department. DOD's comments are reprinted in appendix III. Background: The General Schedule (GS) is the federal government's primary pay and classification system for white-collar employees. Under this system, federal employees are paid according to governmentwide rules and procedures, and federal positions are classified according to the difficulty and responsibility of the work performed. The GS system was created in 1949, when most federal positions involved clerical work or revolved around the execution of established, stable processes--for example, posting census figures in ledgers or retrieving taxpayer records from vast file rooms. Over time, however, federal positions have become increasingly specialized and more highly skilled. In light of this change, a number of federal agencies have attempted to provide managers with greater flexibility in hiring and awarding pay raises to employees by implementing human capital initiatives, such as performance management systems, that reward employees' performance and contribution to the agency's mission. The need for human capital reform regarding these systems has been the subject of a number of previous GAO reviews.[Footnote 20] These reviews have noted, for example, that federal agencies must have modern, effective, credible, and validated performance management systems that are capable of supporting pay and other personnel decisions, and that pay for performance works only with adequate safeguards and appropriate accountability mechanisms in place to ensure that the safeguards are implemented in a fair, effective, and credible manner. In November 2003, the Congress included a provision in the National Defense Authorization Act for Fiscal Year 2004 providing DOD with the authority to establish a pay-for-performance management system as part of NSPS.[Footnote 21] In April 2004, the Secretary of Defense appointed an NSPS Senior Executive to, among other things, design, develop, and implement NSPS. Under the Senior Executive's authority, the PEO was established as the central policy and program office for NSPS. The PEO's responsibilities include designing the human resource/pay-for- performance systems, developing communication and training strategies, modifying personnel information technology, and preparing joint enabling regulations and internal DOD implementing regulations, called implementing issuances. As the central DOD-wide program office, the PEO also directs and oversees the four components' NSPS program managers, who report to their parent components and the PEO. These program managers serve as their components' NSPS action officers and also participate in the development, planning, implementation, and deployment of NSPS. Beginning in April 2006, DOD began phasing (or spiraling) civilian employees into NSPS;[Footnote 22] however, in January 2008 the National Defense Authorization Act for Fiscal Year 2008 prohibited the Secretary of Defense from converting more than 100,000 employees to NSPS in any calendar year and excluded Federal Wage System employees from coverage under NSPS.[Footnote 23] Further, in March 2009, DOD announced that it would delay the conversion of additional organizations to NSPS pending the outcome of a joint review of the system by DOD and OPM. According to DOD, this decision affected roughly 2,000 employees in organizations scheduled to convert to NSPS during the spring of 2009. As a result of these and other legislative changes that resulted in revisions to the NSPS regulations, the PEO has not developed new timelines for phasing any additional civilian employees into NSPS. As table 1 shows, according to DOD, almost 220,000 civilian employees have been phased into NSPS as of September 2009. Table 1: Number of DOD Civilian Employees Phased into NSPS, as of September 2009: Spiral: 1.1; Army: 2,666; Air Force: 3,187; Navy: 4,828; Fourth Estate: 1,611; Total number of employees: 12,292. Spiral: 1.2; Army: 18,084; Air Force: 38,693; Navy: 10,087; Fourth Estate: 9,239; Total number of employees: 76,103. Spiral: 1.3; Army: 28,582; Air Force: 1,102; Navy: 8,559; Fourth Estate: 391; Total number of employees: 38,634. Spiral: 2.1; Army: 11,103; Air Force: -; Navy: 8,194; Fourth Estate: 85; Total number of employees: 19,382. Spiral: 2.2; Army: 20,175; Air Force: -; Navy: 22,487; Fourth Estate: 15,145; Total number of employees: 57,807. Spiral: 2.3; Army: 483; Air Force: -; Navy: 14,128; Fourth Estate: 38; Total number of employees: 14,649. Spiral: 2.4; Army: -; Air Force: -; Navy: 294; Fourth Estate: 15; Total number of employees: 309. Spiral: Total number of employees; Army: 81,093; Air Force: 42,982; Navy: 68,577; Fourth Estate: 26,524; Total number of employees: 219,176. Source: DOD. Note: In June 2009, we reported that according to DOD, about 212,000 civilian employees were under NSPS. See GAO, Human Capital: Continued Monitoring of Internal Safeguards and an Action Plan to Address Employee Concerns Could Improve Implementation of the National Security Personnel System, GAO-09-840 (Washington, D.C.: June 25, 2009). According to a PEO official, no additional organizations have converted to NSPS since March 2009 and the growth of the NSPS workforce is due to routine hiring, transfer, reassignment, and promotion actions into organizations already under NSPS. [End of table] The NSPS performance management process is ongoing and consists of several phases that are repeated during each annual performance cycle. The process begins with a planning phase that involves supervisors (or rating officials) and employees working together to establish performance plans. This includes developing job objectives--the critical work employees perform that is aligned with their organizational goals and focused on results--and identifying contributing factors--the attributes and behaviors that identify how the critical work established in the job objectives is going to be accomplished (e.g., cooperation and teamwork). After the planning phase comes the monitoring and developing phase, during which ongoing communication between supervisors and employees occurs to ensure that work is accomplished; attention is given to areas that need to be addressed; and managers, supervisors, and employees have a continued and shared understanding of expectations and results. In the rating phase, the supervisor prepares a written assessment that captures the employee's accomplishments during the appraisal period. In the final-- or reward--phase, employees should be appropriately rewarded or compensated for their performance with performance payouts. During this phase, employee assessments are reviewed by multiple parties to determine employees' ratings and, ultimately, performance payouts. The performance management process under NSPS is organized by pay pools. A pay pool is a group of employees who share in the distribution of a common pay-for-performance fund.[Footnote 24] The key participants that make up pay pools are the employee, supervisor, higher-level reviewer, pay pool panel, pay pool manager, performance review authority, and, in some instances, the sub-pay pool,[Footnote 25] as shown in figure 1. Figure 1: Example of NSPS Pay Pool Organization: [Refer to PDF for image: flowchart of NSPS pay pool organization] Source: DOD. [End of figure] Within a pay pool, each participant has defined responsibilities under the performance management process: * Employees are encouraged to be involved throughout the performance management cycle. This includes initially working with their supervisors to develop job objectives and identify associated contributing factors; identifying and recording accomplishments and results throughout the appraisal period; and participating in interim reviews and end-of-year assessments, for example, by preparing a self- assessment of their performance during the annual appraisal period. * Supervisors (or rating officials) are responsible for effectively managing the performance of their employees. This includes clearly communicating performance expectations; aligning performance expectations and employee development with organization mission and goals; working with employees to develop written job objectives that reflect expected accomplishments and contributions for the appraisal period and identifying applicable contributing factors; providing employees meaningful, constructive, and candid feedback relative to performance expectations, including at least one documented interim review; making meaningful distinctions among employees based on performance and contribution; and providing recommended ratings of record, share assignments, and payout distributions to the pay pool. * The higher-level reviewer, typically the rating official's supervisor, is responsible for reviewing and approving job objectives and recommended employee assessments. The higher-level reviewer is the first step in ensuring consistency of ratings because this individual looks across multiple ratings. * The pay pool panel (or, in some cases, the sub-pay pool panel) is a board of management officials who are usually in positions of line authority or in senior staff positions with resource oversight for the organizations, groups, or categories of employees making up the pay pool membership.[Footnote 26] The primary function of the pay pool panel is the reconciliation of ratings of record, share distribution, and payout allocation decisions. For example, the pay pool panel may adjust a supervisor's recommended rating of record in order to help ensure the equity and consistency of ratings across the pay pool. * Each pay pool has a manager who is responsible for providing oversight of the pay pool panel. The pay pool manager is the final approving official of the rating of record. Performance payout determinations may be subject to higher management review by the performance review authority[Footnote 27] or equivalent review process. * Finally, the performance review authority provides oversight of several pay pools and addresses the consistency of performance management policies within a component, major command, field activity, or other organization as determined by the component. DOD Continues to Make Progress, but Implementation of Some Safeguards Could Be Improved, and Continued Monitoring of the System's Implementation, Including the Safeguards, Is Needed: DOD continues to take some steps to implement each of the safeguards we reported on in September 2008. However, opportunities exist to continually involve employees in the system's implementation, and implementation of some safeguards--for example, providing effective training--could be improved. Also, we previously reported that continued monitoring of the safeguards was needed to help ensure that DOD's efforts were effective as implementation of NSPS proceeded. However, we found that while DOD monitors some aspects of the implementation of NSPS, it does not monitor how the safeguards specifically are implemented across the department. Because DOD does not monitor the safeguards' implementation, decision makers in DOD lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. Implementation of Some Safeguards Could Be Improved: Involve Employees in the Design and Implementation of the System: DOD has taken a number of steps to involve employees and stakeholders in the design and implementation of NSPS. In our September 2008 report, we noted that DOD solicited comments from employees and unions representing DOD employees during the design of NSPS. We also noted that DOD involved employees during the implementation of the system in workshops and other efforts aimed at refining the system's performance factors. DOD continues to take such steps. For example: * DOD solicited comments from employees and unions on the system's final rule,[Footnote 28] which was published in the Federal Register in September 2008. According to the Federal Register, the final regulations, which became effective in November 2008, include revisions based on 526 comments submitted during the public comment period and on comments from 9 of the department's 10 unions with national consultation rights. * DOD involved employees in efforts to improve the usability of the automated tools that support the NSPS performance and pay pool management processes. Specifically, the PEO and the department's Civilian Personnel Management Service held a series of meetings with employees, rating officials, pay pool managers, and human resource practitioners in early 2008 to address concerns regarding the usability of the automated tools. These meetings allowed the department to gather requirements for the next version of the NSPS automated tools based on lessons learned and user input. Subsequently, DOD established six separate working groups to develop and evaluate the requirements for each of its automated tools. In addition, DOD initiated separate efforts to enhance the usability of the Performance Appraisal Application--the DOD-wide tool for employee self-assessments and appraisals. Specifically, the contractor that developed the Performance Appraisal Application enlisted the assistance of software usability experts to evaluate the tool and recommend changes that would enhance users' experience with it. As a part of this effort, the contractor observed and worked with employees and rating officials to identify changes that could be made to the Performance Appraisal Application to make it more user-friendly. DOD also tested the functionality and usability of the enhancements that were made to the Performance Appraisal Application with over 300 users. * DOD has taken steps to involve the components in the implementation of NSPS through biweekly conference calls held at key phases of the performance management process. According to the PEO, during these calls, PEO and Civilian Personnel Management Service representatives discuss topics submitted by the components, respond to questions regarding such things as NSPS policy and the system's automated tools, and share lessons learned with participants. Further, according to the PEO, these conference calls allow participants to address systemic problems through feedback shared between different levels of the organization. At the locations we visited outside the continental United States, we found that some steps were generally being taken to involve employees in the implementation of NSPS as well. For example, officials at each of the eight locations reported that organizations identified lessons learned that were generally based upon employee feedback gathered following the mock pay pool.[Footnote 29] According to these officials, lessons learned were used to make changes to, among other things, training materials, business rules, and the use of job objectives and contributing factors. For example, two locations limited the number of contributing factors employees should use in their performance plans based upon lessons learned, while one location responded to employee feedback regarding a lack of time to devote to NSPS by mandating that employees be allowed to take time to complete NSPS training. While DOD has taken a number of steps to involve employees in the design and implementation of NSPS thus far, as stated above, we note that one way the department could continue to involve employees as implementation of the system proceeds is to develop and implement an action plan to address employees' perceptions of NSPS, as we recommended in our first assessment of NSPS. However, DOD has not yet done so, which we discuss further in the second objective of this report. Link Employee Objectives to the Agency's Strategic Goals and Mission: DOD continues to take steps to link employee job objectives to the agency's strategic goals, mission, and desired outcomes. As we noted in our 2008 report, DOD's automated tool for employee self-assessments and appraisals--the Performance Appraisal Application--provides a designated area for the mission of the employee's command to be inserted as a guide while employees compose their job objectives and self-assessments. In May 2009, DOD published its evaluation of NSPS for 2008, entitled National Security Personnel System (NSPS) - 2008 Evaluation Report, which included an evaluation of employee performance plans to determine the extent to which employee job objectives are aligned with higher-level organizational goals and thus ensure that employee performance contributes to the achievement of organizational success. The evaluation included 240 employee performance plans encompassing a range of job series, pay schedules, pay bands, and organizations within each of the four components. The evaluation found that 95 percent of the objectives were strongly aligned to higher-level goals and demonstrated a clear, direct, and strong linkage to the organizational mission or relevant strategic goals. During our site visits, we found that the organizations were taking steps to ensure that employees understood how their job objectives link to the organization's strategic goals. This was generally accomplished through documentation requirements in the Performance Appraisal Application and reinforced during employees' discussions with their supervisors. Some organizations have taken additional steps to ensure that employee job objectives link to the organization's strategic goals. For example, at one location we visited the commanding general issued a memorandum to managers emphasizing the importance of ensuring that employee objectives are linked to the command's mission and objectives and that employees understand their roles in achieving those objectives. However, officials at five locations also reported that employees do not always understand this linkage. Training and Retraining in the System's Implementation and Operation: DOD continues to take steps to provide employees with required and other training on the implementation and operation of the NSPS performance management system, but has not yet evaluated the effectiveness of the training that it provides. In our September 2008 report, we noted that DOD encouraged employees who were transitioning to NSPS to receive training that covered skills and behaviors necessary to implement and sustain NSPS; foster support and confidence in the system; and facilitate the transition to a performance-based, results- oriented culture. DOD and each of the components continue to take such steps to provide employees with required and other training on the system, including introductory training for employees converting to NSPS and sustainment training for employees already under the system. While the components are responsible for providing employees with training on the NSPS performance management system, the PEO supports their efforts by offering a variety of departmentwide training courses and other materials. For example, Web-based training modules that the PEO has developed, such as its NSPS 101 and iSuccess courses (see fig. 2 for sample screen shots), provide employees with basic knowledge about NSPS and performance management principles in general, and are used by employees to develop their job objectives. As another example, the PEO developed training guides to educate employees on changes to the NSPS classroom materials resulting from the revised NSPS regulations and implementing issuances. The PEO also developed a Web site for accessing NSPS learning materials, resources, and other tools. In addition, we found that the Air Force has begun incorporating training on NSPS as a normal part of its operations and is working to embed NSPS topics in the regular training provided to Air Force civilians and servicemembers. Figure 2: Sample Screen Shots from the NSPS 101 and iSuccess Training Modules: [Refer to PDF for image: sample computer screenshots] Source: DOD. [End of figure] Although DOD and the components continue to take steps to provide employees with training on NSPS, the department has not yet evaluated the effectiveness of the training provided. We previously reported that it is increasingly important for agencies to measure the real impact of training and thus evaluate their training--for example, by establishing clear goals about what the training is expected to achieve along with agreed-upon measures or performance indicators to ascertain progress toward achieving those goals.[Footnote 30] DOD has outlined the fundamental requirements, or goals, of the training that it provides to employees on NSPS. Specifically, DOD has noted that for NSPS, a training program must be implemented that enables employees to understand better how to succeed, and enables supervisors to communicate performance expectations to their employees, provide feedback to them on their performance against these expectations, and tell them what steps they can take to improve their performance and competencies and manage their careers. However, while DOD has undertaken efforts to understand employees' perceptions of its training, the department has not yet evaluated the effectiveness of the training that it provides. For example, in early 2009 the PEO conducted what PEO officials describe as an ad hoc study of training needs. The PEO's study consisted of conducting sensing sessions with 120 human resource practitioners at 11 locations across the department.[Footnote 31] According to the PEO, the purpose of these sessions included obtaining feedback on existing NSPS learning products and support and exploring options for next generation products. While the PEO's study identified some needed improvements, it does not constitute an evaluation of the department's training--for example, because it did not assess the department's progress toward achieving the goals specified for the training. As another example, DOD's 2008 evaluation report also highlighted deficiencies with the department's training on NSPS.[Footnote 32] Specifically, the report notes that without effective communication and training, NSPS cannot achieve its goal of being a credible and trusted system. Further, three of the report's six key recommendations focus on the need to enhance the effectiveness of the training provided to employees on NSPS: (1) provide more training on the performance management system; (2) provide high-level training for employees and supervisors that explains the pay pool process; and (3) hold mock pay pool panels, which serve as refreshers for continuing panel members and as training for new members. However, like the PEO's study, DOD's 2008 evaluation report does not constitute an evaluation of the department's training--for example, because it did not include an in-depth assessment of DOD's training and also did not assess the department's progress toward achieving the goals for the training, per agreed-upon measures or performance indicators. Of the components, we found that only the Army has taken some steps to evaluate the training it provides to employees on the system. Specifically, the Army assesses the adequacy of NSPS training during on-site reviews that it conducts as part of its implementation of the system. The Army conducted three such assessments during 2008 and an additional four such assessments during 2009. With respect to our site visits, although we found that each of the eight locations provided training on NSPS to employees, officials at each location also expressed concerns over the effectiveness of the training provided. For example, officials at seven locations told us that additional training was needed on writing job objectives or employee self-assessments under the system, while other officials noted that additional training was needed on the pay pool process, use of the system's automated tools, and how the reconsideration process works. Similarly, officials at two locations raised concerns that the training they received did not provide them with the skills they needed to use the Performance Appraisal Application. For example, officials told us that they received training too early and had either forgotten it or the training had become outdated by the time they actually used the Performance Appraisal Application. Further, some program officials raised concerns about their organizations' ability to provide employees with adequate training on the system when the employees are located outside the continental United States. For example, program officials at one location in Germany reported challenges providing employees located in the field with training on NSPS because they lack the resources to send NSPS trainers to those locations. However, until DOD evaluates its training, it will not be able to determine whether the training meets the needs of its employees or the department is making progress toward achieving the goals it specified for the training. Ongoing Performance Feedback and Dialogue between Supervisors and Employees: DOD continues to take steps to ensure that employees receive timely performance feedback that is meaningful, constructive, and in accordance with the department's requirements. In our first assessment of NSPS we noted that DOD's implementing issuances required at least one documented interim performance review and an annual performance appraisal and that the Performance Appraisal Application allowed supervisors and employees to document these feedback sessions. Since then, DOD has taken additional steps to enhance the Performance Appraisal Application by modifying the tool to allow supervisors and employees to identify where they are in the performance appraisal process and help them accomplish required actions in a timely manner. During our site visits, officials at all eight locations told us that NSPS helps ensure the occurrence of performance feedback between employees and supervisors. For example, officials noted that use of the Performance Appraisal Application encourages employee feedback by allowing employees to document and track feedback sessions, and that NSPS encourages direct discussions about performance-related issues, such as developing effective job objectives and establishing performance expectations. System to Better Link Individual Pay to Performance in an Equitable Manner: DOD continues to take steps to better link individual pay to performance as well. As we noted in our 2008 report, the NSPS performance management system uses a multirating system of five rating categories--of which the lowest rating is "1" (unacceptable performance) and the highest rating is a "5" (role model performance)- -that allows distinctions to be made in employee performance and therefore compensation. Since then, DOD added details to the NSPS regulations to facilitate uniform, equitable practices across the department that accord with merit system principles. These include specifying specific share assignment ranges, rounding rules for converting raw performance scores to ratings, and formulas for determining share value and the calculation of performance payouts under NSPS. DOD also clarified that a common share value should apply throughout an entire pay pool. According to the PEO, these changes, in addition to the higher-level review of performance expectations, recommendations for ratings of record, share assignment, and payout distribution have helped to promote a more equitable method for appraising and compensating employees. However, during our site visits, officials at seven of the eight locations told us that they saw the potential for factors other than performance to influence employees' ratings, such as the quality of employees' and supervisors' writing skills. For example, rating officials at one location commented that NSPS does not reward employees based on their performance but rather on how well employees and supervisors can communicate in writing what the employee achieved during the performance management cycle. Similarly, at another location, a pay pool panel member told us that the paperwork submitted to the panel by both the employee and the supervisor must be of very high quality because it can be difficult to defend a high rating recommended for an employee if the assessments are poorly written. DOD's 2008 evaluation report also highlighted concerns from employees and managers over the department's success in linking pay to performance under NSPS. For example, DOD's report noted that while some employees believed that they saw some level of pay for performance under NSPS, others were uncertain. Further, DOD's report also noted that some managers observed that the quality of written assessments contributed significantly to the way in which ratings were substantiated. Means to Ensure That Adequate Agency Resources Are Allocated for System Design, Implementation, and Administration: We found that DOD also continues to take steps to ensure that adequate agency resources are allocated to NSPS. According to 5 U.S.C. 9902(e)(4), to the maximum extent practicable, for fiscal years 2004 through 2012, the overall amount of money allocated for compensation of civilian employees in organizations under NSPS shall not be less than the amount that would have been allocated under the GS system. To meet the requirements of 5 U.S.C. 9902(e)(4), DOD's implementing issuances require that the components certify in writing to the Deputy Secretary of Defense through the Under Secretary of Defense for Personnel and Readiness that the amount expended for NSPS performance-based pay increases is no less than what would have been expended had these positions not been converted into NSPS. Following the 2008 NSPS performance management cycle, each of the components certified that it met this requirement. DOD also continues to capture NSPS implementation costs, and for fiscal year 2008, the PEO reported that NSPS implementation costs were about $61.8 million. According to the PEO, continuing implementation of NSPS will result in some additional program implementation costs, such as for training for NSPS, conducting outreach to employees and others, and improving the system's automated tools. However, the PEO estimates that once the conversion of all non-bargaining unit employees is complete, the system's implementation costs will decrease significantly unless there is a decision to convert bargaining unit employees. Thereafter, according to the PEO, the cost of administering NSPS will be no different than that of any other personnel system. Predecisional Internal Safeguards to Determine if Rating Results Are Consistent, Equitable, and Nondiscriminatory: In our 2008 report, we recommended that DOD take steps to better ensure the consistency and equity of the performance management process by requiring a third party to perform predecisional demographic and other analysis of the pay pool results. DOD did not concur with our recommendation, stating that its postdecisional analysis of the rating results was useful for identifying barriers and any needed corrective action. DOD also stated that if the information obtained from a postdecisional demographic analysis demonstrated that the results were not fair or equitable, for any reason, the process used to achieve those results could be examined with the intent to identify and eliminate barriers to a fair and equitable outcome. However, our review of the postdecisional analyses that the PEO and each of the components completed following the 2007 NSPS performance management cycle and the analyses that each of the eight organizations we visited for our review completed following the most recent performance management cycle in 2008 found these analyses were inconsistent, did not always include an analysis of the rating results by demographics, and were generally conducted at too high a level to provide decision makers with sufficient information to identify potential barriers or corrective actions. For example, we found that following the 2007 performance management cycle, the PEO analyzed the rating results for more than 100,000 employees by select demographic groups, such as race, gender, ethnicity, age, veteran status, and targeted disability, but limited its analysis to the aggregate data from its pay pools. That is, the PEO did not analyze the rating results at the level where decisions are made--in the case of NSPS ratings and payouts, the pay pool level. Similarly, in analyzing the postdecisional analyses that the components conducted following the 2007 performance management cycle, we found inconsistencies in their approaches, primarily because the components were allowed to develop their own approaches for conducting this analysis. For example, only the Army and Fourth Estate included an analysis of the rating results by demographics as part of their respective postdecisional analyses. However, we also found that neither the Army's nor the Fourth Estate's demographic analysis of the ratings provided decision makers with sufficient information to identify possible barriers or corrective actions that could be taken to address such barriers. As with the PEO, this problem results because neither the Army nor the Fourth Estate conducted its analysis at the pay pool level. Of the eight locations we visited for our review, we found that only one organization's postdecisional analysis following the 2008 performance management cycle included an analysis of its ratings results by demographics. Since we issued our 2008 report, DOD has taken steps to promote a degree of consistency in its postdecisional analysis of NSPS ratings and payout data. Specifically, in December 2008, DOD revised its implementing issuance to require the heads of DOD components to conduct an annual analysis of NSPS performance ratings and payouts for subordinate elements, and issue guidance to lower echelons and otherwise act to identify, examine, and remove barriers to similar rating and payout potential for demographic and other groups in the workforce. Further, in May 2009, the PEO issued guidance, entitled Guidance for Conducting Annual Analysis of NSPS Performance Ratings and Payouts, in order to provide the components with a framework and suggested approaches for conducting their annual analysis and to serve as a starting point for identifying and examining barriers. For example, the guidance notes that the NSPS performance management system's rating and payout process has four main outcomes--the rating of record, number of shares awarded, payout, and the distribution of the payout--and that each outcome should be reviewed. The guidance also notes that analysis is best done at the level where decisions are made- -in the case of NSPS ratings and payouts, the pay pool level. Further, the guidance: * expresses the expectation that as the components conduct their analyses, changes and improvements to the guidance will be made; * is careful to ensure that the components understand base parameters for conducting the analysis so it is conducted in a manner that is methodologically sound; * encourages consultation with experts, such as statisticians and human resources researchers, to assist with determining the most suitable analytical models to employ, the statistical tools to utilize, and the standards to adopt in relation to understanding, measuring, and reporting significant findings; and: * makes responsibility for conducting the analysis a shared responsibility between various offices, including the components' legal, equal employment opportunity, and human resources offices, but notes that the components should consider tasking their Office of General Counsel or Office of the Judge Advocate General, whose staff are well positioned to ensure that the components are in compliance with applicable statutes, regulations, and policies, with primary responsibility for conducting the analysis and ensuring that adequate resources are provided in support of the function. While issuance of the May 2009 guidance represents a noteworthy step, the guidance does not address all steps suggested by the Equal Employment Opportunity Commission for identifying and addressing potential barriers to fair, consistent, and equitable ratings. The Equal Employment Opportunity Commission's Instructions to Federal Agencies for EEO Management Directive 715[Footnote 33] provides guidance that agencies can use in identifying and addressing potential barriers.[Footnote 34] The instructions state that "barrier identification and elimination is the process by which agencies uncover, examine, and remove barriers to equal participation at all levels of the workforce." Management Directive 715 further states that "where it is determined that an identified barrier serves no legitimate purpose with respect to the operation of an agency, this Directive requires that agencies take immediate steps to eliminate the barrier." In conducting their analysis, the components' data may uncover barriers or other potential problems. However, understanding why the barrier or problem exists, or what to do to address it, may require that the components take additional steps. To identify and eliminate potential barriers, the directive outlines a four-step process: (1) analyzing workforce data to identify potential barriers; (2) investigating actual barriers and their causes; (3) eliminating barriers, which includes devising a plan for improvement and developing overall objectives for barrier elimination, with corresponding action items, responsible personnel, and target dates; and (4) assessing the success of the plan. The PEO's guidance aims to promote a degree of uniformity and standardization in conducting postdecisional analyses. However, the PEO's guidance does not specify what process the components should follow to investigate potential barriers and their causes, nor does it specify a process for eliminating barriers that are found. By not specifying such steps in its guidance, the components may not follow a consistent approach when investigating barriers, which could hinder their efforts to eliminate them. While not predecisional, we recognize that DOD's approach does provide some benefits, some of which are similar to those of a predecisional analysis. For example, DOD's approach lays out a method of analyzing ratings that would address some of the merit principles in 5 U.S.C. 2301--for example, that employees should receive fair and equitable treatment in all aspects of personnel management and that employees should be protected against arbitrary action, personal favoritism, or coercion for partisan political purposes.[Footnote 35] However, as stated previously, because DOD does not specify what process the components should follow to investigate and eliminate potential barriers, the components may not follow a consistent approach, which could hinder their efforts to ensure fair, consistent, and equitable ratings. Reasonable Transparency of the System and Its Operation: While DOD continues to take steps to ensure a reasonable amount of transparency in its implementation of NSPS, concerns about the overall transparency of the system continue to exist. To improve the transparency of the NSPS performance management system, our September 2008 report recommended that DOD require commands to publish the final rating results to employees. DOD concurred with our recommendation and, in November 2008, amended its NSPS regulations and implementing issuances to require commands to publish the final rating results to employees. Under DOD's revised guidance, performance review authorities are required to communicate the general pay pool results to the NSPS workforce in writing. At a minimum, this includes the number of pay pools (if aggregate pay pool results are necessary), the number of employees rated, the rating and share distributions, the average rating, the average share assignment, the share value (or average share value), and the average payout expressed as a percentage of base salary. At the eight locations we visited, we found that each of pay pools shared this information with employees following the 2008 NSPS performance management cycle. DOD continues to take other steps to ensure a reasonable amount of transparency of the NSPS performance management system. In May 2009, the PEO launched "NSPS Connect," a centralized Web portal for employees, managers, and others to access NSPS products, such as online training courses, fact sheets, tips sheets, and information on the system's automated tools. The PEO also continues to take steps to collect and share lessons learned on the department's experiences implementing NSPS and facilitate lessons learned briefings with the components following each performance management cycle. Further, the PEO continues to report periodically on internal assessments and employee survey results relating to the NSPS performance management system. For example, in May 2009, the PEO published the results of its 2008 evaluation of NSPS. Similarly, as we previously reported, DOD posts the results of its survey of civilian employees on a Web site that is accessible to DOD employees, supervisors, and managers. Officials at each of the eight locations we visited told us that efforts were being made to help ensure transparency of the NSPS performance management system and the rating process. According to officials, among the steps being taken to help ensure transparency, for example, were establishing multiple communities of practice, disseminating business rules and other guidance on NSPS to employees and managers under the system, and publishing monthly newsletters on NSPS. At seven of the locations, officials told us that town hall meetings were held to keep employees informed of NSPS-related developments, and officials at six locations told us that mock pay pool panels were held to show employees how the pay pool process works. However, our site visits revealed some concerns about the overall transparency of the system. For example, at three locations officials expressed concerns over a lack of transparency with regard to their pay pools' business rules, indicating that their business rules either had not been published or were published well after the performance management cycle had begun. At one location, pay pool panel members told us that though it was 6 months into the current performance management cycle, they did not yet have copies of their business rules. Similarly, rating officials at four locations told us that they did not understand what constituted a rating of "4" because neither their pay pools nor business rules provided clear criteria. Meaningful Distinctions in Individual Employee Performance: Although our site visits revealed concerns over the extent to which meaningful distinctions in individual employee performance are being made under NSPS, DOD has taken some steps that include addressing a recommendation we made in our 2008 report aimed at encouraging the use of all available rating categories. Specifically, at all eight locations we visited, officials told us that they did not believe NSPS was being implemented in a manner that encouraged meaningful distinctions in individual employee performance. For example, officials at four locations told us that they were hesitant to give ratings lower than a "3," and officials at six locations told us that they believed that there was a forced distribution of the ratings or manipulation of the ratings in order to achieve a predetermined quota. Further, at one location we found that management stressed the importance of maintaining employee share value at above 2.0, which would result in a higher payout, regardless of the employee's rating. According to the PEO, any effort to limit share value is a roundabout way to establish preset limits on ratings and would constitute forced distribution, which the NSPS regulations prohibit.[Footnote 36] However, in response to concerns about the potential for the forced distribution of performance ratings under NSPS, in April 2009 the department posted to its NSPS Web site a fact sheet emphasizing that the forced distribution of ratings is prohibited under NSPS and describing how meaningful distinctions in performance are made under the system. DOD's fact sheet provides guidance specifying what constitutes the forced distribution of ratings, why the forced distribution of ratings is prohibited, how use of standard performance indicators minimizes the potential for individual bias or favoritism, and how organizations can best apply this information when rating and rewarding employee performance under NSPS. DOD has also noted that if employees believe their rating did not result from meaningful distinctions or are unfair, the system affords them the opportunity to challenge their ratings through a formal process known as reconsideration. See appendix II for further information on the NSPS reconsideration process. Although DOD Monitors Some Aspects of the Implementation of NSPS, It Does Not Monitor How the Safeguards Specifically Are Implemented across the Department: Within DOD, both the PEO and the components are responsible for monitoring the implementation of NSPS. As part of its efforts to monitor the implementation of NSPS, the PEO conducts broad, annual evaluations of NSPS to determine whether the system is on track to achieve certain goals, or key performance parameters,[Footnote 37] by, among other things, monitoring employee perceptions of the system using DOD's survey of civilian employees. In May 2009, the PEO published its first evaluation of NSPS, which focused on determining whether NSPS, as implemented in spiral 1 organizations, was on track to achieve some of the goals specified by DOD and if any improvements were needed.[Footnote 38] While some of DOD's goals for the system lend themselves to comparisons to the safeguards--for example, one of its goals is ensuring a credible and trusted system (which could align with the transparency safeguard)--PEO officials stated that to date, their focus has been limited to understanding how the components have generally implemented NSPS and not on monitoring or assessing the components' implementation of the safeguards. With respect to the components, DOD's implementing issuances state that the heads of DOD components are accountable for the manner in which management in their organizations carries out NSPS policy, procedures, and guidance. However, we found that only the Army has taken steps similar to the PEO to assess whether it is on track to achieve DOD's goals for the system--the other components have not done so. Furthermore, none of the components monitors how the safeguards specifically are implemented within their organizations because there is no requirement to do so. We have previously reported that transitioning to a more performance- oriented pay system is a huge undertaking that requires constant monitoring.[Footnote 39] Further, in our 2008 assessment of NSPS, we noted that continued monitoring of the safeguards was needed to ensure that DOD's actions were effective as implementation of NSPS proceeded. While DOD's efforts to date have helped provide decision makers with some information on how the department is implementing NSPS, including potential areas for changes or improvements, they do not provide decision makers in DOD and the Congress with information to determine whether the safeguards specifically have been implemented effectively. Without monitoring the safeguards' implementation, decision makers in DOD and the Congress lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. For example, in conducting our review, we identified some issues related to the implementation of the safeguards that illustrate the need to monitor their implementation. These include the following: * Ensuring that adequate agency resources are allocated for the system's design, implementation, and administration. We found that each of the components generally lacks visibility over the reasons why organizations have supplemented their pay pool funds. For example, Air Force NSPS program officials told us that for the 2009 payout, while they knew that 8 of the Air Force's 18 major commands supplemented their pay pool funds, they did not know the specific reasons why. According to PEO officials, organizations might elect to supplement their pay pool funds for a variety of reasons--for example, to recruit or retain employees or to compete with other organizations for talent. However, because they do not understand the reasons why pay pools supplement their pay pool funding--which could help DOD and the components understand the extent to which adequate resources have been allocated to the system--decision makers cannot identify or assess any trends in these practices. Indeed, DOD's 2008 evaluation report notes that some employees in organizations that supplemented their pay pools' funding questioned whether the higher funding levels could be sustained over the long term. * Ensuring reasonable transparency of the system and its operation. We found evidence that three of the pay pools we visited deviated from their business rules during the last performance management cycle, indicating a lack of transparency of the performance management process in some instances. DOD's guidance states that a pay pool's business rules are the guiding principles or ground rules that are used throughout the pay pool process, that pay pool panels should establish these principles and hold one another accountable for following them, and that a pay pool's policies--which would include its business rules- -will be made available to employees before the end of the performance cycle. However, at three of the locations we visited, we found evidence that the pay pool had deviated from its business rules during the course of the last performance management cycle. For example, at one location the pay pool's business rules required all recommended ratings be reviewed, noting that the pay pool panel will ensure that all employees receive appropriate consideration and that ratings are fair and consistent. However, officials we spoke with at that location told us that they did not review all recommended ratings in accordance with their business rules. Rather, only the recommended ratings of "4" or "5" were reviewed. As another example, we found evidence that a pay pool at another location used different criteria than what was specified in its business rules for allocating the number of shares to employees. According to component-level NSPS program officials, in order to ensure transparency of the system, pay pools should not deviate from their business rules once those rules are published. However, none of the components requires its pay pools to notify it when such an event occurs, or of the reasons why, though doing so could help provide decision makers with information on the extent to which pay pools are implementing the system in a manner that is transparent to employees. Although DOD Civilian Employees under NSPS Have Mixed Views about the System, DOD Has Not Yet Developed and Implemented a Plan to Address Employees' Negative Perceptions of Some Aspects of the System: DOD civilian personnel have mixed perceptions about NSPS, and although DOD has taken some steps toward addressing employees' concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system. DOD's most recent survey of civilian employees reveals that NSPS employees have mixed perceptions about NSPS. The responses to questions specifically asking about NSPS show positive perceptions about some aspects of performance management under NSPS, including connecting pay to performance, but show negative perceptions about other aspects of performance management, such as the appraisal process. Further, the most recent data indicate that the perceptions of those employees who have worked under NSPS the longest appear to have remained largely unchanged from the negative perceptions we reported in 2008. Moreover, during discussion groups we held at eight locations outside the continental United States, civilian employees and supervisors expressed consistent concerns and negative perceptions of NSPS that are similar to those identified in our 2008 report, although they also identified positive aspects of the system. DOD has taken some steps to address employees' negative perceptions of the system; however, the department has yet to develop and implement an action plan that meets the intent of our prior recommendation because it does not specify such things as the actions DOD intends to take, who will be responsible for taking the action, and the timelines for doing so. We continue to believe that implementing such an action plan is important and note that doing so would be a step that DOD could take to involve employees in the system's implementation--which is one of the safeguards we previously discussed. NSPS Employees Express Mixed Perceptions about the System: NSPS Employees in All Spirals Have Positive Views of Some Aspects of Performance Management but Negative Views of Other Aspects: According to DOD's most recent survey data, some NSPS employees recognize that positive aspects of performance management, such as connecting pay to performance, exist under the system. For example, as shown in table 2, DOD's survey data for 2008 indicate that an estimated 38 percent of NSPS employees agree that differences in their performance are recognized in meaningful ways, as compared with an estimated 33 percent of non-NSPS employees. Further, an estimated 42 percent of NSPS employees agree that pay raises depend on how well employees perform their jobs, as compared with an estimated 25 percent of non-NSPS employees. When asked about how poor performers are dealt with, an estimated 34 percent of NSPS employees, versus an estimated 27 percent of non-NSPS employees, agreed that steps are taken to deal with poor performers. Table 2: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about Performance Management in the 2008 Status of Forces Survey of Civilian Employees: Performance management questions: Differences in performance are recognized in meaningful ways; Employee description: NSPS; Percentage responding: "Agree" 38; Percentage responding: "Neither/nor" 31; Percentage responding: "Disagree" 28. Performance management questions: Differences in performance are recognized in meaningful ways; Employee description: Non-NSPS; Percentage responding: "Agree" 33; Percentage responding: "Neither/nor" 32; Percentage responding: "Disagree" 31. Performance management questions: Pay raises depend on how well employees perform their jobs; Employee description: NSPS; Percentage responding: "Agree" 42; Percentage responding: "Neither/nor" 26; Percentage responding: "Disagree" 30. Performance management questions: Pay raises depend on how well employees perform their jobs; Employee description: Non-NSPS; Percentage responding: "Agree" 25; Percentage responding: "Neither/nor" 29; Percentage responding: "Disagree" 42. Performance management questions: Performance standards/expectations are directly related to the organization's mission; Employee description: NSPS; Percentage responding: "Agree" 70; Percentage responding: "Neither/nor" 21; Percentage responding: "Disagree" 9. Performance management questions: Performance standards/expectations are directly related to the organization's mission; Employee description: Non-NSPS; Percentage responding: "Agree" 65; Percentage responding: "Neither/nor" 25; Percentage responding: "Disagree" 10. Performance management questions: In my work unit, steps are taken to deal with a poor performer who cannot or will not improve; Employee description: NSPS; Percentage responding: "Agree" 34; Percentage responding: "Neither/nor" 31; Percentage responding: "Disagree" 30. Performance management questions: In my work unit, steps are taken to deal with a poor performer who cannot or will not improve; Employee description: Non-NSPS; Percentage responding: "Agree" 27; Percentage responding: "Neither/nor" 30; Percentage responding: "Disagree" 38. Performance management questions: My bonus and cash awards depend on how well I perform my job; Employee description: NSPS; Percentage responding: "Agree" 61; Percentage responding: "Neither/nor" 17; Percentage responding: "Disagree" 23. Performance management questions: My bonus and cash awards depend on how well I perform my job; Employee description: Non-NSPS; Percentage responding: "Agree" 55; Percentage responding: "Neither/nor" 20; Percentage responding: "Disagree" 25. Source: Defense Manpower Data Center. Note: The estimated percentages are based on a 95 percent confidence interval and the margin of error is within +/-1 percent. The response categories are collapsed for positive ("agree") and negative ("disagree") responses. That is, "agree" is the estimated percentage of employees who responded either "agree" or "strongly agree," while "disagree" is the estimated percentage of employees who responded either "disagree" or "strongly disagree." Totals may sum to less than or more than 100 percent as a result of collapsing the positive and negative response categories. [End of table] In comparison, an estimated 47 percent of non-NSPS employees, as compared with an estimated 44 percent of NSPS employees, agreed that their current performance appraisal system motivates them to perform well. Further, an estimated 34 percent of non-NSPS employees, as compared with an estimated 29 percent of NSPS employees, agreed that their performance appraisal system improves organizational performance. Table 3 shows additional comparisons between NSPS and non- NSPS employee responses to questions about performance appraisals. Table 3: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about Performance Appraisals in the 2008 Status of Forces Survey of Civilian Employees: Performance appraisal questions: Performance appraisal is a fair reflection of performance; Employee description: NSPS; Percentage responding: "Agree" : 62; Percentage responding: "Neither/nor" 18; Percentage responding: "Disagree" 19. Performance appraisal questions: Performance appraisal is a fair reflection of performance; Employee description: Non-NSPS; Percentage responding: "Agree" : 65; Percentage responding: "Neither/nor" 19; Percentage responding: "Disagree" 14. Performance appraisal questions: In my most recent appraisal, I understood what I had to do to be rated at different performance levels; Employee description: NSPS; Percentage responding: "Agree" : 59; Percentage responding: "Neither/nor" 16; Percentage responding: "Disagree" 22. Performance appraisal questions: In my most recent appraisal, I understood what I had to do to be rated at different performance levels; Employee description: Non-NSPS; Percentage responding: "Agree" : 65; Percentage responding: "Neither/nor" 17; Percentage responding: "Disagree" 14. Performance appraisal questions: My current performance appraisal system motivates me to perform well; Employee description: NSPS; Percentage responding: "Agree" : 44; Percentage responding: "Neither/ nor" 28; Percentage responding: "Disagree" 28. Performance appraisal questions: My current performance appraisal system motivates me to perform well; Employee description: Non-NSPS; Percentage responding: "Agree" : 47; Percentage responding: "Neither/nor" 31; Percentage responding: "Disagree" 22. Performance appraisal questions: The performance appraisal system I am under improves organizational performance; Employee description: NSPS; Percentage responding: "Agree" : 29; Percentage responding: "Neither/nor" 36; Percentage responding: "Disagree" 36. Performance appraisal questions: The performance appraisal system I am under improves organizational performance; Employee description: Non-NSPS; Percentage responding: "Agree" : 34; Percentage responding: "Neither/nor" 40; Percentage responding: "Disagree" 26. Performance appraisal questions: Procedures for reconsidering performance appraisal ratings are fair; Employee description: NSPS; Percentage responding: "Agree" : 37; Percentage responding: "Neither/ nor" 45; Percentage responding: "Disagree" 19. Performance appraisal questions: Procedures for reconsidering performance appraisal ratings are fair; Employee description: Non-NSPS; Percentage responding: "Agree" : 38; Percentage responding: "Neither/nor" 45; Percentage responding: "Disagree" 17. Source: Defense Manpower Data Center. Note: The estimated percentages are based on a 95 percent confidence interval and the margin of error is within +/-1 percent. The response categories are collapsed for positive ("agree") and negative ("disagree") responses. That is, "agree" is the estimated percentage of employees who responded either "agree" or "strongly agree," while "disagree" is the estimated percentage of employees who responded either "disagree" or "strongly disagree." Totals may sum to less than or more than 100 percent as a result of collapsing the positive and negative response categories. [End of table] Perceptions of Employees under NSPS the Longest Remain Largely Unchanged since GAO's 2008 Report: In our first assessment of NSPS, we reported that the results of DOD's Status of Forces Survey of Civilian Employees indicated that the perceptions of employees who had been under the system the longest had become more negative on questions related to performance management. However, the results of DOD's most recent administration of the survey in 2008 indicate that spiral 1.1 employee perceptions are about the same as the May 2007 survey, as shown in table 4. For example, from the November 2006 through February 2008 administrations of DOD's survey, the percentage of spiral 1.1 employees that agreed that they understood what they had to do to be rated at a different performance level declined from an estimated 59 percent in November 2006 to an estimated 53 percent in May 2007, then remained largely unchanged in February 2008 at an estimated 54 percent.[Footnote 40] Table 4: Estimated Percentage of Responses from Spiral 1.1 Employees to Selected Questions about Performance Management from the May 2006, November 2006, May 2007, and February 2008 Administrations of the Status of Forces Survey of Civilian Employees: Performance management questions: Performance appraisal is a fair reflection of performance; Survey administration: May 2006; Percentage of spiral 1.1 employees responding: "Agree" 67; Percentage of spiral 1.1 employees responding: "Neither/ nor" 20; Percentage of spiral 1.1 employees responding: "Disagree" 12. Performance management questions: Performance appraisal is a fair reflection of performance; Survey administration: November 2006; Percentage of spiral 1.1 employees responding: "Agree" 59; Percentage of spiral 1.1 employees responding: "Neither/ nor" 22; Percentage of spiral 1.1 employees responding: "Disagree" 16. Performance management questions: Performance appraisal is a fair reflection of performance; Survey administration: May 2007; Percentage of spiral 1.1 employees responding: "Agree" 52; Percentage of spiral 1.1 employees responding: "Neither/ nor" 21; Percentage of spiral 1.1 employees responding: "Disagree" 25. Performance management questions: Performance appraisal is a fair reflection of performance; Survey administration: February 2008; Percentage of spiral 1.1 employees responding: "Agree" 54; Percentage of spiral 1.1 employees responding: "Neither/ nor" 20; Percentage of spiral 1.1 employees responding: "Disagree" 25. Performance management questions: Differences in performance are recognized in meaningful ways; Survey administration: May 2006; Percentage of spiral 1.1 employees responding: "Agree" 37; Percentage of spiral 1.1 employees responding: "Neither/ nor" 33; Percentage of spiral 1.1 employees responding: "Disagree" 28. Performance management questions: Differences in performance are recognized in meaningful ways; Survey administration: November 2006; Percentage of spiral 1.1 employees responding: "Agree" 35; Percentage of spiral 1.1 employees responding: "Neither/ nor" 31; Percentage of spiral 1.1 employees responding: "Disagree" 31. Performance management questions: Differences in performance are recognized in meaningful ways; Survey administration: May 2007; Percentage of spiral 1.1 employees responding: "Agree" 32; Percentage of spiral 1.1 employees responding: "Neither/ nor" 31; Percentage of spiral 1.1 employees responding: "Disagree" 34. Performance management questions: Differences in performance are recognized in meaningful ways; Survey administration: February 2008; Percentage of spiral 1.1 employees responding: "Agree" 35; Percentage of spiral 1.1 employees responding: "Neither/ nor" 30; Percentage of spiral 1.1 employees responding: "Disagree" 32. Performance management questions: In most recent appraisal, I understood what I had to do to be rated at different performance levels; Survey administration: May 2006[A]; Percentage of spiral 1.1 employees responding: "Agree" n/a; Percentage of spiral 1.1 employees responding: "Neither/ nor" n/a; Percentage of spiral 1.1 employees responding: "Disagree" n/a. Performance management questions: In most recent appraisal, I understood what I had to do to be rated at different performance levels; Survey administration: November 2006; Percentage of spiral 1.1 employees responding: "Agree" 59; Percentage of spiral 1.1 employees responding: "Neither/ nor" 17; Percentage of spiral 1.1 employees responding: "Disagree" 22. Performance management questions: In most recent appraisal, I understood what I had to do to be rated at different performance levels; Survey administration: May 2007; Percentage of spiral 1.1 employees responding: "Agree" 53; Percentage of spiral 1.1 employees responding: "Neither/ nor" 17; Percentage of spiral 1.1 employees responding: "Disagree" 29. Performance management questions: In most recent appraisal, I understood what I had to do to be rated at different performance levels; Survey administration: February 2008; Percentage of spiral 1.1 employees responding: "Agree" 54; Percentage of spiral 1.1 employees responding: "Neither/ nor" 17; Percentage of spiral 1.1 employees responding: "Disagree" 27. Source: Defense Manpower Data Center. Note: The estimated percentages are based on a 95 percent confidence interval and the margin of error is within +/-2 percent. The response categories are collapsed for positive ("agree") and negative ("disagree") responses. That is, "agree" is the estimated percentage of employees who responded either "agree" or "strongly agree," while "disagree" is the estimated percentage of employees who responded either "disagree" or "strongly disagree." Totals may sum to less than or more than 100 percent as a result of collapsing the positive and negative response categories. [A] This question was not asked on the May 2006 administration of the Status of Forces Survey of Civilian Employees. [End of table] In addition, when asked about the overall impact that NSPS will have on personnel practices in DOD, spiral 1.1 employees' perceptions have become significantly more negative since first converting to NSPS in 2006, but showed little change between the May 2007 and February 2008 surveys. Specifically, the results of the 2008 survey indicate that an estimated 22 percent of spiral 1.1 employees responded that the overall impact of NSPS on the department's personnel practices would be positive, as compared to an estimated 23 percent in May 2007 and an estimated 25 percent in November 2006. Table 5 shows a comparison of spiral 1.1 employee responses over time about the overall impact of NSPS on personnel practices in DOD. Table 5: Estimated Percentage of Responses from Spiral 1.1 Employees about the Overall Impact of the National Security Personnel System from the May 2006, November 2006, May 2007, and February 2008 Administrations the Status of Forces Survey of Civilian Employees: Overall Impact question: Overall, what type of impact will NSPS have on personnel practices in the DOD?; Survey administration: May 2006; Percentage of spiral 1.1 employees responding: "Positive" 40; Percentage of spiral 1.1 employees responding: "Neither/ nor" 35; Percentage of spiral 1.1 employees responding: "Negative" 25. Overall Impact question: Overall, what type of impact will NSPS have on personnel practices in the DOD?; Survey administration: November 2006; Percentage of spiral 1.1 employees responding: "Positive" 25; Percentage of spiral 1.1 employees responding: "Neither/ nor" 32; Percentage of spiral 1.1 employees responding: "Negative" 42. Overall Impact question: Overall, what type of impact will NSPS have on personnel practices in the DOD?; Survey administration: May 2007; Percentage of spiral 1.1 employees responding: "Positive" 23; Percentage of spiral 1.1 employees responding: "Neither/ nor" 30; Percentage of spiral 1.1 employees responding: "Negative" 48. Overall Impact question: Overall, what type of impact will NSPS have on personnel practices in the DOD?; Survey administration: February 2008; Percentage of spiral 1.1 employees responding: "Positive" 22; Percentage of spiral 1.1 employees responding: "Neither/ nor" 28; Percentage of spiral 1.1 employees responding: "Negative" 50. Source: Defense Manpower Data Center. Note: The estimated percentages are based on a 95 percent confidence interval and the margin of error is within +/-3 percent. The response categories are collapsed for positive and negative responses. That is, "positive" is the estimated percentage of employees who responded either "positive" or "very positive," while "negative" is the estimated percentage of employees who responded either "negative" or "very negative." Totals may sum to less than or more than 100 as a result of collapsing the positive and negative response categories. [End of table] NSPS Employees and Supervisors in Our Discussion Groups Expressed Consistent Concerns: As with our first review of NSPS, DOD civilians in our discussion groups at locations outside the continental United States continue to express wide-ranging but consistent concerns about the NSPS performance management system. Although the results of our discussion groups are not generalizable to the entire population of DOD civilian employees, the themes that emerged provide valuable insight into employees' perceptions about the implementation of NSPS thus far. Specifically, during these discussion groups, participants at six locations told us that they were initially optimistic about the intent of NSPS and the concept of pay for performance. Further, some participants indicated that they remain positive about the amount of performance pay and flexibilities afforded to supervisors to rate their employees under the system. However, as with our first review, discussion group participants at all eight locations we visited primarily expressed frustration with and concern about certain aspects of NSPS implementation and the system. The prevalent themes that emerged during our discussion groups include concerns over (1) the negative impact of NSPS on employee motivation and morale, (2) the excessive amount of time spent navigating the performance management process, (3) challenges with job objectives, (4) factors undermining employee confidence in the system, and (5) factors unrelated to job performance affecting employees' final performance ratings. As we noted in 2008, the themes that emerged during our discussion group sessions are not surprising. Our prior work, as well as that of OPM,[Footnote 41] has recognized that organizational transformations, such as the adoption of a new performance management system, often entail fundamental and radical changes that require an adjustment period to gain employees' trust and acceptance. As a result, we expect major change management initiatives in large-scale organizations to take several years to be fully successful. Concerns about a Negative Impact on Motivation and Morale: A prevalent theme from our discussions with both employees and supervisors was that several aspects of NSPS have had a negative impact on employee motivation and morale--consistent with our first assessment of NSPS. Specifically, discussion group participants at all eight locations discussed how various aspects of NSPS--for example, their perception that a rating of "3" is only average--have negatively affected their motivation and morale. Discussion group participants at six of the eight locations also told us they have negative perceptions of what a rating of "3" means. At five of those locations, discussion group participants told us that they continue to believe that a rating of "3" means only "average," in contrast to "valued performer," as it was initially defined to the workforce by DOD. Discussion group participants at five locations also discussed how achieving a rating higher than a "3" seemed to be an unattainable goal. For example, employees at four locations told us that they felt NSPS either does not provide incentives for high performance or encourages only mediocre performance from employees under the system because of the high number of employees receiving "3"-level ratings each year. As another example, supervisors at one location noted that across the installation there is a general feeling that everyone receives a rating of "3," and therefore such a rating is considered average, no matter how DOD defines it. Similarly, discussion group participants at seven locations told us that they felt it was difficult for employees in certain positions to receive a rating of "5" because of the nature of their work or the perceived value their management placed on those positions. At one of those locations, supervisors told us that they felt such things as how the pay pool's business rules were structured affected whether an employee could receive a high rating. At that location, the pay pool's business rules specified that an employee must receive a higher-level award, such as a command or agencywide award, to receive a rating of "5." However, discussion group participants told us that they felt some employees were not in a position to receive such awards because of their positions or the type of work they did. In addition, discussion group participants at all eight locations we visited expressed frustration over the amount of the annual performance payout provided under NSPS. For example, discussion group participants noted that they felt the payout was not significant enough to encourage anything other than average performance. Discussion group participants at six of the eight locations also told us that they felt NSPS discourages employees from seeking additional responsibilities and opportunities that fall outside the scope of their objectives because their payout may not reflect their additional work. In addition, discussion group participants at six locations told us that because supervisory positions under NSPS require such a significant increase in responsibility and effort, and because the maximum allowable pay increase for reassignments is capped at 5 percent,[Footnote 42] some employees may not seek promotion opportunities. Similarly, a discussion group participant at another location expressed frustration that some employees only received their payout in the form of a bonus and not an increase in salary. Discussion group participants at three of the eight locations also expressed concerns that they felt performance payouts under NSPS tended to benefit higher-paid employees at the expense of lower-paid employees. For example, employees at one location expressed concerns that in their pay pool, the higher payouts under NSPS seemed to go to employees at the top of the pay bands. Concerns about an Excessive Amount of Time Spent Navigating the NSPS Performance Management Process: Another prevalent theme at seven of the eight locations and also highlighted in our first assessment was that employees spend an excessive amount of time navigating the performance management process. While the discussion group participants' complaints about the time-and labor-intensive nature of the system were not limited to any one part of the process, discussion group participants at seven locations pointed out that the time and effort required to complete the steps of the NSPS performance management process were significantly greater than what was required of them under previous systems. For example, one supervisor we spoke with speculated that his supervisory duties under NSPS took him six times as long to perform as they had under the GS system, while another supervisor told us that he may have spent from 45 to 50 hours assessing the performance of three employees, a task he could have completed in 10 hours under the GS system. At five of the locations we visited, employees expressed concerns about NSPS potentially affecting their ability to complete their jobs or affecting the mission because of what they perceived as an excessive amount of time required of employees and supervisors in navigating the NSPS performance management process. In some instances, employees spoke of impacts NSPS was having on their supervisors, while others spoke of their own experiences navigating the NSPS performance management process. At three locations, discussion group participants described how what they perceived as an excessive amount of time navigating the NSPS performance management process affected their ability to complete their job-related duties, requiring completion of some NSPS tasks, such as self-assessments and employee ratings, after work hours or on weekends. One employee described feeling inundated with information on NSPS and that it was difficult to stay on top of things while simultaneously performing his job, while another employee estimated that she spent about 2 hours per week on NSPS-related tasks. In some instances, discussion group participants told us that they saw the potential for the excessive time commitment required by NSPS to affect the missions of their organizations. According to one supervisor, any task that takes employees away from their daily work affects the mission, and any task that takes the time and patience of the command's leadership detracts from the mission. Further, in discussing during a site visit the potential for NSPS to impact the organization's mission, one general officer we spoke with described NSPS as "mission ineffective." Concerns about Challenges with Job Objectives under NSPS: Another prevalent theme that emerged from our discussions with both supervisors and employees at all eight locations was that there are challenges with employee job objectives under NSPS. According to DOD, the NSPS performance management system is designed to provide a fair and equitable method for appraising and evaluating performance. As part of the system, DOD established the concept of "job objectives," which are the required tasks of a given job as determined by managers and supervisors, and directed that job objectives be developed and used as the standards for evaluating employee performance. However, supervisors and employees at each of the eight locations discussed challenges they experienced developing their job objectives under NSPS. Specifically: * Although DOD guidance encourages employees to develop job objectives that are specific, measurable, aligned, realistic, and timed--an approach summarized by the acronym S.M.A.R.T.--employees and supervisors we met at six of eight locations discussed how they found it challenging to develop job objectives that are measurable or that follow the S.M.A.R.T. approach. Supervisors at one location objected to the S.M.A.R.T. approach, particularly the "specific" portion, because they felt that job objectives needed to be broad enough to allow employees to discuss any accomplishments they make if they complete additional job activities or other tasks that might arise during the year. Supervisors at two locations discussed how the work they did was nebulous and unpredictable, which made it challenging to develop job objectives that not only reflected the nature of their work but that they could exceed. Similarly, supervisors at another location expressed concerns that employees' job objectives may not reflect the work they do by the end of the performance management cycle because of constant changes within their organization. * According to discussion group participants at four locations, guidance for developing job objectives is either limited or nonexistent, which may result in different approaches to developing job objectives across an organization. At one of these locations, employees told us that their management had not established consistent ground rules for developing job objectives and that as a result some employees' job objectives were based on out-of-date position descriptions. One organization we visited used a mixture of mandatory and employee-specific job objectives; but, according to one employee, little guidance exists to help employees and supervisors when they need to develop personalized job objectives. Employees at another location told us that there were significant differences in the amount of involvement they had in developing their job objectives. For example, one individual told us that employees in her office develop their own objectives, while another said employees in her office are assigned mandatory objectives and were thus unable to provide input into their objectives. * Discussion group participants at six locations expressed concerns that it can be difficult to achieve a high rating for some job objectives. Some locations we visited used mandatory job objectives, which left employees concerned that their job objectives did not accurately capture the full responsibilities of the work they performed. For example, at one location, a uniform, mandatory supervisory objective accounted for half of supervisors' ratings, which, according to one supervisor, diminished the value of the other responsibilities they had. The supervisor expressed further concern that some mandatory job objectives, such as those assigned to government purchase card holders, require a pass-fail evaluation, making it difficult, if not impossible, for the employee to receive a high rating. In one instance, a location we visited required all employees to be rated against a mandatory safety objective. However, according to some supervisors, it did not make sense for everyone to have the mandatory safety objective because for many employees, safety issues were out of their control. Concerns about Factors That Undermine Employee Confidence in the System: During our discussion groups, participants at all eight locations also discussed how various factors undermine employees' confidence in the system and its implementation thus far. For example, discussion group participants at six locations commented that they do not believe that the NSPS performance management system has the ability to rate employees fairly. At the locations we visited, discussion of these concerns centered on such things as the perception of subjectivity and the potential for favoritism under NSPS; a lack of transparency surrounding the pay pool panel process, including a lack of understanding of what employees needed to do to receive higher ratings; and the perception of inconsistencies in interpretations of the standards used to determine employee ratings. One prevalent theme at all eight locations involved perceptions of subjectivity, such as the potential for favoritism under NSPS during the rating and pay pool panel processes. At five locations, participants discussed their frustration with how NSPS takes the responsibility for rating employees out of the hands of supervisors and places it in the hands of the pay pool panel members, who may or may not have any direct knowledge of individual employees' performance. One supervisor told us that NSPS may inadvertently favor employees who work closely or are in direct contact with members of the pay pool panel because those individuals have direct knowledge of the employees and, sometimes, their performance. Similarly, supervisors at another location told us that they did not feel that their pay pool panel understood their jobs and what they do and expressed frustration that the pay pool panel did not seem to be reaching out to their supervisors and higher-level reviewers for additional input on their performance. At five of the eight locations, discussion group participants also told us that they saw the potential for the employee-supervisor relationship to affect an employee's rating--either to the benefit or detriment of the employee. Another prevalent theme at six of the eight locations--a theme also highlighted in our first assessment of NSPS--was a lack of transparency and understanding of the pay pool panel process. Specifically, supervisors at two locations commented that their organizations' pay pool panel processes were not transparent. A supervisor at one location commented that everything "goes into a black vacuum," while another likened the process to a "black box." Employees at that same location referred to the organization's pay pool panel process as a "star chamber," where decisions are made but are not explained to employees. Employees and supervisors at five locations expressed concerns about the amount of information they received from their pay pools and about the process itself; some desired further information to help them better understand the pay pool panel process. In addition, at six of the eight locations, discussion group participants told us that they did not understand what they needed to do to receive a higher rating. For example, an employee at one location told us that she was told by her supervisor that all employees had to receive a rating of "3" because they would have had to "walk on water" to receive a higher rating. Discussion group participants at two other locations also discussed how "walking on water" was a perceived standard for receiving a high rating under NSPS. At three locations, supervisors commented that they were unclear about what they could do to help their employees receive better ratings, while employees at four locations were unclear about what they could do to achieve higher ratings. Discussion group participants at six locations also raised concerns about inconsistent interpretation of the standards used when evaluating civilian employees under NSPS. Discussion group participants reported concerns that military supervisors may rate employees using more stringent standards than their civilian counterparts. Discussion group participants also reported concerns that some military supervisors may not value the NSPS performance management process and sometimes devote less time and effort to the process, which could affect employees' ratings. One civilian supervisor told us that some military supervisors with whom he attended NSPS training had a much harsher perspective of employee performance than their civilian counterparts. For example, he noted that the military supervisors indicated that giving a rating of "1" or "2" was acceptable, whereas he believed civilian supervisors would be more inclined to give an employee a rating of "3." Employees also told us that they do not believe some military supervisors value the work of employees who perform certain job functions, such as providing child care on an installation. Concerns about Factors Unrelated to Performance Affecting Final Ratings: A prevalent theme expressed by discussion group participants at all eight locations we visited is that factors unrelated to performance may affect employees' final performance ratings. Such factors include the existence of a forced distribution or quota of ratings, the writing ability of employees and supervisors, and pay pool panel members' knowledge of employees. For example: * Discussion group participants at all eight locations expressed concerns that their pay pool panels used a forced distribution or quota for ratings, which dictated the number of ratings in each category that could be awarded.[Footnote 43] Employees at one location told us that they were aware of their management's attempts to artificially preserve a higher share value for employees by primarily awarding ratings of "3," regardless of the employees' performance. Further, at three locations discussion group participants told us that their management told them that all employees should expect to receive a rating of "3." Moreover, some discussion group participants told us that they doubted that their actual performance had the bearing it was supposed to have on their final ratings, while others felt the use of a forced distribution or quotas was in direct conflict with the principles of pay for performance under NSPS. While no discussion group participants we met with were aware of any explicit guidance provided to pay pool panels or supervisors that limited the number of certain ratings they assigned employees, employees and supervisors from at least three locations believed that informal guidelines existed or that pay pool panels or supervisors were encouraged to limit the number of certain ratings they could assign. * Discussion group participants at all eight locations also expressed concerns that the writing ability of employees and supervisors may affect ratings--a theme also highlighted in our first assessment of NSPS. Supervisors at one location likened the process of developing employees' assessments under NSPS to a writing contest. Moreover, supervisors told us that they felt their writing ability could unintentionally affect their employees' ratings, noting, for example, that a supervisor's ability to articulate an employee's achievements in writing plays a significant role in supporting a higher rating for that employee. Employees shared the supervisors' concerns, noting that they believed that succeeding under NSPS depended on the quality of their written assessments, rather than their job performance, and that their ratings could suffer if their supervisors did not provide the pay pool panel with well-written assessments. In discussing the potential influence that employees' and supervisors' writing skills may have on a pay pool panel's assessment of an employee, officials at seven of the eight locations acknowledged that in some instances writing skills had affected employees' ratings and could overshadow employees' performance. * Discussion group participants at seven locations also expressed frustration that employee ratings were potentially affected by the extent to which pay pool panel members have personal knowledge of employees or understand the nature of their work in general. Some discussion group participants felt that pay pool panel members' personal knowledge of employees helped some employees receive higher ratings, while others told us that they felt that members of the pay pool panel were too far removed from them and lacked direct knowledge of the work they performed. One employee believed that individuals who were involved in implementing NSPS worked closely with pay pool panel members, or were senior managers who were more likely to receive higher ratings under NSPS than others. Other employees told us that they were concerned about the potential for pay pool panel members to advocate in some way for employees they personally know--for example, by encouraging the pay pool panel to contact a specific employee's supervisor to seek additional information or justification for a rating. As a result, they felt that pay pool panel members' personal knowledge of employees could benefit some employees, but not others. DOD Has Not Yet Developed a Comprehensive Action Plan to Address Employees' Perceptions of the System: In our first assessment of NSPS, we recommended that DOD develop and implement a specific action plan to address employees' perceptions of NSPS, based on guidance published by OPM for conducting annual employee surveys and providing feedback to employees on the results.[Footnote 44] The guidance suggests that after an agency's survey results have been reviewed, the agency has a responsibility to provide feedback to employees on the results of the survey, as well as to let employees know the intended actions to address the results and the progress made on these actions. Further, the guidance suggests that agencies consider the following when developing action plans: * the resources required; * who will be responsible for taking action; * who will be responsible for providing oversight; * if the individuals taking the action have the necessary authority to make things happen; * what coordination, if any, is required, and how it will be accomplished; and: * how agencies will adjust given any changes or delays in their actions. Since then, in June 2009, the PEO issued a departmentwide memorandum entitled "Addressing Key NSPS Workforce Concerns"; however, issuance of this memorandum does not fully meet the intent of our 2008 recommendation. Specifically, the PEO's June 2009 memorandum summarizes key concerns from the department's 2008 evaluation of NSPS, summarizes departmentwide actions that had been taken to date to address employees' concerns about the system, and suggests approaches to enhance local efforts to address workforce concerns. The PEO identified five key areas of concern, which are similar to those identified in our own discussion group sessions with DOD employees and supervisors: (1) performance communication and feedback, (2) understanding of performance management and the pay pool process, (3) trust in the system and its processes, (4) training and information, and (5) the amount of time needed to fulfill performance management responsibilities. The PEO's memorandum urged the components to leverage information from the department's 2008 evaluation of NSPS and focus on the five areas discussed above as they plan their own actions. Further, the PEO's memorandum noted that DOD has taken some steps to address employees' concerns about NSPS--for example, developing and fielding a pay pool training course for employees and rating officials, modifying its implementing issuances to require all performance review authorities to review pay pool panel results on an annual basis, and providing guidance to employees on the prohibition against the forced distribution of ratings. Issuance of the PEO's memorandum represents an important first step. However, because the memorandum does not specify actions the department intends to take, who will be responsible for taking the action, and timelines for addressing areas where employees express negative perceptions of the system, it does not fully meet the intent of our 2008 recommendation. In developing an action plan, we note that OPM recently issued guidance that agencies can use in developing action plans for improving employee satisfaction.[Footnote 45] According to OPM, action plans should clearly (1) state the objectives, (2) identify actions to be taken, (3) provide outcome measures and improvement targets, and (4) describe how progress will be tracked. In addition to identifying the specific actions that will be taken to achieve improvements, OPM's guidance also suggests that agencies specify: * time frames for accomplishing the actions, * who will be responsible for implementing the actions, * who will be affected by the actions, * the resources required, and: * a plan to communicate these actions to managers and employees. We continue to believe that developing and implementing a plan to address employees' perceptions of NSPS could help DOD make changes to the system that could lead to greater employee acceptance and, ultimately, the system's successful implementation. Further, we note that having such a plan is an approach that DOD could take to involve employees in the system's implementation--which is one of the safeguards we previously discussed. Conclusions: As we noted in our first assessment, DOD's implementation of NSPS placed the department at the forefront of a significant transition facing the federal government. However, toward the end of this review, the future of NSPS became uncertain, given the proposed legislation that, if enacted, would terminate the system and require any future system created by DOD to use safeguards similar to those discussed in our report, including ensuring employee involvement in the system and providing adequate training and retraining. In light of the contingent nature surrounding NSPS and the possibility of implementing a different system, sustained and committed leadership will be imperative to provide focused attention necessary to implement any pay-for- performance system within DOD. Key to implementing a fair, effective, and credible system is including safeguards early on in the design of the system. Since we issued our first assessment of NSPS in 2008, we note that DOD has continued to take steps to meet the intent of each of the safeguards. However, with this latest assessment, we note that the department has not implemented the safeguards systematically; for example, it has not ensured that the training provided to employees on the system's operations is effective. Further, DOD has not monitored how the safeguards specifically are implemented by lower-level organizations across the department. As a result, decision makers in DOD lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. Additionally, while DOD has gained experience operating under NSPS, at the time of our review it had not yet developed an action plan for addressing employees' perceptions of the system, as we recommended in 2008. As DOD moves forward with implementing a pay-for-performance system--whether NSPS or another--we believe that it is important for the department to improve upon its implementation of the safeguards and address employees' concerns. Left unchecked, these issues could undermine any future human capital reform efforts within DOD. Recommendations for Executive Action: To help implement a fair, effective, and credible performance management system for its civilian employees--whether NSPS or another- -we recommend that the Secretary of Defense take the following three actions: * Review and evaluate the effectiveness of the department's training. * Ensure that guidance is in place for conducting a postdecisional analysis that specifies what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. * Include, as part of the department's monitoring of the implementation of its system, efforts to monitor and evaluate how the safeguards specifically are implemented by lower-level organizations across the department. Agency Comments and Our Evaluation: In September 2009, we provided DOD with a draft of this report that included three recommendations to better address the safeguards and improve implementation of the NSPS performance management system. Specifically, we recommended that DOD (1) evaluate NSPS training, (2) review and revise its guidance for conducting postdecisional analysis of NSPS ratings, and (3) monitor how the safeguards specifically are implemented. In commenting on a draft of our report, DOD partially concurred with our three recommendations. DOD's comments are reprinted in appendix III. DOD partially concurred with our recommendations, noting the expectation that the Congress would require the department to terminate NSPS by January 1, 2012, and this action, in turn, would require the department to focus on drawing down NSPS in an orderly manner. DOD further stated that it would consider acting on our recommendations to the extent they are relevant as the department moves forward with any future performance management system. We believe that this is a reasonable approach. As discussed above, we recognize the contingent nature surrounding NSPS as a result of provisions in the proposed National Defense Authorization Act for Fiscal Year 2010, which recently passed both Houses of Congress. Accordingly, we revised our recommendations to apply to any future performance management system for the department's civilian employees--whether NSPS or another system. However, we also note that provisions of the proposed legislation would require DOD to implement certain safeguards and issue regulations for that system to provide a fair, credible and transparent performance appraisal system. We therefore continue to believe that our recommendations have merit. We are sending copies of this report to the appropriate congressional committees. We will make copies available to others upon request. The report also is available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to the report are listed in appendix IV. Brenda S. Farrell Director, Defense Capabilities and Management: List of Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Joseph I. Lieberman: Chairman: The Honorable Susan M. Collins: Ranking Member: Committee on Homeland Security and Governmental Affairs: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Howard P. McKeon: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable Edolphus Towns: Chairman: The Honorable Darrell Issa: Ranking Member: Committee on Oversight and Government Reform: House of Representatives: [End of section] Appendix I: Scope and Methodology: As with our first assessment of the National Security Personnel System (NSPS) in 2008,[Footnote 46] we limited our scope in conducting this review to the performance management aspect of NSPS. Therefore, we addressed neither performance management of the Senior Executive Service at the Department of Defense (DOD) nor other aspects of NSPS, such as classification and pay. Determining Implementation of Safeguards and Monitoring Their Implementation: To determine the extent to which DOD has implemented safeguards as part of the NSPS performance management system and monitored the implementation of the safeguards, we used the following safeguards, which we also reported on in our 2008 review: * Involve employees, their representatives, and other stakeholders in the design of the system, to include employees directly involved in validating any related implementation of the system. * Assure that the agency's performance management systems link employee objectives to the agency's strategic plan, related goals, and desired outcomes. * Provide adequate training and retraining for supervisors, managers, and employees in the implementation and operation of the performance management system. * Provide a process for ensuring ongoing performance feedback and dialogue between supervisors, managers, and employees throughout the appraisal period, and for setting timetables for review. * Implement a pay-for-performance evaluation system to better link individual pay to performance, and provide an equitable method for appraising and compensating employees. * Assure that certain predecisional internal safeguards exist to help achieve consistency, equity, nondiscrimination, and nonpoliticization of the performance management process (e.g., independent reasonableness reviews by a third party or reviews of performance rating decisions, pay determinations, and promotions before they are finalized to ensure that they are merit-based, as well as pay panels who consider the results of the performance appraisal process and other information in connection with final pay decisions). * Assure that there are reasonable transparency and appropriate accountability mechanisms in connection with the results of the performance management process, to include reporting periodically on internal assessments and employee survey results relating to performance management and individual pay decisions while protecting individual confidentiality. * Assure that performance management results in meaningful distinctions in individual employee performance. * Provide a means for ensuring that adequate agency resources are allocated for the design, implementation, and administration of the performance management system. To assess implementation of the safeguards, we reviewed the legislative requirements and obtained and analyzed regulations and other guidance for implementing the NSPS performance management system. We also obtained and analyzed other documents, such as DOD's rating results and reconsideration statistics, for the 2007 and 2008 NSPS performance management cycles.[Footnote 47] We also interviewed knowledgeable officials in DOD's NSPS Program Executive Office and the NSPS program offices of the four components--the Army, the Air Force, the Navy, and the Fourth Estate[Footnote 48]--to obtain a comprehensive understanding of their efforts to implement NSPS and each of the safeguards, as well as the processes, procedures, and controls used for monitoring and overseeing implementation of the system. In addition, we conducted site visits to select organizations located outside the continental United States to assess implementation of the safeguards. To allow for appropriate representation by each component, we visited two organizations per component, or eight organizations in total. The organizations we visited were selected based on a number of factors, such as the presence of a large number or concentrated group of civilian employees under NSPS and, when possible, the presence of employees who had converted to NSPS under both spirals 1 and 2. We focused our efforts for this review on visiting organizations located outside the continental United States because our 2008 review focused on assessing implementation of NSPS and the safeguards at locations that were geographically distributed throughout the United States. We elected to focus our site visits in Germany and Hawaii because of the civilian employees located outside the continental United States who had converted to NSPS at the time we initiated our review, more than half were located in either Germany or Hawaii.[Footnote 49] Also, we wanted to determine whether civilian employees located outside the continental United States were experiencing any unique problems or challenges with the system. In Germany, the organizations we visited were the 5th Signal Command; the 435th Air Base Wing; the Defense Finance and Accounting Service; and the George C. Marshall European Center for Security Studies, part of the Defense Security Cooperation Agency. In Hawaii, the organizations we visited were the Commander, Navy Region Hawaii; Headquarters, Pacific Air Force; the Naval Facilities Engineering Command, Hawaii; and the U.S. Army Corps of Engineers, Honolulu District. For each of the organizations we visited, we met with or interviewed the performance review authority, pay pool managers, pay pool panel members, rating officials, and the NSPS program manager or transition team, among others, to discuss the steps they have taken to implement the safeguards or otherwise ensure the fairness, effectiveness, and credibility of NSPS. To assess the organizations' implementation of the safeguards, we compared and contrasted the information obtained during our interviews and supplemented this testimonial evidence with the other relevant documentation we obtained, such as the organizations' pay pool business rules,[Footnote 50] lessons learned, and training materials. Determining DOD Civilian Employees' Perceptions of NSPS: To determine how DOD civilian employees perceive NSPS, we analyzed two sources of employee perceptions or attitudes. First, we analyzed the results of DOD's survey of civilian employees to identify employee perceptions of NSPS and examine whether and how these perceptions may be changing over time. Second, we conducted small group discussions with civilian employees who had converted to NSPS and administered a short questionnaire to the participants at each of the eight organizations we visited. As with our first assessment of NSPS, our overall objective in using the discussion group approach was to obtain employees' perceptions about NSPS and its implementation thus far. Analysis of DOD Survey Results: We analyzed the results of the Defense Manpower Data Center's (DMDC) Status of Forces Survey[Footnote 51] of Civilian Employees--including the May 2006, November 2006, May 2007, and February 2008 administrations--to gauge employee attitudes toward NSPS and performance management in general and identify indications of movement or trends in employee perceptions.[Footnote 52] As we reported in September 2008, we have reviewed the results of prior administrations of DMDC surveys and found the survey results, including the results of the Status of Forces Survey of Civilian Employees, sufficiently reliable to use for several GAO engagements. However, to understand the nature of any changes that were made to its survey methods for administering the survey for 2008 as compared with previous administrations, we also received responses to written questions from and discussed these data with officials at DMDC. Based on these responses and discussions, we determined that DMDC's survey data remain sufficiently reliable for the purpose of our reports on DOD civilian employees' perceptions of NSPS.[Footnote 53] GAO's Discussion Groups with DOD Civilian Employees under NSPS: We also conducted small group discussions with DOD civilian employees and administered a short questionnaire during site visits in February and March 2009. Specifically, we conducted two discussion groups--one with nonsupervisory employees and another with supervisory employees-- at each of the eight locations we visited, for a total of 16 discussion groups. As with our first assessment of NSPS in 2008, our objective in using this approach was to obtain employees' perceptions about NSPS and its implementation thus far because discussion groups are intended to provide in-depth information about participants' reasons for holding certain attitudes about specific topics and to offer insights into the range of concerns about and support for an issue. Further, in conducting our discussion groups, our intent was to achieve saturation- -the point at which we were no longer hearing new information. As we previously reported, our discussion groups were not designed to (1) demonstrate the extent of a problem or to generalize the results to a larger population, (2) develop a consensus to arrive at an agreed- upon plan or make decisions about what actions to take, or (3) provide statistically representative samples or reliable quantitative estimates. Instead, our discussion groups provide in-depth information about participants' reasons for holding certain attitudes about specific topics and offer insights into the range of concerns about and support for an issue. Although the results of our discussion sessions are not generalizable to the entire NSPS civilian population, the composition of our discussion groups was designed to ensure that we spoke with employees from each of the four components at locations outside the continental United States. Because supervisory and nonsupervisory employees have distinct roles with respect to NSPS, we held separate discussion sessions for these groups. To select the discussion group participants, we requested that the organizations we visited provide us with lists of employees who had converted to NSPS. From the lists provided, we selected participants based on their supervisory and nonsupervisory status. To ensure maximum participation of the selected employees, we randomly selected up to 20 participants from each group with the goal of meeting with 8 to 12 individuals in each discussion group and provided the employee names and a standard invitation to GAO's points of contact to disseminate to the employees. At the majority of locations, we reached our goal of meeting with 8 to 12 individuals in each discussion group; however, since participation was not compulsory, in two instances we did not reach our goal of 8 participants per discussion group. Table 6 provides information on the composition of our discussion groups. Table 6: Composition of Discussion Groups: Organization: Army: 5th Signal Command, Funari Barracks, Germany; Discussion group participants: Nonsupervisory employees: 11; Discussion group participants: Supervisory employees: 10; Total discussion group participants: 21; Total NSPS employees assigned to the pay pool visited[A]: 557. Organization: Army: U.S. Army Corps of Engineers, Honolulu District, Fort Shafter, Hawaii; Discussion group participants: Nonsupervisory employees: 7; Discussion group participants: Supervisory employees: 9; Total discussion group participants: 16; Total NSPS employees assigned to the pay pool visited[A]: 246. Organization: Air Force: 435th Air Base Wing, Ramstein Air Base, Germany; Discussion group participants: Nonsupervisory employees: 13; Discussion group participants: Supervisory employees: 9; Total discussion group participants: 22; Total NSPS employees assigned to the pay pool visited[A]: 314. Organization: Air Force: Headquarters, Pacific Air Force, Hickam Air Force Base, Hawaii; Discussion group participants: Nonsupervisory employees: 9; Discussion group participants: Supervisory employees: 9; Total discussion group participants: 18; Total NSPS employees assigned to the pay pool visited[A]: 100. Organization: Navy: Commander, Navy Region Hawaii, Pearl Harbor, Hawaii; Discussion group participants: Nonsupervisory employees: 12; Discussion group participants: Supervisory employees: 15; Total discussion group participants: 27; Total NSPS employees assigned to the pay pool visited[A]: 320. Organization: Navy: Naval Facilities Engineering Command, Hawaii, Pearl Harbor, Hawaii; Discussion group participants: Nonsupervisory employees: 11; Discussion group participants: Supervisory employees: 10; Total discussion group participants: 21; Total NSPS employees assigned to the pay pool visited[A]: 97. Organization: Fourth Estate: Defense Finance and Accounting Service, Kaiserslautern, Germany; Discussion group participants: Nonsupervisory employees: 12; Discussion group participants: Supervisory employees: 4; Total discussion group participants: 16; Total NSPS employees assigned to the pay pool visited[A]: 1,497. Organization: Fourth Estate: George C. Marshall European Center for Security Studies, Garmisch-Partenkirchen, Germany; Discussion group participants: Nonsupervisory employees: 10; Discussion group participants: Supervisory employees: 13; Total discussion group participants: 23; Total NSPS employees assigned to the pay pool visited[A]: 434. Organization: Total; Discussion group participants: Nonsupervisory employees: 85; Discussion group participants: Supervisory employees: 79; Total discussion group participants: 164; Total NSPS employees assigned to the pay pool visited[A]: 3,565. Source: GAO (analysis) and DOD (number of employees assigned to the respective pay pool). [A] The totals listed include the number of civilian employees each organization rated during the 2008 NSPS performance management cycle. [End of table] To facilitate our discussion groups, we developed a discussion guide to help the moderator in addressing several topics related to employees' perceptions of the NSPS performance management system. These topics include employees' overall perception of NSPS and the rating process, the training they received on NSPS, the communication they have had with their supervisors, positive aspects they perceive of NSPS, and any changes they would make to the system, among others. Each discussion group was scheduled for a 2-hour period and began with the GAO moderator greeting the participants, describing the purpose of the study, and explaining the procedures for the discussion group. Participants were assured that all of their comments would be discussed in the aggregate or as part of larger themes that emerged. The GAO moderator asked participants open-ended questions related to NSPS, while at least one other GAO analyst observed the discussion group and took notes. Following the conclusion of all our discussion group sessions, we performed content analysis of the sessions in order to identify the themes that emerged and to summarize the participants' perceptions of NSPS. We reviewed responses from several of the discussion groups and created a list of themes and subtheme categories. We then reviewed the comments from each of the 16 discussion groups and assigned each comment to the appropriate category, which was agreed upon by two analysts. If agreement was not reached on a comment's placement in a category, another analyst reconciled the issue by placing the comment in either one or more of the categories. The responses in each category were then used in our evaluation and discussion of how civilian employees perceive NSPS. Following each discussion group we administered a questionnaire to the participants to obtain further information on their background, tenure with the federal government and DOD, and attitudes toward NSPS. We received questionnaires from 164 discussion group participants. In addition to collecting demographic data from participants for the purpose of reporting with whom we spoke (see table 7), the purpose of our questionnaire was to (1) collect information from participants that could not easily be obtained through discussion, for example, information participants may have been uncomfortable sharing in a group setting, and (2) collect some of the same data found in past DOD surveys. Specifically, the questionnaire included questions designed to obtain employees' perceptions of NSPS as compared to their previous personnel system, the accuracy with which they felt their ratings reflected their performance, and management's methods for conveying overall rating information. Since the questionnaire was used to collect supplemental information and was administered solely to the participants of our discussion groups, the results represent the opinions of only those employees and cannot be projected across DOD, a component, or any single pay pool we visited. Table 7: Composition of Discussion Groups by Demographic Category per Component: Category: Male; Component: Army: 22; Component: Air Force: 25; Component: Navy: 21; Component: Fourth Estate: 17; Component: Total: 85. Category: Female; Component: Army: 15; Component: Air Force: 15; Component: Navy: 27; Component: Fourth Estate: 22; Component: Total: 79. Category: Total; Component: Army: 37; Component: Air Force: 40; Component: Navy: 48; Component: Fourth Estate: 39; Component: Total: 164. Category: American Indian or Alaskan Native; Component: Army: 0; Component: Air Force: 1; Component: Navy: 0; Component: Fourth Estate: 0; Component: Total: 1. Category: Asian; Component: Army: 14; Component: Air Force: 4; Component: Navy: 21; Component: Fourth Estate: 1; Component: Total: 40. Category: Black/African American; Component: Army: 4; Component: Air Force: 2; Component: Navy: 1; Component: Fourth Estate: 4; Component: Total: 11. Category: Hispanic; Component: Army: 0; Component: Air Force: 1; Component: Navy: 2; Component: Fourth Estate: 3; Component: Total: 6. Category: Native Hawaiian or other Pacific Islander; Component: Army: 0; Component: Air Force: 0; Component: Navy: 7; Component: Fourth Estate: 0; Component: Total: 7. Category: White; Component: Army: 16; Component: Air Force: 28; Component: Navy: 14; Component: Fourth Estate: 31; Component: Total: 89. Category: Other; Component: Army: 3; Component: Air Force: 4; Component: Navy: 3; Component: Fourth Estate: 0; Component: Total: 10. Category: Total; Component: Army: 37; Component: Air Force: 40; Component: Navy: 48; Component: Fourth Estate: 39; Component: Total: 164. Source: GAO. Note: Participants voluntarily self-reported demographic information in our questionnaire. [End of table] We visited or contacted the following organizations during our review: Department of Defense: * Defense Finance and Accounting Service, Indianapolis, Indiana; Rome, New York; Columbus, Ohio; and Kaiserslautern, Germany: * Defense Manpower Data Center, Arlington, Virginia: * Defense Security Cooperation Agency, Arlington, Virginia: - George C. Marshall European Center for Security Studies, Garmisch- Partenkirchen, Germany: * Equal Employment Opportunity Office, Arlington, Virginia: * Fourth Estate NSPS Program Management Office, Arlington, Virginia: * NSPS Program Executive Office, Arlington, Virginia: Department of the Army: * 5th Signal Command, Funari Barracks, Germany: * Civilian Personnel Evaluation Agency, Alexandria, Virginia: * NSPS Program Management Office, Alexandria, Virginia: * U.S. Army Corps of Engineers: - Headquarters, Washington, D.C. - Honolulu District, Fort Shafter, Hawaii: - Pacific Ocean Division, Fort Shafter, Hawaii: Department of the Air Force: * 435th Air Base Wing, Ramstein Air Base, Germany: * Headquarters, Pacific Air Force, Hickam Air Force Base, Hawaii: * NSPS Program Office, Arlington, Virginia: Department of the Navy: * Commander, Navy Region Hawaii, Pearl Harbor, Hawaii: * Naval Facilities Engineering Command, Hawaii, Pearl Harbor, Hawaii: * NSPS Program Office, Navy Yard, Washington, D.C. * Office of Civilian Human Resources, Navy Yard, Washington, D.C. We conducted this performance audit from November 2008 through September 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: Reconsideration Process: Department of Defense (DOD) civilian employees who receive performance ratings under the National Security Personnel System (NSPS) have the option of challenging their ratings through a formal process known as reconsideration. The reconsideration process is the sole and exclusive agency administrative process for nonbargaining unit employees to challenge their ratings.[Footnote 54] However, DOD's NSPS regulations also allow for alternative dispute resolution techniques, such as mediation or interest-based problem solving, to be pursued at any time during the reconsideration process consistent with component policies and procedures. Under the reconsideration process, employees may challenge their ratings of record or individual job objective ratings; employees cannot challenge their performance payout, the number of shares assigned, the share value, or the distribution of their performance payout between salary increase and bonus, nor can they challenge their recommended ratings of record, interim reviews, or applicable closeout assessments. In addition, employees who allege that their performance ratings are based on prohibited discrimination or reprisal may not use the reconsideration process; rather, such allegations are to be processed through the department's equal employment opportunity discrimination complaint procedure. Employees who wish to challenge their rating have 10 calendar days from the receipt of their ratings of record to submit written requests for reconsideration to their pay pool managers. Within 15 calendar days of the pay pool manager's receipt of an employee's request for reconsideration, the pay pool manager must render a written decision that includes a brief explanation of the basis of the decision. The pay pool manager's decision is final, unless the employee seeks further reconsideration from the performance review authority. Specifically, if the employee is dissatisfied with the pay pool manager's decision, or if none is provided within the prescribed time frames, the employee may submit a written request for final review by the performance review authority or his or her designee. This request must be submitted within 5 calendar days of receipt of the pay pool manager's decision or within 5 calendar days of the date the decision should have been rendered. The performance review authority then is allotted 15 calendar days from receipt of the written request from the employee to make a decision, which is final. If the final decision is to change the rating of record or job objective rating, the revised rating takes the place of the original one, and a revised performance appraisal is prepared for the employee. According to DOD's 2008 evaluation report, for the 2007 NSPS performance management cycle,[Footnote 55] 2,302 civilian employees out of the 100,465 employees who were rated under NSPS elected to file a request for reconsideration of their ratings, and of these, about 33 percent of the requests were granted. For the 2008 NSPS performance management cycle,[Footnote 56] according to the NSPS Program Executive Office, as of June 2009, 4,296 civilian employees out of the 170,149 employees who were rated under NSPS elected to file requests for reconsideration of their ratings, and of these, about 52 percent of the requests were granted.[Footnote 57] [End of section] Appendix III: Comments from the Department of Defense: Department Of Defense: NSPS: National Security Personnel System: Program Executive Office: 1400 Key Boulevard Suite B200: Arlington, VA 22209-5144: October 23, 2009: Ms. Brenda S. Farrell: Director, Defense Capabilities and Management: U.S. Government Accountability Office: 441 G Street, N.W.: Washington, DC 20548: Dear Ms. Farrell: This is the Department of Defense (DoD) response to the GAO draft report, "Human Capital: Continued Monitoring of Safeguards Is Needed and Opportunities Exist to Address Employee Concerns about the National Security Personnel System" dated September 21, 2009 (GAO Code 351277/GA0-10-102). We thank you for the opportunity to review and comment. Our own internal program evaluation work and other feedback have pointed us to many of the same issues that your report cites. But for our expectation that Congress will require the Department to terminate the National Security Personnel System (NSPS) by January 1, 2012, we would concur in your basic recommendations. They align with actions we have said we would take as NSPS becomes a more fully mature human resources system. As you report, the Department has listened to the workforce and made adjustments since first implementing Spiral 1.1. For example, we modified system rules to increase accountability at senior levels, added NSPS training and reference material that individuals could access for themselves on line, and improved appraisal tools. Now that most organizations have had at least two years of experience with NSPS, this year would have been the time to see if the fundamental safeguards for fair, objective, rigorous ratings (e.g., the multiple levels of review and standard rating criteria) were sound or created their own, undesirable complications for us to correct. Congress' mandate will require that we focus on drawing down NSPS in an orderly way. We therefore will consider acting on your three recommendations to the extent they are relevant as the Department moves forward with any new systems. We appreciate your staff's efforts to review and report on NSPS safeguards, with sensitivity to the cultural change involved and the time and evolutionary actions needed to complete such a transformation. As we noted last year, it is an enormous challenge to ensure a performance management system is rigorous and consequential as well as fair, transparent, and well accepted. Sincerely, Signed by: Brad Bunn: Program Executive Officer: Enclosure: GAO Draft Report Dated September 21, 2009: GAO CODE 351277/GA0-10-102 "Human Capital: Continued Monitoring of Safeguards Is Needed and Opportunities Exist to Address Employee Concerns about the National Security Personnel System" Department Of Defense Responses To Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense direct the National Security Personnel System (NSPS) Senior Executive to review and evaluate the Department's NSPS training to enhance the effectiveness of the training. DOD RESPONSE: Partially concur. If NSPS were not expected to be terminated, we agree that it would be timely to assess NSPS training in terms of outcomes as well as ongoing needs. The DOD training evaluation strategy included this mid-term step. Recommendation 2: The GAO recommends that the Secretary of Defense direct the NSPS Senior Executive to review and revise the Department's guidance for conducting a postdecisional analysis to specify what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. DOD Response: Partially concur. If NSPS, a pay for performance system, were not expected to be terminated, the Department would add process suggestions to help Components adapt their experience with annual barrier analysis methods for the Equal Employment Opportunity program, to rating and payout results. Recommendation 3: The GAO recommends that the Secretary of Defense direct the NSPS Senior Executive to include, as a part of the Department's monitoring of the implementation of the NSPS performance management system, efforts to monitor and evaluate how the safeguards specifically are implemented by lower-level organizations across the Department. DOD Response: Partially concur. If NSPS were not expected to be terminated, the Department would expand on the attention given to how organizations implement and carry out NSPS performance management safeguards, in the DoD human capital accountability program. The nature of future performance management systems will guide how the Department monitors them. [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: Brenda S. Farrell, (202) 512-3604 or farrellb@gao.gov: Acknowledgments: In addition to the contact named above, Ron Fecso, Chief Statistician; Marion Gatling, Assistant Director; Margaret G. Braley; Virginia A. Chanley; William Colwell; Emily Gruenwald; K. Nicole Harms; Cynthia Heckmann; Wesley A. Johnson; Lonnie McAllister; Carolyn Taylor; John W. Van Schaik; Jennifer L. Weber; Cheryl A. Weissman; and Gregory H. Wilmoth made key contributions to the report. [End of section] Related GAO Products: Human Capital: Continued Monitoring of Internal Safeguards and an Action Plan to Address Employee Concerns Could Improve Implementation of the National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-840]. Washington, D.C.: June 25, 2009. Questions for the Record Related to the Implementation of the Department of Defense's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-669R]. Washington, D.C.: May 18, 2009. Human Capital: Improved Implementation of Safeguards and an Action Plan to Address Employee Concerns Could Increase Employee Acceptance of the National Security Personnel System. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-09-464T]. Washington, D.C.: April 1, 2009. Human Capital: Opportunities Exist to Build on Recent Progress to Strengthen DOD's Civilian Human Capital Strategic Plan. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-235. Washington, D.C.: February 10, 2009. High-Risk Series: An Update. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-09-271]. Washington, D.C.: January 2009. Human Capital: DOD Needs to Improve Implementation of and Address Employee Concerns about Its National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-773]. Washington, D.C.: September 10, 2008. The Department of Defense's Civilian Human Capital Strategic Plan Does Not Meet Most Statutory Requirements. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-439R]. Washington, D.C.: February 6, 2008. Human Capital: DOD Needs Better Internal Controls and Visibility over Costs for Implementing Its National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-851]. Washington, D.C.: July 16, 2007. Human Capital: Federal Workforce Challenges in the 21st Century. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-556T]. Washington, D.C.: March 6, 2007. Post-Hearing Questions for the Record Related to the Department of Defense's National Security Personnel System (NSPS). [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-582R]. Washington, D.C.: March 24, 2006. Human Capital: Observations on Final Regulations for DOD's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-06-227T]. Washington, D.C.: November 17, 2005. Human Capital: Designing and Managing Market-Based and More Performance- Oriented Pay Systems. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO- 05-1048T]. Washington, D.C.: September 27, 2005. Human Capital: Symposium on Designing and Managing Market-Based and More Performance-Oriented Pay Systems. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-832SP]. Washington, D.C.: July 27, 2005. Human Capital: DOD's National Security Personnel System Faces Implementation Challenges. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-05-730]. Washington, D.C.: July 14, 2005. Questions for the Record Related to the Department of Defense's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-05-771R]. Washington, D.C.: June 14, 2005. Questions for the Record Regarding the Department of Defense's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-05-770R]. Washington, D.C.: May 31, 2005. Post-hearing Questions Related to the Department of Defense's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-05-641R]. Washington, D.C.: April 29, 2005. Human Capital: Agencies Need Leadership and the Supporting Infrastructure to Take Advantage of New Flexibilities. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-616T]. Washington, D.C.: April 21, 2005. Human Capital: Selected Agencies' Statutory Authorities Could Offer Options in Developing a Framework for Governmentwide Reform. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-398R]. Washington, D.C.: April 21, 2005. Human Capital: Preliminary Observations on Proposed Regulations for DOD's National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-559T]. Washington, D.C.: April 14, 2005. Human Capital: Preliminary Observations on Proposed Department of Defense National Security Personnel System Regulations. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-517T]. Washington, D.C.: April 12, 2005. Human Capital: Preliminary Observations on Proposed DOD National Security Personnel System Regulations. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-432T]. Washington, D.C.: March 15, 2005. Human Capital: Principles, Criteria, and Processes for Governmentwide Federal Human Capital Reform. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-05-69SP]. Washington, D.C.: December 1, 2004. Human Capital: Building on the Current Momentum to Transform the Federal Government. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO- 04-976T]. Washington, D.C.: July 20, 2004. DOD Civilian Personnel: Comprehensive Strategic Workforce Plans Needed. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-753]. Washington, D.C.: June 30, 2004. Human Capital: Implementing Pay for Performance at Selected Personnel Demonstration Projects. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-04-83]. Washington, D.C.: January 23, 2004. Human Capital: Key Principles for Effective Strategic Workforce Planning. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-39]. Washington, D.C.: December 11, 2003. DOD Personnel: Documentation of the Army's Civilian Workforce-Planning Model Needed to Enhance Credibility. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-03-1046]. Washington, D.C.: August 22, 2003. Posthearing Questions Related to Proposed Department of Defense (DOD) Human Capital Reform. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO- 03-965R]. Washington, D.C.: July 3, 2003. Human Capital: Building on DOD's Reform Effort to Foster Governmentwide Improvements. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03- 851T]. Washington, D.C.: June 4, 2003. Posthearing Questions Related to Strategic Human Capital Management. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-779R]. Washington, D.C.: May 22, 2003. Human Capital: DOD's Civilian Personnel Strategic Management and the Proposed National Security Personnel System. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-493T]. Washington, D.C.: May 12, 2003. Defense Transformation: DOD's Proposed Civilian Personnel System and Governmentwide Human Capital Reform. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-03-741T]. Washington, D.C.: May 1, 2003. Defense Transformation: Preliminary Observations on DOD's Proposed Civilian Personnel Reforms. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-03-717T]. Washington, D.C.: April 29, 2003. High-Risk Series: Strategic Human Capital Management. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-120]. Washington, D.C.: January 2003. Acquisition Workforce: Status of Agency Efforts to Address Future Needs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-55]. Washington, D.C.: December 18, 2002. Military Personnel: Oversight Process Needed to Help Maintain Momentum of DOD's Strategic Human Capital Planning. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-237]. Washington, D.C.: December 5, 2002. Managing for Results: Building on the Momentum for Strategic Human Capital Reform. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02- 528T]. Washington, D.C.: March 18, 2002. A Model of Strategic Human Capital Management. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-373SP]. Washington, D.C.: March 15, 2002. Human Capital: Taking Steps to Meet Current and Emerging Human Capital Challenges. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-965T]. Washington, D.C.: July 17, 2001. Human Capital: Major Human Capital Challenges at the Departments of Defense and State. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01- 565T]. Washington, D.C.: March 29, 2001. High-Risk Series: An Update. [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-01-263]. Washington, D.C.: January 2001. [End of section] FOOTNOTES: [1] On May 14, 2009, the Deputy Secretary of Defense requested that the Defense Business Board form a task group to review NSPS to determine (1) if the underlying design principles and methodology for implementation are reflected in the NSPS program objectives; (2) whether the program objectives are being met; and (3) whether NSPS is operating in a fair, transparent, and effective manner. [2] H.R. 2647 and H.R. Conf. Rep. No. 111-288, 1113 (2009). [3] See GAO, Defense Transformation: Preliminary Observations on DOD's Proposed Civilian Personnel Reforms, GAO-03-717T (Washington, D.C.: Apr. 29, 2003); Defense Transformation: DOD's Proposed Civilian Personnel System and Governmentwide Human Capital Reform, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-741T] (Washington, D.C.: May 1, 2003); and Human Capital: Building on DOD's Reform Effort to Foster Governmentwide Improvements, [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-03-851T] (Washington, D.C.: June 4, 2003). [4] See GAO, Human Capital: DOD Needs to Improve Implementation of and Address Employee Concerns about Its National Security Personnel System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-773] (Washington, D.C.: Sept. 10, 2008); Human Capital: DOD Needs Better Internal Controls and Visibility over Costs for Implementing Its National Security Personnel System, [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-07-851] (Washington, D.C.: July 16, 2007); and Human Capital: Observations on Final Regulations for DOD's National Security Personnel System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06- 227T] (Washington, D.C.: Nov. 17, 2005). [5] GAO, Questions for the Record Related to the Implementation of the Department of Defense's National Security Personnel System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-669R] (Washington, D.C.: May 18, 2009). [6] See Related GAO Products at the end of this report for additional reports we have issued related to NSPS and performance management in the federal government. [7] Pub. L. No. 110-181, 1106(c) (2008). Specifically, section 1106(c)(1)(B) directs GAO to conduct reviews in calendar years 2008 through 2010 to evaluate the extent to which DOD has effectively implemented accountability mechanisms, including those established in 5 U.S.C. 9902(b)(7) and other internal safeguards, and to evaluate, as well, the extent that DOD undertakes internal assessments or employee surveys to assess employee satisfaction with NSPS. The accountability mechanisms specified in 5 U.S.C. 9902(b)(7) include those that GAO previously identified as internal safeguards key to successful implementation of performance management systems. For example see GAO, Post-Hearing Questions for the Record Related to the Department of Defense‘s National Security Personnel System (NSPS), GAO-06-582R (Washington, D.C.: Mar. 24, 2006). GAO has emphasized the need for internal safeguards since DOD first proposed NSPS. For example see GAO, Posthearing Questions Related to Strategic Human Capital Management, GAO-03-779R (Washington, D.C.: May 22, 2003). Additionally, our review covered merit system principles that pertain to performance management systems”specifically those stipulated in 5 U.S.C. 2301(b)(2),(3),(7), and (8a). [8] See [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-773] and GAO, Human Capital: Improved Implementation of Safeguards and an Action Plan to Address Employee Concerns Could Increase Employee Acceptance of the National Security Personnel System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-464T] (Washington, D.C.: Apr. 1, 2009). [9] For the purpose of this report, we define safeguards to include accountability mechanisms. We note that we previously identified an independent and credible employee appeals mechanism as a safeguard. For example, see [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06- 582R]. However, although the National Defense Authorization Act for Fiscal Year 2004 (Pub. L. No. 108-136, 1101(a), (2003)) gave DOD the authority to establish a different process for employees to appeal adverse actions than the appeals process available to employees under Title 5, that DOD authority was rescinded by the National Defense Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, 1106(a), (2008). This rescission of authority gave civilian employees under NSPS the same right to appeal adverse actions to the Merit Systems Protection Board that they had under Title 5. 5 U.S.C. 7513(d), 7701. [10] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-741T]. [11] The Department of the Navy's NSPS policies encompass Marine Corps civilians. The Fourth Estate includes all organizational entities in DOD that are not in the military departments or the combatant commands. Examples of Fourth Estate entities are the Office of the Secretary of Defense, the Joint Staff, the Office of the DOD Inspector General, the defense agencies, and DOD field activities. [12] According to the Program Executive Office, as of September 2008, about 85 percent of the civilian employees under NSPS were located in the continental United States, whereas about 15 percent were located outside the continental United States. Of the civilian employees under NSPS located outside the continental United States, about 54 percent were in either Germany or Hawaii. [13] Business rules are the policies that govern a pay pool's operations. They may specify, for example, the pay pool panel's structure, roles and responsibilities, standards of conduct, and the processes used for reconciling employee ratings and allocating shares under NSPS. [14] For the purpose of this report, we use the term officials to refer to the management officials we met with during our site visits, including the performance review authority, pay pool manager, pay pool panel members, rating officials, and NSPS transition manager or members of the NSPS transition team. [15] GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-546G] (Washington, D.C.: March 2004). [16] Specifically, 5 U.S.C. 2301(b)(2) provides that "all employees and applicants for employment should receive fair and equitable treatment in all aspects of personnel management without regard to political affiliation, race, color, religion, national origin, sex, marital status, age, or handicapping condition, and with proper regard for their privacy and constitutional rights." [17] DOD used a phased approach to converting civilian employees to NSPS. Each phase is called a spiral and each spiral has an identifying number associated with it--for example, spiral 1.1. [18] Office of Personnel Management Memorandum, "Guidance on Employee Satisfaction Action Planning" (Aug. 19, 2009). [19] H.R. Conf. Rep. No. 111-288, 1113 (Oct. 7, 2009) to accompany H.R. 2647, the proposed National Defense Authorization Act for 2010. [20] See GAO, Results-Oriented Management: Opportunities Exist for Refining the Oversight and Implementation of the Senior Executive Performance-Based Pay System, [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-09-82] (Washington, D.C.: Nov. 21, 2008); Office of Personnel Management: Key Lessons Learned to Date for Strengthening Capacity to Lead and Implement Human Capital Reforms, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-90] (Washington, D.C.: Jan. 19, 2007); Results-Oriented Cultures: Modern Performance Management Systems Are Needed to Effectively Support Pay for Performance, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-612T] (Washington, D.C.: Apr. 1, 2003); and [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-03-741T]. [21] Pub. L. No. 108-136, 1101 (2003) (codified at 5 U.S.C. 9901- 9904). The National Defense Authorization Act for Fiscal Year 2008 amended 5 U.S.C. 9902. Pub. L. No. 110-181, 1106 (2008). [22] According to PEO officials, DOD originally planned to convert approximately 700,000 civilian employees to NSPS; however, legislative changes have decreased the number of civilian employees eligible to convert to the system. [23] Pub. L. No. 110-181, 1106(a) (2008). The Federal Wage System is a uniform pay-setting system that covers federal appropriated fund and nonappropriated fund blue-collar employees who are paid by the hour. [24] Criteria to distinguish pay pools may include, but are not limited to, organization structure, employee job function, location, and organization mission. [25] Where determined appropriate because of the size of the pay pool population, the complexity of the mission, the need to prevent conflicts of interest, or other similar criteria, sub-pay pool panels may be organized in a structure subordinate to the pay pool panel. Sub- pay pool panels normally operate under the same requirements and guidelines provided to the pay pool panels to which they belong. [26] Pay pool panel members may not participate in payout deliberations or decisions that directly affect their own performance assessments or pay. [27] The senior organization official, usually a member of the Senior Executive Service or a General/Flag officer, serves as the performance review authority. DOD components may provide additional guidance for the establishment of performance review authorities. The responsibilities of the performance review authority may be assigned to an individual management official or organizational unit or group. [28] 5. C.F.R. 9901 (2008). [29] The mock pay pool is a way for organizations to understand the pay pool process. During the exercise, organizations identify ways to improve their pay pool process to achieve greater consistency and ensure fairness in ratings and payouts. Rating officials review their employees and assign mock ratings, numbers of shares, and payout distributions. The panel then practices advocating for employees and reconciling results. [30] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-04-546G]. [31] Of these 120 human resource practitioners, only 1 was located outside the continental United States. [32] DOD's 2008 evaluation focused on determining whether NSPS, as implemented in spiral 1 organizations, was on track to achieve certain goals specified by DOD and if any improvements were needed. These goals include whether a supporting infrastructure is in place for the system. [33] Management Directive 715 is the policy guidance which the Equal Employment Opportunity Commission provides to federal agencies for their use in establishing and maintaining effective affirmative programs of equal employment opportunity under Section 717 of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. 2000(e) et seq., and effective affirmative action programs under Section 501 of the Rehabilitation Act of 1973, as amended by Pub. L. No. 99-506, 103 (1986). [34] Management Directive 715 defines a barrier as "an agency policy, principle, practice or condition that limits or tends to limit employment opportunities for members of a particular gender, race or ethnic background or for an individual (or individuals) based on disability status." [35] 5 U.S.C. 2301(b)(2) and (b)(8)(A). [36] 5 C.F.R. 9901.412(a). [37] DOD has established certain goals for the NSPS performance management system, known as key performance parameters. DOD defines a key performance parameter as "a capability or characteristic that is so significant that failure to meet a minimum 'threshold' can be cause for that element, concept or system to be reevaluated, or the program to be reassessed or terminated." DOD originally identified six key performance parameters for NSPS: (1) High Performing Workforce and Management--whether employees and supervisors are compensated and retained based on their performance and contribution to mission; (2) Agile and Responsive Workforce and Management--whether the workforce can be easily sized, shaped, and deployed to meet changing mission requirements; (3) Credible and Trusted System--whether the system assures openness, clarity, accountability and adherence to the public employment principles of merit and fitness; (4) Fiscally Sound System- -whether aggregate increases in civilian payroll, at the appropriations level, will conform to Office of Management and Budget fiscal guidance, and whether managers will have flexibility to manage to budget at the unit level; (5) Supporting Infrastructure--whether information technology support, training, and change management plans are available and funded; and (6) Schedule--whether NSPS will be operational and stable before November 2009. [38] DOD's 2008 evaluation focused only on spiral 1 organizations under NSPS--which included a total of 108,758 employees. Also, according to the PEO, externally driven delays to converting employees to NSPS subsequently resulted in the department's elimination of the sixth key performance parameter, Schedule. Therefore, DOD's 2008 evaluation did not assess the extent to which the department was on track to achieve this particular goal. [39] GAO, Human Capital: Symposium on Designing and Managing Market- Based and More Performance-Oriented Pay Systems, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-832SP] (Washington, D.C.: July 27, 2005). [40] The first Web-based Status of Forces Survey was conducted in October 2003. Regular administrations of the Status of Forces Survey of Civilian Employees occurred every 6 months from October 2004 through November 2006, and annual administrations commenced in 2007. [41] Office of Personnel Management, Working for America: Alternative Personnel Systems in Practice and a Guide to the Future (Washington, D.C., October 2005). [42] The NSPS regulations (5 C.F.R 9901.353) state that an employee may only receive up to a total of a 5 percent cumulative increase to his or her base salary in any 12-month period as the result of an employee-initiated action, unless an exception is approved by an authorized management official. There are no limits to the number of times an employee may be reassigned by management, however. [43] The NSPS regulations (5 C.F.R. 9901.412(a)) state that the forced distribution of ratings (setting preestablished limits for the percentage or number of ratings that may be assigned at any level) is prohibited. [44] Office of Personnel Management, Annual Employee Survey Guidance (Washington, D.C., November 2006). [45] Office of Personnel Management Memorandum, "Guidance on Employee Satisfaction Action Planning" (Aug. 19, 2009). [46] GAO, Human Capital: DOD Needs to Improve Implementation of and Address Employee Concerns about Its National Security Personnel System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-773] (Washington, D.C.: Sept. 10, 2008). [47] GAO has not independently verified the reliability of DOD's reported reconsideration statistics. [48] The Department of the Navy's NSPS policies encompass Marine Corps civilians. The Fourth Estate includes all organizational entities in DOD that are not in the military departments or the combatant commands. Examples of Fourth Estate entities are the Office of the Secretary of Defense, the Joint Staff, the Office of the DOD Inspector General, the defense agencies, and DOD field activities. [49] According to the Program Executive Office, as of September 2008, about 85 percent of the civilian employees under NSPS were located in the continental United States, whereas about 15 percent were located outside the continental United States. Of the civilian employees under NSPS located outside the continental United States, about 54 percent were in either Germany or Hawaii. [50] Business rules are the policies that govern a pay pool's operations. They may specify, for example, the pay pool panel's structure, roles and responsibilities, standards of conduct, and the processes used for reconciling employee ratings and allocating shares under NSPS. [51] The Status of Forces Survey is a series of Web-based surveys of the total force that allows DOD to (1) evaluate existing programs/ policies, (2) establish baselines before implementing new programs/ policies, and (3) monitor progress of programs/policies and their effects on the total force. [52] DMDC has conducted large-scale, departmentwide surveys of active military personnel since 2002, called the Status of Forces Active Duty Survey. DMDC has also conducted surveys of reserve military personnel for DOD called the Status of Forces Reserve Survey. Since 2003, DMDC has administered its Status of Forces Survey of Civilian Employees, which includes questions about compensation, performance, and personnel processes. All surveys include outcome or "leading indicator" measures such as overall satisfaction, retention intention, and perceived readiness, as well as demographic items needed to classify individuals into various subpopulations. Regular administrations of the Status of Forces Survey of Civilian Employees occurred every 6 months from October 2004 through November 2006, while annual administrations commenced in 2007. [53] In our September 2008 report, we identified areas for improvement with regard to DOD's survey results--for example, use of nonresponse analysis, which is a good survey research practice, to clarify whether those employees who did not respond to DOD's surveys may provide substantively different answers than those who did respond. [54] In contrast, negotiated grievance procedures are the exclusive administrative procedures for bargaining unit employees. However, if a negotiated grievance procedure is not available to a bargaining unit employee or challenging a rating of record or job objective rating is outside the scope of the employee's negotiated grievance procedure, a bargaining unit employee may challenge his or her rating of record or job objective rating in accordance with the DOD's NSPS regulations and implementing issuances. As of May 2009, DOD reported that there were approximately 685 bargaining unit employees and 18 certified bargaining units representing NSPS employees. [55] DOD civilian employees under NSPS received their payout for the 2007 NSPS performance management cycle in January 2008. [56] DOD civilian employees under NSPS received their payout for the 2008 NSPS performance management cycle in January 2009. [57] GAO has not independently verified the reliability of DOD's reported reconsideration statistics. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.