Managing for Results

The Government Performance and Results Act (GPRA) requires agencies to identify program goals and report on their progress in achieving them. GPRA includes a phase during which about 70 programs, ranging from the U.S. Geological Survey's National Water Quality Assessment Program to the entire Social Security Administration, were designated as GPRA pilot projects. These and other government programs have been gaining experience with the act's requirements. GPRA requires GAO to review implementation of the pilot phase and to comment on the prospects for compliance by federal agencies when governmentwide implementation begins in 1997. This report answers the following questions: What analytic and technical challenges are agencies experiencing as they try to measure program performance? What approaches have they taken to address these challenges? How have agencies made use of program evaluations or evaluation expertise in implementing performance measurement?

GAO noted that: (1) the programs included in GAO's review encountered a wide range of serious challenges; (2) 93 percent of the officials GAO surveyed reported at least one as a great or very great challenge, and some were not very far along in implementing the steps required by the Results Act; (3) eight of the 10 tasks rated most challenging emerged in the two relatively early stages of the performance measurement process, identifying goals and developing performance measures; (4) in developing both goals and performance measures, respondents found it difficult to move beyond a summary of their program's activities, such as the number of clients served, to distinguish the desired outcome or result of those activities; (5) sometimes selecting an outcome measure was impeded, instead, by conflicting stakeholder views of the program's intended results or by anticipated data collection problems; (6) issues in the data collection stage were rated as less serious and revolved around the programs' lack of control over data that third parties collected, but programs may have avoided some data issues through selection of measures for which data already existed; (7) the greatest challenge in the analysis and reporting stage was separating a program's impact on its objectives from the impact of external factors, primarily because many federal programs' objectives are the result of complex systems or phenomena outside the program's control; (8) in such cases, it is particularly challenging for agencies to confidently attribute changes in outcomes to their program, the central task of program impact evaluation; (9) the programs GAO reviewed had applied a range of analytic and other strategies to address these challenges; (10) because they had either volunteered to be GPRA pilots or had already begun implementing performance measurement, the programs included in GAO's review were likely to be better suited or prepared for conducting performance measurement than most federal programs; and (11) the challenges experienced by the projects that are pilot testing the Act's requirements suggest that: (a) more typical federal programs may find performance measurement to be an even greater challenge, particularly if they do not have access to program evaluation or other technical resources; and (b) full-scale implementation will require several iterations to develop valid, reliable, and useful performance reporting systems.