Measuring Performance

American taxpayers invested more than $60 billion in military and civilian research and development projects in 1996. The private sector spent more than $110 billion that same year. The technological advancements stemming from these efforts have helped improve the productivity of American workers and, correspondingly, the nation's standard of living. However, although the contributions of research and development to technological advancement are generally recognized, there is no widely accepted method of measuring the outcome of that research. This report evaluates the various indicators that are used to measure the results of research and development. Specifically, GAO discusses the strengths and limitations of the input and output indicators used by the federal and private sectors to measure the results of research and development. GAO also provides a historical perspective on spending for research. GAO summarized this report in testimony before Congress; see: Measuring Performance: Challenges in Evaluating Research and Development, by Allen Li, Associate Director for Energy, Resources, and Science Issues, before the Subcommittee on Technology, House Committee on Science (GAO/T-RCED-97-130, Apr. 10).

GAO noted that: (1) the amount of money spent on research and development, the primary indicator of the investment in research, is useful as a measure of how much research is being performed; (2) having been refined over many years, these data are generally available for the research efforts in both the public and private sectors; (3) however, the level of spending is not a reliable indicator of the level of results achieved by research; (4) unlike the situation with the input measures of R&D, there is no primary indicator of the outputs; (5) output indicators include quantitative analyses of return on investment, patents granted, and other outputs as well as qualitative assessments based on peer review; (6) the companies that GAO spoke with collect data on various output indicators but, in general, make limited used of them in their investment decisions; (7) instead, the companies emphasized that R&D contribute directly to their "bottom line"; (8) because companies are profit-oriented, many of the indicators tracked by the private sector cannot be directly applied to the federal government; and (9) experiences from pilot efforts made under the Government Performance and Results Act have reinforced the finding that output measures are highly specific to the management and mission of each federal agency and that no single indicator exists to measure the results of research.