Measuring Performance

American taxpayers invested more than $60 billion in military and civilian research and development projects in 1996. The private sector spent more than $110 billion that same year. The technological advancements stemming from these efforts have helped improve the productivity of American workers and, correspondingly, the nation's standard of living. However, although the contributions of research and development to technological advancement are generally recognized, there is no widely accepted method of measuring the outcome of that research. The related report, GAO/RCED-97-91, evaluates the various indicators that are used to measure the results of research and development. Specifically, GAO discusses the strengths and limitations of the input and output indicators used by the federal and private sectors to measure the results of research and development. GAO also provides a historical perspective on spending for research.

GAO noted: (1) ATP has funded research projects that would have been funded by the private sector as well as those that would not; (2) the award recipients were nearly evenly divided when asked if they would have pursued their projects if they had not received such funding; (3) GAO also found that in most cases, the participants in its survey did not look for funding from other sources, private or public, before trying to obtain funding from ATP; (4) about half of the 45 applicants that tried to find funding elsewhere before turning to ATP were told by prospective funders that their projects were either too risky or "precompetitive", characteristics that fulfill the aims of ATP; (5) manufacturers viewed the manufacturing extension programs' services positively, as was demonstrated in GAO's national survey of manufacturers who had received substantive services from the programs in 1993; (6) most manufacturers responding to GAO's questionnaire, about 73 percent, reported that they believed that the type of assistance they had received from these programs had positively affected their overall business performance; (7) about 15 percent of the respondents reported that they believed the programs' assistance had not affected their overall business performance; (8) the amount of money spent on R&D, the primary indicator of research investment, is useful as an input measure of how much research is being performed; (9) however, the level of spending is not a reliable indicator of research results; (10) GAO found that there is no primary indicator of R&D results; (11) the companies that GAO spoke with collect data on various output indicators, such as return on investment and patents granted, but in general make limited use of them in their investment decisions; (12) instead, they emphasized that R&D contributes directly to the bottom line; (13) because companies are profit oriented, many of the indicators tracked by the private sector cannot be directly applied to the federal government; (14) determining the specific outcomes resulting from federal R&D is a challenge that will not be easily resolved; and (15) however, in response to recent legislation requiring agencies to report on program results, some progress is being made in measuring the impacts of research.