Federal Research

The federal government will invest $80 billion in fiscal year 1999 on research and development done by government scientists and through grants, contracts, or other agreements with universities, corporations, small businesses, and other members of the research community. GAO was asked to study the peer review and other quality assurance processes that federal agencies use in conducting scientific research and development. GAO reviewed 12 federal agencies to (1) define what is meant by peer review, (2) describe the federal government's peer review policy, (3) describe the peer review practices of the agencies that conduct scientific research, (4) describe other agency quality assurance reviews, and (5) identify which research is not subject to review. GAO found that there is no written governmentwide definition of peer review and that each of the 12 agencies had various policies, orders, or other internal guidance on the conduct of peer reviews.

GAO noted that: (1) there is no written governmentwide definition of peer review; (2) officials at the Office of Science and Technology Policy (OSTP) and at the agencies GAO contacted generally concur that peer review is defined as a process that includes an independent assessment of the technical, or scientific merit of research by peers who are scientists with knowledge and expertise equal to that of the researchers whose work they review; (3) there is no uniform federal policy for conducting peer reviews; (4) through annual budget guidance to federal agencies, OSTP and the Office of Management and Budget encourage funding of research projects that are peer reviewed over those that are not reviewed; (5) officials at OSTP said that peer review practices should not be dictated uniformly for every agency or for all types of federally funded research; (6) rather, the practices should be tailored to agency missions and type of research; (7) each of the 12 agencies that GAO contacted had a variety of policies, orders, or other internal guidance regarding the conduct of peer review; (8) to varying degrees, the 12 agencies use peer review to: (a) assess the merit of competitive and noncompetitive research proposals; (b) determine whether to continue or renew research projects; (c) evaluate the results of the research prior to the publication of those results; (d) establish annual budget priorities for research programs; and (e) evaluate program and scientist performance; (9) all of the agencies use peer review to assess competitive research proposals; (10) the methods for conducting peer reviews vary among and within the agencies; (11) most of the agencies that GAO reviewed also use reviews by agency supervisors or program managers to assess the quality of research proposals, to check the quality of in-progress research, and to evaluate program performance; (12) generally, these quality assurance reviews are not considered independent assessments--a key criterion in the peer review process; (13) these quality assurance reviews occur at both the project and program levels; (14) while agencies reported that almost all research is reviewed either through peer reviews or other quality assurance reviews, a small amount of research may not be reviewed by the agencies in certain circumstances; and (15) examples of research that may be funded without being reviewed include projects that are congressionally mandated or projects that use widely accepted methodologies.