Department of Education

In connection with the Omnibus Appropriations Act, 2009, GAO was required to study the What Works Clearinghouse (WWC), a federal source of evidence about effective education practices. Operating through a 5-year contract awarded by the U.S. Department of Education's Institute of Education Sciences (IES), the WWC reviews education research and disseminates its findings. GAO examined: (1) the extent to which the WWC review process meets accepted standards for research evaluation and how the WWC has responded to recommendations and criticism, (2) how WWC output and costs have changed over time and how its performance is measured, and (3) how WWC products are disseminated and how useful educators find them to be. To conduct its work, GAO reviewed WWC-related documents, analyzed cost and performance data, surveyed all states and a nationally representative sample of school districts, and interviewed IES officials, WWC contractors, researchers, and others.

GAO as well as a congressionallymandated panel of experts, found that the WWC's review process, which includes screening studies to determine if they meet WWC criteria, follows accepted standards for evaluating research on the effectiveness of education interventions. WWC is responding to recommendations made by the expert panel to further improve its review and reporting processes. For example, the panel recommended improvements in the way the WWC presents information to readers on the reasons why studies do not qualify for review. The WWC is revising a report template to include a table summarizing which studies met or did not meet WWC criteria for evaluating research. The WWC has also responded to researchers who have criticized the WWC for presenting limited information because its screening criteria exclude some rigorous research designs that may be appropriate for evaluating certain education programs, such as special education. The WWC responded to this criticism by creating new standards that include two additional study designs and by creating a new product, called a practice guide, which includes a wider range of research. WWC's report output and scope increased under the current contract. For example, the WWC increased its production of various reports, introduced new products, and developed new processes for evaluating research. However, IES had a substantial backlog in its product review process from January 2009 to May 2010. The backlog generally decreased the timeliness of WWC reports, with 20 reports being delayed by up to 6 months. To support the increases in output and scope, WWC's costs doubled from the previous contract to the current one. Both contracts designated about 60 percent of costs to production, while the other 40 percent of costs support other tasks, such as communications, dissemination, and process development. IES' performance goals for the WWC primarily relate to the number of reports produced. However, IES has not developed performance measures related to the cost or usefulness of WWC products. Education uses WWC contractors, Regional Educational Laboratories (RELS) and the Doing What Works (DWW) Web site to disseminate information about WWC products; however, awareness and use of the WWC varies among states, districts, teachers, and principals. WWC contractors disseminate product information in various ways including email alerts and presentations. The RELs host events featuring WWC products for state, district, and school officials and DWW provides resources to educators based on WWC products. Based on our survey, officials from 33 of 38 state education agencies that responded to our survey and an estimated 42 percent of school districts have heard of the WWC. Those states and school districts generally used the WWC to a small or moderate extent to inform decisions on effective practices. Based on our survey, states and school districts reported that they would likely increase their use of the WWC if it included a broader array of information or more timely information. GAO recommends that IES: develop and implement strategies to avoid backlogs in WWC product reviews; establish performance measures related to costs and usefulness; and improve dissemination efforts to promote awareness and use of the WWC. Education generally agreed with GAO's recommendations.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Cornelia M. Ashby Team: Government Accountability Office: Education, Workforce, and Income Security Phone: (202) 512-8403