Education for Disadvantaged Children

The ongoing Longitudinal Evaluation of School Change and Performance (LESCP) is due to be completed in 2001. Although some expected LESCP to serve as a vehicle for evaluating Title I's effectiveness--as the 1997 prospects study had been--others expected it to evaluate specific educational reform policies or changes in instructional practice. This lack of agreement created unclear expectations for the study. LESCP and the prospects study used similar processes for contracting out data collection and analysis, obtaining comments from review panels, and releasing the results. Both obtained advice from two panels of experts. The three- to four-year longitudinal data collection and the complexity of the implementation posed challenges for providing study results in time to meet Congress' five-year reauthorization schedule; thus the Department of Education issued interim reports to Congress. The prospect study's large, representative sample supported fairly strong conclusions about the effect of Title I but did not allow researchers to definitively measure how students would have performed had they no access to Title I services. Depth of information in the LESCP study is liable to be a key strength, but because its sample is small, its conclusions on Title I evaluation, standards-based reform, or instructional practices are likely to be affected.

GAO noted that: (1) while Education staff, contractors, and members of advisory panels all agreed that the purpose of the Prospects study was to assess the effectiveness of Title I of the Elementary and Secondary Education Act, considerably less agreement existed on the precise purpose of the LESCP study; (2) these differences centered primarily around how much emphasis the study woul dplace on the Title I program; (3) the lack of agreement about LESCP's purpose created unclear expectations for the study, making it difficult to predict the degree to which the final report will meet the needs of Congress, other policymakers, and educators; (4) the LESCP and Prospects studies used similar processes for contracting out data collection and analysis, obtaining comments from review panels, and releasing the results; (5) both studies were conducted by outside research organizations under contracts with Educations; (6) for both Prospects and LESCP, Education and the contractors obtained advice from two panels of experts--a congressionally mandated review panel that advised Education on policy issues and a seperate technical panel that provided feedback specific to the individual study; (7) the panels that advised the Prospects study had proportionally greater representation from educators, while LESCP's panels had proportionally more researchers; (8) for both the LESCO and Prospect studies, the 3 to 4 year longitudinal data collection and the complexity of the implementation process posed challenges for providing study results in time to meet Congress' 5-year reauthorization schedule; (9) to provide information for reauthorization, Education issued interim reports to Congress in both cases; (10) for both Prospects and LESCP, major design features influenced the study's ability to address its overall puspose; (11) the design of each study has both strengths and potential limitation; (12) the LESCP study is not yet complete, but the depth of information provided by its longitudinal focus is likely to be a key strength; (13) however, LESCP, which uses a smaller, nonrepresentative sample, suffers from design limitations that will restrict its ability to fully satisfy any of the three potential purposes envisioned by Education, contractors, and panel members; and (14) because several design limitations are directly related to measuring the effects of Title I and standards-based reform, they will likey have a major effect on the conclusions that could be drawn in these areas of education policy.