No Child Left Behind Act
Education Assistance Could Help States Better Measure Progress of Students with Limited English Proficiency Gao ID: GAO-07-646T March 23, 2007The No Child Left Behind Act of 2001 (NCLBA) focused attention on the academic achievement of more than 5 million students with limited English proficiency. Obtaining valid test results for these students is challenging, given their language barriers. This testimony describes (1) the extent to which these students are meeting annual academic progress goals, (2) what states have done to ensure the validity of their academic assessments, (3) what states are doing to ensure the validity of their English language proficiency assessments, and (4) how the U.S. Department of Education (Education) is supporting states' efforts to meet NCLBA's assessment requirements for these students. This testimony is based on a July 2006 report (GAO-06-815). To collect the information for this report, we convened a group of experts and studied five states (California, Nebraska, New York, North Carolina, and Texas). We also conducted a state survey and reviewed state and Education documents.
In nearly two-thirds of 48 states for which we obtained data, students with limited English proficiency did not meet state proficiency goals for language arts or mathematics in school year 2003-2004. Further, in most states, these students generally did not perform as well as other student groups on state mathematics tests for elementary students. Officials in our five study states reported taking steps to follow generally accepted test development procedures to ensure the validity and reliability of academic tests for these students. However, our group of experts expressed concerns about whether all states are assessing these students in a valid manner, noting that some states lack technical expertise. Further, Education's completed peer reviews of assessments in 38 states found that 25 states did not provide adequate evidence of their validity or reliability. To improve the validity of these test results, most states offer accommodations, such as a bilingual dictionary. However, our experts reported that research is lacking on what accommodations are effective in mitigating language barriers. Several states used native language or alternate assessments for students with limited English proficiency, but these tests are costly to develop and are not appropriate for all students. Many states implemented new English language proficiency assessments in 2006 to meet NCLBA requirements, and, as a result, complete information on their validity and reliability is not yet available. In 2006, 22 states used tests developed by one of four state consortia. Officials in our study states reported taking steps to ensure the validity of these tests. However, a 2005 Education-funded review of 17 English language proficiency tests found insufficient documentation of their validity. Education has offered a variety of technical assistance to help states assess students with limited English proficiency. However, Education has issued little written guidance to states on developing English language proficiency tests. Officials in about one-third of the 33 states we contacted told us they wanted more guidance about how to develop tests that meet NCLBA requirements. Education has offered states some flexibility in how they assess students with limited English proficiency, but officials in our study states told us that additional flexibility is needed to ensure that progress measures appropriately track the academic progress of these students. Since our report was published, Education has initiated a partnership with the states and other organizations to support the development of valid assessment options for students with limited English proficiency.
GAO-07-646T, No Child Left Behind Act: Education Assistance Could Help States Better Measure Progress of Students with Limited English Proficiency
This is the accessible text file for GAO report number GAO-07-646T
entitled 'No Child Left Behind: Education Assistance Could Help States
Better Measure Progress of Students with Limited English Proficiency'
which was released on March 23, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony before the Subcommittee on Early Childhood, Elementary and
Secondary Education, Committee on Education and Labor, House of
Representatives:
United States Government Accountability Office:
GAO:
For Release on Delivery Expected at 9:30 a.m. EDT:
Friday, March 23, 2007:
No Child Left Behind Act:
Education Assistance Could Help States Better Measure Progress of
Students with Limited English Proficiency:
Statement of Cornelia M. Ashby, Director:
Education, Workforce, and Income Security Issues:
GAO-07-646T:
GAO Highlights:
Highlights of GAO-07-646T, a testimony before the Subcommittee on Early
Childhood, Elementary and Secondary Education, Committee on Education
and Labor, House of Representatives
Why GAO Did This Study:
The No Child Left Behind Act of 2001 (NCLBA) focused attention on the
academic achievement of more than 5 million students with limited
English proficiency. Obtaining valid test results for these students is
challenging, given their language barriers. This testimony describes
(1) the extent to which these students are meeting annual academic
progress goals, (2) what states have done to ensure the validity of
their academic assessments, (3) what states are doing to ensure the
validity of their English language proficiency assessments, and (4) how
the U.S. Department of Education (Education) is supporting states‘
efforts to meet NCLBA‘s assessment requirements for these students.
This testimony is based on a July 2006 report (GAO-06-815). To collect
the information for this report, we convened a group of experts and
studied five states (California, Nebraska, New York, North Carolina,
and Texas). We also conducted a state survey and reviewed state and
Education documents.
What GAO Found:
In nearly two-thirds of 48 states for which we obtained data, students
with limited English proficiency did not meet state proficiency goals
for language arts or mathematics in school year 2003-2004. Further, in
most states, these students generally did not perform as well as other
student groups on state mathematics tests for elementary students.
Officials in our five study states reported taking steps to follow
generally accepted test development procedures to ensure the validity
and reliability of academic tests for these students. However, our
group of experts expressed concerns about whether all states are
assessing these students in a valid manner, noting that some states
lack technical expertise. Further, Education‘s completed peer reviews
of assessments in 38 states found that 25 states did not provide
adequate evidence of their validity or reliability. To improve the
validity of these test results, most states offer accommodations, such
as a bilingual dictionary. However, our experts reported that research
is lacking on what accommodations are effective in mitigating language
barriers. Several states used native language or alternate assessments
for students with limited English proficiency, but these tests are
costly to develop and are not appropriate for all students.
Many states implemented new English language proficiency assessments in
2006 to meet NCLBA requirements, and, as a result, complete information
on their validity and reliability is not yet available. In 2006, 22
states used tests developed by one of four state consortia. Officials
in our study states reported taking steps to ensure the validity of
these tests. However, a 2005 Education-funded review of 17 English
language proficiency tests found insufficient documentation of their
validity.
Education has offered a variety of technical assistance to help states
assess students with limited English proficiency. However, Education
has issued little written guidance to states on developing English
language proficiency tests. Officials in about one-third of the 33
states we contacted told us they wanted more guidance about how to
develop tests that meet NCLBA requirements. Education has offered
states some flexibility in how they assess students with limited
English proficiency, but officials in our study states told us that
additional flexibility is needed to ensure that progress measures
appropriately track the academic progress of these students. Since our
report was published, Education has initiated a partnership with the
states and other organizations to support the development of valid
assessment options for students with limited English proficiency.
What GAO Recommends:
The GAO report recommended that Education (1) support research on
accommodations, (2) identify and provide technical support states need
to ensure the validity of academic assessments, (3) publish additional
guidance on requirements for assessing English language proficiency,
and (4) explore ways to provide additional flexibility for measuring
annual progress for these students. Education generally agreed with our
recommendations and has taken a number of steps to address them.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-646T].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cornelia Ashby at (202)
512-7215 or ashbyc@gao.gov.
[End of section]
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today to present information from our July 2006
report on the assessment requirements of the No Child Left Behind Act
(NCLBA) as they pertain to students with limited English
proficiency.[Footnote 1] An estimated 5 million children with limited
English proficiency were enrolled in U.S. public schools during the
2003-2004 school year, representing about 10 percent of the total
school population. They speak over 400 languages, with almost 80
percent of students with limited English proficiency speaking Spanish.
These students often have language difficulties that interfere with
their ability to succeed in school and, prior to NCLBA, were often
excluded from statewide assessments. NCLBA's requirements have brought
to the surface a number of challenges to assessing the academic
performance of these students in a valid and reliable manner (that is,
the assessment measures what it is designed to measure in a consistent
manner).
Congress passed NCLBA with the goal of increasing academic achievement
and closing achievement gaps. NCLBA required states to demonstrate that
all students have reached the "proficient" level on a state's language
arts and mathematics assessments by 2014, and states must demonstrate
"adequate yearly progress" toward this goal each year. In addition,
students from groups that traditionally underperform, including
students with limited English proficiency, must meet the same academic
progress goals as other students. For the first time, NCLBA also
required states to annually assess the English proficiency of these
students and to demonstrate that they are making progress toward
becoming proficient in English.
My testimony today will focus on (1) the extent to which students with
limited English proficiency are meeting adequate yearly progress goals,
(2) what states have done to ensure that results from language arts and
mathematics assessments are valid and reliable for students with
limited English proficiency, (3) how states are assessing English
proficiency and what they are doing to address the validity and
reliability of these assessment results, and (4) how the Department of
Education (Education) is supporting states' efforts to meet NCLBA's
assessment requirements for these students. The information being
presented today is from our July 2006 report.
In summary, students with limited English proficiency did not meet
state proficiency goals on language arts and mathematics tests in
nearly two-thirds of 48 states for which we obtained data in the 2003-
2004 school year. Officials in 5 states we studied reported taking
steps to follow generally accepted test development procedures to
ensure the validity and reliability of their academic tests for
students with limited English proficiency. However, a group of experts
we consulted expressed concerns about whether all states were assessing
these students in a valid manner. These experts noted that some states
lack the technical expertise needed to ensure the validity of tests for
these students. As evidence of the challenges states face, Education's
completed peer reviews of 38 states found that 25 did not provide
adequate evidence on the validity or reliability of test results for
these students. We also found that, as allowed under law, most states
offer accommodations, such as a bilingual dictionary, to these students
in order to improve the validity of language arts and mathematics test
results. However, our experts reported that research is lacking on what
accommodations are effective for these students. With respect to
English language proficiency assessments, many states were implementing
new tests in 2006 to meet NCLBA requirements, and as a result, complete
information on their validity and reliability was not available at the
time of our review. Education has offered a variety of technical
assistance to help states assess students with limited English
proficiency. However, Education has issued little written guidance to
states on developing English language proficiency tests. Officials in
about one-third of the 33 states we contacted told us they wanted more
guidance about how to develop tests that meet NCLBA requirements.
To help states assess students with limited English proficiency in a
valid and reliable manner, our recent report included several
recommendations. Education agreed with most of the report's
recommendations and has taken a number of steps to address them.
Specifically, Education has initiated a partnership with the states and
other organizations to support the development of valid assessment
options for students with limited English proficiency.
To determine the extent to which students with limited English
proficiency were meeting adequate yearly progress goals, we collected
school year 2003-2004 state-level data for 48 states, including the
District of Columbia. With regard to assessments, we studied the
testing practices of 5 states in depth (California, Nebraska, New York,
North Carolina, and Texas). We also directly contacted officials in 28
states to discuss their English language proficiency assessments and
Education's guidance regarding these assessments. Further, we convened
a group of experts to discuss states' efforts to implement valid and
reliable academic assessments for students with limited English
proficiency. These experts had significant technical and research
expertise in assessment issues, and some had conducted research focused
on students with limited English proficiency. We also interviewed
Education officials and reviewed relevant Education documents. Finally,
we interviewed officials from major test development companies and
state consortia that are developing English language proficiency
assessments and used a short e-mail survey to obtain information from
the 50 states and the District of Columbia on their use of native
language assessments. We conducted the review in accordance with
generally accepted government auditing standards.
Background:
Students with limited English proficiency are a diverse and complex
group. They speak many languages and have a tremendous range of
educational needs and include refugees with little formal schooling and
students who are literate in their native languages. Accurately
assessing the academic knowledge of these students in English is
challenging. If a student responds incorrectly to a test item, it may
not be clear if the student did not know the answer or misunderstood
the question because of language barriers.
Title I of NCLBA requires states to administer tests in language arts
and mathematics to all students in certain grades and to use these
tests as the primary means of determining the annual performance of
states, districts, and schools. These assessments must be aligned with
the state's academic standards--that is, they must measure how well a
student has demonstrated his or her knowledge of the academic content
represented in these standards. States are to show that increasing
percentages of students are reaching the proficient level on these
state tests over time. NCLBA also requires that students with limited
English proficiency receive reasonable accommodations and be assessed,
to the extent practicable, in the language and form most likely to
yield accurate data on their academic knowledge. In addition, for
language arts, students with limited English proficiency who have been
in U.S. schools for 3 years or more must generally be assessed in
English. Finally, NCLBA also created a new requirement for states to
annually assess the English language proficiency of students identified
as having limited English proficiency.
Accurately assessing the academic knowledge of students with limited
English proficiency has become more critical because NCLBA designated
specific groups of students for particular focus. These four groups are
students who (1) are economically disadvantaged, (2) represent major
racial and ethnic groups, (3) have disabilities, and (4) are limited in
English proficiency. These groups are not mutually exclusive, so that
the results for a student who is economically disadvantaged, Hispanic,
and has limited English proficiency could be counted in three groups.
States and school districts are required to measure the progress of all
students in meeting academic proficiency goals, as well as to measure
separately the progress of these designated groups. To make adequate
yearly progress, each district and school must generally show that each
of these groups met the state proficiency goal and that at least 95
percent of students in each group participated in these
assessments.[Footnote 2] Students with limited English proficiency are
a unique group under NCLBA because once they attain English proficiency
they are no longer counted as part this group, although Education has
given states some flexibility in this area.
Recognizing that language barriers can hinder the assessment of
students who have been in the country for a short time, Education has
provided some testing flexibility[Footnote 3]. Specifically, Education
does not require students with limited English proficiency to
participate in a state's language arts assessment during their first
year in U.S. schools. In addition, while these students must take a
state's mathematics assessment during their first year, a state may
exclude their scores in determining whether it met its progress goals.
Title III of NCLBA focuses specifically on students with limited
English proficiency, with the purpose of ensuring that these students
attain English proficiency and meet the same academic standards as
other students. This title holds states and districts accountable for
student progress in attaining English proficiency by requiring states
to establish goals to demonstrate annual increases in both the number
of students attaining English proficiency and the number making
progress in learning English. States must establish English language
proficiency standards that are aligned with a state's academic
standards in order to ensure that students are acquiring the academic
language they need to successfully participate in the classroom.
Education also requires that a state's English language proficiency
assessment be aligned to its English language proficiency standards.
While NCLBA requires states to administer academic assessments to
students in some grades, it requires states to administer English
language proficiency assessments annually to all students with limited
English proficiency, from kindergarten to grade 12.
Students with Limited English Proficiency Performed below Progress
Goals in 2004 in Almost Two-Thirds of States:
In nearly two-thirds of the 48 states for which we obtained data,
students with limited English proficiency did not meet state
proficiency goals in the 2003-2004 school year. Students with limited
English proficiency met goals in language arts and mathematics in 17
states[Footnote 4]. In 31 states, these students missed the goals
either for language arts or for both language arts and mathematics (see
fig. 1). In 21 states, the percentage of proficient students in this
group was below both the mathematics and the language arts proficiency
goals.
Figure 1: School Year 2003-2004 Comparison of Percentage of Students
with Limited English Proficiency Who Achieved Proficient Scores in
Language Arts and Mathematics with State-Established Progress Goals:
[See PDF for image]
Source: State 2003-2004 report cards available on state Wed sites or
data provided by state officials.
Notes: We obtained data for 42 states from their state Web sites and
contacted state officials in 6 states to obtain these data. Three
states did not report data in a format that allowed us to determine
whether the percentage of students with limited English proficiency met
or exceeded the annual progress goals established by the state.
When states reported proficiency data for different grades or groups of
grades, we determined that students with limited English proficiency
met a state's progress goals if the student group met all proficiency
and participation goals for all grades reported. An Education official
told us that a state could not make adequate yearly progress if it
missed one of the progress goals at any grade level.
All of the states on the map where the proficiency percentage for
students with limited English proficiency met or exceeded the state's
annual progress goal also met NCLBA's participation goals.
We incorporated states' use of confidence intervals and NCLBA's safe
harbor provision in determining whether the percentage of students with
limited English proficiency achieving proficient scores met or exceeded
a state's progress goals. If a state's published data did not
explicitly include such information, we contacted state officials to
ensure that the state did not meet its progress goals through the use
of confidence intervals or through NCLBA's safe harbor provision. In
the following 7 states, the percentage of students with limited English
proficiency was below the state's annual progress goal for language
arts or for both language arts and mathematics, but the student group
met the state's requirements for progress through the safe harbor
provision: Delaware, Idaho, Maryland, Massachusetts, Oklahoma, Rhode
Island, and Utah.
We reported 2004-2005 school year data for Oklahoma, New Mexico, and
Utah because we could not obtain data for the 2003-2004 school year.
Data from Iowa, Massachusetts, and Rhode Island are for the 2002-2004
school years.
Rhode Island did not separately report participation rates for students
with limited English proficiency. Instead, it reported that all
students met the 95 percent participation goal.
[End of figure]
We found that the percentage of elementary school students with limited
English proficiency achieving proficient scores on the state's
mathematics assessment was lower than that for the total student
population in 48 of 49 states that reported to Education in school year
2003-2004. We also found that, in general, a lower percentage of
students with limited English proficiency achieved proficient test
scores than other selected student groups. All of the 49 states
reported that these students achieved lower rates of proficiency than
white students.[Footnote 5] The performance of limited English
proficient students relative to the other student groups varied. In 37
states, for example, economically disadvantaged students outperformed
students with limited English proficiency, while students with
disabilities outperformed these students in 14 states.
Selected States Considered Language Issues when Developing Academic
Assessments, but Validity and Reliability Concerns Remain:
Officials in the 5 states we studied reported that they have taken
steps to address challenges associated with academic assessments of
students with limited English proficiency. However, Education's peer
reviews of 38 states found a number of concerns in assessing these
students. Our group of experts indicated that states are generally not
taking the appropriate set of comprehensive steps to create valid and
reliable assessments for students with limited English proficiency. To
increase validity and reliability, most states offered accommodations
to students, such as providing extra time to complete the test and
offering native language assessments. However, offering accommodations
may or may not improve the validity of test results, as research in
this area is lacking.
States Reported Efforts to Improve Validity of Assessment Results for
Students with Limited English Proficiency:
Officials in 5 states we studied reported taking some steps to address
challenges associated with assessing students with limited English
proficiency. Officials in 4 of these states reported following
generally accepted test development procedures, while a Nebraska
official reported that the state expects districts to follow such
procedures.
Officials in California, New York, North Carolina, and Texas told us
that they try to implement the principles of universal design, which
support making assessments accessible to the widest possible range of
students. This is done by ensuring that instructions, forms, and
questions are clear and not more linguistically complex than necessary.
In addition, officials in some states reported assembling committees to
review test items for bias. For example, when developing mathematics
items, these states try to make language as clear as possible to ensure
that the item is measuring primarily mathematical concepts and to
minimize the extent to which it is measuring language proficiency. A
mathematics word problem involving subtraction, for example, might
refer to fish rather than barracuda. Officials in 3 of our study states
told us they also used a statistical approach to evaluate test items
for bias related to students with limited English proficiency.
Both Education's Peer Reviews and Our Group of Experts Raised Concerns
Regarding State Efforts to Ensure Valid and Reliable Assessment
Results:
Education's completed NCLBA peer reviews of 38 states[Footnote 6] found
that 25 did not provide sufficient evidence on the validity or
reliability of results for students with limited English proficiency.
For example, in Idaho, peer reviewers commented that the state did not
report reliability data for students with limited English proficiency.
As of March 2007, 18 states have had their assessment systems fully
approved by Education.[Footnote 7]
Our group of experts indicated that states are generally not taking the
appropriate set of comprehensive steps to create valid and reliable
assessments for these students and identified essential steps that
should be taken. These experts noted that no state has implemented an
assessment program for students with limited English proficiency that
is consistent with technical standards. They noted that students with
limited English proficiency are not defined consistently within and
across states, which is a crucial first step to ensuring reliability.
If the language proficiency levels of these students are classified
inconsistently, an assessment may produce results that appear
inconsistent because of the variable classifications rather than actual
differences in skills. Further, it appears that many states do not
conduct separate analyses for different groups of limited English
proficient students. Our group of experts indicated that the
reliability of a test may be different for heterogeneous groups of
students, such as students who are literate in their native language
and those who are not. Further, these experts noted that states are not
always explicit about whether an assessment is attempting to measure
skills only (such as mathematics) or mathematics skills as expressed in
English. According to the group, a fundamental issue affecting the
validity of a test is the definition of what is being measured.
The expert group emphasized that determining the validity and
reliability of academic assessments for students with limited English
proficiency is complicated and requires a comprehensive collection of
evidence rather than a single analysis. In addition, the appropriate
combination of analyses will vary from state to state, depending on the
characteristics of the student population and the type of assessment.
The group indicated that states are not universally using all the
appropriate analyses to evaluate the validity and reliability of test
results for students with limited English proficiency. These experts
indicated that some states may need assistance to conduct appropriate
analyses. Finally, they indicated that reducing language complexity is
essential to developing valid assessments for these students, but
expressed concern that some states and test developers do not have a
strong understanding of universal design principles or how to use them
to develop assessments that eliminate language barriers to measuring
specific skills.
Accommodations Can Increase Validity of Assessment Results, but
Research on Appropriate Use Is Limited:
The majority of states offered some accommodations to try to increase
the validity and reliability of assessment results for students with
limited English proficiency. These accommodations are intended to
permit students to demonstrate their academic knowledge, despite
limited language ability. Our review of state Web sites found
documentation on accommodations for 42 states. The number of
accommodations offered varied considerably among states. The most
common accommodations were allowing the use of a bilingual dictionary
and reading test items aloud in English (see table 1). Some states also
administered assessments to small groups of students or individuals,
while others gave students extra time to complete a test.
Table 1: Most Frequently Cited Accommodations in 42 States:
Accommodation: Bilingual dictionary;
Number of states: 32.
Accommodation: Reading items aloud in English;
Number of states: 32.
Accommodation: Small group administration;
Number of states: 29.
Accommodation: Extra time;
Number of states: 27.
Accommodation: Individual administration;
Number of states: 27.
Accommodation: Separate location;
Number of states: 25.
Accommodation: Extra breaks;
Number of states: 25.
Accommodation: Directions in student's native language;
Number of states: 24.
Source: GAO review of state documentation.
[End of table]
According to our expert group and our review of literature, research is
lacking on what specific accommodations are appropriate for students
with limited English proficiency, as well as their effectiveness in
improving the validity of assessment results. A 2004 review of state
policies found that few studies focus on accommodations intended to
address the linguistic needs of students with limited English
proficiency or on how accommodations affect the performance of students
with limited English proficiency.[Footnote 8] In contrast,
significantly more research has been conducted on accommodations for
students with disabilities, much of it funded by Education. Because of
this research disparity, our group of experts reported that some states
offer accommodations to students with limited English proficiency based
on those they offer to students with disabilities, without determining
their appropriateness for individual students. They noted the
importance of considering individual student characteristics to ensure
that an accommodation appropriately addresses the needs of the student.
Native Language and Alternate Assessments May Improve the Validity of
Results but Are Challenging to Implement:
In our survey, 16 states reported that they offered statewide native
language assessments in language arts or mathematics in some grades for
certain students with limited English proficiency in the 2004-2005
school year. For example, New York translated its statewide mathematics
assessments into Spanish, Chinese, Russian, Korean, and Haitian-Creole.
In addition, 3 states were developing or planning to develop a native
language assessment.
Our group of experts told us that this type of assessment is difficult
and costly to develop. Development of a valid native language
assessment involves more than a simple translation of the original
test. In most situations, a process of test development and validation
similar to that of the nontranslated test is recommended. In addition,
the administration of native language assessments may not be
practicable, for example, when only a small percentage of limited
English proficient students in the state speak a particular language or
when a state's student population has many languages. Members of our
expert group told us that native language assessments are generally an
effective accommodation only for students in specific circumstances,
such as students who are instructed in their native language or are
literate in their native language.
Thirteen states offered statewide alternate assessments (such as
reviewing a student's classroom work portfolio) in 2005 for certain
students with limited English proficiency, as of March 2006. Our expert
group noted that alternate assessments are difficult and expensive to
develop, and may not be feasible because of the amount of time required
for such an assessment. Members of the group also expressed concern
about the extent to which these assessments are objective and
comparable and can be aggregated with regular assessments.
Most States Implemented New English Language Proficiency Assessments
but Faced Challenges Establishing Their Validity:
Many states implemented new English language proficiency assessments
for the 2005-2006 school year to meet Education's requirement for
states to administer English language proficiency tests that meet NCLBA
requirements by the spring of 2006.[Footnote 9] These assessments must
allow states to track student progress in learning English.
Additionally, Education requires that these assessments be aligned to a
state's English language proficiency standards. Education officials
said that because many states did not have tests that met NCLBA
requirements, the agency funded four state consortia to develop new
assessments that were to be aligned with state standards and measure
student progress.
In the 2005-2006 school year, 22 states used assessments or test items
developed by one of four state consortia, making this the most common
approach taken by states. Eight states worked with test developers to
augment off-the-shelf English language proficiency assessments to
incorporate state standards. Officials in 14 states indicated that they
are administering off-the-shelf assessments. Seven states, including
Texas, Minnesota, and Kansas, created their own English language
proficiency assessments. Officials in these states said they typically
worked with a test developer or research organization to create the
assessments.
Officials in our study states and test developers we interviewed
reported that they commonly apply generally accepted test development
procedures to develop their assessments, but some are still in the
process of documenting their validity and reliability. A 2005 review of
the documentation of 17 English proficiency assessments used by 33
states found that the evidence on validity and reliability was
generally insufficient.[Footnote 10] The study, which was funded by
Education, noted that none of the assessments contained "sufficient
technical evidence to support the high-stakes accountability
information and conclusions of student readiness they are meant to
provide."
Education Has Provided Assistance, but States Reported Need for
Additional Guidance and Flexibility:
Education has offered states a variety of technical assistance to help
them appropriately assess students with limited English proficiency,
such as providing training and expert reviews of their assessment
systems. However, Education has issued little written guidance on how
states are expected to assess and track the English proficiency of
these students, leaving state officials unclear about Education's
expectations. While Education has offered states some flexibility in
how they incorporate these students into their accountability systems,
many of the state and district officials we interviewed indicated that
additional flexibility is needed to ensure that academic progress of
these students is accurately measured.
Education Has Provided a Variety of Support on Assessment Issues but
Little Written Guidance on Assessing Students with Limited English
Proficiency:
Education offers support in a variety of ways to help states meet
NCLBA's assessment requirements for students with limited English
proficiency. The department's primary technical assistance efforts have
included the following:
* Title I peer reviews of states' academic standards and assessment
systems: During these reviews, experts review evidence provided by the
state about the validity and reliability of these assessments.
Education shares information from the peer review to help states
address issues identified during the review.
* Title III monitoring visits: Education began conducting site visits
to review state compliance with Title III requirements in 2005. As part
of these visits, the department reviews the state's progress in
developing English language proficiency assessments that meet NCLBA
requirements.
* Comprehensive centers: Education has contracted with 16 regional
comprehensive centers to build state capacity to help districts that
are not meeting their adequate yearly progress goals. At least 3 of
these centers plan to assist individual states in developing
appropriate goals for student progress in learning English. In 2005,
Education also funded an assessment and accountability comprehensive
center, which provides technical assistance related to the assessment
of students, including those with limited English proficiency.
* Ongoing technical assistance for English language proficiency
assessments: Education has provided information and ongoing technical
assistance to states using a variety of tools and has focused
specifically on the development of the English language proficiency
standards and assessments required by NCLBA.
While providing this technical assistance, Education has issued little
written guidance on developing English language proficiency assessments
that meet NCLBA's requirements and on tracking the progress of students
in acquiring English. Education issued some limited nonregulatory
guidance on NCLBA's basic requirements for English language proficiency
standards and assessments in February 2003.
However, officials in about one-third of the 33 states we contacted
expressed uncertainty about implementing these requirements. They told
us that they would like more specific guidance from Education to help
them develop tests that meet NCLBA requirements, generally focusing on
two issues. First, some officials said they were unsure about how to
align English language proficiency standards with content standards for
language arts, mathematics, and science, as required by NCLBA. Second,
some officials reported that they did not know how to use the different
scores from their old and new English language proficiency assessments
to track student progress. Without guidance and specific examples on
both of these issues, some of these officials were concerned that they
will spend time and resources developing an assessment that may not
meet Education's requirements. Education officials told us that they
were currently developing additional nonregulatory guidance on these
issues, but it had not yet been finalized.
Education Has Offered Different Accountability Options for Students
with Limited English Proficiency, but State Officials Reported
Additional Flexibility Is Needed:
Education has offered states several flexibilities in tracking academic
progress goals for students with limited English proficiency to support
their efforts to develop appropriate accountability systems for these
students. For example, students who have been in U.S. schools for less
than a year do not have to meet the same testing requirements as other
students. Another flexibility recognizes that limited English
proficiency is a more transient quality than being of a particular
race. Students who achieve English proficiency leave the group at the
point when they demonstrate their academic knowledge in English, while
new students with lower English proficiency are constantly entering the
group (see fig. 2). Given the group's continually changing composition,
meeting progress goals may be more difficult than doing so for other
student groups, especially in districts serving large numbers of these
students. Consequently, Education allowed states to include, for up to
2 years, the scores of students who were formerly classified as limited
English proficient when determining whether a state met its progress
goals for students with limited English proficiency.
Figure 2: Movement of Students In and Out of Limited English Proficient
Student Group and Other Student Groups:
[See PDF for image]
Source: GAO analysis and Art Explosion images.
[End of figure]
Several state and local officials in our study states told us that
additional flexibility would be helpful to ensure that the annual
progress measures provide meaningful information about the performance
of students with limited English proficiency. Officials in 4 of the
states we studied suggested that certain students with limited English
proficiency should be exempt from testing or have their test results
excluded for longer periods than is currently allowed. Several
officials voiced concern that some of these students have such poor
English skills or so little previous school experience that assessment
results do not provide any meaningful information. Instead, some of
these officials stated that students with limited English proficiency
should not be included in academic assessments until they demonstrate
appropriate English. However, the National Council of La Raza, a
Hispanic advocacy organization, has voiced concern that excluding too
many students from a state's annual progress measures will allow some
states and districts to overlook the needs of these students.
With respect to including the scores of students previously classified
as limited English proficient for up to 2 years, officials in 2 of our
5 study states, as well as one member of our expert group, thought it
would be more appropriate for these students to be counted in the
limited English proficient group throughout their school careers--but
only for accountability purposes. They pointed out that by keeping
students formerly classified as limited English proficient in the
group, districts that work well with these students would see increases
in the percentage who score at the proficient level in language arts
and mathematics. An Education official explained that the agency does
not want to label these students as limited English proficient any
longer than necessary. Education officials also noted that including
all students who were formerly limited English proficient would inflate
the achievement measures for this group.
District officials in 4 states argued that tracking the progress of
individual students in this group is a better measure of how well these
students are progressing academically. Officials in one district
pointed to a high school with a large percentage of students with
limited English proficiency that had made tremendous progress with
these students, doubling the percentage of students achieving academic
proficiency. The school missed the annual progress target for this
group by a few percentage points, but school officials said that the
school would be considered successful if it was measured by how much
individual students had improved. In response to educators and
policymakers who believe such an approach should be used for all
students, Education initiated a pilot project in November 2005,
allowing a limited number of states to incorporate measures of student
progress over time in determining whether districts and schools met
their annual progress goals.[Footnote 11]
Prior Recommendations and Agency Response:
We made several recommendations to Education in our July 2006 report.
Specifically, we recommended that Education support additional research
on appropriate accommodations for these students and disseminate
information on research-based accommodations to states. We also
recommended that Education determine what additional technical
assistance states need to implement valid and reliable academic
assessments for these students and provide such assistance. Further, we
recommended that Education publish additional guidance with more
specific information on the requirements for assessing English language
proficiency and tracking student progress in learning English. Finally,
we recommended that Education explore ways to provide states with
additional flexibility in terms of holding states accountable for
students with limited English proficiency.
Education agreed with our first three recommendations and has taken a
number of steps to address them. In recognition of the challenges
associated with assessing students with limited English proficiency and
in response to GAO's report, Education initiated the LEP (Limited
English Proficient) Partnership in July 2006. Under the partnership,
Education has pledged to provide technical assistance and support to
states in the development of assessment options for states to use in
addressing the needs of their diverse student populations. Education's
partners in this effort include the National Council of LaRaza, Mexican
American Legal Defense and Educational Fund, Council of Chief State
School Officers, Comprehensive Center on Assessment and Accountability,
and the National Center on English Language Acquisition. All states
have been invited to participate in this effort. The partnership held
its first meeting in August 2006. In October 2006, officials from all
the states came together to discuss areas for which they need
additional technical assistance. As a result of these meetings,
Education is supporting a variety of technical assistance projects,
including the development of a framework on English language
proficiency standards and assessments, the development of guides for
developing native language and simplified assessments, and the
development of a handbook on appropriate accommodations for students
with limited English proficiency. Education officials told us that they
are planning the next partnership meeting for the summer of 2007 and
expect to have several of these resources available at that time.
Education did not explicitly agree or disagree with our recommendation
to explore additional options for state flexibility. Instead, the
agency commented that it has explored and already provided various
types of flexibility regarding the inclusion of students with limited
English proficiency in accountability systems. However, in January
2007, Education issued a blueprint for strengthening NCLBA, which calls
for greater use of growth models and the recognition within state
accountability systems of schools that make significant progress in
moving students toward English proficiency.
Mr. Chairman, this completes my prepared statement. I would be happy to
respond to any questions you or other members of the subcommittee may
have.
GAO Contacts:
For further information regarding this testimony, please contact me at
(202) 512-7215. Individuals making key contributions to this testimony
include Harriet Ganson, Bryon Gordon, Shannon Groff, Krista Loose,
Michelle St. Pierre, Sheranda Campbell, and Nagla'a El Hodiri.
[End of section]
Related GAO Products:
No Child Left Behind Act: Education's Data Improvement Efforts Could
Strengthen the Basis for Distributing Title III Funds. GAO-07-140.
Washington, D.C.: December 7, 2006.
No Child Left Behind Act: Education Actions Needed to Improve Local
Implementation and State Evaluation of Supplemental Educational
Services. GAO-06-758. Washington, D.C.: August 4, 2006.
No Child Left Behind Act: Assistance from Education Could Help States
Better Measure Progress of Students with Limited English Proficiency.
GAO-06-815. Washington, D.C.: July 26, 2006.
No Child Left Behind Act: States Face Challenges Measuring Academic
Growth That Education's Initiatives May Help Address. GAO-06-661.
Washington, D.C.: July 17, 2006.
No Child Left Behind Act: Improved Accessibility to Education's
Information Could Help States Further Implement Teacher Qualification
Requirements. GAO-06-25. Washington, D.C.: November 21, 2005.
No Child Left Behind Act: Education Could Do More to Help States Better
Define Graduation Rates and Improve Knowledge about Intervention
Strategies. GAO-05-879. Washington, D.C.: September 20, 2005.
No Child Left Behind Act: Most Students with Disabilities Participated
in Statewide Assessments, but Inclusion Options Could Be Improved. GAO-
05-618. Washington, D.C.: July 20, 2005.
Head Start: Further Development Could Allow Results of New Test to Be
Used for Decision Making. GAO-05-343. Washington, D.C.: May 17, 2005.
No Child Left Behind Act: Education Needs to Provide Additional
Technical Assistance and Conduct Implementation Studies for School
Choice Provision. GAO-05-7. Washington, D.C.: December 10, 2004.
No Child Left Behind Act: Improvements Needed in Education's Process
for Tracking States' Implementation of Key Provisions. GAO-04-734.
Washington, D.C.: September 30, 2004.
FOOTNOTES
[1] GAO, No Child Left Behind Act: Assistance from Education Could Help
States Better Measure Progress of Students with Limited English
Proficiency, GAO-06-815 (Washington, D.C.: July 26, 2006).
[2] To be deemed as having made adequate yearly progress for a given
year, each district and school must show that the requisite percentage
of each designated student group, as well as the student population as
a whole, met the state proficiency goal. Further, schools must also
demonstrate that they have met state targets on other academic
indicators, such as graduation rates or attendance. Alternatively, a
district or school can make adequate yearly progress through the "safe
harbor" provision if the percentage of students in a group considered
not proficient decreased by at least 10 percent from the preceding year
and the group made progress on one of the state's other academic
indicators. States also use statistical procedures, such as confidence
intervals, to improve the reliability of the results used to determine
adequate yearly progress.
[3] On September 13, 2006, Education issued a final regulation on this
flexibility. 71 Fed. Reg. 54188 (Sept. 13, 2006).
[4] In 7 of the 17 states, students with limited English proficiency
met a state's adequate yearly progress goals through NCLBA's safe
harbor provision--that is, by decreasing the percentage of students
scoring nonproficient by 10 percent or more and showing progress on
another academic indicator.
[5] Student groups are not mutually exclusive, with each of the ethnic
and racial categories probably including some number of students with
limited English proficiency. For example, the results for a student who
is both white and limited English proficient would be included in both
groups.
[6] As of July 2006, Education had conducted peer reviews of 50 states
and the District of Columbia. However, detailed peer review notes were
available from only 38 states at the time of our review.
[7] Education's approval is pending for 29 states, while approval is
expected for an additional 3 states. Mississippi has received a waiver
from peer review approval for 1 year due to Hurricane Katrina.
[8] Charlene Rivera and Eric Collum. An Analysis of State Assessment
Policies Addressing the Accommodation of English Language Learners. The
George Washington University Center for Equity and Excellence in
Education, Arlington, Virginia: (January 2004).
[9] Education officials told us that the agency has approved an
extension of this deadline for 1 state and is currently considering
extension requests from 2 other states.
[10] Stanley Rabinowitz and Edynn Sato, "Evidence-Based Plan: Technical
Adequacy of Assessments for Alternate Student Populations: A Technical
Review of High-Stakes Assessments for English Language Learners,"
WestEd (December 2005).
[11] See GAO, No Child Left Behind Act: States Face Challenges
Measuring Academic Growth That Education's Initiatives May Help
Address, GAO-06-661 (Washington, D.C.: July 17, 2006), for further
information on Education's pilot project.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
