How States Reported Participation and Performance of English Language Learners in State Assessments

Project conducted in collaboration with the Center for Excellence and Equity in Education, supported by the U.S. Department of Education, OBEMLA, now referred to as OELA.

Research Questions

• To what extent was the participation and performance of ELLs in state assessments being publicly reported?

• To what extent were disaggregated data on accommodated and native language assessments offered?

• What trends are evident in public reporting on ELLs over time?

Research Questions (cont.)

• What trends are evident in reporting in states with high and low ELL enrollment?

• What characteristics of public reports are viewed as best practices for presenting both usable and understood data on LEP students?

Importance of the Study

• Reporting all students’ performance is a required element of NCLB

• NCLB mandates that reported data need to be available to and usable by teachers and administrators as well as parents

• Baseline for future studies of states’ efforts

• Parallels NCEO study on students with disabilities

While it is widely suggested that state and district assessments, and the reporting of their results, has an impact on teaching and learning (Elmore & Rothman, 1999), reporting data is not enough.

“Student performance data must be made available to teachers and the public in ways that spark creative responses”

Ruiz-de-Velasco & Fix, 2000.

The purpose of disaggregation is to gauge whether specific subgroups of students perform at different levels than other subgroups (i.e., whether there is an “achievement gap”) so that interventions can be designed and implemented if needed. Disaggregation is also useful in informing the public about how well a school or district is doing in helping all students to reach state standards. (p.11).

Vincent & Schenck (2001)

A part of the hoped for response to data is

that teachers and administrators can identify what is working for students so that appropriate interventions are implemented for non-English language background students who may have specific needs

(Liu, Albus, & Thurlow, 2000).

Method

• Gathered all available assessment reports as of March 2001 – both public print documents and data on state Web sites

• Examined only data for tests administered during 1999-2000 (dropped 3 states with 1998-99 data)

• Verification letters sent to all assessment directors – 15 responded with corrections or additions; used information from 13 of these states (others were not public)

Reported Data 1999-2000

MNWI

ILIN OH

IA

MO

MI

ND

SD

NE

MT

WY

CO

KS

OK

TX

ARNM AZ

UTNV

IDOR

WA

CA

LA

MSAL GA

FL

SC

NCTN

KY VA WV

NY

PA

MD

DE

NJ

CTRI

MA

ME

NHVT

DC= no data

AK

HI Reported both participation and performance of ELLs for at least one test (n=16)

Reported only performance of ELLs for at least one test (n=3) 

No ELL participation or performance reported (n=32)

                   

Comprehensiveness of Reporting 1999-2000

MNWI

ILIN OH

IA

MO

MI

ND

SD

NE

MT

WY

CO

KS

OK

TX

ARNM AZ

UTNV

IDOR

WA

CA

LA

MSAL GA

FL

SC

NCTN

KY VA WV

NY

PA

MD

DE

NJ

CTRI

MAME

NHVT

DC= no data

AK

HI

ELL performance reported for all tests (n=11)

ELL performance reported for some tests (n=8) 

No ELL participation or performance reported (n=32)

Content Reporting on ELLs

19 of 46 states that tested E/LA and math content reported data on ELLs (41%)

13 of 34 states that tested science (38%)

13 of 39 states that tested writing (33%)

11 of 37 states that tested social studies (30%)

Content Areas

Reading 4 12 3 19

Math 4 12 3 19

Writing 1 10 1 12

Science 4 7 2 13

Social Studies 4 5 2 11

Part R

ATE and

Perfo

rman

ce

Part N

UMBER

and P

erfo

rman

ce

Perfo

rman

ce

Only Total

States

0

10

20

30

40

50

60

70

80

Per

cen

tag

e o

f EL

Ls

State 1 State 2 State 3 State 4

Sample Participation Rate Data

  

  

 

      

Elementary Reading

0

10

20

30

40

50

60

70

80

90

100

CO DE FL ID IN KY LA ME MA NH NJ NC RI TX WI

States

Per

cent

Pro

ficie

nt a

nd A

bove

All Students

ELLs

Example of Gaps in Performance

Reporting on Accommodated and Other Assessments

Three states reported data for ELLs taking tests with accommodations

(Several states indicated only that nonstandard administrations are not reported)

Four states reported data on student performance on native language assessments

1995-98 Data 1999-2000 Data

Arizona California Massachusetts

Delaware Colorado New Hampshire (perf only)

Georgia Delaware New Jersey

Kansas (part only) Florida New Mexico (perf only)

New Jersey (part only) Idaho North Carolina

North Carolina (part only) Illinois Rhode Island (perf only)

Rhode Island Indiana Texas

Virginia Kentucky Virginia

Louisiana Wisconsin

Maine

Trends In Reporting Across Time

Reporting in States with Large and Small Populations

Of the top and bottom 10 states in K-12 ELL enrollment:

Top 10 Bottom 10

Reported ELL Data on:

All Tests 4 4

Some Tests 3

No Tests 3 6

Unique Reporting Practices (in 1999-2000)

Highlighted Descriptions About:

MN

WI

IL IN OH

IA

MO

MI

ND

SD

NE

MT

WY

CO KS

OK

TX

ARNMAZ

UTNV

IDOR

WA

CA

LA

MS

AL GA

FL

SC

NCTN

KY VAWV

NY

PA

MD

DENJCT RI

MA

ME

MENH

VT

AK

HI

DC

Interactive Reporting Online

Example from Delaware:

Table 11-9. Customized Disaggregated Data Report From the Web Reading Performance Levels Location/Group

Grade Year N 5 4 3 2 1

State of Delaware / LEP, Low Income

8 2000 23 0.00% 0.00% 13.04% 26.09% 60.87%

State of Delaware / LEP, Not Low Income

8 2000 16 0.00% 12.50% 31.25% 18.75% 37.50%

State of Delaware / LEP

8 2000 39 0.00% 5.13% 20.51% 23.08% 51.28%

State of Delaware / All Students

8 2000 8088 1.99% 6.95% 58.48% 16.39% 16.18%

Delaware Department of Education (2000). Delaware Student Testing Program Online Reports. at http://delsis.doe.state.de.us/DSTPPublic/

Things to Consider about Interactive Online Reporting…

• Can keep part of the “bigger picture” (e.g., all students tested) in a report when narrowing to smaller subgroups by specific characteristics

• May allow flexibility in combining student characteristics to analyze data

• May be limited for small subpopulations whose numbers are not able to be reported out of concern for student privacy

Things to Consider…

• Can not assume most recent data is available in all online reports – interactive databases may not be updated at the same pace. This needs to be clear.

• May vary in ease of use and interpretation just as other reporting formats

Some Conclusions . . .

• Need clarification of terms used in reporting, and policy information for interpreting data

• Need more data on accommodated and native language assessments, so can better understand their role in an assessment system

• Overall, more states need to report – and this has happened!

Reporting Needs

• Be Clear, Concise, timely, and publicly accessible

• Include participation and performance data at every grade by content area aggregated and disaggregated

• Provide data so readers will not have to do calculations

• Report accommodated results separately and in aggregate

• Ensure proficiency level (e.g., basic, proficient, advanced) is reported in the same manner for ELLs as for mainstream

What can educators do?

Look for your state data onlineGive your state feedback about whether ELL data is comprehensive and clearTalk about the ELL data in your building/district—what does it mean for you?Know whether your state reports in different languages

For more information

http://education.umn.edu/NCEO/