1

2009 MEPAReporting

Teleconference

September 2009

2

Presenters

• Kit Viator, Director of Student Assessment

• Dan Wiener, Administrator of Inclusive Assessment

• Bob Lee, Chief Analyst

3

Today’s Agenda• Welcome • MEPA Overview and Updates • 2009 MEPA Reporting: What’s New and Why?• New Performance Levels and Reporting Scale• Standard Setting: Setting the Cut Score• What the Scores Mean

– Making Reclassification Decisions– Planning Instruction

• Important New Resources• Questions and Answers

4

MEPA Overview• The state’s English language proficiency assessment of LEP

students in grades K-12, based on English Language Proficiency Benchmarks and Outcomes– MEPA-R/W and MELA-O

• Fall testing: Test only those LEP students in grades 1-12 who did not participate in spring 2009 (no fall testing for Kindergarten)

• Spring testing: Test all students in K-12 reported as “LEP” in October 2009 or March 2010 SIMS

• Sub-scaled scores in four tested areas– Reading: 0-30 points– Writing: 0-30 points– Listening: 0-5 points– Speaking: 0-20 points

NOTE: Sub-scaled

scores Overall scaled

score

5

Fall 2009 MEPA-R/W Tests

Grade Span Which Test & Sessions to Administer

1-2

(K-2 in spring 2010)

Level A (one session for Reading and one for Writing)

ORLevel B (one session for Reading and one for Writing)

3-12(grade span tests:

3-4, 5-6, 7-8, 9-12)

Sessions 1 and 2 OR Sessions 2 and 3

6

MELA-O (Listening and Speaking)QMT/QMA Retraining and Requalification

• By January 31, 2010, all QMT/QMAs must be requalified in order to continue to report MELA-O scores for MEPA

• Purpose: To improve consistency and reliability of student ratings

• Fall 2009 QMT Training schedule– Retrain current QMTs: October 13, Westborough– Train new QMT (if no QMT in district):

• September 21-22, Westborough• November 3-4, Norwood

7

Spring 2009 MEPA:What’s New?

• Redesigned tests for grade spans 3-4, 5-6, 7-8, and 9-12

• New K-2 tests• New test questions based primarily on

authentic reading passages• Wide range of difficulty of test questions,

and more constructed responses• Locator survey (K-2)/locator tests (3-12)• New reporting scale: 400-550• New performance levels: Levels 1-5

8

New Tests for Students in K-2

• Student takes either Level A or B, depending on – Level of English proficiency and literacy– Results of K-2 Locator Survey

• What’s tested on K-2 MEPA-R/W:

– Level A: (R) alphabet recognition; match high-frequency words with pictures; answer questions based on story read aloud; (W) write or complete a dictated word; write the word for a picture; write a two-sentence story

– Level B: (R) match or write a word/sentence that matches picture; questions based on story read by student; (W) correct punctuation; letter writing to complete sentence; write a story based on 3 pictures

9

MEPA Reporting: 2005-2008

Beginning

300

Early Inter-

mediate

Inter-mediate Transitioning

400

MEPA Performance Levels

10

Why We Needed New MEPA Performance Levels

395

Many students begin to demonstrate proficiency on MCAS ELA tests

300305

400

2005-2008 MEPA

Beginning

Early Inter-

mediate

Inter-mediate Transitioning

11

MEPA 2005-2008: Distribution of Scaled Scores

• Students at low Beginning and high Transitioning levels were clustered into lower and upper ends of scale.

300 320 340 360 380 400

ss

0

1,000

2,000

3,000

4,000

5,000

Mean = 363.44Std. Dev. = 29.103N = 32,562

300 320 340 360 380 400

1,000

2,000

3,000

4,000

5,000

# S

TU

DEN

TS

SCALED SCORE

0

12

MEPA Reporting: 2009

MEPA Performance Levels

Level 1

Level 2

Level 3

Level 4

Level 5

400 550

13

New MEPA Performance Levels

A student at:• Level 1 cannot yet communicate in English

• Level 2 communicates using simple written and spoken English, with frequent errors

• Level 3 communicates using basic English with occasional errors

• Level 4 is generally fluent, with few errors

• Level 5 communicates effectively in English across all academic subjects

14

Standard Setting: Summer 2009

• Performance levels describe performance at five levels

• Panels of ELL and general educators spent 3 days last summer with ESE determining where one performance level ended and another began (“cut score”)

• Recommendations of panelists for Levels 1-4 were adopted.

• Level 5 was based on panelists’ recommendations plus other evidence, including– Percent of LEP students Proficient on MCAS ELA– Statewide reclassification rate…and was fixed at 500-550 on each test.

15

Reporting Scale Expanded to 400-550

MEPA Scaled-Score Cut Points

Grade Span

Level 1 Level 2 Level 3 Level 4 Level 5

K-2 400 453 466 485 500

3-4 400 432 452 474 500

5-6 400 436 456 479 500

7-8 400 443 464 486 500

9-12 400 450 464 489 500

16

2009 MEPA Grade K-12 Overall Scaled Score Distribution

400400 425 450 475 500 525 550

ss

0

500

1,000

1,500

2,000

2,500

Frequency

Mean = 479.89Std. Dev. = 22.433N = 55,045

# S

TU

DEN

TS

SCALED SCORE550

17

Sample Parent/Guardian Report: Grade 7

• Very similar in appearance to 2005-2008 P/G Report, showing:– student’s score– location within the performance level – likely range of scores if student took the test

multiple times

Level 1 Level 5Level 2 Level 3 Level 4

18

Sample Parent/Guardian Report: Grade 7 (Continued…)

• Relative strengths and weaknesses in four tested areas

19

• Score compared with other students who have been in Massachusetts for 1 to 5+ years– About 45% of students who have been in Massachusetts for

3 years were in Level 3 on this MEPA test, as was this student.

Sample Parent/Guardian Report: Grade 7 (Continued…)

20

Using MEPA Results to Make Reclassification Decisions

• Level 1 and Level 2: Not a candidate for reclassification

• Level 3: Not a likely candidate for reclassification

• Level 4: May be a candidate for reclassification, if other indicators are aligned*

• Level 5: Reclassification recommended, if other indicators are aligned*

* MCAS scores, grades, local assessments, observations

21

Available Now• Guidance on Using MEPA Results to Plan

Sheltered English Immersion (SEI) Instruction and Make Reclassification Decisions for Limited English Proficient (LEP) Students (2009)

• Guide to Interpreting the 2009 MEPA Reports for Schools and Districts

(Both available at: www.doe.mass.edu)

• School “Drop Box” with pre-loaded student results for 2009 MEPA and 2008-2009 MCAS (Department’s Secure Internet Portal)

22

Accessing MEPA Results

• Go to www.mcasservicecenter.com• Click “MEPA”• Then “MEPA Reporting”• Then “Continue”• Log In

23

Accessing MEPA Results (Continued…)

• Choose your district from the dropdown menu

• Select your school or District Office from the school menu

• Enter your MCAS/MEPA five-digit password and login

24

• Select “All Grades” under Administration

• Check the reports you want to view; then “Download”

• If you have trouble viewing the zipped files, click the blue “here” link for help uploading free unzipping software

Accessing MEPA Results (Continued…)

25

Accessing “Dropbox Central” for Principals or their designees

• https://www4.doemass.org

• Log in with your Security Portal password

• Principals and those assigned the MEPA school role by your directory administrator can access “Dropbox Central”

• Select the “2009 MEPA and 2008-2009 MCAS” folder

• Download the .TXT file which can be opened using Excel or other spreadsheet software