annual technical report for english language proficiency ... english language learners with...

Download Annual Technical Report for English Language Proficiency ... English Language Learners with Significant

Post on 16-Mar-2020

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • World-Class Instructional Design and Assessment

    Annual Technical Report for

    ALTERNATE ACCESS for ELLs® English Language Proficiency Test, Series 103, 2015-2016 Administration

    Annual Technical Report No. 4

    Prepared by:

    WIDA Psychometrics/Research Team

    WIDA August 31, 2017

  • Table of Contents 1. Description of Alternate ACCESS for ELLs English Language Proficiency Test ...........1

    Purpose of Alternate ACCESS for ELLs ....................................................................1 Format of Alternate ACCESS ....................................................................................1

    1.2.1 Integration with the Standards ..........................................................................1 1.2.2 Grade-level Clusters ........................................................................................2 1.2.3 Language Domains ..........................................................................................2 1.2.4 Language Proficiency Levels ...........................................................................2

    Test Development ......................................................................................................3 1.3.1 Item Development ...........................................................................................3 1.3.2 Field Test ........................................................................................................4 1.3.3 Scaling ............................................................................................................4 1.3.4 Standard Setting ..............................................................................................5

    Reporting of Results ...................................................................................................7 1.4.1 Scale Scores ....................................................................................................7 1.4.2 Language Proficiency Level Scores .................................................................8

    Test Administration ....................................................................................................9 1.5.1 Test Administrator Training .............................................................................9 1.5.2 Test Security....................................................................................................9 1.5.3 Test Accommodations .....................................................................................9

    Scoring ......................................................................................................................9 1.6.1 Listening and Reading .....................................................................................9 1.6.2 Writing .......................................................................................................... 10 1.6.3 Speaking ........................................................................................................ 11

    2. An Assessment Use Argument for Alternate ACCESS for ELLs: Focus on Assessment Records ......................................................................................................... 13

    The Generic Validation Framework for Alternate ACCESS ..................................... 14 Focus on Assessment Records .................................................................................. 16

    2.2.1 Breakdown of Claims for the Assessment Records Produced in the Alternate ACCESS for ELLs Assessment Program ........................................ 17

    Evidence for Assessment Records Claims of Alternate ACCESS for ELLs............... 18 Summary of Assessment Records Claims, Actions, and Evidence ............................ 24 Visual Guide to Tables and Figures .......................................................................... 26

    2.5.1 Chapter 4 Visual Guide to Tables and Figures................................................ 27 2.5.2 Chapter 6 Visual Guide to Tables and Figures................................................ 28

    3. Descriptions of Student Results ...................................................................................... 29 Participation ............................................................................................................. 29 Scale Score Results .................................................................................................. 29

    3.2.1 Mean Scale Scores Across Domain and Composite Scores............................. 29 3.2.2 Correlations ................................................................................................... 30

    Proficiency Level Results ......................................................................................... 30 Participation by Disability ........................................................................................ 31

    4. Student Results ................................................................................................................ 32 Students excluded from Analysis.............................................................................. 32

    4.1.1 Out-of-grade-level Test Administration ......................................................... 32

  • Participation by Grade-level Cluster ......................................................................... 33 4.2.1 Participation by Grade-level Cluster by State ................................................. 33 4.2.2 Participation by Grade-level Cluster by Gender.............................................. 34 4.2.3 Participation by Grade-level Cluster by Ethnicity .......................................... 34

    Participation by Grade .............................................................................................. 35 4.3.1 Participation by Grade by State ...................................................................... 35 4.3.2 Participation by Grade by Gender .................................................................. 36 4.3.3 Participation by Grade by Ethnicity ................................................................ 36

    Participation by Domain ........................................................................................... 37 4.4.1 Participation by Grade-level Cluster by Domain ............................................ 37 4.4.2 Participation by Grade by Domain ................................................................. 37

    Scale Scores by Domain and Composite ................................................................... 38 4.5.1 Mean Scale Scores by Domain and Composite............................................... 38

    Scale Scores by Grade-level Cluster ......................................................................... 39 4.6.1 Mean Scale Scores by Gender ........................................................................ 39 4.6.2 Mean Scale Scores by Ethnicity ..................................................................... 41

    Scale Scores by Grade .............................................................................................. 44 4.7.1 Mean Scale Scores by Gender ........................................................................ 44 4.7.2 Mean Scale Scores by Ethnicity ..................................................................... 49

    Correlations among Scale Scores by Grade-level Cluster .......................................... 60 4.8.1 Correlations among Scale Scores: Grade-level Cluster 1-2 ............................. 60 4.8.2 Correlations among Scale Scores: Grade-level Cluster 3-5 ............................. 61 4.8.3 Correlations among Scale Scores: Grade-level Cluster 6-8 ............................. 61 4.8.4 Correlations among Scale Scores: Grade-level Cluster 9-12 ........................... 61

    Proficiency Level Results ......................................................................................... 62 4.9.1 Proficiency Level by Grade-level Cluster ....................................................... 62 4.9.2 Proficiency Level by Grade ............................................................................ 64

    Participation by Disability ........................................................................................ 69 4.10.1 Participation by Disability ............................................................................. 69

    5. Analyses of Test Forms: Overview .................................................................................. 70 Background.............................................................................................................. 70

    5.1.1 Measurement Models Used ............................................................................ 70 5.1.2 Sampling ....................................................................................................... 71 5.1.3 Scaling .......................................................................................................... 71 5.1.4 DIF Analyses ................................................................................................. 72 5.1.5 Reliability of Composites............................................................................... 73 5.1.6 Accuracy and Consistency of Classification ................................................... 73

    Descriptions ............................................................................................................. 75 5.2.1 Raw Score Information (Figure A and Table A) ............................................. 75 5.2.2 Scal

Recommended

View more >