Practical Guidelines for the Education of English Language Learners

David J. Francis, Ph.D.Texas Institute for Measurement, Evaluation, and Statistics

Center on Instruction – ELL StrandUniversity of Houston

Presented at LEP Partnership MeetingWashington, DCOctober 28, 2006

The Center on Instruction is operated by RMC Research Corporation in partnership with the Florida Center for Reading Research at Florida State

University; RG Research Group; the Texas Institute for Measurement,Evaluation, and Statistics at the University of Houston; and the Vaughn

Gross Center for Reading and Language Arts at the University of Texas at Austin.

The contents of this PowerPoint were developed under cooperative agreement S283B050034 withthe U.S. Department of Education. However, these contents do not necessarily

represent the policy of the Department of Education, and you should notassume endorsement by the Federal Government.

2006

The Center on Instruction requests that no changes be made to the content or appearance of this product.

To download a copy of this document, visit www.centeroninstruction.org

Practical Guidelines for the Education of ELLsCollaborators: Nonie Lesaux, GSE, Harvard University Mabel Rivera, COI, TIMES, University of Houston Michael Kieffer, GSE, Harvard University Hector Rivera, COI, TIMES, University of Houston

Practical Guidelines for the Education of ELLs Three books:

Research-based Recommendations for Instruction and Academic Interventions

Research-based Recommendations for Serving Adolescent Newcomers

Research-based Recommendations for the Use of Accommodations in Large-scale Assessments

Guide to working with the Documents Single document with separate sections vs. separate

documents Intent was for documents that would work together, but

which could also stand on their own Redundancy of some background material

Conventions Footnotes – for information the reader might need

immediately while reading Endnotes – for references and more detailed explication of

the literature behind a point

Key Reference MaterialsInstruction and Intervention

August & Shanahan (2006) Genessee, Lindholm-Leary, Saunders, & Christian (2006)

Newcomers August & Shanahan (2006); Genessee et al. (2006) Short & Boyson (2004)

Accommodations Abedi, Hofstetter, & Lord (2004). Sireci, Li, & Carpati (2003) Rivera, Collum, & Shafner Willner (2006)

ELLs and NCLB Membership is defined by limited proficiency in an area that

directly affects learning and assessment Group membership is expected to be temporary ELLs face unique set of learning challenges:

to develop the content-related knowledge and skills that define state standards

while simultaneously acquiring a second language, and particularly in the case of young children, at a time when their

first language is not fully developed, and (possibly) to demonstrate their learning on an assessment in the

second language

ELLs and NCLB ELLs present a unique set of challenges to

Teachers Administrators Assessment Systems Accountability Systems

Academic Language is the Key Organizing Principle See Scarcella (2003) for detailed introduction to

Academic Language Development of academic language

is fundamental to academic success in all domains is the primary source of ELLs difficulties with academic

content at all ages and grades can remain a challenge even after students achieve

proficiency on current state language proficiency tests affects ELLs performance on large-scale assessments

Academic Language Impossible to overstate the role that academic

language plays in determining students’ success Good conversational English skills may be

accompanied by limited academic language skills For example, in studies of elementary and middle school

students, including those no longer designated as LEP, mean vocabulary scores below the 20th percentile are not uncommon.

Academic Language Academic Language is comprised of many skills

Vocabulary knowledge (both depth and breadth) Depth – knowing multiple meanings, both common and

uncommon, for a given word Breadth – knowing the meanings of many words, including

multiple words for the same, or related, concepts Written vocabulary as distinct from oral vocabulary Understanding of complex sentence structures and syntax Understanding the structure of argument, academic discourse,

and expository texts

General Organization Foreword Overview

Description of the Population Organization and General Methods

Recommendations for …(multiple sections) Conceptual Framework Specific Recommendations

Who Are English Language Learners? Comprise one of the fastest-growing groups among the

school-aged population in this nation Over 9M students, roughly 5.5M classified as LEP ELL school-aged population has grown by more than

169% from 1979 to 2003 (vs. 12% growth in general) Expected to be 30% of school-aged population in 2015 Over 400 different home languages are represented Spanish is the predominant home language (70%)

Who Are English Language Learners? Largest and fastest growing ELL populations are

Students who immigrated before Kindergarten, and U.S. born children of immigrants

Compared to native English-speaking peers: On Grade 4 NAEP, ELLs were 1/4th as likely to score proficient or

above in Reading and 1/3rd as likely in Math ELLs are less likely to score proficient on State tests

These results may be biased in so far as they reflect performance of students who retain the designation of LEP

Who Are English Language Learners? Some states have begun to look at the performance of ELLs

on State tests after they have gained proficiency in English These reports show that some ELL students do well in

school Nevertheless, many students who have lost the formal LEP

designation continue to struggle with academic text, content, and language

The documents were written with this latter group of current and former LEP students in mind

Book 1: Instruction and Intervention Foreword Overview Reading

Conceptual Framework Recommendations

Mathematics Conceptual Framework Recommendations

Guiding Principles for Planning Instruction and Intervention

Reading is fundamental to the development of content-area knowledge and academic success

Applies to all learners We distinguish three functions for instruction:

augmentation, prevention, remediation To be effective, educators must have a clear

understanding of the specific sources of difficulty or weakness for individual students and groups of students

Guiding Principles for Planning Instruction and Intervention

ELLs often lack the academic language necessary for comprehending and analyzing text

The great majority of ELLs experiencing reading difficulties struggle with the skills related to Fluency Vocabulary Comprehension

These areas are mutually interdependent

Recommendations on Reading Instruction and Intervention ELLs need early, explicit, and intensive instruction

in phonological awareness and phonics in order to build decoding skills These skills are highly correlated across alphabetic

languages (i.e., correlations above .9) K-12 classrooms across the nation must increase

opportunities for ELLs to develop sophisticated vocabulary knowledge

Recommendations on Reading Instruction and Intervention Reading instruction in K-12 classrooms must equip

ELLs with strategies and knowledge to comprehend and analyze challenging narrative and expository texts

Instruction and intervention to promote ELLs’ reading fluency must focus on vocabulary development and increased exposure to print

Recommendations on Reading Instruction and Intervention In all K-12 classrooms across the U.S., ELLs need

significant opportunities to engage in structured, academic talk

Independent reading is beneficial, BUT it must be structured and purposeful, and there must be a good reader-text match

Recommendations on Mathematics Instruction and Intervention Generally much less research to guide recommendations Academic language is as central to mathematics as it is

to other academic areas a significant source of difficulty for many ELLs who

struggle with mathematics ELLs need early, explicit, and intensive instruction and

intervention in basic mathematics concepts and skill ELLs need academic language support to understand

and solve the word problems that are often used for mathematics assessment and instruction

Book 2: Adolescent Newcomers Foreword Overview Elements of Effective

Instruction Organizational Elements

of Effective Programs

Elements of Effective Instruction for Adolescent Newcomers Short & Boyson (2004) August & Shanahan (2006) All middle and secondary school classrooms must

address the language and literacy skills adolescent newcomers need for content area learning

All adolescent newcomers need instruction in academic language, which they need for text comprehension and school success

Elements of Effective Instruction for Adolescent Newcomers Adolescent newcomers need direct, explicit instruction

to support their comprehension of challenging texts Adolescent newcomers must receive intensive

instruction in writing for academic purposes Effective classroom instruction begins with systematic

assessment of students’ strengths and needs, as well as ongoing monitoring of students’ progress

Students with word-reading difficulties need targeted and explicit intervention

Organizational Elements of Effective Newcomer Programs Empirical research is limited Systematic support for assessment and placement of

students Heterogeneous grouping Extended instructional time Coordinated efforts: newcomer programs, programs

for advanced ELLs, and mainstream classes Targeted resources for language and literacy

instruction

Book 3: Accommodations Foreword Overview Review of State Policies Meta-analysis Technical Appendices

Use of Accommodations in Large-scale Assessments Rivera, Collum, & Shafer Willner (2006) Abedi, Hofstetter, & Lord (2004). Sireci, Li, & Carpati (2003)

Content Knowledge and Language Proficiency Assessments of content knowledge are influenced by

students’ language proficiency Assessments with the most linguistically challenging content

show the largest performance gaps between ELLs and native English speakers

It is easier to separate language proficiency from content knowledge in some domains (e.g., mathematics) than in others (e.g., reading language arts)

Appropriate accommodations for ELLs will address their linguistic needs either directly or indirectly

State Policies Educational agencies across the nation provide

accommodations to ELLs as needed The criteria for selection and strategies for

implementation vary by state, according to many factors

Rivera, Collum, & Shafer Willner (2006) have developed a comprehensive taxonomy for thinking about accommodations

Partial Listing of Accommodations Responsive to Needs of ELLs Accommodations of

Testing Conditions Extended time* Breaks offered

between sessions Bilingual

glossaries* Bilingual

dictionaries* English glossaries* English

dictionaries*

Accommodations as Test Modifications Directions read in English Directions read in native

language Directions translated into

native language Simplified English* Side-by-side bilingual

version of the test* Native language test* Dictation of answers

or use of a scribe Test taker responds

in native language

Criteria for Evaluation of Accommodations Effectiveness

Do ELL students who receive the accommodation outperform ELL students who do not receive the accommodation?

Validity Does the accommodation alter the construct validity of

the test? Do non-ELL students who receive the accommodation outperform non-ELL students who do not receive the accommodation?

Practicality

Effective Accommodations for ELLs: Results of a Meta-Analysis 11 studies

Each study used random assignment of ELLs and non-ELLs to testing conditions with and without accommodations

Involved 37 different samples of students Reported 37 different tests of the effectiveness of

accommodations for ELLs

Study Descriptions Grades included

4th: n=11 8th: n=22 5th or 6th: n=2 each

Subject Areas Math: n = 17 Science: n=19 Reading: n=1

Type of test NAEP items: n=22 NAEP and TIMSS: n=6 State Accountability Assessment: n=9 (two different states)

Study Descriptions (cont.) Types of accommodations

Simplified English (n=15) English dictionary/glossary (n=11) Bilingual dictionary/glossary (n=5) Extra time (n=2) Spanish language test (n=2) Dual language questions (n=1) Dual language booklet (n=1)

Results for Fixed Effects ModelResults for Fixed Effects Analysis

Effect Size and 95% Confidence Interval

Test of Mean Effect = 0

Test of Heterogeneity in Effect Sizes Accommodation Number

of Studies Mean

Effect Size

s.e. Lower Limit

Upper Limit Z p Q df(Q) p(Q)

Bilingual Dictionary-Glossary 5 -.096 .065 -.223 .031 -1.479 .139 13.53 4 .009

Dual Language Booklet 1 -.177 .148 -.467 .112 -1.199 .231

Dual Language Questions + Read Aloud in Spanish

1 .273 .195 -.109 .654 1.401 .161

English Dictionary-Glossary 11 .146 .043 .063 .230 3.427 .001 14.804 10 .139

Extra Time 2 .209 .142 -.069 .488 1.473 .141 0.155 1 .693

Simplified English 15 .020 .043 -.064 .104 .473 .637 19.830 14 .136

Spanish Version 2 -.263 .102 -.463 -.062 -2.572 .010 14.465 1 <.001

TOTAL WITHIN 62.789 30 <.001

TOTAL BETWEEN 25.540 6 <.001

OVERALL MEAN 37 .034 .025 -.016 .084 1.342 .180 87.330 36 <.001

Results for Random Effects ModelResults for Random Effects Analysis

Effect Size and 95% Confidence Interval

Test of Mean

Effect = 0

Test of Heterogeneity in

Effect Sizes Accommodation Number of

Studies Mean Effect Size

s.e. Lower Limit

Upper Limit Z p Q df(Q) p(Q)

Bilingual Dictionary-Glossary

5 -.039 .131 -.285 .217 -.298 .766

Dual Language Booklet 1 -.177 .148 -.467 .112 -

1.199 .231

Dual Language Questions + Read Aloud in Spanish

1 .273 .195 -.109 .654 1.401 .161

English Dictionary-Glossary

11 .178 .055 .070 .287 3.232 .001

Extra Time 2 .209 .142 -.069 .488 1.473 .141

Simplified English 15 .018 .061 -.102 .138 0.292 .771

Spanish Version 2 .302 .719 -1.107 1.711 .420 .674

TOTAL WITHIN

TOTAL BETWEEN 9.864 6 <.131

OVERALL MEAN 37 .092 .036 .021 .162 2.550 .011

Summary of Results Of the seven types of accommodations used,

only one had an overall positive effect on ELL outcomes: English language dictionaries and glossaries Produced an average effect, which is positive and

statistically different from zero No indication that this effect varied across the

studied conditions

Summary of Results Findings for native language tests and bilingual

glossaries are mixed Results varied across studies as evidenced by

homogeneity test Too few studies to say conclusive what the important

factors are, but some reasonable candidates are: Matching language of assessment with language of

instruction Ensuring that students are literate in L1

Summary of Results Results for Simplified English were less

promising than expected Test of heterogeneity was not rejected,

indicating that the small average effect is a reasonable characterization of the results of the current studies

Conclusions For any accommodation to be successful in

the testing situation, students must have experience with it during regular instruction

The alignment of curriculum, instruction, and assessment is crucial to the academic success of all students

Accommodations alone will not be effective in raising test scores of ELLs

Conclusions Lack of effects in these studies for Simplified

English is not an indictment of universal design

Research base is limited in important ways: Few studies involving State accountability tests Few studies in reading and language arts No accommodation has been studied definitively