universally designed english proficiency and content area assessments improving the achievement of...

87
Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency Assessments May, 2004

Post on 15-Jan-2016

229 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally Designed English Proficiency and

Content Area Assessments

Improving the Achievement of English Language Learners through Authentic

Proficiency AssessmentsMay, 2004

Page 2: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Training Goals• Determine why we need universally

designed assessments – and for whom

• Identify and give examples of essential elements of universally designed assessments

• Identify and apply considerations for item review to sample test items

Page 3: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

A Quick Definition

Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students, including English language learners!

Page 4: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

But, what does that really mean?

• Do we want to change the standard of performance?NO

• Can we forget about accommodations if we do this? NO

• Is this all figured out – for now and forever? NO

• Is this something that will benefit only English language learners? NO

Page 5: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Think about universal design in architecture and tool design

Curb cuts and ramps

Signage with universal symbols

Door handles rather than knobs

Special pen shapes that are easier to hold

Page 6: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

The goal is to provide optimal

standard assessment

conditions for today’s diverse population of

students

Page 7: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Remember this?

OFFICIAL BALLOT, PALM BEACH COUNTY, FLORIDA

Page 8: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Analysis of Florida Votes

Precincts with primarily Hispanic, African American, or elderly voters had more

than 3 times as many rejected ballots as other precincts.

(New York Times, 11/12/01)

Page 9: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Design is important in

a lot of things –

including assessment

s!http://go.to/funpic

Page 10: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Assessments Discussed in this Training

• English Language Proficiency Listening/ Speaking Test - Computer Based

• English Language Proficiency Reading and Writing Test - Paper/Pencil

• Grade level Academic Content Area Assessment - Computer Based

• Grade level Academic Content Area Assessment - Paper/Pencil

Page 11: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally Designed Assessments Can Increase Reliability and Validity

of Assessments for English Language Learners

Analyses of large data files (Abedi, 2002) found language as a source of measurement error that can negatively impact reliability of test results for ELLs.

Language has also been found to be a source of construct irrelevance that can have a negative impact on the validity of the results for ELLs (Abedi, 2002).

Page 12: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally Designed Assessments Increase Usability for the End Users –

STUDENTS!• In the past, emphasis has been placed

on developing test items without consideration of the students who are the ultimate end users

• The students who are tested have changed and are no longer “even remotely comparable to the designer in skill set, aptitude, expectation, or in almost any attribute that is relevant to the design process” (Rubin, 1994)

Page 13: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Student CharacteristicsJust one of many reasons that we need to be talking about universally designed assessments!

Page 14: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Complete these well-known phrases:1.A penny saved is . . .

2.An idle mind is . . .

3.Don’t bite the hand that . . .

4.It’s always darkest before . . .

5.Strike while the . . .

6.Two’s company, three’s . . .

7.If at first you don’t succeed, . . .

Page 15: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Here’s

what the

kids say!

Page 16: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

A penny saved is . . .

. . . not much.

Page 17: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

An idle mind is . . .

. . . the best way to relax.

Page 18: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Don’t bite the hand that . . .

. . . looks dirty.

Page 19: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

It’s always darkest before . . .

. . . Daylight Savings Time.

Page 20: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Strike while the . . .

. . . bug is close.

Page 21: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Two’s company, three’s . . .

. . . the Musketeers.

Page 22: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

If at first you don’t succeed, . . .

. . . get batteries.

Page 23: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

What seems obvious and clear to test developers may not be all that obvious and clear to students.

It takes consistent effort and guidance to make sure that test items and tests really are accessible to all students.

Page 24: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

This is a BIG DEAL!

Requirements for universally

designed assessments

are showing up in federal legislation

Page 25: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Title I Regulations introduced the need for universally designed assessments – [Assessments must be] designed to be accessible and valid with respect to the widest possible range of students, including students with disabilities and students with limited English proficiency.

Sec. 200.2(b)(2)

Page 26: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Increasingly, we need to think about students with a variety of different language and cultural background characteristics

Page 27: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

The number of English Language Learners in schools has increased

dramatically

• Nearly 4 million LEP students in grades K-12 in 2001-2002 (71.9% increase from 1991-1992)

• Represent about 8.4% of all public school students in the U.S.

• Enrolled in almost half of all public schools nationwide

Page 28: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

The number of ELLs with special needs is increasing too!

• Most recent estimate is 9% of the total ELL population is in Special Education

• Estimated at about 357,000 students

• More than 50% of ELLs with disabilities classified as learning disabled

Page 29: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

The bill for the federal reauthorization of the Individuals with Disabilities Education Act (IDEA) also includes universal design –

UNIVERSAL DESIGN- The State educational agency (or, in the case of a district-wide assessment, the local educational agency) shall, to the extent possible, use universal design principles in developing and administering any assessments….

Sec 612, Senate Bill 1248

Page 30: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

So, when we think about all students, we have to think about students who have a variety of cultural, physical, sensory, and processing characteristics.

Page 31: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Caution

While universally designed assessments can make tests more equitable, producing results that are more valid for all students, they cannot replace

instructional opportunity!

Page 32: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Elements of Universally Designed

Assessments Inclusive assessment population

Measures what it intends to measure

Respects the diversity of the assessment population

Clear format for text

Page 33: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Elements of Universally Designed

Assessments Clear pictures and graphics

Concise and readable text

Amenable to accommodations

Page 34: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Inclusive Assessment Population

Universally designed assessments: Consider all types of

students in the general curriculum from the beginning

Include English language learners and students with disabilities in item tryouts and field testing

Page 35: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Example of Field Test Specifications

• The statewide field test will include students with disabilities and English Language Learners using the accommodations they would normally use in the large-scale assessment.

• There may be need for over sampling of these populations to ensure a valid field test.

Page 36: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally designed assessments reflect good measurement qualities:

Reflects the intended content standard (reviewers have information about what is being measured)

Minimize skills required beyond those being measured

Measures What it Intends to Measure

Page 37: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Suppose a test item requires a student to read an Aesop’s fable

What could this item measure?– Decode text?– Comprehend extended passages?– Extent of vocabulary?– Understand the moral or point of the fable?– Discuss the common elements of any fable?– Compare and contrast fables with news reports?– Articulate the relationship between the fable and the

overall culture? – Anything else? (National Center on Accessing the Curriculum, 2003)

Page 38: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Suggestions by Researchers . . .

• Popham (2001) suggests creating an assessment description for each item that concretely describes what is being tested.

• Chiu & Pearson (1999) suggest determining non-construct behaviors needed to complete task.

Page 39: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Respects the Diversity of the Assessment Population

Accessible to test takers (consider age, gender, ethnicity, disability, socio-economic level)

Avoid content that might unfairly advantage or disadvantage any student subgroup

Page 40: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

According to the National Research Council (1999), bias

arises when:

“Deficiencies in the test itself result in different meanings for scores earned by members of different identifiable subgroups.”

Page 41: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Consider the Effect of Cultural Bias

If you measure from the nose to the tail, how much longer is lizard B than lizard A? Use the ruler provided in your testing kit. Give the answer in centimeters.

A. B.

Page 42: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

To raise money for a trip to the Wolfridge Environmental Learning Center, sixth graders at Johnson Middle School are selling raffle tickets. The raffle prize is an electric scooter worth more than $300. A total of 500 tickets were sold. You bought two raffle tickets, your sister bought three and your father bought one. What is the probability that someone in your family will win the prize?

Page 43: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Clear Format for Text

Universally designed assessments use text that can be read quickly and effortlessly

Standard typeface Type size at least 12 point Wide spacing High contrast Sufficient leading (Spacing) between lines of

text Staggered right margins (no right

justification) No background graphics behind text

Page 44: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Typeface: Standard typeface, with upper and lower

case letters, is better than italic, small caps, or all caps

Adapted from: Making Text Legible by A. Arditi

Page 45: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Type SizeAll type should be at least 12 point

(including captions, keys, labels, and footnotes).Remember that point size differs among typefaces.

Adapted from: Making Text Legible by A. Arditi

Page 46: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Spacing: Space between letters and

between words is wide

Adapted from: Making Text Legible by A. Arditi

Page 47: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

ContrastText should be printed with the highest possible contrast.

Adapted from: Making Text Legible by A. Arditi

Page 48: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Leading: Spacing Between Lines of Text

Adapted from: Making Text Legible by A. Arditi

Page 49: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Staggered Right Margins

Text is easier to read when it is justified on the left and unjustified on the right. This prevents pockets of spaces.

Text is easier to read when it is justified on the left and unjustified on the right. This prevents pockets of spaces.

(APH, 2002) Used with permission

Page 50: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Text with background graphics or other media is difficult to read.

This is a picture of a starfish in-side a vortex. Of course to youit might look like a giant floweror an octopus in a whirlpool. You might even think it is op-art or a computer rendering ofa bad dream. However thisphoto is to demonstrate to you how difficult it is to read text when it is placed over a busy background.

(APH, 2002)Used withPermission

Page 51: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Clear Pictures and Graphics

Clear, non-fuzzy pictures Dark lines (minimum use of gray scale

and shading) Sufficient contrast between colors Color is not relied on to convey important

information or distinctions Label pictures and graphics when

possible

Page 52: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally designed assessments use non-text materials just as carefully as text materials

Only essential illustrations are used (referred to in text and necessary to answer question)

Illustrations for interest may draw attention away from content being assessed

Page 53: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Is this border distracting? Is it necessary?

Page 54: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Fence or Bridge or What?

fence

Add context and labels to help identify pictures

Page 55: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Is the swimmer at the bottom of the page distracting?

Is this a better way to remind test takers to continue?

Page 56: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Concise and Readable Text

Commonly used words Vocabulary appropriate for grade level Minimum use of unnecessary words Idioms avoided unless idiomatic speech is being

measured Technical terms and abbreviations avoided (or

defined) if not related to the content being measured

Sentence complexity is appropriate for grade level

Question to be answered is clearly identifiable

Page 57: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Clear Instructions• Ensure “that test takers can respond to

a task in the manner that the test developer intended” (AERA, APA, NCME, 1999).

• “While writers might think certain expectations are obvious, if they are not explicit in the item, then they are subject to honest misinterpretation in the responses” (Kopriva, 2000, p. 39).

Page 58: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Assessments designed to better include English language learners

benefit all types of students!

• Students have the experience to understand the items

• Language is clear, simple and indicates precisely what is required from student (“Plain language”)

• Questions are amenable to supports that ELLs might use

• Cognitive demands are reasonable

Page 59: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Abedi research suggests that linguistic complexity of test items is a significant source of measurement error for English language learners (and students with disabilities)Examples of Linguistic Modifications:

Tetris Video game

Familiarity or frequency of non-math vocabulary (unfamiliar or infrequent words changed):

Last year’s class vice president The vice president

Length of nominals (long nominals shortened):

(Abedi, et al., 2001)

Page 60: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Linguistic Modifications (continued):

At which of the following times

When

Question phrases (complex question phrases changed to simple question words):

If a marble is taken from the bag

If you take a marble from the bag

Voice of the verb phrase (passive verb forms changed to active):

A report that contains 64 sheets of paper

He needs 64 sheets of paper for each report

Relative clauses (relative clauses removed or recast):

Page 61: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Linguistic Modifications (continued):

If Lee delivers x newspapers Lee delivers x

newspapers

Conditional clauses (conditionals either replaced with separate sentences or order of conditional and main clause changed):

Abedi found that English language learners who received a linguistically modified version of the math test performed significantly better than those receiving the original test items. They also performed significantly faster.

Page 62: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Over 80% of the students also told Abedi they preferred the modified math test because:

• It’s easier to read, and it gets to the point, so you won’t have to waste time

• It’s shorter and doesn’t have, like, complicated words

• I might have a faster time completing that one ‘cuz there’s less reading’

(Abedi, et al., 2001)

Page 63: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Sample Readability Guidelines Use simple, clear, commonly used

words, eliminating any unnecessary words (Abedi, 2002)

Clearly define any technical terms that are used

Break compound complex sentences into several short sentences. State the most important ideas first (Rakow & Gee, 1987)

Introduce one idea, fact, or process at a time (Rakow & Gee, 1987)

Page 64: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Recommendations to Improve Accessibility of Text

• Simple, brief and consistent sentence structure in items

• Consistent and clear paragraph structure

• Present tense and active voice• Minimal paraphrasing and

rewording. If used, identify the original statement in parentheses

From Kopriva (2000)

Page 65: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Recommendations to Improve Accessibility of Text

• Minimal use of pronouns. Follow a pronoun with the term it refers to in parentheses

• High frequency words • Avoid words with double meanings

and colloquialisms. If used, define them in the text.

From Kopriva (2000)

Page 66: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Example Reading Passage Newspaper Excerpt

Para. 1 “When Nicole Zachor, Laura Swanson and Carol Hinz started work on a project for history class a few months ago, the White Bear Lake sophomores had no idea that it would be displayed at the Smithsonian Institution’s National Museum of American History…”

Para. 12 “ This year junior and senior high students started work on their projects in January or February. A project can be a research paper but it can also be a group of individual media presentation, display presentation or performance”

Page 67: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Which of the following is a condition for a student to participate in National History Day?

A. The student must be a junior or senior in high school.

B. The student must be able to go to Washington, D.C.

C. The student must do a project related to the national topic.

D. The student must do the project on his or her own by himself or herself.

¿Cuál de las siguientes es una condicíon para que un estudiante participe en el Día Nacional de Historia?

A. Los estudiantes deben estar en grados once o doce de High School.

B. El estudiante debe estar en posibilidad de ir a Washington, D.C

C. El estudiante debe hacer un proyecto relacionado a un tema nacional.

D. El estudiante debe hacer el proyecto por sí solo.

Side-by-Side Translation

Page 68: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Ordering Pizza (Original Item)The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey.

NO

YES

Based on the results of the survey, answer the following questions:

1.1. What fraction of students would buy Brand X pizza on Friday?What fraction of students would buy Brand X pizza on Friday?

2.2. What percent of students would buy Brand X pizza on Friday?What percent of students would buy Brand X pizza on Friday?

3.3. There are 1200 students in this school. How many students will There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is buy Brand X pizza on Friday if the manager’s survey is accurate?accurate?

Page 69: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Ordering Pizza (Revised Item)Maria surveyed the students in her school to find out if they would like pizza on Friday. She made a circle graph to display the results of her survey.

NO

YES

1.1. What What fractionfraction of students said “yes”? of students said “yes”?

2.2. What What percentpercent of students said “yes”? of students said “yes”?

3.3. There are 1200 students in Maria’s school. There are 1200 students in Maria’s school. How How manymany students said “yes”? students said “yes”?

(From Hanson, et al., 1998)

Page 70: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

• Amount of text not relevant to items• Length of text• Number of long texts• Timing (may be unspoken)• Number of unfamiliar words• Placement of definitions (in text, to side,

separate) – if allowed• Location of native language text – if

provided

Cognitive Demands

Page 71: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Amenable to Accommodations Universally designed

assessments allow needed accommodations to be used

Plan for students who continue to need accommodations

Facilitate the use of accommodations such as bilingual dictionaries or translations (when appropriate), assistive technology, oral presentation, large print, Braille

Page 72: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

What accommodations might be difficult with this item?

Could this item be revised without knowing what it is intended to measure?

Page 73: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Here is a revised graphic that would clarify the item.

Items and revision used by permission from the state of NH and DRC

Page 74: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Universally designed assessments consider the design of the response venue as well as the assessment itself

Large bubbles that avoid most challenges of low vision or difficulty with fine motor skills (Grise, Beattie, Algozzine, 1982)

Consideration of age of students in selecting format (avoid separate answer sheets for younger students) (Tindal and others suggest 4th grade as transition point).

Page 75: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Clear Response Grid

Hard to see

Gray columns changed to yellow.Larger bubbles.Easier to see.

(APH, 2003) Used with permission

Page 76: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Computer-Based Testing

All of the Elements of Universal Design

Plus Some Additional Opportunities and

Challenges

Page 77: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Opportunities

• Efficient administration• Preferred by students• Improved writing performance• Built-in accommodations• Immediate results• Efficient item development• Increased authenticity

Page 78: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Challenges• Use of technology cannot take the place

of content mastery • Issues of equity and skill in computer use• Added challenges for some students• Technological challenges • Security of online data • Lack of expertise in designing accessible

Web pages• Prohibitive development cost

Page 79: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Example: Kentucky Online Assessment

• Web-based, individualized assessment :– Students with IEP or 504 Plan that specifies need for

"reader" as an instructional and assessment accommodation;

– Students who require and routinely use text-reader or screen-reader technologies to access printed material in classroom instruction and assessment;

– Students who have accessed and used the CATS Online Practice Area.

• Based upon success of pilot studies, 16 districts, 31 Schools, & 204 students participated in “live” CATS Online in the spring of 2003

Page 80: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

CATS Online

Page 81: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

How to Develop Universally Designed Assessments

• Develop frameworks for alignment of items with content

• Include elements of universally designed assessments in test specifications

• Develop guidelines for item developers to follow

Page 82: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

How to Develop Universally Designed Assessments

Flag items with bias and design issues:

• Consult with end users – students representing all subgroups

• Consult with bias or sensitivity review teams – representing all subgroups

• Analyze field test data and statewide test results by item and subgroup

Page 83: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

What Reviewers Need to Know• Purpose of the test• Standard tested by each item• Description of test takers (e.g. age,

geographic region)• Test instructions• Overall test and response formats• Use of technology• Field test results by item and subgroup

(when completed)

Page 84: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Review Team Members• Grade-level content experts

• Representatives of major cultural groups

• Representatives of major disability groups

• Second language acquisition experts

• Who else?

Page 85: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Remember!• This is a work in progress

• Considerations put important issues on the table

• Ongoing input is important!

Page 86: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Item Review Materials

• Considerations and notes• Research base for considerations• References• Considerations form for item

reviewers• Sample revised items

Page 87: Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency

Visit: education.umn.edu/nceo

or Search for NCEO

Click on “Universal Design”