universally designed english proficiency and content area assessments improving the achievement of...
Post on 15-Jan-2016
Embed Size (px)
Universally Designed English Proficiency and Content Area Assessments Improving the Achievement of English Language Learners through Authentic Proficiency AssessmentsMay, 2004
Training GoalsDetermine why we need universally designed assessments and for whomIdentify and give examples of essential elements of universally designed assessmentsIdentify and apply considerations for item review to sample test items
A Quick DefinitionUniversally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students, including English language learners!
But, what does that really mean?Do we want to change the standard of performance?NOCan we forget about accommodations if we do this?NOIs this all figured out for now and forever?NOIs this something that will benefit only English language learners?NO
Think about universal design in architecture and tool designCurb cuts and rampsSignage with universal symbolsDoor handles rather than knobsSpecial pen shapes that are easier to hold
The goal is to provide optimal standard assessment conditions for todays diverse population of students
OFFICIAL BALLOT, PALM BEACH COUNTY, FLORIDA
Analysis of Florida Votes
Precincts with primarily Hispanic, African American, or elderly voters had more than 3 times as many rejected ballots as other precincts. (New York Times, 11/12/01)
Design is important in a lot of things including assessments!http://go.to/funpic
Assessments Discussed in this TrainingEnglish Language Proficiency Listening/ Speaking Test - Computer Based English Language Proficiency Reading and Writing Test - Paper/PencilGrade level Academic Content Area Assessment - Computer BasedGrade level Academic Content Area Assessment - Paper/Pencil
Universally Designed Assessments Can Increase Reliability and Validity of Assessments for English Language LearnersAnalyses of large data files (Abedi, 2002) found language as a source of measurement error that can negatively impact reliability of test results for ELLs. Language has also been found to be a source of construct irrelevance that can have a negative impact on the validity of the results for ELLs (Abedi, 2002).
Universally Designed Assessments Increase Usability for the End Users STUDENTS!In the past, emphasis has been placed on developing test items without consideration of the students who are the ultimate end users
The students who are tested have changed and are no longer even remotely comparable to the designer in skill set, aptitude, expectation, or in almost any attribute that is relevant to the design process (Rubin, 1994)
Student CharacteristicsJust one of many reasons that we need to be talking about universally designed assessments!
Complete these well-known phrases:A penny saved is . . . An idle mind is . . .Dont bite the hand that . . . Its always darkest before . . .Strike while the . . .Twos company, threes . . .If at first you dont succeed, . . .
Heres what the kids say!
A penny saved is . . .. . . not much.
An idle mind is . . .. . . the best way to relax.
Dont bite the hand that . . .. . . looks dirty.
Its always darkest before . . .. . . Daylight Savings Time.
Strike while the . . .. . . bug is close.
Twos company, threes . . .. . . the Musketeers.
If at first you dont succeed, . . .. . . get batteries.
What seems obvious and clear to test developers may not be all that obvious and clear to students. It takes consistent effort and guidance to make sure that test items and tests really are accessible to all students.
This is a BIG DEAL!Requirements for universally designed assessments are showing up in federal legislation
Title I Regulations introduced the need for universally designed assessments [Assessments must be] designed to be accessible and valid with respect to the widest possible range of students, including students with disabilities and students with limited English proficiency. Sec. 200.2(b)(2)
Increasingly, we need to think about students with a variety of different language and cultural background characteristics
The number of English Language Learners in schools has increased dramaticallyNearly 4 million LEP students in grades K-12 in 2001-2002 (71.9% increase from 1991-1992)Represent about 8.4% of all public school students in the U.S.Enrolled in almost half of all public schools nationwide
The number of ELLs with special needs is increasing too!Most recent estimate is 9% of the total ELL population is in Special EducationEstimated at about 357,000 studentsMore than 50% of ELLs with disabilities classified as learning disabled
The bill for the federal reauthorization of the Individuals with Disabilities Education Act (IDEA) also includes universal design UNIVERSAL DESIGN- The State educational agency (or, in the case of a district-wide assessment, the local educational agency) shall, to the extent possible, use universal design principles in developing and administering any assessments.Sec 612, Senate Bill 1248
So, when we think about all students, we have to think about students who have a variety of cultural, physical, sensory, and processing characteristics.
CautionWhile universally designed assessments can make tests more equitable, producing results that are more valid for all students, they cannot replace instructional opportunity!
Elements of Universally Designed AssessmentsInclusive assessment populationMeasures what it intends to measureRespects the diversity of the assessment populationClear format for text
Elements of Universally Designed AssessmentsClear pictures and graphicsConcise and readable textAmenable to accommodations
Inclusive Assessment PopulationUniversally designed assessments:Consider all types of students in the general curriculum from the beginningInclude English language learners and students with disabilities in item tryouts and field testing
Example of Field Test Specifications The statewide field test will include students with disabilities and English Language Learners using the accommodations they would normally use in the large-scale assessment.There may be need for over sampling of these populations to ensure a valid field test.
Universally designed assessments reflect good measurement qualities:
Reflects the intended content standard (reviewers have information about what is being measured)Minimize skills required beyond those being measuredMeasures What it Intends to Measure
Suppose a test item requires a student to read an Aesops fableWhat could this item measure?Decode text?Comprehend extended passages?Extent of vocabulary?Understand the moral or point of the fable?Discuss the common elements of any fable?Compare and contrast fables with news reports?Articulate the relationship between the fable and the overall culture? Anything else? (National Center on Accessing the Curriculum, 2003)
Suggestions by Researchers . . .Popham (2001) suggests creating an assessment description for each item that concretely describes what is being tested. Chiu & Pearson (1999) suggest determining non-construct behaviors needed to complete task.
Respects the Diversity of the Assessment PopulationAccessible to test takers (consider age, gender, ethnicity, disability, socio-economic level)
Avoid content that might unfairly advantage or disadvantage any student subgroup
According to the National Research Council (1999), bias arises when:
Deficiencies in the test itself result in different meanings for scores earned by members of different identifiable subgroups.
Consider the Effect of Cultural BiasA.B.If you measure from the nose to the tail, how much longer is lizard B than lizard A? Use the ruler provided in your testing kit. Give the answer in centimeters.
To raise money for a trip to the Wolfridge Environmental Learning Center, sixth graders at Johnson Middle School are selling raffle tickets. The raffle prize is an electric scooter worth more than $300. A total of 500 tickets were sold. You bought two raffle tickets, your sister bought three and your father bought one. What is the probability that someone in your family will win the prize?
Clear Format for TextUniversally designed assessments use text that can be read quickly and effortlessly Standard typeface Type size at least 12 point Wide spacing High contrast Sufficient leading (Spacing) between lines of text Staggered right margins (no right justification)No background graphics behind text
Typeface: Standard typeface, with upper and lower case letters, is better than italic, small caps, or all capsAdapted from: Making Text Legible by A. Arditi
Type SizeAll type should be at least 12 point (including captions, keys, labels, and footnotes).Remember that point size differs among typefaces.Adapted from: Making Text Legible by A. Arditi
Spacing: Space between letters and between words is wideAdapted from: Making Text Legible by A. Arditi
ContrastText should be printed with the highest possible contrast.Adapted from: Making Text Legible by A. Arditi
Leading: Spacing Between Lines of TextAdapted from: Making Text Legible by A. Arditi
Staggered Right Margins
Text is easier to read when it is justified on the left and unjustified on the right. Th