Georgia Modification Research StudySpring 2006

Sharron Hunt

Melissa Fincher

Georgia’s Student Assessment Program

• Includes an array of criterion-referenced tests (including an alternate) and a norm-referenced test

• Built from the ground up to be inclusive– GA includes teachers of special populations in all

phases of test development– GA includes all students in pilots and field tests

Georgia’s Student Assessment Program

• GA provides a menu of traditional accommodation options– Presentation– Response– Setting– Scheduling

• Accommodations are provided during pilots and field tests

Georgia’s Student Assessment Program

• Although Georgia is inclusive, some students still struggle to access the tests

– Schools sought additional avenues for students who were caught between assessments – neither the general or the alternate were completely appropriate for these students

Georgia’s Student Assessment Program

• Georgia looked for avenues to investigate how to make assessments more accessible

• Questions to answer– Who are the students struggling to access the general

assessments?– What additional accommodations/modifications might assist

in making the test more accessible?

• About the same time….– Enhanced Assessment Instruments Grant announced

(ultimately pulled back)– Announcement of the 2% flexibility

Georgia’s Student Assessment Program

• Georgia moved forward with an empirical research study designed to answer those questions

• The Testing Division worked with the – Division for Exceptional Students – Curriculum and Instructional Services Division

Study Purpose

• To investigate the effects of certain modifications on the reliability and validity of the state testing programs

– Criterion-Referenced Competency Tests (CRCT)– Georgia High School Graduation Tests (GHSGT)

• Focus on reading and math

Study Purpose

• This study will help Georgia determine

– What works– Who benefits

• The goal of the study is to help Georgia operationalize additional options for the 2006-2007 school year

What is a modification?

• an alteration in the administration of an assessment that results in a possible change in the content or construct being assessed

– typically either through the addition or removal of content

What is an accommodation?

• an alteration in the administration of an assessment that allows students to participate – how the test is administered or presented– how the student responds

• Construct (what the test measures) remains the same

Georgia Modification Study

• Focused on selected grades and content areas

– Students in grades 4 and 7 took the Reading and Math CRCT (March 2006)

– Students in grade 12 took the ELA and Math GHSGT (April 2006)

Georgia Modification Study

• For security reasons, off-level testing was used

– Grade 4 students took the 3rd grade Spring 2005 CRCT

– Grade 7 students took the 6th grade Spring 2005 CRCT

– Grade 12 students took the Spring 2005 GHSGT

Georgia Modification Study

• Study included both special education students and regular education students

– The number of special education students who took the test last year at each school was identified

• Tested the number of students identified, if possible

– Schools randomly selected an equal number of general education students to participate

Georgia Modification Study

• Included all special education students, with the following exceptions:– GAA students– Students who require Braille or large print

• Set a broad net to help determine the characteristics of students assisted by modifications

Georgia Modification Study

• Each school assigned a condition for this study

– All students within the school took the test under the same condition

• Both special and regular education students

Georgia Modification Study

CRCT (grades 4 and 7)

– Three conditions

– Condition 1: Resource Guides – Condition 2: Read Aloud/Calculator– Condition 3: Traditional Administration

Georgia Modification Study

GHSGT (grade 12)

– Two conditions

– Condition 1: Resource Guide– Condition 2: Traditional Administration

Condition 1 – CRCT & GHSGT

• Resource Guide

– A single sheet (one for each content area) that provides key definitions, examples, multiplication tables etc.

– A short practice test was provided to help introduce the resource guides to students; each item related to something on the resource guide

• Recommended introducing sample tests and guides a couple of days before the administration

Condition 1 – CRCT & GHSGT

• Resource Guide Development

– Committee comprised of Testing, Special Education, and Curriculum specialists reviewed the tests

• identified concepts students might struggle with and discussed appropriate information for inclusion on the guide

Condition 2 - CRCT

• Read Aloud (Reading CRCT)– Teacher read the reading passages and questions to

students• Teacher reads test verbatim

• Calculator (Math CRCT)– Basic function calculators provided to schools

assigned this condition

Condition 3 - CRCT

• Traditional Administration

– Students took tests as they have always done• Special education students received the standard

accommodations they normally receive • Regular education students did not receive

accommodations

Condition 2 - GHSGT

• Traditional Administration

– Students took tests as they have always done• Special education students received the standard

accommodations they normally receive • Regular education students did not receive

accommodations

Georgia Modification Study

• A survey was be provided for each special education student– The person most knowledgeable about the student

completed the survey– This information will be critical in determining who

benefits from the modifications

Georgia Modification Study

• Survey collected important information about each special education student– Student’s disability/ies– Severity of disability/ies– IEP objectives for reading and math– Time in mainstream/general ed classroom– Area/s served– Instructional services– Type of accommodation, if any

Georgia Modification Study

• Administration Procedures

– Follow all administration protocols as outlined in the Examiner’s Manual

– Complete all student demographics on answer documents, including State-Required Codes

Georgia Modification Study

• Administration Procedures

– Followed all administration protocols as outlined in the Examiner’s Manual

– Completed all student demographics on answer documents, including State-Required Codes

– Complete survey for each special education student

Georgia Modification Study

• Key Research Questions

• Are there performance differences for students with disabilities on the CRCT and GHSGT when these assessments are administered under modified conditions?

• Are there performance differences for regular education students on the CRCT and GHSGT when these assessments are administered under modified conditions?

Georgia Modification Study

• Key Research Questions

• How, if at all, do performance differences on the traditional versus modified version of the CRCT or GHSGT differ for students with disabilities compared to regular education students?

• Are the CRCT and GHSGT reliable and valid assessments when administered under modified conditions?

Georgia Modification Study

• Currently analyzing data to answer the research questions

– Analysis at the item level• Are certain items more likely to function differently?

– Analysis at the form level• How does the test form function under each condition

(modified vs traditional) for each group of students?

Georgia Modification Study

Sharron Hunt, Director of Testing

shunt@doe.k12.ga.us / 404.657.0314

Melissa Fincher, Assistant Director of Testing

mfincher@doe.k12.ga.us / 404.651.9405