Summer 2018 IDOE Pretest Workshop

IREAD-3 Retest

Introduction

• Mary Williams (IREAD-3)oAssessment Program Specialist

omwilliams@doe.in.gov

Agenda

• Participation

• Student Transfer Process

• Practice Test Updates

• Test Administration Guidance

• Online & Paper Testing

• Accommodations

• Avoiding Test Invalidations

• Return & Use of Results

• Granting Good Cause Exemptions

IREAD-3 Summer Participation

• Schools may administer the IREAD-3 assessment anytime during the Summer 2018 test window.

• Schools do not need to select a window within the window.

• Each eligible student may take the IREAD-3 retest only once during the summer window.

IREAD-3 Summer Participation

• The summer retest IS for Grade 3 students who:

• did not pass IREAD-3 in March and have since been provided reading remediation/intervention and are not eligible for a Good Cause Exemption,

• were absent during the March test window, or

• enrolled in Grade 3 at an accredited Indiana school after the March test window and before the end of the school year

• The summer retest IS for Grade 4 (and Grade 5) students who:

• did not pass IREAD-3 in Grade 3 (or Grade 4) and are not eligible for a Good Cause Exemption

IREAD-3 Summer Participation

• It is NOT for students who:

• have been granted a Good Cause Exemption,

• completed Grade 3 at a home school, other non-accredited Indiana school, or out-of-state school, or

• have been promoted to Grade 6.

IREAD-3 Participation & Good Cause Exemption Eligibility

Students

Participate

in the

Assessment

Eligibility for Good Cause

Exemption

Students with a Section 504 Plan Yes No

Students with an IEP (SWDs) YesDetermined by

Case Conference Committee

Students with an ILP (ELs) Yes Determined by ILP Committee

Students who have been retained twice Yes Yes

Students who are eligible for the

alternate assessmentNo N/A

IREAD-3 Summer Retest: Important Dates

• May 14 Test sessions created in PearsonAccessnext

• May 14 – July 16 Online Practice Test Window

• May 14 – July 11 Additional Materials Order (TAMs, test books, etc.)

• May 29 – July 20 Test Window (OL)

• June 4 – July 13 Test Window (PP)

• June 18 – August 10 OnDemand Reports in PearsonAccessnext

• July 20: Last day for marking tests “invalid”

• July 16: Last day for materials pick-up (call UPS to schedule)

Student Transfer Process

• CTCs must initiate if a transfer student record already exists in PearsonAccessnext(PAN)

o Confirm student’s correct STN and completed test sessions with former school – Do NOT make assumptions.

o CTC may set up test sessions for student once transfer is approved by CTC of student’s former school.

• CTC of student’s new school should contact Pearson if transfer request is not approved in PAN within one (1) business day.

• Follow process to avoid test invalidations and undetermined results.

• See process document sent via CTC listserv.

Student Test Number (STN)

• Online• Schools who did not enter STNs via the April DOE-TL collection

must manually enter student data in PearsonAccessnext.

• Paper/Pencil • If you are administering a paper/pencil, large print, or Braille test

to a student, follow the instructions in the TAM for entering student information into PearsonAccessnext.

• No barcode labels for summer. CTCs must bubble student information on the front cover of the test book.

IREAD-3: Test Administration Manual

• Schools must use copies reserved from spring testing or place

order during AMO window for additional TAMs, as needed.

• IREAD-3 Summer Retest AMO: 5/14/18 – 7/11/18

• Includes ALL Examiner Scripts• Pencil/Paper and Online

• Practice Test and Operational

• Includes Read Aloud scripts for students with accommodation

• Non-Secure version posted in the TCC may NOT be used to

administer the IREAD-3 operational test.

11

IREAD-3: Practice Test

• Practice tests are required.o Students must participate at least once before completing the

operational test

o Same mode as operational test

o Allow students access to item types, online functionalities

• Practice tests are untimed.

• Administration directions are contained in IREAD-3 TAM.

• Practice Test Windowso IREAD-3 Retest: 5/29/18 – 7/20/18 (OL) 6/4/18 – 7/13/18 (PP)

IREAD-3: Practice Test

• Pearson generated online practice test sessions for sample students based on schools’ April DOE-TL submission.

• Number of sample students approximates number of students submitted in DOE-TL upload.

• Generic Test Tickets

o Include student’s first name and a generic last name

oNot reusable

STCs/CTCs may generate additional sample students and practice test sessions, as needed, for new students and/or repeated practice test administrations.

IREAD-3: Operational Test

• Pearson set up online test sessions for students reported through the April DOE-TL collection.

• Use Accessibility Features and Accommodations for Student Tests report in PAN to confirm accommodations before testing begins.

o Inappropriate use of accommodations leads to undetermined results each year.

• Contact the Pearson Help Desk for assistance, as needed (e.g., updating or creating test sessions, assigning accommodations, user access, etc.).

IREAD-3: Operational Test

• Test sections must be administered in sequential order (e.g., Section 1, 2, 3).

• All test sections do not need to be administered on the same day.

• Students must complete a test section on the same day they start it.

• Be thoughtful about student fatigue when administering multiple test sections in a single day.

• Students may test with their content area teachers.

• Follow directions exactly as outlined in the IREAD-3 Test Administration Manual.

IREAD-3 Online – Part 1

• Students are expected to participate in the online retest.

> Exception…• Any student with an IEP, Section 504 Plan, ILP, or Service Plan

that requires use of paper/pencil test book.

> Limited or no online capacity?• Partner with a neighboring school.

• Refer to “Testing Students at Alternate Sites” memo sent to schools in January.

> Print test tickets and seal codes for use at alternate site.

16

IREAD-3: Online – Part 2• Required Pearson training session for online sites

• Attend one Webinar session

• May 16, 9:00 – 10:30 a.m. ET

• May 22, 1:00 – 2:30 p.m. ET

• Registration is required!

• CTCs and Technology Coordinators who attended Pearson training in preparation for Spring 2018 ISTEP+ / IREAD-3 have already met this training requirement.

Accommodations – Part 1

• Refer to Appendix E of 2017-18 Assessment Program Manual• https://www.doe.in.gov/sites/default/files/assessment/accessibility-and-

accommodations-guidance.pdf

• Same accommodations for IREAD-3 as available for all assessments.

• Confirm correct online accommodations before assigning test sessions for students with IEPs, ILPs, Section 504 Plans.

oStudents with IEP, ILP, or Section 504 Plan onlyoResolve any issues/confirm accommodations before testing begins.

Crosscheck with report in PAN: Accessibility Features and Accommodations for Student Tests

Accommodations – Part 2

• “Manage Student Tests” in PearsonAccessnext

• Changes must be made prior to preparing the test session.

Accommodations – Part 3

• Students with an IEP, ILP, or Section 504 Plan only:

• Online• Text-to-Speech (TTS)

• Paper/Pencil• Large Print, Braille

• Read-Aloud Scripts

• TTS and read aloud accommodations very limited on IREAD-3.

Accommodations – Part 4

• For students with hearing-impairments who receive no phonics instruction and/or whose hearing impairments prevent them from hearing phonics instruction….

• Students will not be asked to respond to items that measure foundational reading sounds.

• A separate form of the test in the online system has been created specifically for these students.

• Verify that students with hearing impairments have pre-assigned online tests (names carried over from spring).

• Notify IDOE if additional students require this accommodated online form.

Accommodations – Part 5

• Standard Edition Paper/Pencil Book• Non-Standard Assessment Accommodation form required (due April 24)

• Based on IEP/ILP/Section 504 Plan

• Large Print and Braille

• Test materials shipped directly to CTCs• Based on school’s submission in April DOE-TL

• Correctly prepare documents for return to Pearson (TAM).

Accommodations – Part 6

• Read aloud scripts are located in the back of the IREAD-3 TAM• Use only with students who test paper/pencil and whose IEP, ILP, or Section

504 Plan requires the read aloud accommodation.

• Reading Comprehension items may NOT be read aloud to any student (see specific guidance below).

Section 1

• Examiner reads aloud some

foundational items to all students;

students independently read

answer choices and mark answers.

Sections 2 & 3

• Only directions preceding each

test section may be read aloud to

students with the read aloud

accommodation.

IREAD-3: Reminders

Before testing (STCs/CTCs):

• Inventory materials from Pearson (e.g., IREAD-3 TAMs, return shipping labels).

• Assign the correct STN to each student.

• Provide each student correct test ticket or test book.

• Review IREAD-3 TAMs with examiners before test window.

IREAD-3: RemindersBefore testing (STCs/CTCs):

• IREAD-3 Test Administration Manual

oSingle manual for use with Spring and Summer 2018 administrations

oContains secure content• STCs/CTCs must distribute and collect TAMs during examiner

training; immediately before and after testing each day.

• Keep TAMs in locked storage when not in use.

oReturn all TAMs to Pearson as soon as testing is complete.

• Call UPS to schedule pickup.

IREAD-3: Reminders

During testing:

• Students must be working in the correct part of the test.

• Students with an IEP, ILP, or Section 504 Plan must have access to the correct accommodation(s), as appropriate.

• Students without a formal plan must not receive accommodations in error.

• Students must not coached or instructed to revisit certain responses.

After testing:

• School personnel must not review test items and/or student responses

• Examiners and STCs/CTCs should check front covers of test books to ensure that the invalidation bubbles were not accidentally filled by adults or students.

Avoid test invalidations and undetermined results!!

IREAD-3: Common Causes of Test Invalidations

Please plan ahead to avoid instances in which test invalidation may result. Here are some common irregularities that result in test invalidations and undetermined results (not an exhaustive list):

• Incorrect STN is assigned for a student.

• Students use incorrect test tickets.

• Students working in the wrong part of test.

• Students without an IEP, ILP, or Section 504 Plan receive accommodation(s) in error.

• School personnel coach or encourage students to revisit certain responses.

• School personnel review test items and/or student responses.

• Invalidation bubbles are accidentally filled in on front cover of test books/answer documents.

Invalidations

• Maintain local documentation for any “Invalid” test sessions.

• STC or CTCs must submit completed Testing Irregularity Report to

IDOE detailing issue(s) that led to test invalidations.

• All test invalidations must be completed in PearsonAccessnext by

July 20.

• Check front cover of each student test book to ensure that

invalidation bubble is not incorrectly marked.

• Once submitted, invalidations MAY NOT BE REVERSED.

Return of Results

29

• Access PearsonAccessnext

• Secure site with unique passwords; available 24/7

• Contains FERPA-protected data! (Family Educational Rights and Privacy Act)

• https://in.pearsonaccessnext.com

• Electronic results (Online)• Beginning June 18

• OnDemand Proficiency Roster

• Undetermined Status Report

• Electronic reports (Online and Paper/Pencil testing)– Beginning August 9

• OnDemand Proficiency Roster

• Undetermined Status Report

• Individual Student Report

• Student Data File (SDF)

Return of Results

• Paper copies (Online and Paper/Pencil testing)

• Delivered by August 20

• One copy of Individual Student Report

• One label for the student’s cumulative folder

• Schools are responsible for promptly communicating results to parents!

• Plan ahead to ensure communication with parents during summer months.

• Notify parents as soon as possible once electronic results are available.

Use of Results

• Data presented on the IREAD-3 Individual Student Report (ISR) present a summative “snapshot” of student performance with regard to foundational reading skills.

• Compare ISR to data gathered locally and look for patterns to help identify and address specific learning needs

• Corporation-, school-, and classroom-level assessments

• Progress monitoring results

• Teacher observations

Use of Results

• IREAD-3 data are used…

• to determine promotion for general education students (Pass/Did Not Pass)

• as one piece of evidence in determining promotion decisions

• Students with disabilities (IEP)

• English learners (ILP)

IREAD-3 and Retention

• Retention…• is identified as a last resort in HEA 1367

• is focused on reading/literacy curriculum and instruction

• requires schools to continue to provide students who do not pass IREAD-3 with Grade 3 reading and literacy instruction

• Literacy instruction may also include other content areas, such as science and social studies.

Methods of organizing students for differentiated instruction

are determined locally.

Students Who Do Not Pass IREAD-3

• Updated guidance offers schools flexibility to determine official grade-level reporting status for Grade 3 students who do not pass IREAD-3 in spring or summer.

• Approved by Indiana State Board of Education (April 2017).

• Continued expectations:

• Schools provide students Grade 3 reading instruction and

• Schools provide students opportunity to retest for two subsequent years, if needed (until students pass IREAD-3 or become eligible for a Good Cause Exemption).

IREAD-3 Good Cause Exemptions

• Good Cause Exemptions that allow for flexibility in determining access to grade four reading and literacy instruction include:

• Students who have previously been retained two times prior to 4th grade.

• Students with disabilities whose Case Conference Committee (CCC) has determined that promotion is appropriate.

• English Learner students whose Individual Learning Plan (ILP) Team has determined that promotion is appropriate.

Good Cause Exemption Eligibility

StudentsEligibility for Good Cause

Exemption

Students with a Section 504 Plan No

Students with an IEP (SWDs)Determined by

Case Conference Committee

Students with an ILP (ELs) Determined by ILP Committee

Students who have been retained twice Yes

Students who are eligible for an

alternate assessmentN/A

IREAD-3 Good Cause Exemptions

• DOE-IR Collection

• Open now through September 30, 2018

• Report STNs of students granted exemptions.

• Submit data as soon as exemptions are granted, if possible.

• Contact the STN Application Center for assistance with reporting exemptions.

IREAD-3 Status Report• Learning Connection

• Permissioned access

• IREAD-3 performance status

• Pass/DNP/Test Not Taken

• Good Cause Exemption information

• Currently reflects all exemptions reported via DOE-IR through March 8.

• Next refresh is scheduled for May 15.

• Reflects data for students reported via DOE-RT

• Select “Admin Reports”• Drill down to specific school

• Select specific grade (3, 4, or 5)

• Select “Run Report”

Educator Resources

• Test Blueprint o Align test expectations with mastery of standards

o More detailed version posted on IDOE website

• Item Specifications

o Offers details on how standards may be assessed

o Released for first time!

• Both resources posted under “Blueprint and Specifications” dropdown: https://www.doe.in.gov/assessment/iread-3

• Use to inform remediation for students who did not pass the assessment.

2017-18 Test Security Training Refresher

Increased Focus on Test Security

• Goalso Deliver accommodations with fidelity.

o Do not deviate from standard test administration and test security protocols outlined in the Test Security and Integrity Training, Examiner’s Manuals, Code of Ethical Practices and Procedures, or Chapter 13 of the Indiana Assessment Program Manual.

o Avoid coaching and reviewing student responses.

• Thoughtful practices limit invalidations and undetermined results.

Key Test Security Documents & Resources• Documents Pursuant to State Board of Education Administrative Rule,

511 IAC 5-5 (Appendix B)

o 2017-18 Code of Ethical Practices and Procedures

o 2017-18 Indiana Testing Security and Integrity Agreement

o 2017-18 Protocol for Reporting and Investigating Alleged Breaches

• 2017-18 Indiana Assessment Program Manual

o Chapter 13 (Testing Policies, Administration, and Security)

o Test Irregularity and Test Security Concern Report forms (Appendix D)

o Social Media or Unallowable Devices Concern Report form (Appendix D)

References and resources are available via the web at http://www.doe.in.gov/assessment/assessment-program-manual

42

Before Testing: Examiners, Staff Training, & IDOE Monitoring

• Review Test Examiner Qualifications. o Examiners must hold an instructional, administrative, or a school services license granted by the Indiana

Department of Education.

• Provide test administration training to staff prior to each testing window.o CTCs and STCs should use relevant information from the Pretest Workshop to prepare this training for

local staff; Documentation staff have completed such training must be kept on file locally.

• Provide a refresher test security training to staff regarding security policies/ procedures related to the particular assessment prior to each testing window. o Staff must receive refresher test security training for the upcoming assessment preceding the test

window to ensure successful test administration and oversight; Documentation staff have completed a refresher training must be kept of file locally.

43

Test Examiners and Proctors must be trained to understand testing procedures and their responsibilities as Test Examiners and Proctors.

Before Testing: IDOE Monitoring Reminder

• Notify principals and STCs that their school may be selected for an unannounced IDOE monitoring visit

• Review Onsite Monitoring Checklist (Appendix C in Indiana Assessment Program Manual) and share with principals and STCs

44

Test Security Details: Secure Storage• CTCs and STCs must control the secure storage, distribution,

administration, and collection of tests and test tickets (e.g., inventory and track materials).

o Oversee and maintain sign-in/sign-out tracking system.

o End of the testing day verification that all materials have been

returned to secure storage.

o NOTE: A principal’s office, teacher’s room, or any location that anyone has unsupervised access to secure materials at any point is not considered a secure location for the storage of test materials, including online test tickets and access codes.

45

Before Testing: Importance of Appropriate Accommodations

• Examiners administering assessments to students receiving accommodations must:o Be familiar with each student’s individual accommodations

o Have received training on accommodations prior to the test administration

o Reviewed and confirmed accuracy of accommodations noted in the Accessibility Features and Accommodation report (PearsonAccessnext)

o Reviewed and confirmed accuracy of accommodations noted in the Test Admin View Details/Student Logins Screen Reader Column (EAS Admin Site)

During 2017-18 monitoring, the IDOE will review documentation of completed trainings, including:

1) the mechanics of administering the test; 2) test security policies/procedures; and 3) accommodations implementation.

Before Testing: Examiner’s Manuals & Student Test Books

• Access to Examiner’s Manuals (EMs)o Examiners must carefully review EMs prior to test administration.

o Examiners may keep the EMs following test administration training for all assessments except IREAD-3.

o For IREAD-3, early access to the Test Administration Manual (TAM) is permitted ONLY during test administration training, as the TAM contains test content.

o Examiners are not to discuss test content contained in the TAM after the training session.

• Access to student test books/answer documentso NO advance access to student test materials

During Testing: Examiner/Proctor Monitoring

• Examiners/Proctors must monitor test sessions to ensure that all students:o Follow instructions

o Do not exchange answers

o Do not interfere with or distract others

o Use only permitted materials or devices; do not have access to cell phones or other unacceptable devices

To ensure implementation fidelity, Examiners/Proctors must focus

entirely on the assessment administration during each test session.

During Testing: CTC/STC Monitoring

• CTCs or STCs (or Designees) must monitor test administration to ensure:

o All examiners and proctors appropriately monitor students.

o Test security protocols are being followed.

o Cell phones and other inappropriate devices are

not accessible during testing.

o Appropriate accommodations are provided for students with an IEP, ILP, Section 504 Plan, or Service Plan.

49

During Testing: IDOE Monitoring• The Office of Student Assessment (OSA)

conducts unannounced onsite monitoring visits during testing windows;

• The purpose of IDOE monitoring is to ensure fidelity of test administration and test security requirements;

• Schools are identified for monitoring based on previously submitted test irregularities, test security concerns, and a random sample derived from Indiana demographic data; and

• Review Section 2 of Chapter 13 in the Indiana Assessment Program Manual as well as the Onsite Monitoring Checklist (see Appendix C) for more details.

2017-18 Assessment Onsite Monitoring Checklist

Corporation Name/Number

School Name/Number

School Staff Interviewed

IDOE Monitor Name(s)

Date & Time Onsite Conducted

Assessment Monitored ISTEP+ Part 1 ISTEP+ Part 2 ISTAR Part 1

ISTAR Part 2 IREAD-3 ECAs

Overall Summary:

___ No additional follow-up needed

___ Additional follow-up as described in one or more Comment sections is needed

Rating Key: (2) All documentation provided, (1) Some documentation provided, (0) No documentation provided

Monitoring Topic: Communication

Questions: 1. How do STCs and school staff receive communication from IDOE and testing vendors about the assessment? 2. Who develops and shares the Testing Schedule with staff? 3. Please provide a copy of the testing schedule.

# Item Rating Comments

1

Copies of assessment communication CTC has provided/sent to STCs (Emails to STC or Meeting agendas with CTC and STC/Memos to STC, etc.)

2 Copies of assessment communication STC has provided/sent to school staff

During Testing: Item Concern

• If a concern arises regarding the functionality of a test item…

…contact the Office of Student Assessment via email at INassessments@doe.in.gov or by calling 317-232-9050.

o Refer to the item by grade level, content area, session or section number, and item number

• NEVER email a copy of the item, a paraphrased item, or answer choices (not even to the IDOE or a Vendor)!

51

During Testing: Inappropriate Actions

It is NEVER appropriate to:

• Coach students by indicating in any way (e.g., facial expressions, gestures, or the use of body language) that an answer choice is correct or incorrect, should be reconsidered, or should be checked;

o Even if an Examiner or Proctor does not provide the correct answer to a student but provides tips, refers to lessons or examples from the classroom, or shares that an answer is incorrect…This is coaching and is not appropriate.

• Allow students to use any type of mechanical, technical or paper device or aid (calculators, computers, read aloud scripts or text-to-speech) unless the test directions allow such use or the device is documented as a necessary and allowable testing accommodation (see Appendix E);

• Answer students’ factual questions regarding test items or vocabulary;

During Testing: Inappropriate Actions

It is NEVER appropriate to:

• Read any parts of the test to students (except as indicated in the test directions, or as documented as an acceptable IEP, Section 504 Plan, Individual Learning Plan [ILP], or nonpublic school Service Plan). In no case may reading comprehension questions be read to the student; or

• Alter students’ answers during or after testing.

• NOTE: Test questions are not to be reviewed by anyone other than the student during the test session, with two exceptions:

o Test Examiners administering IREAD-3, as the IREAD-3 Test Administration Manual (TAM) contains test content; and

o Test Examiners/ who are providing read-aloud accommodations.

Test Security Details: Security Violations

• Test Security Violation – failure to observe/follow testing security procedures:

• All concerns regarding test security violations must be:

o Reported to the STC and CTC;

o Investigated by the STC and CTC (or designees); and

o Immediately reported to IDOE via the submission of the Test Security Concern Report form in Appendix D.

• Common test security violations from last year relate to the following issues: coaching, staff not adhering to test security protocols, the inappropriate use/access to cell phones, and test information being shared in emails or social media.

Test Security Details: Pictures, Social Media Postings or Unallowable Devices

55

• The Social Media or Unallowable Devices Concern Report form has been updated for the 2017-18 school year;

• CTCs no longer need to submit this form if a cell phone or other unallowable device was in the testing room but it was locally verified and documented that the device was not used and no pictures were taken;

• It is important that STCs and CTCs carefully review the form and follow all of the action steps outlined under the description of the issue that best relates to the situation that occurred.

After Testing: Materials Destruction

Review “Materials Destruction

Guidance” chart for more details

56

• All secure items must be collected and managed as defined in the Test Coordinator’s Manual (TCM) for the respective assessment program.

• Any non-secure items must be collected by the CTC prior to discarding following the test administration window.

• Keep appropriate documentation verifying proper disposal and return of secure materials locally on file

Materials Destruction Guidance

Program

Secure Materials Returned to

Vendor

Secure Materials Destroyed Locally

Non-Secure Materials Collected & Discarded

Locally

ISTEP+

ISTAR

ECA

Test Books Used & Unused

Read Aloud Scripts

Scratch paper

Student Logins/Test Tickets

Extra Barcode/Test Book Labels

Practice Test Materials (ISTEP+ Only)

Unused shipping labels

Examiner’s

Manuals

Test Coordinator’s Manual

Extra paper bands

Unused Session Headers & Session Header Summaries

IREAD-3

Test Books Used & Unused

Test Administration Manual

Extra Barcode/Test Book Labels

Practice Test Materials

After Testing: Item Discussion

• Test items are not to be discussed unless and until the items are released on the web by the IDOE. As such, CTCs should ensure:

o Staff members understand that reviewing test questions with students after testing is complete is not acceptable UNTIL/UNLESS the IDOE releases the items by posting them on the web; and

o Staff members understand that discussing, texting, emailing, or posting actual or paraphrased test questions is not acceptable UNTIL/UNLESS the IDOE releases the items by posting them on the web.

57

Contact Information

• Assessment Office INassessments@doe.in.gov

• Telephone 317-232-9050; Toll Free 888-544-7837

• Fax 317-233-2196

• Website http://www.doe.in.gov/assessment

• Learning Connection https://learningconnection.doe.in.gov

• Questions about test content, format, IDOE policies, testing irregularities, etc.

• STN Application Center: (800) 527-4931

• Office of Special Education: (317) 232-0570

• Pearson Help Desk: (866) 683-6668• Monday - Friday, 7:00 a.m. – 7:00 p.m. ET

• Questions about technology issues, online test sessions, ordering test materials, etc.

Thank you for reviewing this presentation!

Let our office know if you have any questions.

59