regents scanning pilot –cfn 201
DESCRIPTION
Regents Scanning Pilot –CFN 201. Amy Reynolds, Assessment Liaison Damarine Reyes, Data/IT Liaison. Agenda. Overview of new Regents data requirements & DOE’s plan to meet these requirements Scanning Process Key points for each step in the process Method - PowerPoint PPT PresentationTRANSCRIPT
1
Regents Scanning Pilot –CFN 201
Amy Reynolds, Assessment Liaison Damarine Reyes, Data/IT Liaison
22
Agenda> Overview of new Regents data requirements & DOE’s plan to meet
these requirements > Scanning Process
Key points for each step in the process Method
> Next Steps: Using Regents data to inform instructional decisions ATS reports overview Activity
> Discussion / Questions > Support
3
Regents Exam Data New York State Department of Education: New Requirement: Beginning in June 2011, NYSED will require that ALL Districts submit item-level data for select Regents exams. This includes the following subjects:
• English Comprehensive (not included for January pilot)• Integrated Algebra• Earth Science• Living Environment• US History & Government• Global Studies & Geography
Impact: Student responses need to be scanned to efficiently capture the required item-level data without impacting the turnaround time for results required by the schools.
Note: •Teachers must still score constructed response portions of exams•Other Regents exams and RCTs will still use the traditional scoring process
4
DOE Data Capture Solution To leverage existing technology and processes currently in use in schools to capture item-level information & provide real-time results. This includes:
• Answer Documents: Generated & printed at school-level• Scoring: Teachers to score Constructed Response items only• Scanning: Using schools’ attendance scanners, locally scan documents on-site• Reporting Results: Automated transfer from ATS to STARS
Benefits:• Increased automation of time-consuming manual processes
• Less overall time required to complete scoring by teachers• No further data entry requirements of aggregate scores
• Reduction in clerical scoring errors• eliminate manual scoring of MC items• adding totals from different sections• application of conversion tables
• Districts/schools/teachers will be able to better utilize item-level results to help inform instruction, similar to Grade 3-8 exams
STARS ATS Attendance Scanner
ATS STARS (and beyond)
1. Schedule students for Regents Exams in STARS
2. Generate & Print answer documents for each scheduled student
3. Answer documents printed for each scheduled student
6. Scan answer documents for upload to ATS
7. ATS receives data, applies scoring key & conversion chart for scale score instantly.
8. ATS passes data nightly to STARS for Report Cards and Transcripts and for reporting to NYSED
Exam DayScoring
4. Student uses pre-slugged answer document
5. Scoring committee marks Open-Ended items and bubbles in score on answer document.
Attendance Scanner
Re-scanning (if needed)
Scanning Process Overview
5
STARS – (Review. Nothing has changed)Schedule students for Regents exam in STARS• Who is responsible for STARS scheduling?
• Program chair of each school usually handles STARS scheduling process
• STARS scheduling process remains exactly the same as before
• Major Consideration: Should I schedule one section or multiple sections for each Regents exam?• One section:
• Shorter scheduling process • Use “Master Schedule” function
• Multiple Sections:• Allows you to break students up into certain groups, assign each group
to a certain room, print attendance rosters by exam room, and print exam invitation reports
• Use “Master Schedule Builder” functionFor support with using STARS, contact your network’s Data/IT person
6
ATSPrint answer documents from ATS to attendance scanner• Who is responsible for ATS printing?
• Pupil accounting secretary
• Print answer documents up to a week before Regents week
• There are three printing options: Regents Scanning Handbook / ATS Screenshots, page 20
1. “Print scheduled students not previously printed”- these students have been scheduled in STARS Print all answer sheets that have not been printed yet
2. “Print by student ID” Print an answer sheet for any student one at a time. This option could be for
another public school student taking the exam at your school3. “Print by name”
Print an answer sheet for a non-public school student who is taking the exam at your school
For support with using ATS, contact your network’s Data/IT person.
7
Log in to ATS. On the screen that appears below type: REGT
8
ATS Screenshots
TAB to RGPR and press RIGHT/ CTRL/RED ENTER
9
ATS Screenshots
Exam DayUse printed answer documents for test taking & scoring • Students use pre-slugged answer documents on the day of the exam to
answer multiple choice questions
• Students and Teachers need to use PENCIL for marking responses on answer documents• Responses are only to be marked in the new answer document
• Proctors ensure that student answer documents are signed and bubbled in appropriately.
• Scoring team bubbles in their scores for the open-ended questions on the teacher portion of the answer document • Teacher portion for all January Regents exams is page 2, except for the
one-page Integrated Algebra form
10
Attendance scanner Scan answer documents for upload to ATS
• Do not scan until you’ve received notice from the DOE• Use your school’s attendance scanner
• Scan the answer documents face up & bar code to the left
• Scan both pages for 2-page answer documents • Batch page 1 documents separately from page 2 documents
• Scanner will print a report of how many documents were scanned successfully; for those not scanned successfully the document will be reprinted with an error message on it
• Rescan answer sheets if necessary• Audit trail captured of re-scans and changes to the item responses
• Scanning must be complete by the end of the Regents window
11
For support with using the attendance scanner, contact your network’s Data/IT person or call the DIIT Helpdesk at (718) 935-5100
ATSATS processes scanned data and makes it
available for use• ATS receives data, applies scoring key and conversion chart
for scale score instantly
• If both pages of a 2-page answer document are not scanned, scale score will not be displayed (Incomplete Record)
• ATS passes data nightly to STARS for Reports Cards and Transcripts, and for reporting to NYSED, etc.
12
Accessing Results
ATS Reports Available for Printing• Document Status Report
• Ensure that answer documents have been properly scanned.
• Item Report• Analyze the Regents results for individual students and inform
decisions on how to move these particular students forward.• Review error report – multiples and omissions
• Item Distribution Report• Analyze the Regents results for the student group as a whole in
order to inform instructional decisions for these students as well as future students.
13
Document Status ReportAllows you to ensure that answer documents have been properly
scanned.
14
Student Name and ID
Raw and Scale Score
Scan Date and Time
# of Answer Sheets Generated
# of Answer Sheets Scanned Errors Warnings
Item ReportAllows you to analyze the Regents results for individual students and
inform decisions on how to move these particular students forward.
15
Student Name
Student ID #Raw and Scale score
• Question number• Answer key • Student answer choices• Open-ended question
scores
Dash = correct answer
Item Distribution ReportAllows you to analyze the Regents results for the student group as a whole
in order to inform instructional decisions for these students as well as future students.
16
Question Number
For each answer choice, number and percentage of students that selected it
Total number of responses for each question
Using data to inform instructional decisionsITEM REPORT
• Questions this report could help you to answer• Which students scored the highest?• Which students scored the lowest?• Which students will have to retake this Regents exam?• Which questions did individual students have trouble with?• Are there any patterns in terms of what skills these questions focus on?
• Next Steps• Plan instructional interventions for students who will need to retake the
Regents• Support students who passed the Regents with further mastery of the areas
that they had trouble with**Keep in mind that you will need to have a copy of the corresponding
Regents exam to see the actual questions which correspond to the data.
17
Quality Assurance – Data Validation• 5% manual verification
• Schools need to manually verify the scores of 5% of the answer documents for each exam have been scored properly.
• Identify two people at school level perform the data verification
• CFN Assessment Liaison will collect signed document from each pilot school• Document will indicate that the school has performed the 5%
manual verification
Review - Scanning Process
Pre-Exam - Regents Scanning Implementation Checklist• STARS – Schedule students for Exams• Supplies: Paper, toner• Printing answer documentsExam Day - Regents Scanning Implementation Checklist• Personnel available to printout extra answer sheets if necessary• Students use PENCIL on the answer documents Post-Exam - Regents Scanning Implementation Checklist• Complete score / open-ended questions. • Wait for official notice from DOE to begin Scanning • Accessing reports• Quality Assurance – Data Validation Documentation
19
20
Ongoing Support for Assessment Liaisons
Technical Support•Scheduling, ATS and STARS
> Contact your Data/IT Liaison•Scanner and printer
> Contact the DIIT helpdesk at (718) 935-5100
Overall Process Support•Contact your Borough Assessment Implementation Director (AID)
•Queens – Barbara Marcisak
Training Documents•Access documents from today’s training in ARIS Connect’s Summative Assessment Implementation Community: http://tinyurl.com/regents-scanning