Slide 1

Trail Blazers Navigating the Landscape of the Smarter Balanced Field Test with ALL Students June 25, 2014 10:15 11:45 am NCSA Next Generation Assessment Systems

Slide 2

Trail Blazers Idaho - Angela Hemingway Montana Judy Snow South Dakota Jan Martin

Slide 3

Session Overview Each state will describe the journey with details addressing such topics as: Decisions as to why all students in the field test Balancing new assessments with existing systems Political landscape Communication strategies across all stakeholders There will be time for questions and discussion at the end of the session.

Slide 4

Idahos Journey Angela Hemingway Director, Assessment and Accountability

Slide 5

Brief Overview of Statewide Assessments in Idaho 199820002002200420062008201020122014 2016 Idaho adopts standards in core subjects First version of ISAT (NWEA) Second version of ISAT (DRC) Direct Writing & Direct Math End Idaho adopts higher standards in Math & ELA Idaho joins Smarter Smarter Pilot Test Idaho Core Standards Implemented & Smarter Field Test Smarter Operational Test, Digital Library, & Interim Assessments Idaho revises standards in core subject areas Online statewide testing begins

Slide 6

Why test ALL students? New standards were to be implemented SY13-14 Some push back from teachers who were concerned that accountability was tied to the old tests and old standards Some were not going to teach the new standards if they were going to be held accountable to the old standards US DOE flexibility allowed for a chance to avoid double-testing Chance to test our infrastructure Allow for students, teachers and administrators to experience a dress rehearsal before the 2015 operational assessments

Slide 7

Testing All Students Logistics and Technologies All students in Grade 3-8 and 11 took both the math and ELA non-PT and PT Over 164,000 students participated with an 88% completion rate Grade 9 and 10 became optional due to test length (~70% of districts chose to test 9 and 10)

Slide 8

What were the students perceptions? Nearly 10,000 students participated in the statewide survey Surveys were developed by grade band Paper and electronic versions were made available to all schools

Slide 9

What did you think about the questions on the test you took today?

Slide 10

What did you think about navigating the test you took today?

Slide 11

How well did the test you took today match what you learned in class this year?

Slide 12

Grade 3- 5: Were you able to use the keyboard to type your answers?

Slide 13

Students Comments LIKES

Slide 14

Students Comments DISLIKES

Slide 15

Technology Coordinator Survey - What went well with the administration of the Smarter Balanced Field Test in your district?

Slide 16

Technology Coordinator Survey - What were the biggest challenges your district faced in administering the Smarter Balanced Field Test?

Slide 17

Technology Coordinator Survey - How did supporting the administration of the Smarter Balanced Field Test in your district compare to supporting administration of the ISAT in previous years?

Slide 18

Principal Survey - What new expenses do you anticipate to incur to administer Smarter Balanced next year?

Slide 19

Principal Survey - What went well with the administration of the Smarter Balanced Field Test in your district or school?

Slide 20

Principal Survey - What were the biggest challenges your district or school faced in administering the Smarter Balanced Field Test?

Slide 21

Idahos Changing Political Landscape Significant support early in the process All looking forward to higher standards and more robust assessments Then it became publicized that no ISRs would be available Opt outs began.but do not appear to be significant in number (survey in progress) Many schools are changing their policies regarding enrollment and attendance to avoid this situation in the future

Slide 22

What we have learned? Information needs to be thoroughly reviewed internally before its public release Training modules and documents are helpful, but need to be streamlined Formation of a Smarter Balanced Committee helped determine areas of additional need/support and should be formed sooner Edmodo site was very helpful, but required significant staff time; guidance should be provided regarding appropriate posts Surveys were helpful, but need slight modifications

Slide 23

Montanas Journey

Slide 24

56 counties 316 districts 824 schools 332 schools with under 50 students 71 one room school houses Big Sky Country 253 Miles: Longest Bus Route

Slide 25

Why Field Test All Students Commitment to CCSS Support for Computer Adaptive testing Partnerships Dress rehearsal/ trial run/pressure test system

Slide 26

The New and the Old

Slide 27

Political Landscape Paper pencil tests Misinformation Opt out Myths

Slide 28

Technology Number of Montana students starting and completing tests: 72,876 (98%) Only one school without internet capacity Technology Assistance Initiative Contract with META and SAM Montana Educational Technologists Association School Administrators of Montana Regional meetings Final report

Slide 29

Communication July 2013Meetings with partners September 2013official email request waiver Dedicated newsletters and website January Assessment Conference Live weekly 20 minute digital blasts (recorded) Webinars (recorded) Transition liaisons Crash course Surveys Montana educator involvement in development activities and Digital Library

Slide 30

Advice for Districts and Schools Form a Team Use Tech Readiness Tool Use Practice Tests Pilot and debrief

Slide 31

Survey Comments The test went very smoothly. The students were engaged and took the challenge (and it was very challenging) seriously. Everything worked well at our school because of good planning, organization and communication. Navigation was too complicated for many of our students. Special ed students were not able to access their accommodations. I thought that there were very good questions that challenged the students and made them think about the mathematics. Went quite smoothly - no problems at all. There were some glitches in the test that kept the students from answering some problems with certain answers. For example Students feedback was that they enjoyed being able to scroll and check the questions before starting the reading passages. Flagging questions and the glossary were also mentioned as tools they enjoyed

Slide 32

Next Steps Accommodations Practice Tests MOU and Vendor Contract Statewide Training Plan

Slide 33

South Dakotas Journey

Slide 34

Starting with a decision Why test students on old standards? US DOE flexibility allowed for a chance to avoid double-testing. Need to pressure-test the technology infrastructure in the state. Allow for students, teachers and administrators to experience the test environment before it counts.

Slide 35

Navigating the uncharted waters of a double-testing waiver Timing of decisions and information from US DOE. Getting specific commitments from districts with Hutterite Colony students. What to do with students in JDC or other placements? Worked with current vendor to provide needed paper/pencil tests to ensure all students would be tested based on waiver expectations.

Slide 36

Smarter Balanced in South Dakota by the numbers 150/151 Public school systems 20/20 BIE/Tribal systems 36/53 Non-public systems 70,560 student took tests 279,137 tests started 278,164 tests submitted 1,200+ district personnel involved 4 SD DOE Staff with tech support from BIT

Slide 37

Were the schools ready? Yes and no. A key learning was while the schools have the technology, how it is used with students vary across the state. Schools with one-to-one Chromebooks, iPads, or other devices less student issues. -Schools with limited access especially in the elementary, more significant issues.

Slide 38

Were the schools ready?

Slide 39

Balancing new assessment with existing systems Science still paper and pencil - regular and alt Alternate assessment included in the waiver with field test participation voluntary at school level ELL testing still paper and pencil for another two years Schools needing paper/pencil reading and math tests Tech readiness Changes in procedures and timelines School staff overwhelmed with all the changes

Slide 40

Balancing new assessment with existing systems Accommodations was the number one issue for most schools Calculators Read aloud versus text to speech Accommodations versus designated supports Resources to support the schools http://doe.sd.gov/octe/SMARTERbalance d.aspx http://doe.sd.gov/octe/SMARTERbalance d.aspx

Slide 41

Special Education Fall workshops across the state on shifts in accommodations for on-line testing Spring testing workshops Supports provided by SBAC and SD DOE The shift to on-line testing and changes in accommodations (including language used) was one of the biggest hills we still are climbing to assist educators in understanding.

Slide 42

Political Landscape 7 bills and 2 resolutions addressing CCSS and assessments 2 bills would have restricted ability of SD DOE to even work with out-of-state entities

Slide 43

Breaking news. Dateline June 22, 2014 Oddly, the Republican convention chose not to ride another big conservative hobbyhorse, Common Core paranoia. Offered an opportunity to debate the oft-bashed curriculum standards, the convention instead approved a resolution that doesn't mention Common Core but requires that any multi- state standards and tests be approved by the Legislature... because of course, Republicans like to involve big government in education, as long as it's a government that they controlbe approved by the Legislature

Slide 44

Communication strategies for various stakeholders Administrators weekly updates through Secretary of Education, updates on department website including webcasts, area administrator meetings Tech coordinators weekly updates, area tech meetings, direct one-on-one support as needed Test coordinators weekly updates from Assessment Office, updates on department website including webcasts, webinars, workshops

Slide 45

Communication strategies for various stakeholders Teachers monthly updates from on-line newsletters, updates on department website including webcasts Parents news media, letters/newsletters sent by schools, district websites Legislators - face to face meetings, presentations at committee hearings, one-on-one discussions General public media, South Dakotans against the common core, State Board of Education meetings

Slide 46

Survey says.

Slide 47

Themes that emerged from the survey feedback: If students and staff did practice tests, the field test went smoother Tech staff must be involved Shift to on-line testing created confusion and concerns with accommodations Text-to-speech and sound the biggest problem across all respondents. Help desk was used with varying success

Slide 48

Themes continued Training materials not necessarily sufficient or timely Too many last minute changes in procedures Wide variance in district level communication procedures Typing/keyboarding an issue for younger students Misconceptions about Smarter Balanced, field tests, and no scores

Slide 49

Mapping the Future More targeted professional development to specific groups, both in-person and web-based Direct meetings with tech coordinators Manuals and training materials must be teacher- friendly Small districts need more staff to support the district test coordinator (typically the high school counselor) Communication on blueprints, achievement standards, and claims reporting results

Slide 50

Some comments from the field Overall, for the field test, it went well. We all knew the test would be challenging. As a parent, my own son came home and shared that he was very glad that the writing part could be done with a keyboard. He state, I dont like to write, but if I could do it on a computer,that makes it a lot easier! I wrote a lot for the Smarter Balanced test!

Slide 51

And on the other hand None (when asked about any positives to share). It is just another reason why students are hating school. Smarter Balanced is another private corporation going to the bank with public school money.