TOM TORLAKSON

STATE SUPERINTENDENT OF PUBLIC INSTRUCTION

2017 Accountability Leadership Institute

Traci Albee, AdministratorCarla Najera-Kunsemiller, Program Consultant

English Language Proficiency and Spanish Assessments Office

The ELPAC and CSA: On the Road to Completion but in Different Gears

1

ELPAC Overview

• Pathway to the ELPAC

• Topics for the State Board

• 2017–18 ELPAC Timeline

• Improvements to the 2018 Summative

Assessment (SA)

• SA Important Differences

• Reporting Hierarchy and Approved Weights for the SA

o Student Score Reports

2

ELPAC Overview (cont.)

• Reclassification Guidance

• English Learner Progress Indicator (ELPI)

• Initial Assessment (IA) Key Information

• IA Reporting Hierarchy and Approved Weights

• Resources and Trainingso Practice Tests

o Administration and Scoring Trainings

o New ELPAC Resources

o Upcoming ELPAC Educator Opportunities

3

IA FT Speaking

Range Finding

Apr./May 2017

Content Review and

Bias and Sensitivity

Review Panel Mtgs.

August 2016

SA Operational Test

Administration

Feb. 1–May 31, 2018

= We are here.

Pathway to the ELPAC

4

General

Performance Level

Descriptors Meeting

June 2016

SA Standard Setting

October 2017

SA Operational Test

Training October–

December 2017

Item Writing

February 2016

IA FT Writing Range

Finding

October 2017

SA/Initial

Assessment (IA)

Field Test (FT)

Training

February 2017

Speaking Range

Finding

October 2016

Speaking Response

Collection

September 2016

SA FT Writing

Range Finding

May 2017

IA FT

Administration

Aug. 28–Sept. 22,

2017

IA Standard Setting

February 2018

Summative

Assessment (SA)

Task Type Pilot

Dec. 2015

SA FT

Administration

Mar. 6–Apr. 14, 2017

Topics Approved or Reviewed by the State Board

• September 2017o Approved revisions to the Summative Assessment (SA) test blueprintso Approved reporting hierarchy of the SA and Initial Assessment (IA)o Approved revisions to the general performance level descriptors (PLDs)

• October 2017o Reviewed the standard setting process and threshold

score validation study

• November 2017o Approved the operational SA preliminary threshold scores and

composite weights

• December 2017o Reviewed the SA Student Score Reports

5

Topics to be Approved or Reviewed by the State Board• January 2018

o Approve the IA PLDs

• February 2018

o Review the IA standard setting process

• March 2018

o Review the IA revised test blueprints

• April–May 2018

o Approve the operational IA preliminary threshold scores and composite weights

• September–November 2018

o Approve the threshold validation study

• November–January 2019

o Approve the English learner reclassification guidelines 6

7

Questions?

8

TOM TORLAKSON

STATE SUPERINTENDENT OF PUBLIC INSTRUCTION

Summative Assessment (SA)

9

Improvements to the 2018 SA

• Audio recordings for Speaking––Summarize an Academic Presentation Grades 3–12

• Grades 3–12 students will be able to write in their Test Books

• Stopping points and break times added

• In grades 6–12, graphics for the Speaking task types (Present and Discuss Information and Summarize an Academic Presentation) are added to the Test Book

• For a complete list, you can access the flyer on the ELPAC.org website.

10

SA Important Differences

• SA Window: February 1–May 31 annuallyo Any student who is identified as an English learner and is

enrolled during the SA window must test.

• Data Correction: There is no Data Review Module for the ELPAC.o Local Educational Agencies (LEAs) need to ensure their

student demographic data in the CALPADS Operational Data Store (ODS) is up-to-date.

o LEAs are encouraged to use Pre-ID labels to ensure further accuracy. ELPAC Pre-ID label costs have been reduced to help with

accurate demographics.

• Materials ordering: LEAs have the ability to order supplemental materials more than once per month.

11

SBE-Approved Reporting Hierarchies for the SA

12

• Address the integrated skills of the 2012 English language development (ELD) standards

• Are the result of a dimensionality study

• Generate valid and reliable information for parents, teachers, and schools/LEAs

K–12 Reporting Hierarchy for the SA

Overall Scale Score

Four Performance Levels

Oral Language Scale Score

Four Performance Levels

Listening

Three Performance Levels

Speaking

Three Performance Levels

Written Language Scale Score

Four Performance Levels

Reading

Three Performance Levels

Writing

Three Performance Levels

13

Questions?

14

Approved Weights for the SA Composite and Overall Scale

Scores

15

Oral Language

Composite

WrittenLanguage

Composite =

Overall CompositeScale Score

Kindergarten 70 30 100

Grades 1–12 50 50 100

SA Student Score Report

CALIFORNIA DEPARTMENT OF EDUCATION 16

First-Year Student Score Report, front page Three-Year Student Score Report, front page

SA Student Score Report(cont.)

CALIFORNIA DEPARTMENT OF EDUCATION 17

Student Score Report, back page (all years)

Reclassification Guidance

• The four criterion identified in California Education Code 313 are still required.

• In April 2017, the CDE provided guidance to LEAs regarding English language proficiency assessment options in 2017–18.

• Further guidance will be forthcoming in early 2018, which will give LEAs local control to use the ELPAC preliminary threshold scores for reclassification.

18

English Learner Progress Indicator (ELPI)

As the ELPI transitions to the ELPAC:

• Results from the California English Language Development

Test (CELDT) will not be used with the ELPAC to calculate

the ELPI.

• The CDE will run data simulations on ELPAC data

beginning in January 2018.

• The Technical Design Group will advise the CDE on the

use of the ELPAC in the ELPI.

• The CDE will seek stakeholder input on using the ELPAC

results in the ELPI.

• The SBE will determine what is reported in the Fall 2018

Dashboard. 19

ELPI (cont.)

• For California Accountability Model and School Dashboard (ELPI) Questions:

o Academic Accountability Team |DASHBOARD@cde.ca.gov| 916-319-0863

• For DataQuest Questions:

o Data Visualization and Reporting Office |CDEdatarequests@cde.ca.gov| 916-322-3245

20

TOM TORLAKSON

STATE SUPERINTENDENT OF PUBLIC INSTRUCTION

Initial Assessment (IA)

21

IA Key Information

22

• The window for the operational ELPAC IA will start July 1, 2018; no more CELDT.

• Trainings for the IA will begin in spring 2018. Every LEA must participate in the training.

• By April 1 of each year, an LEA ELPAC Coordinator must be designated for both the IA and SA.

• LEAs will locally score the IA for the official score using the Local Scoring Tool (LST).

• LEAs must upload their Student English Language Acquisition (SELA) record at the same time as their enrollment record to enter the score into the LST.

• Each year, a rotating 25% of LEAs will be required to send their scored tests to the contractor for statistical item analyses and to identify LEAs that may need assistance with scoring or using the LST.

23

IA Key Information (cont.)

K–12 Reporting Hierarchy for the Initial Assessment

Overall Scale Score

Three Performance Levels

Oral Language

Three Performance Levels

Listening Speaking

Written Language

Three Performance Levels

Reading Writing

24

Questions?

25

TOM TORLAKSON

STATE SUPERINTENDENT OF PUBLIC INSTRUCTION

Resources and Training

26

Practice Tests

• Posted on October 30, 2017 at https://www.elpac.org/resources/practicetests/

• Provide sample test task types and questions

• Include directions for giving the practice test to students

• Provide a sense of how the 2012 ELD Standards are assessed on the ELPAC

27

Practice Tests(cont.)

• Students:o Become familiar with the task types and questionso Learn how to provide their answers

• Parents: o Understand the types of English-language skills

neededo Understand what students need to do on the

actual test

• Teachers: o Review with students and their familieso Create similar tasks for instructional purposes

• Administrators: o Communicate with students, families, and

teachers about the skills assessed in the test 28

Administration and Scoring Trainings

29

• The CDE-sponsored statewide trainings for the first ELPAC operational SA:

o Started October 24, 2017 and will end December 13, 2017

• Upcoming CDE-sponsored ELPAC IA trainings:

o April 10, 2018 through May 4, 2018

o Registration opens February 1, 2018

• SA and IA trainings: Access the registration link on www.ELPAC.org at https://ord.scoe.net/elpac/pg00.php.

SA Regional Administration and Scoring Trainings

Location Day Date

Butte County Office of Ed (COE) Wednesday Dec. 13

Fresno COE Friday andWednesday

Jan. 12Jan. 31

Kern County Superintendent of Schools WednesdayFriday

Nov. 29 (K-5)Dec. 1 (grades 6-12)

Placer COE WednesdayTuesday

Jan. 31, 2018Feb. 20, 2018

Riverside COE (Indio Office) ThursdayTuesday Tuesday

Dec. 14Dec. 19January 23

San Joaquin COE Thursday Nov. 30

San Mateo COE FridayTuesdayMonday

Dec. 1Dec. 19Jan. 22, 2018

30

SA Regional Administration and Scoring Trainings (cont.)

Location Day Date

Santa Clara COE Wed.FridayThurs.

Nov. 29Dec. 13Jan. 4, 2018

Santa Cruz COE Mon.Tues.

Dec. 11 (grades 6-12)Dec. 12 (K-5)

Stanislaus COE Wed.Thurs.Fri.

Nov. 8Dec. 14Jan. 19, 2018

Tehama COE Tues. Jan. 16, 2018

Tulare COE Mon.Thurs. Thurs.

Dec. 11Jan. 11Jan. 18

31

Access the Regional Trainings for the SA on www.ELPAC.org at:

https://www.elpac.org/training/summative/regional

New Resources

• Do we administer the CELDT or the ELPAC in 2017–18?

• ELPAC Assessments Fact Sheet

• A Parent Guide to Understanding the ELPAC

o Translated into seven languages

• Domain Information Sheets

• Matrix 4:Universal Tools, Designated Supports, and Accommodations for the ELPAC

These resources are located on the CDE Web site at http://www.cde.ca.gov/ta/tg/ep/. 32

New Resources(cont.)

33

http://www.cde.ca.gov/ta/tg/ep/documents/howtotestel.pdf http://www.cde.ca.gov/ta/tg/ca/documents/elpacassessmentfs.pdf

Parent Guide to Understanding the ELPAC

34http://www.cde.ca.gov/ta/tg/ep/elpacparentguide.asp

Domain Information Sheets

35

Take Off, Touch Down

Your LEA attended an ELPAC Administration and Scoring Training this fall.

Your LEA has ordered their ELPAC test materials.

You have seen the new movie Coco.

Your LEA is administering the ELPAC Summative in February and/or March.

You can explain at least one difference between CELDT and ELPAC.

You vacationed in another country this year.

You plan to use the new ELPAC resources with parents/guardians and staff.

36

You are bilingual.

TOM TORLAKSON

State Superintendent of Public Instruction

California Spanish Assessment (CSA)

37

CSA Overview

• Purpose and Benefits

• CSA Development Process

• Field Test Item and Passage Review Meetings

• California Educator Reflections

• Pilot Test and Cognitive Labs

• Test Blueprints

• General Achievement Level Descriptors (ALDs)

38

Purpose and Benefits

The Purpose:

The CSA will:

• Measure a student’s competency in Spanish reading/language arts and provide student-level data

Benefits:

• Evaluate the implementation of Spanish reading/language arts programs at the local level

• Provide a high school measure suitable to be used, in part, for the State Seal of Biliteracy

39

Sept. 2017: Pilot Test and Cognitive

Labs

Fall 2018: Field Test

Oct. 2016-July 2017: Item dev and item & passage review

meetings

Sept. 2016: SBE approval of the high

level test design

CSA Development Process

Nov. 2017: SBE approval of ALDs and test blueprints

40

Spring 2019: Operational CSA

= We are here.

Field Test Item and Passage Review Meetings

July 17 through July 21, 2017––CSA Item and Passage Review Meetings in Sacramento

• 28 California educators participated in reviewing approximately 800 field test items

• California educators who participated are native to the following Spanish-speaking countries: Chile, Spain, Mexico, Guatemala, Argentina, Panama, and Peru

41

California Educator Reflections

“The test items are

rigorous and are

comparable to the

SBAC.”

Otilia Vargascorona,

Victor Valley Unified

School District (USD)

“It is obvious

that a great

deal of time and

thought were

invested in the

development of

the CSA test

items.”

Karla Diaz,

Beaumont USD

Providing a

rigorous test will

increase the

quality of the

Spanish taught in

California.”

Claudia Flores,

Palmdale SD

“Technology enhanced-items are a must in

education in the 21st century and must be

reflected in [assessments].”

Sandy Barrientos, Victor Valley UHSD

42

Pilot Test

• The CSA pilot test occurred September 18 through October 6, 2017.

• Six LEAs throughout California participated in the pilot test.

• The CSA pilot test confirmed the functionality of the technology-enhanced items.

• LEAs interested in participating in the fall 2018 field test may e-mail Marques Barron, Program Consultant at mbarron@ets.org.

43

CSA Cognitive Labs

• Conducted in three schools in the San Juan Unified School District

• Included students in grades 3–8 and high school with varying levels of proficiency in Spanish

• One researcher per student

• Participated in a sub-set of questions from the pilot test

Purpose:

• To test how students interact with the technology-enhanced items

• To test how students interact with the user interface

44

Test BlueprintsThey provide the number of items and points by claim and content standard

of an operational assessment for each language-arts domain (Reading,

Writing Mechanics, and Listening) assessed in grades three through eight

and high school.

Key points on the content:

• The linguistic augmentations included in the Common Core State

Standards en Español are included in the blueprint.

• Due to limited funds, the Writing domain will consist of writing mechanics

items that can be machine-scored, rather than constructed-response

items.

• The CSA blueprint does not include Speaking items; speaking is to be

assessed at the local level.

The test blueprints were approved by the board on November 8, 2017. To

view click on:

https://www.cde.ca.gov/be/ag/ag/yr17/documents/nov17item07a3.pdf45

General ALDs

• Provide generic descriptions of student performance level expectations across three levels

• Used to set grade level-specific ALDs and develop reporting ALDs

The ALDs were approved by the SBE on November 8,

2017. To view click on:

https://www.cde.ca.gov/be/ag/ag/yr17/documents/nov17it

em07a4.pdf

46

Contact Information

English Language Proficiency and Spanish

Assessments Office

Phone: 916-319-0784

CELDT e-mail: celdt@cde.ca.gov

ELPAC e-mail: elpac@cde.ca.gov

CSA e-mail: caaspp@cde.ca.gov

CELDT Web page: http://www.cde.ca.gov/ta/tg/el/

ELPAC Web page: http://www.cde.ca.gov/ta/tg/ep/

CSA Web page: http://www.caaspp.org/ (under the Test Administration tab)

Join the CDE’s ELPAC e-mail list by sending a blank e-mail to:

subscribe-elpac@mlist.cde.ca.gov.

47

Questions?

48

During Conference

• Will test examiners be able to pause between the listening items for students with disabilities or section 504s? Yes, further guidance will be provided in the ELPAC Update in mid-December.

• Please e-mail any questions about unlisted resources not included/described in Matrix 4 to ELPAC@cde.ca.gov.

49

During Conference (cont.)

• Does California have a statewide alternate ELP assessment aligned to alternate ELP standards? No, current practice allows IEP teams to determine what alternate assessment is best for assessing their students for each domain, but the state has asked the legislature for funds to develop a new alternate assessment.

• Reminder that with all state assessments, an RFP must be written (@1 year process) and test development takes 3 years.

50

During Conference (cont.)

51