massachusetts department of elementary & secondary education waiting room today’s webinar will...
TRANSCRIPT
Massachusetts Department of Elementary & Secondary Education
Waiting Room
Today’s webinar will begin shortly.
Massachusetts Department of Elementary & Secondary Education
REMINDERS: Dial 800-503-2899 and enter the passcode
6496612# to hear the audio portion of the presentation
Download today’s materials from the sign-in page: Webinar Series Part 7 PowerPoint slides Using Current Assessments in DDMs
Ramping Up for Next Year:Strategies for Using Current Assessments in DDMs
Webinar Series Part 7
DDM and Assessment Literacy Webinar Series# Title Date Length Time
1Introduction: District-Determined Measures
and Assessment Literacy3/14 60 minutes 4-5pm
2 Basics of Assessment 4/4 90 minutes4-
5:30pm 3 Assessment Options 4/25 60 minutes 4-5pm
TA and Networking Session I 7/11 3 hours9am-12pm
4Determining the Best Approach to District-
Determined Measures7/18 60 minutes 4-5pm
5Measuring Student Growth and Piloting
District-Determined Measures8/15 60 minutes 4-5pm
TA and Networking Session II 9/19 3 hours2:30pm-5:30pm
6Integrating Assessments into Educator
Evaluation: Developing Business Rules and Engaging Staff
10/24 60 minutes 4-5pm
7Ramping up for Next Year: Strategies
for Using Current Assessments in DDMs.
12/5 60 minutes 4-5pm
TA and Networking Session III 12/12 3 hours2:30pm-5:30pm
8 Communicating Results 1/23 60 minutes 4-5pm 9 Sustainability 2/27 60 minutes 4-5pm
Massachusetts Department of Elementary & Secondary Education
4
Audience & Purpose Target audience
District teams that will be engaged in the work of identifying, selecting, and piloting District-Determined Measures.
After today participants will understand: Strategies for using current assessments as a
starting place for District Determined Measures
Educator Evaluation Data SY 2012-2013
Massachusetts Department of Elementary & Secondary Education
6
Data Release – 11/21/13 2012-13 educator evaluation
performance rating data released for 213 RTTT districts 37,940 educators
Majority of Educators Received Solid Ratings 85.2% Proficient
Early evidence of differentiation is encouraging 7.4% Exemplary; 6.8% Needs Improvement;
and 0.7% Unsatisfactory
Ramping Up for DDM ImplementationIdentifying DDMs beyond the minimum pilot areas
Massachusetts Department of Elementary & Secondary Education
8
Key Messages for Stakeholders
Massachusetts Department of Elementary & Secondary Education
9
Ramping Up Goal by June 2014: To identify at least
two measures of student growth to match with each educator for use in the 2014-15 school year.
Massachusetts Department of Elementary & Secondary Education
10
Ramping Up Strategies:
Collaborate with other districts (e.g. network at the upcoming TA sessions, work with area collaboratives)
Engage educators in the process of identifying DDMs
Use assessments currently in use in the district as a starting point for DDMs
Massachusetts Department of Elementary & Secondary Education
11
Why use Current Assessments?
FeasibilityPotentially cost effectiveFamiliarity and experience with the assessments
They are Aligned to ContentAssess what is most important for students to learnAssess what the educators intend to teach
They are InformativeTeachers already use the assessments to measure
student learning Final Step
Provide valuable information to schools and districts about an educator’s impact
Massachusetts Department of Elementary & Secondary Education
12
Considerations in DDMs Performance vs. Growth
Most current assessments are measures of performance and not growth
Growth takes into account the different levels of student achievement
Measures of growth should provide all students an equal opportunity to demonstrate growth
Unit vs. Year Most current assessments are designed to assess
short units DDMs that represent a full year provide more accurate
representations of student growth and of an educator’s impact
Massachusetts Department of Elementary & Secondary Education
13
Potential Solutions Bring together multiple assessments of
similar content to measure performance at different points in time
Administer one assessment (or similar versions) at multiple points during the year
Combine multiple assessments of growth into a single DDM (e.g. pre-post tests of multiple units)
Massachusetts Department of Elementary & Secondary Education
14
The larger DDM contextPreparing
Step Key Questions
1. Create a Team Who will be on the team? What are the key responsibilities?
2. Determine Content to Be Measured
What standards and objectives are the most important to measure?
3. Identify Measure What measures are already in use in the district? Do components need to be developed or modified before piloting?
4. Prepare to Implement Measure
What is the timeline? How will measure be scored? How will the data be collected and stored?
ImplementingStep Key Actions
5. Test Administer the measure Score the results Collect the data
6. Analyze Analyze student results Analyze the administration and scoring processes
7. Adjust Modify components as needed
8. Repeat Test, analyze, and adjust
Massachusetts Department of Elementary & Secondary Education
15
New Guidance from ESE Using Current Assessments in District-
Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units
Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf
15
Massachusetts Department of Elementary & Secondary Education
16
Using Current Assessments: Three Steps
1. Identify key content.
2. Ensure that change in performance represents student growth.
3. Select an approach for measuring growth.
Massachusetts Department of Elementary & Secondary Education
17
Using Current Assessments: Three Steps
1. Identify key content.
2. Ensure that change in performance represents student growth.
3. Select an approach for measuring growth.
Massachusetts Department of Elementary & Secondary Education
18
Step 1: Identify Key Content Key Content may include content
that is: A meaningful sample of the
learning Challenging to teach and learn. Under represented in assessment District priorities.
Massachusetts Department of Elementary & Secondary Education
19
Example: 3rd Grade ELA Looking across the curriculum map to
identify a meaningful sample of content
Massachusetts Department of Elementary & Secondary Education
20
Example: 3rd Grade ELAModel Curriculum Unit Name
Writing Standards Covered in Each MCU
1 2 3 4 5 6 7 8 9 10
Independent Readers X
Author Study X X X X
Biography X X X X
Extreme Weather X X
Newspaper X X X
Poetry X
Reading Inquiry X X
Stories Matter
Whose Story X X
Massachusetts Department of Elementary & Secondary Education
21
Example: 3rd Grade ELA Standard 3 reads (W3.3) Write narratives
to develop real or imagined experiences or events using effective technique, descriptive details, and clear event sequences.
Massachusetts Department of Elementary & Secondary Education
22
Example: 9th Grade US History
Are there commonalities in content covered across multiple units aligned to the MA Frameworks.
Massachusetts Department of Elementary & Secondary Education
23
Example: 9th Grade US History
Model Curriculum Unit Name Standards
Constitutional Rights ELA9-10 W.1: Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.
Economies of North & South RH 10.1: Cite specific textual evidence to support analysis of primary and secondary sources, attending to such features as date and origin of the information. WHST 10.9: Draw evidence from informational texts to support analysis, reflection and research
Industrial Revolutions W.H. 6-8.1 b: Support claims with logical reasoning and relevant, accurate data and evidence that demonstrate an understanding of the topic or text, using credible sources.
Westward Expansion HSS Writing.9-10-1: Introduce precise claim(s), distinguish the claim(s) from alternate or opposing claims, and create an organization that establishes clear relationships among the claim(s), counterclaims, reasons, and evidence.
Massachusetts Department of Elementary & Secondary Education
24
Step 1: Identify Key Content Key content may be taught repeatedly
across the year
Key content may be taught once during the year
Massachusetts Department of Elementary & Secondary Education
Step 1: Identify Key ContentQuestion for Engaging Educators
What can a student do after they take your class that they couldn’t do before?
25
Massachusetts Department of Elementary & Secondary Education
26
Using Current Assessments: Three Steps
1. Identify key content.
2. Ensure that change in performance represents student growth.
3. Select an approach for measuring growth.
Massachusetts Department of Elementary & Secondary Education
Step 2: Does change represent growth?
What is Growth?
Educators understand growth. Growth is about improvement and
learning. There is a difference between growth
and how we measure growth. We can measure growth in both simple
and sophisticated ways. 27
Massachusetts Department of Elementary & Secondary Education
28
Step 2: Does change represent growth? Are the assessments similar enough to
support meaningful inferences about student growth during the year? Do early assessments provide meaningful
information about what students do not understand?
Do future assessments provide meaningful information about what students have learned?
Do students have the opportunity to demonstrate different levels of growth?
Massachusetts Department of Elementary & Secondary Education
29
Example: 3rd Grade ELA
In the third grade example Each unit has a CEPA covering the same
content Rubrics are different, hard to compare
Solution Identify similar evidence Modify rubrics by adding consistently worded
rubric items that assess the same content
Massachusetts Department of Elementary & Secondary Education
30
Looking Across Assessments
Unit Name Rubric Items
1 2 3 4 5 6
Author Study Topic developme
nt and clarity
Evidence and
Content Accuracy
Organization
Standard English
Conventions
Variety of Sentences
Precise Use of
Vocabulary
BiographyOrganization: relevant,
well organized
and detailed
Opinion: Includes
persuasive details that support the
writer’s opinion
Illustrations: Includes engaging
illustrations on each page
Text Features: Includes
text features
Voice: Turns the events into a story
NewspaperThe Writing
Process Use of
Technology
Language and
Conventions
Interviewing and
Quotes
Descriptive Details
Organization
PoetryIncludes
description of topic of
poem
Includes senses or emotions evoked by
poem
Specific words or
phrases to support
response
Personal reflection
Whose StoryTopic
development
Evidence and
Content Accuracy
Illustrations Standard
English Conventions
Massachusetts Department of Elementary & Secondary Education
31
Example: 3rd Grade ELA Conclusion:
The rubrics are similar enough to conclude that change in performance represents growth.
Advantages: Do not need to change the tasks students
will complete Rubrics are on a consistent scale Rubric do not need to be identical
Still include important rubric items that are present in only one CEPA (for example, use of technology in the Newspaper Unit)
Massachusetts Department of Elementary & Secondary Education
32
Example: 9th Grade US History Constitutional Rights: Presentation where
students take on different roles Economies of North & South: Write a
journal, letter, … etc. Industrial Revolutions: Persuasive Writing Westward Expansion: Work in small
groups making exhibit
Massachusetts Department of Elementary & Secondary Education
33
Example: 9th Grade US History Conclusion:
The performance tasks are too dissimilar to conclude that change in performance represents growth.
Next Steps: Identify a way to assess the key content
consistently. For example, administer a single assessment
of the key content at multiple times during the year.
Massachusetts Department of Elementary & Secondary Education
34
Using Current Assessments: Three Steps
1. Identify key content.
2. Ensure that change in performance represents student growth.
3. Select an approach for measuring growth.
Massachusetts Department of Elementary & Secondary Education
35
Step 3: Select an approach Think about the scoring approach that
will best capture student learning given the type of measure you have chosen.
For example: Pre-Test/Post Test Repeated Measures Holistic Evaluation Post-Test Only
Learn more Webinar 5 http://www.doe.mass.edu/edeval/ddm/webinar.html Technical Guide B http://www.doe.mass.edu/edeval/ddm/TechnicalGuide-AppxB.pdf
Massachusetts Department of Elementary & Secondary Education
36
Example: 3rd Grade ELA Each student has five time points. Classroom teacher rates all five time
points, which provides information to parents and teachers about growth across the year.
First and last assignments are rated by two additional teachers. The difference is used to create a growth score. (The sum of the four rubric items. They decide to set standards after the pilot year.)
Massachusetts Department of Elementary & Secondary Education
37
Example: 9th Grade US History Each student completes four writing
assignments. The two US History teachers create a
holistic rubric that looks at four aspects of the writing.
The two US History teachers flag disagreements between ratings and meet to resolve these differences.
Massachusetts Department of Elementary & Secondary Education
38
More Information Using Current Assessments in District
Determined Measures: Leveraging the Curriculum-Embedded Performance Assessments From the Model Curriculum Units
Available at http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf.
Closing
Massachusetts Department of Elementary & Secondary Education
40
Attend Next Week’s TA Session Date: December 12, 2013 Time: 2:30pm – 5:30pm Locations: Southeastern MA: Norwell High
School, Little Theatre, 18 South Street, Norwell, MA 02061
Northeastern MA: Groton-Dunstable Regional High School, 703 Chicopee Row, Groton, MA 01450
Western MA: Frontier Regional School, Media Center, 113 North Main Street, South Deerfield, MA 01373
Register: http://www.surveygizmo.com/s3/1462056/Registration-DDM-Technical-Assistance-Networking-Session-3-December-12th-2013
Massachusetts Department of Elementary & Secondary Education
41
Sharing Locally-Developed DDMs
We need your help! Please submit: Assessment directions, materials, items and
prompts Scoring resources (rubrics, answer keys) Other information (steps for administration,
lesson plans, examples of how results are used)
To submit your examples, complete the survey and upload your materials at https://app.smartsheet.com/b/form?EQBCT=52e3762ac7954f93aa40cc879bc38855.
Massachusetts Department of Elementary & Secondary Education
42
Register for Webinar Series Part 8
Part 8: Communicating Results Date: January 23, 2014
Time: 4-5pm EST (60 minutes)Register: https://air-event500.webex.com/air-event500/onstage/g.php?d=596783122&t=a
Massachusetts Department of Elementary & Secondary Education
43
Questions Contact
Craig Waterman at [email protected]
Ron Noble at [email protected]
Tell us how we did: http://www.surveygizmo.com/s3/1469270/District-Determined-Measures-Assessment-Literacy-Webinar-7-Feedback