connecting summative assessment to improving …€¦ · ppt file · web view2016-12-06 · 4....
TRANSCRIPT
Measuring the Power of Learning.™
Connecting Summative Assessment to Improving
Teaching and LearningMay and June 2016
California
http://tinyurl.com/h7d4hub/
Measuring the Power of Learning.™
2
Workshop GoalsParticipants will learn:• the rightful place and purpose of the summative
assessments • to use results to inform the instructional process by
connecting curriculum, instruction and assessment• to access supplementary resources and reference
materials to assist local training• to use tools to reflect on current practice to improve
teaching and learning• a variety of ways to display data to communicate a
particular message
Measuring the Power of Learning.™
3
The Many Faces of Testing
Measuring the Power of Learning.™
4
Overview of Major Assessment Types
http://www.csai-online.org/sites/default/files/resources/4666/CSAI_AssessmentTypes.pdf
Measuring the Power of Learning.™
5
ClassroomFormative
Statewide Summative
Rightful Place/Purpose: Assessment Frequency
and Impact on Instruction
Measuring the Power of Learning.™
6
Standards set expectations
on path to college- and
career- readiness
All students graduate college-
and career- ready
Teachers and schools have
information and tools to improve
teaching and learning
Interim assessments Flexible, open, and used for actionable feedback
Summative assessments benchmarked to college and
career readiness(Grades 3–8 and Grade 11)
Digital Library Formative assessment
tools and practicesfor teachers to improve
instruction
A Balanced Assessment System
Measuring the Power of Learning.™
7
Statewide summative assessments are like icebergs—it pays to pay
attention
Measuring the Power of Learning.™
8
Rightful Place, Rightful Purpose for Statewide Summative Assessment
• Provide a general direction—we must dig deeper to determine cause– Focus on groups, programs, and disaggregation
• Rarely provide definitive answers, but raise many questions, allowing reflection on context and practice
• Provide an entry point into a collaborative, honest conversation
• Provide a necessary story
Measuring the Power of Learning.™
9
Using Summative Results • Comprehensive, complex, and difficult process
– Not a checklist– Not meant to be completed at a single staff meeting
or professional development event• Focus on improving learning
– Not solely about increasing scores• Collaborative process that requires:
– Honesty– Willingness to commit the time– Ability to handle ambiguity– Patience
Measuring the Power of Learning.™
10
Data Examination Is Complicated
Measuring the Power of Learning.™
11
. . . about the rightful place/purpose of summative
assessment results How do you currently use statewide summative
assessment results in the curriculum, instruction, and assessment planning process within your LEA? What is the place and purpose of the
summative assessment in your LEA right now?
Measuring the Power of Learning.™
ITEMS
Evidence Statements
Content Claims
Targets
The Smarter Balanced Hierarchy ofItem Development and Reporting of Scores
Overall Claims
Evidence-Centered Design
Measuring the Power of Learning.™
13
1. Define the domainCommon Core Standards—
Mathematics; English Language Arts/Literacy (ELA)
2. Define claims to be made Four ELA & Four Mathematics ClaimsContent Specifications
3. Define assessment targets Knowledge, Skills, and Abilities
4. Define evidence required Evidence to be Elicited from Student
5. Develop items or performance tasks Methods for Eliciting Evidence
Concepts of Evidence-Centered Design
Measuring the Power of Learning.™
14
English Language Arts and
Literacy in History/Social
Studies, Science, and
Technical Subjects
English Language
Arts/English Language
Development Framework
English
Language Arts/Literacy
Content Specifications
(Claims)
Relationships Between Common Core Standards, California Frameworks, and
Assessments
Measuring the Power of Learning.™
15http://www.smarterbalanced.org/assessments/development/
ELA/Literacy Content Specifications(Grade 7 ELA Example)
Measuring the Power of Learning.™
16
English Language Arts and
Literacy in History/Social
Studies, Science, and
Technical Subjects
English Language
Arts/English Language
Development Framework
English
Language Arts/Literacy
Content Specifications
(Claims)
English Language
Arts/Literacy Item
Specifications
(Claims and targets per
claim)
Relationships Between Common Core Standards, California Frameworks, and
Assessments
Measuring the Power of Learning.™
17http://www.smarterbalanced.org/assessments/development/
Example of Item Specifications(Abbreviated)
Evidence Required for Target 1
The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text.
Measuring the Power of Learning.™
18
English Language Arts and
Literacy in History/Social
Studies, Science, and
Technical Subjects
English Language
Arts/English Language
Development Framework
English
Language Arts/Literacy
Content Specifications
(Claims)
English Language
Arts/Literacy Item
Specifications
(Claims and targets per
claim)
English Language
Arts/Literacy Assessment
Blueprint
Relationships Between Common Core Standards, California Frameworks, and
Assessments
Measuring the Power of Learning.™
19http://www.smarterbalanced.org/assessments/development/
Sample ELA/Literacy Summative Assessment Blueprint
Measuring the Power of Learning.™
20
English Language Arts and
Literacy in History/Social
Studies, Science, and
Technical Subjects
English Language
Arts/English Language
Development Framework
English
Language Arts/Literacy
Content Specifications
(Claims)
English Language
Arts/Literacy Item
Specifications
(Claims and targets per
claim)
English Language
Arts/Literacy Assessment
Blueprint
English Language
Arts/Literacy Summative Assessment
MathematicsMathematics
Framework for California
Public Schools
Mathematics
Content Specifications
(Claims)
Mathematics Item
Specifications
(Claims and targets per
claim)
Mathematics Assessment
Blueprint
Mathematics Summative Assessment
Relationships Between Common Core Standards, California Frameworks, and Assessments
Measuring the Power of Learning.™
21
Student Response
Evidence
Assessment Target
Claim
Common Core State Standards
Assessment Claim
Assessment Target
Evidence
Student Response
Building a Logical Argument
Measuring the Power of Learning.™
22
. . . about the test design and the connections to the
instructional cycle To what extent do the teachers at your school or LEA
understand how the test is constructed and how it is connected to the standards and the frameworks? How does this connection have implications for instructional planning?
Measuring the Power of Learning.™
Review of Available Summative Scores
Measuring the Power of Learning.™
24
• Scale scores• Achievement levels
– not met, nearly met, met, exceeded
• Claim levels– below, near, above
• Target scores– worse, similar, better– insufficient information
Available Summative Assessment Scores
Measuring the Power of Learning.™
25
Scale Score – Achievement Level Relationship – Grade 3 Example
Scale Score Range Smarter Level CA Achievement Level
≤ 2366 Level 1 Standard Not Met
2367 ― 2431 Level 2 Standard Nearly Met
2432 ― 2489 Level 3 Standard Met
≥ 2490 Level 4 Standard Exceeded
Measuring the Power of Learning.™
26
Smarter Balanced ELAThreshold Scale Scores
Grade Level 2 Level 3 Level 4
Grade 3 2367 2432 2490
Grade 4 2416 2473 2533
Grade 5 2442 2502 2582
Grade 6 2457 2531 2618
Grade 7 2479 2552 2649
Grade 8 2487 2567 2668
Grade 11 2493 2583 2682
http://www.cde.ca.gov/ta/tg/ca/sbelathresholdscre.asp
Measuring the Power of Learning.™
27
Low, Medium and High Bands
2367to
2387
2388to
2408
2409to
2431
2432to
2450
2451to
2469
2470to
2489
2416to
2433
2434to
2452
2453to
2472
2473to
2491
2492to
2511
2512to
2532
(Divide the difference by three =
21 pts)
(Divide the difference by three =
21 pts)
(Divide the difference by three =
21 pts)
(Divide the difference by three =
21 pts)
Measuring the Power of Learning.™
28
Low, Medium and High Bands (cont.)
2432
2470
2015
2016
Measuring the Power of Learning.™
29
Comparing Year to Year
http://www.cde.ca.gov/ta/tg/ca/caasppssrinfo.asp
Measuring the Power of Learning.™
30
English Language Arts/Literacy Claim #1
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.
Claims are broad, evidence-based statements about what students know and can do as demonstrated by their performance on the assessments. For example:
Claim Scores: Example of a Claim
Measuring the Power of Learning.™
31
• Reported at an individual and aggregate level• Reported for all claims in ELA and mathematics
Use of Claim Scores
Measuring the Power of Learning.™
32
• Assessment targets connect the content standards to evidence that will be collected from the assessment.
• Targets map the Common Core State Standards (CCSS) onto assessment evidence that is required to support the content categories and claims.
• Targets are used to guide the development of items and tasks that will measure the CCSS.
Assessment Targets
Measuring the Power of Learning.™
33
Target 1
Key Details:Identify explicit textual evidence to support inferences made or conclusions drawn.
Example of an Assessment Target(Grade 7 ELA Claim 1 Example)
Measuring the Power of Learning.™
34
• Reported at an aggregate level relative to group performance• Reported for all claims in ELA, but only for
Claim 1 in mathematics• Only targets with 10 or more items in the item pool for a grade
are included in target reports
Use of Target Scores
http://www.caaspp.org/rsc/pdfs/CAASPP.target-report-FAQs.2016.pdf
Measuring the Power of Learning.™
35
What’s one thing you will take away from this quick review of the scores?
What additional information do you need?
. . . about the summative assessment scores
Measuring the Power of Learning.™
Tools for Analysis of the
Smarter Balanced Summative
Results
Measuring the Power of Learning.™
37
• Content Specifications• Item Specifications• Blueprints• Achievement Level Descriptors (ALDs)
− Policy, range, threshold, and reporting• Claim Descriptors• Score Reports• Additional Data
Supporting Documentation
Measuring the Power of Learning.™
38
• Text descriptions of the knowledge, skills, and processes demonstrated by students at each level.
• Four types of levels or categories of performance– Policy and Content– Range– Threshold– Reporting
Achievement Level Descriptors
http://www.smarterbalanced.org/assessments/scores/
Measuring the Power of Learning.™
39
Base for the ALDs (page xvi)
Measuring the Power of Learning.™
40
Range ALDs
• Composed of grade- and content-specific descriptors
• Describe the cognitive and content rigor that is encompassed within particular achievement levels
• Describe the knowledge, skills, and processes expected of students
• Used by test developers to guide item writing
Measuring the Power of Learning.™
41
Range ALDs (Grade 7 ELA Claim 1 Example)
http://www.smarterbalanced.org/assessments/scores/
Level 1 Level 2 Level 3 Level 4
Measuring the Power of Learning.™
42
Range ALDs (cont.)(Grades 6-8 Math Claim 2, 4 Example)
http://www.smarterbalanced.org/assessments/scores/
Level 1 Level 2 Level 3 Level 4
Measuring the Power of Learning.™
43
Relationship Between Range and Threshold Descriptors
Adapted from Figure 1 in the Initial Achievement Levels Descriptors document.
Students performing at Level 1 should be able to identify textual evidence that minimally supports a basic idea drawn about texts of low complexity.
The student who just enters Level 2 should be able to…
The student who just enters Level 3 should be able to…
The student who just enters Level 4 should be able to…
Students performing at Level 2 should be able to cite relevant textual evidence to support a simple inference, analysis, interpretation, or conclusion …
Students performing at Level 3 should be able to cite sufficient and relevant textual evidence that adequately supports a complex inference…
Students performing at Level 4 should be able to cite strong and thorough textual evidence to support a complex inference, analysis…
Level 1 Level 2 Level 3 Level 4
Measuring the Power of Learning.™
44
• Are used to guide standard setting• Define the minimum performance required for
meeting a particular achievement-level expectation• Reflect the knowledge, skills, and processes that are
expected of studentsNote:• The knowledge, skills, and processes in ALDs are
cumulative. • The student who has achieved the threshold Level 3 is
assumed to have the knowledge, skills, and processes of the range Levels 1 and Level 2 ALDs.
Threshold ALDs
Measuring the Power of Learning.™
45
Threshold ALDs (Grade 7 ELA Claim 1 Example)
Level 1 Level 2 Level 3 Level 4
http://www.smarterbalanced.org/assessments/scores/
Measuring the Power of Learning.™
46
Threshold ALDs (Grade 6-8 Math Claim 2, 4 Example)
Level 1 Level 2 Level 3 Level 4
http://www.smarterbalanced.org/assessments/scores/
Measuring the Power of Learning.™
47
Start thinking about how you might weave these documents into planned professional learning, instructional planning, and local assessment development in the coming school year…
. . . about the documentswe’ve just reviewed.
Measuring the Power of Learning.™
Putting It All Together
Measuring the Power of Learning.™
49
Using Summative Assessment Results in the Instructional Cycle
Improving Teaching and
Learning
Data review and discoveryWhat is the current state of
performance?
Recalling program, practice, policies, etc.
What happened?
Possible connections between performance and
program, practice, policies, etc.
How or why did the performance occur?
Possible actions moving forwardWhat should/will
change?
Measuring the Power of Learning.™
50
Research, Recall, Reflect, Respond
Data review and
discovery
Program, practice, policies
Possible connections
Possible actions moving forward
Measuring the Power of Learning.™
51
Sources of Data: Identifying the Appropriate Scores and Reports
• Consider the reports covered in the a.m. session.
• Consider the “unit of analysis” for your data research.
DATASOURCE
Identify and use
appropriate data sources.
Measuring the Power of Learning.™
52
Research• Engage with the data
to find facts.• Describe what you
observe.• Look for data trends.• Focus on facts, not
conjecture.
RESEARCH
Using the data sources available,
report the facts. Look for trends or areas of
concern. Look for areas of success. This is an objective data
discovery.
Measuring the Power of Learning.™
53
Examples of Research Guiding Questions - Finding the Facts
1. What do you notice about the overall scores?2. What are the percentages of students in each of the
achievement levels?3. Where do the scale scores fall, i.e., high, medium,
low band?4. How do the grade level scores compare with the
Smarter Balanced threshold scores?5. Which scores look most noticeably different from the
population being studied?
Measuring the Power of Learning.™
54
Research (cont.) Example: Comparative Template Using
Threshold Scores and Claim Scores Spring 2015 Smarter Balanced
Scale Score
Achievement Level
Threshold ScoreOr Range
Below–Above Expected
Strongest Claim Area
Weakest Claim Area
Grade or Group Content Area
5 Mathematics 2494 2 (Medium) 2455(2455–2581)
8 Mathematics 2545 2 (Medium) 2504(2504–2585)
11 Mathematics 2589 2 (Medium) 2543(2543–2627)
Measuring the Power of Learning.™
55
Average Scale Score and Performance Level of Overall Score
Percentage of Students in Each
Claim Performance Category
% Below Standard
% At/Near Standard
% Above Standard
Grade Group
ELA/Literacy
Average Scale Score
Performance Level of Overall Score Claims
Percentage in Each Claim Performance Category
7 All
Reading
Writing
Listening
Research/Inquiry
7 EL
Reading
Writing
Listening
Research/Inquiry
7 Hispanic
Reading
Writing
Listening
Research/Inquiry
Research (cont.)Example: Comparative Template Using Claim Scores
Measuring the Power of Learning.™
56
Recall• Consider classroom
program, practice and policies.
• Think about last year:– What happened?– What curriculum
was in place?– What professional
learning occurred?
RECALLFocusing on the prior school year(s), discuss the realities of classroom, school, district program, practice, and policies. What happened? Facts only – no conjecture.
Measuring the Power of Learning.™
57
Examples of Recall Guiding Questions – What Happened?
1. What curriculum did we initiate/continue last year?
2. What professional development opportunities were available to teachers, administrators, and support staff?
3. What environmental factors were at play?4. What changes occurred in our population?5. What particular needs did our students exhibit?
Measuring the Power of Learning.™
58
RecallingExample Recall Template—Observations Only
SCORE: Scale Score in ELAGROUP: Grade 7 Subgroups DATA OBSERVATIONS CONDITIONS FROM LAST
YEAR
What do you notice about the overall scale score?
The overall scale score for Grade 7 ELA is 2525. • Environment?
• Curriculum?• Context?• Professional
Development?
How do these scale scores compare with the threshold scores?
The overall scale score falls into the medium band of Level 2, Standard Nearly Met.
What subgroups are struggling the most?
African American, English learners, SED, SWD
Measuring the Power of Learning.™
59
Resources• What should our
students know or be able to do?
• What are the expectations of our students, based on the range and threshold descriptors?
Evidence Required for Target 1The student will identify text evidence (explicit details and/or implicit information) to support a GIVEN inference or conclusion based on the text.
Recall: program,
practice, policies
Reflect: possible
connections
Measuring the Power of Learning.™
60
Reflect• Use the guiding
questions. • Determine possible
reasons for the performance.
• Keep the conversation honest.
• Explore many possibilities.
REFLECTConnect performance
with prior year practices, programs,
and policies. Consider possible
reasons. Be honest about what occurred
last year and how that may have impacted the
performance that you observe.
Measuring the Power of Learning.™
61
Examples of Reflecting Guiding Questions – Why and how may it
have happened?1. How do these results affirm areas where instruction
was provided?2. What would you consider is the single-most
important factor contributing to the apparent successes/needs as indicated by the scores?
3. Consider what might explain these gaps in achievement. Think about current:
Policies Programs Practices
Measuring the Power of Learning.™
62
Reflecting—Beyond the NumbersExample Reflection Template
Teaching and Learning Inventory: Content Area: ____________________
Component In Place
In Progress
Not Yet
How might these conditions have played a role in the observed
performance?Teachers use a curriculum aligned to the CA state standards and frameworks
Teachers are provided regular professional learning opportunities around the standards.
Teachers use ongoing formative assessment practices and tools
Teacher make use of Interim assessment(s), including SBAC IABs to measure student progress
Teachers have collaborative planning time to discuss effective and responsive teaching practices/strategies
Teachers have sufficient collaborative planning time to examine student work
Measuring the Power of Learning.™
63
Respond• Use guiding questions.• Think about what you
can control.• Ask yourself:
– What other data are available to you?
– How will you know progress is being made?
RESPONDWhat might be some possible ways to move
forward? These responses should tie directly to what was discovered in
the reflection process.
Measuring the Power of Learning.™
64
Examples of Responding Guiding Questions – What’s our path
forward?1. How might we need to alter our curriculum and/or
materials to address the areas of need for the coming year?
2. What evidence do we need during classroom instruction to know that our students are making progress toward meeting the targets for each claim?
3. How might we use the performance tasks to illustrate student performance and to guide the possible intervention?
Measuring the Power of Learning.™
65
Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
Measuring the Power of Learning.™
66
Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2. Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. Overall scale scores for Black or African American students in Grade 8 and Grade 11 fell into Level 2. District wide, 5% of the Black or African American students’ scale scores fell into the Standard Exceeded level, with 15% falling into the Standard Met level. In comparison, 15% of the districts’ All Students’ scores fall into the Standard Exceeded level and 27% fell into Standard Met.
Measuring the Power of Learning.™
67
Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60% of the Grade 3 through Grade 6 students’ claim scores were at the Below Standard level.
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Measuring the Power of Learning.™
68
Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in Key Details, Central Ideas, and Word Meaning.Target scores for the Spring 2015
CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Measuring the Power of Learning.™
69
Using the Resources to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials.
We had new principals at all but two of our schools.
We implemented a new student information system, district wide.
We implemented a new mathematics curriculum in Grades K through 6.
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials. All of our professional learning opportunities were focused on ELA.We implemented a new student information system, district wide.We implemented a new mathematics curriculum in Grades K through 6.
Measuring the Power of Learning.™
70
Using the Tools to Research, Recall, Reflect, and Respond
DATA SOURCE RESEARCH RECALL RESOURCES REFLECT RESPONDIdentify and use appropriate data sources.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of
success. This is an objective data discovery.
.
Using the data sources available, report the facts. Look for trends or areas of concern. Look for areas of success. This is an objective data discovery.
Use the information contained in the
supporting documents to bridge the conversation between reflection and
revealing.
Connect performance with practice and determine possible reasons. Be honest about what occurred last year and how that may have impacted the performance that you observe..
What might be some possible ways to move forward? These responses should tie directly to what was revealed in terms of cause.
Scale Scores
Achievement Levels
Claim Scores
Target Levels
Target scores for the Spring 2015 CAASPP ELA summative test results for the Grade 3 Black or African American students indicate relative weakness in …
Across all claims in ELA, fewer than 10% of all Black or African American students in Grades 3 through 7 had claim scores of Above Standard. Over 60%...
The Spring 2015 CAASPP ELA summative test results for our LEA show that overall scale scores for every grade level fell into Level 2, Standard Nearly Met. Grade 8 and Grade 11 scale scores fell into the high band of Level 2; all other grades’ scale scores fell into the medium band of Level 2.
Overall scale scores for Black or African American students in Grades 3 through 7 fell into Level 1, Standard Not Met. …
Over 30% of our elementary teachers retired at the end of the previous school year, and many of the new teachers were unfamiliar with the district’s materials.
We had new principals at all but two of our schools.
We implemented a new student information system, district wide.
We implemented a new mathematics curriculum in Grades K through 6.
At Grade 3:
Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity.
Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to-high complexity.
Students performing at Level 3 should be able to determine…
At Grade 3:Students performing at Level 3 should be able to use explicit details and information from the text to support answers or basic inferences in texts of moderate-to-high complexity. Students performing at Level 3 should be able to identify or summarize central ideas, key events, or the sequence of events presented in texts of moderate-to-high complexity. Students performing at Level 3 should be able to determine intended meanings of words, including words with multiple meanings, based on context, word relationships, word structure, or use of resources in texts of moderate-to-high complexity.
Measuring the Power of Learning.™
71
. . . using summative assessment results
Choose either Scenario #1 ELA, or Scenario #2 mathematics.
Treat the sample data as if it were your LEA’s data.
Use the Guiding Questions and work your way through the template, answering the questions as best you can.
Practice
Measuring the Power of Learning.™
Telling the Story with the
Data
Measuring the Power of Learning.™
73
Different Scores Tell Different Stories
• What is the message you want to send?• Who is your audience?• What do you want your audience to
remember?• Manage your message through
– Emphasizing– Spotlighting– Focusing
Context Display Level of detail
Measuring the Power of Learning.™
74
ELASpring 2015 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 11
Number of Students Tested 4588 4627 4754 4720 4802 4749 4315
Percent of Students Tested 99 99 99 99 99 98 98
Mean Scale Score 2419.2 2457.4 2494.5 2527.4 2541.7 2555.5 2607.1
Achievement Level Nearly Met
Nearly Met
Nearly Met
Nearly Met
Nearly Met
Nearly Met Met
ELA—Example LEAMean Scale Scores and Corresponding Achievement Levels by Grade
Measuring the Power of Learning.™
75
Comparing ELA Scale ScoresYear to Year: Grade
● 2419
Standard Exceeded
Standard Met
Standard Nearly Met
Standard Not Met
● 2457● 2494
● 2527 ● 2541
● 2607
Spring 2015●
● 2433● 2475
● 2505● 2535 ● 2558
● 2620
Spring 2016●
● 2555 ● 2570
2800
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 112300
Thre
shol
d S
cale
Sco
re R
ange
Measuring the Power of Learning.™
76
Comparing ELA Scale ScoresYear to Year: Cohort
X 2394
Standard Exceeded
Standard Met
Standard Nearly Met
Standard Not Met
X 2437X 2480 X 2481
X 2526O 2550
Spring 2015X
O 2465O 2490
O 2525O 2501
Spring 2016O
2800
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 112300
Thre
shol
d S
cale
Sco
re R
ange
Measuring the Power of Learning.™
77
Take Advantage of the Local Control and Accountability Plan (LCAP) Process
• Examine the scores in the context of your LEA’s strategic plan
• Examine the scores in the context of the purpose of the Local Control Funding Formula and the LCAP
– Eight state priorities– Focus on underperforming groups of students– Consider opportunity gap as well as
achievement gap
Measuring the Power of Learning.™
78
Eight Areas of State Priority Must Be Addressed in LEA LCAP
State Priorities
Other Pupil Outcomes (8)
Course Access (7)
School Climate (6)
Basic Services (1)
Pupil Engagement (5)
Implementation of CA Common Core State Standards
(2)
Parental Involvement (3)
Pupil Achievement
(4)
Measuring the Power of Learning.™
79
ELA/Spring 2015Levels of Performance/ Subgroup*
Black/ African
American Asian FilipinoHispanic/
Latino WhiteTwo or
More Races
Students with
Disabilities SED^English
LearnersAll
Students
Number of Students 4,481 7,474 2,745 8,243 7,029 1,843 3,172 18,618 4,392 32,555
% Exceeded/Met 31% 58% 65% 38% 61% 50% 10% 36% 10% 50%
% Exceeded 8% 25% 26% 11% 25% 18% 3% 10% 1% 19%
% Met 23% 33% 39% 27% 36% 32% 7% 26% 9% 31%
% Nearly 27% 23% 21% 28% 22% 25% 18% 28% 31% 25%
% Not Met 42% 19% 14% 34% 17% 25% 72% 36% 59% 26%
% Nearly/Not Met 69% 42% 35% 62% 39% 50% 90% 64% 90% 51%
*The subgroups American Indian/Alaska Native and Pacific Islander are not released due to insufficient numbers.
^ SED: Socio-Economically Disadvantaged
It’s not just about the whole:Example LEA Subgroups—ELA
Measuring the Power of Learning.™
80
California Achievement Disparities: ELA Subgroups
Measuring the Power of Learning.™
81
Comparable LEAs• Comparable LEAs/schools based on
demographic similarities─ Enrollment─ Poverty─ Ethnicity─ Programs (e.g., students with
disabilities)─ Language fluency
• Geographic proximity─ Within LEA─ Surrounding LEAs─ Both charters and non-charters
Measuring the Power of Learning.™
82
Example LEA Compared to Similar LEAs and State—ELA
ELA Spring 2015Levels of Performance/LEA Example LEA LEA 2 LEA 3 LEA 4 State
Number of Students Tested 32,555 25,327 22,609 28,634 3,154,463
% Exceeded/Met 50% 52% 42% 53% 44%
% Exceeded 19% 23% 15% 19% 16%
% Met 31% 29% 27% 34% 28%
% Nearly Met 25% 21% 25% 26% 25%
% Not Met 26% 26% 33% 21% 31%
% Nearly/Not Met 51% 47% 58% 47% 56%
Measuring the Power of Learning.™
83
Similar LEA Subgroup Comparisons:Students Exceeding Standard
Measuring the Power of Learning.™
84
Similar LEA Subgroup Comparisons: Students Not Meeting Standard
Measuring the Power of Learning.™
85
Which data displays would be useful for your:• Board• Community• Parents• Administrators• Teachers
. . . about the data displayswe’ve just reviewed.
Measuring the Power of Learning.™
Applying the Learning
Measuring the Power of Learning.™
87
Call to Action – Pay Attention but Move Beyond the Scores
• Comprehensive, complex, and difficult process– Not a checklist– Not meant to be completed at a single staff
meeting or a single professional development event
• Focus on improving learning– Not solely about increasing scores– Honest conversations about what might
need to change
Measuring the Power of Learning.™
88
Call to Action – Pay Attention but Move Beyond the Scores (cont.)
• Use the data to communicate—as local policy and practice levers
• Reflect on what you can control to move beyond the scores:
─ Practices─ Policies and Procedures─ Programs
Measuring the Power of Learning.™
89
Summative Assessments Have a Part to Play
• How do you promote the effective use of summative assessment data in your LEA?
• Understand the distinction between– having educators make meaning of the
scores to inform instruction and practice,
– and communicating a message to a particular audience.
Measuring the Power of Learning.™
90
Process Document
Measuring the Power of Learning.™
91
Resources• Handout: Overview of Major Assessment Types
http://www.csai-online.org/sites/default/files/resources/4666/CSAI_AssessmentTypes.pdf
• English Language Arts/English Language Development Frameworkshttp://www.cde.ca.gov/ci/rl/cf/
• Content Specifications, Item Specifications and Blueprintshttp://www.smarterbalanced.org/assessments/development/
• Smarter Balanced Scale Score Ranges• http://www.cde.ca.gov/ta/tg/ca/sbscalerange.asp
• Sample 2015-16 Student Score Reporthttp://www.cde.ca.gov/ta/tg/ca/caasppssrinfo.asp
• Claim Descriptions for ELA and Mathematicshttp://www.cde.ca.gov/ta/tg/ca/elaclaimdescript.asp
• Target Score Report FAQshttp://www.caaspp.org/rsc/pdfs/CAASPP.target-report-FAQs.2016.pdf
Measuring the Power of Learning.™
92
Resources (cont.)• Achievement Level Descriptors
http://www.smarterbalanced.org/assessments/scores/
• California Department of EducationCalifornia Assessment of Student Performance and Progress Officehttp://www.cde.ca.gov/ta/tg/ca/[email protected]
• ETS CAASPP Portalhttp://www.caaspp.org/Help for LEA CAASPP Coordinators:
–[email protected]–800-955-2954