emily jolley let’s measure together project – constructed response
DESCRIPTION
Emily Jolley Let’s Measure Together Project – Constructed Response Mentor: Dr. Tammy Howard, DPI August 6, 2013 kenanfellows.org dpi.state.nc.us /accountability. Kenan Fellows Externship. 5 week externship this summer at the Testing and Accountability office of DPI - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/1.jpg)
Emily JolleyLet’s Measure Together Project –
Constructed Response
Mentor: Dr. Tammy Howard, DPI August 6, 2013
kenanfellows.org dpi.state.nc.us/accountability
![Page 2: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/2.jpg)
Kenan Fellows Externship
• 5 week externship this summer at the Testing and Accountability office of DPI
• Mentor is Dr. Tammy Howard – director of the testing and accountability division for DPI
• Attended department meetings, met with test development staff, item writers, content specialists, psychometricians, and just about everyone else involved in testing and accountability.
![Page 3: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/3.jpg)
NC DPI Accountability Services Division
• The mission of the Accountability Services Division is to promote the academic achievement of all North Carolina public school students and to assist stakeholders in understanding and gauging this achievement against state and national standards.
![Page 4: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/4.jpg)
NC DPI Accountability Services Division
The major thrust of this mission is three-fold: 1. the design and development of reliable
and valid assessment instruments2. the uniform implementation of and access
to suitable assessment instruments for all students
3. the provision of accurate and statistically appropriate reports.
![Page 5: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/5.jpg)
EOCs and EOGs
• Fulfill requirements of Federal NCLB Act of 2001.• Developed by NCSU Technical Outreach for Public
Schools (TOPS) Program as directed by DPI• Questions are written by North Carolina teachers
and then edited by TOPS and DPI.• Standardized administration and grading statewide.• Rigorous development process that takes years to
complete.• Used to assess student learning and growth.
![Page 6: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/6.jpg)
Measures of Student Learning/Common Exams
• Required by Race to the Top Grant.• Developed by NCSU Technical Outreach for Public
Schools (TOPS) Program as directed by NC D.P.I.• Questions are written by North Carolina teachers.• Not standardized, considered a local assessment
administered and graded by individual Local Education Authorities (LEAs)• Used to assess teachers for Standard 6
![Page 7: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/7.jpg)
Item Writing Trainings
• Attended testing item writing training for teachers by TOPS.• Focused on breaking down the standards into
Revised Bloom’s Taxonomy verbs, standard objects, and supporting items.• After learning how to break down the
standards, basic instructions were given about writing items.
![Page 8: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/8.jpg)
Item Writing Trainings
Revised Bloom’s Knowledge Dimensions:•Factual – basic elements•Conceptual – interrelationships•Procedural – how to do something•Metacognition – awareness of what you know
Revised Bloom’s Cognitive Processes:•Remember, Understand, Apply•Analyze, Evaluate, Create
![Page 9: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/9.jpg)
Item Writing TrainingsFor example – breaking down a Civics & Economics standard: CE.C&G.2.7: Analyze contemporary issues and governmental responses at the local, state, and national levels in terms of how they promote the public interest and/or general welfare (eg: taxes, immigration, naturalization, civil rights, economic development, annexation, redistricting, zoning, national security, health care, etc)
Unifying Concept (found in unpacking doc)
ES/CO RBT Verb Object In Terms Of: Examples:
Government Systems, Structure, Function
CE.C&G.2.7
Analyze contemporary issues and governmental responses at the local, state, and national levels
in terms of how they promote the public interest and/or general welfare
taxes, immigration, naturalization, civil rights, economic development, annexation, redistricting, zoning, national security, health care, etc
![Page 10: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/10.jpg)
Most essential standards for high school social studies fall under the understand or analyze range of conceptual knowledge. A few fall under the understand or evaluate range, and a couple standards are procedural knowledge (especially in World History & American History I & II).
![Page 11: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/11.jpg)
Why Constructed Response?
• Constructed Response questions were included in MSLs for a variety of reasons:•Almost none of the new standards are based on
factual knowledge only. Because the new essential standards are more conceptual, having only multiple choice questions is not the best way to assess these standards•Many teachers have been asking for assessments
that more closely reflect students’ learning, and constructed response questions help facilitate that request
![Page 12: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/12.jpg)
Why Constructed Response?
• As teachers, we need to ensure that the written, taught, and tested curriculum are aligned. •We can of course go beyond what the
standards require, but we need to make sure we’re teaching and assessing the students at least at the level the standards are written.
![Page 13: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/13.jpg)
What about the Common Core State Standards?
• We also have literacy standards for history/social studies. These are not technically tested by MSLs, but we should still be teaching them with fidelity to help our students become better readers and writers. A sample reading standard:• CCSS.ELA-Literacy.RH.9-10.8: Assess the extent to which the
reasoning and evidence in a text support the author’s claims.
Unifying ConceptES/CO RBT Verb Object In Terms Of:
Reading in History/Social Studies
RH.9-10.8 Assess the extent to which reasoning and evidence in a text supports the author’s claims
n/a
![Page 14: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/14.jpg)
Almost all of the Common Core State Standards for literacy in high school social studies fall under the apply, analyze, or evaluate ranges of procedural knowledge, with a few falling under the create range of procedural knowledge (especially the writing strand).
![Page 15: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/15.jpg)
Essential Standards Breakdown
• Now, let’s practice breaking down our own standards into this basic format. Use the template from my site to start breaking down your standards:
• The unpacking documents can also be found on my site. It may be more useful (along with the examples in the standards) when figuring out what content to teach.
Unifying Concept (found in unpacking doc)
ES/CO RBT Verb Object In Terms Of: Examples:
![Page 16: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/16.jpg)
Quick Assessment
•When you’ve broken down some of your standards, take the quick survey on my site to assess your comfort with this process.• The survey has some standards from each course. It
also has a few of the released constructed response questions from last fall. You should categorize each by RBT cognitive process and knowledge dimension.• I highly recommend breaking down all the standards
from your course this way (and looking through the unpacking document!)
![Page 17: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/17.jpg)
Constructed Response Criteria
• Now we’re ready to write!• You need a few things to begin writing a
constructed response question. Basic format:• Stimulus or Context Statement (optional): chart,
map, picture, graph, text, quote, intro, or scenario•Prompt or Task Statement: directive to students•Requirements Statement: response specifics• Scoring Criteria: rubric, range 0-4 points
![Page 18: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/18.jpg)
C&E Fall 2012 CR Released Example:
• Stimulus/Context Statement: Throughout its history, American democracy has upheld and strengthened citizens’ individual rights. • Prompt or Task Statement: Take a position that argues for or
against the above statement. • Requirements Statement: Provide at least two specific historic
examples (e.g., legislation, Supreme Court cases, executive orders, or other actions by the federal government) that support your position.
• Scoring Criteria: • Score 0 No response or the response does not address the prompt. • Score 1 Fulfills only 1 of 3 requirements of a level 3 performance • Score 2 Fulfills 2 of 3 requirements of a level 3 performance • Score 3 Takes a position that argues for or against the statement; provides one
historic example that supports the position; provides a second historic example that supports the position
![Page 19: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/19.jpg)
C&E Fall 2012 CR Released Example:
• Throughout its history, American democracy has upheld and strengthened citizens’ individual rights. • Take a position that argues for or against the above
statement. Provide at least two specific historic examples (e.g., legislation, Supreme Court cases, executive orders, or other actions by the federal government) that support your position. • Standard Alignment: • CE.C&G.2.6 Evaluate the authority federal, state, and local
governments have over individuals’ rights and privileges (e.g., Bill of Rights, Delegated Powers, Reserved Powers, Concurrent Powers, Pardons, Writ of habeas corpus, Judicial Process, states’ rights, Patriot Act, etc.).• CE.C&G.3.8 Evaluate the rights of individuals in terms of how well
those rights have been upheld by democratic government in the United States.
![Page 20: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/20.jpg)
Constructed Response Tips
• Start with the objective(s) you want to measure. Most constructed response questions will satisfy more than one objective. • You must use the same RBT level that the standard requires.
(You also must use the same content… obviously!)• You should adhere to the same basic scoring criteria
whether you are creating a 2, 3, or 4 point question. Remember, students don’t see this criteria on the MSL, so you should teach them how to pick out points (there’s a PowerPoint about this on my site). • Review the released questions and use the RBT question
stems if you need help getting started!
![Page 21: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/21.jpg)
Constructed Response
• Choose at least 2 objectives from your course and write your own constructed response using the template provided.
• After writing your constructed response, please trade it with a partner. You should break down each other’s CR and determine if it fits the criteria and matches the content and RBT levels of the standard.
![Page 22: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/22.jpg)
Public Google Spreadsheet
• Please load the constructed response question you created today into the Google spreadsheet found on my site: https://docs.google.com/spreadsheet/ccc?key=0AtoXZ1jIf_QadDJ5N3RDTC0zZlhsSnVpOHpJSEE4Wmc&usp=sharing • This is a public spreadsheet that anyone can
access. I will update it throughout the year with constructed response questions I write (mostly for C&E), and I hope you will do the same.• Please share with everyone!
![Page 23: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/23.jpg)
Other Resources
• TOPS suggested these resources for writing constructed response questions and understanding concept-based instruction:• Lynn Erickson - Concept-Based Curriculum and
Instruction • Lorin Anderson & David Krathwohl - A Taxonomy
of Learning, Teaching, and Assessing•Robert Mager - Measuring Instructional Results •RBT Resources (Section 5):
http://tpri.wikispaces.com/space/content
![Page 24: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/24.jpg)
Contact Info
• Please contact me if you have any questions:
Emily JolleyMillbrook High School
![Page 25: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/25.jpg)
Preparing Students for the Real World
Takes Teachers Who Learn in the Real World
BECOME A KENAN FELLOW
•See first-hand how researchers and industry experts use STEM skills in real-world settings
•Apply that knowledge to better prepare students for the jobs of tomorrow
•Enrich your career through professional development opportunities
•Serve as an ambassador for educational excellence in your school, community and across the state
Find out more: kenanfellows.org
![Page 26: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/26.jpg)
Kenan Fellows Program Timeline
![Page 27: Emily Jolley Let’s Measure Together Project – Constructed Response](https://reader035.vdocuments.net/reader035/viewer/2022062310/568166a2550346895dda8b89/html5/thumbnails/27.jpg)
Kenan Fellows Program Stay Connected
facebook.com/KenanFellows
@KenanFellows
Kenan Fellows Program Group
youtube.com/user/KenanFellows
pinterest.com/kenanfellows