clarifying learning goals & standards: rubrics & exemplars
Post on 14-Apr-2017
285 Views
Preview:
TRANSCRIPT
Clarifying learning goals & standards: rubrics & exemplars
David Carless University of Hong Kong
At EdUHK, January, 20, 2017
The University of Hong Kong
Overview
1. Key concepts2. Students’ views on rubrics & exemplars3. Using rubrics for learning 4. Analyzing exemplars 5. Summary of key implications
The University of Hong Kong
KEY CONCEPTS
The University of Hong Kong
Competing assessment functions The University of Hong Kong
Stimulating productive student learning
Satisfying accountability needs
Judging student achievement
Learning-oriented assessment
Prioritizing student learning as a key aim of all assessment
The University of Hong Kong
The University of Hong Kong
Learning-oriented assessment framework (Carless, 2015a)
Productive assessment task
design
Student engagement with
feedback
Understanding quality academic
work
Importance of learning goals
Where am I going?
How to get there?
The University of Hong Kong
What are rubrics?
• A.k.a. ‘assessment criteria’, ‘grade descriptors,’ ‘marking schemes’
• Rubrics combine criteria and standards in the form of a grid to provide statements of the performance needed to achieve different standards (Carless, 2015, p. 132)
The University of Hong Kong
What are exemplars?
Exemplars are samples of student work used to illustrate dimensions of quality
Usually assignments from a previous cohort
The University of Hong Kong
The University of Hong Kong
Two relevant chapters
Carless, D. (2017). Students’ Experiences of Assessment for Learning. In D. Carless et al., (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer. Jönsson, A. & Panadero, E. (2017). The Use and Design of Rubrics to Support Assessment for Learning. In D. Carless et al., (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.
The University of Hong Kong
EVIDENCE FROM INTERVIEWS WITH STUDENTS
The University of Hong Kong
Research process The University of Hong Kong
Classroom observations
Interviews with teachers & students HistoryLaw
Business
Rubrics data (1)
“Criteria are rather vague to me …”
“I read the criteria descriptions some time ago and I’ve already forgotten them”
“The criteria were similar to previous ones so I didn’t study them carefully or consult them when preparing the assignment”.
The University of Hong Kong
Rubrics data (2)
“The descriptions of assessment criteria are almost the same for every subject with the same key words like critical & analytic”
“The assessment criteria are a bit lifeless. I wanted to know the teacher’s interpretation of the criteria and his personal preference”.
The University of Hong Kong
Rubrics data (3)
“Lists of criteria are all the same, it’s all very vague. Like for an ‘A’, it is excellent mastery and correct application and then ‘B+’ would be above average mastery and good application. You don’t know what it means so you just do what you can. Once you know how to get good academic results, you don’t need to rely on criteria any more because the criteria are always the same.”
The University of Hong Kong
Exemplars data (1)
“Samples are more concrete”
“It is a reference … you can imitate its presentation style”
The University of Hong Kong
Exemplars data (2)
“The exemplar served as a useful example of what a good report should be”
“There is a risk of copying but many of us try to add something more or use our own approach”
The University of Hong Kong
Inferences
Rubrics are only useful for learning if students engage with them
Exemplars are attractive to students but teacher implementation is complex
The University of Hong Kong
RUBRICS & STUDENT ENGAGEMENT
The University of Hong Kong
Dual aspects of rubrics
1. A scoring tool for teachers
2. A potential learning tool for students as aid to self-evaluation
The University of Hong Kong
Rubric options
• Analytic or holistic
• Generic or Task-specific
(see handout)
The University of Hong Kong
Writing rubrics
Standards should be represented with respect to features of work, not impenetrable terms, such as ‘good’ or ‘excellent’ (Ashwin, Boud et al., 2015, p. 259)
The University of Hong Kong
Rubrics activity 1
Students in groups brainstorm their own criteria for an assessment task
The University of Hong Kong
Rubrics activity 2
Students read the criteria for (say) a grade B and then re-write them in their own words
The University of Hong Kong
Rubrics activity 3
Students use rubric to evaluate work of peer (peer evaluation)
AND/OR
Students use rubric to evaluate exemplars
The University of Hong Kong
DISCUSSION 1: RUBRICS
• Share some of your thinking on using rubrics to enhance student learning
• What unanswered issues might you raise in relation to rubrics?
The University of Hong Kong
RATIONALE & USE OF EXEMPLARS
The University of Hong Kong
Why used?
Exemplars convey messages that nothing else can (Sadler, 2002)
The University of Hong Kong
Promote self-evaluation
Illustrate what good work looks like
Benchmark for student comparison with own performance
The University of Hong Kong
Analysis
Identify strengths, weaknesses & suggest how samples could be improved
The University of Hong Kong
When used?
Useful for students at transition stages
When assignments are complex, innovative or hard to explain
The University of Hong Kong
How used?
• Posted on LMS• Optional workshop (Rust et al., 2003)• Part of regular class• Displayed or made available• In part or as a whole
The University of Hong Kong
Benefits for students Gain experience in making judgments
Apply insights to own work
Notice discrepancies between present level & target level
The University of Hong Kong
The University of Hong Kong
Copying model problem
May reduce student creativity
May encourage unproductive copying
The University of Hong Kong
Time issues
• Time in collecting them
• Time in seeking consent for their use
• Time taken away from instruction
The University of Hong Kong
It’s challenging!
Difficulties for some students in evaluating exemplars accurately
The University of Hong Kong
Synthesis on exemplars
• Students like exemplars • They aid learning But • How to implement effectively• How to avoid ‘copying model answer
syndrome’
The University of Hong Kong
Key inference
The quality of the dialogue about the exemplars is a crucial factor determining student learning
The University of Hong Kong
DISCUSSION 2: EXEMPLARS
• Why don’t teachers make more & better use of exemplars?
• What unanswered issues, might you raise in relation to exemplars?
The University of Hong Kong
RECENT EXEMPLARS RESEARCH
The University of Hong Kong
TDG Project
Enhancing dialogic use of exemplars amongst 10 teachers in a Faculty of Education
The University of Hong Kong
The University of Hong Kong
Research question
What are the main features of a specific exemplars dialogue and how is it orchestrated?
The University of Hong Kong
Context of the case study
Participants: • Trainee Science teachers • BEd/BSc double degree • Year 3
Teacher-researcher: Dr Kennedy Chan
The University of Hong Kong
Data collection
• Classroom observation • Open-ended student survey• 2 focus group interviews• Interview with teacher-researcher• Teacher journal• Student artefacts: ‘exit slips’, assignments etc.
The University of Hong Kong
Exemplars provided2 high quality exemplars
– Same format (i.e., reflective essays) but different content focus
– Design aimed at reducing ‘copying’
The University of Hong Kong
Management of dialogue
1. Prior to class students analyzed exemplars;2. Students discussed exemplars in pairs; 3. Students elicited views from peers and made
mini-presentations; 4. Teacher-orchestrated dialogue; 5. Students submitted exit slip of reflections.
The University of Hong Kong
Dilemmas
Students constructing
notion of quality VS
Telling students about quality
Time for students to talk with
peers VS
Time for developing shared understandings
Exemplars as
guide VS
Exemplars as model
Using students’ voices VS
Making teacher’s voice
explicit
The University of Hong Kong
Good Exemplars dialogue
• Airs multiple & divergent viewpoints • Shows linkages between peer talk & whole-
class discussion • Evidences development of student views • Makes explicit some key qualities of exemplars(Carless & Chan, 2016)
The University of Hong Kong
CONCLUSION
The University of Hong Kong
Using exemplars & criteriaAssessment task
Students devise or engage with rubric & quality
Two samples read before class
Peer
discussion Teacher-led dialogue
Student ownership of
insights
The University of Hong Kong
Engaging with quality
Student need to engage with what quality looks like and develop capacities in making judgments
The University of Hong Kong
THANK YOU
The University of Hong Kong
References Ashwin, P., Boud, D., et al., (2015). Reflective teaching in higher education. London: Bloomsbury. Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.Carless, D. (2015b). Excellence in University Assessment: learning from award-winning teachers. London: Routledge. Carless, D. (2017). Students’ experiences of Assessment for Learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.Carless, D., Bridges, S., Chan, C.K.W. & R. Glofcheski, R. (Eds.), (2017). Scaling up Assessment for learning in Higher Education. Dordrecht: Springer.Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing standards‐based assessment effectively: Incorporating discussion of exemplars into classroom teaching. Assessment and Evaluation in Higher Education, 37(2), 149-161.
The University of Hong Kong
References (cont.) Jonsson, A. & Panadero, E. (2017). The use and design of rubrics to support assessment for learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education, 28(2), 147-164.Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.Smith, C., Worsfold, K., Davies, L., Fisher, R. & McPhail, R. (2013). Assessment literacy and student learning: The case for explicitly developing students ‘assessment literacy’. Assessment and Evaluation in Higher Education, 38(1), 44-60.
The University of Hong Kong
top related