rubrics and online assessment
DESCRIPTION
Slides from webinar on 'Rubrics and online assessmentt' given by Eloise Tan in the 2012 Online Assessment and Feedback Module at Dublin City University.TRANSCRIPT
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
/
http://softchalkconnect.com/lesson/files/QARNsW98cnVGeh/Rubrics_Lesson_print.html
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
LI502: Assessment and Feedback in the Online Environment
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Rubric Design and Deep Learning
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
OverviewRubrics: definition, components (based on
(Meeusen, 2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh
Rubrics in the online environmentRubrics for online assessments: online
discussions, wikis, blogs Class admin
What is a rubric?
Do you use them for your assessments?
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Definitions of rubrics from (Meeusen, 2011) http://softchalkconnect.com/lesson/QARNsW98cnVGeh
Criterion –referenced tools
“used to evaluate student performance against a standard of mastery, not just to compare students”
“a scoring guide for evaluating student work”
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
What rubrics can do (Palloff & Pratt, p.33)
• Way to provide feedback• Define characteristics of a high quality
assignment• Help student understand expectations• Establish range of performance
categories• Provide a concrete way of evaluating
their own performance (self-assessment & reflection)
• Takes guesswork out of grading for lecturers
• Aims to achieve reliability, validity, and transparency within marking process
• Helps students to manage time• Helps lecturers clarify what is
important to them in the assessment: “What do I want students to learn from this assessment?”
• Opens up discussion on marking criteria, student learning
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Objectives of rubrics
Validity(Assesses
what it says it will assess)
Transparency
(Makes clear to students
the criteria)
Reliability(Assesses
consistently across
submissions)
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
What could be some challenges or unintended consequences of using rubrics? Enter into chat or raise hand to use Talk
button
Chat room comments: Lead to mechanical approach to
assessmentsOpens up room for conflict relating to
assessmentsCould constrain student creativityMarking is subjective, hard to pin down
criteria Takes a lot of time to design
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Responses to comments Lead to mechanical approach to assessments
We know students already are strategic about their assessments, rubrics help focus their time/effort.
Opens up room for conflict relating to assessmentsRubrics provide a co-constructed tool that can be used by students
to dispute their grades. This could be a positive because they are relating their performance to pre-determined criteria and not making subjective arguments about a grade.
Could constrain student creativityYes, and rubrics could also be seen as a framework through which
students can strategically approach their assessment, instead of hoping that their strategy works.
Marking is subjective, hard to pin down criteria Yes, that is the point of rubrics. To make marking valid, reliable, and
transparent. Takes a lot of time to design
Definitely! Many tools to help though as we will see…
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Holistic v. Analytic RubricsHolistic rubrics
provide a single score based on an overall impression of a student’s performance on a task.
Analytic rubrics provide specific feedback along several dimensions
http://www.schreyerinstitute.psu.edu/pdf/rubricbasics.pdf
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Example of Holistic v. Analytic Webtourhttp://softchalkconnect.com/lesson/files/QA
RNsW98cnVGeh/Rubrics_Lesson_print.html
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Remember
What you do not include in a rubric is just as important as what you put in.
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Parts of a rubric (Meeusen, 2011) Scale: the scale of points to be assigned in
scoring a piece of work on a continuum of quality. Higher points are typically assigned for the best work.
Descriptors: the descriptions for each level of performance that contain criteria and standards by which the performance will be judged
Criteria: Criteria describe the conditions that any performance must meet to be successful. Criteria should describe both strengths and errors.
Standards: standards specify how well criteria must be met.
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Rubrics and online assessment Tools to design rubrics: Rubistar http://rubistar.4teachers.org (free) Rubrics in Moodle 2.2 http://
www.youtube.com/watch?v=KXavtUhDINAPage compiling online rubric tools:
http://www.teach-nology.com/web_tools/rubrics/
Open Educational Resources: Foundations for rubrics can be found online. Example : http://school.discoveryeducation.com/schrockguide/assess.html
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Self, peer assessment and rubricsCould I involve students in setting the
rubrics? Could students use the rubric to self assess
/ peer assess? (Having rubrics readily available promotes self-assessment and reflection)
http://tellio.blogspot.com/
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Designing rubrics for online assessmentsAm I clear on what I am assessing in the
activity? What is my learning objective for the assessment?
Will technical skills / challenges affect students’ ability to carry out the assessment or meet successful criteria?
Careful of: Tendency to quantifyLack of time / preparation/ scaffolding of
technical skills and/or new ways of writing for students
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Evaluating your rubric – some tips1. Does it assess what you want it to?
For an online discussion forum – are you assessing for critical thought displayed in the post? Or are you assessing for number of contributions?
2. Do students have the skills necessary / or the opportunity to gain the skills necessary to achieve success through your rubric? Have you provided tutorials for how to post, use a wiki, etc?
3. Have you paid careful attention to what goes into and what stays out of your rubric?
Is it important to allocate marks for style, presentation, referencing for the assessment? Might be different for different levels.
4. Do the marks allocated for criteria reasonably correlate to the amount of time students should spend on that criterion? 5. Have you shared your rubric with students in advance? Or better
yet involved them in the design? 6. Have you included room in your rubric for feedback
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Sites to help you evaluate your rubricUseful set of questions from the Office of
Provost at Rensselaer Polytechnic Institutehttp://provost.rpi.edu/node/32
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Webquest and DiscussionSearch for rubrics online for 1) online
discussions, 2) wikis , 3) blogs, or 4) any online assessment you would like to discuss
Come back and share/discuss/evaluate rubrics
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Online discussion forum rubricsRubric that combines frequency with
qualityhttp://
www.ankeqiang.org/IEP/Online-Eval.pdf
Rubric that details the different types of posts for forums (responses, initial post, content contribution)
http://www.udel.edu/janet/MARC2006/rubric.html
Also available in our core texts: Crisp, Palloff and Pratt
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Wikis (can be used for group work)Rubric for self / peer assessment of team
members in a grouphttps://
wiki.umn.edu/pub/TeachingWithWikis/AssessingWikis/team_rubric.pdf
Rubric for the collaboration processhttps://wiki.umn.edu/pub/TeachingWithWiki
s/AssessingWikis/wiki_rubric.pdf
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Related to last webinar: Rubrics as self-
assessment and as plagiarism prevention
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Current Featured Resource on Moodle
http://www.docstoc.com/docs/3647772/Tips-For-Effective-Rubric-Design-How-to-design-a-rubric
@t_eloise Dr. Eloise Tan, Learning Innovation Unit, Dublin City University
Questions?