WORKSHOP ON TEACHING AND LEARNING

ASSESSMENT & RUBRICS

MONT FLEUR

18 - 20 MARCH 2012

OutcomesOutcomes

Participants should be able to:

1. align assessment with outcomes

2. select a range of assessment forms available (and approved by UWC)

3. justify the choice of assessment forms

4. explain and defend marks and weightings

5. meet the criteria for reliability and validity

6. create appropriate rubrics

Assessment Assessment

Assessment must be aligned with learning outcomes and modular content.

(Constructive alignment – Biggs)

Assessment provides students with opportunities to show that they can do what is set out in the learning outcomes.

The Purpose of AssessmentThe Purpose of Assessment

‘Assessment defines what students regard as important, how they spend their time and how they come to see themselves as individuals’ (Brown, 2001 in Irons 2008:11)

‘assessment is seen to exert a profound influence on student learning: on what students focus their attention on, on how much they study, on their quality of engagement with learning tasks, and through feedback, on their understanding and future learning’. (Gibbs & Simpson, 2004 in Irons, 2008:11)

Biggs: Assessment Tasks (ATs)Biggs: Assessment Tasks (ATs) provide students the opportunity to demonstrateprovide students the opportunity to demonstrate

whether or not they have achieved the ILOs and what whether or not they have achieved the ILOs and what level their performance is in those ILOs level their performance is in those ILOs

should be appropriately designed or selected to should be appropriately designed or selected to address all the ILOs that we want to assess address all the ILOs that we want to assess

use different assessment methods (tasks) address use different assessment methods (tasks) address different ILOs. There should, therefore, be several kinds different ILOs. There should, therefore, be several kinds of of task.task.

provide the evidence allowing teachers to make a provide the evidence allowing teachers to make a judgment about the level of a student’s performance judgment about the level of a student’s performance against the ILOs and to award a final grade. against the ILOs and to award a final grade.

Common ILOs Common ILOs Possible Assessment Tasks Possible Assessment Tasks (Bi(Bi

DescribeDescribe essay question, exam, oral essay question, exam, oral presentation (peer assessment)presentation (peer assessment)

ExplainExplain assignment, essay question assignment, essay question exam, oral, letter-to-a-friendexam, oral, letter-to-a-friend

IntegrateIntegrate project, assignmentproject, assignmentAnalyseAnalyse case study, assignmentcase study, assignmentApplyApply project, case study, experimentproject, case study, experimentSolve problemSolve problem case study, project, experimentcase study, project, experimentDesign, createDesign, create project, experimentproject, experimentReflectReflect reflective diary, portfolio,reflective diary, portfolio,

self-assessmentself-assessmentCommunicateCommunicate a range of oral, writing or a range of oral, writing or

listening tasks, e.g. presentation, listening tasks, e.g. presentation, debate, role play, reporting, debate, role play, reporting,

assignment,assignment, précis, paraphrasing,précis, paraphrasing, answering questions etc.answering questions etc.

AlignmentAlignmentGraduate attributes

(What are the overarching attitudes, skills and dispositions for UWC students?)

↕Learning Outcomes

(What do I want my students to be able to do?)

↕Assessment Criteria

(What do I need to see to know they can do it?)↕

Teaching and Learning Activities (What will they be able to do, to know, how will their thinking and behaviour change as a result

of the teaching/learning experience?)↕

Assessment Tasks (How can I get them to show me those things?)

↕

Assessment Strategy(How can all my tasks be ‘combined’ to fit the time and cover all outcomes?)

11/04/23

Start with the outcomes we intend students to learn (ILO), and align teaching and assessment to those outcomes.

Outcome statements contain a learning activity (a verb) that students must perform to best achieve the outcome.

Learning is constructed by what students do, not what we teachers do.

Assessment concerns how well they achieve the intended outcomes, not how well they report back to us what we have told them.

(Source: Knight, 2001 www.heacademy.ac.uk)

Summative assessmentSummative assessment

This usually happens at end – making a final judgement about learning in relation to outcome

It usually has a mark associated.

Assessment of learning.

Formative AssessmentFormative Assessment

Assessment for learningused for diagnosis, growth and improvement in student learning takes place during learning and can be ongoing

feedback is important – time consuming so should promote student responsiveness. Electronic feedback sometimes more effective as it cannot be ignored (Winberg, 2008)

Think about feedback in assessmentThink about feedback in assessment

As a teacher: What type of feedback do you give? What happens with it? What would you like to see happen with it?

Cast your mind back to when you were a student … What types of feedback did you get as a student? What did you find helpful? What did you do with it?

Vardi, I (2009) Improving Student Writing with Effective Feedback 2nd Annual SoTL Commons Conferenceconference March 2009

Getting feedback… (Winberg, 2008)Getting feedback… (Winberg, 2008)

Good feedback Good feedback (Winberg, 2008)(Winberg, 2008)

Facilitates the development of self-assessment in learning (reflection)

Encourages teacher-peer dialogue around learning Helps clarify what good performance is (goals, criteria,

expected standards) Provides opportunities to close the gap between current

and desired performance Delivers high quality information to students about their

learning (individual or general) Encourages positive motivational beliefs and self-esteem Provides information to teachers that can be used to

shape the teaching (from HEA guide Enhancing student learning through effective formative feedback, p. 2)

Formal and informal assessmentFormal and informal assessment

Formal - structure and awareness of both lecturer and student that assessment is occurring and how results are documented.

Usually more structured and planned.

Informal - spontaneous and not necessarily documented e.g. suggestions for improvement or revision, further drafts, learning activities to elucidate if students grasp principles.

Reliability and validityReliability and validity

Inter-rater reliability and intra-rater reliability

Content validity – how closely do questions or assessment tasks relate to content/concepts and learning outcomes being assessed?

Traditional and alternative methodsTraditional and alternative methods

Traditional methods focus on the products of learning rather than the process and is high stakes; cumulative and summative; lecturer is assessor

Alternative methods use authentic situations; focus on processes; use artifacts as evidence of student thinking (e.g. portfolios, blogs, podcasts, journals); are metacognitive; learning progress and growth over extended period; use self and peer assessment and feedback (critical friends).

Norm- and Criterion-Referenced Norm- and Criterion-Referenced Approaches to AssessmentApproaches to Assessment

Norm – comparing students to others in a group – bell curve

The UWC Assessment PolicyThe UWC Assessment Policy

Supporting student-centred learning –encouragement of student responsibility for own learning, within reason (e.g. workload)

Assessment committees in each faculty Collegial engagement on assessment tasks Follow principles of good assessment

practices: transparency, clear ILO’s, etc. Use of formative and summative assessment Integrated assessment – theory and practice

Assessing by grading with rubricsAssessing by grading with rubrics

For:For: Student’s performance is appropriately assessed Student’s performance is appropriately assessed

against what they are intended to learn( - against what they are intended to learn( - criterion-referenced.criterion-referenced.

The final grade tells students what they have The final grade tells students what they have achieved and what they need for a better grade.achieved and what they need for a better grade.

Against:Against:

Requires a different mind set for some teachers.Requires a different mind set for some teachers. Initially more work in designing ILOs, suitable Initially more work in designing ILOs, suitable

assessment tasks and rubrics, but once established assessment tasks and rubrics, but once established is no more extra work than marking.is no more extra work than marking.

What is a rubric?What is a rubric? It is a scoring guide or a set of expectations used to

judge student performance. It shows students how well they have performed on an

assignment. It breaks the assignment into parts, using criteria and

levels of performance required for the assignment. It can be used for a wide range of tasks (essays,

research projects, oral presentations, portfolios, etc.) It is especially useful for assessing complex and

subjective subjects.

21

A rubric is a scoring guide or a set of expectations used to judge student performance.

Characteristics are arranged in levels, indicating the degree to which a standard has been met.

Rubrics are especially useful for assessing complex and subjective subjects.

They provide working guides for both teachers and students and are usually handed out before the assignment so that students know what is expected.

Why use rubrics?Why use rubrics? Learners know exactly what is expected; there are

clear targets and expectations. Rubrics protect against evaluator bias because they

are consistent. Rubrics evaluate on the sum of a full range of criteria

rather than a single numerical score. Rubrics empower students because they can use

them to develop their abilities. Rubrics can be created for any content area and can

be modified easily for various grade levels.

Parts of the rubricParts of the rubric

A rubric is a grid made up of four basic parts:a task description (the actual assignment which involves performance expected of the student)a scale of the levels of achievement (marks or descriptions of levels of achievement)the dimensions of the assignment (a breakdown of the skills/knowledge involved in the assignment)descriptions of what constitutes each level of achievement (specific feedback)

04/11/23 Induction workshop for UWC staff 10 & 11 March 2010

23

Title Task Description

Scale Level 1 Scale level 2 Scale level 3

Dimension 1

Dimension 2

Dimension 3

Dimension 4

04/11/23 Induction workshop for UWC staff 10 & 11 March 2010

24

Checklist for a good rubric:Checklist for a good rubric:

____ Rubric Categories

Do the categories reflect the major learning objectives?

____ Levels Are there distinct levels which are assigned names and mark values?

____ Criteria Are the descriptions clear? Are they on a continuum and allowing for student growth?

____ Student-friendly Is the language clear and easy for students to understand?

____ Teacher-friendly Is it easy for the teacher to use?

____ Validity Can the rubric be used to evaluate the work? Can it be used for assessing needs? Can students easily identify growth areas needed?

ReferencesReferences Butcher, C., Davies, C. & Highton, M. (2006). Designing Learning: From Module Outline

to Effective Teaching. London & New York: Routledge. Chapter 6, pp.93-129. Driscoll, A. & Wood, S. (2007) Developing Outcomes-based Assessment for Learner-

centred Education: A Faculty Introduction.Virginia:Stylus. Chapter 4 Irons, A. (2008). Enhancing Learning Through Formative Assessment and Feedback.

London and New York: Routledge Knight, P. (2001). A briefing on key concepts: Formative and summative, criterion and

norm-referenced assessment. Assessment Series No. 7. Generic Network: Learning and Teaching Support Centre

Maki, P.L. (2004). Assessing for Learning:Building a Sustainable Commitment Across the Institution. Sterling: Stylus. Chapter 5 119-152.

McNamara, J. & Burton, K. (2009). Assessment of Online Discussion Forums for Law Students. Journal of Teaching and Learning Practice.6(2): http://ro.uow.edu.au/jutlp (accessed 18 August, 2010).

O’Donavon, B., Price, M. & Rust, C. (2004) Know what I mean? Enhancing student understanding of assessment and criteria. Teaching in Higher Education, 9(3):325-335

Stevens, D.D. & Levi, A.J. (2005). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning. Sterling: Stylus. Chapter 3, pp. 29-46.

04/11/23 Induction workshop for UWC staff 10 & 11 March 2010

26

Do we have a sense ofDo we have a sense of

1. Assessment and alignment

2. Forms of assessment

3. Assessments and marks

4. Reliability and validity

5. Traditional and alternative assessment

6. Norm and criterion-referenced assessment

7. Rubrics?