Creating and Using Rubrics for Formative Assessment and Grading

Facilitators: Prof. Lilian Kwan (關利平教授) Associate Vice President

Dr. Kai Pan Mark (麥啟彬博士) Associate Director (e-Learning) Information Technology Services Centre

1

22 February 2017

Intended Learning Outcomes

After this workshop, you will be able to: 1. Identify the main features of rubrics for

formative assessment tasks. 2. Determine the criteria for assessing

formative tasks in alignment with CILOs. 3. Describe the outcomes of different

performance levels of each criterion to accurately reflect the performance of the class to avoid grade inflation / deflation.

2

What do I know

about my students?

What do I want my students to learn?

How can I facilitate

that learning?

Teaching and

Learning Activities

Learning

Outcomes

Assessment

Learner

Centered

How do I know that learning does happen?

RECAP: Outcomes-based Teaching and Learning (OBTL)

3

RECAP: Three Elements in Outcomes-Based Teaching and Learning

ILO’s

What the

students have to

learn.

TLA’s

Engaging students in

activities to achieve

the stated ILO’s.

AT’s

How well have

the students met

the ILO’s.

4

What is the purpose of assessment tasks?

Formative and Summative Assessment

Formative Summative

Time During a learning activity (e.g. interim assessment of

project report, essay, )

At the conclusion of a learning activity (e.g. final project report & presentation, test,

exam)

Goal To capture students’

progress towards ILOs

based on criteria and

standards defined in

rubrics

To capture students’ achievements at the

end of the semester towards ILOs based

on criteria and standards defined in

rubrics

Feedback Make comments on parts

deserving commendation /

improvement

Final judgement – feedback process

usually absent (but can post overall

feedback on course site)

Form of Feedback

Comments with or without

score/ grade

Grade, score

Frame of reference

Sometimes normative (comparing each student against the others)

Sometimes criterion (evaluating each student according to a set of

common criteria ) – 100% in UIC by 2017/2018. 5

Teaching & Learning Activities can also be Assessment Tasks

• Ask a question

• Case-study

• Concept Map

• Create ‘product’

• Debate

• E-portfolio

• Essays

• Experiments

• Field trips

• Group Discussion

• Individual project

• News article review

• Oral Presentation

• Poster design

• Reflective diary

• Role Play

• Site visits

• Think-Pair-Share

• Video clip critique 6

Who is responsible for assessment? • In most of the cases, only teachers take up the role to

assess students’ work

• Applying rubrics to assess students’ work can be comparatively easy to implement if only one teacher is responsible for assessing students’ work

• More difficult to apply when different teachers are involved

• Should we encourage peer assessment? • advantages / disadvantages? • How can you ensure peer assessment is fair and

objective?

7

Clear description of each performance level of each criterion must be prepared

Design Criteria of Assessment Tasks

Before designing any Assessment Tasks, an important first step

is to decide performance of what outcomes would

represent evidence that a particular learning outcome has

been achieved.

A second step is to ensure that there is an alignment between the learning outcome and the evidence. [The key to achieving alignment mainly rests with the action verbs that we choose which identify what it is a learner can do after successfully completing the Teaching and Learning Activity]

8

Example: Microbiology course

Intended Learning

Outcomes

Teaching and Learning

Activities Assessment Tasks

Identify the

important features of

Ascomycota,

Basidiomycota,

Chytridiomycota and

Zygomycota.

(Knowledge domain)

• Lecture

• Minute Paper

• Group Discussion

• Field Trip

• Online discussion

• Video

Formative

• Log books of labs

• Summary of video

Summative

• Report on field trip

• Quiz

• Examination

Communicate

effectively on

information about

fungi.

(Skill domain)

• Oral presentation from

projects/journal papers/

newspaper articles, etc.

• Writing summary of

projects/journal papers/

newspaper articles, etc.

• Online discussion

Formative

Peer Assessment from

class/group

Summative

Oral Presentation

9

Alignment

10

Formative assessment tasks can be easily adopted as teaching and learning activities. Mobile technology can be used to facilitate the activity.

Why Use Rubrics? • Provide students with clear expectations about what will

be assessed, as well as standards that should be met. • Identify what are the important outcomes to focus on • Give students guidance to self-evaluate and improve

their work and / or evaluate performance of peers • Maintains standard and makes scoring easier and faster

especially when multiple assessors are involved. Adapted from Professor Susan Gano-Phillips’ workshop on “Developing and Applying Rubrics to Assess

Student Learning in GE course and Programmes” & “RUBRIC USE AND DEVELOPMENT” @http://www.bused.org/rsabe/rsabe05.pdf

11

Assessment Rubrics • A systematic scoring guideline

• For evaluation of students’ performance on assessment tasks

• formative ATs such as essays, oral presentation, online collaborative learning activities, group projects, portfolios, etc.

• Consist of detailed descriptions of different levels of performance standards.

• Contains a list of the OUTCOMES which align with CILOs

• Focuses your attention on a few key outcomes at a time

• Guidelines for evaluation of each of these outcomes

Adpated from: http://www.utexas.edu/academic/mec/research/pdf/rubricshandout.pdf & Professor Susan Gano-Phillips’ workshop on “Developing and Applying Rubrics to Assess Student Learning in GE course and Programmes”

12

Essential Parts of the Rubrics

• Descriptors for each performance level that contain criteria and standards by which the performance will be judged.

• Criteria that describe the conditions that any performance must meet to be successful.

• Standards that specify how well criteria must be met.

• A scale of points

13

Assessment tasks to measure students’ performance should • be judged against a set of clear assessment

criteria • define the quality of performance expected of

each of the grades.

Assessment Criteria

14

The expected quality of performance have to be realistic. But gauging the performance level of the class before designing the rubrics is important.

Essential Parts of a Rubric

Excellent 4 / A / 10

Good 3 / B / 8

Satisfactory 2 / C / 6

Marginal Pass

1 / D / 4

Fail 0 / F / 0

Participation Actively involved; shows enthusiasm & engagement; is punctual; supports others; shows initiative & volunteers

Involved; often engaged & enthusiastic; is usually punctual; shows initiative when prompted

Occasionally involved and engaged; little enthusiasm; needs constant guiding and/or prompting

Little to no participation; hampers others; often late; inconsistent response to guiding or prompting

Non-submission or Plagiarism

Criterion

Standards of different Performance levels A scale of

points

Descriptors for Performance levels

Key Steps in Rubric Development (1) • Determine

the major formative assessment tasks for CRA e.g. oral presentation

the important outcomes which are aligned with CILOs

the key components (criteria) of the rubrics

the performance levels

16

Key Steps in Rubric Development (2)

Describe each performance level for each of the criteria, clearly differentiating between them to accurately reflect performance of the class.

17

Key Steps in Rubric Development (3)

18

Trial run

Review

implement

Refine with

students’ feedback

Prepare a Draft or

Review

Rubric for Oral Communication

Criteria

Level of Performance

Outstanding Adequate

Poor

Unacceptable

Introduction Opening statement and statement of thesis

Presence of an engaging introduction that conveys of important topics to audience

Presence of an introduction and clear thesis that is relevant to speech

Inappropriate introduction or lack of a clear statement of thesis

Absence of either an introduction and/or statement of thesis

Organisation Order of points and linkage with introduction and conclusions

The identification of introduction, body, and conclusion are organised in a manner that enhances the power of the speech.

Introduction, body, and conclusion are evident and the body is organised in some logical order (e.g. chronological, topical, spatial, problem-solving).

Logical order of the presentation was weak, along with vague introduction and conclusion.

The components of the speech were disorganised and did not follow in a logical manner.

19

Rubric for Oral Communication

Criteria

Level of Performance

Outstanding Adequate Poor Unacceptable

Verbal Delivery Voice, rate of speech, volume, and enunciation (as applicable)

All elements of verbal delivery effectively strike an appropriate tone.

Clear, audible, and appropriate paced oral communication.

One or more elements of effective verbal delivery are absent, thereby diminishing the presentation.

Speech was inaudible; pace was incorrect and lacked clear diction.

Review / Conclusion Contains summary and conclusion

Eloquently recaps main points and delivers a powerful conclusion.

Summarizes key points and articulates a main conclusion.

Lacks summary of key points or fails to articulate a conclusion.

There is no summary or conclusion, abruptly ending the presentation.

20

Assessment tasks to measure students’ performance should • be judged against a set of clear assessment

criteria • define the quality of performance expected of

each of the grades.

Assessment Criteria

21

The expected quality of performance have to be realistic.

But gauging the performance level of the class

before designing the rubrics is important.

Rubric for Oral Communication – Deflating grade

Criteria

Level of Performance

Outstanding Adequate Poor Unacceptable

Verbal Delivery Voice, rate of speech, volume, and enunciation (as applicable)

Verbally present the ideas in a concise manner with accurate grammar use; strike an appropriate tone.

All elements of verbal delivery effectively strike an appropriate tone; a few grammatical mistakes in the presentation

Clear, audible, and appropriate paced oral communication with an appropriate tone; lack of a focus on presenting ideas

Audible and appropriate paced oral communication with a flat tone.

22

Rubric for Oral Communication – Inflating grade

Criteria

Level of Performance

Outstanding Adequate Poor Unacceptable

Verbal Delivery Voice, rate of speech, volume, and enunciation (as applicable)

Clear, audible, and appropriate paced oral communication with an appropriate tone. Pronunciation mistakes or grammatical mistakes can be accepted when the meaning is not severely affected.

The presentation is made up of complete sentences with the appropriate vocabulary; some pronunciation mistakes or grammatical mistakes are found

The delivery consists of some vocabulary that are meaningful but cannot form a complete sentence.

The presentation is generally inaudible; many pronunciation mistakes or grammatical mistakes are found.

23

What is the standard of ‘A’ for your class - benchmarking ? • Benchmarking – evaluate by comparison with a

standard

• Suggested actions • Adopt the rubrics used in previous years, assuming the

standard of the class is similar to the previous year

• Adopt the recommended rubrics presented online for specific ATs/skills (e.g. VALUE rubrics of Association of American Colleges and Universities - http://www.aacu.org/value)

• Design your own after gauging the standard of your students by using formative assessment tools with differentiating elements

24

Benchmarking – ‘A’ for class participation A: punctual and actively involved in all classes, shows good preparation, always volunteer and shows initiative, supportive to peers

OR

A:punctual and actively involved in over 80% of classes, shows good preparation, volunteer and shows initiative most of the time, supportive to peer

OR

A: punctual and actively involved in most classes, shows good preparation in most of the classes, volunteer and shows initiative sometimes, supportive to peer

OR

A: punctual and actively involved, shows good preparation in most of the classes, volunteer and shows initiative, supportive to peer in most classes

25

Set up assignment

Please set up your assignment as normal.

26

Choose Rubrics as grading method

Please choose “Rubric” under “Grade”.

27

Get started to define the rubrics

First, click the newly created assignment link

Then, choose “Advanced grading” from the left Administration menu.

28

Building / reusing rubrics

You can re-use existing rubrics with this option. Otherwise, use the other option to start building a new set of rubrics.

29

New rubrics: Input the contents

Please click on the respective items to change the criterion, level and points

30

Release assignment to students

The assignment will be released to the students as usual without any additional steps. Once a student clicks on the assignment, the rubrics will be displayed.

31

Release feedback to students

Students will be able to see the comments marked with reference to the rubrics.

32

Display in Gradebook

All grades/ points are converted from the rubrics and recorded properly in the Gradebook.

33

Further applications

• Oral Presentations • Submit the presentation materials in advanced

• Assess and give feedback instantly with laptop computers/ tablets while attending the presentation

• Assessment results instantly available after presentation session

• Vacation assignments and Projects • Useful for interim reports

• Deliver feedback on time

34

Summary of workflow

Design the rubrics

Set up assignment

Choose rubrics as grading

method

Input the rubrics

Release assignment to students

Assess submissions

by rubrics

Release feedback to

students

Display Grade/ Score in

Gradebook

35

Please also refer to http://docs.moodle.org/27/en/Rubrics

Intended Learning Outcomes

After this workshop, you will be able to: 1. Identify the main features of rubrics for

formative assessment tasks. 2. Determine the criteria for assessing

formative tasks in alignment with CILOs. 3. Describe the outcomes of different

performance levels of each criterion to accurately reflect the performance of the class to avoid grade inflation / deflation.

36

Evaluations

Would appreciate if you can give me

some feedback

from all of you so that I can continue

to improve on my workshops.

37

http://ispace.uic.edu.hk/

Thank You for your listening !

38