The Effectiveness of Feedback to

Students and Scoring Rubrics

Craig Taylor

Feedback

What is feedback?

Information that is used to alter the gap between the

students actual level of performance and some

reference level. (Derived from Ramaprasad (1983)).

So information that is provided but in a way that can’t be

used by students (e.g. because students do not

understand it) is not feedback by this definition.

Breakdown of survey responses

Discipline n %

English 68 9.4

Creative writing 42 5.8

Languages 38 5.2

Drama 38 5.2

Philosophy 12 1.7

Education 62 8.6

Law 130 17.9

Theology 16 2.2

Biology 319 44.0

Total 725 100.0

Statistical Analysis of Results

Schools and Humanities Disciplines were compared

using Chi-squared tests.

A p value of < .05 indicates a significant difference.

Specific comments I will make are an interpretation of

this difference. Others are possible.

Philosophy and Theology were excluded as the sample

sizes were too small to be statistically significant.

Frequency of use of feedback for future

assessed work

0%

10%

20%

30%

40%

50%

60%

All of the time Most of the time Sometimes Rarely Never

Pe

rcen

tag

e

Able to use feedback in time

Humanities

Education

Law

Biology

Kinds of feedback received by school

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Written comments Completed marking

templates

Individual verbal comments General verbal comments

Perc

en

tag

e

Types of feedback received

Humanities

Education

Law

Biology

Usefulness of kinds of feedback by school

0%

10%

20%

30%

40%

50%

60%

Written comments Completed marking

templates

Individual verbal comments General verbal comments

Pe

rce

nta

ge

Most useful type of feedback

Humanities

Education

Law

Biology

Satisfaction with kinds of feedback

received

0%

10%

20%

30%

40%

50%

60%

70%

Strongly agree Moderately agree Neutral Moderately disagree Strongly disagree

Pe

rce

nta

ge

Happy with kinds of feedback

Humanities

Education

Law

Biology

Most common individual comments

1. Need to improve handwriting for written comments

2. More detailed comments not just ‘ticks’ and ‘nos’

3. Explanation of where student has gone wrong

4. More specific comments about how to improve work, not just comments about where student went wrong

5. Individual ‘one on one’ verbal feedback with students

6. A better indication of where marks have been lost

7. Providing an ideal essay as a guide to what is expected for top marks

8. Workshopping ideal essay

9. More comments on structure and content, less on just spelling and grammar

10. Comments throughout essay not just at end

11. Using examples to illustrate a better answer to question

12. More positive feedback

13. Letting students know what they have done well

14. Marker not simply expressing their frustration in written comments

15. Avoiding shorthand notation/symbols that student does not understand

Marking Templates – or Scoring Rubrics

Survey results do not show that no useful place for marking

templates

Perhaps there is an overreliance on this form of feedback

But equally more careful attention to the design of templates

might overcome some of the concerns indicated in the

survey

A challenge here is that there is a quite limited literature of

template or rubric design.

Varieties of Scoring Rubrics

Here it is important to note the great variety in these instruments that reflects the different purposes for their use.

Some are mere checklists designed either for ease or efficiency of marking or (in large classes with multiple markers) consistency of marking.

But others may provide students with substantial information about their level of performance, and in some cases may include inserted verbal comments such as those available through i-innotate.

Perceptions of rubrics

As Reddy and Andrade (2010) note perceptions of

rubrics vary for students as compared with instructors.

• Students frequently referred to them as serving the

purpose of learning and achievement

• While instructors focussed on the role of a rubric in

quickly, objectively and accurately assigning grades.

(Ibid p. 439)

Rubric design

Two question proposed by Allen and Tanner (2006) that are important to ask before designing rubrics are:

• What do I want students to know and be able to do?

• How will I know when they know it and can do it well?

As they go on to say, not only important for rubric design but also for the choice of assessment task as most relevant if one is to ‘collect evidence about how outcomes are met.’ (ibid p. 198)

Holistic versus analytic rubrics

An important distinction in the literature is between holistic and analytic rubrics.

Holistic rubrics assess discreet assessment criteria on a point scale with the number of points achieved on that scale being holistically described. For example on a three point scale high medium and low achievement.

Analytic rubrics on the other hand defines much more specifically what would in fact count as say high medium and low achievement.

Example of a holistic rubric

Consider an example from Allen and Tanner (2006): Suppose one is assessing a students explanation of how plants get their food from the soil, and in particular the following criterion of assessment:

Student demonstrates an understanding that food can be thought of as carbon-rich molecules including sugars and starches.

An holistic rubric might give 2 points for complete understanding of the concept; 1 point for incomplete understanding; and 0 points for not addressing the concept at all.

Example of an analytic rubric

But now an analytic rubric might further define these levels of achievement as follows:

2 points: Defines food as sugars, carbon skeletons or starches and glucose.

1 point: Attempts to define food but does not include sugars, carbon skeletons or starches.

0 points: Does not address what could be meant by food.

Challenges for rubric design

Obviously analytic rubrics provide more detailed feedback.

They attempt to define very specifically what ‘language and knowledge’ the student must show to demonstrate competency (Allen and Tanner (2006), p. 198).

But as Allen and Tanner also point out we can think of this as ‘an attempt to make discreet what is fundamentally fuzzy.’ (ibid)

This is perhaps a significant challenge in using these kinds of rubrics in many of the disciplines in this Faculty. What scope is there for creative or innovative answers?

Challenges for rubric design

But there are as Allen and Tanner note ways of making rubrics more effective.

• Describe the levels of quality and proficiency for separate criteria.

• Avoid here using words like ‘good’ or ‘creative’ as these are so general as to be fairly useless for guiding the learner.

• Consider how many levels of mastery learners can make any sense out of. Some educators argue that three levels (basically ‘there’, ‘somewhat there’, and ‘not there yet’) is the most that learners can really process.

Providing rubrics to students

Looking at our survey and qualitative comments:

A concern many students have is perhaps with knowing what is expected of them in a given assessment task.

Students are much more likely to find rubrics useful if handed out with the relevant assignment rather than just afterwards with their grade (Schneider (2006)).

A concern though: students may focus too much on satisfying the criteria in the rubric and not enough directly on the actual assessment task. Which may, for example, inhibit creative answers not anticipated by instructors.

Further innovation in rubric design

Clearly there is much scope for further research and

innovation for more effective rubric design.

We noted a wide variety of different rubric designs

provided to students who took our survey.

One innovation worth noting was a rubric used by

Biology that required students to incorporate feedback

from a previous assessment exercise in a current

assessment task.

Further innovation in rubric design

The survey results again: Biology students were more

likely than students in other schools to use feedback on

previously assessed work in subsequent assignments.

So there is at least prima facie evidence that this

innovation has been effective in encouraging students

to use feedback in this way.

References

• Allen, D., & Tanner, K. (2006). Rubrics: Tools for Making

Learning Goals and Evaluation Criteria Explicit for Both

Teachers and Learners. CBE–Life Sciences Education 5: 197–

203.

• Ramaprasad, A. (1983). On the Definition of Feedback.

Behavioral Science 28, no.1: 4–13.

• Reddy, Y.M., & Andrade, H. (2010) A Review of Rubric use in

higher education. Assessment & Evaluation in Higher Education

35, no. 4: 435–48.

• Schneider, J.F. (2006) Rubrics for teacher education in

community college. The Community College Enterprise 12, no.

1: 39–55.

• Taylor, C., & Burke da Silva K. (forthcoming) An analysis of the

effectiveness of feedback to students on assessed work. Higher

Education Research and Development