2 marking

TO APPLY PAGE STYLESRIGHT CLICK YOUR PAGE>LAYOUT

SCHOOL OF ANIMAL AND VETERINARY SCIENCES

Grading criteria and marking schemes

Liz NormanMassey Universityhttp://www.slidehare.net/liznorman

Sharanne Raidal, BVSc, PhD, FANZCVS, GradDipEd

Australian and New Zealand College of Veterinary ScientistsExaminer Workshop, 6 – 7th February 2016

https://geogteacher.files.wordpress.com/2012/09/markscheme.jpg

http://www.slidehare.net/liznorman



Validity

• The questions must elicit the behaviour we want to evaluate

• Different markers award similar marks for the same candidate response

• Markers reward features we want to evaluate and not reward irrelevant features

Johnson 2013



Purpose of marking schemes

•To help you during question writing What content is important Whether the Q asks what you intended it to ask Whether it is do-able in the time available

•To help you during question marking Helps you decide whether an answer is good enough to

be awarded a mark Facilitates reliable and fair marking



Types of marking scheme

• Model answer – “ideal” response• Point based schemes• Criteria and level (standards) based schemes• Schemes with incorporated principle(s) for

discriminating levels



Model answers

• Not recommended as a sole component of a marking scheme Usually more than would be expected to be given by any

candidate No guidance on how to assess alternative answers to the

model provided No guidance on how to award marks

• Can be a useful adjunct to a markingscheme



Point-based schemes

• Points for each objectively identifiable content point

• Do not indicate the relative importance of the points awarded

• Sum can be more or less than the whole• Rewards quantity not quality



Criteria and level (standards) bases schemes

• Criteria = different dimensions of performance• Level = different quality / standards of

performance on a given criterion

• Generic vs specific for the question• Explicit vs implicit weighting



Grade Description of grade

High Distinction This essay commands attention because of its insightful development and mature style. The response to the text is convincing and elaborated upon with well-chosen and correctly referenced examples. It is written with aptly chosen words, effectively constructed sentences and a keen observation of the conventions of written English.

Distinction This essay provides a thoughtful and well reasoned response to the text with appropriate and correctly referenced examples. The sentences are constructed and words chosen to communicate clearly to the reader. The conventions of written English have been well observed.

Credit This essay provides a competently reasoned response to the text with some appropriate and mostly correctly referenced examples. The sentence structure and choice of words have sufficient precision to communicate the message to the reader. The conventions of written English have been observed.

Pass This essay is satisfactory. It provides an adequate response to the text with sufficient examples and adequate reasoning. The examples are mostly adequately referenced. The sentence structure and choice of words communicates adequately to the reader. The conventions of written English need to be observed more closely.

Fail This essay fails to respond appropriately to the text. The responses are simplistic or incoherent and suggest some significant misunderstanding of the text. The writing lack appropriate structure and has a pattern of errors in word choice with poor grammatical expression. Correctly referenced examples are absent or poorly presented. More attention needs to be paid to the conventions of written English.



Grade Description of grade

High Distinction This response commands attention because of its insightful development and mature style. The response to the question is convincing and elaborated upon with well-chosen and correctly referenced examples. It is written with aptly chosen words, [effectively constructed sentences] and a keen observation of the conventions of written English.

Distinction This essay provides a thoughtful and well reasoned response to the text with appropriate and correctly referenced examples. The sentences are constructed and words chosen to communicate clearly to the reader. The conventions of written English have been well observed.

Credit This essay provides a competently reasoned response to the text with some appropriate and mostly correctly referenced examples. The sentence structure and choice of words have sufficient precision to communicate the message to the reader. The conventions of written English have been observed.

Pass This essay is satisfactory. It provides an adequate response to the text with sufficient examples and adequate reasoning. The examples are mostly adequately referenced. The sentence structure and choice of words communicates adequately to the reader. The conventions of written English need to be observed more closely.

Fail This essay fails to respond appropriately to the text. The responses are simplistic or incoherent and suggest some significant misunderstanding of the text. The writing lack appropriate structure and has a pattern of errors in word choice with poor grammatical expression. Correctly referenced examples are absent or poorly presented. More attention needs to be paid to the conventions of written English.



Analytic vs holistic schemes

• Both are valid• Analytical (criteria scored separately)

Better agreement between examiners Insufficient criteria Overlapping criteria Really just lots of smaller holistic decisions

• Holistic (scored as a whole) Don’t straight jacket examiners Challenging, especially for longer answers Less agreement between examiners



The verbs distinguishing levels of performance are italicised.PASSThe answer has most of the following characteristics:•Correctly describes relevant facts from the case, but...•May or may not identify the relevant theoretical framework•May or may not describe the relevant parts of the theory•Does not or incorrectly applies the theory to the object of the Q•Does not or incorrectly relates the theory to the case evidence.Answers at this level will make simple and obvious connections, but their significance may not be fully articulated. This type of response only meets one part of the task. Sometimes, this type of response deals with terminology but doesn’t develop further.

CREDITThe answer has most of the following characteristics: •Correctly and proficiently describes relevant facts from the case, and•Correctly identifies the relevant theoretical framework•Correctly describes the relevant parts of the theory•May or may not apply the theory to the object of the question•May or may not relate the theory to the case evidence, but uses case evidence well to support argument.Responses at this level outline the influences of a number of factors, but may not bring together and balance their influences. Straight-forward connections between theory and evidence sets may be made, but the meta-connections between them are missed, as is their significance for the whole.



DISTINCTIONThe answer has most of the following characteristics:•Correctly and proficiently describes relevant facts from the case, and•Correctly identifies the relevant theoretical framework•Correctly describes the relevant parts of the theory•Correctly applies the theory to the object of the Q•Correctly relates the theory to the case evidence., and uses case evidence well to support their argument.A response at this level describes the case evidence and the theoretical frameworks and demonstrates good appreciation of how they are integrated. The answer demonstrates an appreciation of the significance to the parts of the question in relation to the whole. The overall answer has a coherent and appropriate structure where the parts of the answer and the connections between them are clearly demonstrated.

HIGH DISTINCTIONA response at this level has all the characteristics of the previous level (Distinction) and in addition demonstrates further insight, extending concepts and theoretical ideas into new but clearly related areas – across a number of areas. The response demonstrates connections not only within the given subject area of the question, but also beyond it . Answers at this level generalise and transfer the principles and ideas underlying the specific instance to a number of other contexts.

Source: Sara Denize, UWS



% Learning outcome

Fail Pass Credit Distinction High Distinction

25 Knowledge and understanding of the selected industry, position & competencies

Little or no knowledge

Basic knowledge

Good knowledge Detailed knowledge

Sophisticated knowledge

25 Demonstrated critical analytical skills & integrative thinking

Good assessment of research findings & application to ....

Methodically assesses research findings & applies it to personal career choices and career decisions

25 Problem solving

25 Effective communication, creative & innovative thought

A below average presentation- little demonstration of involvement from all team members



SOLO taxonomy

Prestructural Question may be rephrased at the answer; almost completely misses the point of the question

Unistructural Able to identify, list, name, enumerate, but does not describe, explain, relate or elaborate multiple aspects of a response

Multistructural Able to list as well as describe distinct aspects of a response (such as being able to describe aetiology, clinical features, management) but unable to explicitly explain causes for observations; unable to present cause-effect relationships

Relational Able to describe multiple aspects of a process and additionally explain or elaborate observations into cause-effect relationships; able to compare similarities and differences between apparently distinct phenomena; this level is taken as suggestion the learner has understood

Extended abstract Highly developed; able to explain mechanisms of phenomena and apply this information to a novel context – able to develop novel hypotheses, theories and deduce principles; creative thinking



SOLO levels in marking schemes

PrestructuralThe task may be engaged, but the student is distracted or misled by irrelevant aspects

UnistructuralThe student focuses on the relevant domain and works with a single aspect

Multistructural The student provides correct material with discrete, separate pieces of information that may be combined to provide a composite picture

RelationalThe student offers an integrated understanding of the information. The whole has a coherent structure and meaning

Extended abstract

Abstract general principles or hypotheses are provided



Question examples at different SOLO levelsInviting a unistructural response:

• Identify descending motor pathways that primarily mediate upright posture and reflex postural adjustments

• List four examples of long acting calcium channel blockers

Inviting a multistructural response:• Describe the structure of the eukaryotic cell membrane.• Describe the clinical features of ischemic stroke.

Inviting a relational response:• Explain the mechanisms of each of the following in a patient with

cirrhosis of the liver: ascites, splenomegaly, haematemesis and neurologic deterioration.



Incorporating principles / rules

• Ideal is a specific criteria- and level-bases scheme with incorporated principle(s) for discriminating levels



AQ

A G

CS

E S

cie

nce

A P

hys

ics

1 F

ou

nd

atio

n T

ier

Ph

ysic

s 1

F S

pe

cim

en

Ma

rk S

che

me

htt

p:/

/ww

w.a

ga.o

rg.u

k/su

bje

cts/

scie

nce

/gcs

e/p

hys

ics-

44

03

/pa

st-p

ap

ers

-an

d-m

ark

-sch

em

e



Incorporating principles

Calais has a warmer winter and a cooler summer than Wroclaw. Explain why. (3 marks)

Marking scheme:

Looking for answers related to distance from the sea, therefore latitude is not credited.•Land heats up quicker than sea (1 mark)•A clear distinction between land and sea heating (2 marks) Ahmed & Pollitt (2011) Improving marking quality through a taxonomy of mark

schemes. Assessment in Education: Principles, Policy & Practice 18: 259-278



Incorporating principles

Marking scheme excerpt:

Discussion should focus on strategies adopted to ensure sustainability and an evaluation of these with regard to whether or not, or to what extent, the Sahel can be sustainably managed. The discussion will depend on the content and whether the overall view is optimistic or pessimistic.

AQA (2013) General Certificate of Education (A-level) Geography Unit 1: Physical and Human Geographyhttp://www.aga.org.uk/subjects/geography/a-level/geography-2030/past-papers-and-mark-schemes



Quality vs quantity

• More complex and unstructured questions allow better assessment of quality over quantity

• In very constrained tasks, only judging how correct the answer is

• In very open tasks, ‘correctness’ is less important and quality is judged instead



SOLO taxonomy

Prestructural Question may be rephrased at the answer; almost completely misses the point of the question

Unistructural Able to identify, list, name, enumerate, but does not describe, explain, relate or elaborate multiple aspects of a response

Multistructural Able to list as well as describe distinct aspects of a response (such as being able to describe aetiology, clinical features, management) but unable to explicitly explain causes for observations; unable to present cause-effect relationships

Relational Able to describe multiple aspects of a process and additionally explain or elaborate observations into cause-effect relationships; able to compare similarities and differences between apparently distinct phenomena; this level is taken as suggestion the learner has understood

Extended abstract Highly developed; able to explain mechanisms of phenomena and apply this information to a novel context – able to develop novel hypotheses, theories and deduce principles; creative thinking



Content vs connections

Lucander et al (2010) The structure of observed learning outcome (SOLO) taxonomy:a model to promote dental students’ learning. Euro J Dent Ed 14(3): 145-150



Writing marking schemes

• Select and organise the criteria/dimensions• Develop clear descriptions for each

level/standard of each criteria• Need to think about poor answers as well as

good ones



Revising mark schemes in use

• Hopefully all types of answer are anticipated

• Sometimes though it is not – can indicate unanticipated problems with the Q

• Marking schemes might need revising after first few candidates marked

• HSE should moderate marking and should encourage team members to report marking issues early in the piece



Key points



Marking schemes

• An indication of the key criteria for discriminating passing answers from failing ones

• Then add in what would be a better performance and what would be a lesser performance

• Think about both quantitative and qualitative aspects of an answer



Marking schemes

Quantitative:• Which parts of the answer are essential

• How many errors in these parts are tolerated

Qualitative• Thinking processes you want to examine

• What connections are essential / important

• How the structure of an answer will help you differentiate a structured, related, connected thought process from a multistructural list of unrelated facts



Marking schemes

• Thinking about how a candidate who includes all / most of the right stuff, but also includes wrong stuff will be graded

• What it indicates about the knowledge structure if there are contradictions / irrelevancies in the answer



Marking schemes

• Keep them brief• Think about them as specifying principles to

apply rather than exhaustive detail• Concentrate on specifying the differentiating

characteristics and how much of this there needs to be

• Don’t use them to educate examiners about the topic



Next session’s task:

For each of your questions:• Think about how you will discriminate a good

from a poor answer• Form this into a marking scheme

Aim is to have a full set of drafted Qs and outlined marking schemes by 5:30pm