why emphasise formative assessment? mantz yorke queen’s university belfast 25 february 2005
TRANSCRIPT
Why emphasise formative assessment?
Mantz Yorke
Queen’s University Belfast
25 February 2005
Formative assessment …
implies no more (and no less) than a discerning judgement about [a] learner’s progress; it is ‘on-going’ in the sense that it goes on all the time; and it is formative in so far as its purpose is forward-looking, aiming to improve future learning (as distinct from the retrospective nature of summative assessment).
Greenwood et al. (2001, p.109)
A typology of formative assessment
Probably the mainapproach in HE
Where circumstancespermit
Via peer assessmentactivities
Over coffee or inthe bar
Problems if assessoris mentor, supervisor
In work-basedsituations
Only if an assessmentrequirement
Where student is acting self-critically
From Formal Informal
Teachers
Peers
Others
Self
A typology of formative assessment
Probably the mainapproach in HE
Where circumstancespermit
Via peer assessmentactivities
Over coffee or inthe bar
Problems if assessoris mentor, supervisor
In work-basedsituations
Only if an assessmentrequirement
Where student is acting self-critically
From Formal Informal
Teachers
Peers
Others
Self
Towards greater autonomy
Probably the mainapproach in HE
Where circumstancespermit
Via peer assessmentactivities
Over coffee or inthe bar
Problems if assessoris mentor, supervisor
In work-basedsituations
Only if an assessmentrequirement
Where student is acting self-critically
From Formal Informal
Teachers
Peers
Others
Self
Formative assessment
Black and Wiliam’s meta-analysis showed a size effect of 0.7
… formative assessment does improve learning …
The gains in achievement [are] among the largest ever reported for educational interventions.
Black and Wiliam (1998, p.61)
However, there are weaknesses . . .
Weaknesses (Subject Review)
In 49 per cent of cases, marking systems could be improved particularly in respect of feedback to students.This sometimes lacked a critical edge, gave few helpful comments and failed to indicate to students ways in which improvement could be made.
QAA (2001, para 28: Subject overview report, Education)
Weaknesses (Subject Review etc.)
In 49 per cent of cases, marking systems could be improved particularly in respect of feedback to students.This sometimes lacked a critical edge, gave few helpful comments and failed to indicate to students ways in which improvement could be made.
QAA (2001, para 28: Subject overview report, Education)
See also QAA (2004) Learning from Subject Review and
Learning from higher education in further education colleges in England
Weaknesses (Foundation Degrees)
Students of about one-half of the programmes
experience some variation in the quality of written
formative feedback. It is not always clear to students
how their assessed work could be improved.
In five cases review teams highlight this as a
serious problem.
QAA (2003, para 56: Review of 33 Foundation Degrees)
Learning Learning Learning
Traditional programme based on year-long units
FormAsst
FormAsst
SummAsst
Academic year
SummAsst
Learning
SummAsst
Learning
Modular programme based on semesters
Formative Assessment?
A challenge for modularity
Formative assessment
• is a more complex matter than some may appreciate
• can be construed as a signalling system
• has an important personal dimension
Assessmenttask
Assessorinterprets
Grade,feedback
Studentinterprets
Studentinterprets
Criteria
Tutor/assessorStudent
Studentperforms
Tutor/assessor’s self-learning
Teacher characteristicsSubject structureProgram specification
Student’s self-theories,general development
The virtue of small steps …
I found having large blocks of work without
assessment difficult – you don’t know if you are
grasping it or not until exam time!
Assignments weekly would be better from my
point of view.
[Female in her 30s, pursuing a science-based FD programme]
The virtue of small steps …
The less individuals believe in themselves, the more
they need explicit, proximal, and frequent feedback
of progress that provides repeated affirmations
of their growing capabilities.
Bandura (1997, p.217)
… and of supportive feedback
Students observed that feedback was given in
such a way that they did not feel it was rejecting
or discouraging . . .
[and] that feedback procedures assisted them in
forming accurate perceptions of their abilities and
establishing internal standards with which to evaluate
their own work
Mentkowski and Associates (2000, p.82)
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
• Programme structures and assessment regulations
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
• Programme structures and assessment regulations
• Quality assurance and enhancement
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
• Programme structures and assessment regulations
• Quality assurance and enhancement
Individuals and groups of teachers
• Following through the implications of the ‘signalling system’
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
• Programme structures and assessment regulations
• Quality assurance and enhancement
Individuals and groups of teachers
• Following through the implications of the ‘signalling system’
• Being imaginative in the use of ‘teaching time’
Optimising formative assessment
Institution and Department
• Culture of support for student learning
• General approach to learning, teaching, assessment
• Programme structures and assessment regulations
• Quality assurance and enhancement
Individuals and groups of teachers
• Following through the implications of the ‘signalling system’
• Being imaginative in the use of ‘teaching time’
• Involving learners in self- and peer-assessment
Threats
1. The concern with standards
2. The legacy of the ‘scientific measurement’ paradigm
3. Increased student/staff ratios
4. Unitisation of curricula
5. Research etc.
6. Students maximising the ratio of grade/effort
What might help? Broad principles
Not trying to do it all yourself
Encouraging the students to engage in self- and/orpeer-assessment, using provided criteria and/orexamples…
… though they are likely to query the provision thatthey are paying for
Actively encouraging students to buy into a culture of learning
What might help? The ‘Patchwork Text’
An example from Sociology (McKenzie 2003)
Choose 4 from the following 7:
• Questions relating to a text• Review of a key text• Case study problem• Presentation relating to a news report• Data analysis exercise• Report on visit(s) to local school• Conduct an interview and analyse it
plus
• A reflective account of the options taken
500-800 words/equivalent each, total no more than 4000
Gibbs’ example from Engineering
The problem: large numbers, non-engagement, heavy marking of problem-sheets
1. On six occasions during the course, students brought worked answersto problem-sheets; problem-sheets randomly distributed; marked in class according to given criteria
2. Problem-sheets returned immediately; marks not recorded
3. Requirement to complete ¾ of the problem-sheets to enter exam
4. Exam (only) used to differentiate student performances
Gibbs (1999, pp.43-44)
Gibbs’ example from Engineering
The problem: large numbers, non-engagement, heavy marking of problem-sheets
1. On six occasions during the course, students brought worked answersto problem-sheets; problem-sheets randomly distributed; marked in class according to given criteria
2. Problem-sheets returned immediately; marks not recorded
3. Requirement to complete ¾ of the problem-sheets to enter exam
4. Exam (only) used to differentiate student performances
Result: Av. Mark 45% c75%; fails near zero Gibbs (1999, pp.43-44)
Key factors in Gibbs’ study
Saved on staff time
Relevant learning (in this case, problem-working)
Rapid feedback, where accuracy was not vital
Students paid attention to the feedback
Involved creativity in pedagogy
Engaged students actively (social pressure?) …
… and encouraged their appreciation of what was expected (implicitly developing self-regulation)
Three wider issues
• Employability
• Widening participation
• Recording student achievement
Three wider issues
• Employability
• Widening participation
• Recording student achievement
Employability;broader personal effectiveness
Subjectunder-standing
Meta-cognition
Skilfulpracticesin context
Personalqualities, includingself-theoriesand efficacybeliefs
E
S
U M
The USEM account
The importance of ‘the personal’
Malleable v. fixed self-theory Dweck (1999)
Learning > performance goals Dweck (1999)
Emotional state Boekaerts (2003)
Locus of control Rotter (1966)
Self-efficacy Bandura (1997)
Learned optimism Seligman (1998)
Practical intelligence Sternberg (1997)
Emotional intelligence Goleman (1996)
Knowledge gain: effect sizes
Meta-analyses Size N studies
Self-system (E of USEM) 0.74 147
Metacognition (M) 0.72 556
Marzano (1998)
Three wider issues
• Employability
• Widening participation
• Recording student achievement
Successful ‘WP institutions’
Inter alia, tend to
• Emphasise early formative assessment
• Be committed to the support of students
• Accentuate the social in programmes
Three wider issues
• Employability
• Widening participation
• Recording student achievement
Issues in the Burgess Report
• Recording of student achievement
• The honours degree classification
• If no honours degree classification, then what?
• Credit: valuing something?
Change
Educational change is technically simple and
socially complex.
Fullan (2001, p.69)
Change
Educational change is technically simple and
socially complex.
Fullan (2001, p.69)
There is no quick fix.