keynote presentation pt.1 at eassessment scotland 14: viewing summative assessment through different...
DESCRIPTION
The first keynote I delivered at #eas14 used Brookfield's notion of lenses of reflection to encourage participants to consider other roles in everyday practice. This presentation focussed on a number of examples, including visualising assessment timelines.TRANSCRIPT
VIEWING SUMMATIVE ASSESSMENT
THROUGH DIFFERENT LENSESPeter Reed
eAssessment Scotland Keynote Presentation
@Reedyreedles#eas14
CC BY flickr photo by ** RCB **
What’s your role?
CC BY-NC flickr photo by mattcornock
What do you see when you think about assessment?
CC BY-ND flickr photo by Adam Hinett
LEARNING TECHNOLOGIST
PROJECT COORDINATOR ACADEMIC
SELF REFLECTION
STUDENTS PEERS
BROKFIELD’S LENSES
LITERATURE
SELF REFLECTION
STUDENTS PEERS
AM I CRITICALLY REFLETIVE?
PROGRAMME SYSTEM(S)
LITERATURE
STUDENTSTHE
INSTITUTIONRECRUITMENT& RETENTION
NSS as driver for change
What can we do to improve the student experience?
Is there a danger in using the NSS as a driver for change?
A BIGGER PICTURE…
FROM THIS…
CCBY-NC-ND flickr photo by Stanford EdTech
WHERE STRATEGIC CHANGE IS NEEDEDTO THIS?
CC BY-NC-SA flickr photo by MizzD
SYSTEMS & ‘THE SYSTEM’
STUDENT AS PRODUCER !=
TRADITIONAL COURSEWORK
RECEIPTING
We're operating in a world in which the rules and processes we have were not invented for
‘Practice remains stubbornly resistant to change’ (JISC, 2012)
FABRIC & CULTURE
FABRIC & CULTURE
WRITTEN EXAMS ONLINE EXAMS
WORK-BASED FORMS ELECTRONIC SIGNATUARES
PATCH UP VS REBUILD
Bigger picture & Perspectives
Rebuilding ‘The System’
CC BY-NC flickr photo by That Canadian Grrl
FOCUSING THE PICTURE
SELF REFLECTION
STUDENTS PEERSLITERATURE
LITERATURE & SUPPORT
SELF REFLECTION
STUDENTS PEERSLITERATURE
TECHNOLOGY & THE LIFECYCLE
EFFICIENCY
ENHANCEMENT
TRANSFORMATION
SPECIFYING
SETTING
SUPPORTING
SUBMITTING
MARKING & FEEDBACK
RECORDING GRADES
REFLECTING
Very modular?
CONSEQUENTIAL VALIDITY
"Students experience the interaction effects of one form of
assessment on another. In any given month they may have to
complete ten assessment tasks, in another month only one. The ways
in which they approach each of these will be influenced by the others.
A task which is intrinsically interesting and which may be approached
meaningfully at any other time may be given short shrift when it is
located among a thicket of examinations.
Very little attention has been given to the compounding effects of
assessment even when we know that it is the total array of demands
in a given period which influences how each one is tackled.”
(Boud p3, in Knight 1995)
ASSESSMENT TIMELINES
What does assessment look like through the student lens? As a whole?
What do you see?
Represents low stakes assessments
Represents medium stakes assessment
Represents high stakes assessments
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
ASSESSMENT TIMELINES
Bunching effect
Represents low stakes assessments
Represents medium stakes assessment
Represents high stakes assessments
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
ASSESSMENT TIMELINES
Little/no feed forward within a module
Represents low stakes assessments
Represents medium stakes assessment
Represents high stakes assessments
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
ASSESSMENT TIMELINES
…or between modules
Represents low stakes assessments
Represents medium stakes assessment
Represents high stakes assessments
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
ASSESSMENT TIMELINES
Strategic planning of summative assessments (ESCAPE Project, Hertfordshire)JISC Design Studio on Transforming A&F
Week 3 6 9 12
Week 3 6 9 12
Week 3 6 9 12
Continuous or low stakes assessments throughout
Split one high stakes summative into two medium stakes assessments & low stakes
Traditional summative assessments at the end of module
Represents low stakes assessments
Represents medium stakes assessment
Represents high stakes assessments
Week 5 10 25 403020 5035 4515
Y1
Y2
Y3
Y4
Y5
Represents formative assessments
Represents re-sit opportunity
Represents high stakes assessments
SEMESTER 1 SEMESTER 2 SEMESTER 3
ASSESSMENT TIMELINES
Draft MBCHB Curriculum
ASSESSMENT TIMELINES
Week 10 13 16 22 26 29 39 433024
YEAR
1
1.011.02
1.031.04
1.051.06
1.071.08
WRITTEN EXAM2 X 1.5 HR PAPERS
RESEARCH & SCHOLARSHIP
Week 4 8 13 26 38 433121 4934 45
2.012.02
2.032.04
2.052.06
2.072.08
OSCERESEARCH & SCHOLARSHIP
WRITTEN EXAM2 X 1.5 HR PAPERS
RESIT
YEAR
2Represents online formative assessments
Represents re-sit opportunity
Represents summative assessments
RESIT
What things pop out at you?
ASSESSMENT TIMELINES
SELF REFLECTION
STUDENTS PEERSLITERATURE
STUDENT WORKLOADGOOD PRACTICE LOST
GOOD PRACTICE LOST
PROGRAMME SYSTEM(S)WEIGHTINGS & FORMATS
IMPACT ON PROCESSES
FEED-FORWARD
LANGUAGE
SYSTEM PERFORMANCE
TESTA Project @ Dundee
USING THE LENSES
SELF REFLECTION
STUDENTS PEERSLITERATURE
ONE MORE THING
‘Learning analytics’
‘refers to the interpretation of a wide range of data
produced by and gathered on behalf of students in
order to assess academic progress, predict future
performance, and spot potential issues’ (L. Johnson,
Smith, Willis, Levine, & Haywood, 2011).
ONE MORE THING
“Assessment analytics offers the potential for
student attainment to be measured across time, in
comparison with individual students’ starting points
(ipsative development), with their peers and/or
against benchmarks or standards” (Ellis, 2013, p.
663).
ONE MORE THING
ONE MORE THING
Ahead of Mark’s keynote, consider:
• How/what might we analyse?
• Reflect through the student lens - ethical issues
your data being analysed?
• Staff lens – what about your data?
• How might we use data to inform future
practice?