touchstone dr simon wilkinson medical education unit university of nottingham medical school ‘ a...
Post on 21-Dec-2015
216 views
TRANSCRIPT
TouchStone
Dr Simon WilkinsonMedical Education UnitUniversity of Nottingham Medical School
‘a local system for local people’
TouchStone
• Assessment Management System– Computer-Based Assessments– Offline assessment support– Surveys/Questionnaires
• Development started 2002
• Licensed to University of Sheffield 2003
• QABME 2009 – innovation/good practice“Touchstone as a resource for blueprinting, standard
setting, receiving external examiners’ comments on examination questions, performing post examination analysis and especially the potential to provide detailed feedback on examination performance to students.”
Why?
Assessment• Are students fit for purpose (GMC)
Quality Assurance• Are assessments fit for purpose
– Peer/External Examiners– Blueprinting– Post-exam item analysis
• Enhanced Annual Review (EAR)
Purpose
How?
Reduce Costs• Reducing time – supporting process in software
• Peer/External examiner review• Standards Setting• Blueprinting• Item Analysis
Purpose
Process
Who?
Purpose
People• Staff – Academic• Staff – Administrative• Staff – Technical• Students
Process
People
Result
Purpose
Process
People
Performance
e-Assessment – Lifecycle
Pre- Exam
Mid- Exam
Post- Exam
• Question writing• Blueprinting• Internal peer review• External Examiner review• Standards Setting
• Questions presented• Answers recorded
• Analysis of performance• Fix incorrect answers• Exclude poor questions• Export marks• Personalised student
feedback
Ref
ine
Item
s
External Examiner Comments
• UoN Requirement – external examiners given access to assessments
Standards Setting – Ebel• GMC Requirement – all knowledge exams standard set
Learning Objectives – the ‘glue’
VLE• Networked Learning
Environment
e-Assessment• TouchStone
e-Portfolio• Insight
LearningObjectives
Personal reflection Progression/Certification
e-Teaching/e-Learning
Blueprinting - sampling• GMC Requirement – all knowledge exams blueprinted
Individual student feedback
Item Analysis – Question Exclusion• Good practice – educational literature
Performance – AcademicPurpose
Process
People
Blueprinting• Sampling• Individual Feedback
Media-rich assessments• Widening learning objectives tested• Assess higher levels of cognitive engagement
Electronic-only question possibilities• Drag and drop labelling• Image Hotspots• Videos• Widening learning styles & objectives tested
Performance – EconomicPurpose
Process
People
Staff - Academic• paper reviews (peer & external examiners)• standards setting – creates pass mark automatically• blueprinting – learning objective sampling• track-changes (automatic)• archiving (automatic)
Staff - Administrative• Paper printing time/costs• Marking/scanning time• Timetabling/room booking
Performance – EconomicPurpose
Process
People
Students• 24/7 access to formative assessments• Personalised feedback on exams• Personal accessibility settings (where needed)
Performance – EconomicsPurpose
Process
People
JISC 2008:
“Assuming the student body in these areas [health and business studies] sat one e-assessed exam per year this alone would represent a cost saving of c. £7 million in staff time.”