summative assessments online: high stakes – low risks
DESCRIPTION
a. Summative Assessments Online: High Stakes – Low Risks. Dr Richard Parsons Centre for Learning and Teaching University of Dundee [email protected]. University of Dundee. Considerable experience of online assessments Blackboard VLE users - PowerPoint PPT PresentationTRANSCRIPT
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Summative Assessments Online: High Stakes – Low
RisksDr Richard ParsonsCentre for Learning and TeachingUniversity of Dundee
learning enhancement unitCentre for Learning and Teaching 22 April 2023
University of Dundee Considerable experience of
online assessments Blackboard VLE users Delivering up to 2000
formative assessment sessions a day with QM Perception
Delivering 50+ summative exams a year with QM Perception
Further expansion expected
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Online Assessment SystemAssessment
hardware design & deployme
nt
Staff and
student educatio
n
Central assessment support
Assessment software design &
deployment
Learning systems
integration
Clear policy and procedure
s
learning enhancement unitCentre for Learning and Teaching 22 April 2023
The Hardest Task
Remains writing quality questions Staff time is precious Students are more critical than ever before Assessments to be aligned Questions need to be prepared ahead of
time Quality runs to feedback, distracters,
variety, depth, currency Authoring Q within the software is
secondary
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Staff and Student Education At Dundee we have an
Online Assessment course Covers technical and
pedagogical aspects 5 weeks long, 3 to 6 hours
a week. Formally accredited by our
Education Faculty (10 SC) Building a bank of staff
skills Most students need little
guidance
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Examples of Assessment Strategies 15 Q from pool of 50, max mark taken (Languages)
Take assessment first, then repeat in an open fashion. Mark taken from first run. (Medicine)
Revision questions with feedback relating to lectures (Art History)
10 numerical questions, each from a pool of 50 randomly generated Q (Life Sciences)
Student revision – students invited to compose 2 Q each with FB, all compiled automatically, checked & made available for open assessment (Env Sci)
Open assessment, one submission only (Law) Summative (range of departments)
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Quality Hardware and Systems Central servers built & maintained by ITS LEU involved in system design Dual servers, one formative, one summative Compaq ML370 Dual Pentium 1400 MHz 2GB
RAM, single NICs Databases
Formative on Oracle Unix serverSummative on local Access
Blackboard VLE – ELS 6.0 Test
QMP examination CAA server CAA-01
Blackboard VLE server replica of production system
Sun Blade 1000, 2 750MHz processors, 2 GB RAM
A1000 300GB RAID5 array
QuestionMark Perception application and database server
Windows 2000 Server, IIS, Access database
Compaq ML370 Dual PIII 1400MHz 2 GB RAM
200 GB RAID5 array
Development VLE environment
Existing & operational
LSExam1
CAA-01 Username Matric no.
QMP formative CAA server CAA-02Blackboard
VLE – ELS 6.0Blackboard VLE web application server – Dumpty
Sun V480 application server 4 pro, 16GB RAM
800GB + RAID5 array
QuestionMark Perception application and database server
Windows 2000 Server, IIS, Oracle 8i database
14000 student users, 1000 staff users Compaq
ML370 Dual PIII 1400MHz 2 GB RAM
200 GB RAID5 array
Operational Sept 2003
Existing & operational
Database server Oracle 8i. Humpty
Sun V480 database server 4 pro, 16GB RAM
800GB + RAID5 array
CAA-01 Database server
Oracle 8i.
LEU (DJW & RP PCs) linked by Netdrive and Oracle 8i client
Assessment publication
LSExam2 CAA-02 xx_username Matric no.
Backup route only
Assessment publication
LEU (DJW & RP PCs) linked by Netdrive and Access publish tool
QMP Test CAA server CAA-03
Development CAA system
Quality Hardware
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Clear Policy and Procedures Central online assessment system required
clear guidance Prepared draft Policy and Procedures Generally follows BS7988:2002 Developed for about a year, inc. consultation Now passed at University Senate Excellent for ensuring quality Runs to 16 pages Additional documentation for Online
Assessment system administrators
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Summative Procedures Assessment must be tested Assessment loaded to both servers,
scheduled on one Central staff attend examinations Use special ”exam” accounts Use Perception Secure Browser Candidates complete Examination
Attendance form Immediately after the exam a
backup of results is taken Use two way radios or mobiles No system is fullproof, 99% success
target ?
learning enhancement unitCentre for Learning and Teaching 22 April 2023
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Final Points of Interest
Once had a problem of results displayed immediately to students finishing a summative
Industrial action on assessments left the machines running unsupported
The best people to assess a subject are experts in the subject, not experts in online assessment
OA is appropriate for usage outside of the sciences
Staff are becoming “managers of teaching” Online assessment is important, but not life
threatening
learning enhancement unitCentre for Learning and Teaching 22 April 2023
The Future for Online Assessment
Secure, stable, performing system Increased range and depth of questions Innovation within and beyond QM Work with the staff on questions Increased sharing/sales of question banks Foresee increased integration with online
learning systems
learning enhancement unitCentre for Learning and Teaching 22 April 2023
Summary
Dundee has an institutional online assessment system
Software, hardware, P&P, staffing, integration and education
Secure and reliable, but straightforward to use for students and staff
Let the academics create the Qs to stimulate learning
Costs are neither insignificant or astronomical Expect the internet developments to continue
P3 Student Management
System
e-Directory
QM Formative server CAA-02
QM Exam server CAA-01
January 2004University of Dundee Managed Learning Environment
User exists 3 weeks before appointmentCompleted daily
Student existsStaff details also stored in SMSModule enrolments of studentsModule assessment mark storageLDAP client?
Create user name and email process LDAP authenticationRemove user process
Library Catalogue accessResource management systemLDAP client?
Module enrolmentBb clientMark return to Bb
Special exam enrolmentMark return
Blackboard VLE – ELS 6.0
GroupWise Web Email
Novell Fileservers Student Desktop
LDAP
Staff Assignment Record
Secure web access
Hardware 2 * 480sVersion 6.0.10Users created from SITS SMS systemModule enrolmentStaff assignments from web systemLDAP clientModule assessment mark storageTraining teaching staff
Richard Parsons, CLT LEU
File store
Dual Novell LDAP servers