using open source assessment and feedback tools
DESCRIPTION
Slides from webinar in using open source assessment and feedback tools 15 Feb 2013TRANSCRIPT
Using open source assessment and feedback tools
2
Introduction
• JISC's Assessment & Feedback programme strand C: Technology transfer
• Open source• Embed use in third party institutions Lessons learned: http://tinyurl.com/afstrandc
• The Tools: Rogō
Full assessment suite Uniqurate
Item and Test Editor, QTI compliant QTIDI project:
QTIWorks• Item and Test renderer, QTI and LTI compliant
LTIConnector and LTIQuizzes• IMS Learning Tool Interoperability demonstrators
Dr Simon WilkinsonUniversity of Nottingham
Rogō OSS
R RogōOpen Source Software
AimThe Rogō OSS project aimed to take software developed at the University of Nottingham and package it up and create a vibrant open source development community.
Rogō OSS
“
”
Enterprise-level e-Assessment
Rogō OSS
• 7 paper types (inc formative, summative)• 15 question types• Searchable – content, keywords, Bloom’s Taxonomy• QTI 1.2 and LTI 1.0• Feedback
– questions– learning objectives
Open Source
Rogō OSS
• Released under GPL 3.0
• Server: Linux, Apache, MySQL, PHP
• Clients:– PC or Mac – IE 8/9, Firefox, Chrome– JavaScript and Flash
• Languages: English, Polish
Partners
De Montfort University
University of Bedfordshire
University of East Anglia
University of Oxford
University of the West of Scotland
Rogō OSS
Challenges
• OSS – raising awareness
• How best to operate?
• Legacy code-base
• Local idiosyncrasies/procedures
Rogō OSS
Timeline
Basic Housekeeping• Name change – TouchStoneRogō• Website• Blog• Mailing list• TRAC – ticketing system• Installation script
Rogō OSS
Timeline
Planned Development• Upgrade script• QTI 1.2 import improved• LTI 1.0 support added
Rogō OSS
Timeline
Further Development• Sub-directory support• UI styling – CSS rewrite• Language abstraction layer
External Consultation• Testing procedures• Openness and sustainability
Rogō OSS
Lessons Learnt
• Installation– Project start time– PHP versions– Sub-directory support– Bugs
Rogō OSS
Lessons Learnt
• Testing– Manual (test plan in TRAC)– Selenium– Unit Testing– PHP scripts
• Modular Design– Language– Authentication
Rogō OSS
Future – Community
• Improved documentation– UI, code and database
• GitHub – access to code in development• Support packages
Rogō OSS
Future – Design
• OO rewrite• Modular design
– Questions
Rogō OSS
UniqurateOnline
Demo Webinar
Sue Milne, University of Glasgow & ELandWeb LtdPaul Neve, Kingston University
Project Aims
The Uniqurate project seeks to
increase the number of open e-assessment content authors,
and as a result, increase the richness of assessment content available to students
by
delivering QTI authoring tools that can be used by a novice user
embedding these tools into several “client” institutions
QTI – what’s that?Question and Test Interoperability specification
Version 2.1 is becoming final about now
Standard for representing questions and tests
Many implementations worldwide: UK, France, Netherlands, Germany, Spain USA, Mexico Korea
Used by Educational institutions for internal assessments Examination authorities (Scotland, Netherlands,
Germany, Korea) Publishers for online content associated with textbooks
Why use QTI?Interoperability:
Share assessment resources with colleagues in same or other institutions Efficiency, less reinvention of wheels Economical – costly to produce good resources, so share and
repurpose
Preserve materials to prevent loss when changing systemsEnsure content separate from softwareAll these contribute to SUSTAINABILITY
Flexibility:20 or so input types – and extendable, e.g. MathsSeveral inputs in one question – engaging resourcesTargeted, individualised feedback, related directly to input
Uniqurate – user-led design
Uniqurate project: JISC Assessment and Feedback programme
Objective: increase usage of QTI in HE by transferring technology from existing tools into "client" institutions
Client partners describe what they need for teaching
Uniqurate derived from user-experience driven Aqurate and multiple interaction power of Mathqurate
Result – hopefully –a useful tool for authoring QTI content accessible to any user
Uniqurate - Question components
Constituent parts providing functionality between question/exercise and student
May map onto QTI elements – but may not! – hence the term question component “What do you want the question to do?”
Questions typically composed of several componentsSeveral inputs in one question
Identify components that provide optimal benefit
Uniqurate - Modes
Friendly Mode drag & drop components
Intermediate Mode edit HTML parts of existing files: ideal for
contextualising
Expert Mode XML editor: customise questions when more control
needed also for packaging non-Uniqurate questions with
media
Uniqurate - output
As QTI XML file
As packaged QTI XML file
As package containing QTI XML file and media: images, stylesheets, etc.
Links...Uniqurate – web application – no installation, just go to
http://uniqurate.kingston.ac.uk/Uniqurate/startup
QTI Support Site (examples, tools, resources...) http://qti-support.gla.ac.uk
Contacts...Graham Alsop (Project Leader)
Paul Neve (Uniqurate Developer) [email protected]
Sue Milne (Consultant) [email protected]
Hands-on Time
Please type any questions or comments in the chat box...
Just in Case...
The following slides are provided in case the live demo is not available...
The questions are quite mathematical in flavour, but authoring resources for other disciplines works in just the same way!
Uniqurate Questions 1: Friendly Mode
Add more components
if needed
One question can have several
(different) inputs
Uniqurate Questions 2: Intermediate Mode
Only HTML content is
visible
Click on MathML to edit
it
Click here to add
MathML
Use \qv{variable-name} in LaTeX to display variables in
MathML
Uniqurate Questions 2: Expert Mode
Files in package QTI XML for this question
Packaging Tests (Uniqurate editor)
QTIDIOnline
Demo Webinar
Sue Milne, University of Glasgow & ELandWeb LtdNiall Barr, University of Glasgow
Project AimsThe project will provide a documented and packaged
version of standards-compliant assessment delivery toolsLearning Tools Interoperability (LTI) connection between
popular VLEs and delivery softwareQTIWorks rendering and responding engine
The tools will be distributed to the project’s client partners Kingston University, Harper Adams University College University of Strathclyde.
Tools
LTI Connector – connect VLE with delivery software
Delivery software:
LTIQuizzes – deliver basic question types
QTIWorks – all-singing, all-dancing delivery – includes facilities for Maths
LTI Connector for VLEs
Learning System
Our simple QTI 2.1 Assessment Tool -
LTIQuizzes
Connecty thing
(LTI)Our fancy QTI 2.1
Assessment Tool – QTI Works
QTIWorks – QTIv2.1 delivery software
Validates questions and tests – reports problems
Supports all QTIv2.1 UK HE profile
Supports input and manipulation of Maths expressions
Full randomisation, including Maths expressions
Feedback may be targeted on analysis of input
Connects to VLE through LTI Connector
… available late summer 2012
Installation?
LTI – Moodle 2, newer versions of Blackboard and Sakai VLEs are already LTI-enabled.
LTIQuizzes – a single .war file needs to be copied to a Tomcat server, then an XML configuration file is edited to add the LTI details (server names and shared secrets).
QTIWorks – online demo at https://www2.ph.ed.ac.uk/qtiworks/ , .war file expected soon for institutional installation on Tomcat server
Links...QTIWorks – online demo at
https://www2.ph.ed.ac.uk/qtiworks/
QTI Support Site (examples, tools, resources...) http://qti-support.gla.ac.uk
Contacts...Niall Barr (Project Leader) [email protected]
Sue Milne (Project Manager) [email protected]
David McKain (QTIWorks Developer) [email protected]
Hands-on Time
Please type any questions or comments in the chat box...
Just in Case...
The following slides are provided in case the live demo is not available...
They are quite mathematical in flavour, but resources from other disciplines work just the same!
QTI WorksQTI Works https://www2.ph.ed.ac.uk/qtiworks/
Use the Demo section to try out questions and tests
Log in to set up LTI delivery data
QTI Works – Demos
Upload questions or
tests to check operation
Validate questions and
tests
Try out and download example
questions and tests from collections
QTI Works – VLE Setup
Upload questions or
tests to use in VLEs
Generate LTI data for setting up questions and tests in VLEs
Uploading an AssessmentContent of uploaded assessments –question or
test – can be changed (e.g. fix a bug)
Date of last change is displayed
Setting Up Assessment Launch DataSaving a delivery with LTI enabled creates launch
data
Data is pasted into Moodle 2 External Tool dialog or equivalent in other VLEs
QTI in Moodle 2 – Question
QTI in Moodle 2 – Question with Solution
QTI in Moodle 2 – Maths Input
Test: Algebra 1
This test is the one used in the Maths T1 topic Algebra 1. It is also in the demo course.
Question in a Test
The indicator at top right shows the question status. No feedback is visible during the test – students have been using these randomised questions in formative mode for several weeks. Candidates return to the list of questions to select another question.
Test feedback & review
These are the two parts of the feedback and review page; the scores for the questions are displayed and the candidate may return to see the questions and their input.