Tools for Online Assessment

Craig Brown

Sarah Honeychurch

John Maguire

Moodle Quiz

Craig Brown

Why use Quizzes• Immediate feedback

• Help student focus

• Give students time to act on feedback

• Not public to peers

• Multiple attempts

• Spot gaps in knowledge

• Save class time

• More individually focused

• Quickly gage progress

Tips for using quizzes• Takes time so plan

• Make questions valid to the course

• Question banks

• Weight your questions

• Feedback for answers

• Review options

• Randomise

• Media and images

• Import question and reuse/repurpose

Quiz for formative assessment• Test quiz

• One question per page

• Asynchronous

• Timing

• Attempts

• Security

Short-AnswerPros

•Automatic marking

•Quick to write

Cons

•No real understanding required to answer

Essay questionPros

•Gives opportunity to demonstrate deep knowledge

•Can help develop writing skills

Cons

•Time to mark

•Can result in poor quality writing if not properly timed

Drag and DropPros

•Automatic marking

•Engaging

Cons

•Time

•Accessibility

Multiple choice questions (MCQ)

Pros

•Automatically marked

•Can cover a wide range of topics

Cons

•Time

•Can confuse students

•Students can ‘Guess’

Discussion

● When is it good to use a quiz?

● Do you think the quiz tool is a useful form of assessment?

● How successful do you think MCQ questions are?

Moodle Assignment

John Maguire

What is Moodle Assignment?

Assignment activity provides an area where students can submit work

electronically for teachers to grade and give feedback on.

Guides to assignment

Moodle.gla.ac.uk

Assignment• File submissions

• Students can upload one or more files

• If pdf is uploaded it can be annotated in the browser.

Assignment• Submission settings

• Submit button• Important when using Urkund, and Group submission

• Submission statement• Required for submissions (Senate Policy)

• Removes need for a paper copy

Assignment• Group submissions

• Submit in groups• Groups members will be able to see anything that is uploaded by others members

• 1 person submits, 1 submits others agree, all submit separate documents.

Ensure groups are setup before the assignment is created

Assignment• Feedback types

• Comments

• Files

• Offline grading worksheet

Grading worksheet

CSV files do not retain formatting, important when using feedback comments

Enter grades in the Grade column

Ignore the Scale column, this simply displays the available scale

Assignment• Blind marking

• Will hide the student’s name.

• If names are revealed they cannot be hidden again

Assignment• Marker allocation

Can be used to allocate markers to specific submissions. This could be used

to ensure that subject specialists are assigned to mark questions about their

specialism.

Assignment• Marking workflow

Marks will go through a series of workflow stages before being released to

students. This allows for multiple rounds of marking and allows marks to be

released to all students at the same time.

Assignment - Urkund• Plagiarism detection

• Develop good academic practice

• Senate office policy (draft)

• Multiple or single submissions

Assignment - Urkund• Plagiarism detection

•Develop good academic practice

• Senate office policy (draft)

• Multiple or single submissions

http://bit.ly/1TESSBQ

Discussion

What is an acceptable percentage when considering plagiarism reports?

Plagiarism or similarity checker?

http://www.gla.ac.uk/services/senateoffice/studentcodes/staff/plagiarism/plagiarismstatement/

DiscussionWhat is plagiarism?

A.Writing or copying a short piece from a source verbatim without stating who the original author is.

B.In collusion with your tutor/teacher, work further on a text that you have yourself written earlier.

C.Translate another author’s text into a different language and use the result without citing the source.

D.Make use of an existing text without stating who the original author is, but make small adjustments that alter the word sequence

and sentence structure, replace words with synonyms, remove or add individual words, and so on.

E.Reading several texts and reworking the contents of these into a new text that isn’t like any of the original sources other than a few

words or a sentence being the same, without citing any of the sources.

F.Transcribe a maximum 3-4 sentence-long text verbatim and state who the original author is in relation to the text.

G.Translate a text you have originated yourself into another language and make use of the result.

H.Build further on a text that has previously been graded without indicating which parts are old.

Online Peer Assessment & Peer Review

Sarah Honeychurch

Why use Peer Assessment/Review?

"..if we want students to develop critical thinking, judgement

and autonomy in assignment production they should be

provided with high-level evaluative experiences similar to

those of experts. Peer review, students evaluating and

commenting on each other's work, is one way to achieve

this.”

http://www.reap.ac.uk/PEER.aspx

Peer assessment or peer review?

• Definitions

– Assessment – gives grade

– Review – gives feedback but no grade

– Two aspects:

• Writing reviews of others’ work

• Receiving reviews on own work from peers

“… many students dislike being asked to mark other students work and receiving marks from other

students. There are many reasons for this, including a belief by students that their peers do not have the

expertise to mark reliably. In contrast, students are usually very positive about the benefits of peer

review processes where marking is not involved.”

http://www.reap.ac.uk/PEER/Research.aspx

Peer assessment?

• Benefits:

– Students (say that they) want more feedback

– Students (say that they) want prompt feedback

– Does not take (much) extra staff time

• Problems:

– Concerns about the reliability of student marking (quantity over

quality?)

– Students don’t like grading peers

Peer Review

• Benefits of receiving peer review

– Accessible language of peers

– Variety of feedback

– Conflict of feedback mirrors professional life (must process and decide what to accept/reject)

• Benefits of giving peer review

– Constructing feedback requires cognitive engagement

• Knowledge construction

• Engagement with assessment criteria

• Developing discipline specific writing skills

– Variety of approaches can stimulate self-reflection

• Issues

– Summative assessment?

Online Tools for peer

assessment/review

• Moodle Workshop

• New LTI tool: ACJ (Adaptive Comparative Judgement)

• Other options

– Moodle Wiki

– Moodle Forums

Moodle Workshop

• Similar to Moodle Assignment

• Options for self, peer and tutor review

• Options for review or assessment

• Option to assign tutor mark for submission and for review

Adaptive Comparative Judgement

• Thurstone (psychophysics); Pollitt

• Students are presented with (random) pairs of submissions and

judge better/worse

• Software uses iterative and adaptive algorithm to sort

submissions

• Staff can “seed” submissions to provide grade boundaries

Pollitt, A (2012) The method of Adaptive Comparative Judgement.

Assessment in Education: Principles, Policy & Practice. 19: 3, 1-

20. DOI:10.1080/0969594X.2012.665354

Adaptive Comparative Judgement

• Benefits

– Timely feedback

– Easier than peer assessment

– Little staff time

– Has inter and intra rater reliability

• Issues

– Trust in new system

– Complex criteria

– Rich feedback

http://www.cambridgeassessment.org.uk/Images/232694-investigating-the-

reliability-of-adaptive-comparative-judgment.pdf

Discussion

• Do you trust students to peer review responsibly?

– Why/not?

• Do you think peer assessment or review are preferable?

• What about ACJ

– Would you be interested in participating in a pilot next academic

session?

Feedback