hanqing zhao final presentation · limitations and future work what can be done to extend this...
TRANSCRIPT
Peer � Marking � In � MoodleHanqing � Zhao
Overview / Project Content
16/5/23
• Assessment in higher education• Marking in COMP1710• Data analysis and Mark calculation in COMP1710• Drawbacks of this peer assessment approach• How to improve the approach (Software functions)• Research on Moodle• Limitations and Future Work
Assessment in higher education
16/5/23
• Formative Assessment (2 disadvantages)
• Summative Assessment
- Exclusiveness - Difficulty of enforcing
16/5/23
Marking in COMP1710
• Formative assessment – quiz on report writing
• Summative assessment – report specifications
• Summative assessment – peer assessment
Combination of both formative assessment and summative assessment:
The Peer Marking in COMP1710
References: HEAd2015ConferencePoster Sabrina B. Caldwell, Tamás D. Gedeon
16/5/23
16/5/23
The Peer Marking in COMP1710
Data analysis and Mark calculation
References: Optimising Peer Marking with Explicit Training: from Superficial to Deep Learning S.B. Caldwell, T.D. Gedeon
• S mark – mark from the students
• D mark – mark from the expert
• SqErr rank – Square difference between S rank and D rank
• SqErr mark – Square difference between S mark and D mark
Problems of this approach
16/5/23
• In order to divide reports into 5 different levels, lecturer and tutors have to mark all the reports first.
• Create unique permutation of five reports for each students
• The reports in the spreadsheets have to be anonymized
• One report should be marked by two students
Need lots of manual work by the teaching staff due to the following element:
The main problem:
16/5/23
How to improve the approach - Idea
One possible approach is: Spreadsheets automation
Two main things included in spreadsheets automation:1. The generation of Excel spreadsheets workbook 2. The selection and permutation of five embedded reports
16/5/23
• The generation of Excel spreadsheets
16/5/23
Technology: Apache POI - a Java API for Microsoft Documents
Environment: Java
Input: Anonymized reports in PDF format
Output: Generated spreadsheets for peer evaluation
How to improve the approach - Method
• The selection and permutation of five embedded reports
program
16/5/23
How to improve the approach - Method
The selection and permutation of five embedded reports
Permutation: Heap's algorithm
- generates all possible
permutations of 5 reports
- then we randomly choose
one permutation and add it in the spreadsheet
Selection: Set a calculator c for every PDF file name, c will increase one when it is chosen, the PDF with lowest c value has higher priority to be chosen.
Research on Moodle
16/5/23
• What is Moodle
• Why should you use a
Course Management
System(CMG)
• What make Moodle special:
- Free and open source
- Educational Philosophy
- Community
16/5/23
Limitations and Future Work
What can be done to extend this project in future:• Moodle Construction Practice• PHP programming
-- E.g. download the spreadsheets from Wattle automatically -- E.g. an Email distribution and collection system to automatically
send notifications to students who didn’t submit reports / spreadsheets
Limitations on the software:1. Not so “automatic” still need manual work: The amount of the spreadsheets that can generated is fixed.2. Haven’t tested in the real-time working environment.
Questions