2013 03-14 (educon2013) emadrid urjc mining student repositories to gain learning analytics an...
DESCRIPTION
2013 03-14 (educon2013) emadrid urjc mining student repositories to gain learning analytics an experience reportTRANSCRIPT
Mining student repositories to gain learninganalytics
An experience report
Gregorio Robles, Jesus M. Gonzalez Barahona
{grex,jgb}@gsyc.urjc.esGSyC/LibreSoft, Universidad Rey Juan Carlos, Madrid, Spain
Berlin, Germany, March 14th, 2013
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
c©2013 Gregorio Robles, Jesus M. Gonzalez-Barahona
All figures are ours, except when the original source is specified.
Some rights reserved. This presentation is distributed under the“Attribution-ShareAlike 3.0” license, by Creative Commons, available at
http://creativecommons.org/licenses/by-sa/3.0/
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Summary: What this talk is about
Engineering students often have to deliver small computerprograms in many engineering courses
Instructors have to evaluate these assignments according tothe learning goals and their quality, but ensure as well thatthere is no plagiarism
We report the experience of using mining software repositoriestechniques
Related efforts (please, see paper)ProcedureToolsLinks and ideas
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Context
3rd-year Telecommunication Engineering students at URJC
Course on multimedia networks
Assignments: small programs that exchange multimediaset-up and content information using standardized networkprotocols: SIP, SDP, RTP, UDP, IP...
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Technologies used by students
Python
git: distributed versioning system
pep8 (a script that checks if the code follows codingguidelines)
wireshark: network protocol analyzer
Scope. The assignment includes:
Program with communication among clients and servers usingstandardized protocols
A live capture with the result of a scenario
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
2. Preprocess
Cloning of the repository
Checking if the files with the assignment exist are have beencorrectly named
Checking if the style guide has been followed (with pep8)
Evaluating the quality of the code (in our case, Pylint)
Retrieving of the git log and analysis (analyzed withCVSAnalY)
Analysis of the wireshark network exchange capture
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
3. Plagiarism detection
A note about ”plagiarism”
We use the non-open-sourcefree web service MOSS(from Stanford University)
Figure: Plagiarism
4. Functional assessment
Black box testing
Domain-specific
In computer networks:standards-oriented
Figure: Black-box testing. Source:goo.gl/3e1Fq
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
5. Post-process
Final grades for theassignment are calculated
Creates file with feedbackfor the student, with inputinformation from all thesteps
Instructors get a report ofthe whole process, includingassignments suspicious ofplagiarism, and errors
6. Personalized exam
Three type of questions:
Code snippets (own andexternal)
Black box questions
Questions about specificscenarios
Personalized exams take from 10to 20 minutes and can be donesimultaneously by many students.
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
6. Personalized exam (example question)
Figure: Personalized exams
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Experience report
Students
Welcome feedback andbecome better in followingassignments
Automatizing allows tobetter understand standards
Instructors
We promised ourselvescomplete automatizing...really not possible
We have a lot of data... butmaybe not enough of theone we want!
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Conclusions/summary
Not fully automatable, but scalable (continuous evaluationpossible)
Offers large and good feedback to students
Domain of the program is very important!
Group assessment could be easily introduced
Discussion: we have a wealth of data, but not always the onewe would really like
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics
Mining student repositories to gain learninganalytics
An experience report
Gregorio Robles, Jesus M. Gonzalez Barahona
{grex,jgb}@gsyc.urjc.esGSyC/LibreSoft, Universidad Rey Juan Carlos, Madrid, Spain
Berlin, Germany, March 14th, 2013
Gregorio Robles, Jesus M. Gonzalez Barahona Mining student repositories to gain learning analytics