learning analytics exemplar template
TRANSCRIPT
Analytics Exemplar:
title
Simon Buckingham Shum Knowledge Media InstituteThe Open University, UK
http://simon.buckinghamshum.net
An Analytics Exemplar has the following elements
1. What kind of research questions are we asking?
2. What theories/methodologies are used?
3. What kind of data is required?
4. What analysis tool are we using?
5. What’s the workflow to get the data into the tool?
6. What visualizations and settings are available?
7. What insights do/might we get?
What kind of research questions are we asking?
• Example: Do mentors with different learner-support roles in
the course have different kinds of social networks?
• Or… Do they engage in different kinds of conversations?
If so, then analytics might identify other learners starting to play
that role, or help compare/contrast mentor performance.
What theories/methodologies are used?
Examples:
• tying SNA metrics to a theory of social learning;
• or a systematic method for analysing a network graph;
• or tying particular discourse patterns to a pedagogical
approach;
• or engagement patterns to particular demographics.
What kind of data is required?
Examples:
Quiz data
• Dropout data
• Demographic data
• Forum data
• Video watching data
What analysis tool are we using?
• Example: NodeXL
• http://nodexl.codeplex.com
Workflow for social network analytics in NodeXL: are learners forming effective peer-relationships?
Import data into OpenRefine
Reshape using template
Export data to CSV
Process in NodeXLand generate network
Martin Hawkseyhttp://mashe.hawksey.info/2013/02/lak13-recipes-in-capturing-and-analyzing-data-using-sna-on-canvas-discussions-with-nodexl-for-when-its-not-a-snapp
What visualizations and settings are available?
• Example: NodeXL gallery
• http://www.nodexlgraphgallery.org/Pages/Default.aspx
What insights do/might we get?
Examples:
• Highly rated mentors of type 1 have social graphs like
this…
• Mentors of type 2 however look like this…
• This confirms Bloggs et al’s prediction that…
other example workflows(discourse analytics)
Workflow for quantifying reflections in forum posts: what elements of reflection are evident?
Convert discussion threads in comma-separated file format
Annotate text segments using custom components for UIMA
Convert results in CSV
Ullmann, T. D., Wild, F., & Scott, P. (2012). Comparing Automatically Detected Reflective Texts with Human Judgements. In 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. Presented at the 7th European Conference on Technology-Enhanced Learning, Saarbruecken, Germany. Retrieved from http://ceur-ws.org/Vol-931/paper8.pdf
Inspect and analyse data with R
Reason over annotations with Drools
Workflow for academic writing analytics: to what extent does student writing display the hallmarks of scholarly argument?
Extract submitted essay drafts from Course XYZ
Convert to text files for XIP
Analyse using rhetorical parser
Render in custom dashboardAnnotate onto source text
Simsek D, Buckingham Shum S, Sándor Á, De Liddo A and Ferguson R. (2013) XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). Open Access Eprint: http://oro.open.ac.uk/37391