Learning Analytics: Exposing Student DataMyles Blaney – University of Edinburgh

2

University of Edinburgh FAQs

• Founded 1582 (its old!)

• Ranked 17th in the world (QS Ranking)

• Over 25k students

• Over 5k staff

• Dolly the sheep & Higgs Boson

• Large and Diverse

3

VLE FAQs

• Blackboard Learn is the main VLE– Moodle trial for Online Distance Learning– Custom VLEs

• 5,500 courses created a year

• Over 70% course coverage

• Multiple 3rd Party Tools Installed and developed

4

Learning Analytics

• What do we mean by analytics?– is the discovery and communication of meaningful patterns in data

Analytics. (2015, March 12). In Wikipedia, The Free Encyclopedia. Retrieved 11:40, March 30, 2015, from http://en.wikipedia.org/w/index.php?title=Analytics&oldid=651076924

• Models– predictive\retention\informative

• Case Studies– Signals, Purdue University (US)– EBEAM, University of Huddersfield (UK) – fullCRM, University of Roehampton (UK)

• BIG DATA (bigger than Godzilla)!

5

Its just data!

• Course Context

• You cant see the data trees from the data forest

• Actionable data

• Interpretation & Repercussions (for all)

• Ethics

• Play the system

• Staff and students have different needs

• Are you interested?

• Image Source: https://media.licdn.com/mpr/mpr/p/3/005/067/090/0500a7a.jpg

6

Its just more data!

• Staff impact\support– What do I do with this data?

• Student impact\support

– I am below\above or just average

• An evolving environment

Image source: http://www.bashyamspiro.com/immigration-meditation/2012/03/08/h-1b-amendments-the-times-they-are-a-changin%E2%80%99/

7

Why now?

• Demand from students (via NSS and EUSA)– How am I doing?– How is the class doing?– Enhance student experience– VLE is part of the student experience

• Demand from schools– info on student activity– Personal Tutors– Courses

• Horizon Report (the curve of doom)

• VLE Focus although wider interest throughout University

8

Were we doing this already?

• Recent Appointment of the Chair in Learning Analytics and Informatics : Professor Dragan Gasevic

• School of Physics - CourseMarks

• Msc Surgical Science (P\T)- Dr Paula Smith– 2014/15 project – ODL Students– Identify students falling behind in their studies– Email student & discuss– Dashboard

9

1ST Step is the hardest

• Project created to review Learning Analytics (Tel009)– What do staff want?– What do students want?– What is available? (is it being used)– Have other institutions implemented?

• Project team– Wilma Alexander (Lead)

– Stephannie Hay

– Myles Blaney

• Image Source:http://www.anneatulaegwu.com/2014/09/baby-steps.html

10

What did we do?

• Literature review– JISC Analytic Series Papers– Case studies– LACE (Learning Analytics Community Engagement)– SOLAR (Society of Learning Analytics Research)

• VLE Tool Review

• Student information gathering events

• Outreach to VLE communities

• User Stories– As a…..– I want to….– So that …..

Image source: https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcRDSWD4Nf5lb0SDbf9jlDe-fGoN6ARs8PlsGDSZP-eyqSIizNQb

11

User Stories Feedback

• 92 stories created

• 18 Staff and 2 students attended the sessions

• Stories were grouped with the project scope and aims:

Mandatory 4

In Scope 38

Already Available 35

Not in scope but of interest 21

Not in scope and already available 6

12

Tool Analysis (Learn)

– BIRT Reports– Course Reports – Stats column option in the grade center – Average grade display – Retention Centre– Performance Dashboard– Google Analytics– 3rd Party tools

• BB Learning Analytics• Eesysoft

13

1ST Step Conclusions

• Manual (e.g. spreadsheets and time)

• Existing tools don’t meet the requirements– however staff do have access to them

• Simplicity (generic)

• Students want to see the cohort (and don’t trust their friends)

• Access (what\when\where\who)

• Impact

• Its not a magical fix

• 2nd Project – if you build it they might come!

14

Student Feedback Sessions

• They love grade data! (especially grades)

• They love charts

• Personal Tutor access beneficial

• Simplicity

• No concerns around data collection

• They like a free lunch

15

2nd Step is meant to be easier

• Build Project Created

• Prototype Pilot

• Student feedback sessions

• Steering group

• Design

• Test, feedback, test, feedback, test feedback, test, feedback

• Live target date July 2015

Image Source: https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTyKuJU5_Opr9DZEQH457P85StD0TfIW_qOj6_UCysPeUNoLvFL

16

University of Edinburgh Student Analytics

• Building Block

• Access controlled at a course level

• Staff will need to enable the tool

• Staff will have access to an admin panel to select grade columns and discussion threads for use.

• Tool can display the following to students– Users click count within the course (heat map) – 4 week period – Their grade vs cohort via bar chart for all releases grades in the GC– Their discussion forum posts vs cohorts per forum

• Staff can access the same information

17

Pilot

18

Student Feedback from Pilot and Survey

• My grader is too strict compared to others

• It makes me paranoid

• I would rather have face-to-face contact

• It will motivate me

• I am happy with the info I already receive

• I can see how I am performing in the class

19

Student Feedback from Survey

• Please scale what information you would find most to least useful?

Yes57%

No40%

N/A3%

• Will it change your Learning Behaviour?

Activity

Grade

Discussion

0 20 40 60 80 100 120 140

• Not really except for a slight increase in paranoia regarding grades

• whether my grades are good or not I can find out myself by talking to my PT, data like this is not instructive to me individually, I have my own needs and goals

20

Survey Q: What do students want to see?

• 38% wanted to have more grade visibility

• 6% didn’t answer the question

• Replies varied– Attendance– Student satisfaction– Grading turn over

“worried this extra info would result in yet another useless email from the university” – student 91

21

3rd Step. Footloose!

• Currently more thoughts & questions than concrete statements

• Mixed response from students

• Follow on project for Moodle

• Management Information Project

• Empower students to take action via data

• Image Source: https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQScVwS6ErIA-sbJMIPrlobTexjNRCp30x2SqpqHbqMC8ODdRje

22

Any Questions

Image Source: http://www.independent.co.uk/migration_catalog/article6240886.ece/alternates/w620/Untitled-2.jpeg