student digital experience tracker experts

Digital student experience Tracker: an update from Jisc Helen Beetham on behalf of Jisc

22/06/2016

Student digital experience tracker pilot results 2016

Background

22/06/2016

Student digital experience tracker pilot results 2016 23

22/06/2016


Development of the Tracker»Consultations 2014-2015: demand for

a better understanding of learners' digital experiences and expectations

»No standard reliable instrument»Tracker questions based on research by

the Jisc Digital student project, inc. studies of learners in HE, FE and skills

»Consultation workshop with staff and students refined the proposed question set to 12 key questions

»Cognitive interviewing with FE and HE students, and early beta testing further refined the questions for piloting

22/06/2016


22/06/2016


The Tracker questions1.Access to digital devices, personal and institutional2.Access to digital services3.Digital activities in courses of study4.Guidance and support5.Overall digital experience6.STOP7.START8.KEEP9. (11) Value of VLE/LMS

22/06/2016


22/06/2016


Aims of the Tracker»gather evidence from learners about

their digital experience, and track changes over time

»make better informed decisions about the digital environment

» target resources for improving digital provision

»plan other research, data gathering and student engagement around digital issues

»demonstrate quality enhancement and student engagement to external bodies and to students themselves

22/06/2016


Reasons for engaging with the Tracker

22/06/2016


Aims of the Tracker

Early findings:» Big appetite for sector benchmarking» Infrastructure, curriculum and

strategy development - plus ‘understand’ student experience

»Desire for a student-facing ‘digital capabilities’ tool to use alongside the Tracker

22/06/2016


Reasons for engaging with the Tracker

22/06/2016


Piloting the Tracker» Survey implemented in BOS,

a system developed for UK HE/FEand already used by many

»50+ education providers applied:24 chosen based on a spread ofinstitutional sizes, types andlocations

» Pilot sites required to involvestakeholders, consider how theywould engage learners, and explorehoped-for benefits

»Teams at each pilot site responsible for: customising the survey, releasing it to learners, maximising completion rates and analysing their own data

22/06/2016


22/06/2016


Support for the pilot

» Six separate guides produced to support the pilot process and to ensure parity of data collected

» Sites had only the guides and collective email/blog postings to reproduce service delivery

» Project negotiated for sector benchmarking data to be made available within BOS

» Pilot sites took different approachesto engaging stakeholders and eliciting learner responses

22/06/2016


22/06/2016


Where we are today»10,753 unique data sets» Initial data report released,

press released» Each site achieved a good

response rate: analysing,understanding andresponding to their data

» Feedback about the pilotprocess (18 out of 24 sites)

»The experience of sixcontrasting sites is beingexplored in more detail,including feedback from learners and other stakeholders

22/06/2016


22/06/2016


More information

22/06/2016


» Full report is available from:http://bit.ly/student-tracker-report

» Further information about the pilot and the process:http://bit.ly/jiscdigidataservice

12

http://bit.ly/student-tracker-report




http://bit.ly/jiscdigidataservice




22/06/2016


At your tables

22/06/2016


»How might the Tracker be useful at your institution?»What would you most want to find out from your

learners?

12

22/06/2016


Headline findings

22/06/2016


22/06/2016


Sample and response rates

22/06/2016


22/06/2016


Headline 1: sector differences

There are significant differences between HE and FE, particularly in the areas of:

»personal access to devices (generally higher in HE)»use of institutional computing and printing (generally higher in FE)» access to institutional digital services (generally better in HE)» access to guidance on digital issues (generally better in FE)»digital activities as part of a course of study (generally more widely

undertaken in HE)

22/06/2016


22/06/2016


Headline 2: BYO is not a panacea

Despite high levels of personal device ownership (especially in HE), learners continue to make extensive use of institutional computing and printing facilities.

22/06/2016


22/06/2016


Headline 3: digital guidance and support

Across the board, access to digital guidance is lower than we might expect. For example:

» Less than half of all students say they have received guidance on: modifying devices to suit their individual needs, or creating a positive online presence.

» Less than two thirds of all students say they have received guidance on: bringing your own device, or developing digital skills relevant to their course.

22/06/2016


22/06/2016


Headline 4: digital activities on course

»HE students are significantly more likely than FE/skills students to have produced work in a digital format, and to have worked online with others. Overall, during the last six weeks:

»9 in every 10 students found information online»7 in every 10 students produced work in a digital format»5 in every 10 students

worked online with others»3 in every 10 students

created a personal recordof their learning

22/06/2016


22/06/2016


Headline 5: overall digital experience

Should we be doing better?

»Around 7 in every 10 students say that when technology is used by teaching staff it enhances their learning experience

» Just over half believe that digital assessments are delivered well

»31.5% of HE students and 44.5% of FE & Skills students agreethat they are told how their personal data is stored and used

»37.5% of HE students and 43.2% of FE & Skills students agreethat learners are involved in decisions about digital services

22/06/2016


22/06/2016


At your tables

»What findings are most surprising/interesting to you?»What (new) questions do you have that we could

explore in future iterations of the Tracker?

22/06/2016


22/06/2016


About the pilot process

22/06/2016


22/06/2016


Being a pilot institution

»A chance to ask questions of Rob Howe, University of Northampton, and Mark Breedon, Basingstoke College

22/06/2016


22/06/2016


Findings about the process

» Pilot sites found the Tracker easy to customise and implement»With some adjustments, especially for learners using assistive

technology, learners found the Tracker easy to complete» Pilot sites valued and used the guidance materials»Many sites are still analysing their data and formulating their

responses»Most sites say that using the Tracker has been 'quite’ or ‘very

useful' – but with some caveats (we are exploring these)

22/06/2016


22/06/2016


Quotes» ‘Thanks to the Tracker we can better understand the needs and

demands of our students...They have sent us a clear message about the importance they attach to digital skills.’

» ‘...particularly useful in being able to gather data locally and to then benchmark more widely. We would also be interested in comparative data over a number of years (to see if some of our interventions are making a difference).’

» ‘It has confirmed ... that learners want to use more types of digital learning and resources.’

» ‘I feel it is important for colleges to be aware of their students' digital experience and there may not be any other mechanism to monitor this. Also the questions are well written and appropriate.’

22/06 Student digital experience tracker pilot results 2016 15

22/06/2016


Quotes» ‘I just wanted to say how valuable we found the survey data. Although we

didn’t get the number of responses we would have liked (this is partly to do with some restrictions we have to avoid student “survey fatigue”) the data we did gather has been evaluated and reported on at key Learning & teaching committees. It has also helped us to prioritise some key plans around the IT infrastructure programme and helped us to identify some other areas where we might need to provide further support.’

» ‘I look forward to seeing the survey evolve and become a regular activity (at least in our institution).’

» ‘Yes please to the Tracker, with some core questions and the ability to add more of our own”

» ‘The Tracker has provided a real insight into the digital experience of our students.’

» ‘The results and comparisons have been really insightful.22/06 Student digital experience tracker pilot results 2016 15

22/06/2016


Changes suggested»Additional questions (optional?) e.g. on» inclusivity and accessibility,

» digital pedagogies (more about tech in the curriculum)

» digital learning and study habits (more about informal uses of tech)

» digital identity, digital literacies, digital capability and confidence

» wellbeing and safeguarding

» Explore more emergent issues, ‘things we don’t already know’ »Delivery to suit students using assistive technologies» (Alongside the Tracker) instrument that assesses learners’ digital

capabilities and gives feedback to the learner...

22/06/2016


22/06/2016


Over to you

Would your institution be interested in using the Tracker? (Show of hands)

What would make you want to use the Tracker (more)?(drivers and potential benefits on green post-its please)

What would be your concerns/barriers?(barriers and potential costs on pink post-its please)

22/06/2016


22/06/2016


Thank you for your contributions!

»Any further questions or contributions before we close?

22/06/2016


22/06/2016


More information

22/06/2016


» Full report is available from:http://bit.ly/student-tracker-report

» Further information about the pilot and the process:http://bit.ly/jiscdigidataservice

12









jisc.ac.uk

22/06/2016


»For more information contact

Sarah Knight: [email protected]

22/06/2016


Helen Beetham: [email protected]

mailto:[email protected]




student digital experience tracker experts

Education