Dr Michael Cejnar, MBBS, FRACP, Founder FIC Technology Pty Ltd

Education Show - Melbourne, VIC

31st August 2018 (V0.1)

Improving outcomes with

ICT Learning Analytics in K-12

Effective ICT Use

http://www.oecd.org/publications/students-

computers-and-learning-9789264239555-

en.htm

OECD – PISA 2015:

Students, Computers

and Learning, Making the Connection

OECD PISA 2015 Analysis –Students, Computers and Learning

• …students who use computers very frequently at

school do a lot worse in most learning outcomes, even

after accounting for social background and student

demographics.

• The results also show …. in the countries that had

invested heavily in ICT for education… no appreciable

improvements.

Andreas Schleicher,OECD Education Director, Directorate for Education and Skills,

OECD PISA 2015 Analysis

Time spent on

Internet at

school/ day.

Source: OECD, PISA

2012 Database, Table 1.5c.

300

350

400

450

500

550

600

2005 2008 2011 2014

Year 6

Year 10

Y6 Profficiency

Y10 Profficiency

Australia

NAP–ICT 2005-2014

58min/day (our data: 90 min)

Post DER ICT Usage in STEM S.Crook

Simon J. Crook, Manjula D. Sharma. Bloom-ing Heck! The Activities of Australian Science

Teachers and Students Two Years into a 1:1 Laptop Program Across 14 High Schools.

Int. J. of Innovation in Science and Mathematics Education, 21(1), 54-69, 2013

ICT Learning

What affects Technology Enabled Learning (TEL)?

• Computer Time – amount, grades, subjects, optimum

• Usage type - Content, vs interactive vs adaptive,

consumptive vs analytical vs creative

• Role of distraction - Off-task, task-switching, copying

• ICT skills – Which important, how assess, how teach,

skills diversity, searching, dig. literacy

• Teaching – support, styles, assessments, sharing

How do we measure & analyse these?

Learning Analytics

Learning Analytics is the measurement, collection, analysis

and reporting of data about learners and their contexts, for

purposes of understanding and optimizing learning and the

environments in which it occurs.

edQuire K-12 Learning Analytics

EdQuire K-12 Live Classroom

Live Classroom

Lesson Learning Flow Display

Reflective Reports

edQuire Data

Present LA Data from edQuire use in:

•25 Schools, independent

•Years: 6-12, 2000 students

•PCs 1:1 (90%), computer labs (10%)

•110,000 lessons (57,045 using PC**) with > 25 million

activities*

•Jan 2018 – to date (Term 1-3)

*Apps and URLs, ** >1 min

Computer Usage

First questions:

• How much time is spent on the computer?

• Do all lessons use computers?

• Different subjects and year levels?

Computer Usage Time

50% of Lessons used

PC > 1 min

30% of all Teaching Time

spend on PC

(110,000 lessons)

In PC-using lessons,

85% of lesson was on

PC

(38 of av. 45 min)

Computer Usage Time

By subject:

Questions to Schools – Do you know:

•How much time is spent on computers?

•What is the optimal usage?

•What quality of the learning on computers?

Student ERIS Model of TEL

1.Engagement

Questions we asked from the data:

• How much time do they spend on lesson task?

• When students are on task what types of resources

do they access?

• How distracted are students during lessons?

• Does resource type determine student positive

engagement?

Engagement – Time on/off task

Top 5

Microsoft Word - 19%

Zoom video conferencing – 8%

Windows Explorer – 8%

Microsoft Outlook – 5%

Google Docs – 4%

Top 5

YouTube – 38%

Spotify – 5%

VLC Media player – 5%

Netflix – 3%

Broforce (Gaming App) - 2%

Mean: 8.1%

Highest Year 10: 14.2%

Lowest Year 12: 6.4%

Engagement – OFF TASK

Engagement - Distractibility

Questions we asked from the data:

• How to measure distractibility

• What does a highly distractibility student look like?

• Who gets distracted?

• How can distraction be reduced?

What is distractibility & how is it measure?

Distractibility Index

D.I is calculated from proportion of Off-task & the number of

switches. High Value indicates more distractible.

29

Student 1 – Highest Distractibility

Distractibility

Start of lesson

8.50am

1st activity – edumate at 8.53am

1st ON to OFF task switch 8.55am

Youtube.com – song

Total time off task 11 minutes.

Total switches from on to off task 29 times

End of lesson – 9.30am

Student Distractibility

Distribution of

durations of

activities

Task Duration or

Dwell TimeStudent Distractibility

by Year

Examples of LA data-driven Teacher Strategies:

1. Blocking: ‘Cold Turkey’ App was applied to highly

distracted students

….but Report data: App installed but not being used, so

reinforced

2.Co-opting Distractions: Personalize instruction to

student’s interest

3.Self-Governance: edQuire student portal was introduced

to students to give feedback on engagement behaviour.

Distractibility – Actionability

2. Co-opting Distractions

Knowing Off-task activities can be co-opted to re-engage

students:

Year 6 Lesson: 10 Subject: English

Students: 21 Total Lesson Time: 600 mins

School: Government primary school Northwest of Sydney.

•Off task ‘DC Database’= DC Comics

•Teacher personalized instruction with

DC comic characters to successfully

re-engage high risk/distractible

students.

3. Student feedback

Students given their own

On-task Engagement Feedback

Self-Regulation

N=153, 8 wk base and 8 wk Fbk

Student’s edQuire Feedback Console

Trial:

Outcomes analysis

Off-Task Relative time

Stu

de

nt E

du

ca

tion

al O

utc

om

e

Intervening for Outcomes

Off-Task Relative time

Stu

de

nt E

du

ca

tion

al O

utc

om

e

2.Resources

Engagement – ON TASK

2.ICT Skills

1.5% used advanced

internet search skills

School LMS

- Canvas

MS Word

Google Docs

PowerPoint

MS Word

Education

Perfect

Research based

activities

19% used advanced

internet search skills

Subject: Science

Year Level: 7, 8, 9,

10 School: Co-ed

Period: Feb 2018

Student N: 307

Subject: Science

Year Level: 7, 8, 9,

10 School: Co-ed

Period: Feb 2018

Student N: 300Student-centered learning

Student-self-directed

Traditional Teaching

email

CAD

ICT Learning Profiles

vs

Science

Bloom Taxonomy - High vs low groups

Bloom Taxonomy variability Subjects

Bloom

Blooms Digital Taxonomy

Learning Profiles by Subject

In Summary

1. Classroom computer efficacy remains

inconsistent and challenging.

2. Effective TEL needs engaged undistracted

students, the right resources and ICT skills

to use them.

3. Real-time practical classroom learning

analytics is demonstrated in K-12 and

provides timely, formative & practical data

for making TEL effective.

Thank You