learning analytics for c21 dispositions & skills

33
LEARNING ANALYTICS FOR C21 DISPOSITIONS & SKILLS Simon Buckingham Shum Knowledge Media Institute, Open U. UK http:// simon.buckinghamshum.net @sbskmi http://net.educause.edu/eliweb121

Upload: simon-buckingham-shum

Post on 06-May-2015

1.484 views

Category:

Education


2 download

DESCRIPTION

EDUCAUSE Learning Initiative talk: http://net.educause.edu/eliweb121

TRANSCRIPT

Page 1: Learning Analytics for C21 Dispositions & Skills

LEARNING ANALYTICS FOR C21 DISPOSITIONS & SKILLS

Simon Buckingham Shum Knowledge Media Institute, Open U. UK

http://simon.buckinghamshum.net @sbskmi

http://net.educause.edu/eliweb121

Page 2: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

2

Page 3: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

3

Page 4: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

4

Page 5: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

5

Page 6: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

6

Page 7: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

Focus of most LA effort

beginning to move

towards these more complex spaces

7

Page 8: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

Focus of most LA effort

beginning to move

towards these more complex spaces

http://solaresearch.org/OpenLearningAnalytics.pdf

8

Page 9: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge   C21  Learning  Capaci5es  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

cri5cal  for  learner  

engagement,  and  authen5c  

learning  

Focus of most LA effort

beginning to move

towards these more complex spaces

9

Page 10: Learning Analytics for C21 Dispositions & Skills

“We are preparing students for jobs that do not exist yet, that will use technologies that have not been invented yet, in order to solve problems that are not even problems yet.”

“Shift Happens” http://shifthappens.wikispaces.com

LEARNING ANALYTICS FOR THIS?

10

Page 11: Learning Analytics for C21 Dispositions & Skills

LEARNING ANALYTICS FOR THIS?

“The test of successful education is not the amount of knowledge that pupils take away from school, but their appetite to know and their capacity to learn.”

Sir Richard Livingstone, 1941

11

Page 12: Learning Analytics for C21 Dispositions & Skills

ANALYTICS FOR… C21 SKILLS?

LEARNING HOW TO LEARN? AUTHENTIC ENQUIRY?

social capital critical questioning argumentation citizenship habits of mind resilience

collaboration creativity metacognition identity readiness sensemaking

engagement motivation emotional intelligence

12 12

Page 13: Learning Analytics for C21 Dispositions & Skills

L.A. FRAMEWORK TO THINK WITH…

Discipline  knowledge   C21  Learning  Capaci5es  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

More LA effort needed

e.g.

1. Disposition Analytics

2. Discourse Analytics

Focus of most LA effort

beginning to move

towards these more complex spaces

13

Page 14: Learning Analytics for C21 Dispositions & Skills

Analytics for learning dispositions

14

Page 15: Learning Analytics for C21 Dispositions & Skills

ELLI: EFFECTIVE LIFELONG LEARNING INVENTORY WEB QUESTIONNAIRE 72 ITEMS (CHILDREN AND ADULT VERSIONS: USED IN SCHOOLS, UNIVERSITIES AND WORKPLACE)

15

Buckingham  Shum,  S.  and  Deakin  Crick,  R.  (2012).  Learning  Disposi5ons  and  Transferable  Competencies:  Pedagogy,  Modelling,  and  Learning  Analy5cs.  Accepted  to  2nd  Interna*onal  Conference  on  Learning  Analy*cs  &  Knowledge  (Vancouver,  29  Apr  –  2  May,  2012).    

Page 16: Learning Analytics for C21 Dispositions & Skills

VALIDATED AS LOADING ONTO 7 DIMENSIONS OF “LEARNING POWER”

Changing & Learning

Meaning Making

Critical Curiosity

Creativity

Learning Relationships

Strategic Awareness

Resilience

Being Stuck & Static

Data Accumulation

Passivity

Being Rule Bound

Isolation & Dependence

Being Robotic

Fragility & Dependence

16

Page 17: Learning Analytics for C21 Dispositions & Skills

ELLI GENERATES A 7-DIMENSIONAL SPIDER DIAGRAM OF HOW THE LEARNER SEES THEMSELF

17

Bristol and Open University are now embedding ELLI in learning software.

Basis  for  a  mentored-­‐discussion  on  how  the  learner  

sees  him/herself,  and  strategies  for  strengthening  

the  profile  

17

Page 18: Learning Analytics for C21 Dispositions & Skills

ADDING IMAGERY TO ELLI DIMENSIONS TO CONNECT WITH LEARNER IDENTITY

18

Page 19: Learning Analytics for C21 Dispositions & Skills

ELLI GENERATES COHORT DATA FOR EACH DIMENSION

19

Page 20: Learning Analytics for C21 Dispositions & Skills

…DRILLING DOWN ON A SPECIFIC DIMENSION

20

Page 21: Learning Analytics for C21 Dispositions & Skills

Plugin visualizes blog categories,

mirroring the ELLI spider

ENQUIRYBLOGGER: TUNING WORDPRESS AS AN ELLI-BASED LEARNING JOURNAL

21

Standard Wordpress editor

Categories from ELLI

21

Page 22: Learning Analytics for C21 Dispositions & Skills

22

ENQUIRYBLOGGER: COHORT DASHBOARD

Page 23: Learning Analytics for C21 Dispositions & Skills

LEARNINGEMERGENCE.NET more on analytics for learning to learn and authentic enquiry

23

Page 24: Learning Analytics for C21 Dispositions & Skills

Analytics for learning

conversations

24

Page 25: Learning Analytics for C21 Dispositions & Skills

DISCOURSE LEARNING ANALYTICS

Effective learning conversations display some typical characteristics which learners can and

should be helped to master

Learners’ written, online conversations can be analysed computationally for patterns signifying

weaker and stronger forms of contribution

25

Page 26: Learning Analytics for C21 Dispositions & Skills

SOCIO-CULTURAL DISCOURSE ANALYSIS (MERCER ET AL, OU)

•  Disputational talk, characterised by disagreement and individualised decision making.

•  Cumulative talk, in which speakers build positively but uncritically on what the others have said.

•  Exploratory talk, in which partners engage critically but constructively with each other's ideas.

Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.

26

Page 27: Learning Analytics for C21 Dispositions & Skills

•  Exploratory talk, in which partners engage critically but constructively with each other's ideas. •  Statements and suggestions are offered for joint consideration.

•  These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered.

•  Partners all actively participate and opinions are sought and considered before decisions are jointly made.

•  Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk.

Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.

SOCIO-CULTURAL DISCOURSE ANALYSIS (MERCER ET AL, OU)

27

Page 28: Learning Analytics for C21 Dispositions & Skills

ANALYTICS FOR IDENTIFYING EXPLORATORY TALK

Elluminate sessions can be very long – lasting for hours or even covering days of a conference

It would be useful if we could identify where quality learning conversations seem to be taking place, so we can recommend those sessions, and not have to sit through online chat about virtual biscuits

Ferguson, R. and Buckingham Shum, S. Learning analytics to identify exploratory dialogue within synchronous text chat. 1st International Conference on Learning Analytics & Knowledge (Banff, Canada, 27 Mar-1 Apr, 2011)

28

Page 29: Learning Analytics for C21 Dispositions & Skills

De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st International Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011)

KMI’S COHERE: A WEB DELIBERATION PLATFORM ENABLING SEMANTIC SOCIAL NETWORK AND DISCOURSE NETWORK ANALYTICS

Rebecca is playing the role of broker,

connecting 2 peers’ contributions in

meaningful ways

29

Page 30: Learning Analytics for C21 Dispositions & Skills

DISCOURSE ANALYSIS

BACKGROUND KNOWLEDGE:

Recent studies indicate …

… the previously proposed …

… is universally accepted ...

NOVELTY:  

...  new  insights  provide  direct  evidence  ...  

...  we  suggest  a  new  ...  approach  ...  

...  results  define  a  novel  role  ...  

 

 

 

 

 

 

OPEN  QUESTION:  …  li[le  is  known  …  

…  role  …  has  been  elusive  

Current  data  is  insufficient  …  

GENERALIZING:  ...  emerging  as  a  promising  approach    

Our  understanding  ...  has  grown  exponen2ally  ...  

...  growing  recogni2on  of  the    

importance  ...  

CONRASTING  IDEAS:  …  unorthodox  view  resolves  …  paradoxes  …  

In  contrast  with  previous  hypotheses  ...  

...  inconsistent  with  past  findings  ...  

SIGNIFICANCE:  studies  ...  have  provided  important  advances  

Knowledge  ...  is  crucial  for  ...  understanding  

valuable  informa2on  ...  from  studies  

SURPRISE:  We  have  recently  observed  ...  surprisingly  

We  have  iden2fied  ...  unusual  

The  recent  discovery  ...  suggests  intriguing  roles  

SUMMARIZING:  The  goal  of  this  study  ...  

Here,  we  show  ...  

Altogether,  our  results  ...  indicate  

Xerox’s  parser  can  detect  the  presence  of  ‘knowledge-­‐level’  moves  in  text:  

Ágnes Sándor & OLnet Project: http://olnet.org/node/512

De Liddo, A., Sándor, Á. and Buckingham Shum, S. (In Press). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work Journal

30

Page 31: Learning Analytics for C21 Dispositions & Skills

NEXT STEPS

SOCIAL LEARNING ANALYTICS: Develop this framework to integrate social, discourse, disposition and other process-centric analytics

DISPOSITION ANALYTICS: Extend the capabilities of the ELLI ‘learning power’ platform using real-time analytics data from online learner activity DISCOURSE ANALYTICS: human+machine annotation of written discourse and argument maps

31

Page 32: Learning Analytics for C21 Dispositions & Skills

IN MORE DETAIL… Social  Learning  Analy5cs  §  Buckingham  Shum,  S.  and  Ferguson,  R.  (2011).  Social  Learning  Analy5cs.  Available  as:  Technical  Report  KMI-­‐11-­‐01,  Knowledge  

Media  Ins2tute,  The  Open  University,  UK.  h[p://kmi.open.ac.uk/publica2ons/techreport/kmi-­‐11-­‐01  

 Discourse  Analy5cs  §  De  Liddo,  A.,  Buckingham  Shum,  S.,  Quinto,  I.,  Bachler,  M.  and  Cannavacciuolo,  L.  (2011).  Discourse-­‐Centric  Learning  Analy5cs.  

1st  Interna*onal  Conference  on  Learning  Analy*cs  &  Knowledge  (Banff,  27  Mar-­‐1  Apr,  2011).  Eprint:  h[p://oro.open.ac.uk/25829  §  Ferguson,  R.  and  Buckingham  Shum,  S.  (2011).  Learning  Analy5cs  to  Iden5fy  Exploratory  Dialogue  Within  Synchronous  Text  

Chat.  1st  Interna*onal  Conference  on  Learning  Analy*cs  &  Knowledge  (Banff,  Canada,  27  Mar-­‐1  Apr,  2011).  Eprint:  h[p://oro.open.ac.uk/28955  

§  De  Liddo,  A.,  Sándor,  Á.  and  Buckingham  Shum,  S.  (2012,  In  Press).  Contested  Collec5ve  Intelligence:  Ra5onale,  Technologies,  and  a  Human-­‐Machine  Annota5on  Study.  Computer  Supported  Coopera*ve  Work.  DOI:  10.1007/s10606-­‐011-­‐9155-­‐x.  h[p://www.springerlink.com/content/23n1408l9g06v062    

 Disposi5on  Analy5cs  §  Ferguson,  R.,  Buckingham  Shum,  S.  and  Deakin  Crick,  R.  (2011).  EnquiryBlogger:  Using  Widgets  to  Support  Awareness  and  

Reflec5on  in  a  PLE  SeQng.  1st  Workshop  on  Awareness  and  Reflec*on  in  Personal  Learning  Environments,  PLE  Conference  2011,  11-­‐13  July  2011,  Southampton,  UK.  Eprint:  h[p://oro.open.ac.uk/30598  

§  Buckingham  Shum,  S.  and  Deakin  Crick,  R  (2012).  Learning  Disposi5ons  and  Transferable  Competencies:  Pedagogy,  Modelling,  and  Learning  Analy5cs.  Accepted  to  2nd  Interna*onal  Conference  on  Learning  Analy*cs  &  Knowledge  (Vancouver,  29  Apr  –  2  May,  2012).  Working  drap  under  revision:  h[p://projects.kmi.open.ac.uk/hyperdiscourse/docs/SBS-­‐RDC-­‐review.pdf  

Page 33: Learning Analytics for C21 Dispositions & Skills

SUMMARY

Discipline  knowledge   C21  Learning  Capaci5es  

Educator  owns  and  manages  a  single  dataset  

Educator  owns  and  manages  mul2ple  datasets  

Learners  add  their  own  datasets  

Hybrid  closed  +  open  datasets  

Hybrid  closed  +  open  analy2cs  

More LA effort needed

We need analytics tuned to generic capacities which equip learners for novel challenges

Focus of most LA effort

mastery of core knowledge and

skills in training is vital, but no longer

sufficient

33