learning analytics at large: the lifelong learning network of 160, 000 european teachers

TeLLNet

EC TEL 2011EC-TEL 2011

Learning Analytics at Large: th Lif l L i N t kthe Lifelong Learning Networkof 160, 000 European Teachers

Ergang Song, Zinayida Petrushyna, Yiwei Cao, and Ralf KlammaInformation Systems and Databases, RWTH Aachen University

Palermo, ItalySeptember 23 2011

Lehrstuhl Informatik 5(Informationssysteme)

Prof. Dr. M. JarkeI5-SPCK-0911-1

September 23, 2011

TeLLNet

MotivationsMotivations How to support lifelong learning (LLL)?

– New means for LLL with rapid development of ICT – Competence assessment methods for LLL in demand

S lf it i f LLL d d

(Meta-) Competence management

– Self-monitoring for LLL needed– Still lack of large data sets– Tools are needed instead of a concept

Self-monitoring

Tools are needed instead of a concept Case study: eTwinning Network

– Continuous professional development for teachers

Learning analyticsfor lifelong learning

p p– Aiming to promote collaborations among schools– Competence gap to recognize and to bridge– Meta-competence

Learning analytics is neededVi l l ti f lf it i



– Visual analytics for self-monitoring– Multiple levels (individual, community, and network)

TeLLNet

Learning AnalyticsLearning AnalyticsLearning analytics is the measurement, collection, analysis and reporting of data about learners and their

Visual analytics

ea ning analytics s e easu e e , co ec o , a a ys s a d epo g o da a abou ea e s a d econtexts, for purposes of understanding and optimizing learning and the environments in which it occurs. (Siemens, 2011)

Visual analytics– It is easier for teachers to understand visualization than statistics

(Breuer et al., 2009)

Data analysis Learning context analysis Learning context analysis

(Cao et al., 2010) Network analysis Network analysis The EC-TEL communities

as an e ample Lehrstuhl Informatik 5(Informationssysteme)


as an example (Pham et al., 2011)

TeLLNet Learning Analytics Contributions to EC TEL so farto EC-TEL so far

2006 - Klamma, Spaniol, Cao, Jarke: Pattern-Based Cross Media Social Network Analysis for Technology Enhanced Learning in Europe

– Media Bases as research tools for TEL – SNA as research methodology for TELSNA as research methodology for TEL

2008 - Petrushyna, Klamma: No Guru, No Method, No Teacher: Self-Observation and Self-Modelling of E-Learning Communities

– In-depth Analysis of a Media Base for TEL– Combination of SNA and content-based measures

2009 - Breuer, Klamma, Cao, Vuorikari: Social Network Analysis of 45.000 Schools: A 2009 Breuer, Klamma, Cao, Vuorikari: Social Network Analysis of 45.000 Schools: A Case Study of Technology Enhanced Learning in Europe

– eTwinning database of European cooperation between schoolsSNA as a tool for teachers– SNA as a tool for teachers

– Visualization and Usability 2010 – Petrushyna: Self-modeling and Self-reflection of E-learning communities



(Doctoral Consortium)

TeLLNet TeLLNet ProjectTeachers' Lifelong Learning NetworksTeachers' Lifelong Learning Networks

T i i T i S T LLN teTwinning

• Founded in 2005• Coordinated by European S h l t

TwinSpace

• Since 2008?• Subject to eTwinning

W b 2 0 f T i i

TeLLNet

•3-year-project within the EU Lifelong Learning Programme (2009-2012)Schoolnet

• Internet platform with workspace and (communication) tools

P j t t b d b

• Web 2.0 for eTwinning• Blogs• Quality labels• Desktop tools

Programme (2009-2012)•Project obejctives: Competence development for teachers in learning networks with social network • Projects must be done by

two or more partners from different countries• Offline activities: Workshops across Europe

networks with social network analysis and scenario building based on eTwinning• Partners• European SchoolnetWorkshops across Europe European Schoolnet• RWTH Aachen University• Open University of the Netherlands• Institute for Prospective Institute for Prospective Technological Studies (IPTS) –Joint Research Centre of the European Commission



TeLLNet

Competence and Meta-Competence

Author DefinitionM Cl ll d Th k l d kill i i d

Developed in lots of areas: Human resource management McClelland

(1973)The knowledge, skills, traits, attitudes, self‐concepts, values, or motives directly related to job performance or important life outcomes and shown to differentiate b i d

Human resource management, vocational education ...

Different definitions in literaturesbetween superior and average performers.

Brown and McCartney ( )

A meta‐competence is the overarching ability under which competence shelters.

b h h h d b l

Common points A set of human characteristics

(knowledge skills abilities ) (1995) It embraces the higher order abilities which have to do with being able to learn, adapt, anticipate and create. Meta‐competences are a prerequisite for the d l f h

(knowledge, skills, abilities...) The performances to enhance Categorized into different types

development of capacities such as judgment, intuition and acumen upon which competences are based and without which competences cannot fl i h

Assessment methods Explicit assessment (questionnaire, test) Implicit assessment flourish

Cheethamand Chivers(2005)

Meta‐competence is the competence that is beyond other competences, and which enables individuals to monitor and/or d l h

Implicit assessment Events to monitor Algorithms to design

C t t tLehrstuhl Informatik 5(Informationssysteme)


develop other competences Competence to computer Automated executable without participation

of questionnaires

TeLLNet

Teachers’ Competence in eTwinning eTwinning Network

Teachers’ Competence in eTwinning Our meta-competenceg

(as of the end of 2010) Teacher Amount %

Sum 135,351 100%

– Higher order competence– Competence to monitor and

develop other competencesProject with projects 26,365 19.4%

with QLs 2,093 1.55%

with EQLs 616 0.46%

develop other competences– Depends on context– Ability to self-monitoring is

with prizes 655 0.48%

Wall post Wall posts sent 10,104 7.47%

Wall posts i d

18,986 14.03%

y gmeta-competence in the contextof LLL Meta

competencereceived

Blog Posts written 4,508 3.33%

Post comments written

441 0.33%

Self- monitoringability

Languaget

Wall-post writing bilitet

ence

e

Post comments received

727 0.54%

Comment Project comments itt

1,531 1.13%

competence

Project performance

ability

Blog writing ability

onal

com

pe

ompe

tenc

e



written

Prize comments written

354 0.26% Project efficiencyetc.

Comment writing ability, etc.

Prof

essi

o

Soci

alco

TeLLNet Data Analysis: Large-Scale Data Set of eTwinningof eTwinning

New tables generated Table Name Records Error

New data for Web 2.0 – Blogs (TwinBlogPost)

Table Name Records number

Error Rate

Affectation 99886 0.00002Institution 71988 0– Comments (TwinBlogComment,

PrizeComment)– Labels (QualityLabel)

Institution 71988 0MyContact 464780 0.000037Prize 892 0.0045PrizeComment 441 0 0091Labels (QualityLabel)

– Tagging, etc. – ProjectMember

PrizeComment 441 0.0091Project 17392 0ProjectGuestBook 3460 0.009ProjectMember 66145 0 0045

– ProjectGuestBook Data cleaning

ProjectMember 66145 0.0045QualityLabel 4886 0Teacher 133693 0TeacherWall 34900 0 00014

Data dumps– 1st Dump (June, 2010)

2 ( 2010)

TeacherWall 34900 0.00014TwinBlog 15235 0.00013TwinBlogComment 2950 0.2783T i Bl P t 31163 0 00064



– 2nd Dump (November, 2010)– 3nd Dump (May, 2011)

TwinBlogPost 31163 0.00064Sum 947811 0.0013

TeLLNet

Competence AssessmentCompetence Assessment Indicator model in Entity Relationship DiagramIndicator model in Entity Relationship Diagram

Performance Indicator

Ff

f fNormwI )(



f

TeLLNet System Architecture ofPrototype CAfePrototype CAfe



TeLLNet Self-monitoring of Teacher Network in CAfein CAfe

Target users– European teachers (teachers‘ workshops)– Administrators & policy-makers



TeLLNet Self-Monitoring of Competence ManagementManagement



TeLLNet Self-Monitoring of Competence Management

Community level ->

Management

Teacher level



TeLLNetDynamic Network Analysis in Progress

The Development Model (Pham et al. 2011 )

Applied to collaboration (project, email) and social media(blog) networksLehrstuhl Informatik 5(Informationssysteme)


pp ed o co abo a o (p ojec , e a ) a d soc a ed a(b og) e o s- To detect the development pattern of project partner community- To compare different networks

TeLLNet Learning Analytics: EC-TEL Community among TEL CommunitiesCommunity among TEL Communities ICALT, ICWL, EC-TEL, IST, AIED (Pham, Derntl and Klamma 2011), , , , ( , )

103

104

r of e

dges

(a) Densification law

ICALT: 0.34889*x1.19760.9

0.95

1

g co

effic

ient

(b) Clustering Coefficient

ICALT

101 102 103 104101

102

Num

ber

Number of nodes

ICWL: 1.1149*x1.0544

ECTEL: 0.40338*x 1.2415

ITS: 0.15818*x 1.3817

AIED: 1.0128*x1.1197

1 2 3 4 5 6 7 8 90.75

0.8

0.85

Clu

ster

ing

Age

ICALTICWLECTELITSAIED

0 08 (c) Maximum Betweenness0 7

(d) Largest connected component

0.04

0.06

0.08

mum

bet

wee

nnes

s

ICALTICWLECTELITSAIED

0 2

0.3

0.4

0.5

0.6

0.7

conn

ecte

d co

mpo

nent ICALT

ICWLECTELITSAIED

1 2 3 4 5 6 7 8 90

0.02

Max

im

Age

1 2 3 4 5 6 7 8 9

0

0.1

0.2

Larg

est c

Age

20(e) Diameter

ICALT

8(f) Average Path Length

ICALT

5

10

15

Dia

met

er

ICWLECTELITSAIED

2

4

6

Ave

rage

pat

h le

ngth ICWL

ECTELITSAIED



1 2 3 4 5 6 7 8 90

Age

1 2 3 4 5 6 7 8 9

0

Age

TeLLNet Node Level Analysis: Structural Holes, Closure and Social CapitalClosure and Social Capital

Structural holes (Burt, 1992)- Nodes are positioned at the interface

between groups (gatekeepers, e.g. node B)Informational nodes: access to information - Informational nodes: access to information from different parts of networks

- Novel ideas by combining information from different groups

- Control the communication between groups

Cl Closure: - Nodes with high clustering coefficient (e.g. node A): embedded in tightly-knit

groupsgroups- More trust and security within coherent communities

Social capital (Coleman, 1990)Lehrstuhl Informatik 5(Informationssysteme)


p ( , )- Individuals and groups deriving benefits from social relationships- Network structural property: can be either structural hole or closure

TeLLNet

ConclusionsConclusions SNA & visualization as tools for competence development in SNA & visualization as tools for competence development in

learning networks– Competence assessment is still limited in performance indicationCompetence assessment is still limited in performance indication

eTwinning case studyComplex data management issues– Complex data management issues

– Visual complexity of networks vs. teachers’ competence Experimenting with web based tools– Experimenting with web-based tools

Learning analytics is the solution for large scale network

Data analysis

Visual analytics

Contextanalytics

Network analysis

Learning analytics



analysis analytics analytics analysis analytics

TeLLNet Learning Analytics for Conference ParticipantsParticipants

At academic conferences/workshopsWhi h t lk t tt d?– Which talk to attend?

– To whom to talk to? CAMRS – Mobile Context-aware Recom-

mendation Services for Conference Participants

???

Room 342: workshopAuditorium: keynote

??


Prof. Dr. M. JarkeI5-SPCK-0911-18 Room 204: paper session Hall: poster session Room 048: round table

learning analytics at large: the lifelong learning network of 160, 000 european teachers

Technology

lifelong learning lll

tellnet competence

tellnet teachers competence

network i5spck

learning desktop toolsnetworks

jarke i5spck

etwinning etwinning

d e contexts