Intelligent Interfaces for Open Social Student Modeling

Peter BrusilovskySharon Hsiao,Tomek Loboda, Julio

Guerra, Jordan Barria-PinedaPAWS Lab,

University of Pittsburgh

Overview

• Goals– Why we are doing it?

• Open Student Models– From ANS to OSM

• Open Social Student Models– QuizMap, Progressor, Progressor+

• Mastery Grids– Topic-level OSLM in Mastery Grids – Concept-level OSLM in Mastery Grids

From Goals to Technologies

• Technologies–Adaptive Navigation Support–Open Student Models–Open Social Student Modeling

• Why to use it– Increase user performance– Increase motivation and retention

Targets Engaged

• Adaptive Navigation Support• Topic-based Adaptation• Open Student Modeling• Social Navigation and Comparison• Open Social Student Modeling• Social Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation

Adaptive Link Annotation: InterBook

1. Concept role2. Current concept

state

3. Current section state4. Linked sections state

4

3

2

1

√

Questions of the current quiz, served by QuizPACK

List of annotated links to all quizzes available for a student in the current course

Refresh and help icons

QuizGuide = Topic-Based ANS

Topic-Based Adaptation

Concept A

ConceptB

ConceptC

Each topic is associated with a number of educational activities to learn about this topic

Each activity classified under 1 topic

QuizGuide: Adaptive Annotations• Target-arrow

abstraction:– Number of arrows –

level of knowledge for the specific topic (from 0 to 3). Individual, event-based adaptation.

– Color Intensity – learning goal (current, prerequisite for current, not-relevant, not-ready). Group, time-based adaptation.

Topic–quiz organization:

QuizGuide: Success Rate

QuizGuide: Motivation

Within the same class QuizGuide session were much longer than QuizPACK sessions: 24 vs. 14 question attempts at average.

Average Knowledge Gain for the class rose from 5.1 to 6.5

• Topic-Based interface organization is familiar, matches the course organization, and provides a compromise between too-much and too-little

• Two-way adaptive navigation support guides to the right topic

• Open student model provides clear overview of the progress

Topic-Based ANS: Success Recipes

Targets Engaged

Adaptive Navigation SupportTopic-based AdaptationOpen Student Modeling• Social Navigation and Comparison• Open Social Student Modeling• Social Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation

Social Navigation

• Concept-based and topic-based navigation support work well to increase success and motivation

• Knowledge-based approaches require some knowledge engineering – concept/topic models, prerequisites, time schedule

• In our past work we learned that social navigation – “wisdom” extracted from the work of a community of learners – might replace knowledge-based guidance

• Social wisdom vs. knowledge engineering

Knowledge Sea – Social Navigation

Farzan, R. and Brusilovsky, P. (2005) Social navigation support through annotation-based group modeling. 10th International User Modeling Conference Lecture Notes in Artificial Intelligence, vol. 3538. Berlin: Springer Verlag, pp. 463-472.

Open Social Student Modeling• Motivation

– Combine benefits of Open Student Models with social navigation and social comparisons

• Key steps– Assume simple topic-based design – Show topic- and content- level knowledge progress of a student

in contrast to the progress of the class– The design should guide students to most appropriate topics

and content• Main challenge

– How to design the interface to show student and class progress over topics?

– We went through several attempts…

16

QuizMap

17

Parallel Introspective Views

18

Progressor

• Topic organization should follow the natural progress or topics in the course

• Clear comparison between “me” and “group”

• Ability to compare with individual peers, not only the group

• Privacy management

OSLM: Success Recipes

The Value of OSLM

Progres

sor

QuizJE

T+IV

QuizJE

T+Portal

Java

Guide

0

50

100

150

200

250205.73

113.05

80.81

125.5

Attempts

AttemptsProg

resso

r

QuizJE

T+IV

QuizJE

T+Portal

Java

Guide

0.00%

20.00%

40.00%

60.00%

80.00%68.39% 71.35%

42.63%

58.31%

Success Rate

Success Rate

The Mechanism of Social Guidancestronger students left the traces for weaker ones to follow

21Time

Topics Strong Weak

The Secret

Targets Engaged

Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress Visualization• Multiple Content Types• Open Source• Concept-Based Adaptation

24

Progressor+ OSLM for two types of content• macro- and micro- comparisons (group or peers)

Students Spent More Time in Progressor+

Quiz =: 5 hours Example : 5 hours 20 mins 25

QuizJET JavaGuide Progressor Progressor+0

50

100

150

200

250

300

350

60.04

150.19

224.7

296.9

69.52

121.23110.66

321.1

Total time spent (minutes)

QuizExample

26

Students Achieved Higher Success Rate

QuizJET JavaGuide Progressor Progressor+0.00%

20.00%

40.00%

60.00%

80.00%

42.63%

58.31%

68.39% 71.20%

Success Rate

p<.01

27

Mastery Grids

28

Mastery Grids: Content Access

29

Mastery Grids: Group and Peer OSLM

MG flexibility

• Parameters to set the visualization:– show hide toolbar or any of its elements– set the (sub) groups: top N, other sub groups– preset values (for example load individual

view by default)– enable/disable recommendation

• Parameters can be specified by group or by user

31

Mastery Grids Engage More

_x00

08_B

aseli

ne

_x00

05_ O

SSM0

204060

Problems Solved

_x00

08_B

aseli

ne

_x00

05_ O

SSM0

10203040

Examples Viewed

And social comparison (OSSM) features strengthen the effect

32

OSSM Engages Persistently

PART 1 PART 210

15

20

25

30Activity by Session

OSM OSSM

Step-wise regression: being in the OSSM group means an increase of about 30 activities, as compared to being in the OSM group.

33

OSSM Group Becomes More Effective• Instructional Effectiveness (Paas & Van

Merriënboer, 1993)Relates performance in problems and time spent

PART 1 PART 2-0.4

-0.2

0

0.2

Effectiveness Score

OSSM

OSM

Targets Engaged

Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress VisualizationMultiple Content TypesOpen Source• Concept-Based Adaptation

Concept-Based Student Modeling

Example 2 Example M

Example 1

Problem 1

Problem 2 Problem K

Concept 1

Concept 2

Concept 3

Concept 4

Concept 5

Concept N

Examples

Problems

Concepts

These cells (first row) shows your progress in the topics of the course

This bar chart shows your progress in the concepts of the course

Each topic has several concepts associated to it. Mouseover a topic to highlight its concepts

This bar chart (upside-down) shows the average progress of the rest of the class on the concepts

Middle row shows the difference between your progress and the progress of the group

Third row shows the progress of the group in blue

Concept level OSLM

An overlayed pane opens indicating which topic you are inspecting (in this case the topic "Comparisons")

The concepts within the selected topic are highlighted

Mousing over this activity

Concepts in the selected activity are highlighted

This gauge estimates the how much you can learn in the selected activity. You will probably learn more in activities that have more new concepts

See more in IUI 2017 Demo! "Concept-Level Knowledge Visualization for Supporting Self-Regulated Learning"

Targets Engaged

Adaptive Navigation SupportTopic-based AdaptationOpen Student ModelingSocial Navigation and ComparisonOpen Social Student ModelingSocial Educational Progress VisualizationMultiple Content TypesOpen SourceConcept-Based Adaptation

Acknowledgements• Joint work with

– Sergey Sosnovsky– Sharon Hsiao– Julio Guerra– Jordan Barria-Pineda

• NSF Grants– EHR 0310576– IIS 0426021– CAREER 0447083

• ADL “PAL” grant to build Mastery Grids

Read About It!• Brusilovsky, P., Sosnovsky, S., and Yudelson, M. (2009) Addictive links: The

motivational value of adaptive link annotation. New Review of Hypermedia and Multimedia 15 (1), 97-118.

• Brusilovsky, P., Hsiao, I.-H., and Folajimi, Y. (2011) QuizMap: Open Social Student Modeling and Adaptive Navigation Support with TreeMaps. Proceedings of 6th European Conference on Technology Enhanced Learning (ECTEL 2011), pp. 71-82

• Hsiao, I.-H., Bakalov, F., Brusilovsky, P., and König-Ries, B. (2013) Progressor: social navigation support through open social student modeling. New Review of Hypermedia and Multimedia

• Brusilovsky, P., Somyurek, S., Guerra, J., Hosseini, R., Zadorozhny, V., and Durlach, P. (2016) Open Social Student Modeling for Personalized Learning. IEEE Transactions on Emerging Topics in Computing 4 (3), 450-461.

• Jordan, B.-P., Guerra, J., Huang, Y., and Brusilovsky, P. (2017) Concept-Level Knowledge Visualization for Supporting Self-Regulated Learning. In: Proceedings of Companion of the 22nd International Conference on Intelligent User Interfaces (IUI '17), Limassol, Cyprus, ACM, pp. 141-144 also available at https://doi.org/10.1145/3030024.3038262.