a hybrid peer recommender system for a online community teachers

A Hybrid Peer Recommender System for a Online Community Teachers

Cristian Miranda*, Julio Guerra**, Denis Parra**, Eliana Scheihing*Southern University of Chile*

University of Pittsburgh**SRS: 3rd International Workshop on Social Recommender Systems ~ at UMAP 2012

Outline

• Introduction to Kelluwen Project• What is Being Recommended?• Recommendation Model• Evaluation/Results• Conclusions/Future Work

Geog. Context: Southern Chile

In this part of South America

Kelluwen Project

• Kelluwen is a mapudungun word that means “Group of People” that we used as “Collaborative Work”.

• The Kelluwen project address deficits in socio-communicate skills of vulnerable students in ages 12~17 from

• Technology involved: The creation of a community of students, teachers and researchers supported by web 2.0 tools.

What is being recommended?

• Our RecSys recommends “comments and suggestions” made by teachers when they conduct these educational activities.

• Educational Activities. These are activities performed in class by students. They report them and comment them in the Kelluwen platform.

• Teachers that are guiding some activity for the first time can have a big support if they receive comments from teachers (peers) that have already conducted that activity .

What is being recommended?

Educational Activity (materials, durations, link to comments, status)

Recommendations: comments & suggestions made by other teachers that already guided the same activity

… MAX 3 students per group…

… ppt too long, might skip some slides…

Recommendation Model (1/3)

• Given:– C: the set of all users– S: the set of all “recommendable” items– u: the utility function ( how useful is an item to a user)– profile of user c, a vector of |p| attributes.– profile of ítem s, a vector of |r| attributes.– R = () rating matrix, where is the rating given by user c to item s– N = () reply matrix, where is the amount of replies that the user i does to

messages sent by user j.

• The utility of an item b for a user a is then given by

= u1: peers’ similarity u2: item value

Recommendation Model (2/3)

• Peers’ similarity considers (u1)Variable ID Values

School Socio-economical Level NS

Low: 1Mean Low: 2

Mean: 3Mean High: 4

High: 5

School Quality of TIC Infrastructure

CT Sum of three values

Size of School Locality TLLower than 10.000 habitants: 1

Between 10.000 and 100.000 habitants: 2Greater than 100.000 bahitants: 3

Mean Number of Students per Classroom

N Mean number of stufents by classroom for each teacher

Recommendation Model (3/3)• Items assessed based on creator’s value (u2):

Variable ID Values

Number of didactical designs applied by teacher t (sending the

suggestion s)

Number of didactical designs applied by teacher t (sending the suggestions s), normalized to the

[0,1] interval

Activity evaluation from the teacher sending the suggestion s

Very well: 0

Well: 1

Bad: 1

Very Bad: 0

Message Nature Activity comment: 0

Suggestion: 1

Number of rating of all messages from teacher t (sending the

suggestion s)

Number of rating of all messages from teacher t (sending the suggestion s), normalized to the [0,1]

interval

Number of ratings of suggestion s Number of rating of suggestion s, normalized to the [0,1] interval

Evaluation(1/2) nDCG• Evaluation done with nDCG (normalized discounted cummulative gain)• Phase 1: 29 teachers provided comments and suggestions to their

educational activities inside the Kelluwen system• Phase 2: # recommendations generated between Oct/Nov 2011: 314• Phase 2: # recommendations that were provided feedback: 96• Here there’s a plot with nDCG of 39 recommendation sets (6 people).

User satisfaction survey

¿Did I consider the recommendation while

executing the educational activities?

70%Used the

recommendations in their educational

activities

¿Were the recommendations useful to develop my activities?

68%Thought that the

recommendations were useful during the activity execution

Totally agree

Disagree

Agree

Totally agree

Conclusions

• We introduced a RecSys that recommends novel items (comments of teachers to educational activities) in a novel domain (a community portal for students, teachers and researchers)

• The RecSys had a good level of acceptance: 68% of the teachers that considered the recommendations found them useful in their educational activities.

Future work

• Incorporate user engagement techniques to obtain more feedback of the utility of recommendations.

• To Develop a “recommendation history”: Users would be able to see which recommendations they have received and how they evaluated them.

• Include new metrics to evaluate the recommender system.

Questions? Thanks

Backup slides

Sensibility Analysis• Best combination of k1 (0.8) and k2 (0.2) values to

combine u1 (peer similarity) and u2 (item and item-creator value) ~ best nDCG

• Encuesta de Satisfacción de Usuarios

¿Aprecio las recomendaciones de mis pares como un mecanismo de apoyo para el desarrollo de las actividades?

70%El sistema adaptativo de

recomendación de pares es un mecanismo de apoyo a sus

actividades docentes

Resultados