Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Affective recommender systems: the role of emotions in recommender systems

Marko Tkalčič, Andrej Košir, Jurij Tasič

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Presentation overview Introduction From data-centric to user-centric Overview of emotions Proposed framework Conclusions

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Introduction It‘s about music, not about recommenders (Eric Bieschke,

Pandora)– Re: It‘s about us, the users

RecSys help us make DECISIONS on content items Bounded rationality theory [Daniel Kahnemann (nobel

prize for economics 2002)]

Decision making = rational + emotional

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

From data-centric to user-centric Early RecSys:

– ratingPredictions(data-centric descriptors)

= descriptors that are available (e.g. from IMDB)

» Genre» Actors» Performers» Timestamps

– Typical modeling:

User ui likes the genre gj under the ck circumstances XX%

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

From data-centric to user-centric In recent years

– shift towards user-centric descriptors

= descriptors that are suspected to carry information

but are NOT available » Emotional responses» Personality

Arapakis, Gonzalez, Hanjalić, Nunes, Tkalčič CAMRA 2010 contest Overlapping with the affective computing community

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

From data-centric to user-centric The data-centric approach is still rooted in the research

community:– It‘s about music, not about recommenders

The community is problem-solving oriented– The existing datasets are real, why building synthetic ones?

Solving existing problems is only a part of research ...

... the other part is generating new knowledge (on how the world works) ...

... which in turn generates new problems ...

... which in turn opens new publishing possibilities

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Overview of emotions Emotions are complex human experiences Strong physiological background Evolutionary based Several definitions We take with simple models, easy to incorporate in

computers:– Basic emotions– Dimensional model– Circumplex model

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Basic emotions Discrete classes model Different sets Darwin: Expression of emotions in man and animal Ekman definition (6 + neutral):

– Happiness– Anger– Fear– Sadness– Disgust– Surprise

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Dimensional model Three dimensions

– Valence– Arousal– Dominance

Each emotive state is a point in the VAD space

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Circumplex model Maps basic emotions dimensional model

Arousal

Valence

high

negative positive

low

neutral

sadness

fear

disgust

surprise

joyanger

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

How to detect emotions? Explicit vs. Implicit Explicit

– Questionnaires (SAM) Implicit:

– Work done in the affective computing community– Different modalities (sources):

• Facial actions (video)• Physiological signals ( GSR, EEG)• Voice• Posture• ...

– ML techniques• Classification (basic emotions)• Regression (dimensional model)

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework Problem statement:

– Research is done in a scattered fashion– Researchers do not benefit from each other‘s work

Goal:– Researchers to identify their position– To benefit from each other‘s work– To establish affective recommender system as a (sub)field?

References are in the paper

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework - 1

Content application

Give conten

t

time

Entry stage Consumption stage Exit stage

Give recommendati

ons

choice

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework - 2

Content application

Entry mood

Detect entrymood

Give conten

t

Exit mood

time

Entry stage Consumption stage Exit stage

Give recommendati

ons

choice

• Context• Decision making• Influence• Diversification

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework - 3

Content application

Entry mood

Detect entrymood

Give conten

t

Content-induced affective state

Observe user

time

Entry stage Consumption stage Exit stage

Give recommendati

ons

choice

• Affective tagging• Affective user profiles

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework - 3

Content application

Entry mood

Detect entrymood

Give conten

t

Content-induced affective state Exit mood

Observe user

time

Entry stage Consumption stage Exit stage

Give recommendati

ons

choice

Detect exit

mood

• Implicit feedback• Evaluation metrics

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

The proposed framework - 3

Content application

Entry mood

Detect entrymood

Give conten

t

Content-induced affective state Exit mood

Observe user

time

Entry stage Consumption stage Exit stage

Give recommendati

ons

choice

Detect exit

mood

• Implicit feedback• Evaluation metrics

• Affective tagging• Affective user profiles

• Context• Decision making• Influence• Diversification

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Conclusions Research is shifting towards the use of emotions in recsys Emotions have shown to improve recommenders‘

performance Research is sparse and not self-aware The proposed framework should put things in place

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Questions Q1: does the framework reflect your view of emotions and

recsys? Q2: did we miss something? Q3: emotions related to diversity, user-centric evaluation? Q4: any other issue?

Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..

Notes