Personalizing Media
Interaction on the Web
Lora Aroyo
© Kierstin Shaylee
1Wednesday, October 3, 12
“interface is the message”
Aaron Koblin: Artfully visualizing our humanity, TED Talk, 2011
Lora Aroyo twitter: @laroyo
2Wednesday, October 3, 12
it’s complicated
Lora Aroyo twitter: @laroyo
3Wednesday, October 3, 12
HCI challengessemanticsuser data
interaction
personalization Lora Aroyo twitter: @laroyo
Flickr: elkabong 4Wednesday, October 3, 12
semantic HCI challenge
• explicit semantics (often from open sources, e.g. LOD) used for system decisions and results
• use facetted presentation, searching and browsing of information
• use typically classifications, typologies or other structures of concepts
• integrate & aggregate data from different sources
Lora Aroyo twitter: @laroyo
5Wednesday, October 3, 12
6Wednesday, October 3, 12
6Wednesday, October 3, 12
6Wednesday, October 3, 12
user data HCI challenge
Lora Aroyo twitter: @laroyo
7Wednesday, October 3, 12
• search, e.g. keyword, faceted
• browse, story lines, narratives through collections
• annotations of multimedia, e.g. (collaborative) tagging, professional annotation of text, images and video, tagging games
• explanations, hints, user feedback, e.g. explanation of recommendation results, explanation of autocompletion suggestions
interaction HCI challenge
Lora Aroyo twitter: @laroyo
8Wednesday, October 3, 12
interaction with semantics
Lora Aroyo twitter: @laroyo
9Wednesday, October 3, 12
© BBC MMVIIIhttp://twitpic.com/il1w/full
10Wednesday, October 3, 12
http://www.bbc.co.uk/programmes/b00c06n2.rdf
Lora Aroyo twitter: @laroyo11Wednesday, October 3, 12
• data challenges multiply with Web openness
• social recommendations are preferred
• it is not about (single) algorithms any more
• it is about the metrics
personalization challenge
Lora Aroyo twitter: @laroyo
12Wednesday, October 3, 12
13Wednesday, October 3, 12
13Wednesday, October 3, 12
13Wednesday, October 3, 12
13Wednesday, October 3, 12
13Wednesday, October 3, 12
13Wednesday, October 3, 12
14Wednesday, October 3, 12
14Wednesday, October 3, 12
14Wednesday, October 3, 12
14Wednesday, October 3, 12
14Wednesday, October 3, 12
14Wednesday, October 3, 12
15Wednesday, October 3, 12
15Wednesday, October 3, 12
15Wednesday, October 3, 12
15Wednesday, October 3, 12
applications
16Wednesday, October 3, 12
© Ugli. on the Walls of BBC London
personalized TV
17Wednesday, October 3, 12
User Perspective
• what is the role of social recommendations?• how do people watch TV together?• how devices influence watching?• what are perceived trade-offs for privacy vs. personalization?
18Wednesday, October 3, 12
Technology Perspective
• what is the role of metadata in TV & Apps?• what are ways to share information in real time?• what are ways to sync TV & other metadata?• what is an easy way for devices to find & talk to each other?
19Wednesday, October 3, 12
choices, choices ...
© VickyBuser, BBC
20Wednesday, October 3, 12
lost in space© RedBee Slides at MIPCube2012
21Wednesday, October 3, 12
always missing something© RedBee Slides at MIPCube2012
22Wednesday, October 3, 12
demand for experience23Wednesday, October 3, 12
© RedBee Slides at MIPCube2012
24Wednesday, October 3, 12
Social Recommendations© Ericsson ComsumerLab: TV & Web Consumer Trends 2011
25Wednesday, October 3, 12
Social Recommendations
I usually find NEW programmes to watch by:
RESULTS OF NOTUBE SOCIAL WEB & TV SURVEY 2012
© NoTube Studies @ BBC
26Wednesday, October 3, 12
Social Recommendations
I usually find NEW programmes to watch by:
RESULTS OF NOTUBE SOCIAL WEB & TV SURVEY 2012
Social TV
RESULTS OF NOTUBE SOCIAL WEB & TV SURVEY 2012
© NoTube Studies @ BBC
26Wednesday, October 3, 12
27Wednesday, October 3, 12
27Wednesday, October 3, 12
27Wednesday, October 3, 12
27Wednesday, October 3, 12
27Wednesday, October 3, 12
interface explorations
28Wednesday, October 3, 12
29Wednesday, October 3, 12
29Wednesday, October 3, 12
30Wednesday, October 3, 12
30Wednesday, October 3, 12
Linking Web Data to User Profiles
31Wednesday, October 3, 12
TV Preferences• Inferred from the Social Web
• tweets, FB likes, last.fm listened, etc.• weighted interests
• Represented using Linked Data web identifiers
• record-linkage:
• NLP:
facebook.com/pages/ dbpedia.org/
http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams
#littlebritain
32Wednesday, October 3, 12
TV Preferences• Inferred from the Social Web
• tweets, FB likes, last.fm listened, etc.• weighted interests
• Represented using Linked Data web identifiers
• record-linkage:
• NLP:
facebook.com/pages/ dbpedia.org/
Brilliant british humor by Matt Lucas & David Walliams - whole range of facinating characters portraying diversity of british society
http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams
#littlebritain
32Wednesday, October 3, 12
http://xmlns.notu.be/aair/33Wednesday, October 3, 12
http://xmlns.notu.be/aair/33Wednesday, October 3, 12
34Wednesday, October 3, 12
TV Program Enrichment
SemanticPattern-based
Recommendation Strategy
RDF GraphTV
Programs
Semantic ContentPatterns for
TV Programs
HybridRecommendation
Strategy
StatisticalSimilarity-based
Recommendation StrategyUser Ratings &
Demographics(BBC EPG
Data)
EPG Metadata(BBC)
Recommendation Service
SimilarityClusters
of Programs
User Data Analysis
End-UsersEnd Users
35Wednesday, October 3, 12
NoTube’s Beancounteraggregate, analyze & profile
36Wednesday, October 3, 12
NoTube’s User Activitiesaggregate
37Wednesday, October 3, 12
NoTube’s User Interestsprofile
38Wednesday, October 3, 12
NoTube’s User Interestsanalyze
39Wednesday, October 3, 12
NoTube’s User Explanationscontrol
40Wednesday, October 3, 12
Recommendations in NoTube
• surface interesting, new & relevant programs to individual and group users• combine in a complementary way different statistical & semantic approaches• define metrics for, e.g. serendipity, diversity, relevance
© Dan Brickley
41Wednesday, October 3, 12
From Predictability to Serendipity
42Wednesday, October 3, 12
From Predictability to Serendipity
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
42Wednesday, October 3, 12
From Predictability to Serendipity
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
documentaryformat
format
format
42Wednesday, October 3, 12
From Predictability to Serendipity
homes and gardens
genre
Grammy Awards
genre
genre art culture and the media
award
award
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
Basic patternsdocumentaryformat
format
format
42Wednesday, October 3, 12
From Predictability to Serendipity
homes and gardens
genre
Grammy Awards
genre
genre art culture and the media
award
award
PaulMerton
anchor
partner
CarolineQuentin
Life ofRiley
actor
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
Basic patternsHomogeneous patterns
documentaryformat
format
format
comedy
genre
42Wednesday, October 3, 12
From Predictability to Serendipity
homes and gardens
genre
Grammy Awards
genre
genre art culture and the media
award
award
PaulMerton
anchor
partner
CarolineQuentin
Life ofRiley
actor
SpikeMilligan
influence
The adventuresof Barry
McKenzie
actor
film
genr
e form
at
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
Basic patternsHomogeneous patterns
documentaryformat
format
format
comedy
genre
42Wednesday, October 3, 12
From Predictability to Serendipity
homes and gardens
genre
Grammy Awards
genre
genre art culture and the media
award
award
PaulMerton
anchor
partner
CarolineQuentin
Life ofRiley
actor
SpikeMilligan
influence
The adventuresof Barry
McKenzie
actor
film
genr
e form
at
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
CharlieChaplin
synopsisenrichment
ShanghaiKnights
form
at
action andadventure
genre
JackieChan
actor
influence
Basic patternsHomogeneous patternsHeterogeneous patterns
documentaryformat
format
format
comedy
genre
42Wednesday, October 3, 12
From Predictability to Serendipity
homes and gardens
genre
Grammy Awards
genre
genre art culture and the media
award
award
PaulMerton
anchor
partner
CarolineQuentin
Life ofRiley
actor
format
synopsisenrichment
Alfred Hitchcock
director
Suspicion
AcademyAwards
awar
d
Thriller
genre
SpikeMilligan
influence
The adventuresof Barry
McKenzie
actor
film
genr
e form
at
Paul Merton looks at
Alfred Hitchcock
SecretGardens
Make ‘em laugh
CharlieChaplin
synopsisenrichment
ShanghaiKnights
form
at
action andadventure
genre
JackieChan
actor
influence
Basic patternsHomogeneous patternsHeterogeneous patterns
documentaryformat
format
format
comedy
genre
42Wednesday, October 3, 12
Weak Semantics & Strong Statistics
43Wednesday, October 3, 12
Weak Semantics & Strong Statistics
Paul Merton
looks at Alfred Hitchcock
PaulMerton
anchor partnerCarolineQuentin
Life ofRiley
act
ordocumentary
form
at
art culture and the media
genre43Wednesday, October 3, 12
Weak Semantics & Strong Statistics
Paul Merton
looks at Alfred Hitchcock
PaulMerton
anchor partnerCarolineQuentin
Life ofRiley
act
ordocumentary
form
at
art culture and the media
genre
woman25 years old1 child
Userprofile
43Wednesday, October 3, 12
Weak Semantics & Strong Statistics
Paul Merton
looks at Alfred Hitchcock
PaulMerton
anchor partnerCarolineQuentin
Life ofRiley
act
ordocumentary
form
at
art culture and the media
genre
25 year old women with
1 child
woman25 years old1 child
Userprofile
43Wednesday, October 3, 12
Weak Semantics & Strong Statistics
Paul Merton
looks at Alfred Hitchcock
PaulMerton
anchor partnerCarolineQuentin
Life ofRiley
act
ordocumentary
form
at
art culture and the media
genre
25 year old women with
1 child
Similar users
woman25 years old1 child
Userprofile
43Wednesday, October 3, 12
NoTube’s N-Screen
N-Screendrag & drop, sharing & TV control
drag & drop, shuffle, sharing & TV control
44Wednesday, October 3, 12
NoTube’s N-Screensimilar programs
45Wednesday, October 3, 12
• means of surfacing content buried in the video collection• when users might reach a dead-end with the recommendations • adds an extra element of serendipity
NoTube’s N-Screenrandom selection
46Wednesday, October 3, 12
NoTube’s N-Screenexplanations
• depending on recommendation strategies• depending on the user’s task
47Wednesday, October 3, 12
♥
INTERESTING PROGRAMMES
watching with friends
suggestions based on programmes others have watched
linking programmes by guests, actors, presenters etc.
sharing suggestions with others in the same room
?
watching 'together apart' in different locations
changing programmes on someone else's TV
♥♥
♥
✔
✘
suggestions based on your past 'Social Web' activities
controlling the TV
drag & drop
swapping suggestions with specific friends
sharing them
✔
♥
✔
♥
sharing to watch immediately
http://notube.tv
? explaining recommendations
'shuffle' option for programme selection
?
finding
USER TESTING RESULTS December 2011, BBC R&D London10 participants : 5 male, 5 female, aged between 20 to 64
drag & drop to change what's playing on TV
♥ ✔ ✘ ? loved liked disliked unsure
For more details:
49Wednesday, October 3, 12
data: open, shared & accessible
infrastructures: interoperable
smart services: relevant content in right context
connected heritage
50Wednesday, October 3, 12
enriching collections
Lora Aroyo twitter: @laroyo
51Wednesday, October 3, 12
discovering links
style: BaroqueteacherOf: Gerrit Dou
studentOf: Pieter Lastman
self-‐portrait
mili<a place: Amsterdam, 1625 to 1650
The Night Watch
http://chip-project.org Lora Aroyo twitter: @laroyo
52Wednesday, October 3, 12
enriching with events
Lora Aroyo twitter: @laroyo
53Wednesday, October 3, 12
discovering links
Lora Aroyo twitter: @laroyo
54Wednesday, October 3, 12
personalized experience
55Wednesday, October 3, 12
56Wednesday, October 3, 12
57Wednesday, October 3, 12
Lora Aroyo twitter: @laroyo
58Wednesday, October 3, 12
Lora Aroyo twitter: @laroyo
http://agora.cs.vu.nl
60Wednesday, October 3, 12
Lora Aroyo twitter: @laroyo
http://agora.cs.vu.nl
60Wednesday, October 3, 12
61Wednesday, October 3, 12
Michiel Hildebrand
Lotte Belice Baltussen
Jacco van Ossenbruggen
Marteen Brinkerink
Johan OomenGuus Schreiber
Riste Gligorov
Marieke van Erp
Lourens van der Meij
Roxane Segers
Piek Vossen
Susan Legêne
Thomas Ploeger
Chiel van den Akker Frank de Bakker
Bibiana Armenta
Iina Hellsten
Geertje Jacobs
Geert-Jan Houben
Chris Welty
Natalia Stash
Yiwen Wang
62Wednesday, October 3, 12
Questions?
@laroyohttp://lora-aroyo.org
63Wednesday, October 3, 12