social tags and music information retrieval (part i)

Social Tags and
Music Information Retrieval
Part I

ISMIR 2008

Paul Lamere Sun Microsystems Inc.

Elias Pampalk Last.fm

Speaker Paul Lamere

Speaker Elias Pampalk

Audience Poll

Have you ever tagged something?

Have you ever tagged music?

Have you ever heard of Last.fm?

Have you ever used tags to help find something?

Do you think social tags are just a web 2.0 fad?

Are you using social tags in your research?

Are you a tag skeptic?

What are social tags?

Word Clouds created with Wordle.net

Social Tags - Delicious.com

Social Tags - Flickr.com

Social Tags - LibraryThing

What is Deerhoof?

Flickr photo by graciepoo

What is Deerhoof?

The Fear It's just Graffiti

Folksonomies are notoriously imprecise, full of tags that are "often ambiguous, overly personalized and inexact" resulting in an "uncontrolled and chaotic set of tagging terms that do not support searching as effectively as more controlled vocabularies do"

Guy & Tonkin, 2006

... if users can continuously add tags to items, at some point it is likely that the whole system will become unusable. A folksonomic system threatens to undermine its own usefulness.

Elaine Peterson, 2006

Flickr Photo by funkandjazz

Abbey E.Thompson. Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy

Outline


Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Outline


Why do people tag?




Data & Tools

Future Research

Conclusion

Discussion


Tags

Short, free text labels applied to content

No structure, no vocabulary limits

Typically applied by the generator or the consumer of the item being tagged

Social Tags

The aggregation of individual sets of tags

Sometimes called a Folksonomy - A user-created bottom-up categorical structure development with an emergent thesaurus

Taxonomy vs. Folksonomy

Taxonomy

Strong labeling

Structured vocabulary

Fixed vocabulary

Folksonomy

Weak labeling

Unstructured vocabulary

Free-for-all vocabulary

D. Turnbull, L. Barrington, G. Lanckriet. Five Approaches to Collecting Tags for Music. ISMIR 2008.

Some characteristics of social tags

J. Furner, M. Smith, and M. Winget. Collaborative indexing of cultural resources: Some outstanding issues.

(a) User tagging is user-oriented. Tags for the resources in a given collection are generated by the members of the community of people who have a demonstrated interest in searching that collection, rather than by professional catalogers or indexers who are tasked with tagging as a means to support others resource discovery. (b) User tagging is empowering. People who might in the past have been accustomed to searching databases by attempting to predict the descriptors used by experts are given the opportunity to record their own knowledge about resources. (c) User tagging is democratic. Taggers are not selected for their expertise by collection managers, but are self-selected according to taggers own interests and goals. (d) User tagging is cheap. Taggers typically volunteer their efforts at low or no cost to collection managers. (e) User tagging is collaborativeif only in the sense that any given record or description of a resource is potentially representative of the work of multiple people. (f) User tagging is distributed. No single person is required to tag all of the resources in a given collection. At the same time, no single resource needs to be tagged by all of the people in a given community. (g) User tagging is dynamic. The description of a given resource may change over time, as different people come to make their own judgments of its nature and importance. (h) User tagging is instructive. The descriptors supplied by taggers may be analyzed with a view to learning about the kinds of aspects of resources that are interesting or significant for the members of the taggers community.

Social Tagging System Attributes

Tagging Rights

owner, group, anyone

Tagging Support

blind tagging, suggestive tagging

Aggregation

bag model, set model

Type of object being tagged

artists, tracks, albums, labels, playlists, clips

# Tagging Rights - who can tag what - * anyone can tag anything (free-for-all) (delicious, last.fm) o Typical for 3rd party content * only contributor (or 'friends') can tag: (flickr) o Typical for user generated content - you probably wouldn't want just anyone to be able to tag your photos on flickr * anyone can tag, but tags are moderated (musicbrainz)# Tagging Support - how does the system help you tag items: * Blind tagging - no support * Tags are recommended o From your own set of personal tags o From other tags already applied to the object being tagged o Hybrid - your own set of tags filtered by the tags others have already applied to the object being tagged o Recommendation - use CF techniques to predict what tags an individual would apply to an item and recommend those * Implications of tag recommendations: o May converge on a vocabulary sooner + Less chance for 'hiphop' 'hip hop' and 'hip-hop' to be used to describe an item o Potentially less diverse tags - if I think something should be tagged 'screamo' but I am suggested 'emo' it may be easier for me to just click on 'emo' and skip adding the new 'screamo' tag o Early taggers can have an undue influence - since their tags will be given as suggestions to future taggers. Without a tagging suggestion system a tag like ATSITWPIOSB would probably not be so prevalent. o May discourage some tagging - if I am about to tag Radiohead with 'alternative' but the tag suggester indicates that Radiohead has already been tagged frequently with 'alternative' I just may skip the tag.

Aggregation - * bag model (=set with weighted items) (last.fm) o potentially more descriptive - tags show a probability distribution o Show distribution of top 20 tags for shins o More common in 'free-for-all' tagging systems * set model (set where each item has the same weight) (flickr) o elias says: how is that an aggregation if per definition there is only one of each? o paul answers: we are aggregating different tags together, each with a count of 1

Type of object - what is tagged. In the music domain this is often:

* Artists tracks * Albums * Labels * Playlists * Clips

What can we learn from social tags?

Insights to user behavior and language usage

how different is rap from hip-hop?

Grouping of items based on tags or taggers

cheese on toast, hooves and paws

Finding social groups with shared interests

people that tag the same items or use the same tags

Generating user profiles from tagging behavior

tag clouds based on tags applied by the user

Representation of taste and/or expertise

Information on how items change over time

track changes to item tags over time

K. Weller. Folksonomies and Ontologies. Two New Players in Indexing and Knowledge Representation.

# Tagging Rights - who can tag what - * anyone can tag anything (free-for-all) (delicious, last.fm) o Typical for 3rd party content * only contributor (or 'friends') can tag: (flickr) o Typical for user generated content - you probably wouldn't want just anyone to be able to tag your photos on flickr * anyone can tag, but tags are moderated (musicbrainz)# Tagging Support - how does the system help you tag items: * Blind tagging - no support * Tags are recommended o From your own set of personal tags o From other tags already applied to the object being tagged o Hybrid - your own set of tags filtered by the tags others have already applied to the object being tagged o Recommendation - use CF techniques to predict what tags an individual would apply to an item and recommend those * Implications of tag recommendations: o May converge on a vocabulary sooner + Less chance for 'hiphop' 'hip hop' and 'hip-hop' to be used to describe an item o Potentially less diverse tags - if I think something should be tagged 'screamo' but I am suggested 'emo' it may be easier for me to just click on 'emo' and skip adding the new 'screamo' tag o Early taggers can have an undue influence - since their tags will be given as suggestions to future taggers. Without a tagging suggestion system a tag like ATSITWPIOSB would probably not be so prevalent. o May discourage some tagging - if I am about to tag Radiohead with 'alternative' but the tag suggester indicates that Radiohead has already been tagged frequently with 'alternative' I just may skip the tag.

Aggregation - * bag model (=set with weighted items) (last.fm) o potentially more descriptive - tags show a probability distribution o Show distribution of top 20 tags for shins o More common in 'free-for-all' tagging systems * set model (set where each item has the same weight) (flickr) o elias says: how is that an aggregation if per definition there is only one of each? o paul answers: we are aggregating different tags together, each with a count of 1

Type of object - what is tagged. In the music domain this is often:

* Artists tracks * Albums * Labels * Playlists * Clips

Example: MusicBrainz

Some MusicBrainz Tag Stats

Artist tags: 37,875

Album tags: 21,423

Track tags: 10,818

Label tags: 1,250

Example: Amazon

Example: MyStrands

Example: imeem

Example: Freesound

Last.fm Case Study

History

Tagging stats

Integration

Tags and their popularity

More stats

July 2005: Felix Miller (CEO) on internal wiki:

Why use tags:

To build playlists by tagging tracks

Categorise your profile and the global music catalogue

Get recommendations based on tags

Summarize your profile

August 2005: launched

Inspired by delicious and flickr

At the same time audioscrobbler merged with Last.fm

History of tagging at Last.fm

Tags: > 50M

Tracks > 50%

Artists > 40%

Albums < 5%

Labels < 1%

Unique tags: > 1.2M

Unique tags applied to more than 10 items: > 130k

Items tagged: > 3.8M

Tags per month: > 2.5M

Taggers per month: > 300k

Last.fm tagging stats

Integration into Last.fm experience

Where are tags visible?

Music landing page

Tag pages

Tag radio

Artist, album, track, label summaries

Tag clouds

User library, ...

How can users tag something?

Desktop software (Last.fm Player & Scrobbler)

Web site

Add to library dialog

Add tag button on item pages (artist, album, track)

Every listed item (charts etc) has multi-function button

Flash player


Tags are visible almost everywhere

It is very easy to tag something


Tag Popularity: Examples

rock >190k >2M

seen live >75k >1M

female vocalists >60k > 500k

chillout >40k > 250k

guitar >20k >90k

happy >9k >40k

icelandic >8k >20k

brazil >4k >15k

inspirational 1.2k >5k

hammond organ >200 >700

berimbau 18k > 200k

officially sh*t > 2k > 12k

i am a party girl here is my soundtrack > 350 > 5.9k

tv themes > 250 > 680

songs that i sing along to but i always forget the words so i say duh duh while trying to sound like i do know the words and no one is falling for it but they keep quiet because they are embarrassed for me
> 60 > 300

gtasa tracks < 30 > 700

artists with disco in their name 2 49

wherein i compile a comprehensive list of songs about zombies
7 36

people # times applied

Tag Categories beyond Genre
by Babs_05 (Last.fm user)

personal meaning

playlists with unusual names

regional

decade

group-specific

descriptive

http://www.last.fm/group/Track-Tag+Bitches/forum/24779/_/319957/1#f4621887

tempo

mood

instruments

films

music to...

album labels

based on radio shows or nightclubs

[Paul Lamere, JNMR 2008]

Tagging Frequencies by Category

[Paul Lamere, JNMR 2008][Klaas Bosteels et al., ISMIR 2008]

Multi-tag search queries51% 7% 4% 2% 5% 26%

Time 3%Other 2%

Tagging vs. Search Frequencies

Obscure Tags

if you fall in love with me you should know these songs by heart

sure go ahead and depress the hell outta me what do I care

that one song

and finally laughing like a maniac saying I am the ruler of this stupid little world called me

I woke up this morning and the sun was gone

I used to suck blood with gothic dogs but now I wear bermuda shorts and [...]

I was never a Swedish teenager with swedish teen angst so now I will attempt at having it

Number of words per tag

[Levy & Sandler, JNMR 2008]

Popular tag radio stations

Pop, rock, jazz, chillout, 80s, indie, dance, hip-hop, alternative, house, pop rock, reggae, electronic , oldies, metal, blues, schlager, ambient, trance, classical

Measured on a day in July 2008.

Almost all of the most popular radio stations are genres!Surprise?

Most frequently applied tags

Rock, seen live, alternative, indie, electronic, pop, metal, female vocalists, alternative rock, classic, rock, punk, jazz, indie rock, electronic, singer-songwriter, folk, hip-hop, ambient, dance, experimental

Metal fans are some of the best taggers ever.

[Paul Lamere, JNMR 2008]

Artist rank (wrt number of tags applied, 1-1000)

Number of tags applied

Distribution of tags

Age vs. Tagging Behaviour

Under 19 Average age of users (years)

lame lame lame lame 18.2

tokio hotel 18.5

xd 18.7

100 members

We have the radio you want! :-D If not, join and help us grow musically... YAY!

Tag radio stations listed:

Cheese on Toast, The Ladies Room, Tambourine, Xylophone & Vibraphone, Forests and Woodland, Insects and Arachnids, They Said Shoes!, Under the Influence..., Awesome Guitar Jams, ...

Group: Subscribers and their tag radio stations

>100 members

This group is to bring together subscribers who have taken the time to tag and share the music they like. No specific genre or participation needed. This is just a way to find and listen to...

Why do people tag?
Last.fm groups focusing on tags

Group: Genre-free tags!

>100 members

"That's right; there's more to tags than silly genres."

lesser known yet streamable artists

cover track

first album

i am a spy here is my soundtrack

Why do people tag?

Group: Thursday Night Party Hat Party

>70 members

OK, I've started tagging weird country music, in anticipation of Frippgamesh's party on August 21. Right now, I'm the only one who's done any tagging, but I was able to tag 11 artists without working up a sweat, starting with the band Rebel Meets Rebel that I had afflicted aniaki with in a recent double-dare.

Why do people tag?

Group: WoopWoop

>30 members

"Listening to a track you're desperate to share with the world? Want to listen to music that others feel the same about?"

Why do people tag?

More Groups:

Pure Track-Taggers

Track-Tag Bitches

Tag Top

Tags are a Joke

We use tags that have never been used before

...

Why do people tag?

Why do people tag?

Motivations

Simplicity

Memory and context

Task organization

Opinion expression & social signaling

Social contribution

Play and competition

Last.fm specific

Collaborative/personal playlists (tag radio)

Tag groups

Outline


Why do people tag?




Data & Tools

Future Research

Conclusion

Discussion

Cold start and obscure content

Tagging something requires:

Knowing it exists (new releases, cold start)

Caring enough to bother (obscure content, long tail)

Problem scope

Many artists, albums, tracks affected

Solution: alternative sources of tags

Autotagging, tag games, ...

Radiohead


Tagged by 24k listeners on Last.fm

Jersey Budd


Weak labeling

E.g. male vocalist is sometimes considered a default

Problem scope:

sparsely tagged items are affected most

Popularity & noise

The Beatles:

tagged emo 68 times on Last.fm

These Silhouettes:

tagged emo 11 times on Last.fm

Solution: normalization

Precision and Recall

Kids having fun

Barney, Paris Hilton, ...

Can be very offensive

Labels/artists trying to help

Problem scope:

Limited to very few artists

Abuse

Abuse example:
Paris Hilton

Do taggers like what they are tagging?

Correlations between taggers

Bot-like patterns

Tag Karma

How useful are contributions of a tagger?

E.g. if artist X is tagged metal, but everyone listening to metal radio skips that artist...

Abuse detection

Examples

that one song

lazy eye

Problem scope

Sparsely tagged items affected most

Noise

Examples

Polysemy: love

music I love

music about love

Synonym: female vocalists

Problem scope

Sparsely tagged items

Ambiguity and Synonyms

89277 female vocalists

15874 female

7150 female vocalist

2716 female vocals

2180 female voices

1424 female artists

955 female vocal

698 female singers

http://blogs.sun.com/plamere/entry/determining_synonyms_from_tags

Synonyms Example

Last.fm community is not an unbiased sample of music listeners worldwide

On average a relatively young audience

Tech-savvy

Tastemakers

Eclectic tastes, interested in discovering new music

Non-mainstream

Some regions are underrepresented (e.g. Africa)

Some styles of music are represented stronger than others. E.g. classical music vs. indie/alternative

Tagger bias

Last.fm community is not an unbiased sample of music listeners worldwide

Electronic music

Nielsen: 3% of total album sales in US

Last.fm: 107k taggers

Country music

Nielsen: 11% of total album sales in US

Last.fm: 23k taggers

Tagger bias in numbers

[Nielsen Report 2007]

Issues with Social Tags

Not enough tags

Cold start

Obscure content (long tail)

Weak labeling

Synonyms and ambiguous tags

Noise

Abuse

Tagger bias

Some styles of music are covered better than others

Not everyone is contributing equally

Outline


Why do people tag?




Data & Tools

Future Research

Conclusion

Discussion

Games (with a purpose)

Idea:

Humans are best at solving some types of problems

Humans like to play games

Build a game that is

fun to play

Solves a hard problem

ESP Game Image Labeling Game

Users try to guess what labels partner will apply to an image

Collected 1.2 million image labels in 4 months

With 5K active gamers could label all 425 million in 1 month

L. von Ahn and L. Dabbish. Labeling images with a computer game. SIGCHI 2004.

Games Tag a Tune

http://www.gwap.com/gwap/gamesPreview/tagatune/

Statistics for 3 months of game play

Total tags: ~260,000

Total songs: ~20,000

High confidence tags: ~50,000

Games played: ~23,000

Unique players: ~6,900

Tags per player per minute: ~4

Games Tag a Tune

Games Major Miner

http://majorminer.com/

* You will be presented with one randomly selected 10-second music clip at a time. * Describe that clip with a word or phrase (we call them "tags", here are some examples) and press enter. * If you're the first person to describe that clip with that tag, you'll get 2 points when the next person tags that clip with that tag. * If you're the second person to describe that clip with that tag, you'll get 1 point immediately. * If more than two people have already tagged that clip with that tag, you won't get any points, but you can try another tag. * Tag each clip as many times as you want, follow one of the "new clip" links to listen to a new one.

Unique asynchronous scoring

Statistics

530 Users

2,345 clips tagged

73,000 user, clip, tag triples

12,000 verified tags

900 unique, verified tags

12,000

3-4 tags per minute per user

5-10 minutes of playtime per user

Data used for the MIREX tagging task

http://www.music-ir.org/mirex/2008/index.php/Audio_Tag_Classification

Games Major Miner

http://majorminer.com/search/

Games Major Miner

Games The Listen Game

http://www.listengame.org/

Statistics from 2 week pilot study

Listen250 dataset

26,000 annotations

1,558 High confidence positive associations

1,630 High confidence negative associations

250 songs

120 words

440 unique players

775 new words suggested (via freestyle round)

~ 5 Tags per user per minute

Dataset available upon request from Doug Turnbull

Games The Listen Game

Games MoodSwings

Y. E. Kim, E. Schmidt, and L. Emelle. MoodSwings:
A collaborative game for music mood label collection. ISMIR 2008.

Collects valence-arousal labels

1 week pilot

100 Users

1K Songs

50K labels

Prototype BBC game not released yet, so no dataSome unique aspects - tied in with radio so you tag data while listening

Games Herd It

"Herd It connects players in real-time and they listen to the same music clip. While they listen,
various fun minigames quiz the players on opinions and trivia about the music. Players earn
points based on the agreement of their answers with the group consensus." - Luke Barrington

Facebook based game by the UCSD team not released yet, so doesn't have any data

Games Moose6 A BBC Prototype

Prototype BBC game not released yet, so no dataSome unique aspects - tied in with radio so you tag data while listeningMoose 6 (kind of named after one of our radio stations - 6 Music)involves players listening to live music radio and tagging the currentlyplaying song or track with both descriptive tags and similar artists.Being live is one of the defining characteristics of radio so we wantedto use that in the game. The game aspect entices people to play yetproduces useful data and may be educational.

Whenever a track is played on 6Music a new round is started and playerswill see a countdown of two minutes starting. The artist and trackdetails are displayed and players should type in as many terms aspossible in that 2 minutes. Players are encouraged to enter two kinds ofterms - descriptive tags and/or similar artists. After two minutes theround finishes and players are given a score based on whether any otherplayers in the round entered the same tags and artists.

Points are scored by matching a tag, where >n players entered that tag,and for matching an artist recommendation (where >n players entered thatartist).The exact details of the scoring scheme are deliberately keptsecret from the players to create some sense of mystery and to avoidinfluencing players' behaviour too much. When the scoring is completethe most popular tags and artists for that round are shown, along withwhat you scored points for. When the next track is played a new roundstarts and the process repeats. An overall leaderboard is created fromthe cumulative scores of all the players. Finally, there are pages foreach round, artist, track and tag showing the aggregated data.

Pros:

Can tag at the phrase/clip level

Potentially can tag into the long tail

Potentially very high tagging rates

Issues:

Challenge to make addictive/fun game

Tags may be more superficial than social tags

Tags may be applied by non-fans

Lowest common denominator labels

Potential for 'gaming' the system

Don't collect many types of useful labels:

context (jogging), opinion (favorite), task organization (check out), social signaling (seen live)

Games (with a purpose)

Experts/Surveys

Hire experts to hand label content

Advantages

Consistent labels

Strong labeling

Fixed, structured vocabulary

Disadvantages

Small pre-determined vocabulary

Very difficult to construct taxonomy

Human-labor intensive (expensive)

Doesn't scale to the long tail

Limited access to data

Experts/Surveys All Music Guide

Pop Tracks: ~13 million

Pop Albums: ~1.4 million

Classical Compositions: ~300 thousand

Biographies: ~84 thousand

Styles: 917

Themes: 86

Moods: 184



Music labeled by paid professional musicians

Fixed vocabulary: ~500 weighted attributes

Structured vocabulary

Experts/Surveys Pandora

66 undergrads paid $10 per hour to label music

1,700 annotations

500 'western popular' songs

135 musically-relevant concepts

Six semantic categories:

emotion aggressive, weird, thrilling, mellow

genre - alternative, bebop, swing, rock, world

Instrument guitar, organ, piano, harmonica

usage at a party, at work, driving, exercising, sleeping

vocals breathy, rapping, monotone, emotional

song catchy, fast, danceable, texture acoustic

http://cosmal.ucsd.edu/cal

Experts/Surveys CAL-500

Web mining

Internet: the largest digital archive of music

Music blogs, reviews, forum discussions, official artist pages, MySpace, ...

Examples: (Google estimated number of results)

Eminem 46M pages

The Beatles 29M pages

Wolfgang Amadeus Mozart 7M pages

Unstructured data

Text information retrieval

Whitman & Lawrence (ICMC 2002)

Google: +music +review

Retrieve text of top ranked pages

Text information retrieval techniques

Issues

Artist names are not unique identifiers

The War, The The, Chicago, Pink (not P!nk)

Solution: add information to query (e.g. album names)

Cold-start (needs at least one web-page)

Quality of tags

Include other artist names, seemingly random noise

Solution: use dictionary

Web mining Using Google et al.

ABBA

100pop

56song

50musical

46swedish

43uk

41group

36band

35dance

31world

29disco

Pampalk et al. (ECDL 2005)http://www.ofai.at/~elias.pampalk/wa/

Web mining Examples

Britney Spears

100pop

99song

52girl

36concert

33dance

24uk

20sexy

18jive

18world

15cool


Web mining Examples

Metallica

100metal

43band

38concert

22heavy metal

20world

20song

18dance

15hard

11black metal

9guitar


Web mining Examples

Ludwig van Beethoven

100piano

67symphony

53classical

22orchestra

20movement

18trio

18solo

15variations

15musical

12violin


Web mining Examples

[Knees et al., ACM MM 2006]

Using Web Mined Tags: nepTune

[Pampalk & Goto, ISMIR 2006]

Using Web Mined Tags: MusicRainbow

[Pampalk & Goto, ISMIR 2007]

Using Web Mined Tags: MusicSun

Sun Microsystems, Inc.

Page

Click to edit the title text format

Click to edit the outline text format

Second Outline Level

Social Tags and Music Information Retrieval ISMIR 2008

Click to edit the notes format

Page

Click to edit the title text format

Presenters Name

Presenters Title

Presenters Company

Click to edit the notes format

Page

Age% Non TaggersTaggers' Median Vocabulary Size

14 1941.3 48.16

19 2237.5 43.67

22 2540.6 47.39

25 3034.8 41.48

30 6028.4 36.013

???Page ??? (???)07/09/2008, 04:34:54Page /

social tags and music information retrieval (part i)

Technology