the quest for musical genres: do the experts and the wisdom of crowds agree?

ISMIR / Philadelphia, US // September, 18th 2008

The Quest for Musical Genres:

Do the Experts and the Wisdom of Crowds Agree?

Mohamed Sordo, Òscar Celma, Martin Blech, Enric Guaus(Music Technology Group ~ UPF)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

motivation

taxonomy (controlled vocabulary)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

motivation

taxonomy (controlled vocabulary)

folksonomy (free text)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

motivation

taxonomy (controlled vocabulary)

VS. folksonomy (free text)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

expert-based

• taxonomy Mp3.com 2005

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

expert-based

• taxonomy 13 seed genres (components) 7 levels 711 genres

Rock Hip-Hop

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

community-based

• folksonomy last.fm

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

community-based

• folksonomy last.fm ~137K artists ~90K tags (after cleaning)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

outline

1) Mapping tags to genres

2) Computing similarity among genres

3) Agreement between experts and wisdom of crowds

4) Reconstructing the taxonomy from the folksonomy

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

outline

1) Mapping tags to genres

2) Computing similarity among genres

3) Agreement between experts and wisdom of crowds

4) Reconstructing the taxonomy from the folksonomy

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

• folksonomy ~ taxonomyJade (artist tags):

90s, illinois, new jack swing, rnb, r and b,

urban, ...

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

• folksonomy ~ taxonomyJade (artist tags):

90s, illinois, new jack swing, rnb, r and b,

urban, ...

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

• folksonomy ~ taxonomyJade (artist tags):

90s, illinois, new jack swing, rnb, r and b,

urban, ...

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

• folksonomy ~ taxonomyJade (artist tags):

R&B, New-Jack-Swing, Urban

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

1) mapping tags to genres

• folksonomy ~ taxonomyJade (artist tags):

R&B, New-Jack-Swing, Urban

(39% tags matched

with MP3.com genres)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

outline

1) Mapping tags to genres

2) Computing similarity among genres

3) Agreement between experts and wisdom of crowds

4) Reconstructing the taxonomy from the folksonomy

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

2) computing similarity among genres

• Taxonomy distance(Doo-Woop, Urban) = 3

Penalty when crossing components distance(Urban, Rock-Pop) = 7

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

2) computing similarity among genres

• Folksonomy LSA (SVD), 50 dim.

Cosine similarity sim(Urban, Doo-Wop) = 0.868 sim(Urban, Pop-Rock) = -0.145

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

outline

1) Mapping tags to genres

2) Computing similarity among genres

3) Agreement between experts and wisdom of crowds

4) Reconstructing the taxonomy from the folksonomy

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) Separate (taxonomy) genre components using (folksonomy) genre sim. intra-component similarity inter-component similarity

• 2) Correlation between (taxonomy) genre path distance and (folksonomy) genre sim. DistanceTAXONOMY(g1, g2) ~???~ SimFOLKSONOMY(g1, g2)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) intra-component similarity, using LSA

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) intra-component similarity, using LSA

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) intra-component similarity, using LSA

Alternative-Rap

Dirty-Rap

West-Coast

Hip-hop

Bass-Music

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) intra-component similarity, using LSA

Blu

es

Hip

-hop

Rock

/Pop

Ele

ctro

nic

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) inter-component similarity centroid for each component

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 1) inter-component similarity Clearly distinguishable from the rest

Hip-hop, Blues, Jazz

Relationships found Country ~ Bluegrass (~ Folk) R&B-Soul ~ Gospel/Spiritual Electronic/Dance ~ Vocal/Easy-Listening New-Age ~ World/Reggae (!)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 2) taxonomy genre distance vs. folksonomy genre sim.

West-Coast

Calypso

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 2) taxonomy genre distance vs. folksonomy genre sim. DistanceTAXONOMY(West-Coast, Calypso) = 8

West-Coast

Calypso

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 2) taxonomy genre distance vs. folksonomy genre sim. DistanceTAXONOMY(West-Coast, Calypso) = 7 SimFOLKSONOMY(West-Coast, Calypso) = 0.04

West-Coast

Calypso

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

agreement experts ~ wisdom of crowds

• 2) taxonomy genre distance vs. folksonomy genre sim.

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

outline

1) Mapping tags to genres

2) Computing similarity among genres

3) Agreement between experts and wisdom of crowds

4) Reconstructing the taxonomy from the folksonomy

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. Get genres at level n

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. For each genre at level n

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. Get all nodes at level n-1 (possible parents)

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. Compute cosine LSA similarity

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. Assign closest parent

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Select closest parent, using folk. genre sim. Compare with taxonomy parent

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Results

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Results

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

reconstruct taxonomy from folksonomy

• Results

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

...and also!

• MIREX 2007 results (Team: IMIRSEL-M2K SVM)

RAPHIPHOP 84.05%BLUES 77.68%EDANCE 77.68%JAZZ 72.53%COUNTRY 71.37%ROCKROLL 69.53%BAROQUE 65.81%METAL 61.11%ROMANTIC 52.79%CLASSICAL 33.33%

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

conclusions

• Consensus in some genres expert, community, and audio

• Discovery in terms of taxonomy/folksonomy coarse / fine grained static / dynamic

• Taxonomy adapts according to the folksonomy

• Do we need experts?

• Are some (wisdom-of-crowds) shepherds more experts than “THE” experts?

ISMIR / Philadelphia, US // September, 18th 2008 // òscar celma // MTG / UPF

future work

• Use more taxonomies and folksonomies

• Agreement measures

Uncovering affinity of artists to multiple genres from social behaviour data (Claudio Baccigalupo, Justin Donaldson, Enric Plaza)

ISMIR / Philadelphia, US // September, 18th 2008

THANKS!!!

Mohamed Sordo, Òscar Celma, Martin Blech, Enric Guaus(Music Technology Group ~ UPF)