machine learning and big data for music discovery at spotify
TRANSCRIPT
![Page 1: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/1.jpg)
Machine Learning & Big Data for Music Discovery
Galvanize NYC, Mar 9th, 2017
Vidhya Murali @vid052Ching-Wei Chen @cweichen
![Page 2: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/2.jpg)
100M users in 60 markets
50M subscribers
Over 30M songs, and 2B playlists
$5B paid to rightsholders
SpotifyMusic for everyone.
![Page 3: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/3.jpg)
30 Million Songs...
![Page 4: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/4.jpg)
What to recommend?
![Page 5: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/5.jpg)
What to recommend?
Discover
![Page 6: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/6.jpg)
Discover
Discover Weekly
How What to recommend?
![Page 7: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/7.jpg)
Many flavors of recommendations
Radio
![Page 8: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/8.jpg)
Many flavors of recommendations
Daily MixRadio
![Page 9: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/9.jpg)
Many flavors of recommendations
This Is:Daily MixRadio
![Page 10: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/10.jpg)
Many flavors of recommendations
Recommended SongsThis Is:Daily MixRadio
![Page 11: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/11.jpg)
‣ Editorial
Recommendation approaches
![Page 12: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/12.jpg)
‣ Editorial‣ Algorithmic
○Content-based
■ Metadata
■ Audio Signals
○Collaborative
■ Usage based
Recommendation approaches
![Page 13: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/13.jpg)
‣ Editorial‣ Algorithmic
○Content-based
■ Metadata
■ Audio Signals
○Collaborative
■ Usage based
‣ Algotorial
Recommendation approaches
![Page 14: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/14.jpg)
‣ Editorial‣ Algorithmic
○Content-based
■ Metadata
■ Audio Signals
○Collaborative
■ Usage based
‣ Algotorial
Recommendation approaches
![Page 15: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/15.jpg)
‣ Find patterns from user’s past behavior to generate recommendations.
‣ Domain independent
‣ Scalable
Collaborative Filtering
![Page 16: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/16.jpg)
Compact representation for each user and item (song): f-dimensional vectors
Latent Factor Models
![Page 17: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/17.jpg)
NLP Models
![Page 18: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/18.jpg)
Context & Co-occurrence is key!
![Page 19: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/19.jpg)
Document : Playlist
Word : Song
NLP Models work great on playlists!
![Page 20: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/20.jpg)
Generating Song Vectors
w1 w2 w3 w4 w5 w6 w7 wn....………..
?
![Page 21: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/21.jpg)
Music in Latent Space
![Page 22: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/22.jpg)
Semantic Regularities
![Page 23: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/23.jpg)
Music + Math = Epic
![Page 24: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/24.jpg)
Songs as vectors
![Page 25: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/25.jpg)
Recommendations
User Profile:
● Aggregation over user interactions on Spotify
● Clustering to capture distinct user tastes/ contexts
● Time Sensitive profiling
![Page 26: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/26.jpg)
‣ Scale of catalog● 30M tracks; 2B playlists● Training
○ 25B data points○ 100M users○ 60 countries represented
Challenges unique to spotify
![Page 27: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/27.jpg)
Data Pipelines
![Page 28: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/28.jpg)
Data Pipelines
Big Table
Big Table
GCS
DATAFLOW
Pub Sub
Scio
![Page 29: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/29.jpg)
‣ Scale of catalog● 30M tracks; 2B playlists● Training
○ 25B data points○ 100M users○ 60 countries represented
‣ Cold-Start○ New Users○ New Music
Challenges unique to spotify
![Page 30: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/30.jpg)
Learning from sound
![Page 32: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/32.jpg)
What’s in a sound?
AmplitudeTime
Frequencies
Loudness
![Page 33: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/33.jpg)
What’s in a sound?
MelodyBeats
Chords
Voices
Instruments
Lyrics
AmplitudeTime
Frequencies
Loudness
![Page 34: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/34.jpg)
What’s in a sound?
MelodyBeats
Chords
Voices
Instruments
Lyrics
AmplitudeTime
Frequencies
Loudness
PopularityEra
RegionGenre
Mood
Purpose
* Some information isn’t encoded in the signal itself, but within the cultural context around the music
![Page 35: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/35.jpg)
Supervised Machine Learning
http://www.nltk.org/
![Page 36: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/36.jpg)
Deep Learning
![Page 37: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/37.jpg)
Deep Learning
1. No feature extraction necessary
2. LOTS of simple learning nodes in many layers
3. Propogate errors backwards to learn optimal weights
4. Needs LOTS of data
![Page 38: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/38.jpg)
Convolutional Neural Networks
![Page 39: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/39.jpg)
Typical Convolutional Neural Network
![Page 40: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/40.jpg)
Deep Learning on Audio at Spotify
Sander Dieleman: http://benanne.github.io/2014/08/05/spotify-cnns.html
Input: Audio spectrogram
Output: Latent Space Vector
![Page 41: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/41.jpg)
Audio vector space
![Page 42: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/42.jpg)
Cold Start? Problem solved! *
* Not completely, of course!
![Page 43: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/43.jpg)
Recommending new music
Release Radar Fresh Finds
![Page 44: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/44.jpg)
Recommendations at Spotify
Recommended SongsThis Is:Daily MixRadio
Discover Weekly
Release Radar
![Page 45: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/45.jpg)
What’s next?
?
![Page 46: Machine Learning and Big Data for Music Discovery at Spotify](https://reader031.vdocuments.net/reader031/viewer/2022030307/58e6036e1a28ab44778b46ef/html5/thumbnails/46.jpg)
Join the band!www.spotify.com/jobs
Ching-Wei (@cweichen): [email protected]
Vidhya (@vid052): [email protected]