Instagram Big Data Analytics

Creative Chaos Amir Imani

TRANSFORM

DATA

COLLECT

STORE

INGEST

INSIGHT

DATA

Kat900K follower 4-5 photo/day 20K likes/ 100 cmnts

Kevin62K follower 1-2 photo/day 500 likes/ 5 cmnts

Scott132K follower 4-5 photo/day 2000 likes/ 100 cmnts

Creative Chaos2K follower 1-2 photo/week 65 likes/ 5 cmnts

Kat850K follower 4-5 photo/day 20K likes/ 100 cmnts

Kevin57K follower 1-2 photo/day 500 likes/ 5 cmnts

Scott113K follower 4-5 photo/day 2000 likes/ 100 cmnts

Creative Chaos2K follower 1-2 photo/week 60 likes/ 5 cmnts

# &

COLLECT

• Extract • Filter • Integrate

• Interactions: like & cmnts • Augmentation: Gender/Language

{"hash":"da3b25df71bf16a847491533e468d359","hash_type":"stream","id":"25bbf43253ed4576ed832d0038c316b4","delivered_at":"Tue, 03 May 2016 04:03:03 +0000","interactions":[{"demographic":{"gender":"female"},"instagram":{"from":{"id":"1765062192","username":"fatemehkavandi","full_name":"Fatemeh Kavandi","profile_picture":"https://scontent.cdninstagram.com/t51.2885-19/s150x150/12501620_208060992908600_18549704_a.jpg"},"type":"like","media":{"id":"1241673596494888825_21515974","created_time":1462239049,"username":"kat_in_nyc","link":"https://www.instagram.com/p/BE7T46uDz95/","type":"image","caption":"my love @sashakichigina ❤ ","filter":"Normal","image":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/13098947_539207262947852_1918000948_n.jpg?ig_cache_key=MTI0MTY3MzU5NjQ5NDg4ODgyNQ%3D%3D.2.l"}},"interaction":{"id":"1e610ceaa6daaa80e019000040de3cfb","type":"instagram","subtype":"like","content":"fatemehkavandi likes kat_in_nyc's image","author":{"avatar":"https://scontent.cdninstagram.com/t51.2885-19/s150x150/12501620_208060992908600_18549704_a.jpg","id":"1765062192","link":"http://instagram.com/fatemehkavandi","name":"Fatemeh Kavandi","username":"fatemehkavandi"},"link":"https://www.instagram.com/p/BE7T46uDz95/","created_at":"Tue, 03 May 2016 03:58:07 +0000","received_at":1462247887.3020000},"language":{"tag":"ms","tag_extended":"ms","confidence":97},"source":{"id":"c9b92d83ee02431bb3e3c263efeef962"}},{"demographic":{"gender":"male"},"instagram":{"from":{"id":"472927476","username":"ildefonsod","full_name":"Ildefonso Domínguez","profile_picture":"https://scontent.cdninstagram.com/t51.2885-19/10540331_303893719790855_791725810_a.jpg"},"type":"like","media":{"id":"1241539081550118008_21515974","created_time":1462223013,"username":"kat_in_nyc","link":"https://www.instagram.com/p/BE61Td6jzx4/","type":"image","caption":"@ksyander ❤ ","filter":"Normal","image":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/13108840_1590136574634198_742652310_n.jpg?ig_cache_key=MTI0MTUzOTA4MTU1MDExODAwOA%3D%3D.2.l"}},

STORE

Scalable Cloud-Storage Full lifecycle support Reliable and CheapS3

{{JSON}

Scalable Cloud-Storage Full lifecycle support Reliable and CheapS3

{{JSON}

INGEST

Fast and general engine for big data processing Open source cluster computing framework built-in modules for streaming, SQL{

Fast and general engine for big data processing Open source cluster computing framework built-in modules for streaming, SQL{

TRANSFORM

Python 3.5

Widely used high-level, dynamic language Open source with variety of libraries Online support community{

INSIGHT

Build a dashboard for: Followers analysis (top likers, etc) Content analysis (top liked, etc) Historic data comparison

{

Gender breakdown Gender breakdown/user

Filters

Overview of likes for each photo

overall Life span

Average Life span Average interaction

Filters

Followers rankingLanguage

• DataSift lacks: follower counts, accurate gender/language, accurate interaction time

• It is expensive (at least $20,000 per year just for the infrastructure) • It’s new API sucks!

{"hash":"5765b1d967610b0b5089a399a81bef59","hash_type":"stream","id":"8b53e742335c0dabc9dcb6ad5c840798","delivered_at":"Wed, 29 Jun 2016 14:45:51 +0000","interactions":[{"instagram":{"count":39,"type":"comment_count","delta":0,"media":{"username":"kat_in_nyc","filter":"Normal","type":"image","from":{"fullName":"kat irlin","username":"kat_in_nyc","profile_picture":"https://scontent.cdninstagram.com/t51.2885-19/11906380_1642296532707452_1269120372_a.jpg","id":"21515974"},"link":"https://www.instagram.com/p/BHOGzD9DCTk/","content":"brooklyn fog with @sashakichigina â¤ï¸â¤ï¸â¤ï¸","created_time":"Wed, 29 Jun 2016 01:44:46 +0000","caption":"brooklyn fog with @sashakichigina â¤ï¸â¤ï¸â¤ï¸","image":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/13534471_286775051671508_916687424_n.jpg?ig_cache_key=MTI4Mjk5Mjg0MTA2NDk4MTczMg%3D%3D.2","tags":[],"id":"1282992841064981732_21515974"}}