insight dataengineering henok_rehearsaldemo
Post on 11-Apr-2017
48 Views
Preview:
TRANSCRIPT
Where is my tweet?Henok Mengistu
Insight Data Engineering Fellow
Silicon Valley, Summer 2016
Motivation
Motivation
But, this number doesn't show how the tweet spreads-out?
But, a re-tweet graph could show
Under the hood
Engineering Challenges
● Stitching the different components ● Re-tweets could arrive out of order
– Spark can't sort across a data stream
– The driver node should collect and sort re-tweets
● I am Henok– Originally, from Ethiopia
– Currently, a PhD student at the University of Wyoming
● Working on Evolutionary Computation● I was also working as a Teaching assistant
– I like soccer, but not skiing
Thank you!
Queries
● On the re-tweet graph
– who are my audiences? ● Geographically, social groups
– Betweenness centrality ● Who is relevant to spread out my tweet?● Identify influential followers
top related