analytics building blockspoloclub.gatech.edu/cse6242/2018spring/slides/cse... · duen horng (polo)...
TRANSCRIPT
![Page 1: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/1.jpg)
http://poloclub.gatech.edu/cse6242CSE6242 / CX4242: Data & Visual Analytics
Analytics Building Blocks
Duen Horng (Polo) Chau Assistant ProfessorAssociate Director, MS AnalyticsGeorgia Tech
Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John Stasko, Christos Faloutsos
![Page 2: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/2.jpg)
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
![Page 3: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/3.jpg)
Building blocks. Not Rigid “Steps”.
Can skip some
Can go back (two-way street)
• Data types inform visualization design
• Data size informs choice of algorithms
• Visualization motivates more data cleaning
• Visualization challenges algorithm assumptionse.g., user finds that results don’t make sense
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
![Page 4: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/4.jpg)
How “big data” affects the process? (Hint: almost everything is harder!)
The Vs of big data (3Vs originally, then 7, now 42)
Volume: “billions”, “petabytes” are common
Velocity: think Twitter, fraud detection, etc.
Variety: text (webpages), video (youtube)…
Veracity: uncertainty of data
Variability
Visualization
Value
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Disseminationhttp://www.ibmbigdatahub.com/infographic/four-vs-big-datahttp://dataconomy.com/seven-vs-big-data/https://tdwi.org/articles/2017/02/08/10-vs-of-big-data.aspx
![Page 5: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/5.jpg)
Two Example Projects from Polo Club
![Page 6: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/6.jpg)
Apolo Graph Exploration: Machine Learning + Visualization
6
Apolo: Making Sense of Large Network Data by Combining Rich User Interaction and Machine Learning. Duen Horng (Polo) Chau, Aniket Kittur, Jason I. Hong, Christos Faloutsos. CHI 2011.
![Page 7: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/7.jpg)
7
![Page 8: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/8.jpg)
7
Beautiful HairballDeath StarSpaghetti
![Page 9: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/9.jpg)
Finding More Relevant Nodes
HCIPaper
Data MiningPaper
Citation network
8
![Page 10: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/10.jpg)
Finding More Relevant Nodes
HCIPaper
Data MiningPaper
Citation network
8
![Page 11: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/11.jpg)
Finding More Relevant Nodes
Apolo uses guilt-by-association(Belief Propagation)
HCIPaper
Data MiningPaper
Citation network
8
![Page 12: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/12.jpg)
Demo: Mapping the Sensemaking Literature
9
Nodes: 80k papers from Google Scholar (node size: #citation) Edges: 150k citations
![Page 13: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/13.jpg)
![Page 14: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/14.jpg)
![Page 15: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/15.jpg)
Key Ideas (Recap)Specify exemplarsFind other relevant nodes (BP)
11
![Page 16: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/16.jpg)
What did Apolo go through?
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
Scrape Google Scholar. No API. 😩
Design inference algorithm (Which nodes to show next?)
Paper, talks, lectures
Interactive visualization you just saw
You will a new Apolo prototype (called Argo)
![Page 17: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/17.jpg)
13Apolo: Making Sense of Large Network Data by Combining Rich User Interaction and Machine Learning. Duen Horng (Polo) Chau, Aniket Kittur, Jason I. Hong, Christos Faloutsos. ACM Conference on Human Factors in Computing Systems (CHI) 2011. May 7-12, 2011.
![Page 18: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/18.jpg)
NetProbe: Fraud Detection in Online Auction
NetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks. Shashank Pandit, Duen Horng (Polo) Chau, Samuel Wang, Christos Faloutsos. WWW 2007
![Page 19: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/19.jpg)
Find bad sellers (fraudsters) on eBay who don’t deliver their items
NetProbe: The Problem
Buyer
$$$
Seller
15
Non-delivery fraud is a common auction fraudsource: https://www.fbi.gov/contact-us/field-offices/portland/news/press-releases/fbi-tech-tuesday---building-a-digital-defense-against-auction-fraud
![Page 20: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/20.jpg)
16
![Page 21: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/21.jpg)
NetProbe: Key Ideas§ Fraudsters fabricate their reputation by
“trading” with their accomplices§ Fake transactions form near bipartite cores§ How to detect them?
17
![Page 22: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/22.jpg)
NetProbe: Key IdeasUse Belief Propagation
18
F A HFraudster
AccompliceHonest
Darker means more likely
![Page 23: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/23.jpg)
NetProbe: Main Results
19
![Page 24: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/24.jpg)
20
![Page 25: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/25.jpg)
20
![Page 26: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/26.jpg)
20
“Belgian Police”
![Page 27: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/27.jpg)
21
![Page 28: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/28.jpg)
What did NetProbe go through?
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination
Scraping (built a “scraper”/“crawler”)
Design detection algorithm
Not released
Paper, talks, lectures
![Page 29: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/29.jpg)
23NetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks. Shashank Pandit, Duen Horng (Polo) Chau, Samuel Wang, Christos Faloutsos. International Conference on World Wide Web (WWW) 2007. May 8-12, 2007. Banff, Alberta, Canada. Pages 201-210.
![Page 30: Analytics Building Blockspoloclub.gatech.edu/cse6242/2018spring/slides/CSE... · Duen Horng (Polo) Chau Assistant Professor Associate Director, MS Analytics Georgia Tech Partly based](https://reader030.vdocuments.net/reader030/viewer/2022041114/5f215c7512bcde298b55eb20/html5/thumbnails/30.jpg)
Homework 1 (out next week; tasks subject to change)
• Simple “End-to-end” analysis
• Collect data using Twitter API
• Store in SQLite database
• Great graph from data
• Analyze, using SQL queries (e.g., create graph’s degree distribution)
• Visualize graph using Gephi
• Describe your discoveries
Collection
Cleaning
Integration
Visualization
Analysis
Presentation
Dissemination