![Page 1: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/1.jpg)
From Internet Usage to Genomics Analysis
with Google BigQuery
Mr. Kanakorn Horsiritham
Prince of Songkla University, THAILAND
![Page 2: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/2.jpg)
Agenda
Big Data
Big Data Analysis
Big Data Tools
Google BigQuery
Prince of Songkla University
Analysis
Google Genomics
Conclusion
![Page 3: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/3.jpg)
Big Data
http://www.viawest.com/sites/default/files/asset/document/ViaWest_Big_Data_Infographic.pdf
(2011)
![Page 4: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/4.jpg)
Source: http://www.intel.com/content/www/us/en/communications/internet-minute-infographic.html
(2013)
![Page 5: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/5.jpg)
Big Data
![Page 6: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/6.jpg)
Big Data Analysis The process of examining big data to uncover
hidden patterns
unknown correlations
useful information
Method
Machine Learning
Data Visualization
Etc ...
http://www.sas.com/en_us/insights/analytics/big-data-analytics.html
![Page 7: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/7.jpg)
Big Data Tools
Tools:
- Hadoop
- Google BigQuery
![Page 8: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/8.jpg)
Hadoop
http://blog.agro-know.com/wp-content/uploads/2015/07/Hadoop_Ecosystem.jpg
![Page 9: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/9.jpg)
Google BigQuery
Google BigQuery
Google Cloud Storage
SQL API
Corperate Data
BI Tools
Google Sheets
Co Worker
Store Data in the cloud Analyze interactively Securely Share
Distribute the Result
![Page 10: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/10.jpg)
Google BigQuery
https://cloud.google.com/bigquery/pricing
Free
Loading/Export Data
Storage Price
$0.02/GB/Month ($20/TB/Month) Storage pricing is prorated per MB, per second.
For example, if you store 1 TB for half of a month, you pay $10
Query Price
First 1 TB of month is FREE
$5/TB
![Page 11: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/11.jpg)
Google BigQuery
Only $7 / Month
![Page 12: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/12.jpg)
Internet Usage Prince of Songkla University
![Page 13: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/13.jpg)
Now, we are here !
Prince of Songkla University
![Page 14: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/14.jpg)
Hadyai
Pattani
Trang
Phuket
Surat
Prince of Songkla University
5 Campus
![Page 15: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/15.jpg)
Credit: Facebook "Care Chayada"
![Page 16: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/16.jpg)
Credit: Facebook "Care Chayada"
![Page 17: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/17.jpg)
![Page 18: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/18.jpg)
![Page 19: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/19.jpg)
![Page 20: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/20.jpg)
![Page 21: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/21.jpg)
![Page 22: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/22.jpg)
![Page 23: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/23.jpg)
![Page 24: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/24.jpg)
Schema
![Page 25: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/25.jpg)
![Page 26: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/26.jpg)
![Page 27: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/27.jpg)
Analysis
![Page 28: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/28.jpg)
How each application uses internet traffic?
![Page 29: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/29.jpg)
How each application uses internet traffic?
![Page 30: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/30.jpg)
How each application uses internet traffic?
![Page 31: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/31.jpg)
What are the URL Category of those SSL Applicaiton?
![Page 32: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/32.jpg)
What are the URL Category of those SSL Applicaiton?
![Page 33: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/33.jpg)
What are the URL Category of those SSL Applicaiton?
![Page 34: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/34.jpg)
How these top 5 applications are used in each hour of day?
![Page 35: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/35.jpg)
How these top 5 applications are used in each hour of day?
![Page 36: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/36.jpg)
![Page 37: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/37.jpg)
How these top 5 applications are used in each hour of day?
![Page 38: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/38.jpg)
How users use internet during a week?
![Page 39: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/39.jpg)
How users use internet during a week?
![Page 40: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/40.jpg)
How users use internet during a week?
![Page 41: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/41.jpg)
![Page 42: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/42.jpg)
How users use internet in week day and weekend?
![Page 43: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/43.jpg)
How users use internet in week day and weekend?
![Page 44: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/44.jpg)
How users use internet in week day?
![Page 45: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/45.jpg)
How users use internet in weekend?
![Page 46: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/46.jpg)
What are the most popular social networking?
![Page 47: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/47.jpg)
What are the most popular social networking?
![Page 48: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/48.jpg)
What are the most popular social networking?
![Page 49: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/49.jpg)
How different do students and staffs use social
networking?
![Page 50: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/50.jpg)
How different do students and staffs use social networking?
![Page 51: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/51.jpg)
How different do students and staffs use social networking?
![Page 52: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/52.jpg)
Which Faculty their students use 'Reference and Research' category?
![Page 53: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/53.jpg)
![Page 54: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/54.jpg)
![Page 55: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/55.jpg)
Google Genomics
![Page 56: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/56.jpg)
Google Genomics
![Page 57: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/57.jpg)
Genomic Browser http://gabrowse.appspot.com/
![Page 58: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/58.jpg)
Genomic Browser http://gabrowse.appspot.com/
![Page 59: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/59.jpg)
VCF Format
![Page 60: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/60.jpg)
Upload into Google Cloud Storage
![Page 61: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/61.jpg)
Google Genomics API
![Page 62: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/62.jpg)
Export to Google BigQuery
![Page 63: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/63.jpg)
![Page 64: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/64.jpg)
![Page 65: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/65.jpg)
![Page 66: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/66.jpg)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 50000000 100000000 150000000 200000000 250000000 300000000
titv
titv
![Page 67: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/67.jpg)
Conclusion
![Page 68: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/68.jpg)
Conclusion
Google BigQuery
• easy to use and not expensive. Just prepare you data, upload and then query
• Process large data within a few seconds
PSU Internet Usage Analysis
• Heavy traffic : video streaming, i.e. youtube and other in the form of web browsing
• When count with number of users, Web browsing and Facebook are used by most users.
![Page 69: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/69.jpg)
Conclusion
• Internet usage peak on workday by office hours and fade down after office hours and weekend
• Staff use internet weekday by office hours and other used by student alone.
• Top 3 Social Networking are Facebook, LINE and Twitter. Both staff and
student use Facebook the most but staff like twitter more than LINE while
student like LINE more than Twitter.
• And Faculty of Science, Mangement Science and Engineering user internet
for Education Purpose the most.
![Page 70: PRESENTATION: From Internet Usage to Genomics Analysis with Google BigQuery](https://reader036.vdocuments.net/reader036/viewer/2022062809/5695d2681a28ab9b029a4d5d/html5/thumbnails/70.jpg)
Conclusion
Google Genomics
• Upload the VCF files created by DNA Sequencer to Google Cloud Storage
• Process with Google Genomics
• Export to Google BigQuery
• Then query the data set