![Page 1: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/1.jpg)
Nico Gaviola Head of Healthcare and Lifesciences UKIE [email protected]
Data exploration at the speed of thought Lessons learned from inside Google
![Page 2: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/2.jpg)
Google’s m ission is t o organize t he wor ld’s inform at ion and m ake it universally accessible and useful.
Sundar Pichai CEO, Google
![Page 3: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/3.jpg)
![Page 4: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/4.jpg)
uploads per minute
users
search index
query response time
500hrs
1B+
100PB+
0.25s
Google computing scale
![Page 5: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/5.jpg)
Hitting the limits, early on...
The Anatom y of a Large -Scale Hype rtextua l Web Search Engine 1996, Se rgey Brin and Lawrence Page Com pute r Science Departm ent, Stanford Unive rsity, Stanford , CA 94305
![Page 6: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/6.jpg)
2012 2013 2002 2004 2006 2008 2010
Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2009 http://research.google.com/pubs/pub35290.html
GFS
MapReduce
BigTable
Single Node to Cluster
![Page 7: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/7.jpg)
Google’s Data Research
2002 2004 2006 2008 2010 2012 2014 2016
GFS
MapReduce TensorFlow
BigTable
Dremel
Colossus
Flume
Megastore
Spanner
Millwheel
PubSub
F1
![Page 8: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/8.jpg)
Google’s Data Products
2002 2004 2006 2008 2010 2012 2014 2016
ML
PubSub
DataFlow
DataStore
DataFlow
Cloud Storage
BigQuery
BigTable
DataProc
Cloud Storage
![Page 9: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/9.jpg)
Programming
Resource provisioning
Performance tuning
Monitoring
Reliability Deployment & configuration
Handling growing scale
Utilization improvements
Typical Big Data Jobs
![Page 10: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/10.jpg)
Big Data with Google Focus on ins ights . Not infras tructure.
Programming
Understanding
![Page 11: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/11.jpg)
Google’s Big Data Vision
Pay $5 per TB
![Page 12: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/12.jpg)
Active contributor to numerous OSS projects
Make migrations eas ier with open APIs
Cus tomers s hould us e us becaus e they love us , not becaus e they are unable to move off
Open Source & APIs
12
![Page 13: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/13.jpg)
Confidential & Proprietary Google Cloud Pla tform 13
You own your data and remain Data Controller
You can delete or remove your data a t
any time
Google does not s hare your content or
pers onal information google.com/ privacy
Strict Internal Policies : a ll acces s es to
cus tomer or cons umer data applications are
logged
Internal data acces s auditing tracks
Googlers
Google Security Model & You!
![Page 14: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/14.jpg)
Example
![Page 15: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/15.jpg)
“Right at the start of the partnership we were able to reduce time to insight from 96 hours to 30 minutes by using BigQuery”
Gary Sanders Head of Digital Analytics
![Page 16: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/16.jpg)
What’s Next?
![Page 17: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/17.jpg)
“Machine learning is a core, trans formative way by which we’re re-thinking how we’re doing everything”
Sundar Pichai CEO, Google
![Page 18: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/18.jpg)
15% reduction in PUE
![Page 19: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/19.jpg)
Fully trained, easy to use Machine Learning models
Cloud Trans la te
Cloud Vis ion
Cloud Speech
Cloud Natural Language Stay tuned…
![Page 20: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/20.jpg)
Use your own data to train models
Cloud Storage BigQuery Cloud Datalab
Cloud Machine Learning
Develop, Model, Train, Tes t
![Page 21: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/21.jpg)
One more thing
![Page 22: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/22.jpg)
Free training courses coming near you!
![Page 23: Data exploration at the speed of thought · nicogaviola@google.com Data exploration at the speed of thought . ... Sundar Pichai . CEO, Google . uploads per minute . users . search](https://reader030.vdocuments.net/reader030/viewer/2022040511/5e5d1ee0daf3cc38345a232b/html5/thumbnails/23.jpg)
Thank you!