"big data & machine learning innovation with google cloud platform", shira kimchi...

20
1 Data & Analytics 3rd Generation Data Platform: From Buckets of Bits to Understanding

Upload: dataconomy-media

Post on 16-Apr-2017

318 views

Category:

Data & Analytics


1 download

TRANSCRIPT

Page 1: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

1Data & Analytics

3rd Generation Data Platform: From Buckets of Bits to Understanding

Page 2: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

2Data & Analytics

Data is Everything.How well you use your data can determine the degree of your success.

Page 3: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

3

Street name

Street number

Street View

Sign

Business facade

Sign Business

name

Traffic light

Traffic sign

Street number

Page 4: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

4Data & Analytics

3rd Gen Data Platforms Challenges

Data access to a variety of data sources.

Develop and build analytic models.

Data preparation, exploration and visualization.

Deploy models and integrate them into

business processes and applications.

High performance and scalability for both

development and deployment.

Perform platform, project and model management.

Page 5: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

55

– Doug Cutting, Hadoop Co-Creator

“Google is living a few years in the future and sending the

rest of us messages”

Page 6: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

6Data & Analytics

Google Cloud Platform VisionSingle-node computing

“Some assembly required”True, on-demand cloud

An actual, global elastic cloud

3rd Wave

Invest your energy in great apps

Colocation

Your kit, someone else’s building.

Yours to manage.

1st WaveToday's Cloud:

Virtualized Data CentersStandard virtual kit, for rent. Still

yours to manage.

2nd Wave

Page 7: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

7Data & Analytics

Bridging the Waves Analyze StoreCapture

BigQuery(large scale SQL)

Cloud Machine Learning

Cloud Pub/Sub

Logs, App Engine

BigQuery streaming

Process

Cloud Dataflow(stream and batch)

CloudStorage(objects)

Cloud SQL(SQL)

CloudDatastor

e(NoSQL)

BigQuery(structured

)

Cloud Dataproc (Hadoop & Ecosystem)

Cloud Bigtable(NoSQLHBase)

Cassandra

hBase MongoDBRabbit MQ

Kafka

Wave 2

Wave 3

Visualise

Cloud DataLab (iPython/Jupyter)

Data Studio 360

Tableau

Qlik

Page 8: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

8Data & Analytics

Exploration &

Collaboration

Databases Storage

Data Preparatio

n & Processin

g

Analytics

Advanced Analytics

& Intelligenc

e

Google Cloud Data Platform

Mobile apps

Sensors and

devices

Web apps

Relational

Key-value

Document

SQL

Wide Column

ObjectStream processing

Batch processing

Data preparation

Federated query

Data catalog

Data exploration

Data visualizatio

nDevelopers

Data scientists

Business analysts

Development

environment for

Machine Learning

Pre-Trained Machine Learning models

Data Ingestion

Messaging

Logs

Page 9: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

9Data & Analytics

Data Preparatio

n & Processing

Cloud Dataflow

Cloud Dataproc

Exploration &

Collaboration

Google BigQuery

Cloud Datalab

Google Analytics 360

Cloud Dataproc

Google Cloud Data Platform

Mobile apps

Sensors and

devices

Web apps

Developers

Data scientists

Business analysts

Data Ingestion

Cloud Pub/Sub

App Engine

Databases/Storage

Cloud SQL

Cloud Bigtable

Cloud Datastore

Cloud Storage

Analytics

Google BigQuery

Google Analytics 360

Cloud Dataproc

Google Drive

Advanced Analytics & Intelligenc

e

Cloud Machine Learning

Translate API

Vision API

Speech API

Page 10: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

10Data & Analytics

Managed Data Services - Focus on Insight vs InfrastructurePB+ Scale, No-Ops, Batch & Streaming of Data

Insights/Programming

Resource Provisioning

Performance Tuning

Monitoring

Reliability

Deployment & Configuration

Handling Growing Scale

Utilization Improvemen

ts

Insights/Programming

Page 11: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

proprietary & confidential | not for distribution

"We are very excited about the productivity benefits offered by Cloud Dataflow and

Cloud Pub/Sub. It took half a day to rewrite something that had previously

taken over six months to build using Spark"Paul Clarke, Director of Technology, Ocado

http://googlecloudplatform.blogspot.co.uk/2015/08/Announcing-General-Availability-of-Google-Cloud-Dataflow-and-Cloud-Pub-Sub.html

Page 12: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

Hadoop + Local SSD

5X the IOPS at 0.5 the cost of AWS local SSD

Up to 1.5TB per instance

680,000 read IOPS and < 1ms latency 1

2

3

Page 13: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

13Data & Analytics

– Mattias P Johansson, Software Engineer, Spotify

“With Google Cloud Platform, we benefitted by having a virtual supercomputer on demand, without having to deal with all the usual space, power, cooling and networking issues. Just a few years ago, we would have needed to use the largest supercomputers on the planet to do what we’re now able to do with Google”

– Mark Johnson, CEO, Descartes Labs

“Right at the start of the partnership we were able to reduce time to insight from 96 hours to 30 minutes by using BigQuery.”– Gary Sanders, Head of Digital Analytics, Lloyds Banking

Group

“Everyone involved unanimously picked GCP. It came down to this: we believe the core technology is better.”

– Peter Bakkam, Platform Lead, Quizlet

Do you feel this way about your Data Warehouse?

Page 14: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

14Data & Analytics

Data Warehouses/Lakes

Machine Intelligence

Data Warehouse is the foundation of something bigger

Predictive +

Prescriptive analytics

=Advanced analytics

Cloud

On Premises Machine

LearningAPIs

Train your own

Models

Page 15: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

15Data & Analytics

Automatically categorize, and automatically extract value

Evaluate the model by applying it against additional manually categorized data, correct and tune

Machine intelligence is already making a huge differenceand there are many, many more opportunities

Capture lots of examples of correct evaluations for that

categorization, and use them to train an ML

model

Identify categorizations that

provide value, categories you’re

already evaluating for by hand today

1 2 3 4

Page 16: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

16| THE LEADERS CIRCLE

Rapidly accelerating use of deep learning at Google

AlphaGoAndroidAppsGmailMapsPhotosRoboticsSpeechSearchTranslationYouTubeand many others ...

Used across areas:

2012 2013 2014 2015

1500

1000

500

0

Number of directories containing model description files

Page 17: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

171717

BETAGA BETA

CloudTranslate API

CloudVision API

CloudSpeech API

Natural Language API

GA

Ready to use Machine Learning models

Page 18: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

18Data & Analytics

Machine learning will drive every successful huge IPO win in the next 5

years.“ ”Eric SchmidtExecutive Chairman, Alphabet Inc

Page 19: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

19Data & Analytics

Now

Stores and Analyzes

Next

Understands

Page 20: "Big Data & Machine Learning Innovation with Google Cloud Platform", Shira Kimchi Google Cloud Platform, Business Manager of MEA

20Data & Analytics 20

Thanks!