aws big data platform_final_fra.pptx

Antonio Alvarez | EMEA BDM for Big Data E-mail: antog@amazon.com @A_AlvarezGarcia

AWS  Big  Data  Pla-orm  

Better Visibility of Your Business

Big  Data  &  The  Cloud  

BIG DATA Platform:

Big Data Challenges:

Capacity Planning & Scalability

Lower Cost, OpEx

Experiment & learn more

Advanced profiles

IT Complexity

Data Variety…

..Volume, velocity Old Answers

& Questions

Managed Services

Fully managed,

secured & automated

services that brings agility &

focus

S3, EMR, Kinesis, Redshift,

DynamoDB:

Collect all data, do Complex

computations and processing it, both in Real-Time &

Batch

Sensors (IoT)

Social

Images

Videos

E. Apps.

Documents

Web Logs

Big Value

Machine Learning

Easy deployment of ML powerful models without the need of ML Experts ready to

be used

Virtually unlimited &

Elastic Resources

No heavy lifting & Reduced Time to Market, parallel processing on

demand

New Answers/questions &

Business Ideas Extract the

meaning from all your data & focus on new business

Ideas, Models, etc..

High Cost & Commitment

IT  Challenges:  SLAs,  Sa;sfac;on,  low  u;liza;on  (all?)  

Massively  Parallel  Processing  (on  demand)  

ON A SINGLE INSTANCE

COST: 4h x $2.1 = $8.4 RENDERING TIME: 4h

ON MULTIPLE INSTANCES

COST: 4 x 1h x $2.1 = $8.4 RENDERING TIME:

Expand to 25 instances

EMR (Steady State)

EMR (Batch Processing)

Shrink to 9 instances

EMR (Steady State)

On and Off Fast Growth

Unpredictable peaks Predictable peaks

USAGE PATTERNS: Flexibility and Agility

Fixed!

Some  References  

netflix

More than 25 Million Streaming Members

   50  Billion  Events  Per  Day  

~10  PB  of  data  stored  in  Amazon  S3  

S3

Data  consumed  in  mul;ple  ways  

S3

EMR

Prod  Cluster  (EMR)

Recommenda;on  Engine  

Ad-­‐hoc  Analysis   Personaliza;on  

EMR

S3EMR

EMR

Prod  Cluster  (EMR)

Query  Cluster  (EMR)

EMR

EMR

Enterprise DWH

AWS  Redshi;  helped  FT  to  increase  performance  (98%  faster  queries),  reduce  TCO  (80%)  and  increase  Agility  

500,000 WRITES PER SECOND DURING SUPER BOWL

FINRA is moving its platform to the AWS Big Data Platform (AWS)

Finra: Financial Industry Regulatory Authority

•  Stores and anlyses: 30B Market events per Day

•  $10 to $20M annual Savings (Estimations)

•  They have increase their Agility, Speed and Cost savings to operate at scale

hVp://aws.amazon.com/solu;ons/case-­‐studies/finra/    

How Much could this cost me? i.e. Real-time Analysis scenario

500MM tweets/day = ~ 5,800 tweets/sec

Kinesis (Ingestion) cost is $0.765/hour

Redshift (DWH) cost is $0.850/hour (for a 2TB node)

S3 (Data Lake) cost is $1.28/hour (no compression)

Total: $2.895/hour

Cost  &  Scale  

Thank you

Contact information: Antonio Alvarez EMEA BDM for Databases & Big Data E-mail: Antog@amazon.com @A_AlvarezGarcia