oracle cloud : big data use cases and architecture

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Guido Guidi Principal Sales Consultant

1

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

Oracle Big Data Architectures, news from OOW


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2


Implementing Hadoop Infrastructure Can Be Hard

• Building your own is complex, risky and time consuming

No compatible public cloud options if you do

• Using a generic public cloud brings its own challenges

No compatible on-premises option if you do

• Focus should be on time-to-value and agility

3

Generic IaaS for Big Data Infrastructure Challenges

• Like building your own infrastructure, except in the cloud, has similar challenges

• On-going responsibility for support and enhancements

• Effort required gets in the way of business goal: using Hadoop to gain deeper business insight

• No on-premises equivalent

Building Your Own Can Impact Business Outcomes

• Burns precious time and skills, may produce uncertain results

• Considerable ongoing operational effort: upgrades, rebalancing, tuning, patching, support

• Both get in the way of the business goal: using Hadoop to gain deeper insights

• No cloud equivalent

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 4

Deliver Big Data Results, Speed Time to Value with Oracle

Optimized public cloud infrastructure, with rich set of tools, workflows

and data sources

Oracle Big Data Cloud service model delivered

in your data center, behind your firewall

On-premises engineered system designed to deliver predictable Hadoop

infrastructure


infrastructure


Big Data on Premise

• Engineered and optimized for Big Data on-premises

• Co-developed with Cloudera

• Eases implementation, operations and growth

• Extended and enhanced by optional Oracle software

• Proven performance, lower cost than build-your-own

• Compatible public cloud equivalents


Oracle Big Data Appliance


Elastically Scale-Out from Starter Rack to Multi-Rack

Starter Full

Multi-Rack • Start with six BDA server nodes and all switches

Add BDA nodes as needed Grow up to 18 racks and 324 nodes in a single cluster Can be configured as single tenant or multi-tenant

• Can expand older machines with new generation servers

HC

6


Big Data Appliance Has Lower Total Cost of Ownership • Significant savings over Build-Your-Own Hadoop cluster

$0

$200.000

$400.000

$600.000

$800.000

$1.000.000

$1.200.000

$1.400.000

Build Your Own Big Data Appliance

Thre

e-y

ear

TC

O

SoftwareLicenses, AllSupport

All hardware

Source: Nik Rouda, and Adam DeMattia, ESG: The Surprising Economics of Engineered Systems for Big Data (with Oracle® and Intel®) December 2015

Three-Year TCO

45% less 45% Lower 3-Year

TCO

http://www.oracle.com/us/technologies/big-data/eng-systems-for-big-data-esg-wp-2852701.pdf




Oracle Big Data Appliance

2X Faster Performance than Do-It-Yourself

Source: Intel White Paper: “Deploying an Apache Hadoop* Cluster? Spend Your Time on BI, Not DIY” September 2015

8

http://www.oracle.com/ocom/groups/public/@otn/documents/webcontent/2745435.pdf










and data sources




infrastructure


and data sources


Hadoop in the Cloud – Two Usage Patterns

• Short-Lived Clusters

– Data is repurposed, and used for a specific use case in a specific workload. Cluster is spun up when needed only

• Key Requirements

– Flexibility • Spin up arbitrary number of nodes quickly

• Expand quickly from very small to very large

• Low management overhead

– Simplicity • Use as is, solve problem, move on

• Long-Lived Clusters

– Data is acquired and augmented continuously, cluster is in permanent use for mixed workloads


– Performance • Raw compute performance across wide

range of workloads

• Time to Availability

– Control of environment • Often requires 3rd party utilities and tuning

for workloads


Oracle Hadoop Offerings in the Cloud – Two Usage Patterns

Short-Lived Clusters


– Flexibility

– Simplicity

• Oracle Big Data Compute Edition –Managed Spark Service

–Managed HDFS Service

Long-Lived Clusters


– Performance

– Control of environment

• Oracle Big Data Cloud Service – Full Cloudera Eco-System

– Engineered Systems backbone


• SHACK (Spark, Hadoop, Akka, Cassandra, Kafka) delivered as a Managed Cloud Service

- Using Hadoop Distribution

- Leveraging Lambda architectural concepts

• Start with 1 node cluster, 2OCPU and scale up/down as needed (up to 100 nodes)

- Independently elastic Storage and Compute with flexible purchase options

- Leveraging Lambda architectural concepts

• Big Data Platform available for new managed Big Data Services

- Big Data Discovery

- IoT Analytics

- Big Data Preparation

- Dala Flow Machine Learning

- Mobile Analytics

Oracle Big Data Cloud Service Compute Edition

Metered, Non Metered Subscription

Oracle Managed Chicago, Ashburn, Slough, Amsterdam

Short-Lived Clusters


Oracle Big Data Cloud Service

• Purpose-built cloud service for big data workloads

• Dedicated and elastic options

• Enhanced with tools, workflows, rich data sources

• Oracle upgrades patches, support and maintains

• Clear and transparent pricing

• Seamlessly works with on-premises Big Data Appliance



Key Features • Dedicated and Elastic Options

• Same software as Big Data Appliance, plus

– Oracle Big Data Connectors

– Oracle Big Data Spatial and Graph

– Oracle Data Integrator Enterprise Edition

• Integrates With Other Oracle Big Data Services

– Big Data Discovery

– Big Data Preparation

– Big Data SQL

– Big Data Visualization

– Oracle Data-As-A-Service (DaaS)

Benefits • Convenient, cost-effective and flexible

• Secure by default

• Comprehensive software stack

• High performance

14

Un-metered Subscription

Oracle Managed

Long-Lived Clusters

Oracle Big Data Cloud Service


Oracle Big Data Cloud Service Dedicated Compute Bursting

• Self Service, on-demand addition of OCPUs and Memory to Cluster – Large expansion chunks with 32 OCPU’s and 256GB of

memory

– Expansion nodes are automatically instantiated as cluster nodes and are shut down when jobs are completed

– Burstable ceiling of 192 OCPUs and 1.5TB memory per cluster • Enables massive workload scalability

– Bursting nodes share InfiniBand fabric • Enables remote execution without network impact

– Hourly Billing rates

• Always Dedicated Compute Capacity

Bu

rst

No

de

s P

ers

iste

nt

No

de

s




and data sources




infrastructure


infrastructure




Oracle Big Data Cloud Machine*

• Big Data Cloud Service, delivered in your data center, behind your firewall

• Near-zero operational effort

• Runs Oracle and non-Oracle software

• Same clear and transparent pricing, pay for what you use

• Complete compatibility with public Oracle Cloud


*planned release

17


Key Features • Hadoop, Spark delivered as a Cloud Machine

– Cloudera Enterprise – Data Hub Edition 5.x –Oracle Big Data Connectors –Oracle Big Data Spatial and Graph –Oracle Data Integrator Enterprise Edition

• Same Infrastructure as in Oracle Big Data Cloud Service

–Oracle Managed and Tested – Start small and grow seamlessly in your data

center

Benefits

• Consistently high performance

• Secure by Default

• Comprehensive Software Stack

Oracle Big Data Cloud Machine The Oracle Cloud@Your Home

@Customer Data Center Subscription

Oracle Managed


Enterprise Management Strategy

Single pane of glass for managing

• Across the stack

– Provide unified solution for hardware and software management

– Complete solution for performance management, lifecycle management and cloud management

• Across on-premise and Oracle Cloud

– Provide comprehensive hybrid cloud management at-par with on-premise capabilities


Security made Easy Key Features

• Kerberized Cluster out of box – Apache Sentry Enabled on Secure Clusters

• Data Encryption built in – At Rest through HDFS Encryption

– In flight for all phases within Hadoop and Spark

• Encrypted Traffic to all Client Tools

• VPN Service

Key Benefits

• Reduced Risk

• Faster Time to Value

Oracle Big Data Security


Deployment choice: on-premises, public cloud on premises, public Oracle Cloud

Precise Equivalents in Different Consumption Models

Same Standards Same Products

Unified Management


ON-PREMISES


Maximum Availability Architecture for Big Data Appliance

White paper available : http://www.oracle.com/technetwork/database/availability/bda-maa-2942174.pdf

Key Features

• Tight integration with Exadata to create a Big Data Management System – Infiniband high speed low latency connection

– Oracle Big Data SQL enables the power of Oracle SQL and provides a single view of data across database and hadoop

• Oracle Data Guard is used to maintain synchronized a standby Oracle database

• Data Replication to a second BDA ensures high availability and data consistency

• The Big Data Management System can be used both in cloud and on premise

Key Benefits

• Designed to tolerate unplanned outages

• End to end application availability

MAA architecture diagram for BDA

oracle cloud : big data use cases and architecture

Technology