oracle autonomous data warehouse cloud...–oracle autonomous data warehouse cloud with 16 ocpus and...

Copyright © 2017 Oracle and/or its affiliates. All rights reserved. |

Oracle Autonomous Data Warehouse Cloud

Date: March 22, 2018Time: 8:30 AM ~ 09:30 AMLocation: Spencer Fox Eccles Business Building

1655 Campus Center Dr., Salt Lake City, UT 84112

Room: 0110, Lower Level

Daniel LiuPrincipal Solution Architect

For Oracle Employees and authorized partners only

UTOUG Training Days 2018

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted


Oracle Database 9i, 10g

• Automatic Storage Management (ASM)

• Automatic Memory Management

• Automatic DB Diagnostic Monitor (ADDM)

• Automatic Workload Repository (AWR)

• Automatic Undo tablespaces

• Automatic Segment Space Management

• Automatic Statistics Gathering

• Automatic Standby Management (Broker)

• Automatic Query Rewrite

Oracle Database 11g, 12c

• Automatic SQL Tuning

• Automatic Workload Replay

• Automatic Capture of SQL Monitor

• Automatic Data Optimization

• Automatic Storage Indexes

• Automatic Columnar Cache

• Automatic Diagnostic Framework

• Automatic Refresh of Database Cloning

• Autonomous Health Framework

4

Journey to Autonomous Data Warehouse• Oracle has been developing sophisticated database automation for decades


Journey to Autonomous Data Warehouse

• With Engineered Systems, Oracle automated database infrastructure

5

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 6

Next Generation of Cloud…


Why You Need An Autonomous Cloud

Less time on Administration

• Less time on infrastructure

• Less time on patching, upgrades

• Less time on ensuring availability

• Less time on tuning

7

More time on Innovation

• More time on database design

• More time on developing new apps

• More time on data analytics

• More time on enriching data sets

Challenge: There are more data management tasks than humans to do the work


Why You Need An Autonomous Cloud

Less time on Administration

• Less time on infrastructure

• Less time on patching, upgrades

• Less time on ensuring availability

• Less time on tuning

8

More time on Innovation

• More time on database design

• More time on developing new apps

• More time on data analytics

• More time on securing data

Challenge: There are more data management tasks than humans to do the work

Payoff is huge - eliminate generic tasks, reduce labor, reduce costs, reduce errors, while

increasing security and availability


AutonomousData Warehouse


Introducing: Autonomous Data Warehouse Cloud

• Easy–Automated management

–Automated tuning: Simply load data and run

• Fast–Based on Oracle’s unique data warehouse technology

• Elastic– Instant scaling of compute or storage with no downtime

10

Expected CY 2017


EasySimple provisioning


Getting Started with Autonomous Data Warehouse Cloud

• Provisioning requires only 5 simple questions:

– Database name?

–Which data center?

– How many CPU’s?

– How many TB’s?

– Admin password?

• New service created in <30 seconds (regardless of size)– Ready to connect via sqlnet


Demo: Provisioning a new ADWC


What You Don’t See Is..

• No decisions for:– Details of the database software

– Configuration of hardware

– Characteristics of the database

– Architecture for backups and availability

• All of this (and more) is automatically configured


EasyAutomated Management


Automated management

• Oracle automates end-to-end management of data warehouse

– Provisioning new database instances

– Growing/shrinking storage and/or compute

– Patching and upgrades

– Backup and recovery

• Full lifecycle managed using ADWC Service Console

21


Demo: ADWC Management Console


EasyAutomated Tuning


Automated Tuning

• “Load and go”

– Define tables, load data, run queries• No tuning

• No special database expertise required

– Good performance out of the box

• Query using any business analytics tool or cloud service – Built-in SQL notebook also included


Demo: Monitoring ADWC







30

Expected CY 2017


FastFast straight out of the box


• Evaluation of real customer workloads and standard benchmarks

– Performance As-Is versus ADWC

– Run on identical hardware

• ADWC out-of-the-box–Optimized, pre-configured DB setup

• Schema as provided by customer

– Indexes, Partitioning, manual DOP

Performance Proof Points for ADWC

WorkloadOracle ADWC

Customer Schema

NetResult

Customer #1 39 34 …as Good

Customer #2 52 53 Faster

Customer #3 1032 911 …as Good





MSTR 1163 1504 Faster

SSB 232 209 …as Good


Benchmarks - Overview of Schema

• Real world customer workloads

• Workloads consists of medium to complex database schemas

– 20 to 100++ tables –

• Sample queries with varying complexities, representing the customer’s most common data analysis.

– Sample queries were run sequentially on each platform and the total runtime was measured.

Confidential


Overview of Cost Comparisons

• Pricing is based on published, publicly available price lists

• Cost the price/performance was calculated based on the processing time and the platform

– All costs are based on 24*7 utilization

• Resources allocated to all platforms are comparable in terms of computational power and storage (as close as possible)

– Resource utilization of queries decided by the system ..

• System configurations:

–Oracle Autonomous Data Warehouse Cloud with 16 OCPUs and the minimal amount of storage necessary for the various workloads, in increments of 1TB.

– Redshift Dense Compute systems, 8 nodes of ds2.xlarge

Confidential

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Confidential

Scenario: ADWCS 8OCPUs / 16OCPUs vs. Amazon Redshift

Same workload on ADWC takes ½ the time, with 2x the resources, cost

remains the roughly the same

because more hardware=higher

cost!







38

Expected CY 2017


ElasticOn-Demand, Instant Scaling


Scaling Out is not Resizing!

Select bigger sized building-block Create required metadata

Put cluster in Read-Only Mode Rebuild

There has to be a better, simpler, more professional way to scale out


Instant Elasticity: Pay for Exactly What you Use

• Size the DW to the exact number of OCPU’s and TB’s required

–Not constrained by fixed building blocks

• Scale the DW on demand

– Independently scale compute or storage

– Resizing occurs instantly, fully online

• Shut off idle compute save money

– Restart instantly


Instant Elasticity

• Customer specifies number of database cores (in OCPU) and database size (in TB)

– CPU’s and storage are independent

– Databases can grow and shrink in either dimension

– Pricing based upon CPU ($/CPU/hour) and Storage ($/TB/Month)

• Examples:

– Adding new storage or compute capacity is just a click on the cloud console

– CPU capacity can be added on-demand for end-of-quarter processing, then reduced after the processing is completed

– CPU capacity can be set to zero, while storage remains in place (for example, to reduce costs for databases which are not used during weekends / nights)


Demo: Scaling ADWC


Enterprise Scaling OutOracle Autonomous Data Warehouse Cloud


Safe and SecureData fully encrypted


Data Security is ON By-Default

• Data is automatically encrypted, by default

– Data is encrypted by default in the cloud, as well as in transit and at rest.

• Oracle staff locked out - well defined “break glass” procedure, with customer approval, for Oracle Operations to access system

• Automated protection from external attacks

– Automatically applies latest security updates online• Quarterly, or off-cycle for high impact security vulnerability

– Native encryption prevents data access from outside the database• Allows keys to be managed by customer’s existing Security Module (HSM)

• Will interface with Database Aware Key Management Cloud Service

47


ROBUSTEnterprise Reliability


Autonomous Data Warehouse Cloud: Reliability

• Enterprise-grade availability – 99.95%– Fault-tolerant, highly-available Exadata infrastructure• Triple-mirrored disks for disk failures

• Real Application Clusters for compute node failures

• And more …

– Automatic backups• Point in time recovery to anytime in last 30 days

• On the roadmap: Mission Critical – 99.995%– Standby database


Automatically Backed Up

• Backups are scheduled on nightly basis

• To Database Backup Cloud Service

• Or Zero Data Loss Recovery Cloud Service in the future for

– Lowest overhead

– Fastest restore

– Lowest data loss

• Retention Time for backups is configurable

50


SAME SKILLSInvestment protection – use your existing DI+BI tools


Supported by a rich Data Warehouse Ecosystem

• Existing tools running on-premise or in Oracle Cloud

– Oracle BI and data-integration tools

– 3rd party BI tools

– 3rd party data-integration tools

• Connectivity via SQL*Net

Oracle Cloud Services

–Oracle Analytics Cloud

–Oracle Data Integration Platform Cloud

–and others…

Oracle Autonomous Data Warehouse Cloud supports:


Built-in SQL Notebooks with Oracle ML

• Collaborative UI for business users and data scientists

– Runs in most browsers

– Easy access to shared notebooks, templates

–Management UI for permissions, scheduler, etc.

• Based on open source Apache Zeppelin

53


Demo: Using SQL Notebooks


Creating Analytical Dashboards


Autonomous Data Warehouse Cloud:Support From Strategic Partners


Demo – Connecting 3rd party applications


Connecting to the Database

• Securely connect using credential wallets via SQL*Net, JDBC, ODBC

• Wallet can be downloaded from the service console or using REST APIs


Location of the wallet

Wallet password

Database service name


Data LoadingNew faster, simpler ways to load data into your ADWC


Data Loading Options

• Data loading via SQL*Net

– Suitable for small volumes of data• SQL*Loader from local filesystem

• ETL scripts that use DML to insert/update data

• Data loading from Oracle Object Storage

– Preferred technique for large volumes of data• Additionally enables data-sharing with other cloud services

– Stage data in Oracle Object Storage, then load into the database using new PL/SQL APIs


Querying Data on the Oracle Object Store

• Define your credentials for the object store

–Oracle Cloud Infrastructure Object Store username and Swift password required

• Credential stored in the database schema once and used for accessing the object store for all queries

• Call the dbms_cloud API for creating an external table on top of the source files


Loading Data from Object Stores

• Source data from files on object stores for data loading

–Oracle Object Store or AWS S3

– Any supported Oracle-loader file format

– Roadmap: any Hadoop file format

ADWC

Object Storage


Demo: Using the Object Store


New Cloud API to Access Object Stores, DBMS_CLOUD

• New PL/SQL package for accessing files in object stores

• No need to manually define external tables for loading files

–Makes it easier to specify the format of the source data


Loading Data from the Oracle Object Store

• Define your credentials for the object store

–Oracle Cloud Infrastructure Object Store username and Swift password required

• Credential stored in the database schema once and used for accessing the object store for all loads

begin

dbms_cloud.create_credential(

credential_name => 'OBJ_STORE_CRED',

username => 'tenant1',

password => ’password'

);

end;

/


Loading Data from the Oracle Object Store

• Load data directly into the target table without any intermediate steps

• Data format in the source file easily specified as JSON

begin

dbms_cloud.copy_data(

table_name =>'CHANNELS',

credential_name =>'OBJ_STORE_CRED',

file_uri_list =>'https://swiftobjectstorage.us-ashburn-

1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',

format => json_object('ignoremissingcolumns' value 'true',

'removequotes' value 'true')

);

end;

/


Querying Data on the Oracle Object Storebegin

dbms_cloud.create_external_table(

table_name =>'CHANNELS_EXT',

credential_name =>'OBJ_STORE_CRED',

file_uri_list => 'https://swiftobjectstorage.us-ashburn-

1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',

format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value

'true'),

column_list => 'CHANNEL_ID NUMBER,

CHANNEL_DESC VARCHAR2(20),

CHANNEL_CLASS VARCHAR2(20),

CHANNEL_CLASS_ID NUMBER,

CHANNEL_TOTAL VARCHAR2(13),

CHANNEL_TOTAL_ID NUMBER'

);

end;

/

select count(*) from channels_ext;


Roadmap: Queries across Cloud Storage and more!

• External tables can be created on data in object stores

–Oracle Object Store or AWS S3

– Any supported Oracle-loader file format• Or any Hadoop file format (e.g. Parquet) – coming soon

– Big Data SQL integration – coming soon

• Seamless queries over object store

• Seamless queries across object store and database

Elastic Data Warehouse Compute

Data Storage

ADWCStorage

Object Storage

Kafka HDFS

Data Storage


MigrationsMoving your DW to ADWC


Easy Migration to Autonomous Data Warehouse Cloud

Migration of existing Oracle systems

• Data pump import using ADWC-aware settings

– Ensures compatible data import

– Violating data structures will be ignored or converted

Migration of Redshift systems

• End-to-end migration using migration workbench

– Convert and create data warehouse schema

– Unload Redshift data into S3

– Load data into ADWC

Migration of other systems

• Migration workbench assistance or manual

77


ArchitectureOverview ADWC Ecosystem


Architecture for Autonomous Data Warehouse

Oracle Exadata Cloud Service

Oracle Database Cloud Service

Express Cloud Service

Data Warehouse Services(EDWs, DW, departmental marts and sandboxes)

Autonomous Data Warehouse Cloud

Service Console

Built-in Access Tools

Oracle ML

Service Management

Data Warehouse

SQL Developer

Developer Tools

Data IntegrationServices

3rd Party BI on-premises

Oracle Data Integration Cloud Service

3rd Party DI on Oracle Cloud Compute

3rd Party DI On-premises


SummaryAutonomous Data Warehouse Cloud


Nearly 4 decades of enterprise innovations now available in the cloud

Industry Leading Data Management Expertise

37 YearsOf Data Management Experience

10 YearsOf Engineered Systems Innovation

Thousands Of GlobalEnterprise Customers

20 Cloud Data Centers World Wide

✓

✓

✓


Highly ElasticQuick and easily scale up and down compute and storage without having to over pay for fixed blocks of resources

Built-in Web-Based NotebookApache Zeppelin based notebooks ready to run from your browser

Database migration utilityDedicated cloud-ready migration tools for easy migration from AmazonAWS Redshift, SQL Server and other databases

Enterprise Grade SecurityData is encrypted by default in the cloud, as well as in transit and at rest.

High-Performance Queriesand Concurrent WorkloadsOptimized query performance with preconfigured resource profiles for different types of users

Oracle SQLADWC is compatible with all business analytics tools that support Oracle Database

Self DrivingFully automated database for self-tuning patching, upgrades itself while the system is running

Cloud-Based Data LoadingFast, scalable data-loading from Oracle Object Store, AWS S3, or on-premises

Oracle Autonomous Data Warehouse Cloud Key Features


Autonomous Data Warehouse Cloud

• Easy

– Fully-managed, pre-configured and optimized for DW workloads

– Simply load data and run

• No need to define indexes, create partitions, etc.

• Fast– Based on Exadata technology

• Elastic

– Instant scaling of compute or storage with no downtime

oracle autonomous data warehouse cloud...–oracle autonomous data warehouse cloud with 16 ocpus and...

Documents