get your head (and data) into the clouds with sqldb

Get Your Head (and Data) into the Clouds with SQLDB, dashDB, and DB2 on CloudKelly Schlamb, IBM Canada Ltd.Session Code: AN69:45am -10:45am, Thursday, September 17th, 2015Platform: DB2 for LUW

2

Agenda

Cloud Introduction

dashDB

SQL Database (SQLDB)

DB2 on Cloud

Getting Started

Note: In the cloud world, where the development and update of services typically follows an agile model, it is not uncommon for changes to be made to the interfaces of these services over time. Therefore, specific instructions or screen shots shown in this presentation may not match exactly with what you see and experience.

3

Cloud computing, often referred to as simply “the cloud,” is the delivery of on-demand computing resources—everything from

applications to data centers—over the Internet on a pay-for-use basis

Computing as a Service over the Internet

4

Its not about Cloud or On-Premise its about Cloud AND On-Premise

Its About

Through 2020, the most common use of cloud services will be a hybrid model combining on-premises and external cloud services.”

Gartner, Cloud Computing Innovation Key Initiative Overview, July 2014

5

Cloud Drives Better Business Economics

Cost Flexibility1Shifts fixed to variable costPay as and when needed

Business Scalability2Provides limitless, cost-effective computing capacity to support growth

Context-Driven Variability 5User defined experiencesIncreases relevance

Ecosystem Connectivity 6New value nets potential new business

Market AdaptabilityFaster time to marketSupports experimentation

3

Source: IBV Analysis

The number one reason to adopt cloud is NOT cost savings, it is agility. Cloud enables businesses to compete faster. And speed kills, a chess grandmaster would lose to a beginner that got four moves every turn.

Masked Complexity4

Expands product sophisticationSimpler for customers and users

6

Cloud Has Three Main Delivery Models

1.IaaS (Infrastructure as a Service)

2.PaaS (Platform as a Service

3.SaaS (Software-aaS) with BPaaS (Business Process-aaS) as a special case

Infrastructure-as-a-Service

Platform-as-a-Service

DatabasesHadoopIntegrationOther Middleware

Software-as-a-Service

ServersStorage Networking

Applications CommerceAnalytics

Business Process-as-a-Service

Payments, Procurement

App

licat

ions

Mid

dlew

are

Har

dwar

e

Do-It-Yourself / Custom

7

"As-a-Service" Explained

Applications

Data

Runtime

Middleware

Operating System

Virtualization

Servers

Storage

Networking

Infrastructure(as a Service)

Applications

Data

Runtime

Middleware

Operating System

Virtualization

Servers

Storage

NetworkingM

anaged by C

USTO

ME

R

Platform(as a Service)

Applications

Data

Runtime

Middleware

Operating System

Virtualization

Servers

Storage

Networking

Software(as a Service)

Managed by

CU

STOM

ER

8

IaaS – Infrastructure-as-a-Service

Bluemix is IBM’s Platform-as-a-Service (PaaS)PaaS lets clients rent middleware services (databases, app servers, storage, Hadoop) without needing to know anything about the underlying hardware, storage and networking they are housed onPaaS still needs to know how much data you have to storeBluemix runs on top of SoftLayer• The services in Bluemix (the PaaS) are consumers of the storage, compute and network resources

provided by SoftLayer (the IaaS)

Applications run on top of BluemixThe applications (SaaS) are consumers of the middleware services provided by the PaaS

PaaS – Platform-as-a-Service

SoftLayer is IBM’s Infrastructure-as-a-Service (IaaS)IaaS lets clients rent hardware, storage and networking without needing to know anything about the underlying data center facilities Servers come with operating systems to allow initial access but are not patched or managed from the default install by the IaaSIaaS lets you to pick where your server will be located

9

SoftLayer Global Footprint

India India

China (Planned)

China (Planned) TokyoTokyo

Hong KongHong Kong

SingaporeSingapore

MelbourneMelbourne

SeattleSeattle

San JoseSan Jose

Los AngelesLos Angeles

Mexico CityMexico City

DenverDenverChicagoChicago

DallasDallas

HoustonHouston

TorontoToronto

MontrealMontreal

BRAZILBRAZIL

New York CityNew York City

Washington D.C.Washington D.C.

MiamiMiami

LondonLondon FrankfurtFrankfurt

AmsterdamAmsterdam

ParisParis

SydneySydney

AtlantaAtlanta

DATA CENTER & NETWORK POINT OF PRESENCEDATA CENTER & NETWORK POINT OF PRESENCE

NETWORK POINT OF PRESENCENETWORK POINT OF PRESENCE

MilanMilan

PerthPerth

10

Bluemix is an open-standard, cloud-based platform for building, managing, and running applications of all types (web, mobile, big data, new smart devices, and so on…)

Go Live in SecondsZero to running in one click. Development plans deploy in seconds. Enterprise plans deploy in 1-2 days.

DevOpsDevelopment, monitoring, deployment, and logging tools allow the developer to run the entire application.

APIs and ServicesA catalog of IBM, third party, and open source API services allow the developer to stitch an application together in minutes.

On-Premise IntegrationBuild hybrid environments. Connect to on-premise assets plus other public and private clouds.

Flexible Pricing Sign up in minutes. Pay as you go and subscription models offer choice and flexibility.

Layered SecurityIBM secures the platform and infrastructure and provides you with the tools to secure your apps.

What is Bluemix?

11

Bluemix: Public, Dedicated, & Hybrid

Services Integration Portability Flexible Deployment Models

DedicatedPublic Bluemix functionality can be accessed in a cost-effective public cloud or an enterprise class managed private environment hosted on SoftLayer.

Public Bluemix Zero cost entry point

No commitment, PAYG and freemium pricing models

Wide array of IBM and 3rd party services

Backed by IBM’s enterprise caliber security

Deployed in Dallas and London. Additional data centers coming in 2H.

Dedicated Bluemix

Fully redundant, dedicated Bluemix environment, deployed to any SoftLayer data center

VPN for private network access

Data Cache and Cloudant part of every environment

Additional Cloud Data Services– like dashDB– will be introduced as optional upsells

12

Data on Premises – Workload Optimized Products

DataStageOn-Prem to Cloud

ETL Connector

BigInsights• Enterprise class Hadoop and real-time• BigSQL for easier analytics• IBM differentiators like GPFS

Cloudant Local• Massively scalable• Eventual consistency model• Built for Systems of Engagement

DB2 BLU• SQL interface• ACID compliance• Columnar, in-memory performance• DB2 Built for Systems of Insight

DB2• SQL interface• ACID compliance• Flexible HA and DR options• Built for Systems of Record

ANALYTICAL TRANSACTIONAL

UNSTRUCTURED

STRUCTURED

Offerings for structured and semi-structured data, transactional and analytics workloads.Knit together with the InfoSphere IIG technologies for data movement and integration.

13

IBM Cloud Data Services

DataWorksData Refinery

Services

Enterprise Hadoop• Bare metal performance• Build on reference architecture• BigInsights enterprise features

Cloudant DBaaS• Global data distribution• Massively scalable• Eventually consistent data model• Built for mobile, Systems of Engagement

dashDB• SQL interface• MPP• ACID compliance• Columnar, in-memory performance• BLU augmented with Netezza in-DB analytics• Built for Systems of Insight

SQL Database (managed)DB2 on Cloud (hosted)• Power of DB2• SQL interface• ACID compliance• Fast Provisioning• Built for Systems of Record

ANALYTICAL TRANSACTIONAL

UNSTRUCTURED

STRUCTURED

Mixed workloads and data types are knit together with DataWorks for true hybrid services

14

Hardware

Software

Admin

Data

Software

Admin

Data

DataLayer

Gues

swor

k, O

verh

ead,

Ris

k

Roll-Your-Own Hosted Database as a Service

DataWarehouse

Data Warehouseas a Service

DB2 on Cloud

SQL Database

15

DB2-Related Cloud Offerings

Hosted Hosted & Managed DBaaS

Transactional

AnalyticD

B2

on

Clo

ud

dashDB

SQLDB

16

Agenda

Cloud Introduction

dashDB


DB2 on Cloud

Getting Started

17

A data warehousing and analytics solution in the cloud– Rapid deployment of large scale data warehouses– Flexible options for both volume and processing speed– Unified architecture that enables hybrid data processing on-premises & cloud

In-database analytic capabilities for the best analytic performance

Scale out your analytics workloads with dashDB MPP

Focus on the business not the business of IT!

IBM dashDB #knowMOREinaDASH#dashDB#DWaaS Follow @GetdashDB

Data warehouse infrastructure out of your way!

18

IBM dashDB Highlights

Key featuresFully managed data warehouse in the cloudIn-memory acceleration capabilities with columnar technology, advanced compression, and buffer pool technologyIn-database predictive analytics & R-supportIntegrated with Cloudant for analytics on JSON data from web & mobile appsTrue “Load & Go” approach – no need to predefine indexesBuilt-in Netezza analytic algorithmsOracle compatibilityOn disk data encryption and secure,encrypted connectionsMPP scale-out

BLU Acceleration

Netezza In-Database

Analytics

Cloudant Database

as a Service

19

dashDB Key Use Cases• Extend on-premises data warehouse

environments to the cloud• Flexible, cost-effective growth• Hybrid cloud models support ground to cloud

Extend / Modernize

• Easy synchronization of JSON to structured data• Allows analytics via standard BI tools• In-database predictive algorithms allow greater

insight for Cloudant users than ever beforeCloudant Analytics

• Robust predictive analytic algorithms• Integrated with R• Watson Analytics ready• Analytics ecosystem with partners

In-Database Analytics

• Data warehousing and analytics in the cloud• Cloud agility and flexibility• Analytics for cloud data, data marts, and

development & test environments

Data Warehouse & Analytics Service

20

Enterprise MPP

• Dedicated, single tenant environment• Bare metal

• 3 - 5 node clusters

• 24 cores per node• 256 GB memory per node• SSD storage (for about 4 TB of

preload data per node)

• $5250 / month (USD) per node

A Wide Range of dashDB Plans to Suit Your NeedsEntry

• Shared, multi-tenant environment

• 20 GB SAN storage capacity

• $50 / month (USD) flat rate• FREE if < 1 GB of

data used

Enterprise - 1TB

• Dedicated, single tenant environment

• Virtual environment

• 16 cores• 64 GB memory• SAN storage (for about

1 TB of preload data)

• $1140 / month (USD)

Enterprise - 4 TB


• Bare metal



• $4560 / month (USD)

Enterprise - 12 TB


• Bare metal



• $7160 / month (USD)

• Fully managed

• High performance data warehousingand analytics in the cloud

• Scale out to meet your data growth needs

21

Key dashDB IntegrationsSystems of Record &

Systems of Engagement as data sources for dashDB

Data cleansing & integration with

numerous options (Object stores, ETL,

DataWorks)

dashDB data warehousing and analytics services

IBM & Third Party Integration with BI, Data

Visualization, and Analytic Apps e.g.

IBM DB2/PDA

Cloudant

Oracle

Other relational systems

IoT, Social Media

Watson Analytics

Cognos

SAS

Tableau

IBM SPSS

Esri

Aginity

dashDB

DataWorks

22

Systems of Insightwww.bluemix.net

www.cloudant.com

SDPSchema Discovery

Process

DataWorksData Refinery

Services

Cloud-Based Systems of Engagement(NoSQL, Mobile Apps, Internet of Things, Social Media, others)

Systems of Record(DB2, Oracle, HDP, flat files, others)

Read/Write(REST API)

ContinuousSynchronization

Read/Write

JDBC / ODBC / REST API

Read/Write(On/Off Prem)

dashDB and the IBM Cloud

www.dashDB.com

SoftLayer Infrastructure as a Service© 2015 IBM Corporation

IBM & Third Party Integrations(Watson Analytics, Cognos, SPSS, SAS,Tableau, ESRI ArcGIS, Aginity, others)

23

Loading Data into dashDB is Easy!

Local file from desktop

SoftLayer swift object store

Amazon S3

InfoSphere DataStage

DataWorks & DataWorks Forge

Synchronizing data from Cloudant database

DB2-based commands/utilities– Remote DB2 load command– DB2 ingest utility– DB2 federation

See load section of dashDB's Knowledge Center– http://www-01.ibm.com/support/knowledgecenter/SS6NHC/

com.ibm.swg.im.dashdb.doc/learn_how/loaddata_overview.html

24

Agenda

Cloud Introduction

dashDB


DB2 on Cloud

Getting Started

25

Updated xx-xx-12

Client Benefits• Simple... Easy to use web console for most common tasks• Resilient… Based on enterprise hardened capabilities, backed up regularly• Secure… Advanced Security and Privacy features

Key Differentiators• Managed DB2 environment

• OS and database software updates performed by IBM• Automatic and manual backups

• Highly affordable – get started for free with tiered pricing for up to 500 GB

• Built-in encryption• Premium plan includes enterprise grade features such as

data masking

Pricing Plans and Availability• Free: Evaluation, getting started; store up to 100MB data • Small: Store up to 10 GB (free if <2 GB used), tens of dollars per month• Premium: Store up to 500 GB, hundreds of dollars per month

IBM SQL Database (SQLDB):Relational Operational DBaaS

26

SQLDB Plans

Free Plan– 100 MB storage limit– 10 concurrent connections– No cost – $0

Small Plan– 10 GB storage limit– 20 concurrent connections– $30 US per month (free if less than 2 GB used)

Premium Plan– 500 GB storage limit– 100 concurrent connections– $400 US per month

27

Loading Data into SQLDB

Various options exist for loading data into SQLDB– Upload locally from your computer

• Excel and CSV files– Load from remote data sources

• Sources are Amazon S3 object store or Swift (SoftLayer object store)S3 not available for Small (plan going away, no further improvements)

• ASCII-delimited files• Can run immediately or be scheduled for a later, optionally repeatable time

– Data can be loaded from InfoSphere DataStage• Via regular DB2 connector

– Federation can be set up with SQLDB as a remote data source, so IUDs can be pushed out to tables in SQLDB

– DataWorks

Loading from Amazon S3 and Swift requires knowledge of the remote data source– S3: Bucket endpoint, access key ID, secret access key– Swift: Public authentication endpoint, user name, and API access key

28

Other SQLDB Features

Backup and recovery– Free plan

• Automated daily online database backup• Multi-tenant environment so restore only initiated by IBM operations team

– Small and Premium plans• Automated daily online database backup• On-demand online database backups can be initiated by user• On-demand restore of backup images (with rollforward to end of backup or end of logs)• Five backups maintained

Monitor connections– Small and Premium plans only

Monitor SQL statements– Number of executions, resources used, etc.– Small and Premium plans only

Monitor data privacy– Run sensitive data and activities reports– Small and Premium plans only

29

Agenda

Cloud Introduction

dashDB


DB2 on Cloud

Getting Started

30

IBM DB2 on Cloud

Provides a hosted DB2 environment that is– Provisioned via Bluemix– Hosted on IBM SoftLayer cloud platform

• Virtual private nodes (not shared) or bare metal depending on configuration size– Administered by your organization's DBA– Paid on a month-to-month basis via subscription model (support included)

Benefits include– Convenience without the loss of control on cost effective infrastructure– Four high performance hardware configurations and two database software tiers to match

capability and affordability needs– BLU Acceleration– Native encryption support included in all configurations ensuring data remains secure in

the cloud

Four t-shirt sized configurations: Small, Medium, Large, X-Large

Two versions of DB2 available: Workgroup Server Edition (Standard) and Advanced Workgroup Server Edition (Advanced)

31

New Ways to Use DB2 on Cloud

DB2 on Cloud is a great way to lift and shift an existing app to the cloud, butthere are other ways to take advantage of this new offering:

Use Case Goal How DB2 on Cloud Helps

Consolidation of smaller on premise databases

Leverage cloud to consolidate sprawling on-prem systems

Streamline initial move to cloud with automated deployment, and use all of your on-premises management processes/tools.

Reduce CAPEX Take Advantage of a monthly charge model

Redundancy without building a new data center

Mitigate risks for unplanned downtime

Easy setup of secondary environment, monthly charges, SoftLayer data centers

Prototyping proof of concept (POC) development projects

Take advantage of the latest DB2 technologies

Experiment in a cost effective environment that doesn’t take up data center space or management bandwidth

Retain relevance to LOB initiatives

Quickly demonstrate a positive ROI associated with IT projects

Manage YtY budget growth

32

Specs(for both

standard & advanced)

Small Medium Large X-Large

Standard $1,000 USD $1,700 USD $3,000 USD $6,000 USD

Advanced $1,250 USD $2,200 USD $4,000 USD $8,000 USD

Nodes Virtual Private Virtual Private Virtual Private Bare Metal

Cores 2 x 2.0 GHz 4 x 2.0 GHz 8 x 2.0 GHz 16 x 2.0 GHz

Memory 8 GB 16GB 32GB 128GB

Storage 100GB, 500GB SAN100GB @ 500 IOPS

100GB & 1TB SAN100GB @ 1,200 IOPS

100GB & 2TB SAN100GB @ 1,600 IOPS

2 x 800GB SSD @ RAID 1 (~800GB)6 x 1TB SATA @ RAID 6 (~4TB)

Network 1Gbps Public & Private Uplinks

1Gbps Public & Private Uplinks

1Gbps Public & Private Uplinks

10Gbps Redundant Public & Private Uplinks

OS RHEL 7.0 RHEL 7.0 RHEL 7.0 RHEL 7.0

DB2 on Cloud Plans4 T-Shirt sized configurations: Small, Medium, Large, X-Large2 versions of DB2: Workgroup (Standard), Advanced Workgroup (Advanced)Single part number for each sizeAll configurations include native encryption

33

DB2

Products/Components

Standard Advanced

DB2 Capabilities and Engine FunctionalityJSON DB2 NoSQL √ √Table Partitioning √ √IBM Mobile Database Sync √ √IBM Mobile Database Support √ √pureXML Storage √ √Time Travel Query √ √Compression Backup √ √Row & Column Access Control √ √Label Based Access Control √ √SQL Replication √ √Websphere MQ √ √High Availability Disaster Recovery √ √Tivoli System Automation √ √Online Reorg √ √Advanced Copy Services √ √Native Encryption √ (1) √Federation Capabilities √ (2) (3) √ (2) (3)

DB2

Products/Components

Standard Advanced

DB2 Capabilities and Engine FunctionalityCompression x √Multi-temperature Storage x √Materialized Query Tables x √Multi-Dimensional Clustering x √Query Parallelism x √Connection Concentrator x √solidDB & solidDB UC x √Plan Lockdown x √Scan Sharing x √DB2 Workload Management(WLM) x √Continuous Data Ingest x √MQ Replication / CDC x √ (4) (5)DB2 Connect x √ (6)Deployment Models AvailableBLU x √BLU Shadow x √Data Partitioning Feature x √

Common Components Advanced Components / Models

DB2 on Cloud: Standard vs. Advanced

1. Native encryption added for DB2 on Cloud Workgroup Edition, beyond standard license image2. Requires DB2 Connect when federating with DB2 z/OS and/or DB2 i-Series 3. Federation includes only DB2 & Informix data sources

4. QRep & CDC licenses - DB2 based replication and logical HA - Restricted to 3 DB2 LUW servers5. Requires DB2 Connect when replicating with DB2 z/OS and/or DB2 i-Series 6. For the purposes of using SQW tooling to access Z-Data

34

Activity Customer IBM

VM specification Chooses from the menu Provides eight configurations (four sizes (S-XL) and two DB2 tiers (standard, advanced))

VM creation

Hardware / VM management

VM/Hypervisor Security patches

DB2 product Installation

DB2 instance creation Single instance created as partof service readiness

HA setup Documentation provided

DB2 Fixpack mandatory upgrades Notify customer about FP availability (email)

DB2 Fixpack/Version optional upgrades Notify customer about FP/Versionavailability (email)

OS patches

Database backup and restore

Setup Encryption at rest

Service deletion Requested when ready Perform deletion of theenvironment (servers/VMs)

DB2 on Cloud – Who Does What?

35

Agenda

Cloud Introduction

dashDB


DB2 on Cloud

Getting Started

36

Getting Started with BluemixTo use dashDB or any of the other services available on Bluemix, first register for the IBM Bluemix platform at https://console.ng.bluemix.net/

If this is your first time using Bluemix, select the SIGN UP option. Otherwise, if you have previously registered for Bluemix then select LOG IN.

An IBM ID is required when you sign up – if you don't already have an IBM ID then one will be created as part of the sign up process.

37

Getting Started with Bluemix (cont.)When you log in, the Bluemix dashboard is shown (if you do not see it then click on DASHBOARD in the top menu).

From here you can create new applications or provision new services. Click on USE SERVICES OR APIS to deploy a new service of dashDB or SQL Database.

38

Finding the Services in the Bluemix CatalogThe dashDB, SQL Database, and DB2 on Cloud services are currently located inthe Data & Analytics section (note that categories and placement might change over time)

39

dashDB: Deploying an Instance of the Entry PlanClick on the dashDB icon in the catalog and then fill in the information on the right hand side:

App: Select "Leave unbound" to specify that it is not to be bound to an existing Bluemix applicationService name: Specify the name of your dashDB serviceSelected Plan: Choose the "Entry" plan

Click CREATE to create the instance. This should take less than one minute to complete.

40

dashDB: Deploying an Instance of the Entry Plan (cont.)

Once the dashDB instance has been created, you will be taken to the dashDB launch screen. Click on LAUNCH to bring up the dashDB console.

41

dashDB: Deploying an Instance of the Entry Plan (cont.)

From the dashDB console, you can perform tasks such as table creation, data load, and running queries.

42

dashDB: Creating Tables Using the Console

43

dashDB: Which Load is Right for Me?

44

dashDB: Loading from Local File on Desktop

45

dashDB: Loading from SoftLayer Swift Object Store

Choose endpoint associatedwith object storage location

User name and API access key available fromSoftLayer's object storage credentials

Once information entered above,browse for source files

46

dashDB: Remote ConnectivityApplications, management tools, ETL tools, etc. require connection information(server, port, user ID, password) to access dashDB

Connection information available in console in Connect -> Connection Settings

Non-SSL-based connections use port 50000, SSL connections use 50001– dashDB servers come with signed digital certificates that allow clients to validate the server’s identity

and secure their communications with SSL/TLS encryption

47

SQLDB: Deploying an Instance of the Free PlanClick on the SQL Database icon in the Bluemix catalog and then fill in the information on the right hand side:

App: Select "Leave unbound" to specify that it is not to be bound to an existing Bluemix applicationService name: Specify the name of your SQL Database serviceSelected Plan: Choose the "Free Beta" plan

Click CREATE to create the instance. This should take less than one minute to complete.

48

SQLDB: Deploying an Instance of the Free Plan (cont.)

Once the SQL Database instance has been created, you will be taken to the launch screen. Click on LAUNCH to bring up the SQL Database console.

49

SQLDB: Deploying an Instance of the Free Plan (cont.)

From the SQL Database console, you can perform tasks such as table creation, data load, and running queries.

50

SQLDB: Working with Tables

Create/drop tables and browse data using the console

51

SQLDB: Loading DataFrom within the console, you can perform quick loads, schedule a load, view scheduled loads, or view the history of scheduled loads

52

SQLDB: Running Queries

53

For More Information

dashDB– Main IBM dashDB product page

• http://www-01.ibm.com/software/data/dashdb/– dashDB Knowledge Center

• https://ibm.biz/dashDBWhatsNew– dashDB Learning Center

• https://developer.ibm.com/clouddataservices/docs/dashdb/– dashDB service description in Bluemix catalog

• https://console.ng.bluemix.net/catalog/dashdb/

SQL Database (SQLDB)– SQLDB documentation in Bluemix

• http://www.ng.bluemix.net/docs/#services/SQLDB/index.html– SQLDB Tutorials on IBM developerWorks

• http://www.ibm.com/developerworks/topics/sql database service– SQLDB service description in Bluemix catalog

• https://console.ng.bluemix.net/catalog/sql-database/

DB2 on Cloud– DB2 on Cloud documentation in Bluemix

• https://www.ng.bluemix.net/docs/#services/DB2OnCloud/index.html– DB2 on Cloud service description in Bluemix catalog

• https://console.ng.bluemix.net/catalog/db2-on-cloud

Kelly SchlambIBM Canada Ltd.Email: [email protected]: @KSchlamb

AN6: Get Your Head (and Data) into theClouds with SQLDB, dashDB, andDB2 on Cloud

Please fill out your session evaluation before leaving!

get your head (and data) into the clouds with sqldb

Documents