mconsulting ibm gold consultant ibm information … gold consultant ibm information champion. ......

THE WHAT, HOW AND WHY

OF DB2 ACCELERATORS

CRISTIAN MOLAROMConsulting

IBM GOLD Consultant

IBM Information Champion

DISCLAIMER

PLEASE BE AWARE THAT THE ACTUAL PROGRAMMING TECHNIQUES,

ALGORITHMS AND ALL NUMERICAL PARAMETERS USED IN EXAMPLES

GIVEN IN THIS PRESENTATION ARE SUBJECT TO CHANGE AT SOME FUTURE

DATE EITHER BY A NEW VERSION OF DB2, A NEW RELEASE, A SMALL

PROGRAMMING ENHANCEMENT (SPE) OR A PROGRAMMING TEMPORARY

FIX (PTF).

THE INFORMATION CONTAINED IN THIS PRESENTATION HAS NOT BEEN

SUBMITTED TO ANY FORMAL REVIEW AND IS DISTRIBUTED ON AN “AS IS”

BASIS WITHOUT ANY WARRANTY EITHER EXPRESS OR IMPLIED. THE USE

OF THIS INFORMATION OR THE IMPLEMENTATION OF ANY OF THESE

TECHNIQUES IS A CUSTOMER RESPONSIBILITY AND DEPENDS ON THE

CUSTOMER’S ABILITY TO EVALUATE AND INTEGRATE THEM INTO THE

CUSTOMER’S OPERATIONAL ENVIRONMENT. WHILE EACH ITEM MAY HAVE

BEEN REVIEWED FOR ACCURACY IN A SPECIFIC SITUATION, THERE IS NO

GUARANTEE THAT THE SAME OR SIMILAR RESULTS WILL BE OBTAINED

ELSEWHERE. CUSTOMERS ATTEMPTING TO ADAPT THESE TECHNIQUES TO

THEIR OWN ENVIRONMENTS DO SO AT THEIR OWN RISK.

DB2 IS A TRADEMARK OF INTERNATIONAL BUSINESS MACHINE

CORPORATION. THIS PRESENTATION USES MANY TERMS THAT ARE

TRADEMARKS. WHEREVER WE ARE AWARE OF TRADEMARKS THE NAME

HAS BEEN SPELLED IN CAPITALS.

[email protected] ©® 2012

Agenda

The WHAT in DB2 Accelerators

The HOW in DB2 Accelerators

The WHY in DB2 Accelerators

Conclusions


THE WHAT IN DB2

ACCELERATORS


What is a DB2 Accelerator?

Definition of “Accelerators for DB2 for z/OS queries”

– Accelerator is a solution that can run queries on behalf of DB2

– Generally a high-performance appliance

– An accelerator server is a specific instance of an accelerator

For certain types of queries processing on an accelerator can be

faster (or much faster) than processing on DB2:

– Business Intelligence queries

– Analytics

IMPORTANT: No changes are required to existing applications

IMPORTANT Transparent to users and applications: unaware of Accelerator

IMPORTANT: Users and applications continue to connect to DB2 for z/OS


What is IDAA?

IDAA = IBM DB2 Analytics Accelerator V2

– Successor of ISAO = IBM Smart Analytics Optimizer

– Improved V2 of ISAO offering

• Complete different platform � Netezza

– GA since November 2011

– Effective Capacity from 32 to 1280 TB

• (Assumes 4x compression ratio)

Requirements:

– z/OS 1.11, or later

– DB2 V9 or DB2 V10.1 for z/OS

– IBM System zEnterprise 196 or 114

– Verify at www.ibm.com/support/

docview.wss?id=swg27022331

IBM land page: www.ibm.com/software/data/db2/zos/analytics-accelerator/


What is IDAA?

IDAA allows integration of OLTP and DWH workload in the SAME DB2


More info?

The Appliance concept

A computer appliance is:

– A separate hardware device with integrated software (firmware)

– Specifically designed to provide a specific computing resource

• DB2 Accelerator � PERFORMANCE!

Known as “appliances” because of their similarities to home

appliances:

– Generally “closed and sealed”

– Not serviceable by the owner

NOT designed to allow the customer to:

– Change the software

– Reconfigure the hardware

Plug & Play Performance solution

IMPORTANT: IDAA provides Transparent integration with DB2 for z/OS


Appliance: isolation and integration

Cannot directly access the appliance

– Service is done through z/OS

– Data is owned and controlled by DB2

Provides many advantages:

– Easy maintenance and administration

– Very fast to deploy and get operative

DB2 integration

– All information about the Accelerator is routed via DB2

– The heartbeat provides active monitoring of the accelerator

– DB2 instrumentation (traces) for IDAA does not require

additional classes or IFCIDs to be started

• New fields were added to existing records

NOTE: IDAA is NOT another database: Total integration with DB2 for z/OS


DB2 Accounting Report Long: Accelerator section

Accounting contains a Distributed activity section

– Even if the application does not do ANY distributed access

– DB2 communicates with IDAA using DRDA

ACCELERATOR IDENTIFIER ACCELERATOR AVERAGE TOTAL ACCELERATOR AVERAGE TOTAL

----------- ---------- ----------- -------- ------ ------------ ----------- ------------

PRODUCT AQT02012 OCCURRENCES 1.00 24 ELAPSED TIME

SERVER IDAATF3 CONNECTS 1.00 24 SVCS TCP/IP 6:31.492821 2:36:35.8277

REQUESTS 2.00 48 ACCUM ACCEL 5:56.198064 2:22:28.7535

TIMED OUT 0.00 0 CPU TIME

FAILED 0.00 0 SVCS TCP/IP 0.000341 0.008187

SENT ACCUM ACCEL 3.357125 1:20.570989

BYTES 8026.33 192632 WAIT TIME

MESSAGES 11.00 264 ACCUM ACCEL 0.037185 0.892430

BLOCKS 0.00 0

ROWS 0.00 0 DB2 THREAD

RECEIVED CLASS 1

BYTES 7467.00 179208 ELAPSED N/A 10:47:16.523

MESSAGES 11.00 264 CP CPU N/A 16:36.545365

BLOCKS 0.00 0 SE CPU N/A 0.000000

ROWS 0.00 0 CLASS 2

ELAPSED N/A 2:47:43.0033

CP CPU N/A 16:35.173834

SE CPU N/A 0.000000


DB2 Statistics: Accelerator section

The DB2 Statistics Class 1 trace provides IDAA information

– The states of the accelerators that are in use

– The amount of processing time that is spent in accelerators

– Counts of the amounts of sent and received information

IDAATF3 ACCELERATION QUANTITY IDAATF3 CONTINUED QUANTITY

-------------------------------- --------------- ----------------------------------- --------------

QUERIES SUCCESSFULLY EXECUTED 24.00 AVG QUEUE LENGTH (LAST 3 HRS) 1.68

QUERIES FAILED TO EXECUTE 0.00 AVG QUEUE LENGTH (LAST 24 HRS) 1.68

ACCELERATOR IN INVALID STATE 0.00 MAXIMUM QUEUE LENGTH 16.00

CURRENTLY EXECUTING QUERIES 5.74 AVG QUEUE WAIT ELAPSED TIME 0.000001

MAXIMUM EXECUTING QUERIES 24.00 MAX QUEUE WAIT ELAPSED TIME 0.000083

CONNECTS TO ACCELERATOR 44.00 WORKER NODES 2.22

REQUESTS SENT TO ACCELERATOR 88.00 WORKER NODES AVG CPU UTILIZATION (%) 7.65

TIMED OUT 0.00 COORDINATOR AVG CPU UTILIZATION (%) 0.31

FAILED 0.00

BYTES SENT TO ACCELERATOR 354814.00 DISK STORAGE AVAILABLE (MB) 5948861.42

BYTES RECEIVED FROM ACCELERATOR 254580.00 IN USE (%) 5.81

MESSAGES SENT TO ACCELERATOR 484.00 IN USE FOR DATABASE (MB) 354179.00

MESSAGES RECEIVED FROM ACCEL 484.00 DATA SLICES 16.31

BLOCKS SENT TO ACCELERATOR 0.00 DATA SKEW (%) 91.69

BLOCKS RECEIVED FROM ACCELERATOR 0.00

ROWS SENT TO ACCELERATOR 0.00 PROCESSORS 17.79

ROWS RECEIVED FROM ACCELERATOR 0.00

TCP/IP SERVICES ELAPSED TIME 4:46:07.281505 ELAPSED TIME IN ACCELERATOR 4:31:41.389834

WAIT TIME IN ACCELERATOR 1.665544 CPU TIME SPENT IN ACCELERATOR 2:14.299979


DB2 commands

-DIS ACCEL DETAIL

-DIS THD (*) ACCEL(*)

DSNV401I -DA12 DISPLAY THREAD REPORT FOLLOWS -

DSNV402I -DA12 ACTIVE THREADS -

NAME ST A REQ ID AUTHID PLAN ASID TOKEN

BATCH AC * 17 IDAA102 IDAA1 DSNTEP10 0032 15480

V437-WORKSTATION=BATCH, USERID=IDAA1,

APPLICATION NAME=IDAA102

V444-DEIBMIPS.IPWASA12.C9272E5E3939=15480 ACCESSING DATA AT

IDAATF3 -::FFFF:10.101.8.100..1400

DISPLAY ACTIVE REPORT COMPLETE

DSN9022I -DA12 DSNVDT '-DIS THD' NORMAL COMPLETION

***

DSNX830I -DA12 DSNX8CDA

ACCELERATOR MEMB STATUS REQUESTS ACTV QUED MAXQ

-------------------------------- ---- -------- -------- ---- ---- ----

IDAATF3 DA12 STARTED 1757 0 0 62

LOCATION=IDAATF3 HEALTHY

DETAIL STATISTICS

LEVEL = AQT02012

STATUS = ONLINE


THE HOW IN DB2

ACCELERATORS


A new access path

TIP: IDAA can be viewed as an additional access path for dynamic SQL


Query Execution: Process Flow

16

DB2 for z/OS

Optimizer

IDA

A D

RD

A R

eq

ue

sto

r

DB2 Analytics Accelerator

App

Application

Interface

Queries executed with DB2 Analytics Accelerator

Queries executed without DB2 Analytics Accelerator

Query execution run-time for

queries that cannot be or should

not be off-loaded to IDAA

SPUSPU

CPU FPGA

Memory

SPUSPU

CPU FPGA

Memory

SPUSPU

CPU FPGA

Memory

SPUSPU

CPU FPGA

Memory

SM

P H

ost

Controlling acceleration

Setting the CURRENT QUERY ACCELERATION special register

Default value is NONE

TIP: There is no way to FORCE IDAA Query execution only. Use RLF for

controlling execution in DB2Q yet

IMPORTANT: be careful with SET CURRENT QUERY ACCELERATION

ENABLE WITH FAIL BACK in case of forgotten or big queries because them

coming back to DB2 may cause performance issues

.-=-.

>>-SET CURRENT QUERY ACCELERATION--+---+------------------------>

>--+-NONE-----------------+------------------------------------><

+-ENABLE---------------+

+-ENABLE WITH FAILBACK-+

'-host-variable--------'


Why a query may not be offloaded to IDAA?

Mandatory criteria

– Cursor is NOT defined as scrollable or row set cursor

– The query is defined as read-only

– The query is dynamic

– The query is a SELECT statement

Main reasons for NOT offloading a Query

– It uses CURRENT QUERY ACCELERATION = NONE (default)

– The accelerator or tables are disabled

– It uses static SQL or a plan with DBRMs

– It is not read-only or the cursor is scrollable or is a rowset cursor

– It references a table or column that is not loaded/enabled

– The optimizer decides DB2 for z/OS can do better


Or7It contains syntax that is not supported

SQL restrictions

SOME SQL restrictions

– The Query uses a special register OTHER than:

• CURRENT DATE

• CURRENT TIME

• CURRENT TIMESTAMP

– It contains a sequence expression (NEXTVAL or PREVVAL).

– It contains a user-defined function (UDF)

– It uses a MIN or MAX function with string values or more than

four arguments

Can identify non eligible queries in DSN_QUERYINFO_TABLE

IMPORTANT: More considerations apply, refer to SG24-8005 for more details

TIP: In some cases, a query can be rewritten to make it eligible


Functions

DB2 scalar functions

supported by IDAA


Workload Assessment

Done by IBM, free or charge

Collect information from DSC

– IFCID 316-318

Assessment of:

– Queries Acceleration

potential

– Estimated Elapsed time

– CPU cost

Documentation

and REXX procedure

Upload data to

IBM IBM lab

Database

Pre-process and

load

Quick

Workload Test

Tool Assessment

Report

AssessmentCustomer

Database


Virtual Accelerators

You can install virtual accelerators with or without IDAA installed

– Applies to DB2 9 for z/OS and DB2 10 for z/OS

EXPLAIN only

DSN_QUERYINFO_TABLE

– Contains information about acceleration of query blocks

EXPLAIN with SET CURRENT QUERY ACCELERATION ENABLE

REASON_CODE=0 � the query is eligible for IDAA offload

EXPLAIN ALL SET QUERYNO=228101 FOR

SELECT SUM(L_TAX)

FROM ONETB.LINEITEM

WHERE L_LINENUMBER=1

AND L_DISCOUNT > 10000

GROUP BY L_SUPPKEY

FETCH FIRST 10 ROWS ONLY;


Adding tables to an Accelerator

Queries eligible for acceleration will be executed in accelerator

ONLY IF all the tables the query is referencing has been defined

and loaded in the accelerator

Defining tables to IDAA: Add Table wizard in Accelerator Studio

Loading tables into an Accelerator

– Stored procedures and batch pipes

– Either from the Accelerator Studio or using batch jobs


Partitions belonging to the same table can be loaded in parallel

User defined degree of parallelism

Updates are done on a per-table or per-partition level

Up to 3TB / hour

– 200 GB / hour for single stream

Loading data into IDAA

NOTE There is no automatic indication that a table needs to be refreshed


THE WHY IN DB2

ACCELERATORS


Performance, performance, performance

Example of observed performance changes (figures from IBM)

– International securities company


IDAA Effects

Class 1

Elapsed

Class 1

CPU

Class 2

CPU

545% 17,921% 27,669%

618% 21,075% 32,545%

442% 22,187% 34,299%

473% 20,123% 31,135%

487% 12,256% 18,924%

719% 1,155,561% 2,233,215%

894% 1,274,866% 2,475,147%

Reduction of CPU consumption in z/OS

Potentially:

– A good portion of the CPU cycles consumed by queries running

in DB2 can be eliminated

• Queries are diverted to execute in IDAA

Amount of reduction depends on:

– The volume of queries that are eligible to run in IDAA

– It is likely complex queries will be selected to run in IDAA

• Normally these queries contribute with much CPU utilization

– Even with only a small subset of queries the potential z/OS CPU

savings could be significant

IMPORTANT: Offloaded queries returning a large result set could consume a

noticeable amount of CPU time in DB2

TIP: IDAA gives an opportunity to reduce costs by CPU reduction


Benchmark workload example

80 concurrent users executing Cognos reports

Reports are a mix of short, intermediate, and long running reports

Total of 26 queries against the DB2 for z/OS database


Before


After


CPU distribution per report

Exploit WLM Client Info to identify CPU utilization per report

Workload driven from Cognos or from zLinux scripts

IMPORTANT: even a few offloaded queries can save lots of resources


Scalability

IMPORTANT: The maximum level of concurrency is 100 queries in IDAA


News from Development

What to expect in the near future

– Reduce the cost of synchronyzing data between DB2 and IDAA

– Low latency replication of changes

– Integration with z/OS WLM

– Removing limitations for routing queries to IDAA

– Performance enhancements

– Adding advanced analytics capabilities

– Providing multi-temperature data capabilities

– Extension of query acceleration to new usage cases


Enable DB2 transition into a truly universal DBMS that provides best

characteristics for both OLTP and analytical workloads

Conclusions

WHAT

HOW

WHY

�A lot better performance

�Reduction of Mainframe TCO

�DB2 integration

�Transparency to applications

�Accelerator = appliance

�OLTP + Analytics in DB2

* Inspired from Simon Sinek’s “Start with Why”


Summary

What we discussed about:

– The WHAT in DB2 Accelerators

– The HOW in DB2 Accelerators

– The WHY in DB2 Accelerators

QUESTIONS?


THANKS!

CRISTIAN MOLARO

[email protected]

@cristianmolaro


mconsulting ibm gold consultant ibm information … gold consultant ibm information champion. ......

Documents