![Page 1: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/1.jpg)
Optimizing Your WebSphere Applications with Optim and pureQuery
Curt Cotner
IBM Fellow, CTO for IBM Database Servers
![Page 2: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/2.jpg)
Development Tools for Your Java Database Applications
![Page 3: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/3.jpg)
1. Select table
Java Data Access in 5 Simple Steps
2. Name bean & select styles 3. Generate test code
4. Map table to bean 5. Select template SQL CRUD
![Page 4: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/4.jpg)
4
What happened?Generate code from table
The code is generated for Bean (aka Java Object) representing the table
Interface with associated SQL or Sample Application with Inline SQL
Implementation of Interface
Optionally a JUnit test case.
![Page 5: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/5.jpg)
5
Generated Test Cases for Your SQL
• Jump start your migrations!
![Page 6: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/6.jpg)
6
SQL analysisSQL executionSQL validation
Developing with pureQueryUnleash SQL from Java
SQL content assist
![Page 7: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/7.jpg)
7
SQL content assist SQL analysisSQL execution
Developing with pureQueryUnleash SQL from Java
SQL validation
![Page 8: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/8.jpg)
8
SQL validationSQL content assist SQL analysis
Developing with pureQueryUnleash SQL from Java
SQL execution
Run SQL with parameters at design time in the Java program WITHOUT writing a test application
![Page 9: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/9.jpg)
9
SQL executionSQL validationSQL content assist
Developing with pureQueryUnleash SQL from Java
SQL analysis
View explain plans for SQL inside you Java programs
![Page 10: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/10.jpg)
10
pureQuery OutlineSpeed up problem isolation for developers – even when using frameworks
• Capture application-SQL-data object correlation (with or without the source code)
• Trace SQL statements to Java source code for faster problem isolation
• Enhance impact analysis by identifying specific application code impacted by a database changes
• Answer “Where used” questions like “Where is this column used within the application?”
• Use with modern Java frameworks e.g. Hibernate, Spring, iBatis, OpenJPA
![Page 11: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/11.jpg)
11
pureQuery Outline
pureQuery Outline View’s 3 categorizations
• How do you look at the relationship between SQL and Java?
![Page 12: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/12.jpg)
Optim Development Studio
• Problem determination and isolation
– with pureQuery outline go to the source of the problematic SQL
• Improve Hibernate data access calls
– Re-write HQL– Use better
performing native SQL
• With performance metrics Identify the hot spots in your hibernate application
Source code correlation
Database object correlation
Identify and change the HQL
![Page 13: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/13.jpg)
Analyzing Performance
![Page 14: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/14.jpg)
Analyze Use of Sensitive Data
See queries accessing sensitive data
Optim Development Studio
Icon identifies sensitive access
Filter to see use of sensitive data
View only SQL accessing
sensitive data
Filter SQL by action
![Page 15: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/15.jpg)
IBM Data Studio
pureQuery For DBAs
and Applicatio
n Developer
s (v1.2) - Part 1
1515
pureQuery ToolsSQL templates and customizations
Create your own SQL templates
Use templates to write SQL that is frequently reused
Use tabs to change the variable names after inserting SQL statement from the template through SQL context assist
Use newly created merge template in your java code using SQL assist
![Page 16: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/16.jpg)
1616
Java/SQL Development on SteroidsWhy the 10x Productivity Improvement
![Page 17: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/17.jpg)
Optimizing Your WebSphere Applications
![Page 18: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/18.jpg)
DB2 ConnectRecommended Deployment Options
Desktop PCsDesktop PCs
Application serversApplication servers
Web application serversWeb application servers
Several options: Personal Edition or DB2 Connect Server or file server
Several options: Personal Edition or DB2 Connect Server or file server
Co-locate DB2 Connecton the application server
Co-locate DB2 Connecton the application server
Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops
Recommendation:• Personal Edition is best for small numbers of end users• DB2 Connect server or file server deployment is best for lots of desktops
Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring
Rationale:• drivers now include the key DB2 Connect gateway features (sysplex workload balancing, connection concentrator, XA support, automatic reconnect, etc.)• fewer potential points of failure• less hardware cost• less system administration cost• fewer network hops (up to 40% better elapsed time)• simplified failover strategy • less complex problem determination and monitoring
![Page 19: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/19.jpg)
Open Source Persistence
Engine
JPA Persistence Engine
JDBC API
Hibernate, iBATIS, EclipseLink,...
JPA for WebSphere, Apache OpenJPA
pureQueryAPI
Data Studio pureQueryPlain JDBC
Data Web Services, Project Zero, sMash
On-ramps to pureQuery
JCC driver
JDBC
.Net Applications
.Net applications
pureQuery
JPA APIOpen Source
Persistence API Web API
ADO .Net
DB2, Informix, and Oracle now…more coming
![Page 20: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/20.jpg)
Client OptimizationImprove Java data access performance– without changing code
• Captures SQL for Java applications– Custom-developed, framework-based, or packaged applications
• Bind the SQL for static execution without changing a line of code– New bind tooling included
• Delivers static SQL execution value to existing DB2 applications– Making response time predictable and stable by locking in the SQL access path
pre-execution, rather than re-computing at access time– Limiting user access to tables by granting execute privileges on the query
packages rather than access privileges on the table– Aiding forecasting accuracy and capacity planning by capturing additional
workload information based on package statistics – Drive down CPU cycles to increase overall capability
• Choose between dynamic or static execution at deployment time, rather than development time
![Page 21: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/21.jpg)
Optim pureQuery Runtime for z/OS
• In-house testing shows double-digit reduction in CPU costs over dynamic JDBC
•
• IRWW – an OLTP workload, Type 4 driver• Cache hit ratio between 70 and 85%• 15% - 25% reduction on CPU per txn over dynamic JDBC
274
360420 446
485524
0
100
200
300
400
500
No
rmalized
Th
rou
gh
pu
t (I
TR
)
Normalized Throughput by API for JDBC Type 4 Driver
-35%
-14%
6%15%
25%
-50%
% in
cre
ase/r
ed
ucti
on
in
CP
U p
er
tran
sn
co
mp
are
d t
o J
DB
C
% increase/reduction in CPU per transaction compared to JDBC using Type 4 driver
![Page 22: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/22.jpg)
Throughput Increase with .NET• Same IRWW OLTP application used for the Java tests but in .NET
• Application access DB2 for z/OS via Windows Application Server (IIS)
• Throughput during static execution increased by 159% over dynamic SQL execution assuming a 79% statement cache hit ratio
![Page 23: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/23.jpg)
pureQuery -- More Visibility, Productivity, and Control of Application SQL
• Capture SQL• Share, review, and optimize SQL• Revise/optimize SQL and validate equivalency
without changing the application• Bind for static execution to lock in service level
or run dynamically • Restrict SQL to eliminate SQL injection
Capture Review Optimize Revise Restrict
![Page 24: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/24.jpg)
Visualize execution metrics
Execute, tune, share, trace, explore SQL
Replace SQL without changing the application
Position in Database Explorer
Visualize application SQL
![Page 25: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/25.jpg)
Have You Heard of SQL Injection?
![Page 26: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/26.jpg)
pureQuery – Stripping Literals from SQL
JDBC app
INSERT INTO T1 VALUES(‘ABC’,2,’DEF’) INSERT INTO T1
VALUES(:h1,:h2,:h3)
pureQueryRuntimeconversion
• pureQuery can identify statements that use no parameter markers, and strip the literals out at runtime• significant performance gains:
• less CPU cost at PREPARE• better use of dynamic statement cache
![Page 27: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/27.jpg)
What Is Heterogeneous Batching?
Data Server
Table 2, operation 2
Table 1, operation 1
Table 1, operation 2
Table 1, operation 3
Table 2, operation 1
Table 3, operation 1
Heterogenous Batching – multiple operations across different tables all execute as one batch
Table 1, operation 4
![Page 28: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/28.jpg)
JDBC Batching vs pureQuery Batching• JDBC batching used by
Hibernate Batcher is currently limited
– Cannot batch entities that map to multiple tables
• Primary and Secondary tables.• Inheritance Join and Table per
class strategies– Cannot batch different operations
against same table• Field level updates• Insert, update
– Cannot batch different entities• pureQuery heterogeneous
batching plug-in– Can batch entities that map to
multiple tables– Can batch different operations
against the same table– Can batch different entities into a
single batch– Combines insert, deletes, updates
into single batch
The advantage of Heterogenous Batching
050
100150200250300350400450500
0 20 40 60 80 100
# of operations per transaction
Elap
sed
Tim
e (m
Sec)
HeteroBatching
No Batching
JDBC Batching
* Preliminary findings based on validation with a test designed to demonstrate heterogeneous batching differences. This is not intended to be a formal benchmark.
![Page 29: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/29.jpg)
OpenJPA and Hibernate --SQL Query Generation
JPA Query
Select emp_obj(), dept_obj()
SQL
Select * from EMP WHERE …Select * from DEPT WHERE …
JPAquery transform
• Hibernate and OpenJPA often rewrite queries• No database statistics are used – entirely heuristic!!!• Can often result in poorly performing queries
![Page 30: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/30.jpg)
Monitoring of Java Applications
![Page 31: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/31.jpg)
Optim Database Performance Manager future directions -- Associate SQL with Java SourceHeat
ChartDashboard
Alerts
SLAs
In-flight analysis
Database: Accounting
Statement text schema E2E elapsed occurrences sort time phys. I/O
SELECT TIME FROM UNIVERSE SAP3 132.13 1323 123.32 1.303
SELECT SALARY FROM PAYMENT … SYSIBM 323.4 221 11.3 32.1
DELETE FROM ACCOUNT WHERE AID = 3…
PROC 23.3 435 32322.3 32.1
TOP 3 currently running SQL Statements
TOP by DS elapsed DS CPU time Physical I/O Sort time
- +
- +
- +
SELECT TIME FROM UNIVERSEStmt text Analyze
Application DS user ID KARN Client IP addr / hostname TPKARN.de.ibm.com Client user ID KARN Client workstation name TPKARN Client application name Jawaw.exe Client accounting N/A application name Online banking application contact [email protected] package West.OLBank class Account method Transfer() source line 314
Time distribution
Force applicationStop SQL
sorting
Resource usageQuery cost estimates 18.456Buffer Pools Data – hit ratio (%) 43.4% Data – physical reads / min 4323 Index – hit ratio (%) 54.2% Index – physical reads / min 3214
Statement information X
DS Proc
USER CPU SYSTEM CPU
Unacc wait
DS sorting
Statement elapsed timeCurrent 132.13 sec last day 239.40 seclast week 15.60 sec
![Page 32: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/32.jpg)
Toughest issue for Web applications – Problem diagnosis and resolution
Web BrowserUsers Web
Server
Application Server
DB2Server
Business Logic
Data A
ccess Logic
Persistence Layer
DB
2 Java Driver
JDBCPackage
EJB
Query Language
![Page 33: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/33.jpg)
What’s so Great About DB2 Accounting for CICS Apps?
z/OS LPARCICS AOR1Txn1 - Pgm1 - Pgm2
CICS AOR2TxnA - PgmX - PgmY
DB2PROD
CICS AOR3Txn1 - Pgm1 - Pgm2
App CPU PLANTxn1 2.1 TN1PLNTxnA 8.3 TNAPLN
DB2 Accounting for CICS apps allows you to study performance data from many perspectives:• By transaction (PLAN name)• By program (package level accounting)• By address space (AOR name)• By end user ID (CICS thread reuse)
This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.
![Page 34: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/34.jpg)
JDBC Performance Reporting and Problem
Determination – Before pureQuery
Application ServerDB2 or IDS
A1
A2
A5
A3
A6
A4
USER1
USER1
USER1
User CPU PACKAGEUSER1 2.1 JDBCUSER1 8.3 JDBCUSER1 22.0 JDBC
What is visible to the DBA? - IP address of WAS app server - Connection pooling userid for WAS - app is running JDBC or CLI
What is not known by the DBA? - which app is running? - which developer wrote the app? - what other SQL does this app issue? - when was the app last changed? - how has CPU changed over time? - etc.
Data A
ccess Logic
Persistence Layer
DB
2 Java Driver
EJB
Query Language
![Page 35: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/35.jpg)
What’s so Great About Optim pureQuery Accounting for WebSphere Applications?
z/OS LPAR
CICS AOR2TxnA (PLANA) - PgmX - PgmY
App CPU TxnA 2.1 TxnB 8.3
Data Studio and pureQuery provide the same granularity for reporting WebSphere’s DB2 resources that we have with CICS:• By transaction (Set Client Application name )• By class name (program - package level accounting)• By address space (IP address)• By end user ID (DB2 trusted context and DB2 Roles)
This flexibility makes it very easy to isolate performance problems, perform capacity planning exercises, analyze program changes for performance regression, compare one user’s resource usage to another’s, etc.
Unix or Windows
WAS 21.22.3.4TxnA (Set Client App=TxnA) - ClassX - ClassY
![Page 36: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/36.jpg)
Simplifying Problem Determination Scenario
Application Developer
• Available for each db access
–SQL text generated
–Access path
–Cost estimates
–Estimated response time
–Elapsed & CPU time
–Data transfer (getpages)
–Tuning advice
Database Administrator
• Available for each SQL
–Application name
–Java class name
–Java method name
–Java object name
–Source code line number
–Source code context
–J-LinQ transaction name
–Last compile timestamp
Java
Profiling
pureQuery
DRDA Extentions
![Page 37: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/37.jpg)
Using pureQuery to Foster Collaboration and Produce Enterprise-ready Apps
Application Server
Catalog data for SQL
ApplicationMeta data
DB2 or IDSProd
A4
A1
A1
A6
A6
A2
A2
A3
A3
A4
A4
A5
A5
A1
A4
A5
PerformanceData
Warehouse
Application Developer Database
Administrator
A1
A6A2 A3
A4 A5
Quickly compare unit test
perf results to production
Use pureQuery app metadataas a way to communicate in terms familiar to both DBA and developer
ApplicationMeta data
DB2 or IDSDev System
A1
A6
A2 A3
A4 A5A1
A4
A5
![Page 38: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/38.jpg)
Customer Job Roles – A Barrier to a “Holistic View”
Application Server
DBServer
Data A
ccess Logic
Persistence Layer
DB
Java Driver
JDBCPackage
EJB
Query Language
WebS
phereC
onnectionP
ool
Busines
sLogic
13
54
2
ApplicationDeveloper
SystemProgrammer DBA
NetworkAdmin
![Page 39: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/39.jpg)
DB2 Performance Expert and Extended Insight• Enables early and rapid problem detection to
prevent impact to production systems– Identify resource shortages across CPU, memory,
and file systems, locking conflicts and deadlocks, and data skew
• Increase ability to meet service level agreements
– Provides optimization and tuning recommendations
• Supports trend analysis and growth planning– Maintains and analyzes performance warehouse
• Supports a variety of database workloads – Includes online transaction processing, data
warehouse, and enterprise content management– Monitors Workload Management environment
• Visualizes the end-to-end response time – See where Java database workloads,
transactions, and SQL requests are spending their time across the database client, application server, and network
Provides in-depth database monitoring, problem isolation, and trend analysis for DB2 for
Linux, UNIX, and Windows databases. Add Extended
Insight Feature for visibility into where Java database
workloads, transactions, and SQL requests are spending their
time
![Page 40: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/40.jpg)
Scenario
It seems that the first application server has a problem. Double-click to drill-down.
In this situation, all applications are equally affected, and the problem seems not to be in the data server.
![Page 41: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/41.jpg)
Scenario - continued
Double-click to drill-down and display detail information.
Most of the time is spent for „WAS connection pool wait“ time.
![Page 42: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/42.jpg)
Scenario – continued
5 second wait time indicates that the maximum number of allowed connections is not sufficient…
… which becomes also evident when comparing the parameters and metrics of this client with other clients.
![Page 43: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/43.jpg)
Where to get IBM Data Studio ?
• IBM Data Studio –www.ibm.com/software/data/studio
• FAQs / Tutorials
• Downloads
• Forum / Blogs
• Join the IBM Data Studio user community
![Page 44: Optimizing Your WebSphere Applications with Optim and pureQuery](https://reader036.vdocuments.net/reader036/viewer/2022062304/56814609550346895db31957/html5/thumbnails/44.jpg)