advanced internal oracle tuning techniques for sap systems

8/6/2019 Advanced Internal Oracle Tuning Techniques for SAP Systems

1/79

0


2/79

2009 Wellesley Information Services. All rights reserved.

Advanced InternalOracle TuningTechniques forSAP Systems

Michael KennedySMS Consulting


3/79

2

What Well Cover

Understanding Oracle SGA and Components

Understanding index design

Proactive review of poorly performing SQL

Index Rebuilds

Oracle Wait Events

SAP transactions for tuning

Oracle best practices

Wrap-up


4/79

3

Oracle SGA Basics

The 3 Most Important Memory Regions:

1. * Shared Pool Holds SQL Statements

2. * Buffer Cache Holds Oracle Blocks

3. * Redo Buffer Holds Transactional Changes


5/79

4

Oracle SGA Basics (cont.)

Tips Normally the Shared Pool Size (shared_pool_size parameter) is no larger than

2-3 GB due to bind variable usage. A very large Shared Pool may cause waitson Cursor Pin: S latches. See SAP Note 690241 for sizing recommendations.

The size of the Buffer Cache (db_cache_size parameter) can vary depending onthe amount of free memory on the server. In general the larger the buffercache the better. Try to keep the Oracle Buffer Hit Ratio above 94% (see SAPNote 789011). Some large customers are running 50-60 GB Buffer Cache.

The size of the Redo Log (log_buffer parameter) is usually no larger than 1 MBas per SAP Note 830576 because of the frequent commit rate.


6/79

5

Oracle SGA and Background Processes


7/79

6

Sessions


8/79

7

Oracle Buffer Cache Used By Sessions

Queries to Use:

select a.object_name, count(b.objd) from dba_objects a, v$bh bwhere a.data_object_id = b.objdand a.object_name = upper('&my_object')group by a.object_name

OBJECT_NAME COUNT(B.OBJD)-------------------- ----------------EDIDC 257,534

select a.object_name, b.num_buf from dba_objects a, X$KCBOQH bwhere a.data_object_id = b.obj#order by 2 desc;

OBJECT_NAME NUM_BUF-------------------- ----------------CE4LBOC 125,822

EKPO 74,081EDIDS~0 54,363EDIDC 48,917EDIDS~0 45,644


9/79

8

Background Processes (DBWR)

Database Writer

Used to write modified (dirty) Oracle blocks from the Oracle

buffer cache to the datafiles.


10/79

9

Background Processes (DBWR) (cont.)

The DBWR writes dirty blocks to disk when:

1. If a user process posts the DBWR because there are too manydirty blocks in memory

2. During an Oracle Checkpoint which should be a LOG SWITCH3. Parameter Settings (i.e., fast_start_mttr_target)

Indications of DBWR issues

If the following WAIT events are constantly showing up

Write Complete Waits

Free Buffer Waits

Solutions to DBWR issues

1. Check for Asynchronous I/O (SAP Note 999524)2. Check the number of DBWRs configured (SAP Note 830576,

793113)

3. Confirm I/O queuing and/or device utilization


11/79

10

Log Writer (LGWR)

Log Writer

Used to write transactional changes from the Log Buffer to the

online redo log


12/79

11

Log Writer (LGWR) (cont.)

The LGWR writes changes when:

1. Every Commit (SAP R/3 commits frequently)

2. The Log Buffer is 1/3 Full

3. Every 3 seconds

Indications of LGWR issues:

If the following WAIT events are constantly showing up

log file sync Sessions are waiting on commits

log buffer space There is no space left in the log buffer

Solutions to LGWR issues:

For Log File Sync issues (SAP Note 619188)

Check the size of the log buffer it may be too large

Check the performance on the redo log filesystems

Try to avoid RAID 5 for redo logs (SAP Note 793113) For Log Buffer Space issues increasing the size of the log buffer may

reduce the wait event. However, only increase the size in small increments(i.e., 500 KB 1 MB) and continue to review filesystem performance. See

SAP Note 619188.


13/79

12

What Well Cover




Index Rebuilds

Oracle Wait Events



Wrap-up


14/79

13

B*Tree Design


15/79

14

B*Tree Indexes

B*Tree indexes are always balanced

All leaf blocks, which hold the row ID, are on the same level

Index entries are always ordered

Index scans use a db file sequential read

Single block fetch (nota multi-block fetch), except for IndexFast Full Scan

Can be slow forlarge index RANGE scans


16/79

15

Optimal Oracle Block Access

What is the optimal number of blocks read foran index?

The optimal number of blocks being accessed in Oracle is

between 3 and 5 During a unique scan down the index, the root block, branch

block, leaf block, then the table data is read into the databasecache. Depending on the height of the index, a unique scan willonly read 3 to 5 blocks.

Use the 3 to 5 blocks as the optimal value to compare withother inefficient SQL statements


17/79

16

What Well Cover




Index Rebuilds

Oracle Wait Events



Wrap-up


18/79

17

Proactive Review of Poor SQL

ST04 Detailed Analysis SQL Requests


19/79

18

Proactive Review of Poor SQL (cont.)


20/79

19

Inefficient SQL

Example of an inefficient SQL statement


21/79

20

Efficient SQL

Example of an efficient SQL statement


22/79

21


ST04 Detailed Analysis SQL Requests Sort on Proc Rows

Notice the HIGH number of PROC ROWS and the HIGH number

of Rproc/Exec This may be an indication of poor code

NOTICE: There WHERE CLAUSE in the ABAP code passingVALUES

SELECT * FROM zsvc_part_match UP TO 1 ROWSWHERE zzmatnr_app = com_vbrp_tab-matnr

AND zz_delete = ' '

AND zzenroll = 'X'.

ENDSELECT.


23/79

22


However, notice that there are NO WHERE clause values in theSQL sent to Oracle other than MANDT

This is an example of where a CHECK FOR INITIAL in ABAP will

fix this query


24/79

23

Reactive Troubleshooting: User Performance Complaint


25/79

24

Example: BSAS SQL Statement

ST04 Detailed Analysis Oracle Session


26/79

25

BSAS Buffer Gets


27/79

26

BSAS Solution

There have been 13,905 executions while the database has beenup for 20 days

That means this query is executed 695 times per day or 28 times

per hour Each execution needs to allocate 459 blocks of memory to find

almost ZERO rows

The BSAS statement has fetched almost 7 million blocksinto memory with only 6 rows found out of the 13,905 executions

Resolution:

Add ZUONR to the custom index BSAS~Z1

Create a new custom index as ZUONR, BUKRS, HKONT, BUDAT


28/79

27

SQL Statement General Rules

In general, RANGE scans may be slower and require additional resources (CPU,memory, I/O)

Try to access the least amount of data down an index

Oracle joins may not be the fastest option depending on the driving table of the

inner join If a join is slow, determine the inner-most index of the explain plan

If the index is not efficient, a custom index may be created

If the CBO is not picking the correct driving index, a HINT should be used

(SAP Note 130480) Check SAP Notes for performance enhancements for suspected transaction

some standard SAP ABAP code may pass the LITERALS instead of BINDvariables

Example: MB51 transaction is slow due to large number of rows in

MSEG/MKPF Applying SAP Note 921164 may change that with LITERALS instead of

BIND variables


29/79

28

What Well Cover




Index Rebuilds

Oracle Wait Events


Oracle best practices Wrap-up


30/79

29

When Should Oracle Indexes Be Rebuilt?

When the index storage quality drops

After large data deletes

When indexes are very large and you would like to

reclaim space If an index has too many extents (greater than 1,000)

When there are SQL statements that do large RANGE SCANS (,

like, between) on the index. Keeping the indexes small willimprove performance.


31/79

30

Index Storage Quality

SAP Note 979054 (Program RSORAISQN)


32/79

31

Index Storage Quality (cont.)

SAP Note 771929

DB02

Detailed Analysis Enter Index Detailed Analysis

Menu: Analyze Index Storage Quality Uses Oracles DBMS_SPACE package to calculate space usage


33/79

32

Rebuilding Indexes

SAP Note 771929

SE38

Program RSORATAD

Ran for one hour on 20 GB index MSEG~0 with 304 million rows


34/79

33

Rebuilding Indexes (cont.)

SAP Note 771929

Analyze Index VALIDATE Structure

Analyze index &index validate structure

Select * from index_stats

Analyze will lock all DML on the table so run on a copy ofproduction

SQL> select name, height, lf_rows, del_lf_rows, pct_used from index_stats;

NAME HEIGHT LF_ROWS DEL_LF_ROWS PCT_USED

-------------------- ---------- ---------- ----------- ----------

TST03~0 3 159608 115497 54


35/79

34

How Do You Monitor Index Creation or Rebuild Status?

create index "EKBE~Z1" on EKBE(MANDT,VGABE, BUDAT, WERKS)tablespace psaprtlonlineparallel (degree 5)

select a.username, a.tablespace, b.sid, a.contents,a.segtype, a.extents, a.blocks*8192 "Bytes"from v$sort_usage a, v$session bwhere a.session_addr = b.saddr

USERNAME TABLESPACE SID CONTENTS SEGTYPE EXTENTS Bytes---------------------- ---------- --------- --------- ---------- ----------------SAPRTL PSAPTEMP 393 TEMPORARY SORT 64 67,108,864SAPRTL PSAPTEMP 384 TEMPORARY SORT 40 41,943,040

SAPRTL PSAPTEMP 367 TEMPORARY SORT 80 83,886,080SAPRTL PSAPTEMP 395 TEMPORARY SORT 61 63,963,136SAPRTL PSAPTEMP 382 TEMPORARY SORT 67 70,254,592

---------- ----------------sum 312 327,155,712

The index in this example is created with a parallel degree of 5and is being sorted in PSAPTEMP

How Do You Monitor Index Creation or Rebuild Status?


36/79

35

How Do You Monitor Index Creation or Rebuild Status?(cont.)

With this script, you can monitor the index being created in thetarget tablespace

select segment_name, tablespace_name, bytes, extents fromdba_segments

where segment_type = 'TEMPORARY'ORDER BY 2

SEGMENT_NAME TABLESPACE_NAME BYTES EXTENTS--------------- --------------- ---------------- ----------

31.680100 PSAPRTL 98,631,680 8463.290980 PSAPRTL 79,626,240 8197.325500 PSAPRTL 85,262,336 82208.153460 PSAPRTL 81,002,496 81237.178036 PSAPRTL 95,420,416 86

---------------- ----------

sum 439,943,168 414


37/79

36

Tips for Index Creations and Rebuild

The pga_aggreagate_target only uses 100 MB for sorts. Using_pga_max_size may help, which is a setting in SAP NetWeaver

BW.

Use the parallel query for faster creation times

Run the create or rebuild index during a low peak time, becausehigh Data Manipulation Language (DML) may cause a table lock(SAP Note 682926)

Use ONLINE keyword so the table is not locked Use NOLOGGING to minimize redo generation

The index will need to be dropped and recreated if using Oracledataguard

The index will need to be dropped and recreated if a recovery isneeded if a backup was not taken after the create or rebuildcommand

PSAPTEMP needs to be at least 1.5 times the size of the largest

index being created in order to hold the ordered data


38/79

37

Tips for Index Creations and Rebuild (cont.)

Schedule Index Rebuilds SE38

Program RSANAORA

List of Indexes in variant

Online Index Rebuilds If locking occurs Kill the Oracle SQL*Net process, NOT the SAP session


39/79

38

What Well Cover




Index Rebuilds Oracle Wait Events




40/79

39

Oracle Wait Events

Users who are waiting are not happy users!

Every session that waits must wait for a specificevent, andknowing where to look can answer why the session is waiting

Areas to look for waits: V$SESSION_WAIT

V$SYSTEM_EVENT

V$SESSION V$LOCK

SAP Transaction ST04 Detailed Analysis Oracle Sessions

C O l W it E t


41/79

40

Common Oracle Wait Events

DB FILE SEQUENTIAL READ Session is waiting for I/O from an explain plan using an index.

A single block I/O is done.

DB FILE SCATTERED READ Session is waiting for I/O when doing a FULL TABLESCAN or

FAST FULL INDEX scan. Multi-block operations are done.

DB FILE PARALLEL WRITE

The DBWR is writing data to the datafiles. LOG FILE SYNC

The session is waiting for the LGWR to write the data from thelog buffer to the online redo logs because of the commit.

LATCH FREE

The session is waiting for an Oracle Latch. Use P2 value todetermine latch name to resolve the issue in 9i. Oracle 10g will

show the full latch name.

C O l W it E t ( t )


42/79

41

Common Oracle Wait Events (cont.)

ENQUEUE

The session is waiting for an Oracle Enqueue, which is most likely eithera(n):

ST Enqueue Space Transaction Enqueue for extent management

TX Enqueue Multiple sessions modifying the exact same rowconcurrently (usually a SELECT FOR UPDATE)

WRITE COMPLETE WAIT

The session is waiting for the DBWR to write the dirty block to disk

FREE BUFFER WAIT

The session is waiting for the DBWR to clean up dirty blocks in the Oraclebuffer cache

LIBRARY CACHE PIN or LIBRARY CACHE LOCK

Waiting for exclusive lock to the library cache and may indicate a SharedPool issue

O l W it E t E l


43/79

42

Oracle Wait Events Example

In this example on Oracle 9i, there are many users waiting for aLATCH FREE for KNB1

Oracle Wait Events Example (cont )


44/79

43

Oracle Wait Events Example (cont.)

In this example on Oracle 10g, there are many users waiting for aLATCH on cache buffers chains

Oracle Wait Events Example (cont )


45/79

44

Oracle Wait Events Example (cont.)

In this example from v$session_wait, the event and the time waiting can beseen

Shown below, a FULL TABLESCAN (db file scattered read) can be seen. Why?FULL TABLESCANS should not be common!

Similar information in ST04 Detailed Analysis Oracle SessionSelect sid, seq#, event, p1,p2,p3, seconds_in_waitfrom v$session_waitwhere event not like SQL% and event not like %ipc%and event not like %timer;

SID SEQ# EVENT P1 P2 P3 SECONDS_IN_WAIT---------- ---------- ------------------------------ ---------- ---------- ---------- ---------------

624 18902 buffer busy waits 354 44041 130 02002 41785 db file scattered read 286 73460 8 0157 30307 db file sequential read 53 320723 1 02245 1775 db file sequential read 354 44209 1 011 48565 log file parallel write 2 20 2 0

Oracle Enqueue Waits


46/79

45

Oracle Enqueue Waits

Usually, there are two types of Oracle Enqueues (not the same asSAP Enqueues)

TX Enqueue is a row level lock (i.e., DATA) and common in

parallel jobs NRIV Locking

Locking on similar tables due to SELECT FOR UPDATE

Non-TX Enqueue (i.e., TM, ST, CI) is usually locking in Oraclekernel and is not DATA related

ST Enqueue due to Dictionary Managed Tablespaces andextent management

Oracle bug

Oracle Enqueue Waits (cont )


47/79

46

Oracle Enqueue Waits (cont.)

Steps to diagnose Oracle Enqueues

SM66 can show concurrent access to a table(e.g., CE3LBOC below)



48/79

47


Use ST04 Detailed Analysis Oracle Session Shows the SQL statement causing the enqueue



49/79

48


Use transaction DB01 to see the TX enqueues and thewaiter/holder

Use the Oracle SID (System ID) shown below in DB01 to map to

the session in ST04



50/79

49


For Oracle 9i, run the following scripts in SQL*Plus to determine TYPEof enqueue. The TYPE determines in what area the problem resides.

The example below shows two sessions with TX enqueue locking

Prompt This will show the TYPE of enqueueSELECT sid, chr(bitand(p1,-16777216)/16777215)||chr(bitand(p1, 16711680)/65535) "Lock",to_char( bitand(p1, 65535) ) "Mode"FROM v$session_wait

WHERE event = 'enqueue;

SID Lock Mode----- ------ ----------------------------------------383 TX 4385 TX 4



51/79

50

For 9i, use the following script to see the object and row of theenqueue prompt. This will show the object and row of enqueue.

In this example, the Object 122961 is where the TX enqueue occurs

col event format a25select a.sid, b.event, a.row_wait_obj#, a.row_wait_file#, a.row_wait_block#,

a.row_wait_row#, b.seconds_in_waitfrom v$session a, v$session_wait bwhere a.sid = b.sidand b.event = 'enqueue'order by 2, 3, 4, 5;

SID EVENT ROW_WAIT_OBJ# ROW_WAIT_FILE# ROW_WAIT_BLOCK# ROW_WAIT_ROW# SECONDS_IN_WAIT---------- --------- ------------- -------------- --------------- ------------- ---------------

381 enqueue 122961 25 25020 0 46383 enqueue 122961 25 25020 0 43385 enqueue 122961 25 25020 0 80




52/79

51


Oracle 10gwill show the TYPE of enqueue in the event

General Rules for Oracle Waits


53/79

52

General Rules for Oracle Waits

Determine the WAIT event for the sessions, then a course ofaction

Full tablescans are not common and probably should be fixed

Large index RANGE scans may be slow due to single blockfetch from db file sequential read

Transactional enqueues (TX) cause row-level ENQUEUE waits thatare usually waiting for the commit due to high concurrency. Try to

commit more frequently. Determine the TYPE of enqueue

If it is TX, then it is most likely application concurrency

If it is NOT TX, then it may be in the Oracle kernel and moreresearch is needed

Concurrent Select for UPDATE statements are a commoncause of TX enqueues

What Well Cover


54/79

53

Understanding Oracle SGA and Components Understanding index design





SAP Transactions for Tuning: SM66


55/79

54

g

Use SM66 as an indicator of normal SAP workload

SAP Transactions for Tuning: ST04


56/79

55

g

Use Transaction ST04 Detailed Analysis Oracle Session asan indicator of a normal workload


57/79

SAP Transactions for Tuning: SE30


58/79

57

Use SE30 to quickly identify if the problem is in SAP or thedatabase layer

SAP Transactions for Tuning: ST12


59/79

58

Combination ST05 and SE30 (see SAP Note 755977) Holds the history of each trace

SAP Transactions for Tuning: ST12 (cont.)


60/79

59

Here we can see SE30 trace information from the ABAP Trace inST12

SAP Transactions for Tuning: ST12 (cont.)


61/79

60

Here we can see ST05 trace information from theST12 trace


62/79

SAP Programs: RSORADJV


63/79

62

Use to access DBA and V$ view information Run in SE38

SAP Programs: RSORADJV (cont.)


64/79

63

Output from RSORADJV

SAP Programs: RSBDCOS0


65/79

64

Output from RSBDCOS0 Can be used to execute OS-level commands from SAP

OS Debugging Methods


66/79

65

truss Trace Unix system calls

Example:

truss p

strace

Print streams information

strace is available for Windows

sar

Displays system activity (CPU, disk, network)

Example:

sar 5 10

vmstat Displays virtual memory statistics

Example:

vmstat 5 10

OS Debugging Methods (cont.)


67/79

66

ping Used to send network packets to host

Examples:

ping ping t

Use these OS tracing methods in conjunction with the 10046

tracing to confirm elapse times

What Well Cover


68/79

67






Oracle Best Practices: Tablespaces


69/79

68

Locally Managed Tablespaces (LMTS) Benefits:

Does not use the single ST Enqueue, which can cause

performance issues Reduces DBAs workload by configuring auto allocate or

uniform extent sizes

Disadvantages:

Still uses pct_used and pct_free, which are not efficient

Concurrent inserts are still sent to the same free block onthe freelist

Oracle Best Practices: Tablespaces (cont.)


70/79

69

ASSM tablespaces Benefits:

Same benefits as listed on previous slide for LMTS

No longer uses the pct_used and pct_free; instead, managesthe block filling based on the percentages

Concurrent inserts are automatically sent to different blocks

Oracle Best Practices


71/79

70

AUM Automatically configures the number of undo segments needed

Minimize ORA-1555 by keeping the changed data in the undo

segments based on a retention time PGA tuning

Allows the PGA to allocate memory up to a certain value that isgreater than the parameters sort_area_size and hash_area_size

Minimizes sorts operations (I/O) to PSAPTEMP

Oracle Best Practices (cont.)


72/79

71

Auto extend Define only one datafile for each tablespace to auto extend in

the event that the DBA cannot add space in time. This willreduce ORA-1653 and ORA-1654 space errors.

Using parallel query

Use parallel query servers to create indexes faster

Use parallel query servers to create tables faster when using

create table as select commands

Oracle Best Practices (cont.)


73/79

72

SPFILE Using the SPFILE aids in the managing of the database and the

parameters that are changed

Analyze table/index validate structure online Can be used to look for corruption without putting a lock on the

table, which happened in previous Oracle versions

Online table reorganization

Tables that do not have LONG or LONG RAW columns can nowbe reorganized online using DBMS_REDEFINITION package.This method is based on materialized views technology.

What Well Cover


74/79

73






Resources


75/79

74

Oracle/SAP Newsletter www.oracle.com/newsletters/sap/current.html

Wait Events

service.sap.com/notes * SAP Note 619188

* Requires login credentials to the SAP Service Marketplace

7 Key Points to Take Home


76/79

75

Understanding the Oracle fundamentals helps identify problemsand provide solutions

The most efficient I/O done, besides blocks already residing incache, will be 3-4 blocks per row return

Proactivelymonitor the system for inefficient SQL statementsusing ST04 and ST05 transactions

Rebuilding indexes can improve runtimes

Always determine what event is causing users to wait

7 Key Points to Take Home (cont.)


77/79

76

Use SAP transactions ST12, SE30, ST05, SM66, and ST04 toquickly identify the area of waits

Set up the Oracle/SAP database per SAPs Best PracticesStandards (ASSM, AUM, SPFILE, etc.)

Your Turn!


78/79

77

How to contact me:Mike Kennedy

[email protected]

Disclaimer

SAP R/3 mySAP mySAP com xApps xApp SAP NetWeaver Duet PartnerEdge and other SAP products and services mentioned herein as


79/79

78

SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver , Duet , PartnerEdge, and other SAP products and services mentioned herein as

well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All

other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor

controlled by SAP.

advanced internal oracle tuning techniques for sap systems

Documents