identifying performance problems in a multitenant environment · logical and physical database...

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

@ChrisAntognini antognini.ch/blog

Christian Antognini

Identifying Performance Problems in a

Multitenant Environment

@ChrisAntognini

Identifying Performance Problems in a Multitenant Environment2 2018-09-19

Senior principal consultant, trainer and partner at Trivadis

christian.antognini@trivadis.com

Focus: get the most out of database engines

Logical and physical database design

Query optimizer

Application performance management

Author of Troubleshooting Oracle Performance (Apress, 2008/14)

OakTable Network, Oracle ACE Director

Agenda

Identifying Performance Problems in a Multitenant Environment3 2018-09-19

1. Introduction

2. SQL Trace

3. Dynamic Performance Views

4. AWR

5. Enterprise Manager 13.3

Introduction

2018-09-19 Identifying Performance Problems in a Multitenant Environment4

Objective of a Performance Analysis

Identifying Performance Problems in a Multitenant Environment5 2018-09-19

Discover the most time-consuming SQL statements or PL/SQL code

invocations

For each of those time-consuming statements, gather additional information

that can help in understanding the problem

Execution plan

Runtime statistics like the number of processed rows

Resource usage profile

Basic Questions that Require Answers

Identifying Performance Problems in a Multitenant Environment6 2018-09-19

Is the problem reproducible at will?

Yes: everything is much easier than if you can’t!

SQL Trace

For irreproducible problems, is it possible to wait till the problem occurs again?

Yes: a real-time analysis has to be carried out

Dynamic performance views

No: a repository holding historical performance statistics is required

AWR

Disclaimer

Identifying Performance Problems in a Multitenant Environment7 2018-09-19

I’m neither covering basic information about Multitenant, SQL trace, V$ views

nor AWR

I consider that you know

the Multitenant architecture

how to use SQL trace, V$ views and AWR

This presentation only covers the differences introduced by Multitenant

Identifying Performance Problems in a Multitenant Environment8 2018-09-19

SQL Trace

Enabling/Disabling SQL Trace

Identifying Performance Problems in a Multitenant Environment9 2018-09-19

No difference compared to non-Multitenant

There is also no specific support for Multitenant

E.g. no cross container enable/disable

The components for which SQL trace was enabled are visible in the root

through CDB_ENABLED_TRACES

Trace Files Location

Identifying Performance Problems in a Multitenant Environment10 2018-09-19

All PDBs plugged in a given CDB write trace data to the same directory

Giving selective OS access to that directory is challenging

Full access can lead to security issues

_TRACE_FILES_PUBLIC can’t be set at the PDB level

As of 12.2, V$ views like V$DIAG_TRACE_FILE and

V$DIAG_TRACE_FILE_CONTENTS can be used instead

SELECT value FROM v$diag_info WHERE name = 'Diag Trace'

Trace Files

Identifying Performance Problems in a Multitenant Environment11 2018-09-19

A trace file can contain information about several containers

Especially for trace files related to background processes (e.g. PX),

whenever possible use TRACEFILE_IDENTIFIER

The container and the service that generated the trace data are identified

*** SESSION ID:(44.48140) 2018-06-13T23:27:45.753105+02:00

*** CLIENT ID:() 2018-06-13T23:27:45.753111+02:00

*** SERVICE NAME:(pdb122) 2018-06-13T23:27:45.753118+02:00

*** MODULE NAME:(SQL*Plus) 2018-06-13T23:27:45.753124+02:00

*** ACTION NAME:() 2018-06-13T23:27:45.753150+02:00

*** CLIENT DRIVER:(SQL*PLUS) 2018-06-13T23:27:45.753156+02:00

*** CONTAINER ID:(3) 2018-06-13T23:27:45.753163+02:00

TRCSESS

Identifying Performance Problems in a Multitenant Environment12 2018-09-19

It doesn’t provide a parameter to select trace data of a specific container

A possible workaround is to use the service name

trcsess [output=<output file name >]

[session=<session ID>] [clientid=<clientid>]

[service=<service name>] [action=<action name>]

[module=<module name>] <trace file names>

TKPROF

Identifying Performance Problems in a Multitenant Environment13 2018-09-19

As of 12.2, it provides the parameter PDBTRACE

It’s unrelated to Multitenant!

It can be used to process a trace file without requiring direct access to the OS

where it’s stored

Trace files can be read through V$ views

E.g. V$DIAG_TRACE_FILE and V$DIAG_TRACE_FILE_CONTENTS

Identifying Performance Problems in a Multitenant Environment14 2018-09-19

Dynamic Performance Views

CON_ID

Identifying Performance Problems in a Multitenant Environment15 2018-09-19

Identifier present in all V$ views

It’s value is either

0: instance-level information

1: root container

n: id of the container

Instance-level Views

Identifying Performance Problems in a Multitenant Environment16 2018-09-19

Some views provide only instance-level statistics

CON_ID = 0

E.g.

V$OSSTAT

V$SGA

V$ASH_INFO

Instance-level and PDB-level Views

Identifying Performance Problems in a Multitenant Environment17 2018-09-19

Some statistics are provided by level-specific views

One view provides cumulative statistics at the instance level (CON_ID = 0)

Another view provides statistics at the PDB level

E.g.

V$(CON_)SYS_TIME_MODEL

V$(CON_)SYSSTAT

V$(CON_)SYSTEM_WAIT_CLASS, V$(CON_)SYSTEM_EVENT

V$(CON_)SYSMETRIC (V$CON views about metrics as of 12.2)

Generic Views

Identifying Performance Problems in a Multitenant Environment18 2018-09-19

Some views show different data depending from where they’re queried

Root: all data

PDB: only PDB-relevant data

E.g.

V$SQL, V$SQL_PLAN_STATISTICS_ALL

V$FILESTAT

V$SESSION, V$PROCESS

– CON_ID = 0 is used for background processes

Make Queries Compatible with Multitenant

Identifying Performance Problems in a Multitenant Environment19 2018-09-19

CON_ID might need to be added

Because of level-specific views, to make queries work at both the instance

level and the PDB level, PDB-level views should be used

Is it always sensible to do so?

Different queries might be better…

Be very careful in mixing data coming from different levels

ASH Reports

Identifying Performance Problems in a Multitenant Environment20 2018-09-19

As of 12.2, when a report is created at the PDB level, the data location can be

specified

SQL> @?/rdbms/admin/ashrpt.sql

…

Specify the location of AWR Data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AWR_ROOT - Use AWR data from root (default)

AWR_PDB - Use AWR data from PDB

…

No major difference

between these two

reports

Identifying Performance Problems in a Multitenant Environment21 2018-09-19

AWR

PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment22 2018-09-19

In 12.1 snapshots are only created at the root level

As of 12.2 they can be created at both the root level and the PDB level

By default only root-level snapshots are taken

Two new sets of views are available: AWR_PDB and AWR_ROOT

Enable PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment23 2018-09-19

Set AWR_PDB_AUTOFLUSH_ENABLED = TRUE

Set the snapshot interval

In case a CDB has many PDB, set AWR_SNAPSHOT_TIME_OFFSET

Note that every container has its own snapshot ids

dbms_workload_repository.modify_snapshot_settings(interval=>30)

AWR Reports (1)

Identifying Performance Problems in a Multitenant Environment24 2018-09-19

Reports created at the root level are superior

As of 12.2, when a report is created at the PDB level, the data location can be

specified

SQL> @?/rdbms/admin/awrrpt.sql

…

Specify the location of AWR Data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AWR_ROOT - Use AWR data from root (default)

AWR_PDB - Use AWR data from PDB

…

No major difference

between these two

reports

AWR Reports (2)

Identifying Performance Problems in a Multitenant Environment25 2018-09-19

As of 12.2, when executed at the PDB level, less information is provided

(there are less differences in 12.1)

PDB level

Root level

AWR Reports (3)

Identifying Performance Problems in a Multitenant Environment26 2018-09-19

A number of tables also provide the PDB name

E.g. SQL ordered by …

Creating a PDB from a Non-CDB

Identifying Performance Problems in a Multitenant Environment27 2018-09-19

In 12.1, after such an operation, AWR data is no longer accessible

But it consumes space

Refer to bug# 25698110

If needed, export the data before the operation takes place

Identifying Performance Problems in a Multitenant Environment28 2018-09-19

Enterprise Manager 13.3

Multitenant Support

Identifying Performance Problems in a Multitenant Environment29 2018-09-19

In general, the Multitenant support

in the performance-related pages

is weak

2+ years old

tweet

Multitenant Support in 13.3

Identifying Performance Problems in a Multitenant Environment30 2018-09-19

For a number of functionalities

Not feasible to focus on a PDB

Access to the CDB is required

Performance Home

The load of a PDB is visible in

the Services and PDBs tabs

Top Activity

PDB name added to some lists

No PDB filter in ASH Report

Support for PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment31 2018-09-19

Summary

Identifying Performance Problems in a Multitenant Environment33 2018-09-19

SQL trace

– No PDB-level trace directory

– Workflow almost identical

V$ views provide expected statistics

AWR

– If possible create reports at the

root level

– Consolidation <> easy

Weak Multitenant support in EM

Questions and Answers

Christian Antognini

Senior Principal Consultant

christian.antognini@trivadis.com

@ChrisAntognini

2018-09-19 Identifying Performance Problems in a Multitenant Environment34