oracle rac, data guard, and pluggable databases: when maa meets multitenant (#oow14 version)

2013 © Trivadis

BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

2013 © Trivadis

Oracle RAC, Data Guard, and Pluggable

Databases: When MAA Meets Multitenant

Ludovico CALDARA

Oracle ACE, Senior Consultant

Trivadis AG

2013 © Trivadis

About Ludovico Caldara

28.09.2014

When MAA Meets Multitenant

2

■ 14 Years DBA (started with 7.3.4)

▪ I do it everywhere (also on Windows)

■ RAC ATTACK Ninja & co-writer

■ RAC SIG Board member

■ OCP (11g, 12c, MySQL) & OCE

■ Italian living in Switzerland

■ http://www.ludovicocaldara.net

■ @ludodba

■ ludovicocaldara

http://www.ludovicocaldara.net/

2013 © Trivadis

Trivadis is a market leader in IT consulting, system integration,

solution engineering and the provision of IT services focusing

on and technologies in Switzerland,

Germany and Austria.

We offer our services in the following strategic business fields:

Trivadis Services takes over the interacting operation of your IT systems.

Our company

O P E R A T I O N

28.09.2014


3

2013 © Trivadis

With over 600 specialists and IT experts in your region

4

11 Trivadis branches and more than

600 employees

200 Service Level Agreements

Over 4,000 training participants

Research and development budget:

CHF 5.0 / EUR 4 million

Financially self-supporting and

sustainably profitable

Experience from more than 1,900

projects per year at over 800

customers

Hamburg

Düsseldorf

Frankfurt

Freiburg Munich

Vienna

Basle

Zurich Bern

Lausanne

4

Stuttgart

28.09.2014

When MAA Meets Multitenant 4

2013 © Trivadis

AGENDA

1. RAC and Multitenant

Why and how

Dealing with PDBs and services

Demo

2. RAC, DG and MULTITENANT

Why and how


Demo

28.09.2014


5

2013 © Trivadis

28.09.2014


Part I

6

2013 © Trivadis

28.09.2014


RAC and Multitenant

Why and How

7

2013 © Trivadis

Multitenant on a single instance: is it worth?

28.09.2014


8

Consolidate on one CDB, one server

2013 © Trivadis


28.09.2014


9


The server fills up? Prepare another server and CDB!

2013 © Trivadis


28.09.2014


10



Create new PDBs on the new server

2013 © Trivadis


28.09.2014


11



Create new PDBs on the new server, REPEAT!

2013 © Trivadis


28.09.2014


12



Create new PDBs on the new server, REPEAT!

A PDB increases? Need to make room for the new workload

2013 © Trivadis


28.09.2014


13

Is that consolidation?

2013 © Trivadis


28.09.2014


14

Is that consolidation? What if you have to change static parameters?

2013 © Trivadis


28.09.2014


15


Bouncing your single-instance CDB takes offline all your PDBs

2013 © Trivadis


28.09.2014


16


Bouncing your single-instance CDB takes offline all your PDBs

You would never stop all your services and reboot the instance, right?

Well, on a single-instance you have to do it!

2013 © Trivadis

RAC empowers Multitenant

28.09.2014


17

With RAC+MT, the Container Database itself is a RAC Database

2013 © Trivadis


28.09.2014


18


The PDBs are still available in case of instance crash

2013 © Trivadis


28.09.2014


19


The PDBs are still available in case of instance crash

Instances can be bounced one at a time without affecting the CDB

2013 © Trivadis


28.09.2014


20

The CDB can accommodate new PDBs by adding new servers/instances

2013 © Trivadis


28.09.2014


21

The CDB can accommodate new PDBs by adding new servers/instances

The workload is distributed across instances. How?

2013 © Trivadis

28.09.2014


RAC and Multitenant

Dealing with PDBs and Services

22

2013 © Trivadis

The services are the key!

28.09.2014


23

The PDBs are mounted by default. Starting the service makes the

cluster open the PDB!

Services on CDBRAC_1 Services on CDBRAC_2

Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis


28.09.2014


24

Singleton services open the PDBs only on the nodes they run on.

Only the blocks of accessed PDBs will populate one instance’s cache.


Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis


28.09.2014


25

Access big PDBs from more than one instance

Either declare the service UNIFORM or create a new one for the same PDB


Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis


28.09.2014


26

Need more room/power?


Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis


28.09.2014


27


Add an instance and modify/relocate some services


Cache on CDBRAC_3 Cache on

CDBRAC_1

Services on CDBRAC_2

Cache on CDBRAC_2

2013 © Trivadis


28.09.2014


28


Add an instance and modify/relocate some services

No need of unplugging/plugging PDBs! They’re in the same CDB!

Services on CDBRAC_1


CDBRAC_1 Cache on CDBRAC_2


2013 © Trivadis


28.09.2014


29

If you stop a service

Services on CDBRAC_1 Services on CDBRAC_3 Services on CDBRAC_2



2013 © Trivadis


28.09.2014


30


The PDB is still open but not accessible




2013 © Trivadis


28.09.2014


31


The PDB is still open but not accessible

Its blocks are aged out from the buffer cache using the LRU mechanism




2013 © Trivadis

Service Failover

28.09.2014


32

After instance crash, a singleton service fails over


Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis

Service Failover

28.09.2014


33

After instance crash, a singleton service fails over


Cache on instance

CDBRAC_2

Cache on instance

CDBRAC_1

2013 © Trivadis

Srvctl syntax for services and PDBs

srvctl add service –db db_unique_name \

–service service_name –serverpool server_pool \

–cardinality uniform –role primary \

–failovertype select –failovermethod basic \

-policy automatic -failoverdelay 2 \

–failoverretry 180 -pdb pluggable_database

srvctl modify service … -pdb pluggable_database

There’s an overall limit of 512 services per CDB, be careful!

28.09.2014


34

2013 © Trivadis

Many PDBs, many services, many changes

Important:

Oracle recommends that you limit configuration changes to the minimum

requirement and that you not perform other service operations while the

online service modification is in progress.

http://docs.oracle.com/database/121/RACAD/srvctladmin.htm#RACAD005

28.09.2014


35

2013 © Trivadis

RAC and Multitenant: Little demo?

28.09.2014


36

CDBATL_2

raca01

CDBATL_1

raca02

CDBATL

MAAZ

2013 © Trivadis

28.09.2014


Part II

37

2013 © Trivadis

28.09.2014


RAC, DG and Multitenant

Why and How

38

2013 © Trivadis

RAC, Data Guard & Multitenant, why?

28.09.2014


39

The reasons of having a Data Guard architecture are the same…

2013 © Trivadis

RAC, Data Guard & Multitenant, why?

28.09.2014


40

The reasons of having a Data Guard architecture are the same…

2013 © Trivadis

Multitenant comes with a great benefit…

28.09.2014


41

Single configuration, simplified administration

CDBATL_1 CDBATL_2

CDBATL

MAAZ

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBGVA_1 CDBGVA_2

CDBGVA

MAAZ

MRP0

2013 © Trivadis


28.09.2014


42


CDBATL_1 CDBATL_2

CDBATL

MAAZ LUDO

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBGVA_1 CDBGVA_2

CDBGVA

MAAZ

MRP0

2013 © Trivadis


28.09.2014


43


CDBATL_1 CDBATL_2

CDBATL

MAAZ LUDO

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBGVA_1 CDBGVA_2

CDBGVA

MAAZ LUDO

MRP0

2013 © Trivadis

… or limitation?

28.09.2014


44

Cannot have different PDBs with different roles

CDBATL_1 CDBATL_2

CDBATL

MAAZ LUDO

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBGVA_1 CDBGVA_2

CDBGVA

MAAZ LUDO

MRP0

2013 © Trivadis

How?

28.09.2014


45

ONE CDB = ONE Data Guard configuration

2013 © Trivadis

How?

28.09.2014


46

ONE CDB = ONE Data Guard configuration

All Data Guard commands (alter database or broker commands) are

executed while connected to the CDB$ROOT

CDBATL_1 CDBATL_2

CDBATL (CDB$ROOT)

MAAZ LUDO

2013 © Trivadis

PDB Creation from PDB$SEED

28.09.2014


47

SQL> create pluggable database MAAZ;

CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

SEED SEED

2013 © Trivadis


28.09.2014


48


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

SEED SEED MAAZ

2013 © Trivadis


28.09.2014


49


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

SEED SEED MAAZ MAAZ

Recovery created pluggable database MAAZ

Recovery copied files for tablespace SYSTEM

Recovery successfully copied file +DATA/CDBGVA/…/DATAFILE/system.435.856973955

from +DATA/CDBGVA/…/DATAFILE/system.280.855055053

Successfully added datafile 24 to media recovery

2013 © Trivadis


28.09.2014


50


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

SEED SEED MAAZ MAAZ

Recovery created pluggable database MAAZ





MOUNTED

OR OPEN

READ ONLY

2013 © Trivadis

PDB Creation from other PDB

28.09.2014


51

SQL> create pluggable database LUDO FROM MAAZ;

CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

MAAZ MAAZ

2013 © Trivadis


28.09.2014


52


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

MAAZ MAAZ LUDO

2013 © Trivadis


28.09.2014


53


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

MAAZ MAAZ LUDO LUDO

Recovery created pluggable database LUDO





2013 © Trivadis


28.09.2014


54


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

MAAZ MAAZ LUDO LUDO






ONLY IF

OPEN READ

ONLY

2013 © Trivadis


28.09.2014


55


CDBATL_1 CDBATL_2

CDBATL

LGWR

LGWR

LNS

LNS

RFS

RFS

CDBATL_1 CDBATL_2

CDBATL

MRP0

MAAZ MAAZ LUDO LUDO






ONLY IF

OPEN READ

ONLY

ACTIVE

DATAGUARD

!!!

2013 © Trivadis

PDB Creation from other PDB WITHOUT ADG

28.09.2014


56

Documentation says: copy the datafiles on the standby before the clone

2013 © Trivadis


28.09.2014


57


But RAC => ASM => OMF => Cannot guess the file names!!

Need to workout a solution so that you don’t mess your OMF with aliases

2013 © Trivadis


28.09.2014


58


But RAC => ASM => OMF => Cannot guess the file names!!

Need to workout a solution so that you don’t mess your OMF with aliases

There are different solutions, too complex to cover here. I’ll just show the

easiest one.

2013 © Trivadis

PDB Creation from other PDB WITHOUT ADG *

28.09.2014


59

SQL> create pluggable database LUDO from MAAZ standbys=NONE;

Backup/Restore PDB from Primary to Standby over the network

Stop Recovery

Alter pluggable database LUDO enable recovery;

Start Recovery

* Not the official way, but much easier IMHO with ASM

2013 © Trivadis

Restore PDB from Primary to Standby

28.09.2014


60

PRIMARY

RMAN> backup as copy pluggable database LUDO format '/tmp/ludo%f.dbf';

$ scp /tmp/ludo*.dbf racb01:/tmp

STANDBY

RMAN> catalog start with ‘/tmp/ludo’

RMAN> set newname for pluggable database LUDO to new;

RMAN> restore pluggable database LUDO;

RMAN> switch pluggable database LUDO to copy;

2013 © Trivadis

Srvctl syntax for services and PDBs with ADG

srvctl add service –db db_unique_name \

–service ro_service_name –serverpool server_pool \

–cardinality uniform –role pyhsical_standby \

–failovertype select –failovermethod basic \

-policy automatic -failoverdelay 2 \

–failoverretry 180 -pdb pluggable_database

Create the same service on the Primary first, so it

exists and can run on the standby

Many services per PDB: don’t forget the 512 service limit per CDB!!

28.09.2014


62

2013 © Trivadis

Connection descriptor for MAA

LUDOAPP = (DESCRIPTION_LIST=

(LOAD_BALANCE=off) (FAILOVER=on)

(DESCRIPTION = (CONNECT_TIMEOUT=5)

(TRANSPORT_CONNECT_TIMEOUT=3) (RETRY_COUNT=3)

(ADDRESS_LIST= (LOAD_BALANCE=on)

(ADDRESS = (PROTOCOL = TCP)(HOST = raca-scan)(PORT = 1521)))

(CONNECT_DATA = (SERVICE_NAME = LUDOAPP)) )

(DESCRIPTION = (CONNECT_TIMEOUT=5)

(TRANSPORT_CONNECT_TIMEOUT=3) (RETRY_COUNT=3)

(ADDRESS_LIST= (LOAD_BALANCE=on)

(ADDRESS = (PROTOCOL = TCP)(HOST = racb-scan)(PORT = 1521)))

(CONNECT_DATA = (SERVICE_NAME = LUDOAPP)) ) )

28.09.2014


63

2013 © Trivadis

RAC, DG and Multitenant: Little demo?

28.09.2014


64

raca01 raca02

racb01 racb02

CDBATL

Data Guard

CDBGVA

2013 © Trivadis


2013 © Trivadis

Thank you!

Ludovico Caldara

Oracle ACE, Senior Consultant

Trivadis AG

[email protected]

@ludodba

oracle rac, data guard, and pluggable databases: when maa meets multitenant (#oow14 version)

Technology