maximizing oracle real application clusters 11g sharing the load with services

47
© 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. James Harding - Database Manager Sabre Sherri Trojan - Database Architect HP October 2, 2009 Maximizing Oracle Real Application Clusters 11g: Sharing the Load with Services

Upload: igoroshka

Post on 28-Apr-2015

96 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

© 2008 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice.

James Harding - Database Manager Sabre

Sherri Trojan - Database Architect HP

October 2, 2009

Maximizing Oracle Real Application Clusters 11g: Sharing the Load with Services

Page 2: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Today

GoldenGate Data Pump Servers

Logical standby DB

Oracle Master DB Oracle

Re-Do Logs

Content Application Servers - CRUD Activity GenGen

GoldenGate Extractor/Replicator

Gen Gen

160 MySQL Database Servers

Oracle Database Servers (3-SCC, 3-CDC)

Gen Gen

AVS DSS

GenGen

Gen

Hist

AVS DSS

GenGen

Gen

Hist

1

23

4

5

6

GG Local Trails

Extract Trail Split

GG Remote Trails

Page 3: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Tomorrow

GoldenGate Data Pump Servers

Logical standby DB

Oracle Master DB Oracle

Re-Do Logs

Content Application Servers - CRUD Activity

GoldenGate Extractor/Replicator

8 Oracle 2 Node RAC Database Clusters - Blades

Oracle Database Servers (3-SCC, 3-CDC)

1

23

4

5

GG Local Trails

Extract Trail Split

GG Remote Trails

Page 4: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Objective• Improve:

−Capacity & Scalability

−Operation Support

−Performance Monitoring & Performance Management Features

• Phase 1:−Connection Load Balance

−Home grown connection pooling

−Clients notify us that there is a problem.

• Phase 2:−Runtime Load Balance

−Oracle connection pooling

−We notify the client that there is a problem.

Page 5: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clark

Page 6: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

APP1 APP1APP2 APP2 APP3

RAC Service Station

APP1 APP1

Load balance to server with least load

Performance

Trace

AWR

Failover

Load balance

Monitor

Event

Services

provide

for:

Page 7: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Sharing the Load with Services4 Main Options:

•Client side connection load balancing

•Server side connection load balancing

•Server side connection load balancing with Load Balancing advisor

•Runtime Load balancing with Load balancing advisor

Page 8: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Client side connection load balancing

Page 9: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Load Balance On Connection

Client Side - TNSNAMESCLIENT1 = (DESCRIPTION = (LOAD_BALANCE = ON) # ON=Random Connections OFF=Sequential connections (FAILOVER = ON) # Activates failover when TCP timeout is reached (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux2-vip)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = racdb) ) )

Random connections one listener is not overwhelmed

Page 10: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Server side connection load balancing

Balances incoming connections across all of the instances that provide the requested database service.

Load Balancing Advisor – Uses current service levels

Page 11: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Connection Load Balancing• Short - Use Load Balancing Advisory, when Load Balancing

Advisory is enabled with Service Goals.• Long - Balances the number of connections per instance

using session count per service.

Service Goals• None• Service Time - Best response time – elapsed time

per call metric. Example: Internet shopping

• Throughput - Better throughput – rate that work is completed

per second metric. Example: Batch

On Connection – Server Side

Load Balancing Advisor : ON or OFF

Page 12: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

•Add a service: srvctl add service -d RACDB1 -s SERV1 -r LINUX1 -a LINUX2 -P basic

-r = Preferred list -a = available list (failover)

•Start service: srvctl start service -d RACDB1 -s SERV1

•List services in OCR: srvctl config service -d RACDB1 -a

•Create TNS Entry

•Use DBMS_SERVICE.MODIFY_SERVICE to set goals and failover

•Verify service in listener: lsnrctl status

•List services in listener: lsnrctl services

Adding a service

Can be done with DBCA, OEM, SRVCTL

Page 13: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

From sqlplus, modify service for goals:

BEGIN

DBMS_SERVICE.MODIFY_SERVICE(

service_name => 'racload'

,failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC

,failover_type => DBMS_SERVICE.FAILOVER_TYPE_SESSION,

,failover_retries => 10,

,failover_delay => 1,

,goal => DBMS_SERVICE.GOAL_SERVICE_TIME

,clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT );

END;

/

Service with Load Balance Advisor

Page 14: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

PMON process communicates with other listeners

•PMON process registers database with database’s local listener•Cross-registers database with listeners on all other nodes in the cluster •PMON by default checks CPU every 3 seconds•PMON updates the listener with statistics. By default, PMON updates TNSLSNR every 10 minutes.•Listener routes incoming connections to the least loaded instance.

Server Side Load Balancing

PMON

Listener ListenerListener

Page 15: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

TNSNAMES entry

LISTENERS_RACDB = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = rac3-vip)(PORT = 1521))

How does PMON which listeners to register with?

Server Side Load Balancing

PMON

Listener ListenerListener

init.ora: REMOTE_LISTENERS=LISTENERS_RACDB

Page 16: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

MMNL sys.sys$Service_metrics_tab

PMONMMON

AQ

Listener Listener ListenerFan Event

ONS$CRS_HOME/opmn/conf/

ons.config

MMNL - Memory Monitor LightMMON - Manageability MonitorONS - Oracle Notification Service

Updated every 30sHas FAN events

Load Balancing Advisor processes

Fast connection failover

racgimon

Updated every few seconds - 10 minutes

OCI.NET

Java JDBC

Page 17: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

2009-03-02 11:54:47.373624 : nsglb:instance:RACNY11 new lbscore:3002009-03-02 11:54:48.422227 : nsglb:instance:RACNY21 new lbscore:1002009-03-02 11:54:49.477567 : nsglb:instance:RACNY21 new lbscore:2002009-03-02 11:54:50.536465 : nsglb:instance:RACNY21 new lbscore:3002009-03-02 11:54:51.589902 : nsglb:instance:RACNY21 new lbscore:4002009-03-02 11:54:52.642743 : nsglb:instance:RACNY11 new lbscore:4002009-03-02 11:54:53.703777 : nsglb:instance:RACNY21 new lbscore:500

• Oracle 10.2 and higher: Listener uses a local value "lbscore“• Lbscore : two dynamic values "goodness" and "delta" • Continuously updated by PMON. • PMON contacts the listener and provides starting values for goodness and delta...

        Listener lbscore = Goodness from PMON         Listener Delta = Delta from PMON

• To account for changes in load between PMON updates the listener will increment the lbscore after each new incoming connection...

        Listener lbscore = Listener lbscore(previous) + Listener Delta

LBSCORE – Surge Protection

Page 18: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

•PMON updates the listener with instance load and dispatcher information•PMON SERVICE_UPDATE’s vary according to the workload of the instance•The maximum interval between these service updates is 10 minutes.

Update listener information

05-MAR-2009 13:31:15 * service_update * RACNY11 * 005-MAR-2009 13:31:16 * service_update * RACTX22 * 005-MAR-2009 13:31:19 * service_update * RACTX21 * 005-MAR-2009 13:31:31 * service_update * RACTX22 * 005-MAR-2009 13:31:52 * service_update * RACNY12 * 0

Setting a Listener Trace:’ lsnrctl set trc_level 16’ produces a trace log that shows the Load Balance Advisory

updates2009-03-05 16:24:17.185722 : nsglgrDoRegister:inst loads: ld1:102 mld1:81920 ld2:17

mld2:60002009-03-05 16:24:17.185746 : nsglgrDoRegister:service:RACNY_STAGING what:4 value:12009-03-05 16:24:17.185763 : nsglgrDoRegister:service:RACNY_STAGING what:2 value:02009-03-05 16:24:17.185799 : nsglgrDoRegister:service:RACNY_RTG what:4 value:1002009-03-05 16:24:17.185817 : nsglgrDoRegister:service:RACNY_RTG what:2 value:1002009-03-05 16:24:17.186112 : nsglgrDoRegister:service:RACNY1 what:4 value:12009-03-05 16:24:17.186129 : nsglgrDoRegister:service:RACNY1 what:2 value:22009-03-05 16:24:17.186157 : nsglgrDoRegister:exit2009-03-05 16:24:17.186193 : nsdo:entry

What:4 = Delta What: 2 = Goodnessld1 = node load - load average ld2 = instance load - number user

sessionsmld1 = max load data/ max cpu

capacity mld2 = max load data/sessions init.ora

Check Listener log. When busy, it updated every few seconds

Page 19: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Load Balance – Multi Cluster

Page 20: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

ractx

racpl racok

rachp

sqlplus name/pass@racload

GOODNESS: 15

GOODNESS: 100

GOODNESS: 15

GOODNESS: 20

racny

Load Balancing

Server side Load Balancing across clusters

Page 21: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

ractx

racpl racok

rachp

sqlplus name/pass@racload

GOODNESS: 15

GOODNESS: 100

GOODNESS: 15

GOODNESS: 20

racny

GOODNESS 1

Load Balancing

Server side Load Balancing across clusters

Goodness: ranking for the quality of service that the service is experiencing at an instance level, including whether access has been restricted from an instance. Lower number is better

Page 22: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

LISTENERS_RACDBALL = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = ractx1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = ractx2-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = racpl1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = racpl2-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = racok1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = racok2-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = rachp1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = rachp2-vip)(PORT = 1521)) )

'ALTER SYSTEM SET REMOTE_LISTENER = LISTENERS_RACDBALL SID='*' SCOPE=BOTH;'

TNS Entry

SPFILE Create RAC service (from unix srvctl command) srvctl config service -d ractx -a srvctl add service -d ractx -s racload -r ractx1, ractx2 -P basic srvctl start service -d ractx -s racload srvctl config service -d ractx -a

From sqlplus, modify service for goals:BEGINDBMS_SERVICE.MODIFY_SERVICE( service_name => 'racload' ,failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC ,failover_type => DBMS_SERVICE.FAILOVER_TYPE_SESSION, ,failover_retries => 10, ,failover_delay => 1, ,goal => DBMS_SERVICE.GOAL_SERVICE_TIME ,clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT );END;/

ractx racpl racok rachp

sqlplus name/pass@racload

GOODNESS: 1 GOODNESS: 100GOODNESS: 1GOODNESS: 10

Verify service in listener: lsnrctl statusList services in listener: lsnrctl

services

Server side Load Balancing across clusters

Page 23: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Bringing a multi-cluster online• Duplicate rac database• Start replication• Add services with goals • Add entry to tnsnames remote_listener entry:• Alter system set remote_listener='LISTENERS_RACDB1';• $ lsnctl status

  should show all 4 instances for each oracle service

  Service "RACDB_MYAPP" has 4 instance(s).    Instance "RACDB11", status READY, has 16 handler(s) for this service...    Instance "RACDB12", status READY, has 16 handler(s) for this service...    Instance "RACDB21", status READY, has 17 handler(s) for this service...    Instance "RACDB22", status READY, has 16 handler(s) for this service...

• Client should set a TCP connect timeout. Syntax varies by client type

Example: SQLNET.OUTBOUND_CONNECT_TIMEOUT=4

Page 24: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Issue: Dispatcher blocked

•Listener log: TNS-12520 errors

•lsnrctl services:Service "RACDB_AS" has 2 instance(s).Instance "RACDB1", status READY, has 11 handler(s) for this

service...Handler(s):D009" established:15499 refused:0 current:972 max:972

state:blocked

•Sqlplus: select count(*) from v$circuit does not show many sessions

Page 25: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Dispatcher Blocked•Reason: Dispatcher defaults to public host. •Check in: ‘network' column of v$dispatcher •'lsnrctl services‘•It's also described in metalink note: 578524.1

Fix: alter system set dispatchers='(address=(protocol=tcp)(host=racdb-vip))

(dispatchers=10)’ (index=0) scope=both sid=‘RACDBxx'; Info:

A listener will hand off a connection to a dispatcher if it is local.

If it is remote, it will send the client a new host/port and the client

will reconnect to the dispatcher.

The fix was to modify the dispatcher with a host vip entry. This specifies

the endpoint listener.

Page 26: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Test Results

Page 27: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Load Balancing

HOST   LOAD  COUNT     

RACNY

RACTX

racny1 0.269 156

racny2 0.729 92

ractx1 2.379 7

ractx2 0.419 119

Page 28: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

0

10

20

30

40

50

60

70

80

90

100

10

-09

:56:3

1

10

-09

:56:3

9

10

-09

:56:4

5

10

-09

:56:5

1

10

-09

:56:5

7

10

-09

:57:0

4

10

-09

:57:1

2

10

-09

:57:1

8

10

-09

:57:2

4

10

-09

:57:3

1

10

-09

:57:3

7

10

-09

:57:4

3

10

-09

:57:5

0

10

-09

:57:5

6

10

-09

:58:0

2

10

-09

:58:1

0

10

-09

:58:1

9

10

-09

:58:2

6

10

-09

:58:3

5

10

-09

:58:4

4

10

-09

:58:5

1

10

-09

:59:0

0

10

-09

:59:0

9

10

-09

:59:1

5

10

-09

:59:2

1

10

-09

:59:2

9

10

-09

:59:3

7

10

-09

:59:4

5

10

-09

:59:5

4

10

-10

:00:0

2

10

-10

:00:1

0

10

-10

:00:1

9

10

-10

:00:2

7

10

-10

:00:3

5

10

-10

:00:4

4

10

-10

:00:5

2

10

-10

:01:0

0

10

-10

:01:0

9

10

-10

:01:1

6

10

-10

:01:2

2

10

-10

:01:2

9

10

-10

:01:3

5

10

-10

:01:4

1

10

-10

:01:4

7

10

-10

:01:5

3

10

-10

:01:5

9

10

-10

:02:0

6

10

-10

:02:1

4

10

-10

:02:2

0

10

-10

:02:2

6

10

-10

:02:3

2

10

-10

:02:3

9

Average CPU Utilization

time

Inst

0

100

200

300

400

500

600

700

10

-0

9:5

6:3

1

10

-0

9:5

6:3

9

10

-0

9:5

6:4

5

10

-0

9:5

6:5

1

10

-0

9:5

6:5

7

10

-0

9:5

7:0

4

10

-0

9:5

7:1

2

10

-0

9:5

7:1

8

10

-0

9:5

7:2

4

10

-0

9:5

7:3

1

10

-0

9:5

7:3

7

10

-0

9:5

7:4

3

10

-0

9:5

7:5

0

10

-0

9:5

7:5

6

10

-0

9:5

8:0

2

10

-0

9:5

8:1

0

10

-0

9:5

8:1

9

10

-0

9:5

8:2

6

10

-0

9:5

8:3

5

10

-0

9:5

8:4

4

10

-0

9:5

8:5

1

10

-0

9:5

9:0

0

10

-0

9:5

9:0

9

10

-0

9:5

9:1

5

10

-0

9:5

9:2

1

10

-0

9:5

9:2

9

10

-0

9:5

9:3

7

10

-0

9:5

9:4

5

10

-0

9:5

9:5

4

10

-1

0:0

0:0

2

10

-1

0:0

0:1

0

10

-1

0:0

0:1

9

10

-1

0:0

0:2

7

10

-1

0:0

0:3

5

10

-1

0:0

0:4

4

10

-1

0:0

0:5

2

10

-1

0:0

1:0

0

10

-1

0:0

1:0

9

10

-1

0:0

1:1

6

10

-1

0:0

1:2

2

10

-1

0:0

1:2

9

10

-1

0:0

1:3

5

10

-1

0:0

1:4

1

10

-1

0:0

1:4

7

10

-1

0:0

1:5

3

10

-1

0:0

1:5

9

10

-1

0:0

2:0

6

10

-1

0:0

2:1

4

10

-1

0:0

2:2

0

10

-1

0:0

2:2

6

10

-1

0:0

2:3

2

10

-1

0:0

2:3

9

Average of goodness

time

Inst

Swingbench with a slow ramp up of 250 sessions

0

2

4

6

8

10

12

14

16

18

10-0

9:5

6:3

1

10-0

9:5

6:3

9

10-0

9:5

6:4

5

10-0

9:5

6:5

1

10-0

9:5

6:5

7

10-0

9:5

7:0

4

10-0

9:5

7:1

2

10-0

9:5

7:1

8

10-0

9:5

7:2

4

10-0

9:5

7:3

1

10-0

9:5

7:3

7

10-0

9:5

7:4

3

10-0

9:5

7:5

0

10-0

9:5

7:5

6

10-0

9:5

8:0

2

10-0

9:5

8:1

0

10-0

9:5

8:1

9

10-0

9:5

8:2

6

10-0

9:5

8:3

5

10-0

9:5

8:4

4

10-0

9:5

8:5

1

10-0

9:5

9:0

0

10-0

9:5

9:0

9

10-0

9:5

9:1

5

10-0

9:5

9:2

1

10-0

9:5

9:2

9

10-0

9:5

9:3

7

10-0

9:5

9:4

5

10-0

9:5

9:5

4

10-1

0:0

0:0

2

10-1

0:0

0:1

0

10-1

0:0

0:1

9

10-1

0:0

0:2

7

10-1

0:0

0:3

5

10-1

0:0

0:4

4

10-1

0:0

0:5

2

10-1

0:0

1:0

0

10-1

0:0

1:0

9

10-1

0:0

1:1

6

10-1

0:0

1:2

2

10-1

0:0

1:2

9

10-1

0:0

1:3

5

10-1

0:0

1:4

1

10-1

0:0

1:4

7

10-1

0:0

1:5

3

10-1

0:0

1:5

9

10-1

0:0

2:0

6

10-1

0:0

2:1

4

10-1

0:0

2:2

0

10-1

0:0

2:2

6

10-1

0:0

2:3

2

10-1

0:0

2:3

9

Average of load

time

Inst

racny11racny12ractx21tactx22

load

cpu goodness

Page 29: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Runtime Load Balancing

Allows Oracle Clients to provide intelligent allocations of connections based on the current service level provided by the database instances when applications request a connection to complete some work.

Page 30: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

• OCI - Oracle Call Interface. Used in C and C++ as well as other languages that are built on top of OCI layer such as perl and PHP

• Thick JDBC - Build around OCI libraries to implement JCBC API standard for Java Language

• Thin JDBC - Native JDBC. Pure JAVA Implementation that does not need underlying OCI layer. Does not support TAF. Does support Fast Connection Failover.

• ODP.NET - Oracle Data Provider (ODP) for Microsoft’s .NET development platform. Based on OCI

Interface

Page 31: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

•Oracle Notification Service (ONS) Communicates/interacts with components like the listener, application, etcserver-side ONS through ORA_CRS_HOME/opmn/conf/ons.config. Use racgons to add middle-tier node info to the OCRNo ONS daemon needed on the client when using Remote ONS SubscriptionOracle Clusterware and RAC utilize ONS to propagate FAN messages both within the RAC cluster, and to client or mid-tier machines

•FAN - Fast Application Notification quickly alert applications about configuration and workload service level changes

•FCF - Fast Connection Failover. (do not use TAF if using FCF)Supports both JDBC Thin and JDBC Oracle Call Interface (OCI) drivers.Requires ONSSubscribes to FAN events

Terms

Page 32: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

To use FAN callouts: place an executable in the directory: CRS_home/racg/usrco on every node that runs CRS Example #! /bin/ksh FAN_LOGFILE= [your path name]/admin/log/`hostname`_uptime.log echo $* "reported="`date` >> $FAN_LOGFILE &

FAN events are published using ONS and Advanced Queuing. • The Connection Manager (CMAN) and Oracle Net Services Listeners are integrated with FAN events. • Listener and CMAN immediately de-register services provided by a failed instance

ONS is used to propagate FAN messages.

Verify ONS is running: srvctl status nodeapps -n {server}   $CRS_HOME/bin/onsctl ping   $CRS_HOME/bin/onsctl debug

Fast Application Notification

Page 33: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Runtime Load Balancing

• JAVA JDBC and JAVA thin − use ONS to subscribe to FAN events− Turning on Fast Connection Failover feature with JDBC includes Runtime Connection

Load Balancing

• OCI − Uses the System Alert Queue (aq) to subscribe to FAN events.− Runtime connection load balancing is enabled by default in a release 11.1 or higher

client talking to server of 10.2 or higher. − The application must have been linked with the threads library.− The OCI environment must be created in OCI_EVENTS and OCI_THREADED mode− DBMS_SERVICE.MODIFY_SERVICE(

 service_name => 'MYSERVICE‘ ,aq_ha_notifications => true

 ,goal => DBMS_SERVICE.GOAL_SERVICE_TIME ,clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT ); END; /

− To view: DBA_OUTSTANDING_ALERTS or DBA_ALERT_HISTORY.

Page 34: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Runtime Load balancing

• Balance connections across all of the Oracle RAC instances when a service starts; this is preferable to directing the sessions that are defined for the connection pool to the first Oracle RAC instance that supports the service

•Remove terminated connections immediately when a service is declared DOWN at an instance, and immediately when nodes are declared DOWN

•Report errors to clients immediately when Oracle detects the NOT RESTARTING state, instead of making the client wait while the service repeatedly attempts to restart

•Balance work requests at run time using load balancing advisory events

Page 35: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Transaction ApplicationFailover

Page 36: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Transaction Application Failover -

TNSNAMES Entry

STOCK = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = uspls330-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = uspls331-vip)(PORT = 1521)) (LOAD_BALANCE = yes) (FAILOVER=on) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = stock) (FAILOVER_MODE = (TYPE = SELECT) (METHOD = BASIC) (RETRIES = 180) (DELAY = 5) ) ) )

Transaction Application Failover can also be associated with a service using the dbms_service package when using the BASIC method

session: Fails over the session; Does not attempt to recover selects.select: Allows users with open cursors to continue fetching on them after failure. Involves overhead on the client side in normal select operations.none: This is the default, in which no failover functionality is used.

METHOD: Specify how fast failover is to occur from the primary node to the backup node: basic: Establishes connections at failover time. This option requires almost no work on the backup database server until failover time. preconnect: Pre-establishes connections. This provides faster failover but requires that the backup instance be able to support all connections from every supported instance.

Page 37: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

BEGINDBMS_SERVICE.MODIFY_SERVICE( service_name => ‘SERV1' ,goal => DBMS_SERVICE.GOAL_SERVICE_TIME ,clb_goal => DBMS_SERVICE.CLB_GOAL_SHORT ,failover_method => DBMS_SERVICE.FAILOVER_METHOD_BASIC ,failover_type => DBMS_SERVICE.FAILOVER_TYPE_SELECT ,failover_retries => 50 ,failover_delay => 5); END;

DBMS_SERVICE package

SQL>

Page 38: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

What To Expect After TAF Failover

TYPE=SESSION• If any uncommitted transaction existed in the session ORA-25402: transaction must roll back

• If no active transactionSELECT statements will get ORA-25401: can not continue

fetchesEven though error may be returned, session will be connected

tosurviving instance and work may continue

TYPE=SELECTSimilar behavior to above, but SELECT statements will continue

fetches without interruption if no transaction in progress

Transaction Application Failover

Page 39: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Monitoring and Tracing

Page 40: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Monitoring ServicesUse OEM Database Maintenance page - Services – Cluster Management Database Services

Set Thresholds per service on elapsed or cpu

Identify notification of alerts via Notification Rules: Service CPU Time (per user call) (microseconds) Service Response Time (per user call) (microseconds)

Page 41: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Tracing Services

execute dbms_monitor.serv_mod_act_trace_enable( - service_name => 'SERV1', - module_name => 'xyz', - (default is all modules)

waits => TRUE, - binds => FALSE);

select * from dba_enabled_traces;

BEGIN DBMS_APPLICATION_INFO.set_module(module_name => 'add_order', action_name => 'insert into orders');END;

Disable:execute dbms_monitor.serv_mod_act_trace_disable( - service_name => 'SERV1', - module_name => 'xyz');

select * from dba_enabled_traces;

To set the module:

Start:

Page 42: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Stopping a Service

Page 43: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Stop a service srvctl stop service -d RACDB -s SERV1

Stop a service and force off connections srvctl stop service -d RACDB -s SERV1 -i RACDB1 -f

Disconnect all sessions for a service: DBMS_SERVICE.DISCONNECT_SESSION(‘SERV1');

Stop a Service

To gracefully take a service offline: Stop the service on an instance to prevent further connections. At a predefined time, disconnect the remaining sessions.

App

App

App

srvctl stop service -d RACDB -s SERV1 DBMS_SERVICE.DISCONNECT_SESSION(‘SERV1');

App

App

App

App

App

Page 44: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Views

Page 45: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Service views:gv$active_services Active services

gv$session Service_name by session

gv$service_stats Stats by service

calls, execute, parse, etc

dba_services goal, failover, etc

gv$servicemetric Goodness and flags

Use group_id=10 for latest entry

Updated every 5 seconds

gv$servicemetric_history Servicemetric history

Contains 1 hour of history

sys.sys$service_metrics_tab LBA advisory events

Updated every 30 secondsHas RLB fan event

Page 46: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

Technology for better business outcomes

Page 47: Maximizing Oracle Real Application Clusters 11g Sharing the Load With Services

47 April 11, 2023

While at Oracle OpenWorld….• Visit HP in Moscone South, Booth #1301• Assess your IT environment – for a chance

to win an HP Mini Notebook and donate to Feeding America!−Take the TCO Challenge

−Modernization Impact Quiz

−Storage and Server Assessment

−Adaptive Infrastructure Maturity Model

−ERP Optimization Assessment