phoug rac presentation final
TRANSCRIPT
-
8/2/2019 PHOUG RAC Presentation Final
1/39
-
8/2/2019 PHOUG RAC Presentation Final
2/39
Who Am I
EBS , RAC DBA Practitioner
12 years in Oracle field
Part of RAC Optimization &
Performance Team @DLL
Special projects & Critical
Services Benchmarking , Performance
Tuning
March 21 | PHL Oracle User Group
Disclaimer
These slides and materials represent the work and opinions of the author and do not constitute official
positions of my current or past employer or any other organization. This material has been peer
reviewed, but author assume no responsibility whatsoever for the test cases.
-
8/2/2019 PHOUG RAC Presentation Final
3/39
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
4/39
#10
AVOIDINGCOMMON RACPROBLEMS
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
5/39
Simple Misconfigurations
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
6/39
Parameter Files :SPFILE & PFILE
create pfile from spfile in
one node
Edit ( Make Changes )
create spfile ..from pfile
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
7/39
#9HA & Fault
Tolerance
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
8/39
Database Challenges: Availability
Real Application Clusters
Be available all the time
> design for fault tolerance
> provide fast andreliable fault recovery
> eliminate maintenancedowntime
Support mission-critical
business operations
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
9/39Apr 14, 2012 | Company Proprietary
Components
Real Application Clusters
> Exploits clusters for very high availability> Overcomes the limitations of traditional failover solutions by
providing:
> Concurrent processing
> Load Balancing
> Fast time to recovery
Masks system failures from users/applications
Applications and users are automatically andtransparently reconnected to another system
Applications and queries continue uninterrupted
Login context maintained
DML transactions are rolled back
-
8/2/2019 PHOUG RAC Presentation Final
10/39
#8
Scalability
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
11/39
March 21 | PHL OUG, Philadelphia
Database Challenges: Scalability
-
8/2/2019 PHOUG RAC Presentation Final
12/39
How does it work
System Settings/Options
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
13/39
Apr 14, 2012 | Company Proprietary
Issues/Troubleshooting
What might go wrong
-
8/2/2019 PHOUG RAC Presentation Final
14/39
#7
RAC &Performance
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
15/39
RAC Performance:
Not only move the jobs up
Average but also how can
We move the entire
Average Up!!!
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
16/39
Best Practices
Resize undo for Optimal Performance
Tune LGWR writes
Tune SQL statement to reducelogical reads
Reduce redo size
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
17/39
#6BATCH & OLTP
ServiceArchitecture
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
18/39
Service Configuration : Operation & Behavior
Each service set up comes with
Preferred Node
Available Node
No Failback
NO Auto Start
Applications dont need to know current servicesconfiguration!
Listeners know services status Listeners control which instance gets a new connection
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
19/39
#5
Concurrency
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
20/39
Apr 14, 2012 | Company ProprietaryApr 14, 2012 | Company Proprietary
Issue : Concurrency
Data archival (and deletion).
Performance of full tablescan.
To improve application
concurrency.
If the rows are inserted withsequence based primary orunique keys, then indexes on that
primary or unique key can be amajor bottleneck.
-
8/2/2019 PHOUG RAC Presentation Final
21/39
#4
Bugs
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
22/39
How does it work in Other Environments?
Issue is Fixed in NextVersion
Work around Does not
work Anymore ASM Crashed..
Alert log has entries of
ORA- errors butinstance did not crash..
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
23/39
#3
PARALLEL QUERY
ON RAC
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
24/39
Feature Overview
PQ slaves inherit service name
Slaves run on all nodes by default, ignoringPREFERRED and AVAILABLE nodes for service
Execution controlled through INSTANCE_GROUPS andPARALLEL_INSTANCE_GROUPS
In 11gR2 :
Slaves run only on nodes where service is active
PARALLEL_INSTANCE_GROUPS overrides defaultbehavior
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
25/39
Data Warehouse Example
March 21 | PHL OUG, Philadelphia
A common requirement of
Oracle data warehousesare rollup and aggregationprocesses whereby super-large tables are read, end-to-end, computingsummary and average
values. Some datawarehouse queries alsohave ORDER BY or GROUPBY clauses, requiringOracle to retrieve and sorta very-large result set.
-
8/2/2019 PHOUG RAC Presentation Final
26/39
#2RAC Failover
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
27/39
March 21 | PHL OUG, Philadelphia
State 1 :
Server 1
Instance 1
Server 2
Instance 2
Server 3
Instance 3
Failover Scenarios
-
8/2/2019 PHOUG RAC Presentation Final
28/39
Server 1
Instance 1
Server 2
Instance 2
Server 3
Instance 3
OLTP Users OLTP Users
No headroom within cluster
to fail over any users in the
event of an unplanned outage
Failover
Concurrent
Manager-Batchh
r-Batch
-
8/2/2019 PHOUG RAC Presentation Final
29/39
Failover
Uncontrolled shutdown of Instance 3
Server 1
Instance 1Server 2
Instance 2
Server 3
Instance 3
Concurrent
Manager -BatchOLTP Users OLTP Users
F il
-
8/2/2019 PHOUG RAC Presentation Final
30/39
Failover
Surviving Instance (1,2) performs instance recovery
Server 1
Instance 1Server 2
Instance 2
Server 3
Instance 3
Concurrent
Manager-BatchOLTP Users OLTP Users
-
8/2/2019 PHOUG RAC Presentation Final
31/39
Transparent Application Failover (TAF)
Recovery with Hot Failover
Shared Memory/Global Area
sharedshared
SQLSQLloglog
bufferbuffer
Shared Memory/Global Area
sharedshared
SQLSQLloglog
bufferbuffer
Shared Memory/Global Area
sharedshared
SQLSQLloglog
bufferbuffer
Shared Disk DatabaseShared Disk Database
-
8/2/2019 PHOUG RAC Presentation Final
32/39
Root Causes of Node Eviction
Network heartbeat lost Voting disk problems
cssd is not healthy
oprocd
Hang check timer
cssd and oclsomon race to suicide
-
8/2/2019 PHOUG RAC Presentation Final
33/39
Why a node is evicted?
Split brain condition I/o Fencing
CRS keeps the lowest number node up
Node eviction detection
-
8/2/2019 PHOUG RAC Presentation Final
34/39
#1
RAC &
EBS
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
35/39
Apr 14, 2012 | Company Proprietary
Concurrent Manager
Concurrent managers aggressively accessesfnd_concurrent_requests table, joining few other tablessuch as fnd_concurrent_programs, fnd_oracle_userid etc.
In a busy environment, fnd_concurrent_table is the topsegment contributing to global cache traffic.
Optimal values for sleep and cache_size is veryimportant.
SGA:
Bigger the SGA Better
-
8/2/2019 PHOUG RAC Presentation Final
36/39
Bonus
March 21 | PHL OUG, Philadelphia
RAC OCR/VotingDisk Recovery
-
8/2/2019 PHOUG RAC Presentation Final
37/39
OCR / Voting Disk Got Deleted
If the OCR has been deleted, then check if the OCR
mirror is OK and vice versa. It may be prudent to usethe OCR mirror to create the OCR.
March 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
38/39
Q&Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R SAMarch 21 | PHL OUG, Philadelphia
-
8/2/2019 PHOUG RAC Presentation Final
39/39
Whats next?
Leave me your BusinessCards
Email me at:[email protected]
Share your feedback at:[email protected]
Thank you.March 21 | PHL OUG Philadelphia
mailto:[email protected]:[email protected]