best practices for conflict detection and resolution in active-active replication environments

25
An Orale White Paper Updated Deember 2012 Best Pactices fo Conflict Detection and Resolution in Active-Active Replication Envionments Using Oacle GoldenGate

Upload: others

Post on 12-Sep-2021

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

An Ora le White Paper

Updated De ember 2012

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Contents

Executive Ove view 1

Int oduction 1

O acle GoldenGate fo Active-Active Replication 3

Key Requi ements fo Active-Active Replication Configu ations 5

Real-Time Low-Impact Data Movement 5

Conflict Detection and Resolution 5

Hete ogeneous Envi onment Suppo t 6

Minimizing Conflicts 6

Application Seg egation 6

P ima y Key Gene ation 7

Allowable Conflicts 7

Conflict Detection 7

Unde standing Conflicts and Complex Resolutions 8

Simple Conflict Resolution Methodologies 8

Time Stamp 8

Trusted Sour e 11

Quantitative Confli t Resolution 14

Conflict Notification and T acking 17

O acle GoldenGate Data Definition Language Replication 21

Conclusion 22

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Executive Ove view

One of the most effective ways to enable inc eased availability and pe fo mance fo

database inf ast uctu es is to establish an active-active eplication envi onment which

dist ibutes database t ansactions ac oss multiple databases Beyond disaste ecove y

and fault tole ance active-active database eplication configu ations facilitate continuous

ope ations offe additional aw computing capacity and p ovide the flexibility to optimize

wo kload management Howeve implementing an active-active eplication solution is not

t ivial The key to success lies in eal-time data movement conflict detection and

esolution and suppo t fo hete ogeneous envi onments Of the th ee conflict detection

and esolution int oduces the most complexity This white pape p ovides best p actices

fo conflict detection and esolution and highlights how O acle GoldenGate 11g Release

2 add esses these challenges

Int oduction

A key objective fo any IT o ganization is to c eate softwa e applications and a database

inf ast uctu e that can scale to meet g owing and changing business needs With

business p ocesses inc easingly mig ating to digital t ansactions the e is a g owing

o ganizational eliance and dependence on the IT g ouprsquos ability to handle la ge volumes

of data and use s with less system downtime Active-active configu ations p ovide

significant pe fo mance and scalability benefits delive exceptional high-availability and

enable continuous ope ations fo not only unplanned inte uptions but also planned

outages such as mig ations upg ades and systems maintenance

In most cases active-active eplication configu ations a e conside ed to be pa t of a

continuous availabilitymdashnot a disaste ecove ymdashplan At the high end of t aditional

disaste ecove y plans the e a e solutions that offe an active-passive configu ation

whe e the active system assumes all the wo kload but when it fails the passive system

1

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

becomes active and assumes the full wo kload Unde no mal ope ating conditions the

seconda y (passive) system doesnrsquot cont ibute to handling the data p ocessing load it is

twice the investment to p ovide the same amount of p ocessing powe as a single

system By compa ison an active-active eplication configu ation not only facilitates ve y

high levels of ecove y point and ecove y time objectives but it also etu ns value on the

investment by adding capacity flexibility and highe pe fo mance to the ope ational data

inf ast uctu e

Implementing an effective active-active eplication configu ation equi es a tho ough

conside ation of technologies available fo enabling the data movement and sha ing

between the database instances Befo e moving fo wa d an o ganization must

unde stand the diffe ent use cases fo active-active eplication configu ations and the

challenges and benefits of each configu ation They must also unde stand the diffe ent

methods fo detecting data conflicts that occu and how to effectively esolve those

conflicts

2

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

O acle GoldenGate fo Active-Active Replication

To effec ively iden ify and respond o da a conflic s organiza ions need con rol over he

movemen of da a across he en erprise When offering ac ive-ac ive da abase replica ion

configura ions Oracle GoldenGa e 11gR2 delivers he infras ruc ure necessary o s reamline da a

movemen o ensure seamless opera ions

Oracle GoldenGa e provides real- ime logical da a replica ion capabili ies o move da e across

he erogeneous IT environmen s wi h subsecond speed The applica ion pla form consis s of

decoupled modules ha can be combined across sys ems o provide maximum flexibili y

modulari y and performance I is an asynchronous solu ion wi h synchronouslike behavior

This archi ec ure facili a es he movemen and managemen of ransac ional da a in four simple

ye powerful s eps

bull Captur Oracle GoldenGa ersquos change da a cap ure echnology iden ifies and replica es da a

changes from da abase log files in real ime using a nonin rusive high-performance low-

overhead approach Oracle GoldenGa e 11gR2 can cap ure da a from any number of

da abases including Oracle DB2 for LinuxUnixWindows DB2 for i-Series DB2 for zOS

as well as hose running on HP NonS opEnscribe SQLMP SQLMX and Sybase All da a

changes are cap ured hrough direc access o na ive da abase ransac ion logsmdashredo logs

where applicablemdash o minimize any impac on sys em performance

bull Rout Once cap ured changed da a ransac ions are placed in queue files (called Trail Files)

and can be delivered o any da a arge including message queues There are no geographic

dis ance cons rain s or impac s Oracle GoldenGa e uses a varie y of ranspor pro ocols as

well as compression and encryp ion echniques prior o rou ing changed da a

bull Enhanc To op imize performance and da a managemen capabili ies a any poin prior o

delivering changed da a from he hos o he arge sys em Oracle GoldenGa e can execu e a

number of buil -in func ions such as fil ering and ransforma ion

bull Apply Oracle GoldenGa e can apply changed da a o mul iple arge s wi h subsecond la ency

o ensure ransac ion in egri y wi h fea ures for conflic de ec ion and resolu ion

Key echnical fea ures ha are in rinsic o Oracle GoldenGa ersquos suppor for ac ive-ac ive

replica ion configura ions include he following

bull Fl xibl topology support and bidir ctional configurations Using a decoupled modular

design Oracle GoldenGa e can suppor a wide varie y of replica ion opologies including one-

o-one one- o-many many- o-one and many- o-manymdashfor bo h unidirec ional and

bidirec ional configura ions For addi ional scalabili y cascading opologies can be crea ed o

elimina e any po en ial bo lenecks By s aging specific se s of da abase changes on he source

3

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 2: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Contents

Executive Ove view 1

Int oduction 1

O acle GoldenGate fo Active-Active Replication 3

Key Requi ements fo Active-Active Replication Configu ations 5

Real-Time Low-Impact Data Movement 5

Conflict Detection and Resolution 5

Hete ogeneous Envi onment Suppo t 6

Minimizing Conflicts 6

Application Seg egation 6

P ima y Key Gene ation 7

Allowable Conflicts 7

Conflict Detection 7

Unde standing Conflicts and Complex Resolutions 8

Simple Conflict Resolution Methodologies 8

Time Stamp 8

Trusted Sour e 11

Quantitative Confli t Resolution 14

Conflict Notification and T acking 17

O acle GoldenGate Data Definition Language Replication 21

Conclusion 22

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Executive Ove view

One of the most effective ways to enable inc eased availability and pe fo mance fo

database inf ast uctu es is to establish an active-active eplication envi onment which

dist ibutes database t ansactions ac oss multiple databases Beyond disaste ecove y

and fault tole ance active-active database eplication configu ations facilitate continuous

ope ations offe additional aw computing capacity and p ovide the flexibility to optimize

wo kload management Howeve implementing an active-active eplication solution is not

t ivial The key to success lies in eal-time data movement conflict detection and

esolution and suppo t fo hete ogeneous envi onments Of the th ee conflict detection

and esolution int oduces the most complexity This white pape p ovides best p actices

fo conflict detection and esolution and highlights how O acle GoldenGate 11g Release

2 add esses these challenges

Int oduction

A key objective fo any IT o ganization is to c eate softwa e applications and a database

inf ast uctu e that can scale to meet g owing and changing business needs With

business p ocesses inc easingly mig ating to digital t ansactions the e is a g owing

o ganizational eliance and dependence on the IT g ouprsquos ability to handle la ge volumes

of data and use s with less system downtime Active-active configu ations p ovide

significant pe fo mance and scalability benefits delive exceptional high-availability and

enable continuous ope ations fo not only unplanned inte uptions but also planned

outages such as mig ations upg ades and systems maintenance

In most cases active-active eplication configu ations a e conside ed to be pa t of a

continuous availabilitymdashnot a disaste ecove ymdashplan At the high end of t aditional

disaste ecove y plans the e a e solutions that offe an active-passive configu ation

whe e the active system assumes all the wo kload but when it fails the passive system

1

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

becomes active and assumes the full wo kload Unde no mal ope ating conditions the

seconda y (passive) system doesnrsquot cont ibute to handling the data p ocessing load it is

twice the investment to p ovide the same amount of p ocessing powe as a single

system By compa ison an active-active eplication configu ation not only facilitates ve y

high levels of ecove y point and ecove y time objectives but it also etu ns value on the

investment by adding capacity flexibility and highe pe fo mance to the ope ational data

inf ast uctu e

Implementing an effective active-active eplication configu ation equi es a tho ough

conside ation of technologies available fo enabling the data movement and sha ing

between the database instances Befo e moving fo wa d an o ganization must

unde stand the diffe ent use cases fo active-active eplication configu ations and the

challenges and benefits of each configu ation They must also unde stand the diffe ent

methods fo detecting data conflicts that occu and how to effectively esolve those

conflicts

2

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

O acle GoldenGate fo Active-Active Replication

To effec ively iden ify and respond o da a conflic s organiza ions need con rol over he

movemen of da a across he en erprise When offering ac ive-ac ive da abase replica ion

configura ions Oracle GoldenGa e 11gR2 delivers he infras ruc ure necessary o s reamline da a

movemen o ensure seamless opera ions

Oracle GoldenGa e provides real- ime logical da a replica ion capabili ies o move da e across

he erogeneous IT environmen s wi h subsecond speed The applica ion pla form consis s of

decoupled modules ha can be combined across sys ems o provide maximum flexibili y

modulari y and performance I is an asynchronous solu ion wi h synchronouslike behavior

This archi ec ure facili a es he movemen and managemen of ransac ional da a in four simple

ye powerful s eps

bull Captur Oracle GoldenGa ersquos change da a cap ure echnology iden ifies and replica es da a

changes from da abase log files in real ime using a nonin rusive high-performance low-

overhead approach Oracle GoldenGa e 11gR2 can cap ure da a from any number of

da abases including Oracle DB2 for LinuxUnixWindows DB2 for i-Series DB2 for zOS

as well as hose running on HP NonS opEnscribe SQLMP SQLMX and Sybase All da a

changes are cap ured hrough direc access o na ive da abase ransac ion logsmdashredo logs

where applicablemdash o minimize any impac on sys em performance

bull Rout Once cap ured changed da a ransac ions are placed in queue files (called Trail Files)

and can be delivered o any da a arge including message queues There are no geographic

dis ance cons rain s or impac s Oracle GoldenGa e uses a varie y of ranspor pro ocols as

well as compression and encryp ion echniques prior o rou ing changed da a

bull Enhanc To op imize performance and da a managemen capabili ies a any poin prior o

delivering changed da a from he hos o he arge sys em Oracle GoldenGa e can execu e a

number of buil -in func ions such as fil ering and ransforma ion

bull Apply Oracle GoldenGa e can apply changed da a o mul iple arge s wi h subsecond la ency

o ensure ransac ion in egri y wi h fea ures for conflic de ec ion and resolu ion

Key echnical fea ures ha are in rinsic o Oracle GoldenGa ersquos suppor for ac ive-ac ive

replica ion configura ions include he following

bull Fl xibl topology support and bidir ctional configurations Using a decoupled modular

design Oracle GoldenGa e can suppor a wide varie y of replica ion opologies including one-

o-one one- o-many many- o-one and many- o-manymdashfor bo h unidirec ional and

bidirec ional configura ions For addi ional scalabili y cascading opologies can be crea ed o

elimina e any po en ial bo lenecks By s aging specific se s of da abase changes on he source

3

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 3: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Executive Ove view

One of the most effective ways to enable inc eased availability and pe fo mance fo

database inf ast uctu es is to establish an active-active eplication envi onment which

dist ibutes database t ansactions ac oss multiple databases Beyond disaste ecove y

and fault tole ance active-active database eplication configu ations facilitate continuous

ope ations offe additional aw computing capacity and p ovide the flexibility to optimize

wo kload management Howeve implementing an active-active eplication solution is not

t ivial The key to success lies in eal-time data movement conflict detection and

esolution and suppo t fo hete ogeneous envi onments Of the th ee conflict detection

and esolution int oduces the most complexity This white pape p ovides best p actices

fo conflict detection and esolution and highlights how O acle GoldenGate 11g Release

2 add esses these challenges

Int oduction

A key objective fo any IT o ganization is to c eate softwa e applications and a database

inf ast uctu e that can scale to meet g owing and changing business needs With

business p ocesses inc easingly mig ating to digital t ansactions the e is a g owing

o ganizational eliance and dependence on the IT g ouprsquos ability to handle la ge volumes

of data and use s with less system downtime Active-active configu ations p ovide

significant pe fo mance and scalability benefits delive exceptional high-availability and

enable continuous ope ations fo not only unplanned inte uptions but also planned

outages such as mig ations upg ades and systems maintenance

In most cases active-active eplication configu ations a e conside ed to be pa t of a

continuous availabilitymdashnot a disaste ecove ymdashplan At the high end of t aditional

disaste ecove y plans the e a e solutions that offe an active-passive configu ation

whe e the active system assumes all the wo kload but when it fails the passive system

1

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

becomes active and assumes the full wo kload Unde no mal ope ating conditions the

seconda y (passive) system doesnrsquot cont ibute to handling the data p ocessing load it is

twice the investment to p ovide the same amount of p ocessing powe as a single

system By compa ison an active-active eplication configu ation not only facilitates ve y

high levels of ecove y point and ecove y time objectives but it also etu ns value on the

investment by adding capacity flexibility and highe pe fo mance to the ope ational data

inf ast uctu e

Implementing an effective active-active eplication configu ation equi es a tho ough

conside ation of technologies available fo enabling the data movement and sha ing

between the database instances Befo e moving fo wa d an o ganization must

unde stand the diffe ent use cases fo active-active eplication configu ations and the

challenges and benefits of each configu ation They must also unde stand the diffe ent

methods fo detecting data conflicts that occu and how to effectively esolve those

conflicts

2

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

O acle GoldenGate fo Active-Active Replication

To effec ively iden ify and respond o da a conflic s organiza ions need con rol over he

movemen of da a across he en erprise When offering ac ive-ac ive da abase replica ion

configura ions Oracle GoldenGa e 11gR2 delivers he infras ruc ure necessary o s reamline da a

movemen o ensure seamless opera ions

Oracle GoldenGa e provides real- ime logical da a replica ion capabili ies o move da e across

he erogeneous IT environmen s wi h subsecond speed The applica ion pla form consis s of

decoupled modules ha can be combined across sys ems o provide maximum flexibili y

modulari y and performance I is an asynchronous solu ion wi h synchronouslike behavior

This archi ec ure facili a es he movemen and managemen of ransac ional da a in four simple

ye powerful s eps

bull Captur Oracle GoldenGa ersquos change da a cap ure echnology iden ifies and replica es da a

changes from da abase log files in real ime using a nonin rusive high-performance low-

overhead approach Oracle GoldenGa e 11gR2 can cap ure da a from any number of

da abases including Oracle DB2 for LinuxUnixWindows DB2 for i-Series DB2 for zOS

as well as hose running on HP NonS opEnscribe SQLMP SQLMX and Sybase All da a

changes are cap ured hrough direc access o na ive da abase ransac ion logsmdashredo logs

where applicablemdash o minimize any impac on sys em performance

bull Rout Once cap ured changed da a ransac ions are placed in queue files (called Trail Files)

and can be delivered o any da a arge including message queues There are no geographic

dis ance cons rain s or impac s Oracle GoldenGa e uses a varie y of ranspor pro ocols as

well as compression and encryp ion echniques prior o rou ing changed da a

bull Enhanc To op imize performance and da a managemen capabili ies a any poin prior o

delivering changed da a from he hos o he arge sys em Oracle GoldenGa e can execu e a

number of buil -in func ions such as fil ering and ransforma ion

bull Apply Oracle GoldenGa e can apply changed da a o mul iple arge s wi h subsecond la ency

o ensure ransac ion in egri y wi h fea ures for conflic de ec ion and resolu ion

Key echnical fea ures ha are in rinsic o Oracle GoldenGa ersquos suppor for ac ive-ac ive

replica ion configura ions include he following

bull Fl xibl topology support and bidir ctional configurations Using a decoupled modular

design Oracle GoldenGa e can suppor a wide varie y of replica ion opologies including one-

o-one one- o-many many- o-one and many- o-manymdashfor bo h unidirec ional and

bidirec ional configura ions For addi ional scalabili y cascading opologies can be crea ed o

elimina e any po en ial bo lenecks By s aging specific se s of da abase changes on he source

3

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 4: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

becomes active and assumes the full wo kload Unde no mal ope ating conditions the

seconda y (passive) system doesnrsquot cont ibute to handling the data p ocessing load it is

twice the investment to p ovide the same amount of p ocessing powe as a single

system By compa ison an active-active eplication configu ation not only facilitates ve y

high levels of ecove y point and ecove y time objectives but it also etu ns value on the

investment by adding capacity flexibility and highe pe fo mance to the ope ational data

inf ast uctu e

Implementing an effective active-active eplication configu ation equi es a tho ough

conside ation of technologies available fo enabling the data movement and sha ing

between the database instances Befo e moving fo wa d an o ganization must

unde stand the diffe ent use cases fo active-active eplication configu ations and the

challenges and benefits of each configu ation They must also unde stand the diffe ent

methods fo detecting data conflicts that occu and how to effectively esolve those

conflicts

2

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

O acle GoldenGate fo Active-Active Replication

To effec ively iden ify and respond o da a conflic s organiza ions need con rol over he

movemen of da a across he en erprise When offering ac ive-ac ive da abase replica ion

configura ions Oracle GoldenGa e 11gR2 delivers he infras ruc ure necessary o s reamline da a

movemen o ensure seamless opera ions

Oracle GoldenGa e provides real- ime logical da a replica ion capabili ies o move da e across

he erogeneous IT environmen s wi h subsecond speed The applica ion pla form consis s of

decoupled modules ha can be combined across sys ems o provide maximum flexibili y

modulari y and performance I is an asynchronous solu ion wi h synchronouslike behavior

This archi ec ure facili a es he movemen and managemen of ransac ional da a in four simple

ye powerful s eps

bull Captur Oracle GoldenGa ersquos change da a cap ure echnology iden ifies and replica es da a

changes from da abase log files in real ime using a nonin rusive high-performance low-

overhead approach Oracle GoldenGa e 11gR2 can cap ure da a from any number of

da abases including Oracle DB2 for LinuxUnixWindows DB2 for i-Series DB2 for zOS

as well as hose running on HP NonS opEnscribe SQLMP SQLMX and Sybase All da a

changes are cap ured hrough direc access o na ive da abase ransac ion logsmdashredo logs

where applicablemdash o minimize any impac on sys em performance

bull Rout Once cap ured changed da a ransac ions are placed in queue files (called Trail Files)

and can be delivered o any da a arge including message queues There are no geographic

dis ance cons rain s or impac s Oracle GoldenGa e uses a varie y of ranspor pro ocols as

well as compression and encryp ion echniques prior o rou ing changed da a

bull Enhanc To op imize performance and da a managemen capabili ies a any poin prior o

delivering changed da a from he hos o he arge sys em Oracle GoldenGa e can execu e a

number of buil -in func ions such as fil ering and ransforma ion

bull Apply Oracle GoldenGa e can apply changed da a o mul iple arge s wi h subsecond la ency

o ensure ransac ion in egri y wi h fea ures for conflic de ec ion and resolu ion

Key echnical fea ures ha are in rinsic o Oracle GoldenGa ersquos suppor for ac ive-ac ive

replica ion configura ions include he following

bull Fl xibl topology support and bidir ctional configurations Using a decoupled modular

design Oracle GoldenGa e can suppor a wide varie y of replica ion opologies including one-

o-one one- o-many many- o-one and many- o-manymdashfor bo h unidirec ional and

bidirec ional configura ions For addi ional scalabili y cascading opologies can be crea ed o

elimina e any po en ial bo lenecks By s aging specific se s of da abase changes on he source

3

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 5: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

O acle GoldenGate fo Active-Active Replication

To effec ively iden ify and respond o da a conflic s organiza ions need con rol over he

movemen of da a across he en erprise When offering ac ive-ac ive da abase replica ion

configura ions Oracle GoldenGa e 11gR2 delivers he infras ruc ure necessary o s reamline da a

movemen o ensure seamless opera ions

Oracle GoldenGa e provides real- ime logical da a replica ion capabili ies o move da e across

he erogeneous IT environmen s wi h subsecond speed The applica ion pla form consis s of

decoupled modules ha can be combined across sys ems o provide maximum flexibili y

modulari y and performance I is an asynchronous solu ion wi h synchronouslike behavior

This archi ec ure facili a es he movemen and managemen of ransac ional da a in four simple

ye powerful s eps

bull Captur Oracle GoldenGa ersquos change da a cap ure echnology iden ifies and replica es da a

changes from da abase log files in real ime using a nonin rusive high-performance low-

overhead approach Oracle GoldenGa e 11gR2 can cap ure da a from any number of

da abases including Oracle DB2 for LinuxUnixWindows DB2 for i-Series DB2 for zOS

as well as hose running on HP NonS opEnscribe SQLMP SQLMX and Sybase All da a

changes are cap ured hrough direc access o na ive da abase ransac ion logsmdashredo logs

where applicablemdash o minimize any impac on sys em performance

bull Rout Once cap ured changed da a ransac ions are placed in queue files (called Trail Files)

and can be delivered o any da a arge including message queues There are no geographic

dis ance cons rain s or impac s Oracle GoldenGa e uses a varie y of ranspor pro ocols as

well as compression and encryp ion echniques prior o rou ing changed da a

bull Enhanc To op imize performance and da a managemen capabili ies a any poin prior o

delivering changed da a from he hos o he arge sys em Oracle GoldenGa e can execu e a

number of buil -in func ions such as fil ering and ransforma ion

bull Apply Oracle GoldenGa e can apply changed da a o mul iple arge s wi h subsecond la ency

o ensure ransac ion in egri y wi h fea ures for conflic de ec ion and resolu ion

Key echnical fea ures ha are in rinsic o Oracle GoldenGa ersquos suppor for ac ive-ac ive

replica ion configura ions include he following

bull Fl xibl topology support and bidir ctional configurations Using a decoupled modular

design Oracle GoldenGa e can suppor a wide varie y of replica ion opologies including one-

o-one one- o-many many- o-one and many- o-manymdashfor bo h unidirec ional and

bidirec ional configura ions For addi ional scalabili y cascading opologies can be crea ed o

elimina e any po en ial bo lenecks By s aging specific se s of da abase changes on he source

3

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 6: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflicct Detection and Resolution in Active-Active Replication Envi onments Using O a acle GoldenGate

or arge sys em differen d da a replica ion requiremen s can be me hrough a single e pass on

he da a source Each se of f s aged da a can con ain unique or overlapping se s of da a a

bull Conflict d t ction and r s solution When wo sys ems are processing da a ransac ions and

he ac ivi y is shared across bo h sys ems de ec ing and addressing conflic s across hem

becomes an essen ial requireremen for any ac ive-ac ive replica ion configura ion Or racle

GoldenGa e 11g Release 2 pprovides a wide varie y of conflic de ec ion and resolu io on op ions

o provide he necessary fle exibili y and adap abili y for a range of requiremen s Con nflic

de ec ion and resolu ion op p ions can be implemen ed globally objec by objec base ed on da a

values and complex fil ers a and hrough even -driven cri eria including da abase erro or

messages

bull H t rog n ity Oracle Go oldenGa e decouples he da a source and arge which en nables he

applica ion o easily facili a e he erogenei y In addi ion changed da a is s aged be w ween he

sys ems in a universal da a f forma (Trail Files) o facili a e por abili y This provides flexibili y

in he choice of hardware o opera ing sys em and da abases for sources and arge s a and can

accommoda e unplanned ou u ages as well as sys em da abase and applica ion main e enance

ac ivi ieswi hou in errup p ion Unlike archi ec ures ha implemen a igh ldquoproces ss- o-

processrdquo coupling his deco oupled archi ec ure provides each module he abili y o p perform i s

asks independen ly of o he er modules or componen s

bull Subs cond lat ncy Oracle e GoldenGa ersquos cap ure enhance rou e and delivery pro ocesses can

move housands of commi ed da a ransac ions be ween sys ems wi h subsecond sp peed

There is very minimal impacc on he source sys em and infras ruc ure hus ensuring g high

performance wi h high da a a volumes

Whe her you are using any mi ix of Oracle Da abase Sybase SQL Server DB2 or evenn HP

NonS op or Terada a Oracle GoldenGa e is an excellen solu ion for improving he

performance accessibili y and d availabili y of your da a across he en erprise

4

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 7: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Figu e 1 In an active-active eplication configu ation O acle GoldenGate delive s continuous system

availability

Key Requi ements fo Active-Active Replication Configu ations

In an ac ive-ac ive replica ion configura ionmdashalso referred o as a mas er-mas er dual-mas er

mul i-mas er or peer- o-peer configura ionmdashmul iple da abase sys ems concurren ly process

da a ransac ions Any changes ha persis on one sys em are reflec ed in he o her sys ems The

key benefi of his ype of configura ion is he abili y o balance he ransac ion workload across

mul iple sys ems Each addi ional sys em in he ac ive-ac ive replica ion configura ion increases

he overall capaci y resul ing in improved response imes and enhanced sys em performance

Ac ive-ac ive replica ion configura ions enable workload par i ioning based on mul iple

a ribu es For example differen applica ions can be rou ed o differen sys ems in he

configura ion or users in a specific region can be serviced by a local da abase server Thus

ac ive-ac ive replica ion configura ions no only offer addi ional capaci y bu hey also offer he

flexibili y o op imize workload managemen

Despi e all he availabili y and performance benefi s ha can be reaped by he business i is

cri ical o poin ou ha implemen ing an ac ive-ac ive replica ion solu ion is no rivial The key

ingredien s include

bull Real- ime bidirec ional da a movemen

bull Conflic de ec ion and resolu ion

bull He erogeneous environmen suppor

Real-Time Low-Impact Data Movement

To load balance users across mul iple da abases all users mus have access o he same da a In

prac ice his requires more han jus moving da a from one sys em o ano her The ideal solu ion

should impose minimal la ency and very low overheadwi hou in roducing in erprocess

dependencies Al hough a synchronous approach using a wo-phased commi would provide

zero la ency i would also lead o high overhead and dependencies across mul iple sys ems In

ac ive-ac ive replica ion configura ions he da a movemen has o be asynchronous ye provide

ldquosynchronouslikerdquo behavior

Conflict Detection and Resolution

In an ac ive-ac ive replica ion configura ion da a collisions are inevi able When wo resources

simul aneously upda e he same record on wo separa e sys ems he ensuing conflic mus be

de ec ed and resolved To suppor a wide varie y of business rules an effec ive ac ive-ac ive

replica ion solu ion mus facili a e differen conflic de ec ion and resolu ion mechanisms

5

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 8: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Hete ogeneous Envi onment Suppo t

Sys ems in an ac ive-ac ive replica ion configura ion migh have differen hardware se ups

opera ing sys ems service packs da abase versions and pa ch levels To ensure con inuous

opera ions during upgrades and main enance opera ions and o provide flexibili y for op imal

resource alloca ion he ac ive-ac ive replica ion solu ion mus provide he erogeneous

infras ruc ure suppor

Minimizing Conflicts

Even wi h he bes conflic resolu ion rou ines here are s ill going o be issues ha are no easy

o handle A key goal in building ac ive-ac ive replica ion environmen s is minimizing he amoun

of conflic s ha happen If you can avoid conflic s on 99 percen of he ables or even on a

single ype of opera ion (such as a dele e) hen you can save an enormous amoun of ime

implemen ing and main aining your environmen This sec ion is going o discuss several ways

ha conflic s can ac ually be avoided or reduced Any way o reduce he amoun of conflic s will

provide a be er experience for all s akeholders

Application Seg egation

Segrega ing applica ion users is one way o avoid conflic s Bo h inven ory- and quan i y-rela ed

conflic s can easily be addressed his way Each server ha is balancing he user load con ains he

primary source for a cer ain ype of produc or service

S ock rades are one ype of produc ha canno have conflic s bu can s ill benefi from an

ac ive-ac ive replica ion environmen By moving he rades on companies ha begin wi h he

le ers AminusM o one server and he le ers NminusZ o ano her you can avoid any conflic s of rading

he same s ock a he same ime Removing he conflic s in such cri ical si ua ions can really allow

his ype of configura ion o succeed bu also provide phenomenal resul s Through he use of

an applica ion server his can be made even easier by having a pool of connec ions o issue he

rades ra her han having he users log on o bo h sys ems

Ano her way o do his is wi h user names If people are frequen ly changing accoun

informa ion or even accoun balances you can move an equal number of people o each server

and avoid nearly all conflic s Because he changed da a from all servers is going o be propaga ed

o all he o her nodes in he even ha he primary node for a group of accoun s goes down

hey can be rou ed o a secondary node un il every hing is back up and running smoo hly Some

companies have gone as far as designa ing primary and secondary load dis ribu ion me hods o

6

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 9: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

ensure ha a single server is no overly burdened by an ou age elsewhere in he environmen

The more servers ha are involved he more i will help o have secondary applica ion

segrega ion and load-balancing s ra egies

P ima y Key Gene ation

I is cri ical ha able primary keys and unique indexes are unique for each da abase par icipa ing

in an ac ive-ac ive replica ion environmen The able primary keys and unique indexes for each

da abase in he ac ive-ac ive replica ion environmen mus con ain informa ion ha iden ifies

which da abase he opera ion occurred on Inser conflic s can be avoided and almos elimina ed

by implemen ing da abase specific primary keys and unique indexes and is probably he easies

me hod of reducing he number of conflic s ha can arise

For Oracle da abases ha are configured o use sequences for able primary keys simply al erna e

he primary key genera ion sequences or rou ines For a wo-server environmen have one

genera e even primary keys he o her odd For an n-server environmen have each genera e keys

s ar ing a a differen value (1 2 3 4 5hellipn) and have heir sequences incremen by he number

of servers in he environmen For a hree-server environmen server one s ar s a 1 and

incremen s by hree (1 4 7 10 13) server wo s ar s a 2 and incremen s by hree (2 5 8 11

14) and server hree s ar s a 3 and incremen s by hree (3 6 9 12 15)

However even hough i is ex remely easy o implemen his me hod migh no be available o

all applica ions

Allowable Conflicts

This goes agains much of wha has been discussed so far however here are going o be cer ain

ye rarer cases where conflic s can jus be ignored One case could be in dele ing informa ion If

an i em is going o be discon inued and he s ore manager dele es i ou of he inven ory sys em

and he da abase adminis ra or does i a he same ime i really doesnrsquo ma er if someone

commi ed heir dele e jus a spli second before he o her user A dele e is a dele e and he resul

will be a discon inued produc Conflic s ha arise in hese si ua ions can jus be ignored A firs

you migh wan o keep rack of hem jus o see how of en hey occur bu once you are

confiden ha here is no harm being done hey can usually be skipped

Conflict Detection

Even hrough applica ion segrega ion and primary key isola ion echniques can significan ly

reduce conflic s in an ac ive-ac ive replica ion configura ion here may s ill be conflic scenarios

ha canrsquo be avoided and need o be addressed hrough conflic de ec ion and resolu ion Oracle

GoldenGa e 11g Release 2 provides buil -in logic o reduce he complexi y of configuring conflic

de ec ion and resolu ion for successful ac ive-ac ive replica ion deploymen s

7

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 10: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Unde standing Conflicts and Complex Resolutions

Differen ypes of conflic s require differen resolu ions In cer ain ins ances he conflic s are

simple and he rules o resolve hem are equally s raigh forward

Take he case of an airline reserva ion sys em ha has an ac ive-ac ive replica ion configura ion

wi h one da abase loca ed in he Uni ed S a es used for online ransac ions and he o her

da abase in Europe used for over he phone ransac ions Two cus omers Joe and Kevin are

reserving heir sea s for fligh 123 from Barcelona o New York Ci y a approxima ely he same

ime Joe is making his sea reserva ion over he phone so when he opera or submi s his sea

reserva ion for 2A i is made o he European Da abase Kevin is making his sea reserva ion

online a approxima ely he same ime so when he submi s his sea reserva ion for 2A i is made

o he US Da abase In he absence of conflic de ec ion and resolu ion processes Joe and

Kevin would have bo h reserved sea 2A successfully

An ac ive-ac ive replica ion configura ion can and mus de ec hese ypes of da a collisions

During he opera ion bo h he pre-change da a and he changed da a need o be cap ured When

delivering he da a he conflic de ec ion process should ma ch he pre-change version of he

da a from he origina ing sys em wi h he pre-upda e version of he record on he arge sys em

Ma ching a primary key or unique key is no sufficien o de ec and resolve conflic s Da a

lookups ransforma ions and cus om business logic could also come in o play and he ac ive-

ac ive replica ion solu ion needs o facili a e hese varia ions

In he example he solu ion mus have he abili y o ma ch he non-key columns and ob ain he

before and af er image of he records I could also be resolved by placing addi ional unique

cons rain s on he objec s or by invoking cus om business logic

Simple Conflict Resolution Methodologies

In an ac ive-passive replica ion environmen a conflic is considered an ou -of-sync record and is

handled individually and manually Such discrepancies need o be immedia ely iden ified and

handled wi h as much au oma ion as possible I is also impor an o use he same resolu ion

procedures on all he sys ems in he ac ive-ac ive replica ion environmen so ha he same

conflic receives he same resolu ion across he board

The wo mos preferred conflic resolu ion me hodologies are ime s amp and rus ed source As

an implemen a ion prac ice i is commonplace o have a da abase procedure for each opera ion

ypemdashone for inser s one for upda es and one for dele esmdash ha can handle 80 percen of he

objec s and heir da a ransac ions

Time Stamp

Wi h he ime s amp me hodology in mos cases he record ha was modified firs ( hough in

some cases las ) always wins For his me hod o work each record mus con ain a imes amp

column ha con ains he da e and ime he record was inser ed or upda ed The easies way o

8

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 11: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

accomplish his if i is no presen in he da a is hrough a da abase rigger or by modifying he

applica ion code o place he imes amp in a able column

I is cri ical o ensure ha he clocks on all da abases are iden ical o one ano her and i rsquos

recommended ha all da abase servers are configured o main ain accura e ime hrough a ime

server using he ne work ime pro ocol (NTP) Even in environmen s where da abases span

differen ime zones all da abase clocks mus be se o he same ime zone or Coordina ed

Universal Time (UTC) mus be used o main ain accura e ime Failure o main ain accura e and

synchronized ime across he da abases in an ac ive-ac ive replica ion environmen will resul in

da a in egri y issues

To de ec a conflic in a imes ampminusbased environmen here are wo simple rules o follow

Firs a emp o apply he row making sure ha he pre-upda e imes amp from he source

sys em is equal o he curren imes amp in he arge sys em If he opera ion succeeds here is

no conflic If i fails hen he second rule is o compare he imes amp of he curren record in

he arge da abase o he af er image of he imes amp from he source da abase The row ha

has he oldes ime s amp value wins

Example 1 Rese ving an Ai line Seat Using Time Stamp Resolution

An airline reserva ion sys em uses he SEAT_RESV able o s ore fligh and sea reserva ion

informa ion for a passenger The SEAT_RESV able is pre-popula ed wi h he airlinersquos fligh and

sea informa ion while he PASSENGER column is null un il a passenger reserves a sea and he

able is upda ed For example before a passenger reserves sea 2A on fligh 123 he able would

look like below

(SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 2A 123 10-10-2012 80000

Re urning o he previous example of Joe and Kevin reserving a sea suppose ha Joe (by

phone) reserved sea 2A a 103000am while Kevin (online) reserved sea 2A one second af er

Joe did a 103001am

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo LAST_UPDATE = (timestamp lsquo2012-10-15 103000rsquo

where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo LAST_UPDATE = (timestamp lsquo2012-10-15 103001rsquo

where ID = 1

9

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 12: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Joe 2A 123 10-15-2012 103000

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT LAST_UPDATE

1 Kevin 2A 123 10-15-2012 103001

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a imes amp

Setup Befo e Image Captu e

As men ioned above he pre-upda e or before image of he imes amp column needs o be

cap ured from he source sys em so ha i can be compared agains he arge imes amp column

o de ermine if a conflic has occurred In order for he imes amp resolu ion me hod o be

successful he applica ion mus always value he imes amp column when upda ing he able In

his example i is required ha he airline reserva ion sys em always values he imes amp column

LAST_UPDATE when upda ing he SEAT_RESV able If he airline reserva ion sys em canno

be configured o always value he imes amp column LAST_UPDATE hen he imes amp

resolu ion me hod canno be used

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for he key column ID and he LAST_UPDATE column

when an upda e occurs on he SEAT_RESV able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE KEYINCLUDING(last_update))

Configu e O acle GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o he Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

10

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 13: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT USEMIN (last_update)))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he LAST_UPDATE column wi h he curren LAST_UPDATE column value when an

upda e occurs o he SEAT_RESV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key ID for uniqueness when comparing rows If he

LAST_UPDATE column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he LAST_UPDATE column values donrsquo ma ch and in his example

he LAST_UPDATE values are differen a conflic has occurred and he

RESOLVECONFLICT parame er would fire The RESOLVECONFLICT parame er in his

example ins ruc s he Delivery process o upda e he PASSENGER and LAST_UPDATE

columns when he LAST_UPDATE value is older han he curren value hrough USEMIN In

his example Kevinrsquos sea reserva ion would be overwri en by Joersquos sea reserva ion in he US

Da abase as Joersquos LAST_UPDATE value is 1 second older han Kevinrsquos Kevinrsquos sea

reserva ion ransac ion would be ignored in he European Da abase as Joersquos LAST_UPDATE

value is 1 second older han Kevinrsquos The da abases remain in sync as sea 2A is now reserved by

Joe in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (also called Replica ) process This is a quick me hod

o de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

T usted Sou ce

Ano her common conflic resolu ion approach is called rus ed source In hese resolu ion

rou ines here is a single rus ed source ha is considered o always con ain he correc da a This

11

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 14: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

could be as simple as a server loca ion or as complex as a da abase user hierarchy The

implemen a ion of his approach is s raigh forwardmdash he decided rus ed source always wins

Example 2 Rese ving an Ai line Seat Using T usted Sou ce Resolution

The airline reserva ion sys em example will be used again bu in he example he

LAST_UPDATE column does no exis on he SEAT_RESV able To handle conflic s he

airline es ablished he rule ha he US Da abase used for online reserva ions will be he rus ed

source This means no ma er wha opera ion or change caused he conflic on he SEAT_RESV

able he US Da abase ransac ions will always win In his example Joe and Kevin are making

heir sea reserva ions for fligh 123 from Barcelona Spain o New York Ci y Joe is making his

sea reserva ion over he phone while Kevin is making his online The opera or assis ing Joe

comple es his sea reserva ion for 2A over he phone and clicks submi Simul aneously Kevin

has also comple ed selec ing sea 2A online and clicks submi

Update to European Database (SEAT_RESV table) Over the Phone

Update EUROSEAT_RESV set PASSENGER = lsquoJoersquo where ID = 1

Update to US Database (SEAT_RESV table) Online

Update USSEAT_RESV set PASSENGER = lsquoKevinrsquo where ID = 1

Before Oracle GoldenGa e replica es he sea reserva ion upda e ransac ions o he European

and US Da abases he sea reserva ion ables would look like he following

European Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Joe 2A 123

US Database (SEAT_RESV table)

ID PASSENGER SEAT FLIGHT

1 Kevin 2A 123

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a rus ed source

Setup Befo e Image Captu e

When using he rus ed source me hod pre-upda e or before image values for all columns in he

able mus be cap ured from he source sys em so ha hey can be compared agains all columns

in he arge da abase o de ermine if a conflic has occurred To accomplish his all columns

12

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 15: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

mus be added o he SEAT_RESV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure all column before image values

Issue he following command o add he SEAT_RESV able supplemen al log group using all he

able columns ID PASSENGER SEAT and FLIGHT The SEAT_RESV supplemen al log

group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA SEAT_RESV COLS (PASSENGER SEAT FLIGHT)

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for all columns when an upda e occurs on he SEAT_RESV

able

European Database Capture

TABLE euroseat_resv GETBEFORECOLS(ON UPDATE ALL)

US Database Capture

TABLE usseat_resv GETBEFORECOLS(ON UPDATE ALL)

Configu e O acle GoldenGate Delive y

Because he US Da abase is he rus ed source in his configura ion he GoldenGa e Delivery

process applying ransac ions o he European Da abase would always overwri e he values in he

SEAT_RESV able when a conflic occurs The GoldenGa e Delivery process applying

ransac ions o he SEAT_RESV able in he US Da abase would always ignore he ransac ions

ha came from he European Da abase when a conflic occurs

By defaul he GoldenGa e Delivery process will wri e o he arge da abase unless he map

s a emen is configured o explici ly ignore i Because of his he COMPARECOLS and

RESOLVECONFLICT parame ers would only need o be added o he US Da abase Delivery

parame er file There is no reason o check for conflic s of ransac ions coming from he US

Da abase because he US Da abase is he rus ed source and always wins The European

Da abase parame er file would jus con ain he s andard map s a emen while he US Da abase

parame er file map s a emen would be configured o ignore conflic s

European Database (Delivery - Overwrite)

MAP usseat_resv TARGET euroseat_resv

US Database (Delivery - Ignore)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE ALL) RESOLVECONFLICT (UPDATEROWEXISTS (DEFAULT IGNORE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of columns when an upda e occurs o he SEAT_RESV able The ALL parame er

ins ruc s he Delivery process o compare all able columns If all column values ma ch he

upda e ransac ion will comple e normally as no conflic has occurred If any of he column

13

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 16: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

values donrsquo ma ch and in his example he PASSENGER values are differen a conflic has

occurred and he RESOLVECONFLICT parame er would fire The RESOLVECONFLICT

parame er ins ruc s he Delivery process o IGNORE he ransac ion if i came from he

European Da abase In his example Kevinrsquos sea reserva ion would overwri e Joersquos sea

reserva ion in he European Da abase because Kevinrsquos ransac ion occurred in he rus ed source

US Da abase Joersquos sea reserva ion ransac ion would be ignored by he GoldenGa e Delivery

process applying o he US Da abase because he ransac ion came from he European

Da abase The da abases remain in sync as sea 2A is now reserved by Kevin in bo h da abases

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery (Replica ) process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Quantitative Conflict Resolution

The me hods previously discussed are fine for mos of he ables normally involved in da abase

ransac ions However here are imes when more-complex rou ines are needed o handle he

differen issues ha can occur A number of differen problems were alluded o in he firs wo

examples where he conflic s were rela ively simple This sec ion addresses he more-complex

quan i a ive conflic resolu ion me hod

Quan i a ive values include angible values such as inven ory accoun balances and sales

informa ionmdashany hing ha has i s value incremen ed or decremen ed by a se amoun

Example 3 Flight Seats Available Using Quantitative Resolution

Prior o Joe and Kevin reserving heir sea s for fligh 123 hey had o purchase heir ini ial

icke s For his example fligh 123 has 10 sea s available and Joe is purchasing 3 icke s for his

family over he phone which u ilizes he European Da abase while simul aneously Kevin is

purchasing 4 icke s for his family online which u ilizes he US Da abase

Update to European Database (FLIGHT_INV table) Over the Phone

Update EUROFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 3 where FLIGHT = 123

14

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 17: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Update to US Database (FLIGHT_INV table) Online

Update USFLIGHT_INV set SEATS_AVAIL = seats_avail ndash 4 where FLIGHT = 123

Before Oracle GoldenGa e replica es he fligh inven ory upda e ransac ions o he European

and US Da abases he fligh inven ory ables would look like he following

European Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 7

US Database (FLIGHT_INV table)

FLIGHT SEATS_AVAIL

123 6

When Joersquos ransac ion is replica ed o he US Da abase i will fail Why Because he before

image value of SEATS_AVAIL is expec ed o be 10 bu ins ead i is 6 from Kevinrsquos purchase of

4 icke s Kevinrsquos ransac ion will also fail for he same reason when i is replica ed o he

European Da abase because i rsquos also expec ing he before image value of SEATS_AVAIL o be

10 There are enough sea s available for bo h Joe and Kevin o comple e heir purchase so

nei her imes amp nor rus ed source is a sui able solu ion In his case SEATS_AVAIL needs o

be handled using a quan i a ive resolu ion The resolu ion needs o look a he ac ual change in

SEATS_AVAIL a he source sys em and apply ha o he arge sys em ra her han using he

ac ual numbers

How would Oracle GoldenGa e 11g Release 2 be configured o de ec his conflic and resolve i

using a quan i a ive resolu ion

Setup Befo e Image Captu e

The preupda e or before image of he SEATS_AVAIL column value needs o be cap ured from

he source sys em so ha i can be compared agains he arge SEATS_AVAIL column value o

de ermine if a conflic has occurred To accomplish his he SEATS_AVAIL column needs o

be added o he FLIGHT_INV able supplemen al log group and he Oracle GoldenGa e

Cap ure processes need o be configured o always cap ure he SEATS_AVAIL columnrsquos before

image values

Issue he following command o add he FLIGHT_INV able supplemen al log group using he

primary key column FLIGHT and he inven ory column SEATS_AVAIL The FLIGHT_INV

able supplemen al log group needs o be crea ed in bo h he European and US Da abases

GGSCIgt ADD TRANDATA FLIGHT_INV COLS (SEATS_AVAIL)

15

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 18: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The following line would be added o he Oracle GoldenGa e Cap ure parame er file for he

European and US Da abases The parame er ins ruc s he Cap ure process o cap ure he before

values in addi ion o he af er values for key column ID and he SEATS_AVAIL column when

an upda e occurs on he FLIGHT_INV able

European Database Capture

TABLE euroflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

US Database Capture

TABLE usflight_inv GETBEFORECOLS(ON UPDATE KEYINCLUDING (seats_avail))

Configu e GoldenGate Delive y

Now ha he Oracle GoldenGa e Cap ure processes have been configured o cap ure he before

image da a required o de ec a conflic he Oracle GoldenGa e Delivery processes need o be

configured o de ec he conflic and resolve i

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases

European Database (Delivery)

MAP usflight_inv TARGET euroflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

US Database (Delivery)

MAP euroflight_inv TARGET usflight_inv COMPARECOLS (ON UPDATE KEYINCLUDING (seats_avail)) RESOLVECONFLICT (UPDATEROWEXISTS (delta_resolution_method USEDELTA COLS (seats_avail)) (DEFAULT OVERWRITE))

The parame er COMPARECOLS ins ruc s he Delivery process o compare he before image

values of he SEATS_AVAIL column wi h he curren SEATS_AVAIL column value when an

upda e occurs o he FLIGHT_INV able The KEYINCLUDING parame er ins ruc s he

Delivery process o use he primary key FLIGHT for uniqueness when comparing rows If he

SEATS_AVAIL column values ma ch he upda e ransac ion will comple e normally as no

conflic has occurred If he SEATS_AVAIL column values donrsquo ma ch and in his example he

SEATS_AVAIL values are differen a conflic has occurred and he RESOLVECONFLICT

parame er would fire The RESOLVECONFLICT parame er in his example ins ruc s he

Delivery process o upda e he SEATS_AVAIL column by sub rac ing he before image value of

he column from he af er image value of he column and hen adding ha value o he curren

value of he column

16

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 19: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Database After Value Before Value Current Value Final Value

European 6 - 10 + 7 = 3

US 7 - 10 + 6 = 3

The COLS parame er ins ruc s he Delivery process which columns o upda e on resolu ion and

he USEDELTA parame er ins ruc s he Delivery process o calcula e he final value In his

example Joe ordered 3 icke s so he curren value of he SEATS_AVAIL column in he

European Da abase would be 7 while Kevin ordered 4 icke s so he curren value of he

SEATS_AVAIL column would be 6 When Joersquos order is applied o he US Da abase he

original inven ory of 10 icke s would be sub rac ed from his af er value of 7 and hen he curren

US Da abase value of 6 would be added for a final value of 3 When Kevinrsquos order is applied o

he European Da abase he original inven ory of 10 icke s would be sub rac ed from his af er

value of 6 and hen he curren European Da abase value of 7 would be added for a final value of

3 The quan i a ive resolu ion rou ines produced he same resul of 3 for SEATS_AVAIL in bo h

da abases and he rows are in sync

Confi m Conflict Detection Resolution Succeeded

Issue he following command in GGSCI on he European and US Da abase servers o display

he curren CDR s a is ics for he Delivery (Replica ) process This is a quick me hod o

de ermine if conflic s are being resolved successfully by he Delivery process

GGSCIgt STATS REPLICAT ltgroupgt REPORTCDR

Total inserts 000 Total updates 100 Total deletes 000 Total dis ards 000 Total operations 100 Total CDR onfli ts 100 CDR resolutions su eeded 100 CDR UPDATEROWEXISTS onfli ts 100

Conflict Notification and T acking

When configuring an ac ive-ac ive replica ion environmen for conflic de ec ion and resolu ion

i is a bes prac ice o rack conflic s hrough an excep ions able The excep ions able con ains

he changes ha were made by he au oma ed resolu ion rou ines Logging hese changes makes

i easy o find ou wha conflic s occurred how hey were handled and wha resolu ion was

aken Excep ion ables assis in roubleshoo ing audi ing and no ifica ion purposes in complex

environmen s

17

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 20: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Example 4 Configu e an Exceptions Table fo Auditing and Notification

In he sea reserva ion example imes amp was used o resolve he conflic of Joe and Kevin

making he sea reserva ion a approxima ely he same ime The conflic was handled correc ly in

he da abase bu Joe and Kevin bo h believe hey have reserved sea 2A Joe made his sea

reserva ion for 2A one second before Kevin and overwro e Kevinrsquos sea reserva ion in he US

Da abase As far as Kevin knows he has reserved sea 2A How would he airline know o no ify

Kevin ha his sea reserva ion was no successful and ha he needed o reserve a new sea The

airline would use he excep ions able o iden ify Kevinrsquos rejec ed sea reserva ions and no ify

him o reserve a new sea

How would Oracle GoldenGa e 11g Release 2 be configured o wri e conflic s o an excep ions

able for sea reserva ions and no ify Kevin o reserve a new sea

C eate Exceptions Table

An excep ions able is recommended o be crea ed in bo h he European and US Da abases and

a bes prac ice is o wri e as much informa ion as possible o he excep ions able abou he

conflic The excep ions able should include all columns in he paren able as well as addi ional

iden ifying informa ion abou he ransac ion Fur her i is recommended o crea e he

excep ions able wi hou any primary key or unique indexes o avoid unique cons rain viola ions

For example he SEAT_RESV excep ions able would be named SEAT_RESV_EXCEPTIONS

and migh look like he following

Name Null Type

TOTAL_CDR_CONFLICTS NUMBER

CDR_FAILED NUMBER

CDR_SUCCESSFUL NUMBER

RESOLUTION_DATE DATE

OPTYPE VARCHAR2(10)

DBERRNUM NUMBER

DBERRMSG VARCHAR2(25)

TABLE_NAME VARCHAR2(20)

PASSENGER_AFTER VARCHAR2(10)

LAST_UPDT_AFTER TIMESTAMP(6)

PASSENGER_BEFORE VARCHAR2(10)

LAST_UPDT_BEFORE TIMESTAMP(6)

PASSENGER_CURRENT VARCHAR2(10)

LAST_UPDT_CURRENT TIMESTAMP(6)

ID NUMBER

SEAT VARCHAR2(3)

FLIGHT NUMBER

18

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 21: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Configu e O acle GoldenGate Delive y

Once he excep ions able has been crea ed in he European and US Da abases he Oracle

GoldenGa e Delivery processes need o be configured o wri e conflic and resolu ion excep ions

o he SEAT_RESV_EXCEPTIONS able

The following lines would be added o Oracle GoldenGa e Delivery parame er file for he

European and US Da abases The map s a emen for SEAT_RESV_EXCEPTIONS needs o

appear immedia ely following he SEAT_RESV map s a emen in he parame er file

European Database (Delivery)

MAP usseat_resv TARGET euroseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP usseat_resv TARGET euroseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from euroseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquoeuroseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

US Database (Delivery)

MAP euroseat_resv TARGET usseat_resv COMPARECOLS (ON UPDATE KEYINCLUDING (last_update)) RESOLVECONFLICT (UPDATEROWEXISTS (min_resolution_method USEMIN (last_update) COLS (passenger last_update)) (DEFAULT OVERWRITE))

MAP euroseat_resv TARGET usseat_resv_ex eptionsEXCEPTIONSONLY INSERTALLRECORDS SQLEXEC (id qry QUERY sele t passengerlast_updt from usseat_resv where id = p1 PARAMS (p1 = id)) COLMAP ( USEDEFAULTS total_ dr_ onfli ts = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_CONFLICTSrdquo) dr_failed = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_FAILEDrdquo) dr_su essful = GETENV(ldquoDELTASTATSrdquo ldquoTABLErdquo ldquousseat_resvrdquo ldquoCDR_RESOLUTIONS_SUCCEEDEDrdquo) resolution_date = DATENOW() optype = GETENV(LASTERR OPTYPE) dberrnum = GETENV(LASTERR DBERRNUM) dberrmsg = GETENV(LASTERR DBERRMSG) table_name = GETENV(GGHEADER TABLENAME) passenger_after = passenger last_updt_after = last_updt passenger_before = beforepassenger last_updt_before = beforelast_updt passenger_ urrent = qrypassenger last_updt_ urrent = qrylast_updt )

19

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 22: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

The parame er EXCEPTIONSONLY ins ruc s he Delivery process o handle errors in he

previous SEAT_RESV able map s a emen if hey occur The parame er

INSERTALLRECORDS ins ruc s he Delivery process o change all ransac ions o inser

s a emen s when wri ing o he SEAT_RESV_EXCEPTIONS able COLMAP explici ly maps

source columns o arge columns and USEDEFAULTS au oma ically maps columns wi h he

same name The TOTAL_CDR_CONFLICTS column s ores he o al number of conflic s ha

have occurred wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_FAILED column s ores he number of conflic s ha

failed o resolve wi h he DELTASTATS parame er keeping a running coun from he previous

ime s a is ics were collec ed The CDR_SUCCESSFUL column s ores he number of conflic s

ha were resolved successfully wi h he DELTASTATS parame er keeping a running coun from

he previous ime s a is ics were collec ed The RESOLUTION_DATE column will s ore he

da e ime he conflic occurred The OPTYPE column will s ore he opera ion ype of inser

upda e or dele e The DBERRNUM column will s ore he da abase error code The

DBERRMSG column will s ore he da abase error message The TABLE_NAME column will

s ore he name of he able ha had he conflic The PASSENGER_AFTER column will s ore

he af er image of he PASSENGER value from he rail file The LAST_UPDT_AFTER

column will s ore he af er image of he LAST_UPDT value from he rail file The

PASSENGER_BEFORE column will s ore he before image of he PASSENGER value from

he rail file The LAST_UPDT_BEFORE column will s ore he before image of he

LAST_UPDT value from he rail file The PASSENGER_CURRENT column will s ore he

PASSENGER value ha is curren ly in he SEAT_RESV able The LAST_UPDT_CURRENT

column will s ore he LAST_UPDT value ha is curren ly in he SEAT_RESV able

Notification

In his example Kevinrsquos sea reserva ion was overwri en by Joersquos sea reserva ion in he US

Da abase and Joersquos ransac ion would be inser ed in o he SEAT_RESV_EXCEPTIONS able

The SEAT_RESV_EXCEPTIONS able would con ain he following row in he US Da abase

US Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 EUROSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

JOE 15-OCT-12 103000 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

20

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 23: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

Kevinrsquos sea reserva ion ransac ion would be ignored and inser ed in o he

SEAT_RESV_EXCEPTIONS able in he European Da abase as Joersquos sea reserva ion is older

han Kevinrsquos The SEAT_RESV_EXCEPTIONS able would con ain he following row in he

European Da abase

European Database (SEAT_RESV_EXCEPTIONS table)

RESOLUTION_DATE OPTYPE DBERRNUM DBERRMSG TABLE_NAME

15-OCT-12 SQL COMPUP 1403 USSEAT_RESV

PASSENGER_AFTER LAST_UPDT_AFTER PASSENGER_BEFORE LAST_UPDT_BEFORE

KEVIN 15-OCT-12 103001 AM 01-OCT-12 81545 AM

PASSENGER_CURRENT LAST_UPDT_CURRENT ID SEAT FLIGHT

JOE 15-OCT-12 103000 AM 1 2A 123

TOTAL_CDR_CONFLICTS CDR_FAILED CDR_SUCCESSFUL

1 0 1

The informa ion s ored in he SEAT_RESV_EXCEPTIONS ables would be used o iden ify

cus omers ha need o re-reserve heir sea s A bes prac ice would be o have a ba ch process

periodically check he excep ion ables where he af er image is no equal o he curren image In

his example he PASSENGER_AFTER value is ldquoKevinrdquo and he PASSENGER_CURRENT

value is ldquoJoerdquo in he European Da abasersquos SEAT_RESV_EXCEPTIONS able From his

informa ion he airline iden ifies Kevinrsquos sea reserva ion was overwri en by Joersquos and no ifies

Kevin ha he needs o re-reserve his sea

The same process ha handles no ifica ions in his example can be used for a number of

differen opera ions such as providing informa ion o he da abase adminis ra ion eam on he

number of excep ions ypes of excep ions and de ails abou hose excep ions Resolving hese

issues wi h he leas impac o your business is cri ical and minimizing conflic s reduces he

complexi y significan ly

O acle GoldenGate Data Definition Language Replication

Oracle GoldenGa e provides Da a Defini ion Language (DDL) replica ion for Oracle Da abase

o allow a user o cap ure and propaga e DDL changes from one sys em o ano her However

special considera ions need o be aken in o accoun if DDL replica ion is o be used in an ac ive-

ac ive replica ion configura ion

DDL opera ions ha al er or change he exis ing able defini ions should no be issued in a

produc ion ac ive-ac ive replica ion environmen These s a emen s would include DDL

opera ions ha drop columns rename ables or modify objec s in such a way ha he DML

s a emen or Oracle GoldenGa e parame er files would also be required o change For

21

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 24: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and Resolution in Active-Active Replication Envi onments Using O acle GoldenGate

example if a column is dropped in one da abase and a he same ime someone inser s a value

in o ha column on a differen da abase his would cause problems ha could no be handled by

conflic resolu ion In order o manage hese ypes of DDL opera ions in an ac ive-ac ive

replica ion environmen one da abase would be quiesced un il only one da abase was ac ive and

he configura ion was now running in an ac ive-passive replica ion mode DDL opera ions could

hen be execu ed and parame er files upda ed on he passive da abase wi hou in erfering wi h in-

fligh ransac ions Af er he passive da abase has been upda ed he roles would be reversed for

he remaining da abase o be upda ed as well Once bo h da abases have been upda ed including

GoldenGa e parame er files he environmen configura ion would re urn o ac ive-ac ive

replica ion mode

DDL opera ions ha donrsquo modify exis ing able defini ions can safely be execu ed in an ac ive-

ac ive replica ion environmen bu should only be execu ed agains one da abase These

s a emen s would include DDL opera ions ha crea e new ables crea e new ablespaces or

crea e users

Conclusion

Ac ive-ac ive da abase replica ion implemen a ions can seem like a daun ing ask especially when

building some of he more-complex conflic de ec ion and resolu ion rou ines This should no

discourage anyone from pursuing such a solu ion for heir business There are remendous

benefi s o improve da abase performance response imes and availabili y and o achieve

significan scalabili y gains by implemen ing an ac ive-ac ive replica ion environmen

Da abases and servers will faili is inevi able To be ready for his companies have inves ed

hundreds of millions of dollars on disas er recovery cen ers and 247 opera ions Ac ive-ac ive

replica ion configura ions deliver excellen business con inui y resul s in a cos -effec ive way

Oracle GoldenGa e is uniquely designed o suppor robus conflic de ec ion and resolu ion

processes o op imize da a accuracy and in egri y in ac ive-ac ive replica ion configura ions

22

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109

Page 25: Best Practices for Conflict Detection and Resolution in Active-Active Replication Environments

Best P actices fo Conflict Detection and

Resolution in Active-Active Replication

Envi onments Using O acle GoldenGate

Updated Decembe 2012

O acle Co po ation

Wo ld Headqua te s

500 O acle Pa kway

Redwood Sho es CA 94065

USA

Wo ldwide Inqui ies

Phone +16505067000

Fax +16505067200

o aclecom

Copy ight copy 2012 O acle ando its affiliates All ights ese ved This document is p ovided fo info mation pu poses only and

the contents he eof a e subject to change without notice This document is not wa anted to be e o -f ee no subject to any othe

wa anties o conditions whethe exp essed o ally o implied in law including implied wa anties and conditions of me chantability o

fitness fo a pa ticula pu pose We specifically disclaim any liability with espect to this document and no cont actual obligations a e

fo med eithe di ectly o indi ectly by this document This document may not be ep oduced o t ansmitted in any fo m o by any

means elect onic o mechanical fo any pu pose without ou p io w itten pe mission

O acle is a egiste ed t adema k of O acle Co po ation ando its affiliates Othe names may be t adema ks of thei espective

owne s

0109