oracle | integrated cloud applications and platform services - … · remote capture and apply for...
Post on 28-May-2020
6 Views
Preview:
TRANSCRIPT
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
BestPracticesforHighAvailabilityandPerformanceTuningforOracleGoldenGateMicroservices
NickWagnerDirectorofProductManagement- Replication
StephanHaisleyMaximumAvailabilityArchitecture(MAA)
October3rd,2017
Confidential– OracleInternal/Restricted/HighlyRestricted
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.
Confidential– OracleInternal/Restricted/HighlyRestricted 2
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgenda
MicroservicesArchitecture
OverviewofOracleActiveDataguard/FSFO
OverviewofnewGridInfrastructureBundledAgent(XAG)
ConfigurationforGoldenGatewithRACandDataGuard
Summary
1
2
3
4
5
Confidential– OracleInternal/Restricted/HighlyRestricted 3
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgenda
MicroservicesArchitecture
OverviewofOracleActiveDataguard/FSFO
OverviewofnewGridInfrastructureBundledAgent(XAG)
ConfigurationforGoldenGatewithRACandDataGuard
Summary
1
3
4
5
Confidential– OracleInternal/Restricted/HighlyRestricted 4
2
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| 5
OracleGoldenGate12.3NewFeaturesNewGoldenGate ServicesArchitectureRESTinterfacesforConfiguration,AdministrationandMonitoringwithincludedHTML5WebapplicationsDeployatcloudscalewithfullysecureHTTPSinterfacesandSecureWebSocketsforstreamingdata
OracleDatabaseSharding SupportEmbedded&fullyautomatedactive-activereplicationwithinandacrossDatabaseShards
AutomaticConflictDetectionandResolution(CDR)ConflictdetectionandresolutionbuiltdirectlyintoOracleDatabaseKernel
HeterogeneousEnhancementsGenericJDBCSupport.MySQL DDLreplication.Remotecaptureandapplyforz/OS
CloudIntegratedUIforDesign,Development,ManagementandMonitoringforGoldenGate CloudService
ParallelApplyHighlyscalableclientsideapplyenginewithgainsofover5xthroughputonOracle
EnhancedBigDataandNoSQL SupportSampleadaptersforOracleNoSQL,MongoDB.SupportforHiveMetadata
ExpandedOracleDatabase12.2SupportLongidentifiers,ExpandedSCN,LocalUndoforPDBs,Top-levelVARRAYs,REFs.ProceduralreplicationofAdvancedQueues,VirtualPrivateDatabase,OnlineRedefinition,….
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
NewServicesArchitectureforCloudandLarge-scaleDeployments
Source Target
TrailFiles
Extract
ManagerGGSCIAdminServer
TrailFiles
Replicat
PumpsDistributionService
CollectorsReceiverService
SourceDatabase
TargetDatabase
HTTPSRESTFulServiceInterfacesManagerGGSCI
AdminServer
MetricsServer
ServiceManager Service
Manager
MetricsServer
Confidential– OracleInternal/Restricted/HighlyRestricted
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
HowOracleGoldenGateWorks– MicroservicesArchitecture
SourceOracle&Non-OracleDatabase(s)
TargetOracle&Non-Oracle
Database(s)
Capture:committedtransactionsarecaptured(andcanbefiltered)astheyoccurbyreadingthetransactionlogs.Asof V.11.2.1,GoldenGateofferstwooptionsforcaptureforOracle;Classic&IntegratedCapture
Trail:stagesandqueuesdataforrouting.
DistributionServer/Receiver:distributesdataforroutingtotarget(s).
Route:dataiscompressed,encryptedforroutingtotarget(s).
Capture
Delivery
TrailFiles Dist.
Service
TrailFiles
Delivery
Capture
Bi-directional
LAN/WAN/InternetOverTCP/IP Trail
Files
TrailFiles
Delivery:appliesdatawithtransactionintegrity.NewwithGoldenGate12c,IntegratedDelivery.
Dist.Service
ReceiverService
ReceiverService
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| 8
CommandLine,Browsers,ProgrammaticRESTful InterfacesAdministerGoldenGate withVarietyofClients
ServiceInterface
Anything viaService IF
Scripts & Interpreters
PLSQL
Browsers
AdminClient
AdminServer
TrailFiles
ReplicatReceiverServer
TrailFiles
ExtractDistributionServer
HTTPS
ServiceManager
MetricsServer
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
SolutionsForEveryone
Confidential– OracleInternal/Restricted/HighlyRestricted 9
Administrators Traditionalists DevOps
• SimpletouseWebinterface
• Getupandrunningquickly
• Muchlowerlearningcurve
• CommandLineaccessviaadminclient – runanywhere
• SimilarGoldenGatecommandsasinGGSCI
• Interactionthesameaspreviousreleases
• UsesRESTfulAPIs
• Runanywhere
• Rollyourownmonitoring
• Automatecommonactivities
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
MicroservicesArchitecture:BrowserBasedClient
Confidential– OracleInternal/Restricted/HighlyRestricted 10
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgendawithHighlight
MicroservicesArchitecture
OverviewofOracleActiveDataGuard/FSFO
OverviewofnewGridInfrastructureBundledAgent(XAG)
ConfigurationforGoldenGatewithRACandDataGuard
Summary
1
2
3
4
5
Confidential– OracleInternal/Restricted/HighlyRestricted 11
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleActiveDataGuardDisasterRecoveryandRead-OnlyOffloadtoanActiveStandby
StandbySitePrimarySite
DataGuard
SYNC/ASYNC
PrimaryDatabase
ActiveStandbyDatabase
Read-writeWorkload
Real-timeReporting
FastIncrementalBackupsObserver
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleDataGuardConcepts• Switchover:Plannedroletransitionfromaprimarydatabasetooneofitsstandbydatabases.– DGMGRL>SWITCHOVERTOCHICAGO• Requiresconnectivitytobothprimaryandstandbydatabases• Ensuresallredohasshipped• Orderlytransition(bothdatabasesagreeoneverything)
• Failover:Unplannedactivity– DGMRL>FAILOVERTOCHICAGO• Noconnectivitywithprimary• Newprimarybecomesthesourceoftruth
OracleConfidential– Internal 13
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgendawithHighlight
MicroservicesArchitecture
OverviewofOracleActiveDataGuard/FSFO
OverviewofnewGridInfrastructureBundledAgent(XAG)
ConfigurationforGoldenGatewithRACandDataGuard
Summary
1
3
2
4
5
Confidential– OracleInternal/Restricted/HighlyRestricted 14
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleGridInfrastructureBundledAgent(XAG)• ClusterwarespecifictomanagingGoldenGateresources– XAGallowsyoutoregisteraGoldenGateinstancewithCRStoprovideHA
• ItsolvesthekeyprocessrelatedissueofensuringavailabilityoftheGoldenGateinstanceinthefaceoffailures.
• Lossofinstance(RACnodefailover)
• Lossofprimarydatabase(DataGuardFailoverintegration)
• UseAGCTLforregistering andstarting/stoppingresources
15
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleGridInfrastructureBundledAgent(XAG)• SupportcominginGoldenGate 12.3.0.1.2 (notyetavailable)• NewversionofXAGforGoldenGateMicroserviceswillbemadeavailableat*
http://www.oracle.com/technetwork/database/database-technologies/clusterware/downloads/index.html
• InstalloutsideofGridInfrastructureORACLE_HOMEandmakesureOSuserPATHfindsthisXAGbeforetheGIinstalledversion
16
*ThecurrentversionofXAG(v8)doesnotsupportGoldenGate Microservices Architecture
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleGridInfrastructureBundledAgent(XAG)HighLevelConfigurationSteps
• CreateOGGdeploymentusingoggca.sh(similartoDBCA)• Chooseto‘IntegratedwithXAG’(seenextslide)• RegisterServiceManageranddeploymentwithXAG– CreatestheCRSresourcesandalldependencieswithfilesystem,rolebasedservice
• XAGqueriestheGoldenGate ServiceManagerforhealthofthedeployment• XAGinteractsdirectlywithServiceManager,nolongerrunningGGSCIandparsingtheoutputfromcommands–Moreintegrated,simplerandrobust
Confidential– OracleInternal/Restricted/HighlyRestricted 17
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleGridInfrastructureBundledAgent(XAG)ServiceManagerCreation
Confidential– OracleInternal/Restricted/HighlyRestricted 18
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ProgramAgenda
MicroservicesArchitecture
OverviewofOracleActiveDataguard/FSFO
OverviewofnewGridInfrastructureBundledAgent(XAG)
ConfigurationforGoldenGatewithDataGuard
Summary
1
2
3
4
5
Confidential– OracleInternal/Restricted/HighlyRestricted 19
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
SampleDeployment
20
Observer
PrimaryDatabase StandbyDatabase
RedoTransport(SYNCorASYNC)
IntegratedExtract Logmining
Server
TrailandotherOGGFilesInDBFS
RedoTransport
OCIConnection
FileI/OWarehouse
BidirectionalGoldenGateReplication
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
SampleDeployment– PostRoleTransition
21
Observer
(OLD)PrimaryDatabase (NEW)PrimaryDatabase
IntegratedExtractLogMining
Server
Trail/Checkpoint/BRFilesInDBFS
LogMiningServer
RedoTransport
OCIConnection
FileI/O
Warehouse
BidirectionalGoldenGateReplication
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseConnectivity• MakeallOGGcomponentsconnecttodatabaseusingRole-BasedServices– Declarativewaytospecifyaserviceshouldbepublishedonlywhenthedatabasehasaspecificrole– PublishaserviceonlywhendatabasehasthePRIMARYrole
• Createrole-basedservice(12csyntax):srvctl add service -db GGS2PRMY -service oggserv -role PRIMARY
-preferred GGS21 -available GGS22
srvctl add service -db GGS2STBY -service oggserv -role PRIMARY -available GGS21
22
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate ConfigurationFileSystem&HostnameRequirements
• ConsistencyofGoldenGatedataandmetadataensuredwithDBFS• StoretrailandcheckpointdataonDBFStoguaranteeconsistencyonroletransitions
• DBFSoffersintegratedhandsoffautomaticfilesystemmounting,replicatedthroughDataGuard
• Providesflexibleroutingoftraildatapostroletransition– UseaVirtualHostname toidentifytheTARGEThostmachine• Transferredbetweenprimaryandstandbyonroletransition
– UsethevirtualhostnamewhencreatingaDistributionServerpath
23
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
CreateDBFSCRSResource• DBFSismountedonRACnodethatwillrunGoldenGate• CreateCRSactionscript(MOSnote1054431.1)–MustcontainSTART,STOP,CHECKfunctions
• CreateCRSresource:crsctl addresourcedbfs_mount -typecluster_resource \
-attr "ACTION_SCRIPT=$ACTION_SCRIPT,CHECK_INTERVAL=30,RESTART_ATTEMPTS=10,\
START_DEPENDENCIES='hard(ora.$DBNAME.db)pullup(ora.$DBNAME.db)',\
STOP_DEPENDENCIES='hard(ora.$DBNAME.db)‘,SCRIPT_TIMEOUT=300"
• DBFSservicewillbemanagedby XAG
24
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate ConfigurationServiceManagerCreation
Confidential– OracleInternal/Restricted/HighlyRestricted 25
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGateConfigurationDeploymentAutoStart
• EachdeploymentisautomaticallyenabledtoautostartwhenServiceManagerstarts• StateofdeploymentispersistentacrossServiceManagerrestarts– IfdeploymentwasrunningwhenServiceManagerwasstopped,thedeploymentwillbestartedautomaticallywhenServiceManagerstarts.
• MustenableGoldenGate Extract/Replicat autostart– Extract,DistributionPathandReplicat autostartisenabledwiththeAdministrationServeruserinterface
Confidential– OracleInternal/Restricted/HighlyRestricted 26
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate ConfigurationGoldenGate ProcessAutoStartandAutoRestart
Confidential– OracleInternal/Restricted/HighlyRestricted 27
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
• Autorestarttaskscanalsobecreatedusingadminclient:OGG (not connected) 1> HELP SHOWSYNTAX *AUTORESTART*
Admin Client Command Summary:ADD AUTORESTART - Create an autorestart rule.Syntax:| ADD AUTORESTART <rule-name> ( EXTRACT | REPLICAT | ER ) <group-name-wildcard>| [ RETRIES <retries-number> ]| [ WAITSECONDS <wait-number> ]| [ RESETSECONDS <reset-number> ]DELETE AUTORESTART - Remove an autorestart rule.Syntax:| DELETE AUTORESTART <rule-name>INFO AUTORESTART - Returns information about autorestart rules.Syntax:| INFO AUTORESTART <rule-name-wildcard>
Confidential– OracleInternal/Restricted/HighlyRestricted 28
GoldenGate ConfigurationGoldenGate ProcessAutoRestart
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
RegisterGoldenGate withBundledAgent(XAG)• RegisterwithXAGattheprimary(asroot)
agctl addgoldengate GGPRMY--gg_home /u01/oracle/goldengate/gg123_MA--service_manager--config_home /mnt/dbfs/goldengate/deployments/ggsm01/etc/conf--port9100--adminuser admin--useroracle--groupoinstall --vipname xag.GG_MA1-vip.vip--filesystems dbfs_mount --db_services ora.ggdg.oggserv.svc --dataguard_autostart yes
• RegisterwithXAGatthestandby(asroot)agctl addgoldengate GGSTBY--gg_home /u01/oracle/goldengate/gg123_MA--service_manager--config_home /mnt/dbfs/goldengate/deployments/ggsm01/etc/conf--port9100--adminuser admin--useroracle--groupoinstall --vipname xag.GG_MA1-vip.vip--filesystems dbfs_mount --db_services ora.ggdg.oggserv.svc --dataguard_autostart yes--nodesstdby1
• StartExtractusingAgentControl– agctl startgoldengate ggprmy
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate withBundledAgent(XAG)ExampleCommands
• CheckGoldenGate statusagctl statusgoldengate ggprmy
• StartGoldenGateagctl startgoldengate ggprmy --nodeGGsourceNode1
• StopGoldenGateagctl stopgoldengate ggprmy
• Replocate GoldenGate toanotherRACnodeagctl relocategoldengate ggprmy --nodeGGsourceNode2
Confidential– OracleInternal/Restricted/HighlyRestricted 30
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
DataGuard– BoundedDataLoss(ASYNC)• DataGuardinMaxPerformance(ASYNC)modepermitsdatalossonthestandby• AmountofdatalosswithFSFOcontrolledbyFastStartFailoverLagLimit(defaultis30seconds)• ExtractmustonlyminefromredoalreadyappliedtoStandby– Preventsdatabeingreplicatedtoatargetdatabaseandthenmissingfromthesourceafteradatalossfailover
• AddIntegratedExtractparameterTRANLOGOPTIONS HANDLEDLFAILOVER
• IfFSFOisnotenabled,specifyTRANLOGOPTIONS FAILOVERTARGETDESTID n• IdentifieswhichlogarchivedestinationofstandbyExtractmustremainbehind
31
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
DataGuard– BoundedDataLoss(ASYNC)EffectsofTRANLOGOPTIONSHANDLEDLFAILOVER
• AfterDGfailoverifnewstandby(oldprimary)cannotbequeriedforredoapplystatus,Extractabends with:ERROR OGG-02077 Extract encountered a read error in the asynchronous reader thread and is abending: Query to retrieve applied SCN of the target standby database failed.
• Canconfiguretimeouttoallowstandbyreinstatementafterroletransition– TRANLOGOPTIONSDLFAILOVER_TIMEOUT300– Ifstandbynotavailablebytheendofthetimeout,Extractwillabend.
• NOTE:Duringtimeoutperioddataisbeingreplicatedthathasnotyetbeenappliedtostandby,sothereistemporarydatadivergence.
Confidential– OracleInternal/Restricted/HighlyRestricted 32
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate withRACNodeFailurePerformanceMetricsServer
Confidential– OracleInternal/Restricted/HighlyRestricted 33
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
GoldenGate withDataGuardRoleTransitionTargetReceiverServer
Confidential– OracleInternal/Restricted/HighlyRestricted 34
AfteraDataGuardroletransitionthetargetdatabaseReceiverServerautomaticallyrefresheswithnewsourcehost.
Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|
ConfigurationRecommendationSummary• BundledAgentinstalledoutsideofGridInfrastructureinstalldirectory• GoldenGate 12.3.0.1.2andabove• GoldenGate IntegratedExtractandIntegratedReplicat• Role-basedservices forGoldenGate databaseconnectivity• DBFSfor GoldenGate sharedfilesystem• IntegratedExtractparameters• TRANLOGOPTIONSHANDLEDLFAILOVER• TRANLOGOPTIONSFAILOVERTARGETDESTIDn• TRANLOGOPTIONSDLFAILOVER_TIMEOUT300
35
top related