Transcript
Page 1: OpenEdge Database Performance Tuning

1

OpenEdgeDatabasePerformanceTuningTomBascom,WhiteStarSoftwareAbstract:Usersarecomplainingthatyourdatabaseisslow?Cometothissessiontofindoutwhatyoucandoaboutthat!Whatarethecriticalperformancetuningparametersandoptions?Howdoyouknowwhichonestotweakandwhatvaluestouse?Jointhissessiontofindouttheanswerstoallthesequestionsandmore!

Page 2: OpenEdge Database Performance Tuning

OpenEdgeDatabasePerformanceTuning

aka:HowToMakeItGoFast!

TomBascom,[email protected]

Page 3: OpenEdge Database Performance Tuning

AFewWordsabouttheSpeaker

•  TomBascom:Progressuser&roamingDBAsince1987•  Partner:WhiteStarSoftware&DBAppraise,LLC

– ExpertconsultingservicesrelatedtoallaspectsofProgressandOpenEdge.– RemotedatabasemanagementserviceforOpenEdge.– Authorof:– Simplifyingthejobofmanagingandmonitoringtheworld’sbestbusinessapplications.

–  [email protected]

3

Page 4: OpenEdge Database Performance Tuning

{inc/disclaimer.i}

•  Yourkilometeragewillvary.•  Thesetipsareinnospecialorder.Butsometimesorderdoesmatter.

•  Inanidealworldyouwillchangeonethingatatime,testandremovechangesthatdonothavethedesiredeffect.

•  Therealworldisn’tveryideal.

Page 5: OpenEdge Database Performance Tuning

TheList!

Page 6: OpenEdge Database Performance Tuning

#30PickYourBattles

Theperformanceenhancementpossiblewithagivenimprovementislimitedbythefractionofthe

executiontimethattheimprovedfeatureisused. --Amdahl’sLaw

Page 7: OpenEdge Database Performance Tuning

#30Inotherwords:

•  Tryingtoimprovesmallthingsthatnobodynoticesprobablyisn’ttheroadtofameandfortune

•  Bigqueriesthatreturnlotsofdataandwhicharefrequentlyusedbylotsofuserswillbemuchmorenoticeable

Page 8: OpenEdge Database Performance Tuning

#29StayCurrent

•  UptodatereleasesofProgress,yourOSandyourapplicationareessentialcomponentsofawelltunedsystem.

•  Youcannottakeadvantageofthebesttechniques,improvedalgorithmsornewhardwarewithoutstayinguptodate.

•  Failuretostayuptodatemaymeanpoorperformance,increasedcostsandgeneraluncompetitiveness.

Page 9: OpenEdge Database Performance Tuning

#29StayCurrent•  TRXthroughputinv8->v9(CCLP)•  Major4GLexecutionspeedimprovementsin9.1EandOE10•  64bitplatformsupportandlargememory.•  OE10–Type2StorageAreas•  SignificantimprovementsinDBreadperformance•  10.2b06andoe11added–lruskipsand–prefetch*•  11.4addstablepartitioning•  11.7bringsCDC…

Page 10: OpenEdge Database Performance Tuning

#28Set*rangesize

•  Thedefaultvalueof50isuseless.•  Withoutthefullsetoftablesenabledformonitoringmanydiagnostictechniquescannotbeused.

•  Youneedtorestartthedbtochangethesesodoitproactively:– Notwhenyoualreadyhaveaproblem.– Roundupabittoavoidneedingtoadjustwitheveryschemachange.– Keepaneyeonyourcoverage!Don’tgetoutofsync.

Page 11: OpenEdge Database Performance Tuning

•  Type2storageareasarethefoundationforalladvancedfeaturesoftheOpenEdgedatabase.

•  DatablocksinType2areascontaindatafromjustonetable!•  Type2areashave“clustersizes”of8,64or512.

•  Clustersizesof0or1areType1areasL

#27Type2StorageAreas

d "Data":20,128;512 /db/dbname_20.d1d "Indexes":21,1;8 /db/dbname_21.d1d "LOBs":22,64;512 /db/dbname_22.d1

11

Page 12: OpenEdge Database Performance Tuning

#27Type2StorageAreas

•  Alwaysusetype2areas…•  …forareasthatcontaindata,indexesorLOBS.•  Theschemaareaisatype1areaL

12

Page 13: OpenEdge Database Performance Tuning

#27Type2StorageAreas

•  Alwaysusetype2areas…•  …forareasthatcontaindata,indexesorLOBS.•  Theschemaareaisatype1areaL

•  ThusNOdatashouldeverbeintheschemaarea!

13

Page 14: OpenEdge Database Performance Tuning

#27Type2StorageAreas

•  Alwaysusetype2areas…•  …forareasthatcontaindata,indexesorLOBS.•  Theschemaareaisatype1areaL

•  IfyouthinkthatyouhavealegitimateexceptionIexpecttoseeadetailedtalkaboutitnextyear.

14

Page 15: OpenEdge Database Performance Tuning

#26RapidReaders

•  Similartoarunaway–consumesawholeCPU•  Butisactuallydoingdb“logical”IO•  Usuallycausedby:

– Tablescans– Poorindexselection.– Unmonitoredbatchprocessesandapp-servers.– Reallybadalgorithmchoices.

Page 16: OpenEdge Database Performance Tuning

#26RapidReaders

Direct Auto Sampling JSON 15255 4576 0.227 ProTop Version 3.3h 2015/09/27 18:49:53 dbappraise 28 0 /db/dbappraise dbappraise Hit% 99.06 Commits: 0 New RM: 0 Oldest TRX: 00:00:00 Connections: 10 Log Reads: 194,472 Undos: 0 From RM: 0 Curr BIClstr: 0 Brokers: 1 OS Reads: 1,835 Lock Tbl HWM: 48 From Free: 0 Oldest BIClstr: 0 4gl Servers: 0 Rec Reads: 95,269 Curr # Locks: 0 Examined: 0 Checkpoints: 1,139 SQL Servers: 0 LogRd/RecRd: 2.04 Modified Bufs: -16 Front2Bk: 0 Curr AI Extent: Disabled 4gl Clients: 6 Log Writes: 0 IO Response: 3.33 Remove Lk: 0 Curr AI Seq#: 0 SQL Clients: 0 OS Writes: 0 BogoMIPS: 0.00 Empty AI Exts: -1 App Server: 0 Rec Creates: 0 BogoMIP%: 0.00 Full AI Exts: -1 BIW: 1 Rec Updates: 0 AIW: 0 Rec Deletes: 0 Notes: 0 0 APW Writes: 0 APWs: 2 Rec Locks: 10 BIW/AIW Write% 0 0 APW Write% 0 WDOG: 0 Rec Waits: 0 Writes to Log: 0 0 Bufs Scanned: 0 Local: 2 Idx Blk Spl: 0 BIW/AIW Writes: 0 0 APW Scan Wrts: 0 Remote: 0 Resrc Waits: 0 Partial Buf Wr: 0 0 APW Q Wrts: 0 Batch: 4 Latch Waits: 2 Busy Buf Waits: 0 0 Chkpt Q Wrts: 0 TRX: 0 Empty Buf Wts: 0 0 Flushed Bufs: 0 Blocked: 0 ┌───────────────────────────────────────────────────────── Table Activity ─────────────────────────────────────────────────────────┐ │ Tbl# Area# Table Name #Records Turns Create Read v Update Delete OS Read │ │ ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── │ │ > 8 22 activity 0.00 0 57,351 0 0 1,731 │ │ 6 10 site 0.00 0 37,874 0 0 0 │ ┌──────────────────────────────────────────────────────── User IO Activity ────────────────────────────────────────────────────────┐ │ Usr# Name PID Flags Blk Ac v OS Rd OS Wr Hit% Rec Lck Rec Wts Line# Program Name │ │ ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── │ │ > 16 tom 14505 S4B 194216 1826 0 99.06% 0 0 34 ./test/churn.p │ │ 12 tom 22668 S4 62 0 0 100.00% 6 0 630 lib/vstlib.p │

Page 17: OpenEdge Database Performance Tuning

#25-lruskips100

•  Itissimpleandeffective.•  Eliminatesalotofpointlessinternalhousekeeping.•  Bigbenefitforbusysystems!•  Nonegativeimpactonquietsystems.•  Makestheimpactofbadcodeslightlylessawful.•  Goaheadandset–lruskips2100whileyou’reatit.

Page 18: OpenEdge Database Performance Tuning

#24Transactions

•  Distinguishbetweena“businesstransaction”anda“databasetransaction”.

•  Donottrytoabuseadatabasetransactiontoenforceabusinessrule:– HugeLockTablesareasignofthis.– Youmayneedtocreate“reversing(business)transactions”.– Orrestartabletransactions.

•  Forlargedatabaseoperations“chunk”yourtransactions.

Page 19: OpenEdge Database Performance Tuning

#23“Chunking”Transactions define variable i as integer no-undo.

outer: do for customer transaction while true:

inner: do while true:

i = i + 1.

find next customer exclusive-lock no-error. if not available customer then leave outer. discount = 0. /* the “work” part of things… */

if i modulo 100 = 0 then next outer. end.

end. /* outer */

Page 20: OpenEdge Database Performance Tuning

#22DefineBufferTableforTable.•  Veryusefulininternalprocedures.•  Itpreventstheaccidental“borrowing”ofabuffer.•  Limitsthedamagefromunintentionalfreereferences.

procedure x: define buffer customer for customer. /* ß comment me out! */ find first customer no-lock. display cust-name. end. find last customer no-lock. display cust-name. run x. display cust-name.

Page 21: OpenEdge Database Performance Tuning

#21Parallelize

•  ManylegacyprocessesweredesignedwhenthesystemwasmuchsmallerorwhenmultipleCPUswereunusual.

•  Stepoutsideofthesingle-threadedboxandconsiderwhatportionsofyoursystemcouldbenefitfrombeingparallelized:

•  MRPRuns•  NightlyProcessing•  Reports•  DataExtracts•  DataImports

Page 22: OpenEdge Database Performance Tuning

#21Parallelize$mbprodbname–pexp.p–param“01|0,3000”$mbprodbname–pexp.p–param“02|3000,6000”$mbprodbname–pexp.p–param“03|6000,9999”/*exp.p*/definevariablestartCustasintegerno-undo.definevariableendCustasintegerno-undo.startCust=integer(entry(1,entry(2,session:parameter,“|”))).endCust=integer(entry(2,entry(2,session:parameter,“|”))).outputtovalue(“export.”+entry(1,session:parameter,“|”).foreachcustomerno-lockwherecustNum>=startCustandcustNum<endCust:exportcustomer.end.outputclose.quit.

Page 23: OpenEdge Database Performance Tuning

#21Parallelize

•  Butdon’tgocrazy!

Page 24: OpenEdge Database Performance Tuning

#20Delete1recordatatime

•  Becarefulwithlargescalepurgeprocesses.•  DoNOT“chunk”deletes.•  Anddonotparallelizethem.

Page 25: OpenEdge Database Performance Tuning

#19Implement-bithold

•  Thebifileshouldnotbeallowedtogrowbeyondapproximately1/4ththefreespaceofthefilesystemholdingit.

•  Thatisbecausecrashrecoverymayrequire2xto3xthesizeofthebifiletocompleteandrequestingextradiskspaceatthatpointintimeisgenerallynotpossible.

•  Ifyouhaveneverhadaproblemwithexcessbigrowth,thisisawaytoensurethatyouneverdo.

-bithold 500

Page 26: OpenEdge Database Performance Tuning

#18Otherargs

•  Themostimportantpropertyinconmgr.properties•  Useforparametersthatarenotsupportedbyexploder.•  Alsoallowsyoutopointtoa.pffileandmaybeevenavoidhavingtouseOpenEdgeExploreratall!

Page 27: OpenEdge Database Performance Tuning

#17Dump&Load

•  Youshouldn’tneedtoroutinelydump&load.•  Ifyou’reonOE10+,•  Usingtype2areas,•  Thathavebeenwelldefined,•  Andyouhaveawell-behavedapplication.

•  TherestofusbenefitfromanoccasionalD&L.

Page 28: OpenEdge Database Performance Tuning

#17Dump&Load

•  Donotbeafraidtousetheproutilbinarydump&load.•  “Binary”referstotherecordcontents–nottheblockstructure.•  Binarydumpisportable:

– UpwardsacrossProgressversions(i.e.v9tooe11)– BetweenallPlatforms(i.e.SolaristoWindows)– Between“bitness”(i.e.32bitWindowsv9to64bitLinuxOE11)

•  Mostpeoplecangetbywithsomesimplescripts.

Page 29: OpenEdge Database Performance Tuning

#16LargerdbBlocks

•  LargerblocksresultinmuchmoreefficientIO.•  FewerIOopsmeanlesscontentionfordisk.•  Movingfrom1kto4kishuge.4kto8kisrelativelylesshugebutstillvaluable.

•  8kworksbestinmostcases.Especiallyread-heavyworkloads.•  Betterdiskspaceefficiency(tighterpacking,lessoverhead).

•  Don’tforgettoadjust–BandRowsPerBlock!

Page 30: OpenEdge Database Performance Tuning

#16LargerdbBlocks

•  LargeBlocksreduceIO,feweroperationsareneededtomovethesameamountofdata.

•  Moredatacanbepackedintothesamespacebecausethereisproportionallylessoverhead.

•  Becausealargeblockcancontainmoredataithasimprovedoddsofbeingacache“hit”.

•  LargeblocksenableHWfeaturestobeleveraged.EspeciallySANHW.

Page 31: OpenEdge Database Performance Tuning

#15ManageTempFileIO

•  Temp-fileIOcanexceeddbIO.•  Sometimesby2:1,3:1ormore!•  -TisolatestempfileIO.•  -thelpsyoutocrudelydiagnosethesourceofIO.•  -yprovidessomedetailregardingr-codeswapping.•  -mmaxbuffersr-code,4096isagoodstartforChUI,16384forGUI.•  Memorymappedprocedurelibrariescacher-code.•  Use–Bt&-tmpbsizetotune4GLtemp-tables.

Page 32: OpenEdge Database Performance Tuning

#15ManageTempFileIO-rw-r--r-- 1 VEILLELA users 579312 Oct 19 15:16 srtrAyhEb -rw-r--r-- 1 wrightb users 35697664 Oct 19 15:16 srtH6miqb -rw-r--r-- 1 STEELEJL users 36772864 Oct 19 15:16 srtz37kyb -rw-r--r-- 1 THERRIKS users 0 Oct 19 07:12 srt--Elab -rw-r--r-- 1 root users 17649 Oct 19 15:16 lbiV6Qp7a -rw-r--r-- 1 root users 34704 Oct 19 15:16 lbi-TymMa -rw-r--r-- 1 wrightb users 811008 Oct 19 15:16 DBIHDmiqc -rw-r--r-- 1 BECKERLM users 8192 Oct 19 11:06 DBI--Abac -rw-r--r-- 1 CALUBACJ users 8192 Oct 19 09:16 DBI--Abyc

CLIENT.MON (-y) Program access statistics: Times Bytes Reads from temp file: 0 0 Writes to temp file: 0 0 Loads of .r programs: 14 524594 Saves of compilation .r's: 0 0 Compilations of .p's: 0 0 Checks of files with stat: 165 0

Page 33: OpenEdge Database Performance Tuning

#14UpdateStatistics

•  SQL-92usesacostbasedoptimizer•  Butitcannotoptimizewithoutknowledgeofthecost!(datadistribution).

•  Monthlyorquarterly“updatestatistics”isappropriateformostpeople.

•  Orwhen20%ofyourdatahaschanged.•  Thisisadataintenseprocess:

– Runitduringoffhoursifyoucan– Youmightwanttoonlydoafewtables/indexesatatime

Page 34: OpenEdge Database Performance Tuning

#14UPDATESTATISTICS

34

/*genUpdateSQL.p**mprodbName–pgenUpdateSQL.p-param”updstats.sql"**sqlexp-useruser-passwordpassWord-dbdbName-Sport-infileupdstats.sql–outfileupdstats.log*/outputtovalue(session:parameter).foreach_fileno-lockwhere_hidden=no:putunformatted"UPDATETABLESTATISTICSANDINDEXSTATISTICSAND”“ALLCOLUMNSTATISTICSFORPUB."'"'_file._file-name'"'";"skip"commitwork;”skip.end.outputclose.

Page 35: OpenEdge Database Performance Tuning

#13ProgressAppServer

•  Usedtoreducenetworktrafficandlatency.•  Whenproperlyimplementeditwillminimizethepathlengthbetweenbusinesslogicanddatapersistencelayers.

•  IOW,forbestperformance,theAppServershouldliveonthesameserverasthedatabaseanduseaself-serviceconnection.

•  InaVMenvironmentyourappservermightnotbeself-servicebutitshouldbeonthesamephysicalhost.

Page 36: OpenEdge Database Performance Tuning

#12fork()&exec()definevariableiasintegerno-undo.definevariablefSizeasintegerno-undo.etime(yes).doi=1to1000:inputthroughvalue("ls-ld..").import^^^^fSize.inputclose.end.displayetimefSize.

definevariableiasintegerno-undo.definevariablefSizeasintegerno-undo.etime(yes).doi=1to1000:file-info:file-name="..".fSize=file-info:file-size.end.displayetimefSize.

3140ms,atleast1000callstoeachofopen(),close(),fork(),exec(),read()completewithmultiplecontextswitchesperinvocation.

16ms,1,000stat()calls.

Page 37: OpenEdge Database Performance Tuning

#11-spin

•  Allnewmachines,evendesktops,laptops,phonesandwatchesarenowmulti-core.

•  DoNOTusetheoldX*#ofCPUsruletoset–spin.Itisbogus.•  Biggerisnotalwaysbetterwith–spin!•  Modestvalues(5,000to10,000)generallyprovidethebestandmostconsistentresultsforthevastmajorityofpeople.

•  Usereadprobe.ptoexplore.•  CheckoutRichBanville’sSuperbExchange2008Presentation!

Page 38: OpenEdge Database Performance Tuning

#10biclustersize•  TheideaistoreducethenumberandfrequencyofcheckpointsgivingAPWsplentyoftimetowork.

•  LargerbiclusterspermitspikesintheworkloadtotakeplacewithoutambushingtheAPWs.

•  Easytooverlookwhenbuildingnewdbviaprostrctcreate…•  512isthedefaultOE10biclustersize.•  8192isreasonableforsmallsystems.•  16384is“agoodstart”forlargersystems.•  LongerREDOphaseonstartupsodon’tgetcrazy.•  NOTagoodideafor“Workgroup”databaselicenses.

–  ForWGsmallvalues(512or1024)arebetter.

Page 39: OpenEdge Database Performance Tuning

#10biclustersize

$grep‘(4250)’dbname.lg(4250) Before-ImageClusterSize:524288. (=512k)Thevalueaboveistheoe10defaultvalueof512ksolet’smakeitlarger:

$proutildbname-Ctruncatebi-bi16384…(1620)Before-imageclustersizesetto16384kb.(1621) Before-ImageClusterSize:16777216.(=16384k)$proutildbname-C-bigrow8

Page 40: OpenEdge Database Performance Tuning

#9APW,AIW,BIW&WDOG

•  AlwaysstartaBIW•  AlwaysstartanAIW•  StartWDOG•  OneorTwoAPWsareusuallyenough:

– DoNOTfollowtheold“1APWperdisk”suggestion.– Toomanyisjusta(small)wasteofCPUcycles.–  IfyouareconsistentlyflushingbuffersatcheckpointsincreasebiclustersizeandaddanAPW(oneatatimeuntilbufferflushesstop).

Page 41: OpenEdge Database Performance Tuning

#8BeWaryofVirtualization•  Virtualizationisnotmagic!•  Thedefaultassumptionwithvirtualizationisthatyoudonotreallyneedalloftheresourcesavailabletoyou.

•  Overcommittingresourceswillhurtperformance.•  Virtualizationcannotcreatecapacityfromthinair.•  AsaDBAyoumustsizedbserversforpeakload.WithsharedresourcesyoumustsizeforsimultaneouspeaksinmultipleVMs!(Sys-adminsoftenassumesimultaneousvalleysforallresources…)

Page 42: OpenEdge Database Performance Tuning

#8Notmagic…

•  APartnertellsacustomer“youcansupport50connectionsonthatserver”.

•  Thecustomerhas200users...•  Sotheyvirtualizetheserverandbringup4VMs!

Page 43: OpenEdge Database Performance Tuning

#8RightSizeYourVirtualEnvironment

•  AdatabaseserverisNOTthesameasafileserver•  DoNOToverallocateResources•  DoNOT“thinprovision”•  Ensurethatyoualwayshavesparecapacity

Page 44: OpenEdge Database Performance Tuning

#7MinimizeNetworkTraffic

•  UseFIELDSlistinqueries.•  Use–cacheand–pls.•  NO-LOCKqueriespackmultiplerecordsintoarequest.•  Watchoutforclient-sidesortingandselectiononqueries.•  RememberthatCAN-DOisevaluatedontheCLIENT(yetanotherreasonnottouseit).

•  Use-noautoresultlist/FORWARD-ONLYfordynamicqueries.

Page 45: OpenEdge Database Performance Tuning

#7MinimizeNetworkTraffic

•  Useasecondarybrokertoisolatehighactivityclients(suchasreports).

•  Set–Mmto8192orlarger.•  Use–Mnand–Matokeepthenumberofclientsperserverlow(3orless).

•  Use–Mi1tospreadconnectionsacrossservers.

Page 46: OpenEdge Database Performance Tuning

#7MinimizeNetworkTraffic

•  JumboFrames!•  -prefetchDelay•  -prefetchFactor100•  -prefetchNumRecs10000

Page 47: OpenEdge Database Performance Tuning

ImpactofMessageSize&PrefetchOptions

totMsgs! qryRecv recSent recs/qry etime nettime Description...

198 ! 97 1758 18 10 208 -Mm1024

208 ! 102 1758 17 14 222 -Mm4096

192 ! 94 1758 19 9 201 -Mm8192

180 ! 88 1758 20 11 191 -Mm16384

162 ! 79 1758 22 14 176 -Mm32600

152 ! 74 1758 24 8 160 -prefetchDelay

154 ! 75 1758 23 12 166 -prefetchDelay-prefetchFactor100

8 ! 2 1758 879 22 30 -prefetchDelay-prefetchFactor100-prefetchNumRecs10000

foreach_indexfields(_field-name)no-lock:end.

Page 48: OpenEdge Database Performance Tuning

ImpactofMessageSize&PrefetchNumRecs

totMsgs! qryRecv recSent recs/qry etime nettime Description...

146 71 1758 25 13 159 -prefetchDelay-prefetchFactor100

102 49 1758 36 8 110 -Mm1024-prefetchNumRecs10000

28 12 1758 147 9 37 -Mm4096-prefetchNumRecs10000

16 6 1758 293 13 29 -Mm8192-prefetchNumRecs10000

10 3 1758 586 9 19 -Mm16384-prefetchNumRecs10000

8 2 1758 879 23 31 -Mm32600-prefetchNumRecs10000

8 2 1758 879 22 30 -prefetchDelay-prefetchNumRecs10000

8 2 1758 879 22 30 -prefetchDelay-prefetchFactor100-prefetchNumRecs10000

-prefetchNumRecsdominates!

Page 49: OpenEdge Database Performance Tuning

ImpactofBasicCodingApproachestotMsgs! qryRecv recSent recs/qry etime nettime Description...

3519 0 1758 ? 139 3658 dowhile…findno-lock5276 0 1758 ? 174 5450 dowhile…findshare-lock28 12 1758 147 7 35 foreachno-lock8 2 1758 879 22 30 FENLfields()

5277 1758 1758 1 155 5432 FEshare-lock3520 1758 1758 1 134 3654 FEexclusive-lock3519 1758 1758 1 87 3606 openquery3519 1758 1758 1 84 3603 openqueryfields()7 2 1758 879 29 36 openqueryfields()cache507 2 1758 879 30 37 openqueryfields()cache500020 8 1758 220 113 133 sql89select*8 2 1758 879 37 45 sql89select_field-name

No,IamnotendorsingSQL89

Page 50: OpenEdge Database Performance Tuning

NestingandJoins

totMsgs! qryRecv recSent recs/qry etime nettime Description...587 195 1950 10 33 620 nestedFE587 195 1950 10 33 620 joinedFE6183 195 4748 24 207 6390 joinedFEw/sortoninnerfield33 14 1951 139 150 183 TToption

/*simplenesting*/foreach_fileno-lock:foreach_fieldno-lockof_file:end.end.

/*uglysortcriteria*/foreach_fileno-lock,each_fieldno-lockof_fileby_field-name:end.

/*useajoininsteadofnesting*/foreach_fileno-lock,each_fieldno-lockof_file:end.

Page 51: OpenEdge Database Performance Tuning

#6TheBufferCache•  ThecurefordiskIOisRAM.•  UseRAMtobufferandcacheIOops.•  Efficiencyof–B:

–  Islooselymeasuredbyhitratio.–  Changesfollowaninversesquarelaw.–  Tomakeanoticeablechangeyoumustmakealargechangeto–B.

•  100,000is“agoodstart”(800MB@8kblocks)•  1,000,000isnotunusual•  8,000,000ismycurrentlargestcustomervalueinproduction

Page 52: OpenEdge Database Performance Tuning

#6TheBufferCache

Layer Time #ofRecs

#ofOps CostperOp

Relative

Progressto–B 0.96 100,000 203,473 0.000005 1-BtoFSCache 10.24 100,000 26,711 0.000383 75FSCachetoSAN 5.93 100,000 26,711 0.000222 45-BtoSANCache 11.17 100,000 26,711 0.000605 120SANCachetoDisk 200.35 100,000 26,711 0.007500 1500

-BtoDisk 211.52 100,000 26,711 0.007919 1585

InBigBYouShouldTrust!

(Approximately4recordsperreadopinnon–Bcases.)

Page 53: OpenEdge Database Performance Tuning

#5IOSubsystems

•  Usemorethanonedisk:– Afastdiskcando150to200randomIOOps/sec.– Kbytes/secisameasureofsequentialIO.– OLTPismostlyrandom.

•  Don’twastetimetryingto“manuallystripe”.•  Instead,use“hardware”stripingandmirroring.•  IsolateAIextentsforsafety,notperformance.•  Isolatetemp-file,application,OSand“other”IO.

Page 54: OpenEdge Database Performance Tuning

#5TheSANScam

•  External,SharedstorageisNOTaperformanceenhancer•  DBIOopsarerandomandseektimeiscritical

– Thebettertunedyourdb,themorerandomtheIObecomes

•  Accessingdataoveranetworkconnectionisonly“fast”inthefeveredimaginationofadelusionalsalesperson

•  Lightspeed=1footpernanosecond

Page 55: OpenEdge Database Performance Tuning

RAID#5“Don’tbecheatedfirstandsurprisedlater!”

•  Greatperformancewhenthereisnoload:– Andwhentherearenodiskfailures.– AndifyourdatabaseisroughlythesamesizeastheSANcache.– Whichnobodyelseisusing.– AlloftheRAMthatyoucanuse–cleverlyplacedwhereitwilldoyoutheleastamountofgood.

– Alloftheperformanceofasinglediskwithnoneofthecostsavings.

Page 56: OpenEdge Database Performance Tuning

#5RAID6

•  HowcanwepossiblymakeRAID5Worse?•  Addanotherparitydisk!

Page 57: OpenEdge Database Performance Tuning

#5BAARF.com-EnoughisEnough

•  BattleAgainstAnyRaid(Free,Four,FiveorFix)

–  http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt–  http://www.facebook.com/pages/BAARF–  http://gurucollege.net/rants/baarf-or-why-raid5-isnt-safe/–  http://www.unitrends.com/documents/whitepapers/unitrends-wp-russian-roulette-and-raid-5.pdf

•  Don’tbecheatedfirstandsurprisedlater!

Page 58: OpenEdge Database Performance Tuning

#4IndexCompact

•  CompactsIndexes.•  Removesdeletedrecordplaceholders.•  Improves“utilization”=fewerlevels&blocksandmoreindexentriesperread.

•  Runsonlineoroffline.•  Availablesinceversion9.

Page 59: OpenEdge Database Performance Tuning

#4IndexCompact

•  DoNOTsettarget%for100!•  Considercompactingwhenutilization<70%•  …andblocks>1,000.

proutildbname–Cidxcompacttable.indextarget%

INDEX BLOCK SUMMARY FOR AREA "APP_FLAGS_Idx" : 96 ------------------------------------------------------- Table Index Fields Levels Blocks Size %Util Factor PUB.APP_FLAGS AppNo 183 1 3 4764 37.1M 89.9 1.2 FaxDateTime 184 2 2 45 259.8K 72.4 1.6 FaxUserNotified 185 2 2 86 450.1K 65.6 1.7 INDEX BLOCK SUMMARY FOR AREA "Cust_Index" : 10 ------------------------------------------------------- Table Index Fields Levels Blocks Size % Util Factor PUB.Customer Comments 13 1 1 1 874.0B 21.5 1.0 CountryPost 14 2 2 4 9.0K 56.5 1.9 CustNum 12 1 2 4 9.9K 62.2 1.8 Name 15 1 2 9 22.5K 62.9 1.7 SalesRep 16 1 1 1 1.3K 33.6 1.0

Page 60: OpenEdge Database Performance Tuning

#2SolidStateDisks

•  Virtualizationisnotmagic–butSSDisprettydarnclose!•  Literallythousandsoftimesfasterthanrotatingrust.•  Perfectlysafeandreliable.•  But--disableWindowsdiskdefragprograms.•  Willnotcureallperformanceproblems–buttheysuredohelpwithIOthroughput.

•  ItisstillnecessarytooptimizecodeproperlyanduselotsofRAMfor–B(see#6)

Page 61: OpenEdge Database Performance Tuning

#1Stupid4GLTricks

•  Badcodewilldefeatanyamountofheroictuningandexcellenthardware.

•  Luckilybadcodeisoftenadvertisedbytheperpetratorashavingbeendevelopedto“improveperformance”.

•  Justbecauseafeatureofthe4GLcandosomethingdoesn’tmeanthatitshouldbeusedtodoit.

Page 62: OpenEdge Database Performance Tuning

#1Stupid4GLTricks/* SR#1234 – enhanced lookup to improve performance! */ update cName. find first customer where cName matches customer.name use-index custNum no-error. - Or – /* different variations used in different bits of code… */ find first customer where can-do( cName, name ) use-index custNum no-error.

Page 63: OpenEdge Database Performance Tuning

#1Stupid4GLTricks

•  Thereisalotofcrapcodeintheworld.•  Stopdumpster-divingforcode!•  Don'temulateit,eliminateit.

Page 64: OpenEdge Database Performance Tuning

Questions?

64

Page 65: OpenEdge Database Performance Tuning

ThankYou!

65


Top Related