bench tuning bestpractices

8/13/2019 Bench Tuning BestPractices

1/84

Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from lide 121


2/84

Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from lide 122

Insert Pict!re "e

MySQL Performance:Benchmarks, Tuning and BestPractices..

#imitri $%A&'C"($

)y*+ Performance Architect Oracle


3/84

'he follo-ing is intended to o!tline o!r general prod!ct

direction. It is intended for information p!rposes only, and

may not e incorporated into any contract.

It is not a commitment to deliver any material, code, or

f!nctionality, and sho!ld not e relied !pon in maingp!rchasing decisions. 'he development, release, and timing

of any feat!res or f!nctionality descried for Oracles

prod!cts remains at the sole discretion of Oracle.


4/84

Are you imitri!..

es, its me 345"ello from Paris6 745

Passionated y ystems and #ataases Performance

Previo!s 18 years !n 9enchmar Center

tarted -oring on )y*+ Performance since v:.2:9!t d!ring all that time ;!st for f!n only 745

ince last years officially )y*+ Performance f!ll time no-

http3//dimitri.free.fr/log / dimitri


5/84

Agenda

Overvie- of )y*+ Performance=orload oriented t!ning and )y*+ Internals

Performance improvements in )y*+ 8.> ? 9enchmar res!lts

Pending iss!es..

Progress in )y*+ 8.@ Performance* ? A


6/84

Why MySQL Performance ?...


7/84

"hy #enchmarking MySQL!..

Any sol!tion may loo good eno!ghB...


8/84


(ntil it did not reach its limit..


9/84


And even improved sol!tion may not resist to increasing loa


10/84


And reach a similar limit..


11/84


A good enchmar testing may help yo! !nderstand ahead resistance of yo!r sol!tion to incoming potential prolems 7


12/84


9!t eep it in mind3

ven a very po-erf!l sol!tion !tleaved in -rong hands may still eeasily roen6... 345


13/84

The Main MySQL Performance Tuning#1Best Practice is... ???..


14/84

The Main MySQL Performance Tuning#1 Best Practice is... ???..

USE YOUR BRAIN!!! :-)


15/84

The Main MySQL Performance Tuning#1 Best Practice is... ???..

USE YOUR BRAIN!!! :-)

IN FA!IN FA!

!"IS IS !"E!"IS IS !"EMAIN SLIE$ %&''MAIN SLIE$ %&''

IN FA!IN FA!!"IS IS !"E!"IS IS !"E

MAIN SLIE$ %&''MAIN SLIE$ %&''


16/84

Think ata#ase Performance from the #eginnin

erver3

"aving faster CP( is still etter6 :2 cores is good eno!gh 745 O is important6 4 +in!D, olaris, etc.. Eand =indo-s too65

%ight mallocE5 li66 E+in!D3 ;emalloc, olaris3 li!mem5

torage3

#ont !se slo- diss6 EeDcept if this is a test validation goal 3455 # helping random access6 EindeD/data5 more and more cheaper

F is important6 4 GF, (F, *F, &DF, H':, H', HF, etc..

O


17/84

Seek for your #est o%tion..

Sec(r)*ySec(r)*y

Lo+er Pr)ceLo+er Pr)ce

PerformancePerformance


18/84

&no' your %(atform (imits ) features..

)y ac!p is finished on +in!D faster than on olaris same

9e s!re first there is rea((yno more I/O activity once ac!p is finis $eep in mind +in!D !ffering..

+in!D distro3 )y*+ Performance has D regression6 FiD it6

"o- did yo! see itL M O!r *A test is taing D times more time..

=hich engineL M Inno#9.. =hat is innod


19/84

Ad*ise: Benchmark your %(atform ) %rototy%e$

"ave a clear goal6

Other-ise3 Ive otained all these res!lts, and no-... so -hatL.. =ant to sim!late yo!r prod!ction -orloadL..

'hen ;!st sim!late it6 Emany = availale, not al-ays O/free5

"ard to sim!lateL 4 adapt some generic tests

=ant to no- capacity limits of a given platformL till try to foc!s on the test -hich are most significant for yo!6

=ant ;!st to validate config settings impactsL

Foc!s on tests -hich are potentially depending on these settings

Or any, if the goal to prove there are not depending 745 =ell, ;!st eep thining ao!t -hat yo!re doing 745


20/84

Po%u(ar +eneric Test "ork(oads MySQL

ysench

O+'P, %O/%=, 14tale, since v0.8 4tale, lots load options, deadlo #9'2 / 'PCC4lie

O+'P, %=, very compleD, gro-ing d, no options, deadlocs

In fact !sing mostly only 2 tales6 Ethans Performance chema 7455

d'% O+'P, %O/%=, several tales, one most hot, config!rale, no deadlo

linench EFaceoo5

O+'P, %=, very intensive

#9': #=", compleD heavy !ery, loved y OptimiJer 'eam 745


21/84

Test "ork(oad

9efore to do something compleD...

9e s!re first yo!re comfortale -ithasicB operations6

ingle taleL

)any talesL

hort !eriesL

+ong !eriesL

%ememer3 any compleD load ;!strepresents a miD of simple operations..

o, start from as simple as possile..

And then increase compleDity progressively..


22/84

The -nfiniti*e Loo% of ata#ase Tuning...

Ser,erSer,er

S*ora-eS*ora-e

OSOS

B En-)neB En-)ne

A/)ca*)onA/)ca*)on

#1 Mon)*or)n-#0 !(n)n-

# O*)m)2a*)on#3 Imro,emen*45'#6 7...-o*o #1

#1 Mon)*or)n-#0 !(n)n-# O*)m)2a*)on#3 Imro,emen*45'#6 7...-o*o #1


23/84

The -nfiniti*e Loo% of ata#ase Tuning...

Ser,erSer,er

S*ora-eS*ora-e

OSOS

B En-)neB En-)ne

A/)ca*)onA/)ca*)on

#1 Mon)*or)n-#0 !(n)n-

# O*)m)2a*)on#3 Imro,emen*45'#6 7...-o*o #1

#1 Mon)*or)n-#0 !(n)n-# O*)m)2a*)on#3 Imro,emen*45'#6 7...-o*o #1

E,en )f )n

869ca5e5

*he ro:/emI5 here$$$ ;&'

E,en )f )n

869ca5e5*he ro:/emI5 here$$$ ;&'


24/84

MySQL nter%rise Monitor

Fantastic tool6

#id yo! already try itL.. #id yo! see it liveL..


25/84

/ther Monitoring Too(s

Cacti

GaiD agios

tc.........................................................................................

dim_STAT -ell, Im !sing this one, sorry 745

all graphs -ithin presentation -ere made -ith it

details are in the end of presentation..


26/84

MySQL Performance

InnoBInnoB

MySQLMySQL

S*ora-e

BBU< SS

S*ora-eBBU< SSSer,er< OS< FSSer,er< OS< FS

In*erna/ L)m)*5..In*erna/ L)m)*5..

Q(ery O*)m)2a*Q(ery O*)m)2a*)

onf)-(ra*)on Se**)n-onf)-(ra*)on Se**)n-5

A/)ca*)on on*en*)onA/)ca*)on on*en*)on5

!here )5No S)/,erB(//e*$$$

!here )5No S)/,erB(//e*$$$


27/84

Basic Tuning

(nderstanding "= platform limits

helps yo! to deploy yo!r )y*+ erver in the most optimal -ay.. (nderstanding )y*+ erver internals

helps yo! to config!re yo!r dataase settings in the most optimal -a

!se the est adapted torage ngine

(nderstanding of yo!r =orload helps yo! to t!ne the -hole sol!tion in the most optimal -ay 745

20Q of no-n iss!es covering R0Q of oserved prolems..

o, adapt some est practices from the eginning..


28/84

Storage ngines: use -nnoB 0 MySQL 1.2 345

)yIA)

'ale level locing for everything, cacheEs5 for indeDes only asily spending 80Q of time on syscalls to readE5 data..

)any gloal locs, no transactions, no recovery..

Inno#9

%o- locing, transactions for everything, 9!ffer Pool #o!le -rite !fferL Checs!msL

innod


29/84

"ork(oad: 6ead4/n(y oriented

9igger 9!ffer Pool E9P5 is etter

9P S dataset N IO4o!nd '%H list Eernel3 trD E-hile Parser time not more than :Q according to profiler5..

%ead4Only transactions6


30/84

-nnoB: 6ead4/n(y Transactions in 1.2

ysench O+'P


31/84

-nnoB: 6ead4/n(y Transactions in 1.2 7A%r.89;

ysench O+'P


32/84

-nnoB : fa(se sharing of cache4(ine < true ki((er

%O or %= =orloads

ame symptoms in 8.8 ? 8.> 3 no *P improvement et-een 1> an!ser sessions3


33/84

-nnoB : fa(se sharing of cache4(ine fi=ed$

%O or %= =orloads

K8B patch6 345 Over =87$5times etter on ysench O+'P


34/84

MySQL -nterna(s: ki((er L/>&?o%en mute=

)y*+ 8.8 and efore3

$eep tale3

FiDed3 several tale open cache instances

9!t it doesnt mean yo! can !se a small tale


35/84

MySQL 1.2 -nterna(s : (o' ta#(e?o%en?cache

)y*+ 8.> 3

ot ig eno!gh tale


36/84

MySQL 1.2 -nterna(s : (o' ta#(e?o%en?cache 785

)y*+ 8.> 3

ot ig eno!gh tale


37/84

MySQL 1.2 -nterna(s : ta#(e?o%en?cache?instanc

)y*+ 8.> 3

=hen +OC$


38/84

- B i t ( hi


39/84

-nnoB: irty %ages, (ushing

T " $%&

'%(

Buffer oo*+ free

+ ,ata

+ ,irtyinno,ma/,irtyages

inno,ufferoo*sie

inno,*ogfi*esie

#irect dependence on %#O log siJe

O'3 o direct dependence

on amountof dirtypages and %#O siJe6

#epends on -orload6

"o-ever, igger %#Oallo-s more dirty pages..

And recovery is -ayfaster today6

- B 6/ ( t i t


40/84

-nnoB: 6/ (og constraints

T " $%&

Buffer o+ free

+ ,ata

+ ,irty

%#O log constraints3 EAl-ays monitor Checpoint Age6665

Cyclic, need free space

Checpoint age3 diff et-een the c!rrent + in %#Oand the oldest dirty page +

Checpoint age cannot o!t4pass the maD checpointage Eredo log siJe5

If Checpoint age UN @/R of )aD age NU Fl!sh A++ dirty6 NU A$A f!rio!s fl!shingB...

Adaptive Fl!shing3

$eep %#O !nder )aD age

%especting IO capacity limit

- B Ad ti ( hi


41/84

-nnoB: Ada%ti*e (ushing

)y*+ 8.83

stimation ased

ometimes -ors 745

)y*+ 8.> 3

9ased on %#O -rite rate T I/O capacity )aD

Involving atch fl!shing -ith pages to fl!sh Eprogressive, dependion %#O Qfree5 T page age limit Eaccording %#O rate5

'!ning3

innod


42/84

Ada%ti*e (ushing: MySQL 1.2 *s 1.1

O+'P


43/84

-nnoB : 6esisting to acti*ity s%ikes in 1.2

d'% %T= -ith spies

> i


44/84

ser >oncurrency scenarios

ingle !serL..

=ith a igger code path today 8.> simply cannot e faster than 8.8

9!t then, -hy yo!re not considering *!ery CacheL 745

)ore !sersL..

(p to R41> conc!rrent !sers all internal contention are not yet hot

o, 8.> -ill not e etter yet..

)ore than 1> !sersL..

'hen yo!ll feel a real difference, !t if yo! have at least 1>cores 745

Or if yo! have really a lot of conc!rrent !sers

9!t dont forget other 8.> improvements either6 On4line ##+, 9inlog gro!p commit, )emcached, etc..

Cigh >oncurrency Tuning


45/84

Cigh >oncurrency Tuning

If ottlenec is d!e a conc!rrent access on the same data Edapplication design5 M as dev team to re4design 745

If ottlenec is d!e )y*+/Inno#9 internal contentions, the

If yo! cannot avoid it, then at least dont let them gro- 745

'ry to increase Inno#9 spin -ait delay Edynamic5

'ry innod


46/84

-nnoB S%in "ait e(ay

%O/%= =orloads3

=ith more CP( cores internal contentions ecome more hot..

9ind mysld to less cores helps, !t the goal is to !se more cores 74

(sing innod


47/84

Tune -nnoB S%in "ait e(ay

otes 3

is the maD random delay on sleepB -ithin a spin loop in -ait for loc

Ideally sho!ld e a!to.. -hile the same t!ning -ors for 8.8 as -ell

Keneral r!le3 defa!lt is >, may need an increase -ith more cores

'est3 :24"'/ :2/ 2/ 1>cores, spin delay N > / V> 3

Thread Poo( MySQL


48/84

Thread Poo( MySQL

one of these sol!tions -ill help to increase performance6

itll ;!st help to eep the pea level constant Eand yo! yet need todiscover on -hich level of conc!rrency yo!re reaching yo!r pea 745

'hreadPool in )y*+ 8.8 and 8.> is a-are if I/O are involve

o, etter than innod thread conc!rrency setting or tasset

)ay still re!ire spin -ait delay t!ning6

'he m!st for high conc!rrency loads6

)ay still start to sho- a difference since :2412R conc!rrent !sers6 Eadepends on -orload5..

$eep in mind that O sched!ler is not a-are ho- to manage !serthreads most optimally, !t 'hreadPool does 745

Thread Poo( in MySQL 1 2


49/84

Thread Poo( in MySQL 1.2

O+'P


50/84

Thread Poo( in MySQL 1.2

O+'P


51/84

Thread Poo( in MySQL 1.D Cea*y /LTP?6"

-nnoB Cigh >oncurrency: AC-


52/84

-nnoB Cigh >oncurrency: AC-

Adaptive "ash IndeD EA"I5

"elps a lot on %ead4Only -orloads

In fact it helps al-ays !ntil itself ecome not actively modified

A"I contention is seen as its tr


53/84

-nnoB Purge

P!rging Esimilar to Karage Collecting5

ince )y*+ 8.83 p!rge thread

ince )y*+ 8.>3 p!rge threadEs5 E!p to :25

"aving P!rge follo-ing -orload is very important6

D.3 On aggressive %= got 00K9 of !ndo records -ithin fe- ho!rs

'hen it too days to reach Jero in "istory +ength..

'he main prolem is the past M ho- to dose itL..

ince 8.>3 -ith many threads, P!rge ecome a!to4stale itself

till missing a dynamic config option to say ho- many p!rge threadsr!n in parallel right no- E!t itll e fiDed soon 7455

-nnoB : Purge im%ro*ement in 1 2


54/84

-nnoB : Purge im%ro*ement in 1.2

everal P!rge 'hreads 3

O'3 activation is a!to4magical EIm serio!s 7455

-nnoB : Purge im%ro*ement in 1 2


55/84

-nnoB : Purge im%ro*ement in 1.2

FiDed maD p!rge lag code6

innod


56/84

Testing A%%(es4to4A%%(es...

Comparing )y*+ 8.> vs 8.8 3

#ont have K83 dead..

#ont have open tale cache instances3 ad..

#ont have improved Adaptive Fl!shing7 ad..

#ont have fiDed P!rge ? +ag3 danger6..

#ont have inlog gro!p commit and !se inlog3 dead..

tc. etc. etc.

O'3 some improvementB are also fiDes -hich are maing st!ff-oring properly, !t coming -ith additional overhead Elie P!rge5..

O'3 -hen comparing 8.> and 8.8 eep in mind that Performancechema is enaled y defa!lt in 8.>, and not in 8.8, so thin to disain oth Eas 8.8 also has a -ay less PF instr!mentation5..

Co%e you did not miss 345


57/84

Co%e you did not miss 345


58/84

Co%e you did not miss 345 7;5


59/84

Co%e you did not miss 345 7;5

Sys#ench /LTP 6/ Ecores4CT 7A%r 89;5


60/84

Sys#ench /LTP?6/ Ecores CT 7A%r.89;5


61/84


62/84


63/84

Sys#ench /LTP 6/ E4ta# ;8cores4CT 7A%r.89


64/84

Sys#ench /LTP?6/ E ta# ;8cores CT 7A%r.89

Sys#ench /LTP 6/4tr= E4ta# ;8cores4CT 7A%r.


65/84

Sys#ench /LTP?6/ tr= E ta# ;8cores CT 7A%r.

Sys#ench /LTP 6" Ecores4CT 7A%r.89;5


66/84

Sys#ench /LTP?6" Ecores CT 7A%r.89;5

Sys#ench /LTP 6" 2cores4CT 7A%r.89;5


67/84

Sys#ench /LTP?6" 2cores CT 7A%r.89;5

Sys#ench /LTP 6" ;8cores4CT 7A%r.89;5


68/84

Sys#e c / ? ; co es 7 % 9 ;5

Sys#ench /LTP 6" E4ta# ;8cores4CT 7A%r.89


69/84

y ? 7 %

Things are changing constant(y, stay tuned 345


70/84

g g g y, y 3 5

)y*+/Inno#9 calaility3

200@ 3 !p to 2CP(...

200R 3 !p to CP( cores

200V 3 !p to 1>CP( cores ET!n5

2010 3 !p to :2CP( cores ETOracle5

2012 3 !p to RCP( cores..

201 3 WLL 745

O'3 on the same "= performance is etter from version to versio

Inno#9 today3

At least D4R times etter performance than 24: years ago 745

Capale of over 100$ :00$ 199&*PE65 T F' ? )emcached

MySQL 1.2: Pending issues


71/84

y g

IndeD loc..

+oc


72/84

y % g 3 5

IndeD loc.. SNN fiDed6

+oc


73/84

y 7 % 5

O+'P


74/84

y 7 % 5

O+'P


75/84

y 7 % 5

O+'P


76/84

y 7 % 5

O+'P


77/84

y 7 5

O+'P


78/84

O+'P


79/84

O+'P


80/84

O+'P...

#!e higher )#+ contentions, -or in progress..

MySQL 1.D: M68 7Se%.89;5


81/84

O+'P


82/84

O+'P

)#+ contention, -or in progress..

e' 'ords a#out dim_STAT7as youG(( ask 3455


83/84

All graphs are !ilt -ith dim_STATEhttp3//dimitri.free.fr5

All ystem load stats ECP(, I/O, et-or, %A), Processes,...5

)anly for olaris ? +in!D, !t any other (IH too 345

Add4Ons for Oracle, )y*+, Postgre*+, Xava, etc.

)y*+ Add4Ons3

mysl'A' 3 all availale data from sho- stat!sB

mysl+OA# 3 compact data, m!lti4host monitoring oriented

mysl=AI' 3 top -ait events from Performance C")A

Innod'A' 3 most important data from sho- innod stat!sB

innod)('H 3 monitoring Inno#9 m!teD -aits

innod)'%IC 3 all co!nters from the )'%IC tale

And any other yo! -ant to add6 345

TCAH& I/ $$$


84/84

TCAH& I/ $$$

All details ao!t presented materials yo! may find on3

http3//dimitri.free.fr4 dim

bench tuning bestpractices

Documents