aix vios diskandadapterqueuetuningv1.2

8/11/2019 AIX VIOS DiskAndAdapterQueueTuningV1.2

1/46

IBM Americas Advanced Technical Skills

AIX/VIOS Disk and Adapter IO Queue TuningAIX/VIOS Disk and Adapter IO Queue Tuning

Dan BradenDan Braden

IBM AIX Advanced Technical SkillsIBM AIX Advanced Technical Skills

Version: 1.2Version: 1.2

Date: July ! 2"1#Date: July ! 2"1#

$ 2"1# International Business Machines! Inc.


2/46


A%&'()*+D,+M+'TS.............................................................................................................................-

D ISCLAIMERS............................................................................................................................................5T RADEMARKS............................................................................................................................................5F EEDBACK.................................................................................................................................................5

V ERSIO

!"DATES

....................................................................................................................................5I'T(D/%TI('.............................................................................................................................................0

1.T+ AIX I( STA%& A'D /+/+S........................................................................................................3

#.#.T$EAI% IO STACK................................................................................................................................'.$O(IO )!E!ES(ORKADASSOCIATEDATTRIB!TES..........................................................................*#.+.C$A,I,DEVICEATTRIB!TEVAL!ES................................................................................................###.-.M!LTI"AT$IO CODEAD)!E!ET!I,..........................................................................................#'

#.-.#.SDD and SDD"CM.....................................................................................................................#'

2.T((*S T( M('IT( T+ /+/+S..................................................................................................1#

'.#.T$EIOSTATCOMMAD.........................................................................................................................#-'.'.T$ESARCOMMAD..............................................................................................................................#5'.+.T$EFCSTATCOMMAD.........................................................................................................................#5'.-.T$EITERACTIVETO"ASCOMMAD.....................................................................................................#&'.5.T$EITERACTIVEMOCOMMAD......................................................................................................#*'./.MO RECORDI,S............................................................................................................................#0'.&.T$ESDD"CM "CM"AT$COMMAD....................................................................................................'''.*.T$ESDD DATA"AT$COMMAD...........................................................................................................'+

4.T/'I', T+ /+/+S.............................................................................................................................2#

+.# C$ECKFORSA1STORA,E"ROBLEMSFIRST........................................................................................'-+.'$O(TOT!ET$E)!E!ES...................................................................................................................'-

+.'.#.T2nin3 hdisk 42e2ede67h...........................................................................................................'5+.'.'.T2nin3 8cs Device )2e2e A77ri927es...........................................................................................'&

+.+.AFTER:O!RIITIALT!I,...............................................................................................................'0+.-.T!I,ORDER.....................................................................................................................................'0+.5.($ATARE,OOD; REASOABLEAD"OORIO SERVICETIMESESIVIO EVIROMETS......................................................................................++

+./.#.Av=idin3 VIOS O27a3es..............................................................................................................+5+./.'.T?= S7ra7e3ies 7= Limi7 In8li3h7 IOs 7= 7he S7=ra3e...................................................................+/+./.+.T2nin3 vSCSI )2e2es.................................................................................................................+&+./.-.T2nin3 "IV )2e2es..................................................................................................................+*+./.5.T2nin3 Shared S7=ra3e "==l )2e2es...........................................................................................+0

+./.T$EORETICALT$O!,$TSOS$AREDVS. DEDICATEDRESO!RCES.......................................................-@

#.+STIMATI', A55*I%ATI(' 5+6(MA'%+ IM5(V+M+'T.............................................##

A55+'DIX: +*AT+D 5/B*I%ATI('S..............................................................................................#-

'@#- In7erna7i=nal B2siness Machines; Inc.

"a3e '


3/46


Ta7le o8 6i9ures:

6i9ure 1 AIX I( stack and 7asic tuna7les...................................................................................................06i9ure 2 to;as D sa6i9ure 12 Sa


4/46


Ackno@led9e


5/46


Versi=n #.# J re=r3anied and 26da7ed ?i7h ne? ma7erial

Versi=n #.' J added in8=rma7i=n a9=27 limi7 =n "IV vir72al ada67ers 8=r n2mcmdelems


"a3e 5


6/46


Introduction

This 6a6er is in7ended 8=r IBM "=?er Ss7ems c2s7=mers; 2sin3 AI%; IBM Technical Sales S6ecialis7sand c=ns2l7an7s ?h= are in7eres7ed in learnin3 m=re a9=27 72nin3 IO 42e2es 8=r =67imal 6er8=rmance; and

h=? 7= d= i7. I8 =2r a66lica7i=n has a disk IO 9=77leneck; 7his 6a6er ?ill hel6 =2 eval2a7e 7he 6r=9lemand 6=7en7iall im6r=ve 6er8=rmance.

The 6a6er e6lains h=? IO 42e2in3 ?=rks; and e6lains h=? 7= 72ne 7he 42e2es 7= im6r=ve 6er8=rmance;incl2din3 in VIO envir=nmen7s. This ?ill hel6 ens2re =2 d=n7 have 2nnecessar IO 9=77lenecks a7 7hese42e2es. Thanks 7= M==res la?; disk IO is 3e77in3 rela7ivel sl=?er c=m6ared 7= 6r=cess=rs and mem=r;and 9ec=min3 m=re and m=re a 9=77leneck 7= 6er8=rmance. Red2cin3 IO la7enc 8r=m 7he a66lica7i=ns6=in7 =8 vie? im6r=ves 6er8=rmance; and 72nin3 7hese 42e2es is im6=r7an7 8=r hi3h IO"S 7hr2627. Thisd=c2men7 eamines 72nin3 7he 42e2es 8=r 7he hdisk driver and ada67er drivers; incl2din3 in VIO envir=nmen7s. I7 d=esn7 eamine IO 72nin3 8r=m 7he a66lica7i=n 7= 7he hdisk driver.

This 6a6er c=n7ains 9es7 6rac7ices ?hich have 9een c=llec7ed d2rin3 7he e7ensive 6eri=d =8 7ime 7eam c=llea32es and I have s6en7 ?=rkin3 in 7he AI% envir=nmen7. I7 is 8=c2sed =n AI% versi=ns 5.+; /.# and &.#.


"a3e /


7/46


1. The AIX IO Stack and Queues

1.1. The AIX IO Stack

F=ll=?in3 is 7he IO s7ack 8r=m 7he a66lica7i=n 7= 7he disk

6i9ure 1 AIX I( stack and 7asic tuna7les

This 6a6er is c=ncerned ?i7h 72nin3 7he AI% disk driver and ada67er driver 42e2e sies. =7e 7ha7 even7h=23h 7he disk is a77ached 7= 7he ada67er; 7he hdisk driver c=de is 27ilied 9e8=re 7he ada67er driver c=de.S= 7his s7ack re6resen7s 7he =rder s=87?are and hard?are c=me in7= 6la =ver 7ime as 7he IO 7raverses 7he

s7ack. The disk s29ss7em is 76icall SA a77ached; and 7he disk s29ss7em ?ill have i7s =?n in7ernal IOs7ack. =7e 7ha7 7his IO s7ack als= eis7s in 7he VIOS; 7h=23h 7heres n= a66lica7i=n; =7her 7han 7he VIOS82nc7i=n =8 vir72aliin3 IOs 8r=m VIOCs and sharin3 IO ada67ers. The 8ile ss7em laers and VMM als=aren7 2sed 8=r VIO vir72alied IO ece67 8=r 8ile 9acked hdisks ?hich aren7 c=vered in 7his 6a6er.


"a3e &


8/46


1.2. Ho IO !"e"es #ork and Associated Attrib"tes

AI%s disk and ada67er drivers each 2se a 42e2e 7= handle IO; s6li7 in7= an inservice 42e2e; and a ?ai742e2e. IO re42es7s in 7he inservice 42e2e are sen7 7= 7he s7=ra3e; and 7he 42e2e sl=7 is 8reed ?hen 7he IOis c=m6le7e GAI% 3e7s ei7her 7he da7a 8=r a read; =r 7he ackn=?led3emen7 8=r a ?ri7e; and re72rns 7he res2l7

9ack 26 7he s7ackH. IO re42es7s in 7he ?ai7 42e2e s7a 7here 2n7il an inservice 42e2e sl=7 is 8ree; a7 ?hich7ime 7he are m=ved 7= 7he inservice 42e2e and sen7 7= 7he s7=ra3e. IO re42es7s in 7he inservice 42e2eare als= called in8li3h7 8r=m 7he 6ers6ec7ive =8 7he device driver.

The sie =8 7he hdisk driver inservice 42e2e is s6eci8ied 9 7he 42e2ede67h a77ri927e; ?hile 7he sie =87he ada67er driver inservice 42e2e is s6eci8ied 9 7he n2mcmdelems a77ri927e. And 3enerall in 7his 6a6er 7he inservice 42e2e sie is als= called 7he 42e2e sie; 7he 42e2e de67h; =r n2m9er =8 42e2e sl=7s.This 6a6er ?ill als= re8er 7= 7he 7radi7i=nall 2sed ada67er driverN since ada67ers ini7iall had =ne 6=r7;7h=23h n=? =87en have m2l7i6le 6=r7s ?hen i7 ac72all re8ers 7= 7he 8cs device =r s6eci8icall an ada67er6=r7s driver.

$eres h=? 7= sh=? 7he 8cs ada67er 6=r7 and hdisk a77ri927es

root # lsattr -EHl fcs0attribute value description user_settable

intr_priority 3 Interrupt priority Falselg_term_dma 0x800000 ong term !" $ruemax_xfer_si%e 0x&00000 "aximum $ransfer 'i%e $ruenum_cmd_elems (00 "aximum )umber of *+"")! Elements $rues,_fc_class ( F* *lass for Fabric $rue

root # lsattr -EHl dis.0attribute value description user_settable

/*" /*"friendvscsi /at *ontrol "odule Falsealgoritm fail_over lgoritm $ruecec._cmd test_unit_rdy Healt *ec. *ommand $ruecec._interval 10 Healt *ec. Interval $ruecec._mode enabled Healt *ec. "ode $ruemax_transfer 0x20000 "aximum $)'FE 'i%e $ruepvid 00c2c1c4b35f(6440000000000000000 /ysical volume identifier False7ueue_dept 3 ueue !E/$H $ruereserve_policy no_reserve eserve /olicy $rue

A77ri927es in 9l2e are =8 in7eres7 re3ardin3 72nin3 7he 42e2es. B=7h 6hsical and vir72al disk drives 76i call s266=r7 c=mmand 7a33ed 42e2in3; ?hich all=?s 2s 7= have m2l7i6le in8li3h7 IO re42es7s 7= a sin3le

hdisk. This im6r=ves 6er8=rmance in several ?as.

A 6hsical disk can =nl d= =ne IO a7 a 7ime; 927 kn=?in3 several =8 7he IO re42es7s all=?s 7he disk 7= d=7he IOs 2sin3 an eleva7=r al3=ri7hm 7= minimie ac72a7=r m=vemen7 and la7enc. Vir72al disks 76icallare 9acked 9 man 6hsical disks; s= can d= man IOs in 6arallel. The eleva7=r anal=3 is a66r=6ria7e8=r individ2al 6hsical disk drives; ?here 6assen3ers are IO re42es7s; 7he eleva7=r is 7he disk arm; and 7he8l==rs are 7he disk 7racks. I8 7he eleva7=r ?en7 7= 7he 8l==rs in 7he =rder 7he 9277=ns ?ere 62shed G=r 7he =r


"a3e *


9/46


der =8 IO re42es7sH; ra7her 7han 7he ?a 7he n=rmall ?=rk; 7he avera3e service 7ime 8=r each 6assen3er?=2ld 9e l=n3er. Th2s; s29mi77in3 m2l7i6le in8li3h7 IOs 7= a disk s29ss7em all=?s i7 7= 8i32re =27 h=?7= 3e7 7he m=s7 7hr2627 and 8as7es7 avera3e IO service 7ime. Increasin3 7he 42e2e sie is like increasin3 7he6assen3er ca6aci7 =8 7he eleva7=r.

The=re7icall; 7he maim2m IO"S =ne mi3h7 achieve 8=r a L! is

Maim2m L! IO"S 42e2ede67h1 Gav3. IO service 7imeH

Th2s; increasin3 42e2ede67h; and c=nse42en7l 7he n2m9er =8 in8li3h7 IOs; increases 6=7en7ial IO"S and7hr2627. Similarl

Maim2m ada67er IO"S n2mcmdelems1 Gav3. IO service 7imeH

The maim2m in8li3h7 IOs a ss7em ?ill s29mi7 7= SA s7=ra3e is 7he smalles7 =8 7he 8=ll=?in3

The s2m =8 7he hdisk 42e2ede67hs The s2m =8 7he ada67er n2mcmdelems

The maim2m n2m9er =8 in8li3h7 IOs s29mi77ed 9 =2r a66lica7i=nGsH

As IOs 7raverse 7he IO s7ack; AI% needs 7= kee6 7rack =8 7hem a7 each laer. S= IOs are essen7iall 42e2eda7 each laer; and 2sin3 42e2e sl=7s =r 9288ers 7= kee6 7rack =8 7hem. ,enerall; s=me n2m9er =8 in8li3h7IOs ma 9e iss2ed a7 each laer and i8 7he n2m9er =8 IO re42es7s eceeds 7ha7 n2m9er; 7he reside in a?ai7 42e2e 2n7il 7he re42ired res=2rce 9ec=me availa9le. S= 7here is essen7iall an inservice 42e2e and a?ai7 42e2e a7 each laer; ?i7h 7he sie =8 7he inservice 42e2e limi7in3 7he n2m9er =8 in8li3h7 IOs.

(hile 7his 6a6er d=esn7 eamine IO 72nin3 7he 8r=m 7he hdisk driver 7= 7he a66lica7i=n; =ne can als= d=

IO 72nin3 a7 7he 8ile ss7em laer; as 8ile ss7em 9288ers limi7 7he maim2m n2m9er =8 in8li3h7 IOs 8=reach 8ile ss7em. Als= a7 7he LVM device driver laer; hdisk 9288ers; 6928s; limi7 7he n2m9er =8 in8li3h7IOs 8r=m 7ha7 laer. The hdisks have a maim2m n2m9er =8 in8li3h7 IOs 7ha7Ps s6eci8ied 9 i7s42e2ede67h a77ri927e. And FC ada67ers als= have a maim2m n2m9er =8 in8li3h7 IOs s6eci8ied 9n2mcmdelems. The disk s29ss7ems 7hemselves 42e2e IOs and individ2al 6hsical disks can acce67m2l7i6le IO re42es7s 927 =nl service =ne a7 a 7ime. There are man 42e2es ?i7hin 7he =6era7in3 ss7emand 7he hard?are; incl2din3 ?i7h TC"1I"; mem=r; and even 7he 42e2e =8 c=mmands 7= 2se 7he 6r=cess=rs.

$ere is =ne 76e =8 hdiskPs de8a2l7 a77ri927es Ga77ri927es var acr=ss di88eren7 disk s29ss7emsH

# lsattr -El dis.33/_.ey_value none eserve 9ey $ruelocation ocation abel $ruelun_id 0x55&5000000000000 ogical :nit )umber I! $ruelun_reset_spt yes 'upport '*'I :) reset $ruemax_transfer 0x20000 ) $ruenode_name 0x5005041300c061ab F* )ode )ame Falsepvid none /ysical volume identifier False7_type simple ueuing $;/E $rue7full_dly (0 delay in seconds for '*'I $'9 'E$ F: $rue


"a3e 0


10/46


7ueue_dept (0 ueue !E/$H $ruereserve_policy single_pat eserve /olicy $ruer,_timeout 10 E!(51 ?@&A

This indica7es 7he val2e can 9e an?here 8r=m # 7= '5/ in incremen7s =8 #. In 3eneral 8=r device a77ri927es7ha7 have a 2serse77a9le val2e =8 Tr2e; =2 can dis6la all=?a9le val2es 2sin3

# lsattr -l BdeviceC -a BattributeC

$erePs a FC ada67erPs a77ri927es

# lsattr -El fcs0bus_intr_lvl 15403 =us interrupt level Falsebus_io_addr 0xdec00 =us I+ address Falsebus_mem_addr 0xe8020000 =us memory address Falseinit_lin. al I)I$ in. flags $rueintr_priority 3 Interrupt priority Falselg_term_dma 0x800000 ong term !" $ruemax_xfer_si%e 0x&00000 "aximum $ransfer 'i%e $ruenum_cmd_elems (00 "aximum number of *+"")!' to 7ueue to te adapter $rue

pref_alpa 0x& /referred _/ $rues,_fc_class ( F* *lass for Fabric $rue

C2rren7l de8a2l7 42e2e sies Gn2mcmdelemsH 8=r FC ada67ers ran3e 8r=m '@@ 7= 5@@; ?i7h maim2mval2es =8 '@-* =r -@0/.

The FC ada67er als= direc7l accesses a c=n8i32ra9le am=2n7 =8 ss7em mem=r 7= h=ld in8li3h7 IO da7a;?hich als= ac7s as a 42e2e since i8 7here isn7 s288icien7 8ree mem=r 7= h=ld 7he da7a; 7hen 7he IO ?ill 9e9l=cked 2n7il 7here is. This DMA mem=r area is c=n7r=lled 9 7he ma8ersie a77ri927e; and c=n7r=ls7?= se6ara7e 7hin3sQ 7h2s; is a 9i7 c=m6le. This a77ri927es val2e s6eci8ies 7he ma IO sie 7ha7 7he ada67er

?ill send 7= 7he disk s29ss7em G6r=vided 7he L!s ma7rans8er a77ri927e isn7 smallerH. The sie =8 7heDMA mem=r area is n=7 e42al 7= 7he ma8ersie val2e. B de8a2l7 8=r 7he * ,9 d2al 6=r7 ada67er;?i7h 7he de8a2l7 ma8ersie =8 @-@@@@'5/ KB; 7he DMA mem=r area sie is #/ MB. !sin3 an=7her all=?a9le val2e 8=r ma8ersie increases 7he mem=r area 7= #'* MB. T= 3e7 7he 82ll 9and?id7h=8 7he ada67er; =ne needs 7he DMA mem=r area =8 #'* MB; 7h=23h 8=r man c2s7=mers; 7he de8a2l7 iss288icien7. Chan3in3 7his val2e isn7 al?as 6=ssi9le; de6ends =n 7he ss7em hard?are; and has s=merisks disc2ssed in 7he ne7 sec7i=n.


"a3e #@


11/46


Since 7he DMA mem=r area sies s266=r7ed varies acr=ss ada67ers; =2 can 2se

# fcstat D! Bfcs#C

T= see 7he val2e =8 7he DMA mem=r sie =8 =2r ada67er; i8 =2 have a recen7 versi=n =8 AI% =r =8 7heVIOS OS. This all=?s =2 7= 7r 7he al7erna7ive ma8ersie se77in3s; and see i8 7he DMA 6==l siechan3es. Sam6le =27627 is sh=?n in sec7i=n '.+ =n 8cs7a7.

1.$. %han&in& De'ice Attrib"te Val"es

In 3eneral =ne chan3es device a77ri927e val2es 2sin3 7he chdev c=mmand

# cdev Dl BdeviceC -a BattributeCBne, valueC

$=?ever; 7= chan3e 7hese a77ri927es 8=r hdisk and 8cs devices; 7he m2s7 n=7 9e in 2se. Th2s; 76icall7hese val2es are chan3ed d2rin3 se726 =r 6lanned main7enance. One can make 7he chan3e in 7he ODM via7he J" 8la3; s= 7ha7 7he chan3e 3=es in7= e88ec7 ?hen 7he ss7em is re9==7ed =r 7he device is rec=n8i32red

# cdev Dl BdeviceC -a BattributeCBne, valueC -/

Be a?are 7ha7 a87er makin3 7he chan3e 7= 7he a77ri927e in 7he ODM; 7he lsa77r c=mmand dis6las 7he val2ein 7he ODM; n=7 7he val2e 9ein3 2sed 9 7he kernel. :=2ll need 7= kee6 7rack =8 ?ha7s ac72all in 2se 97he kernel 8=r 72nin3.

Chan3in3 hdisk a77ri927es is 6=ssi9le 6r=vided 7he aren7 in a varied =n V,; =r =6ened and accessed via=7her a66lica7i=ns.

Ada67er a77ri927es can 9e chan3ed 6r=vided n= disks are 2sin3 7he ada67er. Ass2min3 =ne has m2l7i6le

6a7hs 7= all disks via red2ndan7 ada67ers and =2re 2sin3 m2l7i6a7h c=de c=m6lian7 ?i7h 7he AI% "CM;7hen =2 can dnamicall chan3e 7he ada67er a77ri927es =2 s7=6 2sin3 =ne ada67er 9 6277in3 all disk6a7hs 2sin3 7he ada67er in7= 7he De8ined s7a7e; make chan3es 7= 7he ada67er a77ri927es; rec=n8i32re 7heada67er; 7hen rec=n8i32re 7he 6a7hs 9ack in7= 7he Availa9le and ena9led s7a7e via 7he c83m3r c=mmand.This can 9e achieved via

# rmdev Dl Bfcs#C# cdev Dl Bfcs#C -a BattributeCBne, valueC# cfgmgr Dl Bfcs#C

D2e 7= limi7ed mem=r =n "CI $=s7 Brid3e G"$BH chi6s; i7s n=7 al?as 6=ssi9le 7= increase

ma8ersie 8=r all FC ada67ers. This is less =8 an iss2e =n ne?er hard?are; and =n =lder ss7ems?here 8e?er ada67ers share "$Bs; s= i7 is c=n8i32ra7i=n de6enden7. The =nl ?a 7= 8ind =27 i8 =2 canchan3e 7he val2e 8=r each ada67er is 7= 7r chan3in3 i7. $=?ever; 9e a?are 7ha7 i8 =2re 9==7in3 8r=mSA and =2r c=n8i32ra7i=n d=esn7 have s288icien7 "$B mem=r; 7his can ca2se 9==7 8ail2re. Lack =8"$B mem=r res2l7s in hdisk devices c=n8i32rin3 in7= a De8ined; ra7her 7han Availa9le s7a7e; and i8 =2r9==7 disk is =ne =8 7hem; 9==7 ?ill 8ail. Rec=ver inv=lves 3=in3 in7= SMS and chan3in3 7he val2e 9ack7= 7he de8a2l7; =r res7=rin3 8r=m a mkss9.


"a3e ##


12/46


Ass2min3 =2 can 9==7; 8ail2re 7= c=n8i32re hdisks =r 6a7hs in7= 7he Availa9le s7a7e ?ill lead 7= err=rs in7he err=r l=3 7ha7 l==k like

=E !"_E I!E)$IFIE 00530E1 G

*lass H $ype :)9) esource )ame /*I!" >>> !escription :)!E$E"I)E! E+

/robable *auses ';'$E" I+ =:' '+F$


13/46


SDD adds a laer =8 42e2in3; ?hile SDD"CM 2s7 ch==ses a 6a7h 8=r each IO.

SDD has a v6a7h device 8=r each l=3ical disk 6l2s an hdisk device 8=r each 6a7h 7= i7; ?hile ?i7h SDD"CM ?e 2s7 have an hdisk ?i7h 6a7hs 7= i7 lis7ed ?i7h 7he ls6a7h c=mmand. There is als= a d6= device

8=r SDD. $erePs 7he d6= devicePs a77ri927es 8=r =ne release =8 SDD

# lsattr -El dpoEnterpr_maxlun 100 "aximum :)' allo,ed for Enterprise /roducts $rueirtual_maxlun 5&( "aximum :)' allo,ed for irtuali%ation /roducts Falsepersistent_resv yes 'ubsystem 'upports /ersistent eserve *ommand False7dept_enable yes ueue !ept *ontrol $rue

(hen 4de67hena9lees; SDD ?ill =nl s29mi7 42e2ede67h IOs 7= an 2nderlin3 hdisk G?here42e2ede67h here is 7he val2e 8=r 7he 2nderlin3 hdiskPs 42e2ede67h a77ri927eH. (hen 4de67hena9len=;SDD 2s7 6asses =n 7he IOs direc7l 7= 7he hdisk driver. S= 7he di88erence is; i8 4de67hena9lees G7he de8a2l7H; IOs eceedin3 7he 42e2ede67h ?ill 42e2e a7 SDD; and i8 4de67hena9len=; 7hen IOs eceed 7he

42e2ede67h ?ill 42e2e in 7he hdiskPs ?ai7 42e2e. In =7her ?=rds; SDD ?i7h 4de67hena9len= and SDD"CM d= n=7 42e2e IOs and ins7ead 2s7 6ass 7hem 7= 7he hdisk drivers. =7e 7ha7 a7 SDD #./; i7Ps 6re8era9le 7= 2se 7he da7a6a7h c=mmand 7= chan3e 4de67hena9le; ra7her 7han 2sin3 chdev; as 7hen i7Ps a dnamicchan3e; e.3.; da7a6a7h se7 4de67h disa9le ?ill se7 i7 7= n=. S=me releases =8 SDD d=nP7 incl2de SDD 42e2in3; and s=me d=; and s=me releases d=nP7 sh=? 7he 4de67hena9le a77ri927e. Ei7her check 7he man2al 8=r=2r versi=n =8 SDD =r 7r 7he da7a6a7h c=mmand 7= see i8 i7 s266=r7s 72rnin3 7his 8ea72re =88.

(i7h SDD =ne can s29mi7 42e2ede67h 6a7hs 7= a L!; ?hile ?i7h SDD"CM; =ne can =nl s29mi742e2ede67h IOs 7= 7he L!. Th2s; i8 =2 s?i7ch 8r=m SDD 2sin3 - 6a7hs 7= SDD"CM; 7hen =2Pd ?an77= se7 7he SDD"CM =r AI% "CM hdisks 42e2ede67h 7= - 7ha7 =8 SDD hdisks 8=r an e42ivalen7 e88ec7ive 42e2e de67h.


"a3e #+


14/46


2. Tools to Monitor the Queues

Basic c=mmands 7= m=ni7=r 7he 42e2es incl2de

(hen 2sin3 7he AI% "CM =r m2l7i6a7h c=de c=m6lian7 ?i7h 7he M"IO archi7ec72reo i=s7a7 8=r hdisk driver 42e2es

o sar 8=r hdisk driver 42e2es

o 8cs7a7 8=r ada67er driver 42e2es

o 7=6as

o nm=n Gin7erac7ive =r via MO rec=rdin3sH

(hen 2sin3 SDD"CMo 6cm6a7h

(hen 2sin3 SDDo da7a6a7h

=7e =2 ?ill s7ill 2se i=s7a7; sar; 8cs7a7; 7=6as and nm=n ?i7h SDD and SDD"CM.

2..1 The iostat %ommand

F=r AI% i=s7a7 is 7he 9asic 7==l 7= m=ni7=r 7he hdisk driver 42e2es. The i=s7a7 D c=mmand 3enera7es =27627 s2ch as

dis.1 xfer Jtm_act bps tps bread b,rtn2>4 (>(" &6>0 0>0 (>("

read rps avgservminserv maxserv timeouts fails 0>0 0>0 0>0 0>0 0 0,rite ,ps avgservminserv maxserv timeouts fails &6>0 38>6 &>& &60>( 0 07ueue avgtimemintime maxtime avg,7s% avgs7s% s7full &5>0 0>0 83>4 0>0 0>0 &31

av3?4s avera3e ?ai7 42e2e sieav3s4s avera3e service 42e2e sieav37ime avera3e 7ime s6en7 in 7he ?ai7 42e2e in mss482ll J ra7e =8 IOs s29mi77ed 7= a 82ll 42e2e 6er sec=nd

The s482ll val2e has chan3ed 8r=m ini7iall 9ein3 a c=2n7 =8 7he 7imes ?ePve s29mi77ed an IO 7= a 82ll42e2e; 7= n=? ?here i7Ps 7he ra7e =8 IOs 6er sec=nd s29mi77ed 7= a 82ll 42e2e. The eam6le re6=r7 sh=?s7he 6ri=r case Ga c=2n7 =8 IOs s29mi77ed 7= a 82ll 42e2eH; ?hile ne?er releases 76icall sh=? l=?er val2esand decimal 8rac7i=ns indica7in3 a ra7e.

I7Ps nice 7ha7 i=s7a7 D se6ara7es reads and ?ri7es; as ?e ?=2ld e6ec7 7he IO service 7imes 7= 9e di88eren7?hen ?e have a disk s29ss7em ?i7h cache. This hel6s 2s eval2a7e 7he IO service 7imes m=re acc2ra7el.The m=s7 2se82l re6=r7 8=r 72nin3 is 2s7 r2nnin3 i=s7a7 Dl ?hich sh=?s s7a7is7ics since ss7em 9==7; as


"a3e #-


15/46


s2min3 7he ss7em is c=n8i32red 7= c=n7in2=2sl main7ain disk IO his7=r Gr2n lsa77r El ss@; =r smi77 ch3ss 7= see i8 7he i=s7a7 a77ri927e is se7 7= 7r2eH. The a27h=rPs 6re8erred i=s7a7 c=mmand 8la3s are

# iostat -!$l K BintervalC K B#intervalsC LL

This lis7s 7he da7a in a l=n3 8=rma7 G=ne line 6er hdiskH. The R 8la3 rese7s minim2m1maim2m val2es 8=reach in7erval; 7he JT adds a 7ime s7am6; and 3enera7es =27627 7ha7 is 7== ?ide 7= dis6la here in 7his 8=n7;927 i7 c=n7ains 7he da7a a9=ve 8r=m i=s7a7 JD =n =ne line 8=r each hdisk. $eres sam6le =27627

# iostat -!$l

'ystem configuration lcpu&( drives3 pats8 vdis.s2

!is.s xfers read ,rite 7ueue time-------------- -------------------------------- ------------------------------------ ------------------------------------ -------------------------------------- --------- Jtm bps tps bread b,rtn rps avg min max time fail ,ps avg min max time fail avg min max avg avg serv act serv serv serv outs serv serv serv outs time time time ,7s% s7s% 7fulldis.0 0>5 2>19 0>4 &>(9 3>29 0>& &>( 0>& 30>0 0 0 0>1 8>3 &>( 16>2 0 0 &8>& 0>0 &2(>2 0>0 0>0 0>( 01(052dis.& 0>0 (5>09 (>6 (5>09 5(>& (>6 0>2 0>& (&8>6 0 0 0>0 5>( 0>8 (28>2 0 0 0>0 0>0 0>0 0>0 0>0 0>0 01(052dis.( 0>0 3>09 0>2 3>09 0>0 0>2 0>( 0>& &62>& 0 0 0>0 0>0 0>0 0>0 0 0 0>& 0>0 &62>& 0>0 0>0 0>2 01(052

Me7rics in 9l2e are 7he main =nes eamined 8=r 72nin3 7he 42e2es.

2..2 The sar %ommand

The sar d c=mmand 3enera7es =27627 s2ch as

&15056 device Jbusy av7ue r@,s 9bss av,ait avserv&15&00 dis.& 0 0>0 0 0 0>0 0>0 dis.0 0 0>0 0 0 0>0 0>0

av?ai7 J avera3e 7ime s6en7 in 7he ?ai7 42e2e in msavserv J avera3e IO service 7ime in ms

av42e J avera3e n2m9er =8 IOs in 7he ?ai7 42e2e

This c=mmand d=esn7 dis7in32ish 9e7?een read and ?ri7e service 7imes; 2nlike i=s7a7.

2..$ The ,cstat %ommand

F=r 8i9re channel GFCH ada67er 42e2es; 7he 8cs7a7 c=mmand is 2sed 7= l==k 8=r 9l=cked IOs in 7he ada67erPs42e2es; e.3.

# fcstat fcs0

FI=E *H))E '$$I'$I*' E/+$ fcs0>>>'econds 'ince ast eset 456432(>>>F* '*'I dapter !river Information )o !" esource *ount 0

)o dapter Elements *ount &02828)o *ommand esource *ount &36&5618

>>>


"a3e #5


16/46


The val2e = C=mmand Res=2rce C=2n7 is 7he n2m9er =8 7imes; since 7he ada67er ?as c=n8i32red G76icall 9==7 7imeH; 7ha7 an IO ?as 7em6=raril 9l=cked ?ai7in3 8=r res=2rces; s2ch as an ada67er 9288er d2e7= a n2mcmdelems a77ri927e val2e 7ha7 is 7== l=?. =ner= val2es indica7e 7ha7 increasin3n2mcmdelems ma hel6 im6r=ve IO service 7imes. O8 c=2rse i8 7he val2e incremen7s sl=?l; 7hen 7he

im6r=vemen7 ma 9e ver small; ?hile 42ickl incremen7in3 val2es means 72nin3 is m=re likel 7= have ameas2ra9le im6r=vemen7 in 6er8=rmance. One can calc2la7e 7he ra7e =8 9l=cked IOs as 7he 9l=cked c=2n7divided 9 7he sec=nds since las7 rese7. $ere ?ere 3e77in3 n= c=mmand res=2rce 9l=cks a7 a ra7e =8 a9=27' 6er sec=nd. Ass2min3 ?e red2ce la7enc 9 # ms 9 72nin3 7he 42e2e; =verall savin3s in an h=2r ?=2ld9e

' 9l=cks1s # ms19l=ck +/@@ s1hr &.' sec each h=2r

S= 7his isn7 a si3ni8ican7 9=77leneck and ma n=7 9e ?=r7h ?=rrin3 a9=27. T6icall 7he am=2n7 =8 7ime?ai7in3 8=r a 9288er 7= 8ree is ver small. (e are ?ai7in3 8=r 7he s7=ra3e 7= c=m6le7e ne7 in8li3h7 IO.

The 8cs7a7 has recen7l 9een enhanced ?i7h 7he JD 8la3 ?hich 6r=d2ces =27627 as 8=ll=?s

# fcstat -! fcs0

FI=E *H))E '$$I'$I*' E/+$ fcs0

!evice $ype 8b /*I Express !ual /ort F* dapter?df&000f&&2&08a03A?adapterpciexdf&000f&&2&08a0A

>>>> I+ !" pool si%e 0x&000000

F* '*'I dapter !river ueue 'tatistics

)umber of active commands 0 Hig ,ater mar. of active commands &80 )umber of pending commands 0 Hig ,ater mar. of pending commands 6& )umber of commands in te dapter !river Held off 7ueue 0 Hig ,ater mar. of number of commands in te dapter !river Held off 7ueue 0

F* '*'I /rotocol !river ueue 'tatistics )umber of active commands 0


"a3e #/


17/46


Hig ,ater mar. of active commands &80 )umber of pending commands ( Hig ,ater mar. of pending commands 48

>>>F* '*'I dapter !river Information )o !" esource *ount 0 )o dapter Elements *ount 0 )o *ommand esource *ount &4&&643>>>dapter Effective max transfer value 0x&00000

These s7a7is7ics are es6eciall 2se82l in 72nin3 7he ada67er 42e2es. I7ems hi3hli3h7ed in 9l2e sh=? 7he s7a7is7ics =8 in7eres7 and =7her 2se82l in8=rma7i=n incl2din3 7he ada67er 76e; (("; s266=r7ed s6eed; 7her2nnin3 s6eed; 7he sie =8 7he DMA 6==l mem=r area; and 7he ma8ersie val2e c2rren7l 2sed 9 7hekernel.

2..( The Interacti'e topas %ommand

S7ar7in3 7he in7erac7ive 7=6as c=mmand ?i7h

# topas -!

Or 2s7 6ressin3 D ?hile in 7=6as ?ill ield 7his disk de7ail 6anel

Fi32re ' 7=6as D sam6le =27627

KBR KB readART Avera3e Read IO service TimeKB( KB (ri77enA(T Avera3e (ri7e IO service Time

A)( Avera3e )2e2e (ai7 avera3e 7ime ?ai7in3 in 7he 42e2eA)D Avera3e )2e2e De67h avera3e n2m9er =8 IOs in 7he ?ai7 42e2e

This c=mmand; like i=s7a7; re6=r7s 9=7h read and ?ri7e IO service 7imes; and 42e2e ?ai7 7ime as ?ell.

"ressin3 7he d ke ?hile in 7he disk de7ail 6anel; ?ill al7erna7e 8r=m 7he disk de7ail 6anel 7= 7he ada67er6anel


"a3e #&


18/46


Fi32re + 7=6as ada67er 6anel

"ressin3 d7hen v?ill sh=? 7he vir72al ada67er 6anel

Fi32re - 7=6as vir72al ada67er 6anel

=7e 7ha7 7he 6er8=rmance me7rics re8er 7= avera3e val2es 8=r 7he L!s a77ached 7= 7he ada67er. Th2s 7heA)( and A)D re8er 7= ?ai7 7ime 8=r 7he L!s in 7heir hdisk driver 42e2es; n=7 ?ai7 7ime in 7he ada67er42e2e.

2..- The Interacti'e nmon %ommand

One en7ers 7he in7erac7ive nm=n 7==l via

# nmon

"ressin3 D?ill ccle 7hr2 8=2r di88eren7 disk screens; 7he 8irs7 sh=?in3 each disks 7hr2627; 7he sec=ndsh=?in3 7he disk sie; n2m9er =8 6a7hs and an ada67er c=nnec7ed 7= 7he disk; 7he 7hird screen sh=?n 9el=?; and 7he 8=2r7h screen sh=?s disk 7hr2627 s7a7is7ics ?i7h a 3ra6h indica7=r =8 7hr2627. $eres 7he 7hirdscreen sh=?in3 IO service 7imes


"a3e #*


19/46


Fi32re 5 in7erac7ive nm=n disk IO service s7a7is7ics

(ai7 avera3e ?ai7 7ime in 7he 42e2e in msServ) Sie Avera3e service 42e2e sie(ai7) Sie Avera3e ?ai7 42e2e sie

Serv) F2ll 2m9er =8 IO re42es7s sen7 7= a 82ll 42e2e 8=r 7he in7erval

I8 =2ve a l=7 =8 hdisks; =2 can 6ress 7he .N s29c=mmand 7= 2s7 sh=? 7he 92s disks.

2.. /)O/ 0ecordin&s

One can se726 MO rec=rdin3s 8r=m smi7 via

# smitty topas -C 'tart )e, ecording -C 'tart local recording -C nmon

$=?ever; 7he smi7 6anel d=esn7 lis7 7he =67i=n ?e need 7= 3e7 disk IO service 7imes; s6eci8icall 7he Jd

=67i=n 7= c=llec7 disk IO service and ?ai7 7imes. Th2s; =ne needs 7= 2se 7he c=mmand line ?i7h 7he nm=nc=mmand 7= c=llec7 and re6=r7 7hese s7a7is7ics. $eres =ne se7 =8 =67i=ns 8=r c=llec7in3 7he da7a

# nmon -df9")+/;M -, 2 -s BintervalC -c Bnumber of intervalsC

The ke =67i=ns here 8r=m a disk vie?6=in7 incl2de

d c=llec7 and re6=r7 IO service 7ime and ?ai7 7ime s7a7is7ics incl2des 7he FC ada67er sec7i=n G?hich als= meas2res "IV 7ra88ic =n VIOS FC ada67ersHV incl2des 7he disk v=l2me 3r=26 sec7i=n

T= 3e7 7he rec=rdin3 7hr2 7he MO Analer 7==l Ga s6readshee7 7==l 7ha7 r2ns =n "Cs and 3enera7es 6er

8=rmance 3ra6hs; =7her =27627; and is availa9le via 7he in7erne7H; i7s rec=mmended 7= kee6 7he n2m9er =8in7ervals less 7han +@@. Sam6les =8 7he main re6=r7s =8 in7eres7 8=r IO 42e2e 72nin3 8=ll=?


"a3e #0


20/46


Disk Read Service Time msec/!er d"## 1$/2%/1&

0

2

4

6

8

10

12

14

hdisk1

hdisk0

hdisk2

Avg. WAvg. Max.

Fi32re / MO analer disk read IO la7enc

Disk Read Service Time msec/!er d"## 1$/2%/1&

0

2

4

6

8

10

12

14

17:39

17:40

17:40

17:40

17:41

17:41

17:41

17:42

17:42

17:42

17:43

17:43

17:43

17:44

17:44

17:44

17:45

17:45

17:45

17:46

17:46

17:46

17:47

17:47

17:47

17:48

17:48

17:48

17:49

17:49

hdisk1 hdisk0 hdisk2

Fi32re & MO analer disk read IO la7enc =ver 7ime


"a3e '@


21/46


Disk IO Reads 'er second d"## 1$/2%/1&

0

10

20

30

40

50

60

70

17:39

17:40

17:40

17:40

17:41

17:41

17:41

17:42

17:42

17:42

17:43

17:43

17:43

17:44

17:44

17:44

17:45

17:45

17:45

17:46

17:46

17:46

17:47

17:47

17:47

17:48

17:48

17:48

17:49

17:49

Thous

ands


Fi32re * MO analer disk read IO"S =ver 7ime

Similar re6=r7s eis7 8=r ?ri7e service 7imes Gn=7 sh=?nH. Als= 6r=vided are 3ra6hs =8 avera3e ?ai7 7ime in7he hdisk driver 42e2e 8=r IOs

Disk (ait Queue Time msec/!er d"## 1$/2%/1&

0

10

20

30

4050

60

70

80

90

hdisk0

hdisk2

hdisk1

Avg. WAvg. Max.

Fi32re 0 MO analer disk 42e2e ?ai7 7ime


"a3e '#


22/46


Disk (ait Queue Time msec/!er d"## 1$/2%/1&

0

10

20

30

40

50

60

70

80

90

17:39

17:40

17:40

17:40

17:41

17:41

17:41

17:42

17:42

17:42

17:43

17:43

17:43

17:44

17:44

17:44

17:45

17:45

17:45

17:46

17:46

17:46

17:47

17:47

17:47

17:48

17:48

17:48

17:49

17:49


Fi32re 0 MO analer disk 42e2e ?ai7 7ime; =ver 7ime

The ra? da7a is als= availa9le in case i7s di88ic2l7 7= read 7he 3ra6hs; e.3.; 7he avera3e disk ?ai7 7ime 8=rhdisk@ G7he nearl invisi9le ver narr=? 9l2e 9ar in 8i32re 0; is #.+ ms.H. MO rec=rdin3s have 7he9ene8i7 7ha7 =2 can 42ickl see ?hen 7he hdisk 42e2e is =ver8l=?in3; 7he d2ra7i=n; and h=? l=n3 7he IOsare ?ai7in3 in 7he 42e2e.

2.. The SDD+%) pcmpath %ommand

SDD"CM 6r=vides 7he 6cm6a7h 42er devs7a7s and 6cm6a7h 42er ada67s7a7s c=mmands 7= sh=?hdisk and ada67er 42e2e s7a7is7ics. :=2 can re8er 7= 7he SDD1SDD"CM man2al 8=r sn7a; =67i=ns; e6lana7i=ns =8 all 7he 8ields and man =7her 2se82l c=mmands incl2din3 7he a9ili7 7= disa9le all 6a7hs 7= as6eci8ic h=s71s7=ra3e 6=r7 ?i7h =ne c=mmand. $erePs 7he =27627 =8 6cm6a7h 42er devs7a7sN 8=r =nehdisk

!evice # 0 !EI*E )"E dis.( $otal ead $otal


23/46


dapter # 0 $otal ead $otal


24/46


&. Tunin) the Queues

$.1 %heck ,or SA/3Stora&e +roblems First

Be8=re 72nin3 7he 42e2es; =2 sh=2ld make s2re 7here aren7 6r=9lems ?i7h 7he SA =r 7he s7=ra3e inhi9i7in3 6er8=rmance. Fiin3 7hese 6r=9lems can res=lve 42e2e ?ai7 iss2es; as re8lec7ed in 7his 8=rm2la

Ma L! IO"S 42e2ede67h 1 Uav3 IO service 7ime

I8 ?e red2ce 7he IO service 7ime 9 8iin3 7he SA1s7=ra3e; ma IO"S ?ill increase; and IOs ?=n7 9e 2sin3 7he 42e2e sl=7 as l=n3; red2cin3 2sa3e =8 7he 42e2e sl=7s; and red2cin3 ?ai7 7ime in 7he 42e2e. Fas7erIOs 8r=m 7he s7=ra3e can 8i =2r 42e2e ?ai7 6r=9lem ?i7h=27 ad2s7in3 7he 42e2e sies.

The 8cs7a7 c=mmand is 2sed 7= eamine FC link heal7h ?here i7 sh=?s 7he 8=ll=?in3 s7a7is7ics

# fcstat fcs0

FI=E *H))E '$$I'$I*' E/+$ fcs0G'econds 'ince ast eset 554(40GI/ *ount 0)+' *ount 0Error Frames 0!umped Frames 0in. Failure *ount 0oss of 'ync *ount 26

oss of 'ignal 0/rimitive 'e7 /rotocol Error *ount 0Invalid $x


25/46


(hen =2 increase 7he 42e2ede67hs and 7he n2m9er =8 in8li3h7 IOs 7ha7 are sen7 7= 7he disk s29ss7em;7he IO service 7imes are likel 7= sli3h7l increase; 927 7hr=23h627 ?ill als= increase. I8 IO service 7imess7ar7 a66r=achin3 7he disk 7ime=27 val2e; 7hen =2Pre riskin3 s29mi77in3 m=re IOs 7han 7he disk s29ss7emcan handle. I8 =2 s7ar7 seein3 IO 7ime=27s and err=rs in 7he err=r l=3 indica7in3 6r=9lems c=m6le7in3 IOs;7hen 7his is 7he 7ime 7= l==k 8=r hard?are 6r=9lems =r 7= make 7he 6i6e smaller 9 red2cin3 42e2e sies.

Once IO service 7imes s7ar7 increasin3; ?ePve 62shed 7he 9=77leneck 8r=m 7he AI% disk ada67er driver 7=s=me?here 9el=? 7he hdisk driver in 7he s7ack. This c=2ld 9e 7he ada67er driver; 7he SA; a VIOS; =r 7hedisk s29ss7em G=r c=m6=nen7s ?i7hin i7H i7sel8. T6icall i7 ?ill 9e 7he disk s29ss7em and =87en 7he6hsical disks ?i7hin i7.

Be c=3nian7 i8 7he IO is hi3h ra7e asnchr=n=2s se42en7ial IO. In s2ch a case; ?e h=6e and e6ec7 7he a66lica7i=n 7= ini7ia7e man IO 7hreads s= IOs are sen7 7= 7he s7=ra3e as s==n as 6=ssi9le 7= drive 26 7hr2627.This 76icall 8ills 7he 42e2es makin3 IOs ?ai7; s= ?e d=n7 e6ec7 7= 9e a9le 7= make 7he 42e2es 9i3en=23h 7= av=id 8illin3; n=r d= ?e necessaril need 7=. E.,.; c=nsider i8 ?e have a #@ ,B 7a9le; an a66lica7i=n needs 7= read 2sin3 a 76ical ma IO sie =8 '5/ KB. S= 7he a66lica7i=n iss2es -@;0/@ '5/ KB read

re42es7s asnchr=n=2sl; 7= read 7he en7ire 7a9le. !nless ?e have l=7s =8 hdisks and ver 9i342e2ede67hs; 7he 42e2es ?ill 9e 82ll 8=r a?hile Gin 7his case 2s7 a 8e? sec=ndsH. Then 7heres s7ill 7he42es7i=n =8 ?he7her 7he s7=ra3e can handle 7ha7 man in8li3h7 IOs; ?he7her ?ell r2n in7= in7erc=nnec79and?id7h limi7s; and ?he7her increasin3 42e2ede67h hel6s 6er8=rmance. Increasin3 42e2ede67h 7=handle s2ch IO =nl hel6s 7= 7he e7en7 i7 increases 7hr2627; re3ardless =8 ?ai7 7ime in 7he 42e2e. Increasin3 42e2ede67h here increases h=? man IO re42es7s can 9e d=ne in 6arallel; ass2min3 7he s7=ra3e cand= m=re IOs in 6arallel ?i7h=27 si3ni8ican7l de3radin3 IO la7enc =r r2nnin3 in7= =7her SA1s7=ra3e 9=77lenecks.

$.2.. 1 T"nin& hdisk 4"e"e5depth

T?= a66r=aches 7= 72nin3 42e2e de67h are #H 9ase 7he 42e2e de67hs =n ac72al IO re42es7s =2r a66lica7i=n3enera7e =r 'H 2se a 7es7 7==l and 72ne 7he 42e2es 9ased =n ?ha7 7he disk s29ss7em can handle. The ndisk7==l G6ar7 =8 7he ns7ress 6acka3e availa9le =n 7he in7erne7 a7 h77611???0-#.i9m.c=m1c=lla9=ra7i=n1?iki1dis6la1(iki"76e1ns7ress H can 9e 2sed 7= s7ress 7he disk s29ss7em 7= see?ha7 i7 can handle; and 7= 6r=vide 7he da7a 7= 3enera7e IO"S vers2s. IO service 7ime 3ra6hs 8=r 7he s7=ra3e.The a27h=rPs 6re8erence is 7= 72ne 9ased =n =2r a66lica7i=n IO re42iremen7s; es6eciall ?hen 7he disk isshared ?i7h =7her servers.

F=r 72nin3; ?e can ca7e3=rie 7he si72a7i=n in7= =ne =8 / cases

I( Service Ti


26/46


ueues=7 8ill

in3Fillin3 =7 8illin3 Fillin3 =7 8illin3 Fillin3

Tunin9%ase andAction:

#. =72nin3

needed

'. Increase42e2e

sies

+. Add re

s=2rces 9el=?

7he hdisk driv

er

-. Add re

s=2rces 9el=?

7he hdisk driv

er. Ma9e

72ne42e2ede67h.

5. Check 7he ss7em 8=r 9=77le

necks 9el=? 7he hdisk driver.T2ne 7he s7=ra3e; add s7=ra3e

res=2rces; and1=r red2ce 42e2e

sies

/. Red2ce 42e2e

sies and add re

s=2rces 9el=? 7he

hdisk driver

Fi32re #@ J T2nin3 Si72a7i=ns

(e like 7= 72ne 7he 42e2es; s= 7he 42e2es d=n7 8ill 26; and s7ill have 3==d IO service 7imes; leavin3 2s incase #. Limi7ed s7=ra3e res=2rces =87en leave 2s in case + =r -; 3e77in3 7he m=s7 7hr2627 6=ssi9le 8=r

Case ' is ver c=mm=n. The de8a2l7 42e2ede67h val2es aren7 a66r=6ria7e 8=r ever=ne; s= increasin37he 42e2e sies is im6=r7an7. F=r eam6le 7he de8a2l7 vSCSI hdisk has a 42e2ede67h =8 +; limi7in3 i7sIO"S 9and?id7h 7= +1Uav3 IO service 7ime. In ch==sin3 h=? m2ch 7= increase 42e2e de67h; s=me =8 7hec=nsidera7i=ns incl2de

$=? man main7enance ?ind=?s d= =2 3e7 7= 72ne 7his>


"a3e '*


29/46


This 8cs@ da7a sh=?s an eam6le =8 an ada67er 7ha7 has s288icien7 val2es 8=r n2mcmdelems andma8ersie. =ner= val2es indica7e a si72a7i=n in ?hich IOs ?ai7ed in 7he ada67er driver 42e2e d2e7= lack =8 res=2rces. Increase n2mcmdelems 8=r n=ner= val2es =8 = C=mmand Res=2rce C=2n7?hen 7he ra7e is si3ni8ican7. Increase 7he DMA 6==l sie 8=r n=n er= val2es 8=r = DMA Res=2rce ?hen7he ra7e is si3ni8ican7; i8 6=ssi9le. =7e 7ha7 chan3in3 ma8ersie isn7 al?as 6=ssi9le; and can lead7= 9==7 8ail2re. "lease see sec7i=n #.+ Changing Device Attribte !ales7ha7 disc2sses 7his i8 =2 haven7alread.

$ere; similar 7= increasin3 42e2ede67h; ?ed increase n2mcmdelems 7= s=me7hin3 m=re 7han

ne? n2mcmdelems n2mcmdelems U9l=cked IO ra7e1UIO"S ra7e

= ad2s7men7 =8 7he DMA mem=r sie is 6=ssi9le =n vir72al FC ada67ers; 2s7 =n 7he real ada67er.

$.$. A,ter 6o"r Initial T"nin&

A87er chan3in3 7he val2es; r2n =2r a66lica7i=n and l==k a7 7he s7a7is7ics 8=r 6eak IO 6eri=ds and 72nea3ain; i8 needed. (hile en7irel elimina7in3 42e2e ?ai7 7ime is 7he 3=al; 7he re72rns 8r=m inves7in3 7ime in7his 72nin3 diminish as 8e?er IOs ?ai7 in 7he 42e2es; and as 7he ?ai7 less 7ime as ?ell.

$.(. T"nin& Order

Re3ardin3 7he =rder 7= 72ne 7he s7ack; =ne 3enerall 72nes 42e2ede67h; n2mcmdelems; andma8ersie 7=3e7her in =ne main7enance ?ind=?. T2nin3 7he l=?er laers =8 7he s7ack 8irs7 Ge.3.; 72n

in3 7he ada67er 42e2e sies 9e8=re 7he hdisk 42e2e siesH can res=lve 42e2e ?ai7s hi3her in 7he s7ack. On7he =7her hand; 72nin3 hi3her laers in 7he s7ack 76icall increase 7he n2m9er =8 in8li3h7 IOs sen7 7= 7hel=?er laers ?hich can res2l7 in m=re 9l=ckin3 7here. S= 72nin3 7hese is an i7era7ive 6r=cess.

$.-. #hat Are 7ood8 0easonable and +oor IO Ser'ice Times9

(ha7 is 3==d; reas=na9le; =r 6==r is a 8ac7=r =8 7he s7=ra3e 7echn=l=3; 7he s7=ra3e cache sies; and 7he IO?=rkl=ad 8=r 7he disk s29ss7em. Ass2min3 n= IOs are 42e2ed 7= a $DD; a 76ical read ?ill 7ake s=me?here 8r=m @ 7= #5 ms; =r s=; de6endin3 =n h=? 8ar 7he ac72a7=r has 7= 7ravel Gseek 7imeH; h=? l=n3 i7

7akes 7he disk 7= r=7a7e 7= 7he ri3h7 sec7=r Gr=7a7i=n 7ime 9ased =n s6indle s6eedH; and h=? l=n3 i7 7akes 7=read 7he da7a G7rans8er 7imeH. Then 7he da7a m2s7 m=ve 8r=m 7he s7=ra3e 7= 7he h=s7. T6icall 7he 7ime isd=mina7ed 9 seek 7ime W r=7a7i=n 7ime; 7h=23h 8=r lar3e IOs 7rans8er 7ime als= can 9e si3ni8ican7. This incl2des 7imes 7= 7rans8er 7he IO 7=18r=m 7he disk; as ?ell as sendin3 i7 acr=ss vari=2s in7er8aces Gs7=ra3e9ackend; s7=ra3e 7= h=s7 in7er8ace; e7c. incl2din3 FC; SAS; iSCSI and FC=EH. S=me7imes 7he da7a ?ill 9ein disk s29ss7em read cache; in ?hich case 7he IO service 7ime is ar=2nd # ms =r 9e77er. T6icall 8=rlar3e disk s29ss7ems 7ha7 arenP7 =verl=aded ?i7h #5K R"M disks; read IO service 7imes ?ill avera3e


"a3e '0


30/46


ar=2nd 5#@ ms. (hen small rand=m reads s7ar7 avera3in3 3rea7er 7han #5 ms ?i7h #5K R"M disks; 7hisindica7es 7he s7=ra3e 27ilia7i=n is hi3h.

(ri7es 76icall 3= 7= ?ri7e cache Gass2min3 i7 eis7sH ?i7h avera3e IO service 7imes 76icall less 7hana9=27 '.5 ms; 927 7here are ece67i=ns. I8 7he s7=ra3e is snchr=n=2sl mirr=rin3 7he da7a 7= a rem=7e si7e;

?ri7es can 7ake m2ch l=n3er as ?e have 7= add 7he in7ersi7e r=2nd 7ri6 la7enc. And i8 7he IO is lar3e Gsa/- KB =r lar3erH 7hen 7he 7rans8er 7ime 9ec=mes m=re si3ni8ican7 and 7he avera3e 7ime is sli3h7l ?=rse. I87herePs n= cache; 7hen ?ri7es 7ake a9=27 7he same as reads. S=me disk s29ss7ems d=n7 cache lar3e ?ri7eIOs and send 7hem direc7l 7= 7he s7=ra3e.

$eres a 7a9le sh=?in3 e6ec7ed IO service 7imes and IO"S 8=r li3h7l l=aded disk 7echn=l=3ies

Fi32re ## IO"S 8=r di88eren7 disk 7echn=l=3ies

These 7imes are sim6l calc2la7ed as IO"S #1 Gr=7a7i=nal la7enc W av3. seek 7imeH

In reali7; $DDs can d= m=re =r less IO"S a7 hi3her1l=?er IO service 7imes res6ec7ivel. And =ne cancharac7erie a $DD in an IO"S vs. IO service 7ime char7 as 8=ll=?s Gin 7his case 8=r a &'@@ R"M SATAdisk d=in3 - KB IOsH


"a3e +@


31/46


Fi32re #' Sam6le IO"S vs. IO service 7ime 3ra6h

S= ?hile 7his &'@@ R"M SATA disk can achieve =ver #*@ IO"S; ?e ma n=7 ?an7 7= drive i7 a7 7ha7 ra7e;and 6re8er a l=?er IO"S ra7e and a 9e77er IO service 7ime. The 6ercen7a3es =n 7he 3ra6hed line are 7he$DD 27ilia7i=n. Th2s; ?ha7s reas=na9le de6ends =n 7he ?=rkl=ad and 7he c2s7=mers 6er8=rmance re42iremen7s. F=r a $DD like 7his; an IO service 7ime =8 '@ ms is reas=na9le; 2nless =2ve 6lanned 7= d=8e?er IO"S and 3e7 9e77er IO service 7imes 7= mee7 =2r a66lica7i=n 6er8=rmance re42iremen7s. Similarchar7s eis7 8=r disk s29ss7ems; and =ne can make s2ch a char7 8=r s7=ra3e all=ca7ed 7= an L"AR.

I8 7he IO is lar3e 9l=ck se42en7ial; 7hen 9esides 7he increased 7rans8er 7ime; ?e e6ec7 IOs 7= 42e2e a7 7he6hsical disk; and IO service 7imes 7= 9e m2ch l=n3er =n avera3e. E.,.; i8 an a66lica7i=n s29mi7s 5@ IOsGsa 5@ /- KB IOs readin3 a 8ile se42en7iallH 7hen 7he 8irs7 8e? IOs ?ill have reas=na9l 3==d IO service7imes; ?hile 7he las7 IO ?ill have had 7= ?ai7 8=r 7he =7her -0 7= 8inish 8irs7; and ?ill have a ver l=n3 IO

service 7ime. (e h=6e 7= 42e2e 26 a 92nch =8 IOs 8=r se42en7ial IO 7hreads; and s= ?e h=6e 7he ?ill 9e?ai7in3 in 7he 42e2e read 7= 3=. Th2s; ?e d=n7 3e7 c=ncerned ?i7h l=n3 IO service 7imes 8=r se42en7ialIO; and ins7ead 8=c2s =n 7hr2627.

IOs 7= SSDs18lash are 76icall less 7han # ms and 8=r SSDs18lash in disk s29ss7ems; 76icall less 7han 'ms; and =ccasi=nall hi3her. IO la7encies 8=r 8lash have 9een im6r=vin3; as sh=?n in 7his char7


"a3e +#


32/46


Fi32re #+ eMLC SSD 6er8=rmance

(here 7he 8lash is im6lemen7ed als= ma77ers. The n2m9ers in 7he char7 are 8=r SAS a77ached SSDs. I8 7he8lash is in a disk s29ss7em; add a9=27 @.5 ms 7= 7he la7enc 8=r 7he IO 7= 3= 7hr2 7he disk s29ss7em 9ackend in7er8aces and als= 7hr2 7he SA 8=r ?ha7 is e6ec7ed. I8 2sin3 VIO; 7hen add @.@5 ms G5@ micr= sec=ndsH 7= la7enc 7= 3= 7hr2 7he VIOS. The FC a77ached IBM FlashSs7em can achieve @.+ ms 8=r IOs7hanks 7= i7s 2se =8 a da7a 6a7h 2sin3 hard?are ra7her 7han s=87?are.

!nders7andin3 ?ha7 reas=na9le IO service 7imes are 8=r =2r s=l27i=n is im6=r7an7 in 72nin3 7he 42e2es; s==2 kn=? i8 =2 have a 9=77leneck 9el=? 7he hdisk driver; =r n=7.

$.-..1 Stora&e %ache )ana&ement Al&orithm :,,ects on IO Ser'ice Times

S=me disk s29ss7ems 2se cache mana3emen7 al3=ri7hms; ?hich s=me7imes 7ake a?a ?ri7e cache 8=r aL!. This =cc2rs ?here 7he IO"S ?ri7e ra7e 7= a L! is ver l=?; and ?here 7here is a hi3h read ra7e8r=m 7he L!; =r heav 2se =8 7he cache else?here. In s2ch cases; 7he disk s29ss7em ma 7ake all =rnearl all 7he ?ri7e cache 8=r a L! and 2se i7 else?here. The res2l7 ?hen an a66lica7i=n sends a ?ri7e in7his si72a7i=n is 7he disk s29ss7em cache mana3emen7 al3=ri7hms have 7= 7ake 7he 7ime 7= 8ree 26 and all=ca7e s=me cache 8=r 7he ?ri7e; and 6lace 7he ?ri7e da7a in 7he cache s= 7he disk s29ss7em can 7hen send


"a3e +'


33/46


9ack an ackn=?led3emen7 7ha7 7he ?ri7e is c=m6le7e. This adds si3ni8ican7 la7enc 7= 7hese IOs; and =2ri=s7a7 da7a ma l==k like

# iostat Dl!!is.s xfers read ,rite-------------- -------------------------------- ------------------------------------ ------------------------------------

Jtm bps tps bread b,rtn rps avg min max time fail ,ps avg min max time fail act serv serv serv outs serv serv serv outs

dis.0 0>3 (1>49 3>& &6>39 4>59 &>2 &>4 0>2 &6>8 0 0 &>1 0>8 0>1 1>6 0 0dis.& 0>& 508>1 0>& 343>0 &35>1 0>& 8>& 0>5 (2>4 0 0 0>0 0>8 0>1 &>0 0 0dis.( 0>0 14>8 0>0 0>0 14>8 0>0 0>0 0>0 0>0 0 0 0>0 0>8 0>4 &>0 0 0dis.3 &>& 34>39 2>2 (5>&9 &(>(9 (>0 0>8 0>3 &0>2 0 0 (>2 2>2 0>1 138>2 0 0dis.2 80>& 33>1" 56(>5 33>1" 38>(9 586>2 (>2 0>3 853>1 0 0 3>& 1>5 0>5 450>3 0 0dis.5 53>( &1>6" 302>( &1>6" (&>59 30(>( 3>0 0>3 &>0' 0 0 (>0 &1>2 0>4 426>3 0 0dis.1 &>& (&>49 2>( &>69 &6>89 0>& 0>1 0>5 0>8 0 0 2>0 (>4 0>1 265>1 0 0

=7e 7he hi3h ?ri7e IO service 7imes in red and 7he hi3h read IO"S in 9l2e. Overall; 7he L"AR is d=in30@+.* IO"S; 927 la7enc is =nl a88ec7ed 8=r &.- IO"SQ 7h2s; less 7han #X =8 7he IOs are a88ec7ed.

The disk s29ss7em mi3h7 =88er s=me 72nin3 7= 6reven7 7his; 927 i8 n=7; 7here isn7 an eas me7h=d 7= deal?i7h i7. C=nsider 7ha7 since 7he im6ac7 is 7= a small 8rac7i=n =8 7he IOs; 7ha7 =verall i7 isn7 likel 7= have asi3ni8ican7 6er8=rmance im6ac7. Th2s; =ne needs 7= 9e a?are =8 7his; 6ri=r 7= decidin3 7he ?ri7es are 7==

sl=?.

$.. T"nin& !"e"e Si;es in VIO :n'ironments

IO 7hr2 VIO is d=ne ei7her ?i7h vSCSI =r "IV; 2sin3 a vir72al SCSI =r a vir72al Fi9re Channel GvFCHada67er as sh=?n in 8i32res #- and #5 res6ec7ivel.

Fi32re #- J vSCSI Archi7ec72re


"a3e ++


34/46


Fi32re #-5 J "IV Archi7ec72re

These dia3rams sh=? 7he l=3ical c=nnec7i=ns =8 7he vir72al and real hard?are; 927 7he d=n7 sh=? 7he IOs7ack; s= le7s l==k a7 7ha7


"a3e +-


35/46


Fi32re #5/ J VIO IO S7ack T2nin3

=7e 7ha7 ?e d= n=7 3= 7hr2 7he hdisk driver in 7he VIOS ?i7h "IV. Th2s; "IV IO s7a7is7ics are n=7ca672red via i=s7a7; sar =r =7her c=mmands a7 7he VIOS 7ha7 access 7he hdisk driver s7a7is7icsY (e can s7illca672re IO s7a7is7ics via 8cs7a7 and nm=n 7ha7 access 7he ada67er driver s7a7is7ics.

(i7h vSCSI; ?e 72ne hdisk 42e2ede67h a7 9=7h 7he VIOS and 7he VIOC. And ?e als= 72ne 7he real FCada67ers c=nnec7ed 7= 7he s7=ra3e. The ada67er als= has a 8ied 42e2e; s= 7= av=id 42e2in3 7here; ?e limi77he n2m9er =8 hdisks 2sin3 7heir ada67ers 9ased =n 7heir 42e2e sies.

(i7h "IV; ?e 72ne 7he hdisk 42e2e de67hs a7 7he VIOC; 7he vFC ada67ers a7 7he VIOCs; and 7he realada67ers a7 7he VIOSs. One 7hin3 ?e can7 chan3e 8=r 7he vFC is 7he DMA mem=r sie; ?hich is 8iedre3ardless =8 7he ma8ersie a77ri927e val2e. I7s im6=r7an7 7= kn=? 7he vFC ada67ers have an e88ec7ive limi7 =8 '5/ in8li3h7 IOsQ 7h2s; increasin3 n2mcmdelems 9e=nd '5/ ?=n7 have an a88ec7.

$... 1 A'oidin& VIOS O"ta&es

T= chan3e device a77ri927es means 7he cann=7 9e in 2se. Disks ma66ed 7hr2 vSCSI =r vFCs ma66ed 7hr2real FC ada67ers are in 2se. And =87en 7here are several vFCs 2sin3 a sin3le real FC. Th2s; chan3in3 7hereal FC ada67ers s266=r7in3 "IV; ?i7h=27 a VIOS =27a3e; is di88ic2l7.

A sim6le a66r=ach is 7= 2se Live "ar7i7i=n Mi3ra7i=n GL"MH 7= m=ve 7he L"ARs; 2sin3 7he VIOS; 7= an=7her ss7em; make 7he chan3e 7= 7he real ada67er a77ri927es; 7hen 2se L"M 7= m=ve 7he L"ARs 9ack.


"a3e +5


36/46


An=7her a66r=ach is essen7iall 7= re6lace 7he ada67er; seeh776116ic.dhe.i9m.c=m1in8=cen7er16=?erss1v+r#m51inde.s6


37/46


#. Se7 all 7he 42e2e sies 7= 7he minim2m sie 7ha7 can handle 7he 6eak ?=rkl=ad. Se7 vSCSI hdisk42e2ede67h 7= ma7ch a7 7he VIOS and 7he VIOC. T2ne ada67er n2mcmdelems and DMA mem=r sie a7 7he VIOS and n2mcmdelems a7 7he VIOC; 7= handle 7he 6eak ?=rkl=ads.

'. Se7 7he 42e2e sies a7 7he VIOS 7= 7heir maim2m val2es; and limi7 7hem a7 7he VIOCs.

S7ra7e3 # has 7he vir72e =8 minimiin3 7he 2se =8 res=2rces Gm=s7l mem=r 7= handle increased 42e2esiesH. S7ra7e3 ' has 7he vir72e =8 minimiin3 VIOS =27a3es 8=r 72nin3 and red2cin3 adminis7ra7=r ?=rk;927 2ses e7ra mem=r. B=7h a66r=aches ins2re 7ha7 ?e d=n7 have hdisk driver 42e2e 9=77lenecks a7 7heVIOS.

C2s7=mers ma 7ake 7he a66r=ach 7ha7 9es7 8i7s 7heir needs.

$... $ T"nin& 'S%SI !"e"es

(hen 2sin3 vSCSI; =ne c=n8i32res vir72al SCSI ada67ers 8=r 7he VIOC 7= access s7=ra3e c=n8i32red =n 7he

VIOS. F=r each vir72al SCSI ada67er; 7here ?ill 9e a vscsi device in a VIOC; and a ma7chin3 vh=s7 devicein 7he VIOS. (e d=n7 have kn=9s 7= chan3e 42e2e sies 8=r 7he vscsi ada67ers; ins7ead ?e ch==se h=?man ?e crea7e and ?hich hdisks 2se ?hich ada67ers.

These vir72al ada67ers have a 8ied 42e2e de67h. There are 5#' c=mmand elemen7s =8 ?hich ' are 2sed 97he ada67er; + are reserved 8=r each vSCSI L! 8=r err=r rec=ver and 7he res7 are 2sed 8=r IO re42es7s.Th2s; ?i7h 7he de8a2l7 42e2ede67h =8 + 8=r vSCSI L!s; 7ha7 all=?s 8=r 26 7= *5 L!s 7= 2se an ada67er?i7h=27 9l=ckin3 7here G5#' 'H 1 G+ W +H *5 r=2ndin3 d=?n. S= i8 ?e need hi3her 42e2e de67hs 8=r 7hedevices; 7hen 7he n2m9er =8 L!s 6er ada67er is red2ced. E.,.; i8 ?e ?an7 7= 2se a 42e2ede67h =8 '5;7ha7 all=?s 5#@1'* #* L!s. (e can c=n8i32re m2l7i6le vSCSI ada67ers 7= handle man L!s ?i7h hi3h42e2e de67hs. Each vSCSI ada67er 2ses addi7i=nal mem=r and res=2rces =n 7he h6ervis=r. F=r a 8=rm2

la; 7he maim2m n2m9er =8 L!s 6er vir72al SCSI ada67er Gvh=s7 =n 7he VIOS =r vscsi =n 7he VIOCH isITG5#@1G)W+HH ?here ) is 7he 42e2ede67h =8 all 7he L!s Gass2min3 7he are all 7he sameH. $eres a7a9le 8=r 32idance

vS*SI hdisk+ueue de'th

Ma hdisks'er vscsiada'ter,

& - de!ault /

1$ &0

2% 1

&2 1%#%

1$$ %

12 &

2/2 2

2/# 1


"a3e +&


38/46


Fi32re #/& J vSCSI L! limi7s

Z Maim2m 7= ens2re n= 9l=ckin3 =8 IOs d2e 7= lack =8 vscsi 42e2e sl=7s

(hen 2sin3 8lash 9acked L!s; ?here =2 e6ec7 rela7ivel hi3h IO"S; ?e c2rren7l rec=mmend 2sin3 a

sin3le vSCSI ada67er 6er L!. This is d2e 7= =7her limi7a7i=ns ?i7h 7he vSCSI ada67er and 9eca2se 8lashIOs are s= 8as7.

F=r LV vSCSI hdisks; ?here m2l7i6le VIOC hdisks are crea7ed 8r=m a sin3le VIOS hdisk; 7hen =ne ma7ake a dedica7ed res=2rce; shared res=2rce =r an in 9e7?een a66r=ach 7= 7he VIOS hdisk 42e2e sl=7s. See7he sec7i=n -.5 Theoretical Thoghts on Shared vs" Dedicated Resorces.

$... ( T"nin& /+IV !"e"es

(hen 2sin3 "IV; ?e have vFC and real FC ada67er 6=r7s; and =87en have m2l7i6le vFCs 7ied 7= a sin3le

real FC ada67er 6=r7.

I8 =2 increase n2mcmdelems =n 7he vir72al FC GvFCH ada67er 6=r7; 7hen =2 sh=2ld make s2re 7he se77in3 =n 7he real FC ada67er 6=r7 is e42al =r 3rea7er. (i7h m2l7i6le vFCs 6er real FC; 7he real FC 6=r7s res=2rces 9ec=me a shared res=2rce 8=r 7he vFCs. Kee6 in mind 7ha7 n2mcmdelems is e88ec7ivel limi7ed7= '5/ =r smaller; s= 7here isn7 an 9ene8i7 7= increasin3 n2mcmdelems 7= 3rea7er 7han '5/ 8=r vFCs.

:=2 can 2se 7he 8cs7a7 c=mmand 8=r 9=7h 7he vir72al ada67er 6=r7 as ?ell as 7he real ada67er 6=r7 8=r 72nin3;and de7erminin3 ?he7her =r increasin3 n2mcmdelems =r ma8ersie mi3h7 increase 6er8=rmance.

S= 8=r heav IO and es6eciall 8=r lar3e IOs Gs2ch as 8=r 9ack26sH i7Ps rec=mmended 7= se7 ma8ersie

s2ch 7ha7 ?e 3ive 7he maim2m DMA mem=r 7= 7he 8cs device; ?hen 7he ss7em c=n8i32ra7i=n ?ill all=? i7.

$... - T"nin& Shared Stora&e +ool !"e"es

$eres a dia3ram =8 a Shared S7=ra3e "==l GSS"H cl2s7er


"a3e +*


39/46


Fi32re #&* J Shared S7=ra3e "==l Cl2s7er

The SS" and SS" cl2s7er re6=si7=r L!s are all=ca7ed 8r=m a RAID 6r=7ec7ed SA disk s29ss7emGsH 7=all 7he VIOSs in 7he SS" cl2s7er. The SS" L!s are 76icall ver lar3e; ?hile 7he cl2s7er re6=si7=rL! is a9=27 #@ ,B. These L!s are c=n8i32red as hdisks a7 7he VIOS. The VIOS adminis7ra7=r crea7esvir72al hdisks 8r=m 7he SS" and ma6s 7hem 7= VIOCs 8=r ne? =r eis7in3 L"ARs. This rem=ves 7he need

8=r 7he s7=ra3e1SA adminis7ra7=rs 7= 3e7 inv=lved; and s6eeds 7he 6r=cess. The vir72al disks 7he VIOSadminis7ra7=r crea7es are s6read =27 acr=ss 7he L!s in 7he SS" 7= 9alance 7he IO ?=rkl=ad acr=ss 7hem.SS"s als= =88er 7hin 6r=visi=nin3 and sna6sh=7 ca6a9ili7 as ?ell.

Fr=m a 72nin3 6ers6ec7ive; ?e 72ne

FC ada67er 42e2es a7 7he VIOS

42e2ede67h =n 7he SS" hdisks a7 7he VIOS

42e2ede67h =8 7he vir72al disks a7 7he VIOCs

(e =nl need 7= s7=6 2sin3 7he hdisks a7 7he VIOC 7= chan3e 7heir a77ri927es. T= chan3e a77ri927es a7 7heVIOS ?ill 76icall re42ire a VIOS re9==7.

One sh=2ld als= 3ive s=me 7h=23h7 7= makin3 s2re 7he servers d=n7 =verr2n 7he s7=ra3e ?i7h IO re42es7s.One a66r=ach 7= 7his is 8=r 7he s7=ra3e adminis7ra7=r 7= m=ni7=r 7he s7=ra3e res=2rce 2se; and aler7 7heserver adminis7ra7=rs a9=27 i7. Then 7he server adminis7ra7=rs can inves7i3a7e and red2ce 42e2e sies asnecessar; 76icall ?hile 7he s7=ra3e 7eam ?ill 9e 3e77in3 ne? res=2rces 7= handle 7he demand.


"a3e +0


40/46


Desi3nin3 7he 42e2e sies 3e7s in7= sharin3 vs. dedica7in3 s7=ra3e res=2rces; 8r=m 7he maim2m in8li3h742e2e sl=7s ?i7hin 7he disk s29ss7emGsH 7= 7he vir72al disk 42e2e sl=7s a7 7he VIOCs.

Fr=m a shared res=2rce 6ers6ec7ive

The vir72al disks a7 7he VIOCs share 7he SS" hdisk 42e2es a7 7he VIOS The SS" hdisks share 7he FC ada67er 42e2es a7 7he VIOS

The VIOSs share 7he disk s29ss7em 42e2es 8=r i7s limi7 =8 in8li3h7 IOs

(e can a88ec7 h=? 7hese res=2rces are dedica7ed =r shared 9 ?here and h=? m2ch =ne limi7s 7he 42e2esies.

$.. Theoretical Tho"&hts On Shared Vs. Dedicated 0eso"rces

)2e2e sies limi7 7he maim2m n2m9er =8 in8li3h7 re42es7s 8=r s7=ra3e. S= red2cin3 7he 42e2e sie =8sa an hdisk; 3ives 7he =7her hdisks access 7= m=re 42e2e sl=7s 82r7her d=?n in 7he s7ack; s2ch as 7heada67er 42e2e sl=7s; all 7he ?a 7= 7he disk s29ss7emGsH 42e2e sl=7s. Th2s 9 red2cin3 42e2e sies 8=rhdisks; L"ARs; VIOSs; ada67ers and ss7ems; ?e 6r=vide m=re =8 7he s7=ra3e res=2rces 7= 7he =7herhdisks; L"ARs; VIOSs; ada67ers and ss7ems res6ec7ivel.

T6icall ?e have man hdisk drivers sharin3 m2l7i6le ada67ers and ada67er drivers; 7h2s; 7he FC 42e2esl=7s are a shared res=2rce 8=r 7he hdisk drivers

Fi32re #*0 J Ada67er and 6=r7 hdisk 42e2e c=nnec7i=ns

Th2s; i7Ps 6=ssi9le 7= ens2re 7ha7 ?e never 8ill 7he ada67er 6=r7 42e2es; 9 makin3 S!MGhdisk@42e2ede67h; hdisk# 42e2ede67h; ... hdiskM 42e2ede67hH U S!M G8cs@ n2mcmdelems; 8cs#n2mcmdelems; ... 8cs n2mcmdelemsH. This ass2mes 7ha7 IOs are evenl s6read acr=ss 7he ada67ers.And m=s7 m2l7i6a7h c=de d=es 9alance IOs acr=ss 7he ada67ers G=r a7 leas7 canH.


"a3e -@


41/46


Th=23h envir=nmen7s =87en have man m=re hdisks 7han FC 6=r7s; and ens2rin3 ?e ?=nP7 8ill 7he ada67erdrivers can lead 7= small val2es 8=r 42e2ede67h; and 82ll 42e2es =n 7he hdisk drivers.

S= 7here is 7he dedica7ed res=2rce a66r=ach; 7he shared res=2rce a66r=ach; and in 9e7?een dedica7ed andshared. Takin3 7his sim6le eam6le ?here ) re6resen7s 7he 42e2e sie 8=r 7he device driver

Fi32re #0@ J Dedica7ed ada67er 6=r7 42e2e sl=7 res=2rce

This ?=2ld 9e c=nsidered a dedica7ed res=2rce a66r=ach; ?here #@ =8 7he ada67er driver 42e2e sl=7s arededica7ed 7= each hdisk driver. $ere ?e kn=? ?ePll never s29mi7 an IO 7= a 82ll 42e2e =n 7he ada67er driver.

Al7erna7ivel

Fi32re '@# J Shared ada67er 6=r7 42e2e sl=7 res=2rce

This ?=2ld 9e c=nsidered a shared res=2rce a66r=ach ?here 7he #@ ada67er 42e2e sl=7s c=2ld 9e 8illed 268r=m a sin3le hdisk driver.

And herePs an eam6le =8 s=me7hin3 in 9e7?een


"a3e -#


42/46


Fi32re '' J Ada67er 6=r7 42e2e sl=7 res=2rce J 6ar7l dedica7ed; 6ar7l shared

$ere; 7here ?ill al?as 9e a7 leas7 5 42e2e sl=7s availa9le in 7he ada67er driver 8=r ei7her hdisk driver.

There are 6r=s and c=ns 7= each a66r=ach. The 9ene8i7 =8 7he dedica7ed res=2rce a66r=ach is 7ha7 7he res=2rces all=ca7ed ?ill al?as 9e availa9le 927 76icall 7here ?ill 9e 8e?er res=2rces availa9le 7= each 2ser

=8 7he res=2rce Ghere 7he res=2rce ?ePre c=nsiderin3 is 7he ada67er 42e2e sl=7s; and 7he 2sers =8 7he res=2rce are 7he hdisk drivers sendin3 IO re42es7s 8r=m 7he hdisk 42e2esH. The 9ene8i7 =8 7he shared res=2rce a66r=ach is 7ha7 ?ePll have m=re res=2rces 8=r an individ2al 2ser =8 7he res=2rce ?hen i7 needs i7and i7 ?ill 9e a9le 7= 3e7 3rea7er 7hr2627 7han in 7he dedica7ed res=2rce a66r=ach. The a27h=r 3enerall6re8ers a shared res=2rce a66r=ach; as i7 3enerall 6r=vides 7he 9es7 =verall 7hr2627 and 6rice 6er8=rmance.

=7e 7ha7 7his si72a7i=n =8 shared res=2rces =cc2rs in several 6=ssi9le ?as 9e=nd hdisk drivers 2sin3ada67er drivers. I7 is als= inv=lved ?hen

Several LV vSCSI hdisks 8=r a sin3le hdisk =n a VIOS

Several vFC ada67ers 2sin3 a sin3le real FC ada67er Several L"ARs 2sin3 7he same disk s29ss7em Several ss7ems 2sin3 SS"s vSCSI hdisks 2sin3 a vscsi ada67er

S= similar a66r=aches can 9e 2sed 8=r sharin3 =r dedica7ed 7h=se res=2rces as ?ell.

(eve 7aken 7?= a66r=aches 7= 72nin3 42e2es here. One a66r=ach 2ses IO s7a7is7ics 7= de7ermine h=? 9i37he 42e2es sh=2ld 9e; ?hile 7he =7her a66r=ach l==ks a7 7he 6=7en7ial n2m9er =8 IOs 7ha7 mi3h7 arrive 7= a42e2e. The la7er a66r=ach ?ill 9e m=re c=nserva7ive in av=idin3 42e2e ?ai7; and ?ill als= have lar3er avera3e n2m9er =8 re42es7s in 7he service 42e2e.

In 6rac7ice =ne sh=2ld 3enerall sie 7he servers and s7=ra3e 9ased =n 7he a66lica7i=n res=2rce needs; se7267he envir=nmen7; 7es7 i7s 6er8=rmance; and se7 7he 42e2es a66r=6ria7el 7hen; 7akin3 in7= acc=2n7 6lanned3r=?7h.

Ma7hema7icall; 7he IO 42e2es here are cl=ses7 7= an M1D1c 42e2e. M re6resen7s IO arrivals and is ass2med 7= have a "=iss=n dis7ri927i=n. D re8ers 7he 8ac7 7ha7 IOs arrivin3 7= 7he 42e2e ?ill need s=me 7ime7= service 7hem; and c re8errin3 7= 7he n2m9er =8 42e2e sl=7s ?here9 26 7= c IOs can 9e serviced in 6aral


"a3e -'


43/46


lel. An M1D1c42e2e m=del is de8ined as a s7=chas7ic 6r=cess ?h=se s7a7e s6ace is 7he se7 [@;#;';+;...\?here 7he val2e c=rres6=nds 7= 7he n2m9er =8 c2s7=mers in 7he ss7em; incl2din3 an c2rren7l in service.In 42e2in3 7he=r; an M1D1c 42e2e re6resen7s 7he 42e2e len37h in a ss7em havin3 cservers; ?here arrivals are de7ermined 9 a "=iss=n 6r=cess and =9 service 7imes are 8ied Gde7erminis7icH. Seeh77611en.?iki6edia.=r31?iki1M1D1k42e2e .


"a3e -+
http://en.wikipedia.org/wiki/M/D/k_queuehttp://en.wikipedia.org/wiki/M/D/k_queue


44/46


%. stimatin) A''lication 3er!ormance Im'rovement

"r=ec7in3 a66lica7i=n 6er8=rmance im6r=vemen7 9ased =n disk la7enc im6r=vemen7; 8r=m 7he a66lica7i=ns 6=in7 =8 vie?; has challen3es. S=me =8 7he challen3es incl2de

Disk IOs are =87en d=ne in 6arallel

Man IOs are d=ne asnchr=n=2sl and d=n7 9l=ck 7he a66lica7i=n; ?hile 7he a66lica7i=n is9l=cked 8=r man =7her IOs

(ri7es 7end 7= 9l=ck a66lica7i=ns m=re 7han reads

IO la7enc im6r=vemen7s 8=r reads and ?ri7es di88er

ever7heless; 9 makin3 s=me ass2m67i=ns; ?e can crea7e an es7ima7e; =r a ran3e =8 6=7en7ial im6r=vemen7; 2sin3 Amdahls la?.

Ass2min3 ?e can im6r=ve IO la7enc LX and 7he disk 6ar7 =8 7he =9 is DX; 7hen 7he 6=7en7ial 6er8=rmance im6r=vemen7 G8=r a 9a7ch =9; =r 8=r a66lica7i=n res6=nseH is DL. E.,.; ass2min3 IO service 7imeavera3es & ms; ?ai7 7ime in 7he 42e2e avera3es + ms; and 7ha7 ?e can elimina7e 7ime in 7he 42e2e; 7hen L+1G&W+H +@XQ 7h2s im6r=vin3 la7enc +@X 8r=m 7he a66lica7i=ns 6=in7 =8 vie?. Ass2min3 7he disk?=rk is 5@X =8 7he =9 7ime; 7hen ?e can r2n 7he =9 in 5@X +@X #5X less 7ime.

(i7h=27 kn=?in3 7he 6=r7i=n =8 7he =9 7ha7 is disk rela7ed; ?e d=n7 kn=? 7he ac72al im6r=vemen7; 927 i7627s 7he 6=7en7ial im6r=vemen7 in7= 7he ran3e =8 @L; and a c=26le =8 da7a 6=in7s a7 di88eren7 la7encies Gasseen 9 7he a66lica7i=nH can 6r=vide an es7ima7e =8 D.

"er8=rmance im6r=vemen7 7= a 9a7ch =9 is re8lec7ed in 7he r2n 7ime. F=r OLT" a66lica7i=ns; red2c7i=nsin IO la7enc are re8lec7ed in a66lica7i=n res6=nse 7ime 7= a 42er126da7e; =r 7= maim2m a66lica7i=n7hr2627. Since 7he n2m9er =8 in8li3h7 7ransac7i=ns is =87en limi7ed 9 mem=r; 7he a9ili7 7= d= 7ransac 7i=ns 8as7er means ?e essen7iall increase 7he ma T"S 8=r an L"AR 9 red2cin3 IO la7enc G=r al7erna7ivel ?e c=2ld red2ce ss7em res=2rces and kee6 7he same T"S ?i7h 8as7er IOsH.

An=7her me7ric ?e can calc2la7e is h=? m2ch IO 7ime ?e can save 9 elimina7in3 7he 42e2e ?ai7 7ime

IO"S Uav3 7ime in 7he 42e2e 6er IO IO la7enc savin3s 6er sec=nd

2s7 9e a?are 7ha7 7his isn7 7ime savin3s 8=r 7he a66lica7i=n. E.,.; i8 ?ere ?ai7in3 in 7he 42e2e =n avera3e ' ms 8=r each IO; and ?ere d=in3 #@;@@@ IO"S; 7he savin3s in IO la7enc ?=2ld 9e '@ sec=nds 6ersec=nd. $=?ever ?e can7 save m=re 7han # sec=nd ever sec=nd; and 9eca2se 7he IOs are d=ne in 6arallelQ 7h2s; 7he real a66lica7i=n savin3s is s=me7hin3 less 7han =ne sec=nd ever sec=nd. B27 7his me7ric isan indica7=r ?he7her 72nin3 ?ill hel6 6er8=rmance si3ni8ican7l and ?he7her 7he 72nin3 e88=r7 is ?=r7h 7he7ime.


"a3e --


45/46



"a3e -5


46/46


A''endi4 Related 3u"lications

The 629lica7i=ns lis7ed in 7his sec7i=n are c=nsidered 6ar7ic2larl s2i7a9le 8=r a m=re de7ailed disc2ssi=n =87he 7=6ics c=vered in 7his ?hi7e 6a6er.

AI% 5.+ "er8=rmance Mana3emen7 ,2ideN; IBM

AI% /.# "er8=rmance Mana3emen7 ,2ideN; IBM

AI% &.# "er8=rmance Mana3emen7 ,2ideN; IBM

"=?erVM Vir72alia7i=n =n IBM Ss7em 6 In7r=d2c7i=n and C=n8i32ra7i=n; IBM Red9==ks

h77611???.red9==ks.i9m.c=m1red6ieces1a9s7rac7s1s3'-&[email protected]

"=?erVM Vir72alia7i=n =n IBM Ss7em 6 Mana3in3 and M=ni7=rin3; IBM Red9==ks

h77611???.red9==ks.i9m.c=m1red6ieces1a9s7rac7s1s3'-&[email protected]

Fi9rechannel I1O "er8=rmance T2nin3 =n AI% 2sin3 8cs7a7Q A $=?7= and !sa3e ,2ideh77611?+@+.i9m.c=m1s266=r717echd=cs1a7smas7r.ns81(e9Inde1TD#@/#''

IBM Ss7em 6 and AI% In8=rma7i=n Cen7erh77611629li9.9=2lder.i9m.c=m1in8=cen7er16series1v5r+1inde.s6 GAI% 5.+Hh77611629li9.9=2lder.i9m.c=m1in8=cen7er1ss7ems1sc=6e1ai1inde.s6 GAI% /.#Hh77611629li9.9=2lder.i9m.c=m1in8=cen7er1ai1v&r#1inde.s6 GAI% &.#H
http://www.redbooks.ibm.com/redpieces/abstracts/sg247940.htmlhttp://www.redbooks.ibm.com/redpieces/abstracts/sg247590.htmlhttp://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsphttp://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsphttp://publib.boulder.ibm.com/infocenter/systems/scope/aix/index.jsphttp://publib.boulder.ibm.com/infocenter/systems/scope/aix/index.jsphttp://publib.boulder.ibm.com/infocenter/aix/v7r1/index.jsphttp://www.redbooks.ibm.com/redpieces/abstracts/sg247940.htmlhttp://www.redbooks.ibm.com/redpieces/abstracts/sg247590.htmlhttp://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsphttp://publib.boulder.ibm.com/infocenter/systems/scope/aix/index.jsphttp://publib.boulder.ibm.com/infocenter/aix/v7r1/index.jsp

aix vios diskandadapterqueuetuningv1.2

Documents