injecting realistic burstiness to a traditional client-server benchmark
DESCRIPTION
Injecting Realistic Burstiness to a Traditional Client-Server Benchmark. Ningfang Mi College of William and Mary Giuliano Casale SAP Research Ludmila Cherkasova Hewlett-Packard Labs Evgenia Smirni College of William and Mary - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/1.jpg)
© 2006 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice
Injecting Realistic Burstiness to a Traditional Client-Server Benchmark
Ningfang Mi College of William and Mary
Giuliano Casale SAP Research
Ludmila Cherkasova Hewlett-Packard Labs
Evgenia Smirni College of William and Mary
Presenter: Lucy Cherkasova
![Page 2: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/2.jpg)
2 International Conference on Autonomic Computing and Communications (ICAC) 2009
Origin of Burstiness
• Enterprise and Internet applications:
Clients DB Server
Front Server
Web + Application
Server
HTTP request
HTTP reply
SQL query
SQL reply
Burstiness
??
Highly Correlated Arrivals
?
![Page 3: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/3.jpg)
3 International Conference on Autonomic Computing and Communications (ICAC) 2009
Client-Server Benchmark
• E.g., TPC-W (On-line bookstore Web site)
• Exponentially distributed user think timesExponentially distributed user think times
Clients DB Server
Front Server
Web + Application
Server
HTTP request
HTTP reply
SQL query
SQL reply
Burstiness
??
Highly Correlated Arrivals
?
![Page 4: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/4.jpg)
4 International Conference on Autonomic Computing and Communications (ICAC) 2009
• Accounts for randomness and variability … • … but not for burstinessbut not for burstiness
Can we ignore burstiness in the arrival process?
Typical Client-Server Benchmark
BurstinessBurstinessVariabilityVariability
Serv
ice t
ime
Serv
ice t
ime
Request number Request number
![Page 5: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/5.jpg)
5 International Conference on Autonomic Computing and Communications (ICAC) 2009
Why Need to Inject Burstiness?
• Burstiness impacts the performance of resource allocation mechanisms.
• Example: Session-based admission control (SBAC)−User session: sequence of transaction requests−Session is a unit of work−Typically, long sessions are “sales”.−Useful system throughput is the number of
completed sessions−Admission controller admits/rejects sessions
based on observed CPU utilization of the server (a combination of last measurement and some history).
L. Cherkasova, P. Phaal. Session Based Admission Control: a Mechanism for
Peak Load Management of Commercial Web Sites. IEEE J. TOC, June 2002.
![Page 6: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/6.jpg)
6 International Conference on Autonomic Computing and Communications (ICAC) 2009
SBAC
Reject a new session when utilization is above the threshold
Abort an accepted session when the server queue is full
highly undesirable
Front ServerWeb +
ApplicationServer
DB Server
New Client Arrival
Requests from already accepted clients
limited server queue
![Page 7: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/7.jpg)
7 International Conference on Autonomic Computing and Communications (ICAC) 2009
Impact of Burstiness
• We performed experiments for the same workload with different arrival patterns: non-bursty vs bursty
• Aborted ratio = aborted sessions/accepted sessions
highly undesirable
Queue Size Non-bursty Bursty
250 0.04% 11.37%
512 0.00% 6.28%
800 0.00% 2.50%
![Page 8: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/8.jpg)
8 International Conference on Autonomic Computing and Communications (ICAC) 2009
Why Need to Inject Burstiness? (2)• Service level agreement (SLA)
−support given response time guarantees for accepted sessions
• SLA of 1.2s can be supported for 98% of requests with queue size =250 for non-bursty traffic
• Only 90% of requests meet SLA=1.2s bursty traffic.
0
0.2
0.4
0.6
0.8
1
1.2
250 512 800
90th
95th
98th
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
250 512 800
90th
95th
98th
Queue Size Queue Size
Resp
onse
Tim
e (
s)
Resp
onse
Tim
e (
s)
Non-Bursty Bursty
![Page 9: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/9.jpg)
9 International Conference on Autonomic Computing and Communications (ICAC) 2009
Limitations of Standard TPC-W
• Think times are drawn randomly from the exponential distribution identical for all clients
• Exponential think times are incompatibleincompatible with the notion of burstiness.
Need to inject burstiness into user think times.
![Page 10: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/10.jpg)
10 International Conference on Autonomic Computing and Communications (ICAC) 2009
Our Methodology
•Basic Idea: modify the distribution of client think time to create bursty arrivals−Regulate the arrivals by using a 2-phase
Markovian Arrival ProcessMarkovian Arrival Process (MAP).• MAPs are variations of popular On/OFF traffic
models that can be easily shaped to create correlated inter-arrival times
• All clients share a MAP(2) to draw think times
• A new module for client-server benchmarks
−Regulate the intensity of traffic surges by using the index of dispersionindex of dispersion. • A simple tunable knob of burstiness
![Page 11: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/11.jpg)
11 International Conference on Autonomic Computing and Communications (ICAC) 2009
Index of Dispersion (I)• Popular burstiness index in networking• Definition
− SCV – the squared coefficient of variation (variance/mean2)− ρk – autocorrelation coefficients
• i.e., correlation of service times− Exponential: I = SCV = 1
)21(1
k
kSCVI variabilityburstines
s
BurstinessBurstinessVariabilityVariability
Serv
ice
tim
e
Serv
ice
tim
e
Request number Request number
![Page 12: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/12.jpg)
12 International Conference on Autonomic Computing and Communications (ICAC) 2009
Markovian Arrival Process (MAP)
• MAPs have ability to provide variabilityvariability and temporal localitytemporal locality.
• We use a class of MAPs with two states only
Normal
Traffic
λlong
Traffic Surge
λshort
2 states: λshort > λlong
pl,s
ps,l
ps,spl,l
time
Num
. of
arr
ivals
pl,s, ps,l, ps,s, pl,s shape correlation
![Page 13: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/13.jpg)
13 International Conference on Autonomic Computing and Communications (ICAC) 2009
MAP Fitting
• Input − Estimated mean service demands at servers: E[Di]
− Mean user think time E[Z]
− The pre-defined index of dispersion I
• Output− A MAP(2) to draw user think times
![Page 14: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/14.jpg)
14 International Conference on Autonomic Computing and Communications (ICAC) 2009
MAP Fitting (2)
Key: determine (Key: determine (λλshortshort,, λλlonglong, , ppl,sl,s,, p ps,ls,l))• Condition for traffic surge
• Condition for normal traffic
• Mean think time
• We use non-linear optimizer to search for such f and ps,l and find a MAP(2) to best match the predefined I
fDEi ishort /)(1
])[),(max(1 ZEDENfi ilong
)][
][(
1
1
,,
short
longlssl ZE
ZEpp
Departure > Arrival
Arrival > Departure the arrival rate is f times higher than the throughput of the system
the arrival rate is f times slower for balanced system throughput
Balancing the height and the width of the burst
![Page 15: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/15.jpg)
15 International Conference on Autonomic Computing and Communications (ICAC) 2009
Realistic values for Burstiness
−What is the range of realistic values for defining burstiness via index of dispersion I ? • Exponential: I = SCV = 1
• Bursty: values of thousands,
−e.g., FIFA World Cup 1998, one of the servers over 10 days, I = 6300
![Page 16: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/16.jpg)
16 International Conference on Autonomic Computing and Communications (ICAC) 2009
TPC-W Testbed
• On-line bookstore Web site • Testbed: clients + front server + DB server
−Constant number of emulated browsers (EBs)
• User session−sequence of transaction requests
−think time (mean=7 sec) between two transaction requests
• 14 transactions types grouped in three mixes:−Browsing mix
−Shopping mix
−Ordering mix
![Page 17: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/17.jpg)
17 International Conference on Autonomic Computing and Communications (ICAC) 2009
Validation – Arrival Process
• Arrival clients to the system (front server)
Think times drawn by a MAP(2) with I create the bursty conditions.
Shopping Mix
Non-bursty (I=1)
Time (s)
Num
ber
of
act
ive c
lients Bursty (I=4000)
Time (s)
Num
ber
of
act
ive c
lients
![Page 18: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/18.jpg)
18 International Conference on Autonomic Computing and Communications (ICAC) 2009
Validation – Utilization DistributionShopping Mix
Non-bursty (I=1) Bursty (I=4000)
pd
fpd
f
pd
fpd
f
Utilization (%)Utilization (%)
Utilization (%) Utilization (%)
Front
DB
![Page 19: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/19.jpg)
19 International Conference on Autonomic Computing and Communications (ICAC) 2009
Validation - Average Latency
0
500
1000
1500
2000
2500
3000
3500
200 400 600 800 1000 1200Number of EBs
non- bursty
I=4000
Browsing Mix
Resp
onse
tim
e (
ms)
0
200
400
600
800
1000
1200
1400
1600
200 400 600 800 1000 1200Number of EBs
non- bursty
I=4000
Shopping Mix
Resp
onse
tim
e (
ms)
![Page 20: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/20.jpg)
20 International Conference on Autonomic Computing and Communications (ICAC) 2009
Validation – Latency Distributions
0%
20%
40%
60%
80%
100%
0 2000 4000 6000 8000 10000
non-bursty
I=4000
0%
20%
40%
60%
80%
100%
0 1000 2000 3000 4000 5000
non-bursty
I=4000
Browsing Mix
CD
F
Shopping Mix
Response time (ms) Response time (ms)
CD
F
0.83
2.98
0.04
1.25
![Page 21: Injecting Realistic Burstiness to a Traditional Client-Server Benchmark](https://reader036.vdocuments.net/reader036/viewer/2022062408/568139fd550346895da1c51c/html5/thumbnails/21.jpg)
21 International Conference on Autonomic Computing and Communications (ICAC) 2009
Conclusion• Burstiness critical for autonomic system design
− need representative benchmarks for system evaluation− need reproducible and controllable bursty workloads
• Traditional client-server benchmarks ignore burstiness in arrival flows− e.g., TPC-W with exponential think times
• Explicitly inject burstiness − a simple and tunable parameter: index of dispersion− can introduce different intensity of traffic surges
• http://www.cs.wm.edu/~ningfang/tpcw_codes/
• Supported by NSF grants CNS-0720699 and CCF-08114171 and HPLabs gift.