palm calculus part 1 - performance evaluationperfeval.epfl.ch/printme/palm1post.pdf · palm...

29
Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec 1 May 2015

Upload: others

Post on 04-Jul-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Palm CalculusPart 1

The Importance of the Viewpoint

JYLeBoudec

1May2015

Page 2: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

1. Event versus Time AveragesConsider asimulation,stateStAssumesimulationhasastationary regime

Consider anEventClock:timesTn atwhich some specific changesofstateoccurEx:arrival ofjob;Ex.queuebecomes empty

Eventaverage statistic

Timeaverage statistic

2

Page 3: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Example: Gatekeeper; Average execution time

3

0 90 100 190 200 290 300

50001000

Real time t (ms)

job arrival

50001000

50001000

Execution time for a job thatarrives at t (ms)

Viewpoint 1: System Designer Viewpoint 2: Customer

Page 4: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Example: Gatekeeper; Average execution time

4

0 90 100 190 200 290 300

50001000

Real time t (ms)

job arrival

50001000

50001000

Execution time for a job thatarrives at t (ms)

Viewpoint 1: System Designer Viewpoint 2: Customer

Two processes, with execution times 5000 and 1000

5000 10002 3000

Inspector arrives at a random timered processor is used with proba

90100 5000

10100 1000

4600

Page 5: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Sampling Bias

Ws andWc aredifferentAmetricdefinitionshouldmentionthesamplingmethod(viewpoint)Differentsamplingmethodsmayprovidedifferentvalues:thisisthesamplingbias

PalmCalculus isasetofformulasforrelatingdifferentviewpoints

CanoftenbeobtainedbymeansoftheLargeTimeHeuristic

5

Page 6: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large Time Heuristic Explainedon an Example

Wewant torelate andWe apply thelargetimeheuristic

1. Howdowe evaluate these metrics inasimulation?

where indexofnext greenorred arrow at orafter6

Page 7: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large Time Heuristic Explainedon an Example

2. Breakoneintegral into pieces that matchthe ’s:

7

Page 8: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large Time Heuristic Explainedon an Example

2. Breakoneintegral into pieces that matchthe ’s:1

…1

1⋯

1⋯

1 ⋯

1⋯

8

Page 9: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large Time Heuristic Explainedon an Example

3. Compare

9

Page 10: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large Time Heuristic Explainedon an Example

3. Compare

10

Page 11: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

This is Palm Calculus !

11

cov ,

Page 12: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

In which case do we expect to see 

A. Sn=90,10,90,10,90;Xn=5000,1000,5000,1000,5000B. Sn =90,10,90,10,90;Xn =1000,5000,1000,5000,1000C. BothD. NoneE. Idon’tknow

12

Sn = 90, 10, 90, 10, 90; Xn 

Sn = 90, 10, 90, 10, 90; Xn 

Both

None

I don’t know

72%

0%0%

28%

0%

Page 13: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Solution

IncaseA, and arepositivelycorrelated(whentheintervalislong,soistheprocessingtime),i.e. .BythePalmcalculusformula:

IncaseB,thecorrelationisnegative,therefore

AnswerA

13

Page 14: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

The Large Time Heuristic

Formallycorrectifsimulationisstationary

Itisarobustmethod,i.e.independentofassumptionsondistributions(andonindependence)

14

Page 15: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Other «Clocks»

15

Flow 1 Flow 2

Flow 3

Distribution of flow sizesfor an arbitrary flowfor an arbitrary packet

Page 16: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Which curves are for the per‐packet 

viewpoint ?

A. AB. BC. ItdependsD. Idon’tknow

16

A B

It depends

I don’t know

71%

18%

0%

12%

Page 17: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Solution

AnswerATherearemorepacketsinthelargeflows.Somorepacketsexperiencealargeflowsize.

17

Page 18: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

18

Load Sensitive Routing of Long-Lived IP FlowsAnees Shaikh, Jennifer Rexford and Kang G. ShinProceedings of Sigcomm'99

ECDF, per flow viewpoint

ECDF, per packet viewpoint

Page 19: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Meanflowsize:perflowperpacket

19

Flow 1 Flow 2

Flow 3

Distribution of flow sizesfor an arbitrary flowfor an arbitrary packet

Page 20: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large «Time» Heuristic

1. Howdowe evaluate these metrics inasimulation?

perflow ∑

perpacket ∑where when packet belongs toflow

20

Page 21: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large «Time» Heuristic

1. Howdowe evaluate these metrics inasimulation?perflow ∑

perpacket ∑where when packet belongs toflow

2. Putthepackets side byside,sorted byflow

21

Flow n=1 Flow n=2 Flow n=3

p=1 p=2 p=3 p=4 p=5 p=6 p=7 p=8 p=9

1⋯

1⋯

1

Size Size Size

Page 22: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large «Time» Heuristic

3. Compare

22

Flow n=1 Flow n=2 Flow n=3

p=1 p=2 p=3 p=4 p=5 p=6 p=7 p=8 p=9

1

1 1

1 1 1 1var

1var

Size Size Size

Page 23: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Large «Time» Heuristic for PDFs of flow sizes

Putthepackets side byside,sorted byflow

1. Howdowe evaluate these metrics inasimulation?

23

Flow n=1 Flow n=2 Flow n=3

Page 24: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

24

Page 25: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

25

Page 26: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

Cyclist’s Paradox

Onaroundtriptour,thereismoreuphillthandownhill

26

Page 27: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

The km clock vs the standard clock

ℓ speedfortheℓ kilometer1

ℓℓ

meanof ℓ

∑ 1ℓℓ

harmonicmeanof ℓ meanof ℓ

27

Page 28: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

BorduRail claims that only 5% of trains arrivals are late

BorduKonsum claims that 30% of train users suffer from late train 

arrivals

A. AtleastoneofthemliesB. Thenumberofpassengersina

latetrain passengersinaveragetrain

C. Thenumberofpassengersinalatetrain passengersinaveragetrain

D. Idon’tknow

28

At least o

ne of the

m lie

The nu

mbe

r of p

assengers i.

The nu

mbe

r of p

assengers in

I don

’t know

0% 0%

44%

56%

Page 29: Palm Calculus Part 1 - Performance Evaluationperfeval.epfl.ch/printMe/palm1Post.pdf · Palm Calculus Part 1 The Importance of the Viewpoint JY Le Boudec May 2015 1. 1. Event versus

29

Solution