lbe tests between infn & ucl

1

LBE Tests between INFN & UCL

http://www.cisco.com/

http://www.ukerna.ac.uk/

http://www.spirentcom.com/

2

Contents

• Purpose/Scenario • Equipment/Topology• UDP TESTS

– Per frame-size 1:1 proportion (2 flows)– Per frame-size 5:5, 7:3, 9:1 proportion– Visual summary of results

• TCP TESTS (ongoing…..)• Conclusions• Future work

3

2 Possible test Scenarios

• Bottleneck(s) in the core network [Activity 2 in Project GN1 (Geant);D9.9].

• Bottleneck(s) in the edge-networks (NRENs) only: This is supported by the idea that a network can deploy

LBE incrementally, in points where congestion is more likely to occur (currently in the NRENs only)

4

Objectives…..

• Assess– BE protection under congestion due to the

LBE attack in terms of:• Throughput (IP and TCP layer)• Packet Loss• Jitter (OWD Standard Deviation)

– OWD Distribution buckets were engineered to achieve a compromise between lost of accuracy (buckets too large) and the network-noise/clocks non-synchronization vulnerability (buckets too small)

• Reordering (mainly for BE)…ongoing

5

Edges Equipment

NRENs + GEANT (“Overprovisioned”…some noise to filter out)

• 1 x 7200 Cisco Router. (1x GigE + 3 x feth + 1 x eth )

• 1 x SmartBits 6000B chassis (with 12 x 10/100 BaseT and 2 x GigE) in each site

• Italian site: 2 Linux boxes dual-processor 2.4.9 (red hat 7.3)

• UK site: 1 Supermicro super server 6002P6, Dual Intel Processor, 1G memory, Intel card

100Mb

2x100Mb 1 Gbps

Policy Applied

Cisco 7200

100Mb

100Mb

6

Mngm_port128.40.4.159

1A6: 128.40.255.33/30

7200

fe1/0:128.40.255.34 /30128.40.255.234/30 secondary

ge0/0:128.40.169.217 /29 SW:6500

mbng1/pc58MBNG2

JANET

GEANT

GARRCabletronSS8600

1Gb

1Gb100Mb

1 Gb

GW:131.154.99.253/24

Cabletron Smart Switch Router2000

mbng1/pc58Sunlab3f

mbng1/pc58Sunlab2f

fe:131.154.99.3/24

fe:131.154.99.2/24

1A1: 131.154.99.72/24

Mngm_port: 131.154.99.73

: Gigabit1A2: 131.154.99.74/24

eth1: 128.40.255.233/30

HUB

GW:128.40.169.222

100Mbps Bottleneck !!!Policy applied 99% BE 1% LBE

7

64 100 744047 550478 193569 26.0157 N/A 110095.6 40515181

128 100 422297 422297 0 0 N/A 84459.4 74324272

256 100 226448 226448 0 0 50 45,290 86,231,398

384 100 185642 154782 30860 16.62 60 30,956 90,640,339

512 100 117481 117481 0 0 N/A 23496.2 92856982

Per frame size 100Mbps 1BEflow Throughput achieved

8

Test Layout1 LBE Flow

1 BE Flow

Frame sizes set :

64,128,256,384,512,1180,1500,1518

Load Set ;

from 10Mbps…to 100Mbps per port

100Mbps

100Mbps

9

• 0 packets dropped both in the BE and LBE output queues of the router.

• 0 packets dropped in the router input interface

• 0 packets ignored

• 8184514-4194010=3990504 frame lost in the network..48.7% of the sent packets

• 35Mbps is the Max. IP throughput for 40Mbps Link!!!! sure not for 100Mbps…..Clearly we can’t go above the associated input rate which is

[35.22Mbps]/[8*(64-18)]=95706 packets/sec!!!

NO DIFFERENTIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues……..

Traffic is seriously blocked by the network….the maximum IP level throughput for a 64 Bytes frame size flow on a 100Mbps link would be 54.7 Mbps…..and we reach 35Mbps………..

Similar router (we saw the same in bck2bck) along the path reached their Max. Input rate !!!! (95706 packets/sec)

35Mbps

64UDP 64Fr.Size 1:1 BE+LBE

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

35,000,000

40,000,000

offered wire rate load

IP R

x.t

hro

ug

hp

ut

UDP_BE

UDP_LBE

Total

UDP_BE 5,251 10,95 16,42 17,10 17,07 17,08 17,28 17,49 17,45 17,51

UDP_LBE 5,251 10,95 16,42 17,27 17,27 17,30 17,54 17,67 17,58 17,71

Total 10,50 21,90 32,85 34,37 34,35 34,38 34,83 35,16 35,04 35,22

10 20 30 40 50 60 70 80 90 100

UDP 64f.s. 1:1 BE+LBE

0

10

20

30

40

50

60

70

80

wire rate offered traffic (Mbps)

Pa

ck

et

Lo

ss

%

UDP_BE

UDP_LBE

UDP_BE 0 0 0 21.9 37.6 48 54.9 60.1 64.6 68

UDP_LBE 0 0 0 21.2 36.9 47.3 54.2 59.7 64.3 67.7

10 20 30 40 50 60 70 80 90 100

10

1. 0 packets dropped both in the BE and LBE output queues of the router.

2. 0 packets dropped in the router input interface

3. 179884 packets ignored..5,3% of all- rate packets that entered the input interface (higher % for the high rate packets)

4. 4645260 – 3380105 = 1265155 frame lost in the network..27% of the frame sent

5. At least….74.3 Mbps IS the Max. IP throughput possible for a 128f.s. on a 100Mbps link

Traffic is partially blocked by the network….the remaining is then blocked at the input interface where that size provokes a lot of packet ignored…

If the 64B packets had arrived at the router…we would have seen packets both ignored and flushes……he same thing would have happened for 128B packets…

NO DIFFERENTIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues

74.3Mbps

128UDP 128Fr.Size Throughput 1:1 BE+LBE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

wire rate offered load (Mbps)

Rx.

IP

Th

rou

gh

pu

t

UDP_BE

UDP_LBE

Total

UDP_BE 7,430 14,86 22,29 29,72 37,15 37,02 36,97 37,01 36,99 36,98

UDP_LBE 7,429 14,86 22,29 29,72 37,15 37,34 37,29 37,41 37,41 37,43

Total 14,85 29,72 44,59 59,45 74,31 74,36 74,26 74,42 74,40 74,41

10 20 30 40 50 60 70 80 90 100

11

0 packets dropped both in the BE and LBE output queues of the router.

0 packets dropped in the router input interface

48767 packets ignored….2.6% of the all-rate packets that entered the input interface (higher % for high rate packets)

2490930 - 1813031 = 677899 frame lost in the network..27% of the packets sent

86.2Mbps is the Max. IP throughput possible for a 256f.s. on a 100Mbps link

NO DIFFERENZIATION: traffic can’t arrive at the router in order to congest the output diff. serv. queues

86.2Mbps

Traffic is partially blocked by the network….the remaining is then blocked at the input interface where that size provokes packet ignored…

256UDP 256f.s. 1:1 BE+LBE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000

100,000,000


Rx.

IP

th

rou

gh

pu

t (b

ps)

UDP_BE

UDP_LBE

Total

UDP_BE 8,617 17,24 25,86 34,49 43,11 43,10 43,06 43,05 43,05 43,06

UDP_LBE 8,617 17,24 25,86 34,49 43,11 43,23 43,22 43,25 43,25 43,28

Total 17,23 34,49 51,73 68,98 86,23 86,34 86,29 86,31 86,31 86,35

10 20 30 40 50 60 70 80 90 100

Less than for 128…..

12

1. 0 packets dropped in the BE queue BUT 232844 packets dropped in the LBE output queue of the router.


3. 2374 packets ignored in the input interface…1.6% of the packets that entered the interface (higher % for high rate packets)

4. 1701722 - 1471758 = 229964 frame lost in the network…13.5%

5. 89.6Mbps would be the IP throughput BE should get at 200Mbps offered load according to 384f.s. and to the policy applied (99%)…

PARTIAL DIFFERENZIATION: traffic arrive at the router in order to congest the output diff. serv. Queues but it is still dropped in the network and ignored in the input interface as well…..

UDP 384 f.s.1:1 BE+LBE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000

100,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

UDP_BE

UDP_LBE

Total

UDP_BE 9,049 18,11 27,17 36,23 45,29 54,35 60,08 60,06 60,00 60,05

UDP_LBE 9,051 18,11 27,17 36,23 45,29 36,26 30,63 30,68 30,63 30,69

Total 18,10 36,23 54,35 72,47 90,59 90,61 90,72 90,75 90,64 90,75

10 20 30 40 50 60 70 80 90 100

384

Loss %

0

10

20

30

40

50

60

70

wire rate offered load Mbps

% o

f lo

ss

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0.01 5.25 17.1 26.4 33.7

UDP_LBE 0 0 0 0 0 33.3 51.7 57.7 62.4 66.1

10 20 30 40 50 60 70 80 90 100

Less than 256

13

1. 0 packets dropped in the BE queue BUT 339579 packets dropped in the LBE output queue of the router.


3. 0 packets ignored in the input interface

4. 1292284- 1045468 = 246816 frames lost in the network…19% of the sent frames

5. 91.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…the target throughput is 91.8 – 88.07= 3.1Mbps far…

UDP 512 Th. 1:1 BE+LBE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000

100,000,000


Rx

IP T

hro

ug

hp

ut

(bp

s)

UDP_BE

UDP_LBE

Total

UDP_BE 9,274 18,57 27,85 37,14 46,42 55,71 64,99 74,28 83,51 88,07

UDP_LBE 9,274 18,57 27,85 37,14 46,40 37,23 27,98 18,69 9,427 4,943

Total 18,54 37,14 55,71 74,28 92,83 92,95 92,97 92,97 92,94 93,01

10 20 30 40 50 60 70 80 90 100

UDP 512 Loss 1:1 BE+LBE

0

10

20

30

40

50

60

70

80

90

100


Lo

ss

% UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0.01 0 0 0 0.06 5.15

UDP_LBE 0.12 0 0 0 0.04 33.2 57 74.8 88.7 94.7

10 20 30 40 50 60 70 80 90 100

UDP 512 St.Dev. 1:1 BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000


Sta

ndar

d de

viat

ion

(Mic

rose

c.)

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0 0 0 0 0

UDP_LBE 0 0 0 0 0 0 1219 5686 5281 1655

10 20 30 40 50 60 70 80 90 100

512

14

1. 464 packets dropped in the BE queue!!!!...traffic

2. arrives finally



5. 0 frames lost in the network

6. 95.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE gets]….it’s ok!!!


0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx.

IP th

roug

hput

(bp

s)

UDP_BE

UDP_LBE

Total

UDP_BE 9,654 19,36 29,05 38,73 48,41 58,12 67,78 77,44 87,15 95,92

UDP_LBE 9,658 19,36 29,05 38,73 48,41 38,85 29,21 19,52 9,840 1,169

Total 19,31 38,73 58,10 77,46 96,83 96,98 96,99 96,97 96,99 97,09

10 20 30 40 50 60 70 80 90 100


0

20

40

60

80

100

120


Pac

ket

Lo

ss %

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0 0 0.02 0 0.93

UDP_LBE 0 0 0 0 0 33.1 56.9 74.8 88.7 98.8

10 20 30 40 50 60 70 80 90 100

UDP 1180 St.Dev. 1:1 BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

10 20 30 40 50 60 70 80 90 100


OW

D S

tand

ardr

Dev

iatio

n (M

icro

sec.

)UDP_BE

UDP_LBE

1180

15

1. 357 packets dropped in the BE queue




5. 96.52Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE get]…..it’s ok!!!


0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

UDP_BE

UDP_LBE

UDP_BE 9,717 19,49 29,24 38,99 48,74 58,53 68,24 77,99 87,73 96,65

UDP_LBE 9,712 19,49 29,24 38,99 48,74 39,15 29,43 19,72 9,947 1,221

10 20 30 40 50 60 70 80 90 100

UDP 1500 Loss. 1:1 BE+LBE

0

20

40

60

80

100

120


Packet

Lo

ss %

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0 0 0 0 0.9

UDP_LBE 0.4 0 0 0 0 33 57 75 89 99

10 20 30 40 50 60 70 80 90 100

UDP 1500f.s. St.Dev. 1:1 BE+LBE

0

5000

10000

15000

20000

25000

10 20 30 40 50 60 70 80 90 100


OW

D S

tan

da

rd D

ev

iati

on

(M

icro

se

c.)

UDP_BE

UDP_LBE

1500

16





5. 97.02Mbps would be the IP throughput BE should get at 200Mbps offered load according to 1518f.s. and to the policy applied (99%) [it also equalises what 100M BE get]….nearly 0.3 Mbps missing!!!


0

20

40

60

80

100

120


Pa

ck

et

Lo

ss

%

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0 0 0 0 0.9

UDP_LBE 0 0 0 0 0 33 57 75 89 99

10 20 30 40 50 60 70 80 9010

0

UDP 1518f.s. Th. 1:1 BE+LBE

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

UDP_BE

UDP_LBE

UDP_BE 9,71 19,5 29,2 39,0 48,7 58,5 68,2 78,0 87,7 96,6

UDP_LBE 9,71 19,5 29,2 39,0 48,7 39,1 29,4 19,7 9,96 1,22

10 20 30 40 50 60 70 80 90 100

UDP 1518 St.Dev.1:1 BE+LBE

0

5000

10000

15000

20000

25000

10 20 30 40 50 60 70 80 90 100


OW

D S

tan

da

rd D

ev

iati

on

(M

icro

se

c.)

UDP_BE

UDP_LBE

1518

17

Zoom 512Zoom 512f.s. Th. 1:1 BE+LBE

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

w ire rate offered load (Mbps)

Rx. I

P Th

roug

hput

(bps

)

UDP_BE

UDP_LBE

UDP_BE 83,5 84,4 85,4 86,2 87,2 88,0 88,1 88,1 88,0 88,8 88,2

UDP_LBE 9,01 8,49 7,51 6,66 5,70 4,94 4,82 4,85 4,87 4,14 4,74

90 91 92 93 94 95 96 97 98 99 100

Zoom 512f.s. Loss 1:1 BE+LBE

0

20

40

60

80

100

120


Pack

et L

oss

%

UDP_BE

UDP_LBE

UDP_BE 0 0.06 0 0.07 0 0.16 1.12 2.11 3.22 3.39 4.93

UDP_LBE 89.2 90 91.2 92.3 93.5 94.4 94.6 94.6 94.6 95.5 94.9

90 91 92 93 94 95 96 97 98 99 100

Zoom 512 Standard Deviation 1:1 BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

90 91 92 93 94 95 96 97 98 99 100


OW

D St

anda

rd D

evia

tion

(Mic

rose

c.) UDP_BE

UDP_LBE

1. 91.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalizes what 100M BE get]…nearly 4Mbps missing…

18

Zoom 1180

Zoom 1180f.s. OWDSt.Dev. 1:1 BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

90 91 92 93 94 95 96 97 98 99 100


OW

D S

tand

ard

Dev

iatio

n (M

icro

sec.

)UDP_BE

UDP_LBE

Zoom 1180f.s. Th. 1:1 BE+LBE

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx.

IP T

hrou

ghpu

t (bp

s)

UDP_BE

UDP_LBE

UDP_BE 83,5 88,1 89,0 90,0 91,0 91,9 92,9 93,9 94,8 95,8 95,8

UDP_LBE 9,39 8,88 7,90 6,93 5,98 5,01 4,06 3,07 2,12 1,17 1,21

90 91 92 93 94 95 96 97 98 99 100


0

20

40

60

80

100

120


Rx.

IP T

hrou

ghpu

t (bp

s)

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0 0 0 0 0 0 1

UDP_LBE 89 90 91 92 93 95 96 97 98 99 99

90 91 92 93 94 95 96 97 98 99 100

464 packets dropped in the BE queue!!!!...traffic arrives finally

0 packets dropped in the router input interface

0 packets ignored in the input interface

0 frames lost in the network

95.8Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE gets]….it’s ok!!!

19

Zoom 1500


0

20

40

60

80

100

120


Pa

ck

et

Lo

ss

%

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0.8707

UDP_LBE 95.858 96.67 97.727 98.735 98.748

96 97 98 99 100

Zoom 1500f.s. OWDStandarDeviation 1:1 BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

90 91 92 93 94 95 96 97 98 99 100


OW

D S

tan

dar

d D

evia

tio

n (

mic

rose

c.)

UDP_BE

UDP_LBE





5. 96.52Mbps would be the IP throughput BE should get at 200Mbps offered load according to 512f.s. and to the policy applied (99%)…[it also equalises what 100M BE get]…..it’s ok!!!


0

50000000

100000000

150000000


Rx. I

P Th

roug

hput

(bps

)

UDP_BE

UDP_LBE

UDP_BE 89783117 94572941 95549875 96524438 96650112

UDP_LBE 3876912 3148954 2172019 1221168 1221168

96 97 98 99 100

20

ZooZZZ

Zoom 1518


0

20

40

60

80

100

120


Pa

ck

et

Lo

ss

%

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0 0.8711273

UDP_LBE 95.667889 96.669034 97.903274 98.724832 98.747447

96 97 98 99 100





5. 97.02Mbps would be the IP throughput BE should get at 200Mbps offered load according to 1518f.s. and to the policy applied (99%) [it also equalises what 100M BE get]….nearly 0.3 Mbps missing….


0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

UDP_BE

UDP_LBE

UDP_BE 93,626,400 94,603,200 95,248,800 96,552,000 96,679,200

UDP_LBE 4,056,000 3,151,200 2,004,000 1,231,200 1,221,600

96 97 98 99 100

21

Interleaving(1)Interleaving frame sizes below 512 with frame size 1500

• 64 : NO for both combinations

• 128 : NO for both combinations

• 256 : “YES” (Great improvement!!)…but only if BE is 1500 and LBE is 256…….see next slide…..

• 384 : PERFECT for both combinations

• 512 : PERFECT for both combinations

22

Interleaving(2)UDP BE 1500 LBE 256

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000

100,000,000


Rx

. IP

Th

rou

gh

pu

t (M

bp

s)

UDP_BE

UDP_LBE

UDP_BE 38,99 48,74 58,49 68,23 77,97 86,49 87,51

UDP_LBE 34,49 43,11 34,51 25,87 17,29 8,581 8,877

40 50 60 70 80 90 100

UDP BE 1500 + LBE 256

0

10

20

30

40

50

60

70

80

90

100

wire offered rate (Mbps)

pac

ket

Lo

ss %

UDP_BE

UDP_LBE

UDP_BE 0 0 0 0.02 0.03 1.43 10.24

UDP_LBE 0 0.01 33.3 57.13 74.93 88.94 89.71

40 50 60 70 80 90 100

UDP BE 256 + LBE 1500

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

40 50 60 70 80 90 100


Rx.

IP

Th

rou

gh

pu

t (M

bp

s)

UDP_BE

UDP_LBE

UDP BE 256 + LBE 1500

0

10

20

30

40

50

60

70

80

90

wire rater offered load (Mbps)

Pa

ck

et

Lo

ss

%

UDP_BE

UDP_LBE

UDP_BE 0 0 1.01 0.02 4.46 15.09 23.51

UDP_LBE 0 0 34.22 57.06 70.37 73.73 76.38

40 50 60 70 80 90 100

23

Tests with disproportion :

10 flows per Smartbits port ( overall 20 flows)

• Case 1:1(again..)

5 BE + 5 LBE flows per port (i.e. X=Y=5)

• Case 7:3

7 LBE + 3 BE flows per port (i.e. X=3;Y=7)

• Case 9:1

9 LBE + 1 BE flows per port (i.e. X=1;Y=2)

X BE + Y LBE Flows

X BE + Y LBE Flows

24

1500f.s. 5:5 (5LBE+5BE per port sharing the BW)

1500f.s. Th. 1:1(5,5) LBE-BE

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 73,104,09677,986,397 82,873,44087,724,91592,619,072 96,624,029

LBE 24,563,26119,699,930 14,834,227 9,935,328 5,098,080 1,218,797

75 80 85 90 95 100

1500f.s. Loss 1:1(5,5) LBE-BE

0

20

40

60

80

100

120


Pa

ck

et

Lo

ss

%

BE

LBE

BE 0 0 0 0.01 0 0.88

LBE 66.4 74.74 82.1 88.68 94.5 98.75

75 80 85 90 95 100

1500f.s. OWD St.Dev. 1:1(5,5) LBE_BE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

75 80 85 90 95 100


OW

D S

tan

da

rd D

ev

iati

on

(b

ps

)BE

LBE

[5/(5+5)= 0.5] * 200Mbps * [(1500-18/1500+20)=0.975] = 97.5Mbps(*) (*) this cannot be achieved.....99%(97.5)=96.52Mbps can be achieved ……It is Fine!!!!!!!!!!!!!!!!

25

1500f.s.7:3 (7LBE+3BE per port sharing the BW)

1500f.s. Th. 7:3 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 43,862, 46,786, 49,721, 52,640, 55,557, 58,488,

LBE 53,814, 50,900, 47,957, 45,040, 42,138, 39,214,

75 80 85 90 95 100

1500f.s. Loss 7:3 LBE-BE

0

10

20

30

40

50

60

70

80


Pa

ck

et

Lo

ss

%

BE

LBE

BE 0 0.02 0 0 0.03 0

LBE 47.42 53.38 58.67 63.33 67.5 71.27

75 80 85 90 95 100

1500 St.Dev. 7:3 LBE-BE

0

5000

10000

15000

20000

25000

75 80 85 90 95 100


OW

D S

tan

da

rd D

ev

iati

on

(M

icro

se

c.)

BE

LBE

[3/(3+7)= 0.3] * 200Mbps * [(1500-18/1500+20)=0.975] = 58.5Mbps…it is fine!!!!……..obvious…....there is no loss!!!

A : 200Mbps = 140Mbps LBE + 60Mbps BE offered load

26

1500 9:1 ( 9LBE+1BE per port sharing the BW)

1500f.s. 9:1 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 14,030, 15,597, 16,574, 17,542, 18,523, 19,496,

LBE 79,684, 82,088, 81,132, 80,118, 79,190, 78,206,

75 80 85 90 95 100


0

10

20

30

40

50

60

w ire rate offred load (bps)

Pa

cket

Lo

ss %

BE

LBE

BE 0 0 0 0.03 0 0

LBE 39.44 41.52 45.61 49.27 52.5 55.43

75 80 85 90 95 100

1500f.s. St.Dev. 9:1 LBE-BE

0

2000

4000

6000

8000

10000

12000

14000

16000

75 80 85 90 95 100


OW

D S

tan

da

rd D

ev

iati

on

(M

icro

se

co

nd

s)

BE

LBE

[1/(9+1)= 0.1] * 200Mbps * [(1500-18/1500+20)=0.975] = 19.5Mbps…it is fine!!.........obvious…….there is no loss!!!!!

It is even better than the 7:3 case…

200Mbps = 180Mbps LBE + 20Mbps BE offered load

27


1180 Th. 1:1 (5,5) LBE-BE

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

120,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 72,620,3577,441,2582,306,7887,140,7091,937,4495,947,73

LBE 24,377,8319,525,3114,685,829,837,0275,014,2621,169,437

75 80 85 90 95 100

1180f.s. Loss 5:5(1,1) LBE-BE

0

20

40

60

80

100

120

wire rate offered load (mbps)

pa

ck

et

los

s %

BE

LBE

BE 0 0.02 0 0 0.04 0.91

LBE 66.43 74.79 82.16 88.71 94.55 98.79

75 80 85 90 95 100

1180f.s. St.Dev. 5:5(1,1) LBE-BE

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

75 80 85 90 95 100


OW

D S

tan

dar

d d

evia

tio

n (

Mic

rose

c.)

BE

LBE

[5/(5+5)= 0.5] * 200Mbps * [(1180-18/1180+20)=0.968] = 96.8Mbps(*) (*) this cannot be achieved.....99%(96.8)=95.832Mbps can be achieved…. It is fine!!!

28

1180f.s. Th. 7:3 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 43,572,211 46,472,563 49,384,070 52,271,408 55,184,774 58,096,282

LBE 53,394,365 50,523,760 47,606,675 44,674,717 41,798,534 38,909,338

75 80 85 90 95 100

1180f.s Loss 7:3 LBE-BE

0

10

20

30

40

50

60

70

80


Pa

ck

et

los

s %

BE

LBE

BE 0 0 0 0.02 0 0

LBE 47.48 53.41 58.69 63.38 67.54 71.3

75 80 85 90 95 100

1180f.s. OWD St.Dev. 7:3 LBE-BE

0

500

1000

1500

2000

2500

3000

3500

4000

4500

75 80 85 90 95 100


OW

D S

tan

da

rd D

ev

iati

on

(M

icro

se

c.)

BE

LBE


[3/(3+7)= 0.3] * 200Mbps *[(1180- 18/1180+20)=0.968] = 58.08Mbps…..it is fine!!!!......obvious……...there is no loss!!!


29

1180f.s. 9:1 (9LBE+1BE per port sharing the BW)1180f.s. Th. 9:1 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 13,934,704 15,490,854 16,457,638 17,428,141 18,387,488 19,365,427

LBE 79,116,397 81,505,469 80,503,360 79,549,590 78,562,355 77,640,192

75 80 85 90 95 100


0

10

20

30

40

50

60


Pa

ck

et

Lo

ss

%

BE

LBE

BE 0 0 0.02 0 0.04 0

LBE 39.47 41.54 45.66 49.28 52.55 55.45

75 80 85 90 95 100

1180f.s St.Dev. 9:1 LBE-BE

0

100

200

300

400

500

600

700

800

900

75 80 85 90 95 100


OW

D S

tan

dar

d D

evia

tio

n (

Mic

rose

c.)

BE

LBE

[1/(9+1)= 0.1] * 200Mbps * [(1180-18/1180+20)=0.968] = 19.36Mbps……it is fine!!!.......obvious……..there is no loss!!!It is even better than the 7:3 case…


30


512 Th.1:1 (5,5) BE+LBE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000

100,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 69,642, 74,281, 78,881, 83,568, 88,138, 89,332,

LBE 23,285, 18,692, 14,015, 9,394,6 4,816,6 3,669,0

75 80 85 90 95 100

512 Loss 1:1 (5,5) BE+LBE

0

20

40

60

80

100

120


pac

ket

loss

%

BE

LBE

BE 0 0 0.05 0 0.08 3.79

LBE 66.56 74.84 82.24 88.76 94.54 96.05

75 80 85 90 95 100

512 Standard Deviation 1:1 (5,5) BE+LBE

0

2000

4000

6000

8000

10000

12000

14000

16000

75 80 85 90 95 100


Rx.

IP T

hrou

ghpu

t (bp

s)BE

LBE

[5/5+5= 0.5] * 200Mbps * [(512-18/512+20)=0.928] =100Mbps BE * 0.928 = 92.8Mbps(*) this cannot be achieved.....99%(92.8)=91.8Mbps can be achieved……The target throughput is 91.8. – 89.3 = 2.5Mbps far……but this is not a news….see 512 1:1….

31

512 f.s. 7:3 (7LBE+3BE per port sharing the BW)

512 Th. 7:3 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000


Rx

. IP

Th

rou

gh

pu

t (b

ps

)

BE

LBE

BE 41,775,011 44,569,075 47,334,685 50,117,683 52,780,541 52,635,107

LBE 51,133,347 48,405,677 45,563,398 42,825,453 40,228,198 40,366,518

75 80 85 90 95 100

512 Loss 7:3 LBE-BE

0

10

20

30

40

50

60

70

80


pa

ck

et

Lo

ss

%

BE

LBE

BE 0.02 0 0.04 0.05 0.27 5.53

LBE 47.56 53.45 58.76 63.4 67.42 68.95

75 80 85 90 95 100


0

100

200

300

400

500

600

700

800

900

1000

75 80 85 90 95 100


OW

D St

anda

rd D

evia

tion

(Mic

rose

c.) BE

LBE

[3/(3+7)= 0.3] * 200Mbps * [(512-18/512+20)=0.928] = 60Mbps BE * 0.928 = 55.68Mbps... The target Throughput is 55.68 – 52.63 = 3.05Mbps (5%loss) far due to the presence of 140Mbps LBE traffic……SEE BLUE ARROW IN SLIDE 13


A

B

B : 190Mbps = 133Mbps LBE + 57Mbps BE offered load

[3/(3+7)= 0.3] * 190Mbps * [(512-18/512+20)=0.928] = 57Mbps BE * 0.928 = 52.896Mbps... The target Throughput is 52.896-52.780 = 116kBps (0.27%loss) far due to the presence of 140Mbps LBE traffic……INTERPOLATE THE VALUES IN SLIDE 13 FOR 57Mbps BE STANDALONE

32

512 9:1 (9LBE+1BE per port sharing the BW)

512 Th. 9:1 LBE-BE

0

10,000,000

20,000,000

30,000,000

40,000,000

50,000,000

60,000,000

70,000,000

80,000,000

90,000,000


Rx

. IP

th

rou

gh

pu

t (b

ps)

BE

LBE

BE 13,928,429 14,851,616 15,784,288 16,713,798 17,102,675 17,082,915

LBE 78,998,109 78,106,538 77,130,394 76,246,726 75,894,998 75,939,261

75 80 85 90 95 100

512 Loss 9:1 LBE-BE

0

10

20

30

40

50

60


Pa

cke

t L

oss

%

BE

LBE

BE 0 0.03 0 0 3.06 8.01

LBE 36.98 41.58 45.71 49.31 52.2 54.57

75 80 85 90 95 100


0

200

400

600

800

1000

1200

1400

75 80 85 90 95 100


OW

D S

tan

da

rd D

evi

ati

on

(M

icro

sec

on

ds

)

BE

LBE

[1/(9+1)= 0.1] * 200Mbps * [(512-188/512+18)=0.928] = 20Mbps BE Wire rate * 0.928 =18.56Mbps……The target Throughput is 18.56 – 17.05 = 1.51Mbps (8%loss) far… due to the presence of 180Mbps LBE traffic……SEE RED ARROW IN SLIDE 13

[1/(9+1)= 0.1] * 190Mbps * [(512-188/512+18)=0.928] = 19Mbps BE Wire rate * 0.928 =17.632Mbps……The target Throughput is 17.632 – 17.102 = 1/2Mbps (3%loss) far… due to the presence of 171Mbps LBE traffic….……..INTERPOLATE THE VALUES IN SLIDE 13FOR 19Mbps BE STANDALONE

A

B


B : 190Mbps = 171Mbps LBE + 19Mbps BE offered load

33

The network noise for 512 f.s. doesn’t affect the results significantly because it is absent in point 1 and not dominant in point 2.

1. Slide 7 and 13 demonstrates that 512 frame size flow can reach the theoretical target throughput when the offered wire rate is 60,57,20,19 and 100 Mbps respectively.

2. The network noise provokes an equal drop probability for both classes…(Geant transparency)……so…if it was dominant we should have seen a smaller drop % for BE when its load was smaller….however, we saw quite the reverse, that is, a higher drop % when BE load was smaller.

34

64 95706pps input rate exceeded

128 NO

256

384 NO-ISH

512 YES-ISH

1180 YES-VERY GOOD

1500

1518 YES

1:1 Diff. Serv. Evaluation(Cisco7200)

35

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

XXXXXXXXXX

Very Bad

Very Very Bad

Bad

Good

Very Good

Very Very Good

1:1 5:5 7:3 9:1

1500, 1180

512

36

StandaloneThroughput

80

82

84

86

88

90

92

94

96

# of flows

TC

P T

hro

ug

hp

ut

BE Throughput

LBE Throughput

BE Throughput 85.98 92.26 93.72 93.86 94.54 95

LBE Throughput 87.68 93.14 93.32 93.88 94.04 95.06

1 2 4 8 12 20

Throughput Vs. # of flows

0

20

40

60

80

100

120

# of flows per each class

TC

P T

hro

ug

hp

ut

(Mb

ps)

BE Throughput

LBE Throughput

aggregate throughput

BE Throughput 79.02 86 88.3 90.94 93.54 94.2

LBE Throughput 12.62 9.76 7.16 6.345 7.304 12.96

aggregatethroughput

91.64 95.76 95.46 97.285 100.84 107.16

1 2 4 8 16 32

Loss %

0

0.5

1

1.5

2

2.5

# of flows per each class

Loss

% %Be loss

%LBE loss

%Be loss 0.00155 0.0042 0.01538 0.05454 0.16679 0.58784

%LBE loss 0.05703 0.42317 0.78629 1.91059 1.19613 1.51166

1 2 4 8 16 32

•As regards BE, a greater # of flows lose more because more TCP flows have to adapt…

•BE aggregate gets….:

1BE+1LBE: 79.02/85.98=91%

2BE+2LBE : 86/92.26=93%

4BE+4LBE:88.3/93.72=94%

8BE+8LBE:90.94/93.86=96.8

16BE+16LBE:93.54/94.77= 98%

32BE+32LBE:94.2/95 = 99%

37

32 flows BE TCP + 1 flow UDP LBE 512&1500

It seems that, for 512f.s., beyond 80x2=160 Mbps LBE offered load…the input interface goes crazy again….and while BE TCP backs-off…LBE UDP fill the gaps created by TCP; This is why LBE “steals” BW to LBE…….

I think this is the situation where the LBE BW. percentage assigned has to be 0%...but CISCO doesn’t support it

Bear in mind that when LBE is 160Mbps, the load at the input interface is constituted by the TCP load as well which is around 94Mbps

UDP LBE 512f.s. + 32flows TCP BE

0

20

40

60

80

100

120

LBE UDP per port Load (Mbps)

Th

rou

gh

pu

t(B

lue

&P

ink

)) A

ND

Lo

ss

%(Y

ell

.)

TCP BE Throughput

LBE Loss % (512)

LBE IP Throughput (512)

32 TCP Standalone

TCP BEThroughput

92.5 93.3 92.3 49 26.22

LBE Loss %(512)

94.78 95.64 96.42 72.72 60.94

LBE IPThroughput(512)

5.815 5.662 5.315 45.593 72.534

60 70 80 90 100


0

20

40

60

80

100

120

60 70 80 90 100

LBE UDP per port Load (Mbps)

Th

rou

gh

pu

t(B

lue

&P

ink

) A

ND

Lo

ss

%(Y

ell

.)

32 TCP BE Throughput

LBE Loss (1500f.s.)


32 TCP Standalone

38

Better reaction for the 64 aggregate with respect to the 32 one……..the higher the # of flows the higher the reactivity!!!!!!!!!!!


0

20

40

60

80

100

120

LBE UDP per port Load

Th

rou

gh

pu

t(B

lue&

Pin

k) A

ND

L

oss

%(Y

ell.

)

TCP BE Throughput

LBE Loss (512f.s.)


64 TCP Standalone

TCP BEThroughput

93.4 93.7 90.03 63.908 39.045

LBE Loss(512f.s.)

97.89 98.93 97.23 79.79 64.21

LBE IPThroughput(512)

2.349 1.396 4.111 33.771 66.464

60 70 80 90 100

64 TCP flows instead of 32

39

Conclusions:

UDP• Router input interfaces suffer a lot from high input rate…it

prevents traffic from arriving at the Diff. Serv. output interfaces.

………so…..small frame sizes LBE traffic has to be carefully rate- limited, when necessary, in the router input interface.

• For small frame sizes BE is not protected from LBE attack…the bigger the disproportion the worse the situation…it is likely that this is going to be a problem for Gbps speed….

......so….such form of rate-limiting has to be function of the traffic disproportion present in the system as well….

TCP• The policy works better when the number of flows increases

40

Future work

TCP + UDP

• Try N flows TCP BE + UDP LBE with and without burstiness and check if BE TCP is protected

• Apply a 0% BW reservation to LBE UDP traffic

TCP

• Investigate whether, in the case of a small number of TCP flows, it is actually the unreliability of the measurements obtained with a very small number of TCP flows which is behind the bad performance of the policy, rather than the policy itself not working with a small number of flows.

41

THANK YOU………….

lbe tests between infn & ucl

Documents

mbps bottleneck

mbps linktraffic

throughput ip

lbe tests

flowsper framesize

lbeper frame size

lbe output queues

cisco router