Capacity Allocation ParadoxCapacity Allocation Paradox

Isaac KeslassyIsaac Keslassy

Joint Work with Asaf Baron and Ran Ginosar

EE Department, Technion, Haifa, Israel

2

The Capacity Allocation ParadoxThe Capacity Allocation Paradox

Node B

Node C

Node A

CA

CB

CR

Router

Unlimited queuesFinite (small) buffers

RA

RB

Capacity Allocation Paradox:Adding Capacity Can Destabilize the Network

3

Fast Security CheckSlow Security Check

StableUnStableMarakana Soccer StadiumMarakana Soccer Stadium

Brazillian Line

Argentinian Line

Fast Security Check

Fast Swipe Ticket Entrance

Safety Check

Enter the stadium

4

MotivationMotivation Small buffer networks are widely used

When QoS not met: add capacity [Guz et al., ’06]May destabilize the network

Network On-Chip Interconnection of Computers

SpaceWire

5

Previous Work: Selfish RoutingPrevious Work: Selfish Routing Braess’s Paradox (1968)

Difference: We assume fixed routing

6

Previous Work: Cyclic DependencyPrevious Work: Cyclic Dependency Kumar & Seidman (1990)

Instability even though capacity > data rate

Dai, Hasenbein & Vande Vate (1998) Adding capacity may destabilize a network

Differences: No cycles in dependency graph Single router

Each packet visits router only once Several simple arbitration policies Independent of initial conditions

New fundamental reason: Finite buffers

7

A General PhenomenonA General Phenomenon

Node B

Node C

Node A

CA

CB

CR

Router

Unlimited queues

Finite (small) buffers

RA

RB

When buffer is full:1. Blocking: Wormhole Routing2. Dropping (with retransmission): Store And Forward

Arrivals:Periodic, Poisson…

CA=2

CR=2CA=1

8

IntuitionIntuition

Node B

Node C

Node A

CB=1

Router

Buffer of 1 bit

1 [pkt/T]

1 [pkt/T]

Assume A has priority:

(a) CA=1A1

B1

A2

B2

A3

B3

2T

Share of CR

2

T

1

3T(a)

(b) CA=2 A1B1 (1)

2T

Share of CR

2

T

1

3T

(b)

A2B1 (2)

A3B2 (1)

T/2 3T/2 5T/2

9

What are the conditions for stability?What are the conditions for stability? Necessary conditions:

A AC R

R A BC R R

B BC R

Node A

Node B

Node C

Stability Regions, CR =0.273Mflits/sec

CA

[Mflit/sec]

CB [

Mfli

t/se

c]

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

BC

AC

CR is constantRA = RB

10

Stability Regions, CR =0.273Mflits/sec

CA [Mflit/sec]

CB [

Mfli

t/se

c]

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Case #1: Case #1:

Buffers in the router hold no more than one data unit

A B RC C C

Node C

Queue A

Queue B

Buffer A

Buffer B

Necessary conditions are also sufficient.

Stability Regions, CR =0.273Mflits/sec

CA [Mflit/sec]

CB [

Mfli

t/se

c]

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

?

CA

CB

CR

11

EPRR

CA [Kf/s]

CB [

Kf/

s]

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

Example 1: Analysis Stability PictureExample 1: Analysis Stability Picture

1

42

3 2

0

0

CA [Kf/s]

CB [

Kf/

s]

CR = 273[Kf/s] (Constant)

RA = RB = 100[Kf/s]

CB=110CB=150

CA=110CA=300CA=190

12

Analytic Stability regions, CRC

=0.273Mflits/sec

CAR

[Mflit/sec]

CB

R [

Mfli

t/se

c]

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Case #1Case #2Case #3

Example #1 – Capacity AllocationsExample #1 – Capacity Allocations

Node A

Node B

Node C

CR=273

StableUnStableStable1

42

3 2

RA = 100

RB = 100 1000 [flits/pckt]Buffer Size: 16 Flits

Exhaustive Round Robin, Wormhole

13

Analytic+Simulation Stability regions for CR =0.273Mf/s

CA [Kf/s]

CB [

Kf/

s]

125 150 175 200 225 250 275 300 325

125

150

175

200

225

250

275

300

325

Results – Simulation Stability Results – Simulation Stability RegionsRegions

1

42

3 2

CR = 273[Kf/s] (Constant)

RA = RB = 100[Kf/s]

14

EPRR

CA

[Kf/s]

CB [

Kf/

s]

550 600 650 700

150

200

250

300

350

400

450

500GPS

CA

[Kf/s]

CB [

Kf/

s]

550 600 650 700

150

200

250

300

350

400

450

500RRPF

CA

[Kf/s]

CB [

Kf/

s]

550 600 650 700

150

200

250

300

350

400

450

500

Example #2 – Wormhole RoutingExample #2 – Wormhole Routing

1000 [flits/pckt], Buffer Size: 16 Flits, RA = 500kf/s, RB = 100kf/s

Exhaustive Round Robin Round-Robin GPS

15

Exhastive, CRC=2.1Mbit/sec

CAR [Mbit/sec]

(b)

CB

R [

Mbi

t/se

c]

1.25 1.5 1.75 2 2.25 2.5

1.25

1.5

1.75

2

2.25

2.5

Priority, CRC=2.1Mbit/sec

CAR [Mbit/sec]

(a)

CB

R [

Mbi

t/se

c]

1.25 1.5 1.75 2 2.25 2.5

1.25

1.5

1.75

2

2.25

2.5

Example #3 – Store and forwardExample #3 – Store and forward

Poisson Arrivals with Parameters: A = 100, B = 100Packet Length 10^4 bitBuffer Size 3-4 packets

Strict Priority, CR = 2.1[Mbit/s] Exhaustive RR, CR = 2.1[Mbit/s]

16

Round-Robin, CRC=6.1Mbit/sec

CAR [Mbit/sec]

(c)

CB

R [

Mbi

t/se

c]

5.25 5.5 5.75 6 6.25 6.5

1.5

2

2.5

3

3.5

Exhastive, CRC

=6.1Mbit/sec

CAR

[Mbit/sec]

(d)

CB

R [

Mbi

t/se

c]

5.25 5.5 5.75 6 6.25 6.5

1.5

2

2.5

3

3.5

Example #3 – Store and forwardExample #3 – Store and forward

Poisson Arrivals: A = 500B = 100

Packet = 10^4 bitBuffer 3 packets

RR, CR = 6.1[Mbit/s]Exhaustive RR, CR = 6.1[Mbit/s]

All packets need to arrive

sometime

17

SummarySummary Adding capacity may destabilize even a

simple network

The scheduling algorithm affects the stability of the network (even if work-conserving)

GPS arbitration: always stable

18

Thank you.Thank you.