scheduling techniques for hybrid circuit/packet …./mmukerje/solstice-conext15-final.pdf ·...
Post on 23-Mar-2018
231 Views
Preview:
TRANSCRIPT
Scheduling Techniques for Hybrid Circuit/Packet
NetworksHe Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman,
George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren
Hybrid circuit/packet networks: Can we get best of both?
Circuit Switching Packet Switching
High link rate
Low link rate
Limits on num flows+ Explicit flow setup
Any-to-any communication
This talk: commun. restricted to pairs
+ non-trivial circuit reconfig time
Hybrid networks in datacenters*
*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …
RACK RACK RACK RACK RACK RACK RACK RACK
Circuit Packet
Hybrid networks in datacenters*
*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …
A B CABC
Datacenter workloadsare skewed
Senders
Receivers
Senders
ReceiversHybrid networks in datacenters*
*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …
A B CABC
Prior work showsmajor benefits
(“Keep circuits busy!”)
Hybrid networks in datacenters*
*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …
Packet switch as “escape hatch”
for difficult demands
Need an explicit sequence of circuit
configurations
Hybrid networks in datacenters*
How do proposed hybrid networks schedule demand?They don’t! They use an oracle
*Helios [SIGCOMM ’10], c-Through [SIGCOMM ’10], Flyways [SIGCOMM ’11], …
Does prior scheduling work help?
iSLIP BvN
Works per-packet (can’t identify “difficult” to schedule demands)
Works on all demand simultaneously
(can identify “difficult” demands)
Both assume circuit reconfig is free (use many configurations)
(Fast heuristic per-packet) Optimal
Does prior scheduling work help?BvN
Both assume circuit reconfig is free (use many configurations)
Time
iSLIP
Contribution: Solstice
• Characterizing
• Optimal formulation and bounds
• A fast, heuristic-based hybrid scheduler “ ”
• of Solstice: performs 2-3x better than previous solutions and within 14% of optimal up to a 128-port switch
the hybrid scheduling problem
Solstice
Evaluation
Outline
• Model
•
•
•
the hybrid scheduling problem
Solstice
Evaluation
Goal of hybrid scheduling
?
A B CABC
Schedule all demand in minimum duration
Goal of hybrid scheduling
Schedule all demand in minimum duration
duration = total circuit transmission time+
number of configurations * circuit reconfig cost
Configuration selection and duration
What goes topacket switch
The hybrid scheduling problem
?
A B CABC
Understanding circuitsKey properties
of circuit switching
Configuration
Duration
A B CABC
Understanding circuitsKey properties
of circuit switching
Configuration
Duration
Src Dest
A
B
C
A
B
C
A B CABC
A
B
C
A
B
C
Src Dest
Configuration
Duration
The hybrid scheduling problem
?Circuit Switch Packet Switch
A
B
C
A
B
C
A B CABC
The hybrid scheduling problem
?Circuit Switch Packet Switch
A
B
C
A
B
C
A B CABC
A B CABC
How to compute the output?
Outline
• Model
• The hybrid scheduling problem
• Solstice
• Evaluation
Solstice: intuition
• What can we take advantage of?
• Skew in workload (i.e., large + small demands)
• Circuit switch is efficient for large demands
• Packet switch is efficient for small demands
Solstice: key ideaSchedule largedemands first
on circuit
A B CABC
D
D
Solstice: key ideaSchedule largedemands first
on circuit
Circuit SwitchLoop
A B CABC
D
D
A
B
C
A
B
C
D D
Solstice: key idea
Circuit Switch
A B CABC
D
D
Schedule largedemands first
on circuitLoop
A
B
C
A
B
C
D D
A
B
C
A
B
C
D D
Solstice: key ideaTerminate when
remainder can be served by
packet switch
Circuit Switch
A B CABC
D
D
A
B
C
A
B
C
D D
A
B
C
A
B
C
D D
Packet SwitchA B C
ABC
D
D
Solstice: key idea
Large demand alwayson circuit switch
(efficient)
Small demand alwayson packet switch
(efficient)Circuit SwitchA
B
C
A
B
C
D D
A
B
C
A
B
C
D D
Packet SwitchA B C
ABC
D
D
Solstice: example
A B CABC
D E
DE
Solstice: exampleTotal
Duration
0Reconfig time = 2.5
. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .
Solstice: exampleServableon Packet
0Total
Duration
0Reconfig time = 2.5 Packet BW = 1/10
. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
0Reconfig time = 2.5 Packet BW = 1/10
. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
0Reconfig time = 2.5 Packet BW = 1/10
. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .
. 5 40 . 5 5 . . 40 5 . . . 10 4045 5 . . . . 40 10 . .
A
B
C
A
B
C
D
E
D
E
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
0Reconfig time = 2.5 Packet BW = 1/10
40
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
0Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
40Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: exampleServableon Packet
0Threshold
32Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: exampleServableon Packet
4.25Threshold
32Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: exampleServableon Packet
4.25Threshold
16Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Threshold too large
Solstice: exampleServableon Packet
4.25Threshold
8Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: example
A
B
C
A
B
C
D
E
D
E
Servableon Packet
4.25Threshold
8Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Incompleteconfiguration
Solstice: exampleServableon Packet
4.25Threshold
4Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E . 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
Solstice: example
A
B
C
A
B
C
D
E
D
E
Servableon Packet
4.25Threshold
4Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
A
B
C
A
B
C
D
E
D
E
. 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
MultipleConfigs
. 5 . . 5 5 . . . 5 . . . 10 . 5 5 . . . . . 10 . .
A
B
C
A
B
C
D
E
D
E
Solstice: exampleServableon Packet
4.25Threshold
4Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
Solstice: exampleServableon Packet
4.25Threshold
4Total
Duration
42.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E . 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Solstice: exampleServableon Packet
4.25Threshold
4Total
Duration
47.5Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E . 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Solstice: exampleServableon Packet
4.25Threshold
4Total
Duration
50Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E . 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Solstice: exampleServableon Packet
5Threshold
4Total
Duration
50Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E
. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Servableon Packet
5
Solstice: exampleThreshold
4
. . . . . . . . . . . . . . . . . . . . . . . . .
. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Total Duration
50Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E
Servableon Packet
5
Solstice: exampleThreshold
4
. . . . . . . . . . . . . . . . . . . . . . . . .
. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .
Total Duration
50Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E
ScheduledAll Demand
Servableon Packet
5
Solstice: exampleThreshold
4
. . . . . . . . . . . . . . . . . . . . . . . . .
Total Duration
50Reconfig time = 2.5 Packet BW = 1/10
40
A
B
C
A
B
C
D
E
D
E
5
A
B
C
A
B
C
D
E
D
E
ScheduledAll Demand
2 Configs
. 5 . . . . . . . 5 . . . 5 . 5 . . . . . . 5 . .Remainderon Packet
Outline
• Model
• The hybrid scheduling problem
• Solstice
• Evaluation
Inefficiencies
• Two possible ways to be inefficient
• Circuit reconfig “downtime”
• Link is active but sender has no data to send
• Focus on first one in this talk; second in paper
Metrics
• Total schedule duration
• total circuit transmission time + number of configurations * reconfig time
• Number of circuit configurations used in schedule
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)Schedule duration
Better
Worse
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)Schedule duration
Better
Worse
Lower Bound
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)Schedule duration
Upper Bound
Better
Worse
Lower Bound
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)
Solstice
Schedule duration
Solstice at most 14% from optimal for these counts
Better
Worse
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)
Solstice BvN
Schedule duration
Better
Worse
8 12 16 32 64 128Number of ports
0
2
4
6
8
10
12
14
16
Tim
e(m
s)
Solstice BvN iSLIP
Schedule durationSolstice
2-3x better
iSLIP poor overall. Why?
BvN gets worse here. Why?
Better
Worse
8 12 16 32 64 128Number of ports
0
50
100
150
200
Num
confi
gs
Solstice BvN iSLIP
Number of configurations
iSLIP uses many configurations
throughout
BvN almost 3x number of
configurations
Better
Worse
Other results in paper
• Effects of skew (up to 10% worse)
• Effects of sparsity (up to 26% worse)
• Runtime complexity analysis
• Augmenting Solstice with offline computation
Conclusion
• Characterizing the hybrid scheduling problem
• Optimal formulation and bounds
• A fast, heuristic-based hybrid scheduler “Solstice”
• Evaluation of Solstice: performs 2-3x better than previous solutions and within 14% of optimal up to a 128-port switch
Scheduling Techniques for Hybrid Circuit/Packet
NetworksHe Liu, Matthew K. Mukerjee, Conglong Li, Nicolas Feltman,
George Papen, Stefan Savage, Srinivasan Seshan, Geoffrey M. Voelker, David G. Andersen, Michael Kaminsky, George Porter, and Alex C. Snoeren
Backup Slides…
Solving Optimally• Use an ILP to solve the problem
• Provides intuition and a lower bound (i.e., this is the best you can do)
• has to work through n! possible configurations
• Slow (e.g., 12-port takes minutes, 16-port takes hours)
Stuffing . 13 10 70 4 . . 14 12 6965 . . 12 1415 33 2 . 1112 7 3 1 .
. 14 10 70 4 . . 17 12 6971 . . 13 1415 70 2 . 1112 14 69 3 .
= Variable row/col sum (but max is 98)
Fixed row/col sum (98) =
Pick large entries and increase until maxed
Runtime complexity• Dominated by circuit configuration selection
• (equivalent to finding a perfect matching in a bipartite graph)
• O(n * non-zero entries in rows) + O(n * log2 n)
• If we assume number of non-zero entries is constant (in line with prior work on dc workloads), total is O(n2 * log2 n)
Simulation Setup• 20 microsecond reconfiguration time
• accumulation period of 3 ms, schedule all traffic
• 100Gbps circuit links, 10Gbps packet links
• sparse matrices (4 large and 12 small demands per port on 64 port switch)
• skewed matrices (small demands total 30% of demand)
• 100 random matrices for testing
8 12 16 32 64 128Number of ports
0
20
40
60
80
100
Util
izat
ion
(%)
iSLIP BvN Solstice
Circuit utilizationNumber of configshas great impact
on circuit utilization
Worse
Better
8 16 24 32 40 48 56 64Number of demands per port
2.02.53.03.54.04.5
Tim
e(m
s)
2.8 3.
1 3.3 3.
5 3.6
3.7
3.7 3.
9Solstice
Sparsity
Default
Worse
Better
only 26% worse
. 3 . . . . . 4 . . . . . 4 . . . . . 142 . . . .
32 13 10 70 424 29 14 12 6965 1 10 12 1415 33 2 42 1112 7 3 1 3
Skew
0 5 10 15 20 25 30 35 40 45 50Bandwidth requested by small flows (%)
2.02.53.03.54.04.5
Tim
e(m
s)
2.7 2.
9 3.0
3.0
3.0 3.1
3.1
3.1 3.2 3.2
3.3
Solstice
Worse
Better
only 6% worseDefault
. 3 . 40 2 2 . 4 . 40 1 45 . 4 . . 2 48 . 142 . 3 4 .
. 43 . 40 4242 . 41 . 4041 45 . 44 . . 42 48 . 4142 . 43 44 .
top related