routing algorithms

© Sudhakar Yalamanchili, Georgia Institute of Technology (except as indicated)

Routing AlgorithmsRouting Algorithms

ECE 8813a (2)

OverviewOverview

• Routing Algorithm fixed by Routing function Selection function

• Determinant of key properties Connectivity Deadlock and routing freedom Fault tolerance adaptivity

A single function for deterministic protocols

ECE 8813a (3)

TaxonomyTaxonomy

Routing Algorithms

Unicast multicast

Centralized Source Distributed Multi-phase

Table Lookup Finite State Machine

Deterministic Adaptive

Progressive Backtracking

Profitable Misrouting

Complete Partial

# Destinations

Routing Decisions

Implementation

Adaptivity

Progressiveness

Minimality

Number of Paths

ECE 8813a (4)

Generic Router ArchitectureGeneric Router Architecture

ECE 8813a (5)

ElaborationElaboration

• Source Routing Static paths defined at the source Extensions to street-sign routing

• Multiphase routing Used for reliable routing Load balancing

• Irregular networks typically need table look-up or source routing

• Interval routing Making table-based routing more efficient

ECE 8813a (6)

Interval RoutingInterval Routing

• Each output link corresponds to an interval of nodes Union of intervals at a node is the set of all destination nodes Must be able to distinguish invalid intervals (at the edges of a

mesh)• Use overlapping intervals for fault tolerance

0 4 8 12

1 5 9 13

2 6 10 14

3 7 11 15

Channel Node Interval

X+ 6 8-15

X- 6 0-3

Y+ 6 4-5

Y- 6 7

ECE 8813a (7)

Some Finer PointsSome Finer Points

• Oblivious vs. deterministic protocols Typically referring to use of network state Deterministic protocols are oblivious

o Converse may not be true

• Fully, maximal, and true fully adaptive Fully: maximize alternative physical paths Maximal: maximize all routing options True: no constraints on VC usage

o Deadlock recovery-based methods

ECE 8813a (8)

Classes of Routing AlgorithmsClasses of Routing Algorithms

• Deterministic Routing

• Partially Adaptive

• Fully Adaptive

• Maximally Adaptive

• Non-minimal Routing

• Routing in MINs

ECE 8813a (9)

Deterministic Routing AlgorithmsDeterministic Routing Algorithms

• Strictly increasing or decreasing order of dimension

• Acyclic channel dependencies

Mesh Binary Hypercube

Deterministic routing Deterministic routing

ECE 8813a (10)

ToriTori

• Dimension order routing• Create an acyclic channel

dependency graph with the following routing function c

0i

when j<i, c

1i when

j >i

n0 n1

n3 n2

c0

c1

c2

c3

c13

n0 n1

n3 n2

c10

c11

c02

c12

c01

c03

c00

c10

c11

c12

c01

c02

c03


ECE 8813a (11)

Deterministic Routing Algorithms: Deterministic Routing Algorithms: Implementation Issues Implementation Issues

• Relatively, the most inexpensive to implement Absolute addressing Relative Addressing

• Header update Necessary to maintain uniformity of implementation

• Source routing Street sign addressing: only encode the turns


ECE 8813a (12)

Partially Adaptive Routing AlgorithmsPartially Adaptive Routing Algorithms

• Trade-off between hardware resources and adaptivity Maximize adaptivity for given resources Minimize resources required for a given level of

adaptivity

• Typically exploit regular topologies

Partially Adaptive Routing Partially Adaptive Routing

ECE 8813a (13)

Planar Adaptive RoutingPlanar Adaptive Routing

• Packets are routed adaptively in a series of two dimensional planes Order of planes (dimensions) is arbitrary

• Routing in two dimension uses two virtual networks Increasing and decreasing networks

Fully adaptive Planar adaptive Planar adaptive

A0

A1

A0

A1A2


ECE 8813a (14)

Adaptive Routing in Two Dimensions Adaptive Routing in Two Dimensions

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

Increasing Network Decreasing Network Di+1

Di


ECE 8813a (15)

PAR in Multidimensional NetworksPAR in Multidimensional Networks

• Routing is fully adaptive in a plane

• When can you skip a plane?

Dimension i

Dimension i+1

Dimension i+2

A1

A0


ECE 8813a (16)

PAR PropertiesPAR Properties

• Each plane is comprised of the following channels

Ai = di,2 + di+1,0 + di+1,1

• Three virtual channels/link in meshes and six virtual channels/link in Tori


ECE 8813a (17)

The Turn ModelThe Turn Model

• What is a turn? From one dimension to another : 90 degree turn To another virtual channel in the same direction: 0

degree turn To the reverse direction: 180 degree turn

• Turns combine to form cycles• Goal: prohibit the least number of turns to

break all possible cycles

abstract cycles XY routing west-first routing


ECE 8813a (18)

Turn ConstraintsTurn Constraints

• Choice of prohibited turns is not arbitrary

• Alternative designs Three combinations unique (within symmetry)

o Three algorithms: west-first, north-last, negative-first

equivalent


ECE 8813a (19)

West First Routing West First Routing

• Fully adaptive to the east, deterministic to the west (for non-minimal routing)

• Non-minimal is partially adaptive to the west

deterministic region

fully adaptive region


ECE 8813a (20)

Generalization of the Turn ModelGeneralization of the Turn Model

• Identify channel classes and prohibit turns between them

• Cycles are infeasible within a channel class• Transitions between channel classes are acyclic• Partially adaptive: number of shortest paths are

reduced

Application to Binary hypercubes•Base is e-cube routing•Identify up channels and down channels•Adaptively route though one class and then the other

•Turns prohibited from one class to the other


ECE 8813a (21)

Fully Adaptive RoutingFully Adaptive Routing

• Using resources in the form of buffers, channels and networks

• Migration from initial proposals to more efficient implementations

Fully Adaptive Routing Fully Adaptive Routing

ECE 8813a (22)

Structured Buffer PoolsStructured Buffer Pools

• Positive hop algorithm D+1 buffers at each node (D = diameter) Nodes request/use buffers in strictly increasing order Minimal path algorithm, valid for any topology Large buffer requirements: O(Diameter)

buffer 0

buffer 1

buffer D-1


ECE 8813a (23)

Structured Buffer Pools: ExtensionStructured Buffer Pools: Extension

subset S-1subset 1subset 0

• Negative hop algorithm Partition nodes into non-adjacent subsets Order subsets Down transitions request higher numbered buffer,

else same numbered buffer Number of buffers required in each node is given by

+ 1

set 0

set 1

S

SD )1(


ECE 8813a (24)

Extensions to Wormhole SwitchingExtensions to Wormhole Switching

replace central buffers with equivalent number of virtual channels across each physical channel

Acyclic central buffer dependencies are transformed into

acyclic virtual channel dependencies

• Basic version produces unbalanced used of virtual channels distance rarely equal to diameter

• Extension: bonus cards. Number is equal to unused hops: diameter - #req Use this number to increase the number of choices of virtual

channels at any node Not the same as adaptivity


ECE 8813a (25)

Virtual NetworksVirtual Networks

• Establish multiple virtual networks Each network works for a specific destination set Routing functions are acyclic, but typically not

connected

• Establish constraints between virtual networks


ECE 8813a (26)

Virtual Networks in a MeshVirtual Networks in a Mesh

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

Y

X

X+Y+ X-Y+

• Each virtual network is constructed to have acyclic channel dependencies Routing function in a virtual network is not connected

• Packets are injected into the appropriate virtual network• Fully adaptive, no transitions between networks• 2n virtual networks with (n.2n) virtual channels/node


ECE 8813a (27)

OptimizeOptimize

• Reduce the number of networks by 50% with one additional channel/network

• Extensions to k-ary n-cubes by introducing levels in each virtual network One level for each wrap-around channel (n+1).(n+1).2(n-1) virtual channels/node for dimensions >0 See Figure 4.20

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

Virtual network 0 Virtual network 1


ECE 8813a (28)

Extensions to ToriExtensions to Tori

• Addition of layers (virtual networks) for each wrap-around connection

• Number of layers increases by #dimensions


ECE 8813a (29)

Deadlock Avoidance via Message Deadlock Avoidance via Message DependenciesDependencies

• Two virtual channel classes across each physical link Adaptive channels & deterministic channels

• Fully adaptive use of adaptive channels Keep track of #dimension reversals for each message

o Moving from a dimension p to a lower dimension q Label each channel with the DR# of the message Messages cannot block on a channel with lower DR#

o If no channel available, permanent transition to the deterministic channel

Dependencies between messages are acyclic


ECE 8813a (30)

Summary of Design TechniquesSummary of Design Techniques

• Ordered use of topological features Dimensions Packaged components Paths

• Ordered use of resources Buffers Channels

• Order the message population Each message is uniquely identified by some

attribute, e.g., number of wrap-around channel crossings

Order blocking based on message population membership

ECE 8813a (31)

Design MethodologyDesign Methodology

• Start with a network, set of channels C1, and routing function R1

R1 is connected and deadlock free and may be deterministic/adaptive, minimal/non-minimal

• Split each physical channel into a set of additional virtual channels and define the new routing function

Set of channels includes escape channels and adaptive channels

Selection function can be defined in many ways• For wormhole switching, verify that the extended channel

dependency graph is acyclic: likely if R is restricted to minimal paths

))((),(),( 11 CCCyxRyxR xy

ECE 8813a (32)

Example Binary HypercubeExample Binary Hypercube

• Start with dimension order e-cube algorithm

• Add additional channels for adaptive routing

0 1

54

6 7

32

ECE 8813a (33)

Maximally Adaptive RoutingMaximally Adaptive Routing

• Establish a relationship between routing freedom and resources Maximize adaptivity for fix resources Minimize resources for target adaptivity

• Relationship between adaptivity and performance Not obvious Unbalanced use of physical or virtual channel

resources

Maximally Adaptive Routing Maximally Adaptive Routing

ECE 8813a (34)

In 2D Meshes: Double YIn 2D Meshes: Double Y


0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

increasing network decreasing network

X+

X- Y1-

Y1+ Y1+

Y1-

X-

X+

Y2+

Y2-

X-X+

X-

Y2+

X+Y2-

Y1Y2

6

Y2

Y1

ECE 8813a (35)

In 2D Meshes: Mad-YIn 2D Meshes: Mad-Y

X+

X- Y1-

Y1+ Y1+

Y1-

X-

X+

Y2+

Y2-

X-X+

X-

Y2+

X+Y2-

• Permit turns From the Y1 channels to the X+ channels From X- channels to Y2 channels

• Remove unnecessary turn restrictions

• Still overly restrictive!

permitted

6

Y2

Y1

No coupling between Y2 and Y1


ECE 8813a (36)

In 2D Meshes: Opt YIn 2D Meshes: Opt Y

X+

X- Y1-

Y1+ Y1+

Y1-

X-

X+

Y2+

Y2-

X-X+

X-

Y2+

X+Y2-

Y2+ Y1+

Y1+ Y2+ Y2- Y1-

Y1- Y2-

• Further reduce the number of restrictions Only restrict turns from Y1 to X- Turns from X- to Y1 and 0-degree turns in Y only

when X offset is 0 or positive

• Extensions to multidimensional meshes• Basic idea: fully adaptive routing in one set of

channels, and dimension order in the other set until specified lower dimension traversals are complete


ECE 8813a (37)

Routing with Minimum Buffer RequirementsRouting with Minimum Buffer Requirements

• Key Idea: Organize packet traffic into disjoint groups that use

separate buffers in each node Place acyclic routing restrictions in buffer usage

• Based on node orderings


ECE 8813a (38)

Node Labeling for 2D TorusNode Labeling for 2D Torus0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

15 14 13 12

11 10 9 8

7 6 5 4

3 2 1 0

15 14 12 13

11 10 8 9

3 2 0 1

7 6 4 5

0 1 3 2

4 5 7 6

12 13 15 14

8 9 11 10

right increasing node ordering left increasing node ordering

outside increasing node orderinginside increasing node ordering


ECE 8813a (39)

Algorithm 1Algorithm 1

• Algorithm Packet moves from the injection queue to the A

queue Stay in the A queues as long as we cancan move to right

along at least one dimension along a minimal path Transition to the B queues under same rule for left

traversals Transition to the C queue and remain there until

packet is delivered

• Note the de-coupling of node labeling from buffer labeling


ECE 8813a (40)

ApplicationApplication

0

1

2

4

5

6

7

3

76543210

• Orderings analogous to virtual planes Note the orderings are

acyclic

• Extensions to edge buffers Check Algorithm 2


ECE 8813a (41)

True Fully Adaptive RoutingTrue Fully Adaptive Routing

• Adaptivity extends across physical and virtual channels

• Deadlock recovery vs. deadlock avoidance


ECE 8813a (42)

Non-minimal RoutingNon-minimal Routing

• Non-minimal routing Wormhole degrades performance while VCT has less

secondary effects Fault tolerance is the main motivator

• Classes Backtracking Randomized routing and the Chaos router

Non-Minimal Routing Non-Minimal Routing

ECE 8813a (43)

Backtracking ProtocolsBacktracking Protocols

• Backtracking search + resource reservation

• Constrain the search Minimal paths vs. #misroutes


ECE 8813a (44)

OptimizationOptimization

• Sensitive to choice of switching technique Naturally suited to circuit switching and pipelined circuit

switching Overhead is large with SAF

• Deadlock is avoided by not blocking on busy channels• Livelock is avoided by maintaining and using search

history In the header: large headers In the routers: local state, headers comparable to e-cube

• Protocol variations Multi-links k-family exhaustive: profitable and misrouting limited misrouting multi-phase


ECE 8813a (45)

Routing in MINsRouting in MINs

• “Square” vs. non-square MINs vs. dilated MINs• Permutation routing and centralized control• Message routing, distributed control, and contention

• Basic elements of a logkN network and topological

equivalence Single source destination path Logarithmic delay

7

6

5

4

3

2

1

0

7

6

5

4

3

2

1

0

Routing in MINs Routing in MINs

ECE 8813a (46)

The Blocking ConditionThe Blocking Condition

sn-1,sn-2,…,s2,s1,s0,dn-1,dn-2,…,d2,d1,d0

rn-1,rn-2,…,r2,r1,r0,tn-1,tn-2,…,t2,t1,t0

• Addresses of ports at intermediate switches are computed from the source-destination addresses Blocking condition: two paths collide iff they compete

for the same link at any stage


ECE 8813a (47)

Self Routed MINsSelf Routed MINs


ECE 8813a (48)

Self Routed MINs (cont.)Self Routed MINs (cont.)

• Self routing property Routing decision a function of onlyonly the destination

• Computation of destination routing tag tn-

1,tn-2,…,t2,t1,t0

ti = di+1 0 <= i <= n-2, and tn-2 = d0 for butterfly MINs

ti = dn-i-1 0 <= i <= n-1 for Omega and Cube networks


ECE 8813a (49)

Self Routed MINs: exampleSelf Routed MINs: example


ECE 8813a (50)

Routing in Bidirectional MINsRouting in Bidirectional MINs

• Routing uses the function FirstDifference(S,D) Identifies the nearest stage to “turnaround”

• Multiple choices of switches at last stage Can randomize forward path selection


ECE 8813a (51)

Relevance of Fat TreesRelevance of Fat Trees

76543210 15

14

13

12

11

1098

• Popular interconnect for commodity supercomputing

• Active research problems Efficient, low latency

routing Fault tolerant routing Scalable protocols

(coherence)


ECE 8813a (52)

Segment Based RoutingSegment Based Routing

• Topology dependent vs. topology agnostic routing Reliability Increasing important on-chip

• Restriction-based approach Multiple restriction options

o Select restrictions based on performance goals Source based routing

o Routing table generation

Topology Agnostic Routing Topology Agnostic Routing

ECE 8813a (53)

Key Idea: Segments & SubnetsKey Idea: Segments & Subnets

• Partition topology into subnets and then segments in a subnet

• Goal: islands of regularity


ECE 8813a (54)

RequirementsRequirements

• Avoid deadlock in a segment

• Avoid deadlock when traversing multiple segments

• Ensure routing connectivity when physical connectivity exists

• Avoiding congestion in path construction


ECE 8813a (55)

Construction of SegmentsConstruction of Segments

• Search for initial segment + “regular” segments Unitary segments

• Add one bidirectional restriction in each segment

• Source routing

Starting node

Terminal node

Unitary segment

bridge segment


Failed links

ECE 8813a (56)

Deadlock FreedomDeadlock Freedom

• One routing restriction per segment No cycles in a segment

• Every cycle contains a segment Hence cannot be “closed” to create deadlock

• No cycle from the start node back to itself Cannot create cycles across subnets

• Think of a subnet as a union of 1-D segments


ECE 8813a (57)

PerformancePerformance

From A. Mejia , J. Flich, J. Duato, Sven-Arne Reinomo and Tor Skeie, “Segment Based Routing: An Efficient Fault-Tolerant Routing Algorithm for Meshes and Tori,” Proceedings of the International Parallel and Distributed Processing Symposium, April 2006.


ECE 8813a (58)

Region-Based RoutingRegion-Based Routing

• Problem: reduce the size of routing tables for on-chip routers

• Recognize that finite state machine routers implicitly check for region membership Think meshes

• Generalize the idea of regions Can naturally be adapted for fault tolerant routing


ECE 8813a (59)

Example of RegionsExample of Regions

• What are the characteristics of these regions? Note #regions = f(routing options)

{node set}{node set}

From J. Flich, A. Mejia, P. Lopez, and J. Duato, “Region-Based Routing: An Efficient Routing Mechanism to Tackle Unreliable Hardware in Networks on Chip,” Proceedings of the First International Symposium on Networks on Chip, May 2007


ECE 8813a (60)

ApproachApproach

• Observe that table-based routing is really region based Each entry identifies a region

• Merge entries into compact region specifications at each switch

• Region construction is based on the paths Any set of paths fault tolerant routing No virtual channels


ECE 8813a (61)

Region ConstructionRegion Construction

• Start with routing restrictions• Compute deadlock free-paths• Compute regions • Coalesce regions (pack if necessary to cap

#regions)• Construct routing table



@each node @each node

ECE 8813a (62)

Creating RegionsCreating Regions

• Coalesce routing options based on inputs and outputs

• Represents a compact routing table



ECE 8813a (63)

ImplementationImplementation

• Initialization of region registers and parallel evaluatiom of all regions



ECE 8813a (64)

Hardware OverheadsHardware Overheads

• Each region requires Four registers that define the region Mask registers to define input and output ports Logic to determine routing options

• Hardware cost grows as the number of regions Growth as f(network_size) is much slower



ECE 8813a (65)

Microarchitecture IssuesMicroarchitecture Issues

• Routing algorithm performance is sensitive to resource allocation schemes in the router

• Key resource management functions include Routing function Selection function Arbitration/scheduling

• Mismatch can lead to poor performance

ECE 8813a (66)

Resource Allocation: Selection FunctionsResource Allocation: Selection Functions

• Selection function may be oblivious or informed Common to favor minimal paths and lightly loaded links

• Examples: Meshes: minimum congestion, maximum flexibility, straight

lines• Unlike routing functions, selection function must be

serialized Result updates the channel status – a centralized resource

VC status/control

VC buffer

Selection function

Routing function

Input VC

Output VCs

ECE 8813a (67)

Selection FunctionsSelection Functions

• Favor adaptive channels Improve probability of escape channel availability Time dependent selection functions: give adaptivity a

chance

• Selection functions for real time traffic Separate best effort and guaranteed packets via VCs

or virtual networks

• Note the impact on bisection utilization

• Selection functions for cache coherent systems?

ECE 8813a (68)

Resource Allocation: ArbitrationResource Allocation: Arbitration

• Tradeoffs: channel bandwidth vs. message sizes and types Mix of buffering strategies across message types

• All three strategies must be co-designed for a tuned system

arbitration

Flow controlMessage size

ECE 8813a (69)

Routing, Selection & ArbitrationRouting, Selection & Arbitration

• Input driven vs. output driven scheduling Output driven scheduling requires replication of

routers amongst inputs

• Lessons from the microprocessor world Impact of complexity, workloads, and concurrency

• Impact of…. Symmetry of the topology Locality of traffic Packet size

Locality, uniformity

Irregular, hot spot

deterministic routing

adaptive routing

ECE 8813a (70)

Summary Summary

• Best routing algorithm driven by multiple considerations

Deterministic vs. adaptive

Uniform vs. non-uniform traffic

Packet sizes

Power envelopeOn-chip vs. off-chip

Locality of traffic

Hot spots

Compatible micro-architecture

Symmetric vs. asymmetric topology

routing algorithms

Documents