Gaussian Interconnections for On-Chip Networks
Ramón Beivide and Enrique VallejoUniversity of Cantabria, [email protected]
R. Beivide, E. Vallejo 2Microgrid Workshop
Index Introduction: Why a Topology? Dense Gaussian Networks and other topologies Different layouts Routing:
ideas: Adaptive routing, deadlock avoidance, fault tolerance Unicast routing Broadcast Routing
Perfect placement of resources Expansibility:
Increasing number of nodes in a Gaussian network Hierarchical Gaussian networks
Some ideas about cache coherence
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 3Microgrid Workshop
Introduction
Future trends: many PE on a chip Possible interconections: bus, MIN, direct
network Bus-based interconnections do not scale – they
do not provide a sufficient bandwith when there are many PEs. MIN hard to implement in a chip.
Direct networks with a given topology: The way to connect different routers in the chip
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 4Microgrid Workshop
Mesh Network
Number of Nodes N:
N = b x b = b2
Diameter k:
k = (b-1) + (b-1) = 2b-2
i 1+i 2+i -1+i -2+i
2i 1+2i 2+2i -1+2i -2+2i
0 1 2 -1 -2
-i 1-i 2-i -1-i -2-i
-2i 1-2i 2-2i -1-2i -2-2i
b
b
Nb
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 5Microgrid Workshop
Number of Nodes N:
N = (b+1)2 + b2
Diameter k= b + b = 2b 0 1 2 -1 -2 3 -3
i 1+i 2+i -1+i -2+i
2i 1+2i -1+2i
-i 1-i 2-i -1-i -2-i
-2i 1-2i -1-2i
3i
-3i
2 b+1
Diamond Network
2112
Nb
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 6Microgrid Workshop
Torus Network
Number of Nodes N:
N = b x b = b2
Diameter k = b -1
Nb i 1+i 2+i -1+i -2+i
2i 1+2i 2+2i -1+2i -2+2i
0 1 2 -1 -2
-i 1-i 2-i -1-i -2-i
-2i 1-2i 2-2i -1-2i -2-2i
b
b
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 7Microgrid Workshop
0 1 2 -1 -2 3 -3
i 1+i 2+i 2+i 2+i
2i 1+2i 2+i
-i 2+i 2+i 2+i 2+i
-2i 2+i 2+i
3i
-3i
2b+1
Number of Nodes N:
N = (b+1)2 + b2
Diameter k = b
2112
Nb
Dense Gaussian Network
• Same # links as torus, withperipheral links.• Lower mean distance andDiameter.
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 8Microgrid Workshop
Topological properties comparative
Topology Nodes Diameter Aprox. Diam.
Average Distance
Aprox. Aver. Dist
2-D Mesh
2-D Torus
Dense Gaussian
2WN
22 W N 12
12
2
NWWW
2N
122 2 kkN k
13121
2
Nkk
32N
2N
32 N
N222 W3
2W2WN
Lower average distance and diameter
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 9Microgrid Workshop
Area comparativeGaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 10Microgrid Workshop
3i -2-i -1-i 3 2 -i 1
2i 1+2i 1-2i -1+2i 2-i -2i 1-i
i 1+i 2+i -1+i -2+i -3i 1-2i
0 -1 -2 -3
Different Layouts
2314
20191718
0
232422
21
5
131514
78 6
101211 9
16
Different layouts for the same network:•Mesh-like layout•Without peripheral links, bounded link length
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 11Microgrid Workshop
Routing ideas Adaptive routing: in-fligh packets can choose their
(minimal) path from info in the Routing Record (jumps in each direction), depending on congestion or other parameters.
Deadlock avoidance: Bubble routing proposed as a cost-effective deadlock avoidance mechanism (used in IBM’s Blue Gene). Only 2 virtual channels needed per link.
Fault-tolerant routing: Inmunet proposed as a fast, efficient mechanism to detect link failures and restore network performance.
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 12Microgrid Workshop
Unicast Routing
0 1 2 -1 -2 3 -3
1+i 2+i -1+i -2+i
1+2i -1+2i
-i 1-i 2-i -2-i
-2i 1-2i -1-2i
3i
-3i
Route from a to b:Routing record generatedFrom the difference: dest-source (x, y)
-1-i
i
Example:i – (-1-i) = 1+2i (x=1, y=2)1 jump to the right, 2 jumps up Movement from sourcenode to the origin (node 0)generates routing record.
Example 2: The movement makesthe arrow fall outside the originalnetwork Peripheral links used
Translations from surroundingreplicas of the network areconsidered, to obtain an optimal RR
2i
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 13Microgrid Workshop
P 1 2 -1 -2 3 -3
i 1+i 2+i -1+i -2+i
2i -1+2i
-i 1-i 2-i -1-i -2-i
-2i 1-2i -1-2i
3i
-3i
1+2i
NW NE
SW SE
Broadcast Routing
• Triangle-based broadcast• Minimum number of steps• The same for any node (due to peripheral links)• Balanced use of resources• Simple routing based on labels (see abstract)
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 14Microgrid Workshop
Perfect placement of resources
Resource distribution overthe network.
All nodes have resources within a given distance (example: distance 1)
Resource example: I/O portsProcessing elementsMemory banks...
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 15Microgrid Workshop
Expansibility: Increasing # nodes
Increasing Gaussian network: Network can be expanded with the number of nodes necessary to increase diameter in 1 unit: 4k +4.
Alternatively, hierarchical Gaussian networks have been proposed, joining several Gaussian networks with another gaussian pattern. Useful for CMPs sceneries, for example (different latency, link length, etc. in each hierarchical level): Lower level: interconnection between different cores inside a
chip. Fast and reliable, with low latency Higher level: interconnection between different chips. Slower
and with higher latency.
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 16Microgrid Workshop
Expansibility: Increasing # nodes
Gaussian Interconnections for On-chip Networks
Lower level (on-chip) with adense Gaussian pattern.Higher level, with the samepattern.
Central routers will have 8 links:4 internal links4 external links
Route from one node to another:1) Route to the central router of the same gaussian2) Route in the higher level to the desired gaussian.3) Route from the central router of the dest. Gaussian, to the destiny node.
R. Beivide, E. Vallejo 17Microgrid Workshop
Cache coherence in Gaussian networks Recent proposals based in broadcast, such as TokenB
(M. Hill) can beneficiate from a Gaussian interconnection:
Broadcast block requests (latency optimized with Gaussian interconection).
Unicast response with grants (Tokens) to use memory blocks, between different nodes and main memory.
There is no need to maintain a directory for coherence. Broadcast network can work as a bus with a snoopy-like
protocol.
Gaussian Interconnections for On-chip Networks
R. Beivide, E. Vallejo 18Microgrid Workshop
R. Beivide, E. Vallejo 19Microgrid Workshop
Additional comments (not presented) Dense Gaussian Networks are isomorphic to
Dense Midimew Networks. However, these two topologies are not isomorphic in the general case (not dense). In this work, related to Dense Gaussian networks, properties studied for both Gaussian and Midimew topologies are presented.
References in the next slide will be thus referred to both Midimew and Gaussian networks
R. Beivide, E. Vallejo 20Microgrid Workshop
Commented References (I) Midimew networks were first introduced in:
R. Beivide, E. Herrada, J.L. Balcázar, Agustín Arruabarrena, “Optimal Distance Networks of Low Degree for Parallel Computers”. IEEE Transactions on Computers, Vol. 40, No 10, Oct 1991, pp. 1109-1124. This paper introduces properties, analysis and some rectangular (mesh-like) layouts. Unicast routing is also proposed, but based on labeling nodes with integer labels (instead of Gaussian numbers).
Bounded link-length layouts were introduced in:E. Vallejo, R. Beivide y C. Martínez, “Practicable Layouts for Optimal Circulant Graphs”. Proceedings of the “13th Euromicro Conference on Parallel, Distributed and Network-based Processing”, Lugano, Switzerland, Feb. 2005. A previous work on Midimew folding, which obtained a worse result (maximum link length 4) is the following one:Francis C. M. Lau, Guihai Chen, “Optimal Layouts of Midimew Networks”. IEEE Transactions on Parallel and Distributed Systems, Vol 7, No 9, pp 954-961
R. Beivide, E. Vallejo 21Microgrid Workshop
Commented References (II) Gaussian Networks will be introduced in:
C. Martínez, R. Beivide, J. Gutierrez and E. Gabidulin. "On the Perfect t-Dominating Set Problem in Circulant Graphs and Codes over Gaussian Integers". Accepted for presentation at ISIT’05, September, Australia.This paper also deals with perfect resource placement.
Broadcasting in Dense Gaussian Networks will be introduced in:R. Beivide, C. Martínez, E. Vallejo, J. Gutierrez, C. Izu, “Gaussian Interconnection Networks”. Accepted for presentation at the Spanish Parallelism Conferences, Sept. 05, Granada, Spain.This paper also introduces unicast routing in terms of the Gaussian numbers (instead of integer labels)
Hierarchical Gaussian Networks will be introduced in:Miquel Moretó, Carmen Martínez, Enrique Vallejo, Ramón Beivide, Mateo Valero, “Hierarchical Topologies for Large-scale Two-level Networks”, Accepted for presentation at the Spanish Parallelism Conferences, Sept. 05, Granada, Spain.
R. Beivide, E. Vallejo 22Microgrid Workshop
Commented References (III) Bubble routing is described in
V. Puente, C. Izu, R. Beivide, J.A. Gregorio, F. Vallejo and J.M. Prellezo, “The Adaptative Bubble Router”, Journal of Parallel and Distributed Computing. Vol 61 - nº 9, September 2001
Inmunnet was introduced inV. Puente, J.A. Gregorio, F. Vallejo and R. Beivide. "Immunet: A Cheap and Robust Fault-Tolerant Packet Routing Mechanism". 31th Annual International Symposium on Computer Architecture (ISCA-31), pp. 198-209, 2004.
Token Coherence was presented in:M. M. K. Martin, M. D. Hill, and D. A. Wood. "Token Coherence: Decoupling Performance and Correctness". 30th Annual International Symposium on Computer Architecture (ISCA-30), pp. 182-193, 2003.