network-on-chip (noc): a new soc paradigm dr. konstantinos tatas
TRANSCRIPT
![Page 1: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/1.jpg)
NETWORK-ON-CHIP (NOC):
A New SoC Paradigm
Dr. Konstantinos Tatas
![Page 2: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/2.jpg)
PRESENTATION OUTLINE
Introduction
Part A
Motivation – SoC Communication
Current Solutions
NoC Concept
Part B
Work@MicroLab
Summary
![Page 3: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/3.jpg)
THE MANY CORES ERA
Source: International Roadmap for Semiconductors 2007 edition (http://www.itrs.net/)
![Page 4: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/4.jpg)
THE GROWING GAP: COMPUTATION VS. COMMUNICATION
Taken From ITRS, 2001
2:1
9:1
![Page 5: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/5.jpg)
Design complexity - high IP reuse Efficient high performance interconnectScalability of communication architecture
GROWING CHIP DENSITY
1998ASIC - 0.35 m
2012SoC - 22nm
Memory, I/O
P
![Page 6: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/6.jpg)
Variety of dedicated interfaces Poor separation between computation and communication. Design Complexity Unpredictable performance
TRADITIONAL SOC NIGHTMARE
The architecture is tightly coupled
DMA CPU DSP
MemCtrl. Bridge
MPEGI oo
The “Board-on-a-Chip” Approach
C
System Bus
Control Wires Peripheral Bus
![Page 7: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/7.jpg)
COMPUTATIONAL DEMANDS OF FUTURE MULTIMEDIA APPLICATIONS - MEMORY BANDWIDTH SCALES PROPORTIONAL
Source: K. Uchiyama., VLSI Circuit Digest of Technical Papers, p 6, 2008.
BW~
20
12
-20
15
tod
ay
K. Uchiyama., “Power-Efficient Heterogeneous Parallelism for Digital Convergence”, VLSI Circuit Digest of Technical Papers, IEEE p 6-9, June 2008Jian Li, “3D Integration opportunities and challenges”, ISCAS 2008 tutorial on 3D
![Page 8: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/8.jpg)
SHARED ADDRESS SPACE COMMUNICATIONS
![Page 9: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/9.jpg)
SYSTEM BUS
![Page 10: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/10.jpg)
CROSS-BAR
![Page 11: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/11.jpg)
MULTI-STAGES NETWORK ON CHIP
![Page 12: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/12.jpg)
AN NOC EXAMPLE
•Source: ossum, Intel @ MPSoC’07
![Page 13: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/13.jpg)
Regular topologies: general-purposed on-chip multiprocessors
Custom topologies:
NOC TOPOLOGIES
![Page 14: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/14.jpg)
NOC VS. “OFF-CHIP” NETWORKS
What is Different?
Routers on Planar Grid Topology
Short Point-To-Point Links between routers
Unique VLSI Cost Sensitivity:
Area-Routers and Links Power
M odule
M odule M odule
M odule M odule
M odule M odule
M odule
M odule
M odule
M odule
M odule
![Page 15: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/15.jpg)
No legacy protocols to be compliant with …No software simple and hardware
efficient protocolsDifferent operating env. (no dynamic
changes and failures)
NOC VS. “OFF-CHIP” NETWORKS
![Page 16: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/16.jpg)
No legacy protocols to be compliant with …No software simple and hardware
efficient protocolsDifferent operating env. (no dynamic
changes and failures)
NOC VS. “OFF-CHIP” NETWORKS
Custom Network Design – You design what you need!
![Page 17: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/17.jpg)
No legacy protocols to be compliant with …No software simple and hardware
efficient protocolsDifferent operating env. (no dynamic
changes and failures)
Custom Network Design – You design what you need!
NOC VS. “OFF-CHIP” NETWORKS
M odule
M odule M odule
M odule M odule
M odule M odule
M odule
M odule
M odule
M odule
M odule
Replace
M odule
M odule M odule
M odule M odule
M odule M odule
M odule
M odule
M odule
M odule
M odule
Example1: Replace modules
![Page 18: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/18.jpg)
No legacy protocols to be compliant with …No software simple and hardware
efficient protocolsDifferent operating env. (no dynamic
changes and failures)
NOC VS. “OFF-CHIP” NETWORKS
M odule
M odule M odule
M odule M odule
M odule M odule
M odule
M odule
M odule
M odule
M odule
Example2: Adapt Links
Adapt Links
M odule
M odule M odule
M odule M odule
M odule M odule
M odule
M odule
M odule
M odule
M odule
Custom Network Design – You design what you need!
![Page 19: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/19.jpg)
•Compare the cost of:NoCNon-Segmented Bus (NS-Bus) Segmented Bus (S-Bus) Point-To-Point (PTP)
NOC COST SCALABILITY VS. ALTERNATIVES
![Page 20: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/20.jpg)
Bus NoC
Longer connections
higher parasitic
capacitance
Performance does not
downgrade with network
scaling
Arbitration grows and
becomes a bottleneck
Arbitration and routing are
distributed
Bandwidth is limited and
shared by all cores
Aggregated bandwidth
scales with network size
Latency is wire-speed
once arbitration granted
control
Multiple hops increase
latency
Well-known and simple
concepts
Further study needed
WHY NOC?
![Page 21: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/21.jpg)
Communication infrastructure
Communication paradigm selection
Application mapping optimization
Programming model
Physical design
Design automation/tool-flow integration
WHICH ARE THE MAIN CHALLENGES?
![Page 22: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/22.jpg)
Circuit Switching A real or virtual circuit establishes a direct connection between source and destination.
Packet Switching Each packet of a message is routed independently. The destination address has to be provided with each packet.
Store and Forward Packet Switching The entire packet is stored and then forwarded at each switch.
Cut Through Packet Switching The flits of a packet are pipelined through the network. The packet is not completely bu ered in each switch.ff
Virtual Cut Through Packet Switching The entire packet is stored in a switch only when the header flit is blocked due to congestion.
Wormhole Switching is cut through switching and all flits are blocked on the spot when the header flit is blocked.
BASIC SWITCHING TECHNIQUES
![Page 23: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/23.jpg)
Phases:Circuit SetupTransmissionTear Down
Disadvantages:Exclusive allocation of resourcesLong setup phase
Advantages:High performance - throughput and latencyLow power consumptionLow overhead during transmission phasePredictable transmission
CIRCUIT SWITCHING (ARE THEY NOC?)
![Page 24: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/24.jpg)
PACKET SWITCHING VS CIRCUIT SWITCHING
![Page 25: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/25.jpg)
NOC ROUTER
![Page 26: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/26.jpg)
NoC-based MPSoC
• nodes– Processing Elements (PEs),
such as CPUs, custom IPs, DSPs, etc.
– storage elements (embedded memory blocks),
• Routers• Links• Network Interfaces (NIs)• Often a switch together
with its host node memory is referred to as a tile.
![Page 27: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/27.jpg)
NoC Topologies
• Regular/irregular
• Direct/indirect– each node has a direct point-to-point link to a
subset of other nodes in the system, called neighboring nodes
![Page 28: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/28.jpg)
2D Mesh
IP IP IP
IP IP IP
IP IP IP
R R R
R R R
R R R
•simplest and most popular topology for NoCs.
•Every switch, except those at the edges, is connected to four neighboring switches and one node.
![Page 29: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/29.jpg)
2D Torus
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R
IP
R•layout of a regular mesh except that nodes at the edges are connected to switches at the opposite edge via wrap-around routing channels.
•Every switch has five ports
• The limitation of this topology affects the long end-around connections
![Page 30: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/30.jpg)
Octagon
IP
IPIP
IPIP
IP
IPIP
RR
R
R
R
R
RR
•well-established direct topology found in NoCs.
•ring of 8 nodes connected by 12 bi-directional links.
•links provide two-hop communication between any pair of nodes in the ring
•simple algorithms for fast yet efficient shortest-path routing.
•In case a platform consists of more than eight nodes, the octagon is extended to multidimensional space
![Page 31: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/31.jpg)
Fat-tree and butterfly fat-tree• nodes are connected to an
architecture's external switch • switches have point-to-point links
to other switches. • processing units and memory
modules are assigned to the leafs of the trees,
• switches are placed at the vertices,
• communication involves climbing up and down some part of the tree.
• A pair of coordinates is used to label each node, ($l$, $p$), where $l$ denotes a node's level and $p$ gives its position within this level.
R
IP IP
R
IP IP
R
IP IP
R
IP IP
R
R R
IPIPIP IPIPIP IPIPIP IPIPIP
R
R R R R
R
![Page 32: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/32.jpg)
Polygon• widely accepted
topology• packets travel in a loop
from one router to the next.
• We can add chords to the circle
• if chords are inserted only between opposite routers, the topology is called a spidergon.
R
IPIP
R
R R
R R
IPIP
IP IP
![Page 33: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/33.jpg)
Star
• central router in the middle of the star, • computational resources, or subnetworks, in the spikes of the star. • The capacity requirements of the central router are quite large, • significant possibility of congestion in the middle of the star
IP
IP
IPIP
IP
R COREIP
R R
R R
IP
IP
IP
![Page 34: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/34.jpg)
Flow Control
• intra-switch
• switch-to-switch– Buffered– Bufferless
• end-to-end
![Page 35: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/35.jpg)
ACK/NACK• handshaking protocol • When a sender puts data on
the link, it activates a VALID signal.
• When the receiver is ready to consume the valid data, it activates the corresponding ACK signal.
• If the data is corrupt or there is no buffer space to store them, a NACK signal is activated instead.
• Upon receipt of a NACK, the sender starts resending flits starting from the not acknowledged one
• inherently supports fault tolerance,
• additional buffer space required to keep sent flits in case retransmission is required.
![Page 36: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/36.jpg)
Stall/go
• requires just two control wires
• one going forward, signifying data availability,
• one going backward and signaling either a condition of buffers filled ("STALL") or of buffers free ("GO")
![Page 37: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/37.jpg)
Credit-based
• transmitter has a "credit" counter • initialized to the value of empty buffer slots of the receiver • decrements it every time a flit is sent. • The credit counter must be updated in case the receiver consumes or
forwards a flit and therefore increases its buffer space. • a credit value that is sent back to the transmitter to be added to the current
value of the credit counter. • transmitter stalls when the credit value is zero and • resumes when its value increases again.
![Page 38: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/38.jpg)
NI Design• logic required to connect the nodes to the NoC. • NIs can differ significantly depending on the nature of the
node• Using a NI allows IPs and communication infrastructure
to be designed independently• One end of a NI is connected to a router using the
selected flow control protocol • the other to the node IP • Since most IPs are designed to communicate through a
bus, the NI uses a bus interface • NI is not simply a protocol adapter from a processor bus
to a router port. • Ideally, the NI must offer the processing cores the view
of a shared memory system, and the network itself should be transparent.
![Page 39: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/39.jpg)
NI services
• adaptation services – packetization/depacketization– protocol conversion and clock domain crossing. – absolute minimum services required of the NI so that data can
be sent and received on the NoC• transaction reordering services, • error and flow control services
– error detection and/or correction – request retransmission when required
• route computation services– Source routing
• upper layer services– Cache coherence
![Page 40: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/40.jpg)
Typical NoC Packet Format
• Header– routing and network control information. – In the case of distributed routing the information required is the destination and source
addresses – in the case of source routing the complete routing information is written– In the case of variable packet size a length field is required
• Payload• Tail
– sequence number– error control fields such as hamming code or CRC fields
Header Tail
Packet type
SA DA
Routing
OR
SAHOP
0HOP
1 . . . HOPN
Length SNError
Control
Payload
Control Address Data
![Page 41: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/41.jpg)
Source vs Distributed Routing
• In source routing the entire routing path is computed at the source and appended to the packet. – The routers do not make any routing decisions,
• in distributed routing, the routing path is decided in a hop-by-hop basis at each router even for deterministic routing algorithms. – The only information required to be found in the packet is the
destination address.
• The advantage of source routing is that it requires simple routers and can easily support irregular architectures. Its disadvantage is that it does not provide adaptiveness and requires more complex NIs and packets.
![Page 42: NETWORK-ON-CHIP (NOC): A New SoC Paradigm Dr. Konstantinos Tatas](https://reader035.vdocuments.net/reader035/viewer/2022081501/56649e115503460f94afd019/html5/thumbnails/42.jpg)
Source vs Distributed Routing
R R R R
R R R R
R R R R
R R R R
(0,1) (1,1) (2,1)
(2,2)
(0,1) (1,1) (2,1) (2,2)
E E S PE . . . SR principle
. . . (2,2)
DA
DR principle