Issues and Trends in Router Design

Presented by:Sudarshan Krishnaprasad

August 28th, 2003

Contents Introduction Components of a router

Anatomy of a router Network processor

Functionalities of IP routers Route processing Packet forwarding Route look up Special functions of a router

Router Architecture IP Packet Processing in a Distributed Router Architecture Switching Fabric Conclusion References

Introduction

What are routers? Determines the route your packet takes in travel from the

source to destination and also routes them in that direction. When does routing take place?

Sending to an IP address IP packet comes to the router

What does the router do? The router determines where the destination is and how it

should be reached by consulting its routing tables Why is it difficult?

Heterogeneous link technologies Scheduling support for differential service. Participate in complex distributed algorithms to generate

routing tables Insatiable need for bandwidth

Introduction The three different routing classes are:

Access routers:These allow homes and small businesses to connect

to the internet service provider Enterprise routers:

They link tens and thousands of computers within a campus or an enterprise

Backbone routers:They link together ISP’s and enterprise networks

with a long distance trunks.

Components of a Router Input Port:

It is the point of attachment of physical link and is the point of entry for incoming packets

Switching fabric: This interconnects the input ports and the output

ports. Input queued router and output queued router.

Output port: Stores the packets and schedules them for service

on an output link Routing Processor:

Participates in routing protocols and creates a forwarding table that is used for packet forwarding.

Components of a Router

Line interfaces: physically attach multiple transmission systems to the node and provide framing functionality

Network processors: provide the intelligence and processing power to analyze packet headers, look up routing tables, classify packets based on their destination and source addresses and other control information and (often complex) rules, and provide queuing and policing of packets

Switch Fabric: which provides high-speed (ideally non-blocking) interconnection of the node’s packet processing units

System Processor: which performs control point functions such as route computation and box and network management.

Components of a Router Network Processors

Fast, economical, but rarely flexible enough for rapid adaptation to protocol or standard changes.

Will implement performance critical packet forwarding functions in software

The bit stream processors receive the serial stream of packet data and extract the information needed to process the packet, (IP source/destination address, type of service (TOS) bits, or TCP source/destination port numbers)

The packet is then written into the packet buffer memory The extracted control information is fed to the processor

complex, which constitutes the programmable unit of the NP.

Components of a Router if needed, extracts additional

information from the packet and submits the relevant part to the search engine, looks up the medium access control (MAC) or IP address, classifies the packet, or does a virtual circuit/path identifier (VCI/VPI) lookup if the packet is recognized as an asynchronous transfer mode (ATM) cell using the routing and bridging tables and appropriately designed hardware assists.

Based on the results returned, the processor instructs the scheduler to determine the appropriate departure time of the packet.

Upon packet transmission through the bit stream processor, the necessary modifications to the packet header are performed.

Functionalities of a Router Generally consists of several network interfaces to the

attached networks, processing modules, buffering modules, internal interconnection networks

Generic IP router architecture as shown: Interface cards perform the inbound and outbound

packet forwarding CPU in router performs functions like path

computations, routing table maintenance, and reach ability propagation.

Basically, Route processing Packet forwarding Router special services

Functionalities of a Router

Route Processing Routing protocols are the means by which routers gain

information about the network. Routing protocols map network topology and store their view of that topology in the routing table. E.g.: Routing Information Protocol and OSPF

Routing table consists of entries that specify the destination and next hop router through which the datagram should be sent.

Construction and maintenance using routing protocols Route calculation consists of determining the route: network,

subnet, network prefix or host. Static routing

Routing table entries are created by default (bootstrap file, ICMP) Tables cannot be configured once created Cannot reroute across a disabled link and alarms Network manager maintains the routing table Basically for small networks, there is a single connection point to

other networks, no redundant routes.

Functionalities of a Router Dynamic routing

Used for internetworking across wide area networks Automatically reconfigures the routing table and

recalculates the least expensive path Routers broadcast packets to signify their presence and

communicate, cost of connections and load levels Reconfiguration must be quick without incorrect

information The routing table is updated dynamically If multiple routes are found, the protocol chooses the best

route and bypass the problems of broken links. Two principal routing protocol types

Interior gateway protocols (IGP’s) within an autonomous system eg: RIP, OSPF and IS-IS

Exterior gateway protocols (EGP’s) between autonomous systems

Eg: Border Gateway Protocol (BGP)

Packet Forwarding Forwarding Process

Packet forwarded from I/p to O/p based on contents of IP Packet’s networks layer remains unchanged, data-link layers and

physical transmission schemes change radically Checks for errors and sees If entire packet is received, if

malformed entire packet is discarded Verifies the TTL and makes sure the packet does not circulate

forever, If TTL reaches 0, ICMP message sent to host, router changes only TTL and checksum

Unicast, multicast and broadcast packets Broadcast storm for unicast packets Destination IP used to lookup routing tables, indicating the

interface to forward out of and IP of the next router. Link address is determined and prepended in the IP.

Large Packet is divided into fragments ARP (Address resolution protocol) converts the IP of the next hop

to a data-link, and similar process is repeated if necessary

Route Lookup

Looking up a route in routing table (finding the longest prefix match) is a performance bottleneck

The address classes A, B, and C (allowing sites to have 24, 16, and 8 bits respectively for addressing) proved too inflexible and wasteful of the resources, and Class C addresses were given out which increased the tables massively.

Classless Inter-Domain routing was introduced which aggregated the addresses where advertised as a whole instead of individual addresses.

Routers have a default route (0.0.0.0) usually overridden, eg: if one router connects to the internet then all routers use this default route to connect to the internet.

The longest prefix match used radix trees or modified particia trees

These lookup alogirithms have complexity based on the number of bits, 32 for IPv4 Performance depends on data, might have to backtrack

Some routers use route cache to cache the recent addresses to avoid the routing table (not suitable for backbone routers)

Route Lookup Typically 2 types of packets

arrive at the router, packets to the router or packets forwarded to the

network.

Check the packets explicitly if it belongs to the router, saves time by not going to the massive routing table.

The recent algorithms for routing table lookups use small and compressed data structure using a small amount of memory. Hence result in faster lookups

Special Functions of Routers Authentication and access services (Packet

filtering for security). Routers often used between company networks

and the internet to prevent unauthorized access Foils most attacks and makes it easy to back

track (ISP). Also used in congestion control. (Discadring

packets algorithm) Used to define traffic statistics (ISP to

implement usage based charging schemes)

Router Architecture Processing power, memory bandwidth and

internal bus bandwidth are main bottle necks. Routers are mainly dedicated systems not

running any specific application tasks Every network interface is equipped with proper

processing power and buffer space to solve the processing power problem

internal bus bandwidth can be solved with an effective switch, allowing simultaneous transfers between network interfaces.

To cope with high aggregate packet rates, we need efficient processing and memory components, switch based router architecture is used in most commercial routers for this reason.

Router Architecture Each network interface provides the

processing power and the buffer space needed for packet processing tasks related to all the packets flowing through it.

They perform specific tasks and offer QoS, depending on the protocol the port specifies the packets to predefined classes

The exact features of the processing components depend on the functional partitioning and implementation details.

Interfaces interconnected via a high performance switch and a CPU is used to perform centralized tasks

The MSI performs physical layer functions and SFI is responsible for preparing the protocol on its next hop

Based on the kind of functionality desired by the protocol there are two paths involved.

Router Architecture

Critical data path processing (fast path)

The processing tasks directly related to packets being forwarded through the router can be referred as the time critical processing tasks.

These processing tasks comprise all protocols involved in the critical path (e.g., Logical Link Control, (LLC) Sub network Access Protocol ( SNAP) and IP) as well as ARP which can be processed in the network interface because it needs direct access to the network, even though it is not time critical.

The time critical tasks mainly consist of header checking, and forwarding (and may include segmentation) functions.

Generally, the fast path of IP routing requires the following functions:

IP packet validation, destination address parsing and table lookup, packet lifetime control (TTL update), and checksum calculation.

While they are not trivial, it is possible to implement them in hardware, thereby providing performance suitable for high-speed routing.

Router Architecture Non critical data path

processing (slow path)

Packets destined to a router, such as maintenance, management or error protocol data are usually not time critical.

Typical examples of these non-time critical processing tasks are error protocols (e.g., ICMP), routing protocols (e.g.,RIP, OSPF, BGP), and network management protocols (e.g., SNMP).

These processing tasks need to be centralized in a router node and typically reside above the network or transport protocols.

Network interface implements all IP packet forwarding protocols and all others are by the CPU.

Router Architecture

IP packet process1. IP header validation: forwarding logic verifies all layer 3

information (header length, packet length, protocol version, checksum etc)

2. Route lookup and header processing: Looks up the IP destination to lookup output port and performs forwarding operations (TTL decrement, checksum increment etc)

3. Packet classification: forwarding engine examines layer 4 and higher layer packet attributes relative to QoS and access control

4. Performs the following functions in parallel:• Associates the packet to the right output port• Redirects the packet to a different destination• Drops the packet in case of congestion, or security policy and • Performs the appropriate accounting functions

5. The forwarding engine notifies the system controller that packet has arrived.

6. The system controller reserves a memory location or the arriving packet

IP Packet Process

7. Once packet is in shared memory, system controller assigns the appropriate o/p port.

8. O/P port extracts the packet from shared memory

9. Once packet is received in O/P port memory location is made available for new traffic.

Switching Fabric Responsible for transferring packets

from the input modules to the appropriate output modules

Different popular approaches: Shared memory, output queued. Shared medium, output queued. Distributed output buffered, output queued

(commercially used) Space division switch fabric: the crossbar

switch (commercially used)

Switching Fabric Shared medium switch fabric:

Packets routed by means of a shared medium e.g.: bus, ring or dual bus, bus is the simplest

Used TDM to share the medium Incoming packets are

sequentially broadcasted on the bus, filters at the O/P determine if the packet is destined to that O/P

If packets are more than the port speeds, Input queuing is used.

Normally used for broadcast approach.

Switching Fabric Shared memory switch fabric

Packets are converted from serial to parallel form and written sequentially into a dual port RAM.

Outgoing packets are de-multiplexed and then converted from parallel to serial again.

Output buffer approach with a centralized buffer for all ports. Main advantage of statistical sharing

Must write into memory at least once and memory access time imposes a limit on packet transfer.

Multicasting not a natural approach. Packet must be in memory until read by all multicast ports.

Switching Fabric

Distributed output buffered switch fabric

Independent paths exist between all N2 possible pairs of inputs and outputs. In this design, arriving packets are broadcast on separate buses to all outputs.

Address filters at each output determine if the packets are destined for that output.

Appropriate packets are passed through the address filters to the output queues

No conflict among the I/p and o/p ports, output queuing approach, multicast is natural

All hardware operates at the same speed.

Switching Fabric Used in most designs as no speed up

factor is present. Unfortunately, the quadratic N2 growth

of buffers means that the size N must be limited for practical reasons

The port speed S can be increased to the physical limits of the address filters and output buffers.

Hence, this approach might realize a high total throughput NS packets per second.

Switching Fabric Space division switch fabric: the crossbar switch This architecture appears to be especially convenient for

providing QoS guarantees, serious limitations are :

the switch fabric must operate at a rate at least equal to the aggregate of all the input links connected to the switch.

One of the most popular interconnection networks used for building input buffered switches is the crossbar because of its (i) low cost, (ii) good scalability and (iii) non-blocking properties.

In this architecture buffering occurs at the inputs, and the speed of the memory does not need to exceed the speed of a single port.

Given the current state of technology, this architecture is widely considered to be substantially more scalable than output buffered or shared memory switches, but presents a lot of technical challenges, eg HOL

One way of reducing the effect of HOL blocking is to increase the speed of the input/output channel

Switching Fabric

•Another way is VOQ, Instead of maintaining a single FIFO at the input, a separate queue per each output can be maintained at each input.•Thus a combination of input buffered and output buffered switch is required, i.e., Combined Input and Output Buffered (CIOB).•Here, the shared buffer provides a repository for the heads of all input queues and hence serves as a contention resolver.

Switching Fabric

ConclusionIP provides a high degree of flexibility in building

large and arbitrary complex networks. Internetworking routers capable of forwarding aggregate data rates at terabits per second will be the need for performance network environment.

We mainly discussed the architecture and design overview of the components that have highest effect on performance.

Significant advances have been made in router design to address the most demanding needs and promise to scale to the necessary data rates and system sizes. A system and software structure that optimally distributes both packet by packet and control processing tasks among the processors will be crucial for a network router.

References Issues and Trends in Router Design:

S. Keshav and R. SharmaCornell University

On the Design of IP routers: Router Architectures James Aweya Nortel Networks

Technologies and Building Blocks for Fast Packet Forwarding Werner Bux et. Al IBM