TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

Software Datapath Acceleration for Stateless Packet Processing

June 22, 2010

Ravi MalhotraSoftware Architect

FTF-NET-F0817

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 2

► What can be accelerated• Stateless and stateful• Various applications• Sample TCP offload

►Soft Data Path Engine• Architecture• Feature set• Packet flow• DPE API

►Performance• Soft DPE advantage• Leverage key hardware offloads

Agenda

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

Stateful Path and Stateless Packet Processing

►Most network packet processing protocols can be broken down into two paths

• Stateless path, also known as the data path, requires quick and efficient switching/routing of packets

Can be broken down into packet identification (classification) and forwarding• Stateful path, also known as the control path, requires more processing and has

more inherent latency than the data path

►Stateful control path requires 90% of the code and is used 10% of the time.

►Stateless data path requires just 10% of the code and is used 90% of the time.

►This session focuses on how to accelerate the 10% of the code in the stateless path to increase packet processing performance.

3

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 4

Stateless Data Path for Different Applications

Application Data Path Control Path

Layer 2 bridging FDB lookup, VLAN add/delete, Learning

Aging, STP

IPv4 forwarding Dest-cache lookup, L2 modify

LPM route-table lookup, ARP, IP Options

NAPT 5-tuple lookup, IP/Port/L2 modify

Connection setup/destroy, policy, ALG

Firewall Access control list, pin-holes Stateful packet inspection, ALG

IPSec 5-tuple lookup, encap/decap+ crypto

SA setup, security policy

QoS Enforcement – sched, police, congestion, shaper

Policy, provisioning

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 5

Connection AgingSubsystem

Networking Stack

NetfilterConnection

Tracking

DPA Control Module Event Handlers

Asynchronous Low Level API

NetFilter Hooks Asynchronous Offload

Mechanism

Stateless Data Path Engine

Rule/Stream Tables

Data Path

Classifier/Action Table

Connection established/

assured event

New Connection

request

Create Rule

Create StreamAck Ack

Dynamic Connection Offloading with Soft DPA

(L4 TCP NAPT flow, no QoS)

Control Path

Lkup MissLkup HitPkt Flow

Ctrl Flow

Connection Offload Success

Probe Status

Run Aging

Ageoutlist

Aging

status

FIN/ACK (terminate pkt)

Connection destroyed

Delete Stream

Delete Rule

Conn destroy/ageout

Connection Deleted

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 6

Current – Linux Forwarding

eTSEC or FM-QM-BM eTSEC or FM-BM-QM

Linux Network Stack

eTSEC/QM Driver eTSEC/QM Driver

Control Plane Applications (DHCP/DNS/IGMP etc)

Linux KernelLinux User-space

e500 Cores

SEC (QM)SEC/QM Driver

Platform SoC

P1, P2 P1010, P1020, P2020, 85xx, 83xx

P3, P4, P5 P4080, P3040, P5020

Control Packet flow

Configuration flow

Data Packet flow

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 7

Stateless Data Path Procesing in QE

Network Interface Network Interface

Linux Network Stack

UCC UCC

Control Plane Applications (DHCP/DNS/IGMP etc)

Interworking microcode

Control LogicNetwork Driver

Linux Kernel

Linux User-space

RISC Microcode

DPE API

e500 Core

QE R

ISC C

ores

Existing Solution for 8360 and 8323

SEC

Completely re-used from

Linux

Platform SoC

P1, P2 8323, 8360, 8569

Control Packet flow

Configuration flow

Data Packet flow

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 8

Stateless Data Path Processing in Software

VeTSEC

Linux Network Stack

VeTSEC Driver VeTSEC Driver

Control Plane Apps (DHCP/DNS/IGMP/IKE etc)

Linux Kernel

Linux User-space

e500 Cores

Completely re-used from

QE based Platforms

SEC

VortiQa CP + NMS

Platform SoC

P1, P2 P1010, P1020, P2020, 85xx

P3, P4, P5 N/A

DPE API

SEC Driver

Control Packet flow

Configuration flow

Data Packet flow

VortiQa Network Stack

Soft Data Path Engine

VeTSEC

Control Logic

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 9

Soft Data Path Engine Feature List

►Stateless packet processing (all stateful processing including ALG, SPI firewall, ARP, routing, learning etc. done by control-path)

►Offloads following stateless processing• IPv4 forwarding• NAPT/firewall (ACL) processing• Layer 2 switching with VLAN• IPSec forwarding• Quality of service

►Support for the following interfaces:• Ethernet• VLAN• PPPoE• WLAN

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 10

Soft Data Path Engine Feature List (cont.)

►Maintenance• Per-flow statistics and aging

►Platform support• Multicore support over VeTSEC

►Provides a standard configuration across platforms• Integrates seamlessly with Linux networking stack and applications using

SWANG package• Integrates seamlessly with VortiQa networking stack and customer network

stacks

►Leverages hardware acceleration (hashing, scheduling, classification, security) where available

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 11

Data Path Engine

Soft Data Path Engine – Functional Model

Packet Parsing

Rx Processing

Classification/Lookup

Policer

Ingress HM-ops Egress HM ops

Scheduler/Shaper

Tx Processing

Backplane Processing / Inter-plane/processor communication

Control Plane (s)

Other DataPath Engine

Network Interface Network Interface

Recycle

Recycle

Application Offload– Crypto, PME etc.

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 12

ETHLogicalPort1

MA

C

Scheduler

Shaper

ETHLogicalPort2

MA

C

Scheduler

Shaper

Data Path Engine API Architecture Overview

LookupControl

Path

Tx Q1

Rx Queues

Stream1

BufferManager1

Stream2

BandwidthManager1

HdrMan1

Tx Q2

Stream3

Tx Q3 Send(data, stream3)

Match

Stream5

Rx Q1

Rx Q2

Stream4

error traffic

PHY Classification

Rule_1Rule_2Rule_3

BandwidthManager2

Tx queues

BufferManager2

HdrMan3

PHY HdrMan2

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

Soft Data Path Engine Performance Advantage

Results on P2020 RDB - 1200/600/400 : 2-core SMP Linux

13

Significant (2x to 5x) performance improvement over native Linux

IPv4 NAPT IPSec

0.00

1.00

2.00

3.00

4.00

5.00

6.00

0

500

1000

1500

2000

64 390 1500

% Increase

Thro

ughp

ut in

Kpp

s

NAPT ‐ Linux NAPT ‐ Soft DPA % Diff

0.00

0.50

1.00

1.50

2.00

2.50

0

500

1000

1500

2000

64 390 1500

% Increase

Thro

ughp

ut in

Kpp

s

IPv4 ‐ Linux IPv4 ‐ Soft DPA % Diff

0.00

0.50

1.00

1.50

2.00

2.50

3.00

0

100

200

300

400

500

600

64 390 1456

% Increase

Thro

ughp

ut in

Kpp

s

IPSec ‐ Linux IPSec ‐ Soft DPA % Diff

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

Soft Data Path Engine Multicore Scaling

Results on P2020 RDB - 1200/600/400: 1-core non-SMP vs. 2-core SMP Linux

14

Scaling factor of > 1.8x when migrating from 1-core to 2-core

Scaling Limited by SEC HW

IPv4 NAPT IPSec

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

Soft Data Path Engine Flow Scaling

Results on P2020 RDB - 1200/600/400 : 2-core SMP Linux – 64 byte traffic

15

Low performance degradation for handling multiple flows

IPv4 NAPT IPSec

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 16

Data Path Hardware Acceleration

Core(s)

Ingress Offload Egress OffloadAutonomous Processing

Look-AsideOffload

Drivers/API

Network Stack (SMP optimized)

GenericOffload

Autonomous aware

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 17

Hardware Acceleration Support

Offload Feature Advantage

Hash calculation Packet distribution to multiple cores, flow-pinning, table lookup

Coarse classification Offload stateless ACL processing

Ingress

Packet parsing Avoid software overhead

Hardware buffer management

No buffer alloc/free operations in softwareGeneric

Hardware queue management

Simpler packet Rx/Tx, efficient stashing (to L1/L2), leaves room in cache for other data

Egress Hardware QoS Avoid software overhead, mitigate DoS attacks, prioritize CPU cycles

Core Backside L2 cache Faster access for multiple flow tables

Look-Aside

Protocol-aware cryptography

Offload protocol encapsulation/decapsulation, sequence tracking etc.

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 18

Hardware Acceleration Advantage

100.00

150.00

200.00

250.00

300.00

Baseline IPv4 + QoS

Shaping + WFQ

WRED Policing Hash results in 

FD

Parse results in 

FD

HM ops in HW

HW Buffer Mgmt

HW Queue Mgmt

Stash on Dequeue

BS L2 cache

Relative speedu

p

Thro

ughp

ut in

kpp

s

Cycles Throughput (kpps) Absolute Tput %

Hardware Acceleration provides upto 2.5x improvement

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLinkand VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 19

Summary

► Software data path engine• Optimized packet processing path• Consistent interface across platforms• Easy integration with network stacks• Single solution across QorIQ LE/ULE platforms

► Performance advantage• Flexibility to leverage hardware acceleration• Optimized for multicore scaling

TM