negst2008 @ toulouse 2008/6/24 network monitoring on 10gbe y. kodama itri (information technology...

NEGST2008 @ Toulouse 2008/6/24

Network monitoring on 10GbE

Y. KodamaITRI (Information Technology Research Institute)

AIST (National Institute of Advanced Industrial Science and Technology)


ITRI, AIST

• GTRC (Grid Technology Research Center) was finished March 2008.

• Research activities of GTRC were merged to ITRI (Information Technology Research Institute), and continues research of Grid middleware and infrastructure.

• The director of ITRI is Sekiguchi who was the director of GTRC.


GtrcNET-10

• GtrcNET is a fully programmable hardware network box.

• GtrcNET-1, which has 4 ports of GbE, was produced in 2003.

• GtrcNET-10, which has 3 ports of 10GbE, was produced in 2005.


Architecture of GtrcNET-10p3FPGA(XC2VP100)

10GbEXENPAK.

10GbEXENPAK

10GbEXENPAK

10GbEMAC

10GbEMAC

10GbEMAC

4bit x 3.125GHz

SO-DIMMDDR333

SO-DIMMDDR333

SO-DIMMDDR333

MICTORUSB2.0 GPS

SystemACE/CF

64bi

t x

156.

25M

Hz

64bit x 162MHz x 21GBytes


Functions • Delay emulation: Up to 800 ms delay without packet losses by 1 us step. • Random frame loss: The rate of frame losses is controlled by 2-31.• Output bandwidth Control: pacing (154Mbps-10Gbps) or shaping / policin

g by token bucket (1Mbps-10Gbps)• Buffer size control: It controls the buffer size storing input frames up to 1

GByte by 1 KByte step. • Frame generation: It generates UDP frames with several parameters, such

as frame length, bandwidth, burstiness, VLAN tag, etc. • Bandwidth measurement: It measures all of input and output bandwidth ev

ery 10ms. Two of ports are measured more precisely every 100 us.• Per-stream bandwidth measurement: using stream ID specified by any 8

bit field in header, per-stream bandwidth is measured every 1 ms. • Frame capture: It captures frames in 1 GBytes of memory. It is available to

capture any combination of all frames or filtered frames, and whole fields or selected header fields.

• Capture Forwarding: Captured header fields are transmitted by frames. The number of captured data in a frame is controlled. Destinations are distributed up to 16 nodes.


Per-stream bandwidth

GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0

1/ge01 1/xg1 CH0 CH1

grmpi031/ge02

Str 1

Str 2

1/xg2 1/ge25

grmpi041/ge26

Stream ID Field


Capture frames

GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0

1/ge01 1/xg1 CH0 CH1 1/xg2 1/ge25

CH2

Capture only one-way trafficSpeed of reading captured data is slow via USB.Duration of capture is very short

full frames: 800ms for wire-rate trafficselect header: 17sec for wire-rate traffic, when first 64bytes in 1500 bytes frame are captured,

duplicate

Store off-chip memory 1GB

CH0.RxorCH1.Rx

gnetctrl3

USB


Capture Forwarding

GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0


CH2

1/xg4grmpi05

1/ge19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

Application data

Captured data

1/xg2 1/ge25

CH0.RxorCH1.Rx

capture forwarding traffic: 550Mbps if 64 bytes of header in 1500 bytes IP length are captured in wire-rate traffic, and 10 date are packed in a forwarding frame.

Store off-chip memory 1GB

duplicate


Capture bidirectional traffic (1)

GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0

1/ge23 1/xg1CH0 CH1

CH2

1/xg4grmpi05

1/ge19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

Application data

Captured data

1/xg2 1/ge25

CH0.Rx

GtrcN

ET

-10p3

(S/N

:2)

CH0 CH1

CH2

USB

grmpi04

eth1: 192.168.2.104(00:07:E9:3A:C5:9F)

CH1.Rx1/ge21 1/xg3

two GtrcNET are required.



GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

C4948

vlan2

DE

LL 6248vlan10

0

Gi1/23 Te1/49 CH0 CH1

CH2grmpi05

Gi1/19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

1/xg2 1/ge25

CH2.RxApplication data

Mirrored data

Captured data

Mirror: Rx/Tx/Both

Config:192.168.2.232

Gi1/47

Te1/50

sum of bidirectional traffic should be less than 10Gbps. mirrored port should be able to receive packets.



Application data

Mirrored data

Captured data

grmpi01 grmpi02

GtrcNET-10p3 (S/N:2)

Gi1/23 Gi1/25Gi1/18 Gi1/32

grmpi04

grmpi05

Gi1/19

Gi1/21

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

eth1: 192.168.2.104(00:07:E9:3A:C5:9F)

gnetctrl3

USB

150.29.149.156

Config:192.168.2.232

Gi1/47

Mirror :Rx

Mirror: Tx

Te1/50CH1

Te1/49

CH0

C4948

vlan1

C4948

vlan2



grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0


1/ge23

CH0

1/xg2

1/ge251/xg3 1/xg4

grmpi051/ge19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

CH1

Mirror: Rx, Tx

Application data

Mirrored data

Captured data

1/xg1



GtrcN

ET

-10p3

(S/N

:4)

grmpi01 grmpi02

DE

LL6248

vlan1

DE

LL 6248vlan10

0


grmpi051/ge19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

1/xg4 1/ge25

CH2.RxApplication data

Mirrored data

Captured data

Mirror: Rx, Tx

1/xg2

1/xg1

CH2


Distributed destination

• Destination are changed in round-robin up to 16 nodes. • Traffic and stored data are divided to the destination nodes.

grmpi01 grmpi02

DE

LL 6248vlan1

DE

LL 6248vlan10

0


1/ge23

1/xg1 CH0

1/xg2

1/ge251/xg3 1/xg4

grmpi051/ge19

eth1: 192.168.2.101 eth1: 192.168.2.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

CH1

Mirror: Rx, Tx

Application data

Mirrored data

Captured data

grmpi04

Gi1/21

eth1: 192.168.2.104(00:07:E9:3A:C5:9F)


Packet filtering in capturegrmpi01 grmpi02

DE

LL 6248

Cisco 494

8


1/ge1

CH0

1/xg4

1/ge11/xg1 Te1/49

grmpi05

1/ge26

eth1: 192.168.2.101eth1.101:192.168.101.101eth1.102:192.168.102.101

eth1: 192.168.2.102eth1.101:192.168.101.102eth1.102:192.168.102.102

eth1: 192.168.2.105(00:07:E9:5C:34:9C)

gnetctrl3

USB

150.29.149.156

CH1

Mirror: Rx, Tx

Application data

Mirrored data

Captured data

1/xg3

mark = ((pos0 & mask0)== val0) and/or ((pos1 & mask1) == val1)


Example of packet filtering

• capmark 0 0f 8000 8000:– TTL >= 128

• capmark 1 0a ffff 8100 and 0b ffff 0065– VLAN tag is 101(0x65)

• capmark 1 15 ffff 1389 or 16 ffff 1389– Either srcport or dstport is 5001

• capmark 2 02 00ff 0001– Random 1/256

VLAN frame detect mode is supported.


Summary

• Many monitoring functions are available on GtrcNET-10.

• Requirements for these functions are given by co-research with AIST and INRIA Lyon.

• Experimental results and analysis on real network monitoring will be presented by Paulo INRIA Lyon.

• Please visit our web page

http://www.gtrc.aist.go.jp/gnet/

negst2008 @ toulouse 2008/6/24 network monitoring on 10gbe y. kodama itri (information technology...

Documents