washington washington university in st louis [email protected] msr tutorial group exercises

WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

[email protected]

MSR Tutorial Group Exercises

2WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

Fred Kuhns - 04/19/23

Exercise Goals• Get you familiar with the tools we use for configuring,

testing and analyzing the MSR• Get you familiar with building and running an MSR• Not worry about a lot of MSR internal details



General Info• Each group will be using 3 PCs

– Data Source ($DSRC) Linux– Monitor Switch Control Processor ($SWCP), NetBSD– MSR Control Processor ($MSRCP), Linux

• All utilities work on both Linux and NetBSD• You must use NetBSD when building:

– MSR Kernels and Plugins

• The exercises will start out by giving you very specific directions.

• Later exercises leave out some details that you should be able to fill in from what you learned in earlier exercises.



General Info (continued)• MSRs are configured with serial cables to SPCs on Ports

4 and 5. See aliases spc4 and spc5 in the readme file and your login script.

• We are going to be configuring and testing a router based on the WUGS and SPCs from the hardware on up.

• There are a LOT of details involved. Your login script defines macros/variables to simplify the configuration process. In particular the hosts are defined by the variables:

– MSRCP, SWCP and DSRC



Preliminaries• Prior to you doing the exercises we have already

defined a few simplifying macros and built the software utilities:

– See $MSRTUT/README for a complete list of variables and aliases

– setenv REPODIR ~

– setenv WUARL ${REPODIR}/wu_arl

– setenv WUGS ${WUARL}/wugs

– setenv WUTILS ${WUARL}/utilities/

– setenv WUTUT ${WUARL}/TUTORIAL

– setenv MSR ${WUARL}/msr

– setenv MSRTUT ${WUTUT}/msr

– cd ${WUGS}/src

– gmake OSTYPE={Linux | NetBSD} depend

– ${WUARL}/Make.script install



IP Configuration for Exercises

vci = 50

MSR-#Switch

Switch

CP

Host

Data

Src

Host

P4

P1

MonitoringSwitch

P3 P1

P2 P2

P5 P5

P0 P4

P6 P6

P7 P7

P3

NH

MSR

(P3)

P0MSR

CP

IP Router(Switch + SPCs + CP)

192.168.216.1

192.168.200.1

192.168.200.2

192.168.216.2

192.168.216.2

192.168.204.1

192.168.208.1

192.168.220.1

192.168.216.1

192.168.224.1

192.168.228.1

192.168.220.2

vci = 51

192.168.205.2

vci = 50

192.168.204.2

192.168.205.1



Group Host Assignments

Group #

Data Source

${DSRC}(Linux)

Switch CP

${SWCP}(NetBSD)

MSR CP${MSRCP}

(Linux)

Port 4 SPCserial line

Port 5 SPCserial line

1 mouser demand5 demand4demand5tip spc1

demand4cu -l ttyS0 -s 9600

2 tabby deak demand0deak

tip spc0demand0

cu -l ttyS0 -s 9600

3 motti nmvc2 nmvc1deak

tip spc4demand7

cu -l ttyS0 -s 9600

4 abyssinian gussie demand3nmvc2

tip spc1tomcat

cu -l ttyS0 -s 9600

5 tomcat nmvc3 demand8demand5tip spc0

demand7cu -l ttyS1 -s 9600



Preliminaries - Host ATM config• All steps are also recorded in the

${MSRTUT}/cfgs/README file. You can simple open this file, then cut and paste the commands

• Logging on to the Tutorial hosts:– login: msr#

password: Msr#Tut– root password, where needed: TutR00T (those are

zero’s)



Preliminaries - Host ATM config1.Login to the Data Source host, open a local window (Linux)

• open local window - use the alias ‘local’ defined in login scriptalias local '/usr/X11R6/bin/xterm -j -sb -sl 1000 -bw 3 -T "Local" &'

• go to config directory and become the root user

• cd ${MSRTUT};su

• Assuming atmarpd has not be started,

• /etc/init.d/atm start

• ps -agx | grep | egrep "atmsigd|ilmid“

• kill <atmsigd_pid> <ilmid_pid>

• Configure the APIC interface

• ./apic.cfg port 4

• exit root

• exit



Preliminaries - Host ATM config2.Open window on the monitoring Switch CP (NetBSD Host)

• use alias swcp, this is already defined in your login scriptalias swcp 'ssh -f $SWCP /usr/X11R6/bin/xterm -ls -j -sb -sl 1000 -bw 3 -T "SWCP$$SWCP$"'


• cd ${MSRTUT}/cfgs; su

• configure local APIC interface for the switch cp

• ./apic.cfg port 1 swcp

• exit root user

• exit

• Start up GBNSC and configure Monitoring Switch

• ${WUGS}/bin/NetBSD/newGBNSC.init -P3551 -init init.GBNSC.switch1

config.GBNSC.switch1 &

• ${WUGS}/bin/NetBSD/Jammer 0.1 ${SWCP} 3551 group.js



Preliminaries - Host ATM config3.Open window on the MSR CP (Linux Host)

• use the alias ‘msrcp’ defined in your login scriptalias msrcp 'ssh -f $MSRCP /usr/X11R6/bin/xterm -j -sb -sl 1000 -bw 3 -T "MSRCP$$MSRCP$"‘


• cd ${MSRTUT}/cfgs;su

• Assuming atmarpd has not be started,

• /etc/init.d/atm start

• ps -agx | grep | egrep "atmsigd|ilmid“

• kill <atmsigd_pid> <ilmid_pid>

• configure local APIC interface for the switch cp

• ./apic.cfg port 0

• exit root user

• exit



Preliminaries - Building a kernel• Go to your Switch CP host window (must be NetBSD)

• The following environment variables are defined in your login script

• SYS = ${MSR}/usr/src/sys• CONF = ${SYS}/arch/i386/conf• DKERN =${SYS}/arch/i386/compile/MSR_SPC• PKERN = ${SYS}/arch/i386/compile/MSR_SPC_PERF

• Building a kernel (On NetBSD)– cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR– Debug kernel

1. cd $CONF; config MSR_SPC 2. cd $DKERN; make depend;make

– Performance kernel 1. config MSR_SPC_PERF2. cd $DKERN;make depend;make



Preliminaries - Building a kernel• Copy kernel symbols into the filesystem (still on SWCP - NetBSD)

– cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR

– su # become the root user

• vnconfig -t spc24MB -v -c /dev/vnd0d SPC24MB.fs

• mount /dev/vnd0d /mnt

• ${WUTILS}/bin/NetBSD/mknl ${DKERN}/netbsd /mnt/netbsd or${WUTILS}/bin/NetBSD/mknl ${PKERN}/netbsd /mnt/netbsd

• umount /mnt

• vnconfig -u /dev/vnd0d

– exit

– cp /usr/MSR/SPC24MB.fs ${MSRTUT}/FS/SPC24MB.fs



Preliminaries - SPC Kernel continued• Go to the MSRCP window. Copy kernel and filesystem to the

staging area

– cd /usr/MSR

– cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR

– rm /usr/MSR/netbsd.MSR

– cp ${DKERN}/netbsd /usr/MSR/netbsd.DEBUG orcp ${PKERN}/netbsd /usr/MSR/netbsd.PERF

– ln -s /usr/MSR/netbsd.DEBUG /usr/MSR/netbsd.MSR orln -s /usr/MSR/netbsd.PERF /usr/MSR/netbsd.MSR



Preliminaries - MSR Configuration• Go to the MSRCP window

– Download kernels and configure switch• cd ${MSR}/cp/configuration• ./Linux/MSR_Config -f config.MSR

– set port (SPC) default parameters• cd ${MSRTUT}/cfgs• ./defconfig.cmd

– start up GBNSC for use by the GUI monitoring tool• ${WUGS}/bin/Linux/newGBNSC -P3551

config.GBNSC.MSR &• Start up the GUI: go to the Data Source Host’s local window

– /pkg/jdk1.2.2/bin/java -jar ${MSR}/apps/javaGUI/pubgui.jar– then load the msrgui.txt configuration file and arrange windows



Preliminaries - Traffic Generation• We will use sendpkts for all exercises in the MSR tutorial

• sendpkts - uses default socket interface and requires configuration of IP and ATM VCs on data sources

– ${WUTILS}/bin/Linux/sendpkts -S -n 10 -a 1 –l 192.168.216.2 192.168.211.2

• AAL5Generator - uses APIC User mode library and cannot coexist on the same VCIs as sendpkt,

– undo atm: su; sh ${WUGRP}/cfgs/IP/DataSource.atm.undo;exit

– ${WUTLS}/AAL5Generator -pacer 8 \-src 192.168.216.2 -dst 192.168.211.2 \ -frate 1000 -svci 50 -seconds 20 -packet 20

– Use sendCommand utility to change AAL5Generator’s output

• Sending commands to the ports using sendcmd and monitoring debug messages with monmsgs.

– ${MSR}/bin/Linux/monmsgs &

– ${MSR}/bin/Linux/sendcmd -p 4 -c policy -s set_dflags -d 0x30 -t 10000



Exercise 1 - Part 1• Update the Tutorial Directory

1. cd $MSRTUT2. cvs update -A -d

• Default routes using the built in Simple route module and table

3. copy $MSRTUT/cfgs/hosts to /etc/hosts on both the SWCP and DSRC hosts

4. in the MSR CP window, Download a DEBUG kernel and verify

5. In the Data Source window generate IP traffic from Data Source (192.168.216.2) to port 5 of the MSR and verify with the GUI$WUTILS/bin/Linux/sendpkts -x 1000 -p 100 -a 1 -S -s 5050

-l 192.168.216.2 192.168.220.2



Exercise 1: Part 1• Bidirectional traffic using an echo server on SWCP

6. In the SWCP window, start up a IP packet echo server$WUTILS/bin/NetBSD/serv -p 5050 -e 1This will echo all packets received back to he sendpkt command

7. In the DSRC window, send packets to the SWCP host (192.168.204.2)$WUTILS/bin/Linux/sendpkts -x 1000 -p 100 -a 1 -S -s 5050

-l 192.168.216.2 192.168.204.2this command sends single cell packets at a rate of 1000 per second. Verify with the GUI that packets are “flowing” in both directions.



Exercise 1, Part 1: Default Routes

vci = 50

MSR-#Switch

Switch

CP Host

Data

Src

Host

P4

P1

MonitoringSwitch

P3 P1

P2 P2

P5 P5

P0 P4

P6 P6

P7 P7

P3

NH

MSR

(P3)

P0MSR

CP

192.168.216.1

192.168.200.1192.168.200.2

192.168.216.2

192.168.216.2probe(A)

192.168.204.1

192.168.208.1

192.168.220.1

192.168.216.1

192.168.224.1

192.168.228.1

probe(B)192.168.220.2

vci = 51192.168.205.2

vci = 50192.168.204.2

192.168.205.1

probe(C)

IP ForwardingSimple, Defaultroutes, at Port 4



Exercise 1, Bidirectional Traffic

vci = 50

MSR-#Switch

Switch

CP Host

Data

Src

Host

P4

P1

MonitoringSwitch

P3 P1

P2 P2

P5 P5

P0 P4

P6 P6

P7 P7

P3

NH

MSR

(P3)

P0MSR

CP

IP Forwardingusing FIPL atPort 4

192.168.216.1

192.168.200.1192.168.200.2

192.168.216.2

192.168.216.2

probe(A)

192.168.204.1

192.168.208.1

192.168.220.1

192.168.216.1

192.168.224.1

192.168.228.1

probe(B)

vci = 50

192.168.204.2

Destination Route = OutVIN = <P1, S0>,Maps to Port 1, VCI = 50

probe(C)

serv (echo)

sendpkts



Exercise 1, Part 2• Enable the FIPL module and define the routing tables

– In the MSRCP window

• turn fipl on at port 4${MSR}/bin/Linux/sendcmd –p 4 –c policy –s set_fipl –t 10000

• Install routes port 4, add route to port 5, subport 0 ${MSR}/bin/Linux/fip –s –p 4Enter command (h for help): a 192.168.220.0/24 320

– Verify, in the DSRC Window execute the command• $WUTILS/bin/Linux/sendpkts -x 1000 -p 100 -a 1 -S -s 5050

-l 192.168.216.2 192.168.220.2

• Verify traffic follows correct route with the GUI



Exercise 1, Part 3• Add routes from port 4 to port 1, subports 0 and 1

– in the MSRCP window• add routes to existing fip session

Enter command (h for help): a 192.168.204.0/24 64...Enter command (h for help): a 192.168.205.0/24 65

• quite fip Enter command (h for help): quite

• Define the return routes from port 1 to port 4– in the MSRCP window

• To simplify the process, the script fipl.cmd enables fipl and defines default routes to all virtual interfaces.$MSRTUT/cfgs/fipl.cmd port 1or to enable fipl and define routes on all ports $MSRTUT/cfgs/fipl.cmd



Exercise 1, Part 3• Send packets from the data source (DSRC) to the Switch CP

host (SWCP) at its alias IP address on subnet 192.168.205.– Open a second local window on the data source host– in one window send packets to 192.168.204.2

• $WUTILS/bin/Linux/sendpkts -x 1000 -p 100 -a 1 -S -s 5050-l 192.168.216.2 192.168.204.2

– in the other window send packets to 192.168.205.2• $WUTILS/bin/Linux/sendpkts -x 1000 -p 100 -a 1 -S -s 5050

-l 192.168.216.2 192.168.205.2

– Verify bidirectional traffic using the GUI - the echo server should still be running on the switch CP host.



Exercise 1, Part 3: SubPorts

vci = 50

MSR-#Switch

Switch

CP Host

Data

Src

Host

P4

P1

MonitoringSwitch

P3 P1

P2 P2

P5 P5

P0 P4

P6 P6

P7 P7

P3

NH

MSR

(P3)

P0MSR

CP

192.168.216.1

192.168.200.1192.168.200.2

192.168.216.2

192.168.216.2 probe(A)

probe(C)

192.168.204.1

192.168.208.1

192.168.220.1

192.168.216.1

192.168.224.1

192.168.228.1

probe(B)

vci = 51

vci = 50192.168.204.2

192.168.205.1

Destination Route = OutVIN = <P1, S1>,Maps to Port 1, VCI = 51

Returning traffic not shown

192.168.205.2



Exercise 2: Fair Queuing• In SWCP window, stop the echo server (do a ^C in the window)

– Restart the server but without the echo option$WUTILS/bin/NetBSD/serv -p 5050 -e 0

• Set interface and internal rates (internal rate includes a speed advantage)

– in the MSRCP window $MSR/bin/Linux/sendcmd -p 1 -c policy -s set_trate

-d 10000 -d 0 -d 150 -t 10000 -w info0) 50000 (0xc350) # G = Max rate for APIC interface1) 10000 (0x2710) # T = Total Rate = Link Rate + Internal Rate2) 4000 (0xfa0) # L = Link Rate (aka External rate)3) 150 (0x96) # S’ = Speedup factor (percent)4) 6000 (0x1770) # Internal Rate (port to port rate)

– All rates are in Kbps.



Exercise 2: Fair Queuing• Default behavior is effectively a FCFS, tail drop packet scheduler. In

the two Data source windows, send traffic and observe packet scheduling behavior in the GUI

– turn on source 1, approx 3.4 Mbps$WUTILS/bin/Linux/sendpkts -c -x 8000 -p 100 -a 1 -S -s 5050

-l 192.168.216.2 192.168.204.2– in other window turn on source 2, approx 2.5Mbps

$WUTILS/bin/Linux/sendpkts -c -x 6000 -p 100 -a 1 -S -s 5050 -l 192.168.216.2 192.168.205.2

• With monitoring the GUI, turn on Queue State DRR– In the MSRCP window, send command to enable qsdrr on port 1

$MSR/bin/Linux/sendcmd -p 1 -c policy -s set_ps -d 2• The two flows not fairly share available BW, go boack to the FCFS

scheduler (in the MSRCP window)– $MSR/bin/Linux/sendcmd -p 1 -c policy -s set_ps -d 0



Exercise 3: Plugins• Compiling and downloading a plugin

– in the SWCP window (Need NetBSD), build the object file (combined.o)cd ${MSR}/rp/plugins/msr_templatemake

– Then in the MSR CP window, download plugin to running SPC kernel (msr_template is hte entry point name)cd ${MSR}/rp/plugins/msr_templatesetenv WU_ARL_ROOT $WUARL$WUTILS/bin/Linux/pluginDownload -p 5

-e msr_template -s combined.oPlugin successfully loaded with module ID: 0

– The downloader prints out the plugin id number. You will need this in later commands.



Exercise 3: Plugins• Next, turn on debug messages for the plugin, classifier and pcu

modules– In the MSR CP window

$MSR/bin/Linux/sendcmd -p 5 -c set_debug -l verbose -m plugin -m ctl -m pcu -m classify

## Create an instance of the plugin• Turn on the general match classifier

– In the MSR CP window$MSR/bin/Linux/sendcmd -p 5 -c policy -s set_gen -d 1

• Create one instance of the plugin– In the MSR CP window

$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s create -i 100 -w infothis creates instance number 0 of the plugin (see debug messages or the return value with -w info is used with the sendcmd command)



Exercise 3: Plugins• For a plugin to be called, it must be bound to either a general or exact

match filter. In this exercise we use the general match filters - the exact match interface is still “sensitive”

– Now create a general match filter that the plugin will be bound to.

– addfltr - Add fltr: gid, fid, flags, src/msk, sp, dst/msk, dp, pr

– gid = 0 for general match and 1 for exact match

– use flags = 0 and set filter id to 0 for the first filter

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 0 -F 0 -n 0 -n 0 -d 0 -n 192.168.220.0 -n 255.255.255.0 -d 0 -d 0

-f is the filter id for use when we bind the pluginThis will match any packets with a destination network prefix equal to 1923.168.200.0/24.



Exercise 3: Plugins• Now that we have a plugin instance and a general match filter is

installed, we need to bind the plugin instance to the filter

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 0 -g 0 -f 0This binds plugin instance ID 0 to general match filter ID 0.

• Send a message to the bound plugin. The plugin will both printout and return the number of packets it has seen

– $MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0

• Send packets that will match the filter.

– In one of the the Data Source windows, send 10 packets$WUTILS/bin/Linux/sendpkts -n 10 -a 1 -S -s 5050 -l 192.168.216.2 192.168.220.2

• Then send message to plugin and read the number of packets

– MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0 -w info



Exercise 3: Plugins• Get a list of loaded classes, instances and filters - prints to SPC

window– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s clist -t 100000 -w info– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s ilist -t 100000 -w info– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s flist -g 0 -w info

• Unbind the instance– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 0 -g 0 -f 0

• Free the instance– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 0

• Remove general match filter– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 0 -g 0

• Unload a plugin (ignore error message)– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unload -d 0

• Check on plugin status– $WUTILS/bin/Linux/pluginStat -p 5 0



Exercise 3: Plugins• Download the plugin a second time and and create three instances

– cd $MSR/rp/plugins/msr_template/– $WUTILS/bin/Linux/pluginDownload -p 5 -e msr_template -s combined.o

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s create -i 100 -w info



• Create three filters– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 1 -F 0 -n 0 -n 0 -d 0

-n 192.168.220.0 -n 255.255.255.0 -d 0 -d 0





Exercise 3: Plugins• Bind instances

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 0 -g 0 -f 0



• Now send packets so that each of the plugins will receive a different number of packets.

– In the Date Source Window$WUTILS/bin/Linux/sendpkts -n 5 -a 1 -S -s 5060 -d 5090

-l 192.168.216.2 192.168.220.2$WUTILS/bin/Linux/sendpkts -n 8 -a 1 -S -s 5060 -d 5090

-l 192.168.216.2 192.168.221.2$WUTILS/bin/Linux/sendpkts -n 11 -a 1 -S -s 5060 -d 5090

-l 192.168.216.2 192.168.222.2



Exercise 3: Plugins• Send messages to the different instances

– $MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0– $MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 1– $MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 2

• Print out information about loaded classes, instances and fliters:

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s clist -t 100000 -w info– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s ilist -t 100000 -w info– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s flist -g 0

• Unbind instances– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 0 -g 0 -f 0– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 1 -g 0 -f 1– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 2 -g 0 -f 2



Exercise 3: Plugins• Free instances

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 0



• Remove filters– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 0 -g 0

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 1 -g 0

– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 2 -g 0

• Unload class– $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unload -d 0

washington washington university in st louis [email protected] msr tutorial group exercises

Documents