Enhancing Network Management with Cisco

Data

Brian Borowski – brianb@uwo.ca

Ed Gibson – egibson@uwo.ca

Water’s Hardware

• Debian Stable Release 2.4 Kernel

• Dual P3 @1Gig CPU’s

• 1.5 Gig RAM

• Onboard IDE - – (2) 60 Gig IDE on Channel 0

– (1) CDROM on Channel 1

• Promise PD-2067 PCI - IDE Interface Card– (2) 60 Gig IDE on Channel 0

– (2) 60 Gig IDE on Channel 1

• Total of 360 Gig IDE Drive Space

Water’s Partitions

• 60 Gig - /, /usr• 60 Gig - /usr1• 120 Gig (linear) - /nf1 {gate1, gate2, hse}• 120 Gig (linear) - /nf2 {nsc cat, uc cat}• /nf1 & /nf2 maintained at 90% capacity• gate2 - 65 days• hse - 60 days• cat's - 40 days

NetFlow Versions

NetFlow Version

Comments

1 Original

5 Standard and most common

7 Specific to Cisco Catalyst 6500 and 7600 Series Switches

Similar to Version 5, but does not include AS, interface, TCP Flag & TOS information

8 Choice of eleven aggregation schemes

Reduces resource usage

9 Flexible, extensible file export format to enable easier support of additional fields & technologies; coming out now MPLS, Multicast, & BGP Next Hop

• Version 1• Source Interface - SNMP Index of input interface• Source IP Address• Destination Interface - SNMP Index of output interface• Destination IP Address• Protocol - IP protocol type

– (i.e, ICMP = 1, TCP = 6; UDP = 17)

• Source Port - TCP/UDP source port number or equivalent• Destination Port - TCP/UDP destination port number or

equivalent• Packets - Packets in the flow• Octets - Total number of Layer 3 bytes in the packets of

the flow• Start Time - Unix time• Lifetime - flight time of the flow

Netflow Toolkit• Utilize a netflow "toolkit" available at

– http://www.splintered.net/sw/flow-tools/– We have made some modifications

•  flow-capture– data collector - receives data from router– one instance per source (unique udp port)– we currently have 7 instances– zip compresses data in 16 blocks (4:1 compression) – one file per hour named rawflow.mm-dd-yyyy.hh– files >2GB required later kernel and current fileutils

glibc, bash, ETC. – busy periods ~ 4-5GB/hr– custom -> filename format

• flow-print– Decompress and Display Data– Number of output formats

• 0 short format– Sif SrcIPaddress Dif DstIPaddress Pr SrcP DstP Pkts Octets

• 1 long format– Sif SrcIPaddress DIF DstIPaddress Pr SrcP DstP Pkts Octets StartTime EndTime

Active B/Pk Ts Fl R1 CS MC

• 6 custom format – Sif SrcIPaddress DIF DstIPaddress Pr SrcP DstP Pkts Octets start_time lifetime

• generally used as a piped filter– cat ~/gate1flows/rawflow.05-14-2004.10 | flow-print -f6 | more

• custom -> format type 6 (Unix time format)• custom -> display ports and protocols in decimal

• flow-stat

• Library of predefined reports

• Command Line Parameters of Interest– -P use percents instead of counters– -Sn sort descending on field n– -sn sort ascending on field n– -fn format n

• flow-stat(cont’d))• 0 Overall Summary• 1 Average packet size distribution• 2 Packets per flow distribution• 3 Octets per flow distribution• 4 Bandwidth per flow distribution• 5 UDP/TCP destination port• 6 UDP/TCP source port• 7 UDP/TCP port• 8 Destination IP• 9 Source IP• 10 Source/Destination IP• 11 Source or Destination IP• 12 IP protocol• ……..

• flow-filter

• Utility to filter traffic • Uses “kludge” of Cisco ACL format to

filter IP’s (defaults file flow.acl)

• Command parameters links to ACL reference

• e.g. more flow.acl

ip access-list standard uwoweb permit 129.100.2.71

ip access-list standard cuda permit host 129.100.1.101

• flow-filter (cont’d)• $ flow-filter -hUsage: flow-filter [-hko] [-a src_as_filter] [-A dst_as_filter] [-b big|little] [-C comment] [-D dstaddr_filter_name] [-d debug_level] [-f acl_fname] [-i input_filter] [-I output_filter] [-p srcport_filter] [-P dstport_filter] [-r ipprot_filter] [-S srcaddr_filter_name] [-t tos_filter] [-T tcp_flags_filter] [-x nexthop_filter_name] [-z z_level]

• Flow-Cat

• Facilitates the concatenation of multiple flow files into a single pipe stream.

• e.g. flow-cat rawflow.05-14-2004.* | flow-print -f6 | {some utility}

• Other tools included in this toolkit but are beyond the scope of this presentation.

• flow-filter – clunky & limiting

• flatten data with flow-print then process

• Unix piping

• grep, awk, sort

• perl one liners• perl -ne '@a=split(“ “,$_);if(($a[4]==6)&&($a[6]==53)){print $_}'

• toolbox of scripts to perform repetitive functions • slice - pull particular time sections out of a particular day

– eg. slice gate1 3 11 12 13 • flowcleaver - filter netflow output for a particular value or

range in a specified column– eg. … | flow-print –f6 | flowcleaver –sip 129.100.87 | …

• l3srcsum - total and index (flow,packet,octet) input stream by IP source

• l3destsum - total and index (flow,packet,octet) input stream by IP destination

• l4srcsum - total and index (flow,packet,octet) input stream by destination port

• l4destsum - total and index (flow,packet,octet) input stream by source port

• timefill – convert from Unix time to calendar time

Identify Traffic Anomalies

• Top N analysis– By octet – By conversations– By port

• Focus analysis on anomalies– Identified by MRTG– Syslog– Netflow– IDP …

• Scanner detection & BOTs

Security Incident Verification

• Complaint received associated with network activity two weeks ago

• Netflow can verify conversation took place

• What ports involved (web,smtp,IRC)

• Irrefutable second source of transaction verification

Protocol Distribution

• Application analysis of network traffic

• Percentage of Web Traffic

• Locate ftp servers

• Tracking BOT communication channels

• Virus Infection Detection

Traffic Distribution

• What networks/users are hogging the Internet Pipe?

• What hosts are seeing significant inbound traffic requests

• Are there outside hosts dominating your outbound traffic?

NetFlow Uses

• Attack Mitigation• User (IP)

monitoring• Application

monitoring

• Attack Mitigation• User (IP)

monitoring• Application

monitoring

• Billing• Chargeback• AS Peer

Monitoring

• Billing• Chargeback• AS Peer

Monitoring

• Traffic Engineering

• Traffic Analysis

• Traffic Engineering

• Traffic Analysis

Ap

pli

cati

on

s • Attack Mitigation• User (IP)

monitoring• Application

monitoring

• Attack Mitigation• User (IP)

monitoring• Application

monitoring

• Billing• Chargeback• AS Peer

Monitoring

• Billing• Chargeback• AS Peer

Monitoring

Net

wo

rk L

ayer

AccessAccess DistributionDistribution DistributionDistribution AccessAccessCoreCore

Net

Flo

wF

eatu

res

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks

• NetFlow MPLS Egress Accounting

• BGP Next-hop (v9)

• Multicast NetFlow (v9)

• NetFlow MPLS Egress Accounting

• BGP Next-hop (v9)

• Multicast NetFlow (v9)

• MPLS Aware NetFlow (v9)

• BGP Next-hop (v9)

• Sampled NetFlow

• MPLS Aware NetFlow (v9)

• BGP Next-hop (v9)

• Sampled NetFlow

• NetFlow MPLS Egress Accounting

• BGP Next-hop (v9)

• Multicast NetFlow (v9)

• NetFlow MPLS Egress Accounting

• BGP Next-hop (v9)

• Multicast NetFlow (v9)

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks

• Aggregation Schemes (v8)

• “show ip cache flow” command

• Arbor Networks

Futures• ……

• brianb@uwo.ca

• egibson@uwo.ca