tein2 measurement and monitoring workshop netflow
TRANSCRIPT
Passive Measurements - Netflow• Netflow• Setting up Netflow on a router• Using Netflow• Establishing exports• Configuring a collector• Analysing the data
Netflow• Netflow was developed by Cisco in response to demands by
customers to have accessible detailed information about the IP traffic in the network
• Cisco IOS NetFlow efficiently provides a key set of services for IP applications, including network traffic accounting, usage-based network billing, network planning, security, Denial of Service monitoring capabilities, and network monitoring. NetFlow provides valuable information about network users and applications, peak usage times, and traffic routing.
Configuring Netflow on a router• On the router:
– At interface level:• ip route-cache flow• ip flow ingress
• On switch/routers it is different:– ip route-cache flow gives only supervisor routed packets– Need to cover switched packets
• mls flow ip interface-full• The router is now deploying Netflow but the data is not
being exported but it is available local to the router.
sh ip cache flowRouter> sh ip cache flowIP packet size distribution (10608M total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480.003 .433 .082 .040 .032 .017 .012 .005 .004 .003 .005 .005 .003 .004 .003512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.003 .003 .019 .039 .274 .000 .000 .000 .000 .000 .000IP Flow Switching Cache, 4456704 bytes
24273 active, 41263 inactive, 916467676 added1635772219 ager polls, 0 flow alloc failuresActive flows timeout in 30 minutesInactive flows timeout in 15 seconds
IP Sub Flow Cache, 794760 bytes24273 active, 24879 inactive, 916467371 added, 916467371 added to flow0 alloc failures, 42240 force free3 chunks, 1089 chunks addedlast clearing of statistics never
show ip cache flowProtocol Total Flows Packets Bytes Packets Active(Sec)
Idle(Sec)-------- Flows /Sec /Flow /Pkt /Sec /Flow /FlowTCP-Telnet 217147 0.1 8 173 1.1 4.7 22.2TCP-FTP 2002298 1.2 8 58 10.6 3.8 23.6TCP-FTPD 1062429 0.6 570 920 381.3 4.3 14.5TCP-WWW 248915610 156.5 17 607 2695.6 3.6 19.1TCP-SMTP 10305099 6.4 25 579 167.5 7.3 14.8TCP-X 635761 0.3 1 415 0.7 0.3 35.6TCP-BGP 97043 0.0 3 63 0.2 15.6 19.7TCP-NNTP 77040 0.0 98 137 4.7 75.2 21.0TCP-Frag 30143 0.0 40 71 0.7 8.7 21.9TCP-other 146547079 92.1 20 497 1853.3 6.1 21.3UDP-DNS 210767500 132.5 2 78 309.0 2.5 22.1UDP-NTP 1694221 1.0 1 77 1.2 1.2 22.0UDP-TFTP 11996 0.0 5 175 0.0 14.4 22.2UDP-Frag 28878 0.0 118 74 2.1 28.1 23.5UDP-other 272715645 171.5 4 228 856.0 2.3 23.0ICMP 20214614 12.7 10 150 133.7 6.0 22.2IGMP 162 0.0 2 37 0.0 3.4 23.0IPv6INIP 94956 0.0 1 192 0.1 7.8 20.4GRE 86581 0.0 3107 136 169.1 126.5 18.9IP-other 939688 0.5 139 652 82.2 28.4 21.1Total: 916443890 576.3 11 498 6669.9 3.5 21.4
sh ip cache flow
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP PktsGi0 139.230.245.21 Gi1 66.240.190.132 06 0C33 0050 8Gi0 130.116.2.21 Gi1 66.232.156.5 11 0035 0035 1Gi1 150.203.2.50 Gi0 202.85.138.71 11 801E 0035 8Gi0 139.230.245.21 Gi1 66.240.190.132 06 0C3A 0050 9
sh ip cache flowUseful tips:For spotting large numbers of packets in a flow:
Router>sh ip cache flow | include KGi1 137.111.130.211 Gi0 218.250.5.246 06 0E3F 4158 34KGi0 139.230.245.21 Gi1 38.113.141.170 06 3D86 0050 20KGi1 149.171.161.17 Gi0 218.111.150.47 06 691F 0AAF 12KGi1 149.171.160.62 Gi0 219.78.167.2 06 459D 0FAF 10KGi1 38.113.141.170 Gi0 139.230.245.21 06 0050 3D86 15KGi1 149.171.161.17 Gi0 203.218.241.65 06 0F9F 26E4 12K
Top talkers• Configure terminal
ip flow-top-talkers
Sort-by packetsCache-timeout 2000top 5
• Sample output:Router>show ip flow top-talkers
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP PktsGi0 202.158.192.34 Local 202.6.112.239 2F 0000 0000 45KGi1 202.6.112.5 Null 233.29.147.222 11 800E DC91 8733Gi1 202.6.112.12 Gi2 203.59.48.154 2F 0000 0000 2624Gi2 203.19.110.254 Gi1 202.6.112.12 11 D606 2707 2339Gi2 203.19.110.254 Gi1 202.6.112.5 11 D606 2707 2339
Sampled Netflow• Sampling rates (if required):flow-sampler-map SAMPLE
mode random one-out-of 100exit
Configuring flow-export on a router• Flow data can be exported as a UDP stream from the router
to a collection machine• The commands for enabling this are normally:
ip flow-export source Loopback0
ip flow-export version 5 origin ASip flow-export destination 10.1.1.1 9991
• Configuring flow-export on a switch/routermls nde sender version 5
Now verify router export…Router> show ip flow exportFlow export v5 is enabled for main cacheExporting flows to 1.1.1.1 (9991)Exporting using source interface Loopback0Version 5 flow records, origin-as916688250 flows exported in 30556276 udp datagrams0 flows failed due to lack of export packet0 export packets were sent up to process level0 export packets were dropped due to no fib0 export packets were dropped due to adjacency issues0 export packets were dropped due to fragmentation failures0 export packets were dropped due to encapsulation fixup failures0 export packets were dropped enqueuing for the RP0 export packets were dropped due to IPC rate limiting
Router>
Configuring flow-export – the receiver
• Many netflow tools available• Many in-house developed solutions eg AARNet
NFA• Flowtools (http://www.splintered.net/sw/flow-
tools/) is a software package for collecting and processing NetFlow data from Cisco and Juniperrouters
Flowtools• Flow-tools is library and a collection of programs used to
collect, send, process, and generate reports from NetFlow data.
• The tools can be used together on a single server or distributed to multiple servers for large deployments.
• The flow-toools library provides an API for development of custom applications for NetFlow export versions 1,5,6 and the 14 currently defined version 8 subversions.
• A Perl and Python interface have been contributed and are included in the distribution.
Flowtools• Flows are exported from a router in a number of different
configurable versions. A flow is a collection of key fields and additional data. The flow key is {srcaddr, dstaddr, input, output, srcport, dstport, prot, ToS}. Flow-tools supports one export version per file.
Formats..• Flows are exported from a router in a number of
different configurable versions. A flow is a collection of key fields and additional data. The flow key is {srcaddr, dstaddr, input, output, srcport, dstport, prot, ToS}. Flow-tools supports one export version per file.
• Export versions 1, 5, 6, and 7 all maintain {nexthop, dPkts, dOctets, First, Last, flags}, ie the next-hop IP address, number of packets, number of octets (bytes), start time, end time, and flags such as the TCP header bits.
Formats• Version 5 adds the additional fields {src_as,
dst_as, src_mask, dst_mask}, ie source AS, destination AS, source network mask, and destination network mask.
• Version 7 which is specific to the Catalyst switches adds in addition to the version 5 fields {router_sc}, which is the Router IP address which populates the flow cache shortcut in the Supervisor
Formats• Version 6 which is not officially supported by Cisco adds in
addition to the version 5 fields {in_encaps, out_encaps, peer_nexthop}, ie the input and output interface encapsulation size, and the IP address of the next hop within the peer.
• Version 1 exports do not contain a sequence number and therefore should be avoided, although it is safe to store the data as version 1 if the additional fields are not used.
• Version 8 IOS NetFlow is a second level flow cache that reduces the data exported from the router. There are currently 11 formats, all of which provide {dFlows, dOctets, dPkts, First, Last} for the key fields.
V8 Formats• V8
8.1 Source and Destination AS, Input and Output interface8.2 - Protocol and Port8.3 - Source Prefix and Input interface8.4 - Destination Prefix and Output interface8.5 - Source/Destination Prefix and Input/Output interface8.9 - 8.1 + ToS8.10 - 8.2 + ToS8.11 - 8.3 + ToS8.12 - 8.5 + ToS8.13 - 8.2 + ToS8.14 - 8.3 + ports + ToS
Flowtools• The following programs are included in the flow-tools distribution.• flow-capture - Collect, compress, store, and manage disk space
for exported flows from a router.• flow-cat - Concatenate flow files. Typically flow files will contain a
small window of 5 or 15 minutes of exports. Flow-cat can be used to append files for generating reports that span longer time periods.
• flow-fanout - Replicate NetFlow datagrams to unicast or multicast destinations. Flow-fanout is used to facilitate multiple collectors attached to a single router.
• flow-report - Generate reports for NetFlow data sets. Reports include source/destination IP pairs, source/destination AS, and top talkers. Over 50 reports are currently supported.
Flowtools• flow-report - Generate reports for NetFlow data sets. Reports
include source/destination IP pairs, source/destination AS, and top talkers. Over 50 reports are currently supported.
• flow-tag - Tag flows based on IP address or AS #. Flow-tag is used to group flows by customer network. The tags can later be used with flow-fanout or flow-report to generate customer based traffic reports.
• flow-filter - Filter flows based on any of the export fields. Flow-filter is used in-line with other programs to generate reports based on flows matching filter expressions.
• flow-import - Import data from ASCII or cflowd format.• flow-export - Export data to ASCII or cflowd format.
FlowTools• flow-send - Send data over the network using the NetFlow protocol.• flow-receive - Receive exports using the NetFlow protocol without
storing to disk like flow-capture.• flow-gen - Generate test data.• flow-dscan - Simple tool for detecting some types of network
scanning and Denial of Service attacks.• flow-merge - Merge flow files in chronoligical order.• flow-xlate - Perform translations on some flow fields.• flow-expire - Expire flows using the same policy of flow-capture.• flow-header - Display meta information in flow file.• flow-split - Split flow files into smaller files based on size, time, or
tags.
Flow-capture• flow-capture [-h] [-A AS0_substitution] [-b big|little] [-C
comment] [-c flow_clients] [-d debug_level] [-Ddaemonize] [-e expire_count] [-f filter_fname] [-Ffilter_definition] [-E expire_size] [-m privacy_mask] [-nrotations] [-N nesting_level] [-p pidfile] [-R rotate_program] [-S stat_interval] [-t tag_fname] [-Tactive_def|active_def,active_def...] [-V pdu_version] [-zz_level] {-w workdir} {localip/remoteip/port}
Using flow-capture• % flow-capture -z0 -N0 -V5 –n95 -w/var/local/flows /1.1.1.1/9991
• This will create a flow capture file in the /var/local/flows directory– It will initially by named tmp-v5.YYYY-MM-DD.HHMMSS+0700– When the collection period (15 minutes) expires it will be renamed
• ft-v5.YYYY-MM-DD.HHMMSS+TZ00
– % flowprint < ft-v5.YYYY-MM-DD.HHMMSS+TZ00
• To anonymise use the –m privacy_mask
flow-cat• The flow-cat utility processes files and/or directories of
files in the flow-tools format. The resulting concatenated data set is written to the standard output or file specified by -o. If file is a single dash (`-') or absent, flow-cat will read from the standard input.
flow-cat [-aghmp] [-b big|little] [-C comment] [-ddebug_level] [-o filename] [-t start_time] [-Tstart_time] [-z z_level] [file|directory...]
% flow-cat ft-v05.2001-05-01.* | flow-print
flow-stat • The flow-stat utility generates usage reports for flow data
sets by IP address, IP address pairs, ports, packets, bytes, interfaces, next hops, autonomous systems, ToSbits, exporters, and tags.
flow-stat [-hnpPw] [-d debug_level] [-f format] [-Ssort_field] [-s sort_field] [-t tally_lines] [-Ttitle]
Formats0 Overall Summary1 Average packet size distribution2 Packets per flow distribution3 Octets per flow distribution4 Bandwidth per flow distribution5 UDP/TCP destination port6 UDP/TCP source port7 UDP/TCP port8 Destination IP9 Source IP10 Source/Destination IP11 Source or Destination IP12 IP protocol13 octets for flow duration plot data14 packets for flow duration plot data15 short summary16 IP Next Hop
17 Input interface18 Output interface19 Source AS20 Destination AS21 Source/Destination AS22 IP ToS23 Input/Output Interface24 Source Prefix25 Destination Prefix26 Source/Destination Prefix27 Exporter IP28 Engine Id29 Engine Type30 Source Tag31 Destination Tag32 Source/Destination Tag
Flow-filter• The flow-filter utility will filter flows based on user selectable criteria. The
IP address filters are defined in flow.acl or by the filename specified by -f.• Other filters such as input interface and ports are defined on the
command line. These filters accept range and negation operators, ie -i1-15 for input interfaces 1 through 15 or -i1,15 for input interfaces 1 and 15, or !1,15 for not input interfaces 1 and 15.
flow-filter [-hko] [-a src_as_filter] [-A dst_as_filter] [-bbig|little] [-C comment] [-D dstaddr_filter_name] [-ddebug_level] [-f acl_fname] [-i input_filter] [-Ioutput_filter] [-p srcport_filter] [-P dstport_filter] [-ripprot_filter] [-S srcaddr_filter_name] [-t tos_filter] [-Ttcp_flags_filter] [-x nexthop_filter_name] [-z z_level]
Flow-tools are modular• To produce a report on top source/destination AS using symbolic
names:% flow-cat ft-v05.2005-09-07.* | flow-stat –f20 –n –S4
• To produce a report on top destination IP address report by outbound traffic:
% flow-cat ft-v05.2005-09-07.* | flow-filter –I 5 | flow-stat -f8 -P -p -S3
• To produce a report on top destination IP address report by outbound traffic:
% flow-cat ft-v05.2005-09-07.* | flow-filter –I 5 | flow-stat –f9 -P -p -S3
Flowscan
• FlowScan examines flow data and maintains counters reflecting what was found. Counter values are stored using RRDtool, a database system for time-series data. Finally, FlowScan uses visualization capabilities of both RRDtool and other front-ends to report on the processed flow data.
Other tools• http://www.caida.org/tools/utilities/flowscan/ flowscan• http://www.ntop.org/netflow.html ntop• http://www.paessler.com/prtg prtg• http://www.mindrot.org/flowd.html flowd• http://netflow.cesnet.cz/index.php Netflow Monitor