autofocus: a tool for automatic traffic analysis
DESCRIPTION
AutoFocus: A Tool for Automatic Traffic Analysis. Cristian Estan, University of California, San Diego. Who is using my link?. Informal problem definition. Traffic reports. Analysis. Applications: 50% of traffic is Kazaa Sources: 20% is from Steve’s PC. Gigabytes of measurement data. - PowerPoint PPT PresentationTRANSCRIPT
AutoFocus: A Tool for AutoFocus: A Tool for Automatic Traffic AnalysisAutomatic Traffic Analysis
Cristian Estan,
University of California, San Diego
October 2003 AutoFocus - NANOG 29 2
Who is using my link?Who is using my link?
October 2003 AutoFocus - NANOG 29 3
Informal problem definitionInformal problem definition
Gigabytes of measurement
data
Analysis Traffic reports
Applications:
50% of traffic is Kazaa
Sources:
20% is from Steve’s PC
October 2003 AutoFocus - NANOG 29 4
Informal problem definitionInformal problem definition
Gigabytes of measurement
data
20% is Kazaa from Steve’s PC
50% is Kazaa from network A
Analysis Traffic reports
October 2003 AutoFocus - NANOG 29 5
AutoFocus: system structureAutoFocus: system structure
Trafficparser
Web basedGUI
Trafficanalyzer
Grapher
(sampled) NetFlow data or Packet header traces
categories
names
October 2003 AutoFocus - NANOG 29 6
System detailsSystem details
Availability Downloadable Free for educational, research and non-profit use
Requirements Linux or BSD (might run on other Unix OSes) 256 Megs of RAM at least 1-10 gigabytes of hard disk (depends on traffic) Recent Netscape, Mozilla or I.E. (Javascript) Needs no web server – no server side scripting
October 2003 AutoFocus - NANOG 29 7
Traffic analysis approachTraffic analysis approach
Characterize traffic mix by describing all important traffic clusters
Multi-field clusters (e.g. flash crowd described by protocol, port number and IP address)
At the the right level of granularity (e.g. computer, proper prefix length)
Analysis is automated – finds insightful data without human guidance
October 2003 AutoFocus - NANOG 29 8
Traffic clusters: exampleTraffic clusters: example
Incoming web traffic for CS Dept. SrcIP=*, DestIP in 132.239.64.0/21, Proto=TCP, SrcPort=80, DestPort in [1024,65535]
October 2003 AutoFocus - NANOG 29 9
Traffic reports automatically list significant traffic clusters
Describe only clusters above threshold (e.g. T=total of traffic/20)
Compression removes redundant clusters whose traffic can be inferred from more specific clusters
Traffic reportTraffic report
October 2003 AutoFocus - NANOG 29 10
Automatic cluster selectionAutomatic cluster selection
10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.8 10.0.0.9 10.0.0.10 10.0.0.14
15 35 30 40 160 110 35 75
10.0.0.2/31 10.0.0.4/3150 10.0.0.8/31 10.0.0.10/31
70 270 35 75
10.0.0.0/30 10.0.0.4/30 10.0.0.8/30 7530550 70
10.0.0.0/29 10.0.0.8/29120 380
10.0.0.0/28 500
10.0.0.14/31
10.0.0.12/30
October 2003 AutoFocus - NANOG 29 11
Automatic cluster selectionAutomatic cluster selection
10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.8 10.0.0.9 10.0.0.10 10.0.0.14
15 35 30 40 160 110 35 75
10.0.0.2/31 10.0.0.4/3150 10.0.0.8/31 10.0.0.10/31
70 270 35 75
10.0.0.0/30 10.0.0.4/30 10.0.0.8/30 7530550 70
10.0.0.0/29 10.0.0.8/29120 380
10.0.0.0/28 500500
120 380
305
270
160 110
Threshold=100
10.0.0.14/31
10.0.0.12/30
October 2003 AutoFocus - NANOG 29 12
270
120
500
305
380
160 110
Automatic cluster selectionAutomatic cluster selection
10.0.0.8 10.0.0.9
10.0.0.0/29 10.0.0.8/29
10.0.0.8/31
10.0.0.8/30
10.0.0.0/28
120 380
160 110
Compression keeps interesting clusters by removing those that can be inferred from more specific
ones
305-270<100
380-270≥100
October 2003 AutoFocus - NANOG 29 13
Single field report exampleSingle field report example
Source IP Traffic
pkts.
10.0.0.0/29 120
10.0.0.8/29 380
10.0.0.8 160
10.0.0.9 110
AutoFocus has both single field and
multi-field traffic reports
October 2003 AutoFocus - NANOG 29 14
Graphical user interfaceGraphical user interface
Web based interface Many pre-computed traffic reports Interactive drill-down Traffic categories defined by user
October 2003 AutoFocus - NANOG 29 15
Traffic reports for weeks, days, three
hour intervals and half hour
intervals
October 2003 AutoFocus - NANOG 29 16
Traffic reports measure traffic
in bytes, packets and flows, have
various thresholds
October 2003 AutoFocus - NANOG 29 17
Single field report
October 2003 AutoFocus - NANOG 29 18
October 2003 AutoFocus - NANOG 29 19
Colors – user defined traffic categories
Separate reports for each category
October 2003 AutoFocus - NANOG 29 20
The filter and threshold allow
interactive drill-down
October 2003 AutoFocus - NANOG 29 21
The filter and threshold allow
interactive drill-down
October 2003 AutoFocus - NANOG 29 22
The filter and threshold allow
interactive drill-down
October 2003 AutoFocus - NANOG 29 23
Case study : SD-NAPCase study : SD-NAP
Structure of regular traffic mix Backups from CAIDA to tape server
FTP from SLAC Stanford
Scripps web traffic
Web & Squid servers
Large ssh traffic
Steady ICMP probing from CAIDA
Unexpected events
October 2003 AutoFocus - NANOG 29 24
Backups from CAIDA to tape server
Structure of regular traffic mixStructure of regular traffic mix
SD-NAP
October 2003 AutoFocus - NANOG 29 25
Backups from CAIDA to tape server
Semi-regular time pattern
Structure of regular traffic mixStructure of regular traffic mix
SD-NAP
October 2003 AutoFocus - NANOG 29 26
Steady ICMP probing from CAIDA
Structure of regular traffic mixStructure of regular traffic mix
SD-NAP
The flow view highlights
different traffic clusters
October 2003 AutoFocus - NANOG 29 27
Analysis of unusual eventsAnalysis of unusual events
Sapphire/SQL Slammer worm Find worm port & proto automatically
October 2003 AutoFocus - NANOG 29 28
Analysis of unusual eventsAnalysis of unusual events
Sapphire/SQL Slammer worm Can identify infected hosts
October 2003 AutoFocus - NANOG 29 29
How can AutoFocus help you?How can AutoFocus help you?
Understand your regular traffic mix better
Better planning of network growth
Better traffic policing
Understand unusual events
More effective reactions to worms, DoS attacks
Notice effects of route changes on traffic
October 2003 AutoFocus - NANOG 29 30
Benefits w.r.t. existing toolsBenefits w.r.t. existing tools
Multi-field aggregation Automatically finds right granularity Drill-down
Per category reports Using filter
October 2003 AutoFocus - NANOG 29 31
Thank you!Thank you!
Beta version of AutoFocus downloadable from
http://ial.ucsd.edu/AutoFocus/
Any questions?
Acknowledgements: Stefan Savage, George Varghese, Vern Paxson, David Moore, Liliana Estan, Mike Hunter, Pat Wilson, Jennifer Rexford, K Claffy, Alex Snoeren, Geoff Voelker, NIST,NSF
October 2003 AutoFocus - NANOG 29 32
October 2003 AutoFocus - NANOG 29 33
Definition: unexpectednessDefinition: unexpectedness
To highlight non-obvious traffic clusters by using unexpectedness label 50% of all traffic is web Prefix B receives 20% of all traffic The web traffic received by prefix B is 15%
instead of 50%*20%=10%, unexpectedness label is 15%/10%=150%