connect. communicate. collaborate experiences with tools for network anomaly detection in the...
TRANSCRIPT
Connect. Communicate. Collaborate
Experiences with tools for network anomaly detection in the GÉANT2 core
Maurizio Molina, DANTE
COST TMA tech. Seminar
Samos, 23rd Sep 2008
Connect. Communicate. CollaborateThe GÉANT Network
• DANTE operates GÉANT2• Backbone network for National Research and Education
Networks in Europe• 30+ NRENs, 2 global connectivity providers (Telia and
GCrossing), peerings with other research networks (Abilene, Canarie, Clara, TEIN2, SINET…)
Connect. Communicate. Collaborate
The GÉANT Network (IP layer)
• 20 Juniper routers
• tenths of GBit/s of aggregated traffic
• Main accesses and the backbone 10Gbit/s
Pls see www.dante.net
Connect. Communicate. CollaborateThe Services
• So…. Just a big pipe? No!• Services
– Dedicated L1-L2 circuits via multiple technologies– Performance Monitoring services (perfSONAR)– Support for federation of National AA Infrastructures
(eduGAIN) and wireless roaming (eduROAM)– Security Service Very NEW!
NEW!
Connect. Communicate. Collaborate
The vision:enhance NRENs security
• NRENs have their (+ - evolved…) CERTs to deal with security
• and DANTE can filter traffic on GÉANT upon NRENs request….
! BUT !
• Can we be more proactive to NREN CERTs exploiting
the visibility of the GN2 core?
Connect. Communicate. Collaborate
The vision (cont.):enhance NRENs security
• Approach: NetFlow (+ Routing data) & good processing tools
NetFlow v5 collector
• Netflow collected on all peering interfaces
• 1 / 1,000 Sampling
• ~3k flows/s
Connect. Communicate. Collaborate
Proof of concept: Can we identify anomalies in the core?
• Anomalies are often “hidden”
Requirements:
High detection rate
Low false positives
Anomaly classification
Evidence collection
NfSen
Connect. Communicate. Collaborate
From “volume” to “IP feature entropies”
Connect. Communicate. Collaborate
•“IP features entropies”•Simple linear filter
Connect. Communicate. CollaborateDrilling down on peaks Connect. Communicate. Collaborate
-Concentration of DST IPs and DST ports receiving flows
-Dispersion of SRC IPs and SRC ports
• IRC server in Slovenia, receiving a lot of 60 bytes syn pkts on port 6667, mainly from a /16 Subnetwork of an University in the Netherlands.
• Likely a “BotNet war”?
Connect. Communicate. Collaborate
Drilling down on peaks (cont.) Connect. Communicate. Collaborate
- Concentration of SRC and DST IPs and SRC ports
- Dispersion of DST ports
• Portscan of host in CARNET, from 4 hosts, 29 bytes packets
Connect. Communicate. CollaborateOpen source tools
• Results:– anomalies are observable in the GÉANT2 core– Novel methodologies (IP Features entropy) for their
classifications are applicable• Limits:
– NfSen does not fuse NetFlow and Routing data– Extensions would need to be run (and tuned) on all
ingress/egress points– No support, no guaranteed development
Connect. Communicate. CollaborateCommercial tools
• Test started Jun 08 (3 tools)– Tool 1
• PCA, entropy– Tool 2
• Large scale DDoS and Worm spread– Tool 3
• Per host behaviour
Connect. Communicate. CollaborateTool 1 (as a security tool…)
• Two main novel elements– Principal Component Analysis (PCA)– Both Volume and IP features Entropy anomaly
detection• Address what makes anomaly detection a complex task
– PCA: single parameter to control detection sensitivity, even if anomalies are attributed to specific OD pairs
– Entropy: Detection of both low volume (scans) and high volume (DoS) anomalies
Connect. Communicate. CollaborateTool 2
• Well-established (and expensive!) solution for detecting “large” events
• Originally based on large volume shifts only• Now enhanced to give alerts on “fingerprints” (e.g.
communication with C&C servers)– Shared by (part) of the user community (50 out of 120)
• No usage of routing data– though “zones” can be manually created via BGP
prefixes lists• Traditional threshold based detection (although adaptive)
Connect. Communicate. CollaborateTool 3
• Per host behavioural analysis• rather complex “scoring” system to distinguish normal from
abnormal behaviour. Proprietary algorithms• Doesn’t use routing info
– though “zones” can be manually created via BGP prefixes lists
• Potentially attractive methodology• Concerns on scalability and accuracy with 1,000 sampling
Connect. Communicate. Collaborate
lessons learnt and directions for research
• Manual validation is required to confirm/correct anomalies– More automatic intelligence to help this process– Fusion with other data sources (router logs?
Honeynets?)• Detection space of 3 tools often disjoint
– (Standard) anomaly injection• Operations need supported tools to support services• If choice is among published but “not a tool” or “secret but
supported and (claiming to) work” => risk to stick to those!– Fill the gap towards TOOLS!