BotMiner: Clustering Analysis of Network Traffic for Protocol- and

Structure-Independent Botnet Detection

Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee College of Computing, Georgia Institute of Technology

Presented by Joshua Cox

Botnet

Group of compromised computers Controlled by remote commands Malicious activities

DDoS attacks – spam phishing – identity theft

Protocols IRC, HTTP, P2P

Centralized Botnets

Botmaster sends command to designated C&C server

Bots request commands from server

P2P Botnets

No C&C server Botmaster sends

command to any bot

Bots share commands with neighbors

Rishi

Detects IRC based botnets Monitors traffic

Suspicious nicknames Suspicious servers Uncommon server ports

BotSniffer

Network based anomaly detection All bots within a botnet will share similar

traffic patterns Works with IRC and HTTP botnets Does not detect P2P botnets

BotHunter

Works with IRC and HTTP P2P botnets

Relies on “Infection Lifecycle Model” What if we change the lifecycle?

BotMiner Objective

Detect groups of compromised machines that are part of a botnet

Independent of C&C communication structure and content

Minimal false positives Resource efficient detection

BotMiner Architecture

BotMiner Architecture

C-plane Monitor

Who is talking to whom? TCP and UDP traffic flows

time, duration source, destination packet count, bytes transferred

Manageable log size less than 1GB per day for 300 Mbps

network

A-plane Monitor

Who is doing what? Detects malicious activities

scanning – binary downloading spamming – exploit attempts

Snort with custom plugins expandable

C-plane Clustering

Which machines have similar communication patterns?

C-plane monitor logs → cluster reports

C-plane Clustering

Basic Filtering Remove internal flows Remove one way flows

C-plane Clustering

White Listing Remove flows to popular destination Google, Yahoo, etc.

C-plane Clustering

Aggregation C-flow: all traffic flows over a period of

time that share the same source, destination, and protocol

C-plane Clustering

Feature Extraction flows per hour – bytes per packet packets per flow – bytes per

second

C-plane Clustering

Two-step Clustering Coarse-grain and Refined clustering X-means clustering algorithm

A-plane Clustering

Which machines have similar activity patterns?

A-plane monitor logs → cluster reports

A-plane Clustering

Activity Type Clustering scan – spam binary download – exploit

A-plane Clustering

Activity Feature Clustering target subnet – similar binary spam content – exploit type

Cross-plane Correlation

Which machines are in a botnet? Botnet score

Number of clusters Score of other hosts in cluster Activity weighting

Which bots are in the same botnet?

Test Case

Georgia Tech campus network up to 300 Mbps

Ran monitors for 10 days wide variety of protocols

Obtained traces for 8 botnets IRC, HTTP, and P2P

Botnets Used

Overlaid malicious traffic on normal traffic Mapped IPs from random hosts to bots

Filtering Results

Internal/External filter reduces data by 90% 10 billion packets reduced to 50k C-flows

Detection Results

All botnets detected 99.6% bot detection 0.3% false positive rate

Limitations

Traffic randomization and mimicry C-plane cluster evasion

Individual or group commands A-plane cluster evasion

Delay bot tasks Cross-plane analysis evasion

References• Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee. BotMiner: Clustering Analysis of

Network Traffic for Protocol- and Structure-Independent Botnet Detection. 17th USENIX Security Symposium (Security'08), San Jose, CA, 2008.

• D. Pelleg and A. W. Moore. X-means: Extending k-means with efficient estimation of the number of clusters. In Proceedings of the Seventeenth International Conference on Machine Learning (ICML’00), pages 727–734, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc.

• J. Goebel and T. Holz. Rishi: Identify bot contaminated hosts by irc nickname evaluation. In Proceedings of USENIX HotBots’07, 2007.

• G. Gu, P. Porras, V. Yegneswaran, M. Fong, and W. Lee. BotHunter: Detecting malware infection through ids-driven dialog correlation. In Proceedings of the 16th USENIX Security Symposium (Security’07), 2007.

• G. Gu, J. Zhang, and W. Lee. BotSniffer: Detecting botnet command and control channels in network traffic. In Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS’08), 2008.