Distributed Network Intrusion Detection

An Immunological Approach

Steven HofmeyrStephanie ForrestPatrik D’haeseleer

Dept. of Computer Science University of New Mexico

Albuquerque, NM

{steveah, forrest, patrik}@cs.unm.eduhttp://cs.unm.edu/~steveah/research.html

Introduction

• Intrusion detection:– Assume that systems are not secure.– Attempt to detect violations of security policy (intrusions) by monitoring and

analyzing system behavior.– Construct a model of normal behavior and look for deviations from the model

(anomaly detection).

• Building the model (defining self):– TCP/IP traffic over a broadcast LAN.– Based on Network Security Monitor (NSM).

• Every computer on the network should participate in IDS:– Distributed detection– Use negative-selection algorithm

• Diversity of protection:– Permutation masks

Background: Defining Self

• The right approach:– Anomaly detection– Sparsely connected graph– Normal patterns reasonably stable – Attackers highly likely to perturb graph

• Disadvantages:– Heavyweight– Single point of failure– Not scalable

• NSM: Network Security Monitor (UCDavis){Mukherjee et al. Network Intrusion Detection. IEEE Network, pp26-41, 1994}

10.10.10.2

20.20.20.5

Datapath triple(10.10.10.2, 20.20.20.5, ftp)

BroadcastLAN

External host

Internal host

The Biological Viewpoint

Universenonself

self

Detectionsystem

• Self (proteins) = normal datapath triples

• Nonself (proteins) = triples generated during an attack

• Universe = Self Nonself

• Anomaly detection:– Detection system trained on self– Detection system classifies new triples as self (normal) or nonself

(anomalous)

• NSM: a single monolithic detector matching self (positive detection)

How the Immune System Distributes Detection

• Advantages of distributed negative detection:– Localized (no communication costs)– Scalable– Tunable– Robust (no single point of failure)– Negative selection algorithm minimizes false positives

self

Single Detector Multiple Random Detectors

self self

• Immune system: Many small detectors matching nonself (negative detection).

The Negative Selection Algorithm

self self1. Randomly generate a

detector string.

2. Does the detector string match self? NO YES

3. If no, accept

If yes, go to 1.

(regenerate). ACCEPT REJECT

Results in a set of valid detectors self

self

Applying Negative Detection to Network Traffic

• Representation:– SYN packet triples mapped to 49-bit strings

• Generalized detection:– Partial matching with r-contiguous bits rule

0110100101

1110111101 1110111101

0100110100

Match No Match

r = 4

Triple

Detector

• Consequences of Partial Matching:– Advantage: Lightweight (few detectors per host)– Disadvantage: Holes limit detection

Holes

Host 1 Host 3Host 2

• Problem: Holes limit detection for any partial match rule.

• Solution: A different permutation mask for each host.

Overcoming Holes

• Result: In the broadcast network, detection is limited by the intersection of all hole sets.

Total Coverage

Experimental Setup

• UNM CS subnet of 50 machines on a switched segment. – 100 49-bit string detectors per machine

• Training set (self):– Collected over 43 days

– 1 266 000 TCP SYN packets

– 3763 unique binary self strings

• Normal test set (supposedly self):– Collected over 7 days

– 182 629 TCP SYN packets

– 626 unique binary self strings

• Abnormal test set (nonself):– 8 different incidents, 7 real occurrences, 1 synthetic

– Real abnormal behavior includes: massive portscanning, limited probing, address-space probing, local host compromise

– Synthetic: 200 random connections between internal (LAN) hosts

Experimental Results

• Low false positives:– P(false positive per self string) = 0.000304– 55 strings, but only 10 unique– Effectively: under 2 false alarms per day

• High detection rates with few detectors– 100% successful detection: 8 out of 8 abnormal incidents detected– Only 100 detectors per host

• Permutation masks improve detection– Up to an order of magnitude improvement– Overcomes hole limitation

• Normal is reasonably stable.

Title:

Creator:

Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

The Problem of Incomplete Self Sets(Suppose the training set is incomplete)

• Activation threshold:– Detector is not activated on every match.

– Must have exceeded x matches before activation.

– No time horizon.

– Helps with stealth attacks (distributed in time).

– Reduced false positives by an order of magnitude.

• Adaptive activation:– Tune local activation thresholds dynamically.

– Whenever a detector matches its first pattern, the activation threshold for that computer is reduced by 1.

– Has a time horizon (threshold gradually returns to default value).

– Hypothesized to help with distributed coordinated attacks.

Experimental ResultsIntrusions with and without permutation masks

Anomaly SignalIncident Fraction Threshold 1 Threshold 10

nonself permutation No permutation

Phear 1.00 1.00 0.50 0.09

Cartan 0.44 0.44 0.43 0.34

Dt03ln93 0.17 0.17 0.16 0.16

Xtream 0.62 0.62 0.59 0.61

Cougar 0.54 0.58 0.53 0.49

Sauron 0.10 0.10 0.09 0.09

Pc35nl 1.00 1.00 0.84 0.43

Synthetic 1.00 0.94 0.33 0.01

Experimental and Theoretical Results:Permutation Masks Overcome the Hole Limit

Title:

Creator:

Preview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.

Pushing the Immune Metaphor

• The analogy thus far:– Distributed networks and immunology– Combining negative detection and network

intrusion detection– Diversity via permutation masks

• For the future:– Distributed generation of detectors– Dynamic detector sets– Adaptation and memory (misuse detection)