network traffic analysis optimization for signature-based intrusion detection systems dmitry s....

NETWORK TRAFFIC ANALYSIS OPTIMIZATION FOR SIGNATURE-BASED INTRUSION DETECTION SYSTEMS

Dmitry S. Kazachkin

student, Computational systems lab at CMC MSU

Dennis Y. Gamayunov

scientific advisor, PhD, Computational systems lab at CMC MSU

INTRODUCTION

High-performance traffic analysis at gigabit channels

Optimization of a given signature set matching

Packet – the unit of analysis

Operations number minimization for a single packet

SIGNATURE FORMAL DEFINITION (1)

Packet – pair {H, P}

Header (H) – {H1,H2,...,Hn}, Hi belongs to finite space

Payload (P) – text line of unrestricted length

State vector (C) – {С1,С2,...,Сk}, Ci belongs to finite space


Header condition – logical predicate; Hi – arguments

Payload condition – set of functions: CondPi(P,C) — logical predicate; payload and

state vector are arguments Effectsi = {Effecti,j(P,C)} — represents side effect

on variable Сi, performed on CondPi(P,C) success

Reaction – element of event classes finite set


Signature – triplet <SH,SP,R>, where SH – set of header conditions SP – ordered set of payload conditions R – reaction.

RES(<SH,SP,R>,H,P,C): {R}, if all the header conditions return ‘true’ and

all the payload conditions successfully evaluate consecutive.

, else

SNORT SIGNATURESSTRUCTURE

Action – action performed on rule activation (usually, ‘alert’)

Header – context-independent conditions on packet header

Options – context-dependent conditions on packet payload

Info – rule info and message, generated at rule activation

alert tcp any:80 -> any:any (content:”qwerty”; msg:”Panic”)

action header options info

SNORT SIGNATURES FORMALIZATION (1)

Header H1 — protocol type (TCP / UDP / ICMP / other IP)

H2 and H3 — IP and port of packet source

H4 — packet direction (from server/to server)

H5 and H6 — IP and port of packet destination

Payload – packet payload byte string

State vector – C = {C1}, C1 – position of text marker



SNORT SIGNATURES FORMALIZATION (2)

CondHk(H) – logical expression, operating {Hi} only

<CondPk(P,C), Effectk(P,C)> – pair of functions representing pattern matching performed after marker

Reaction – finite set of identifiers, which represent action and info sections



R-LANG LANGUAGEFORMAL SIDE

Automaton - (K, PS, T, PT, s0, I, g q) S – states set; PS – state predicates set; T – transitions set; PT – transition conditions set;

s0 – initial state; I – automaton exemplars set; g – global environment; q – global timer queue

automaton exemplar – series of slices NK x L x Q x E NK – current state; L – local environment; Q – local timer queue; E – local events queue

R-LANG LANGUAGEDECLARATIVE SIDE

Scenario Events subscription Variables declaration – scenario fields Functions declaration – scenario methods States declaration

One is marked as initial Transitions declaration

Transition type – consuming/nonconsuming/unwinding

Condition – logical function Body – code block

SIGNATURE CONVERSIONTO R-LANG (1)

S.T. Eckmann, "Translating Snort rules to STATL scenarios“

Typified events on every packet SrcIP, SrcPort, Direction, DstIP, DstPort Payload

Scenario field variable Current

Header conditions – logical expressions on {Hi}

Payload conditions – boolean function calls, with side-effect on Current

Reactions – alert sending code blocks

SIGNATURE CONVERSIONTO R-LANG (2)scenario sc(<event corresponding to packet type> ev)

{

<Secondary variables definition>

initial state st0;

consuming transition st0->st0

event <event corresponding to packet type> (

<Header condition 1> && … && <Header condition N>

) {

<Restoring state vector>

if(<Payload condition 1>)

…

if(<Payload condition M>)

<Reaction>;

}

};

HEADER-BASED OPTIMIZATION (1) Conditions alternative — a set of pairs <SPi,Ri>, where

SPi – ordered set of payload conditions

Ri — reaction

Alternative-containing signature – a pair <SH, SA>, where SH — set header conditions SA — condition alternative.

Set of signatures with the same header are converted to alternative-containing signature this way:

Alternative-containing signature evaluation result is defined as:

HEADER-BASED OPTIMIZATION (2) R-lang transition body of the alternative-containing

signature model: <Restoring state vector>

if(<Series of payload conditions 1>) <Reaction 1>;


if(<Series of payload conditions 2>) <Reaction 2>;

…


if(<Series of payload conditions N>) <Reaction N>;

We get the same RES set with less operation by grouping signatures with the same header conditions set.

519 different header condition sets for Snort base of 6372 signatures grants a good performance speed-up

HEADER-BASED OPTIMIZATION (3)

Tree-style order provides even more economy.

consuming transition st0->st0

event <event corresponding to packet type>(true){

if(<Header condition 1>) {

if(<Header condition 2>)

<predicate alternative for header granted by 1,2>



<predicate alternative for header granted by 1,3,4>

}

}

Snort analysis engine use that header optimization only, that does not allow achieving further speed-up on a fixed signature set.

PREDICATE TREE Fact: Condition chains contained in conditions

alternative can have the same beginnings.

Predicate tree ST – tree: the edges are marked with a payload condition the nodes are marked with a reaction set, possibly empty the leafs are marked with non-empty reaction sets

A B

A C DC

A

{R1}

{R2}

{R3}

A B

C DC

{R1}

{R2}

{R3}

PREDICATE TREE EVALUATION Predicate tree evaluation

recursive traversal of tree from root node conditions control sub-trees evaluation reaction performing if current node is marked context restoration after sub-tree evaluation

RES(ST, P,C) – {all the reaction achieved}

Statement: Let predicate tree ST is converted from condition alternative SA. Then

RES(ST,P,C)=RES(SA,P,C).

ADDITIONAL OPTIMIZATIONS

Synonymic bunch – set of leafs: the same {R} the same edges parent

PCRE union Aho-Corasick algorithm

{R}

{R}

{R}

A2

An … {R}

B

A1

STATIC RESULT ANALYSIS

Tree profit = edges number in the tree - total number of payload conditions

Relational tree profit = tree profit / total number of payload conditions

For 2 signature groups (Samba): ~2000 signatures tree profit > 11000 relational tree profit > 80%

Overall relational tree profit ~62%. 40 pairs of identical signatures

DYNAMIC RESULT ANALYSIS N seconds of flood with legal packets, a bad packet after

them

Test 1 – legal traffic is ignored by most rules FTP “HELP” command

Test 2 – legal traffic is checked by lots of rules specially constructed HTTP request

Seconds of alert delay

Before Optimization After optimization

Test 1, 5 seconds 123 10




CONCLUSION

Proposed methods provides significant speed-up of packet analysis at experimental IDS.

Though speed-up is significant, it is still not enough. There is a need for other optimizations, e.g. parallel signature matching.

REFERENCES

1. Snort IDS, http://www.snort.org

2. S.T. Eckmann, "Translating Snort rules to STATL scenarios" presented at the 4th International Symposium on Recent Advances in Intrusion Detection (RAID 2001), Davis, CA, October 2001, LNCS 2212, pp. 69-84.

3. M. Norton, "Optimizing Pattern Matching for Intrusion Detection," white paper, Sourcefire Inc., 2004 [Online] Avaliable: http:// docs.idsresearch.org/OptimizingPatternMatching/ForIDS.pdf.

4. D.U. Gamayunov, “Network objects behavior analysis for detecting computer attacks” PhD thesis, Faculty of Computational Math and Cybernetics, Moscow State University, Moscow, 2007.

5. S.T. Eckmann, G. Vigna, and R. A. Kemmerer. “STATL: An Attack Language for State-based Intrusion Detection” Dept. of Computer Science, University of California, Santa Barbara, 2000.

6. G. Vigna, R. Kemmerer, "NetSTAT: A Network-based Intrusion Detection Approach." Proceedings of the 14th Annual Computer Security Application Conference, Scottsdale, Arizona, December 1998.

7. M. Roesch. "Writing Snort Rules: How To write Snort rules and keep your sanity" [Online] Avaliable: http://www.snort.org.

Thank you for attention

network traffic analysis optimization for signature-based intrusion detection systems dmitry s....

Documents

header h

single packet slide

h logical expression

port of packet source

variable i

c logical predicate

serverto server h

p t transition conditions