efﬁcient techniques for fast packet classiﬁcationseclab.cs.sunysb.edu/alok/alok_cisco.pdf ·...

Efficient Techniques for Fast PacketClassification

Network Reading Group

Alok Tongaonkar, R Sekar

Stony Brook University

Sept 16, 2008

What is Packet Classification?

Packet Classification

A mechanism that

inspects network packets

determines how to process a packet based on the valuesof header fields and/or the payload.

Fundamental Operation

Identify the rules Ri that match a packet p from rules{R1, ..., Rn} whereRi : condition → actione.g., R1: dhost = PLUTO && dport = HTTP && content:‘‘Bad command’’ → DENY

Efficient Techniques for Fast Packet Classification

Applications

Firewalls

Identify highest priority matching rule

Intrusion Detection Systems

Use unordered rules

Identify all matching rules

Network Monitoring

Packet-filtering – whether a packet satisfies any of theconditions


Previous Techniques

Naive technique: Berkeley Packet Filter(BPF)

Match one rule at a timeA test that occurs in multiple rules is tested once on behalfof each of the rules


Previous Techniques

Trie-based techniques: PathFinder, Dynamic Packet Filter(DPF)

Identify common prefixes and share them{F1, F2}

type = IP

proto = TCP

{F1} {F2}

dport = A dport = B

{F1, F2}

{F1, F2}


Previous Techniques

DAG automaton: Berkeley Packet Filter(BPF+)

Recognize some equivalent states

Use data flow analysis to eliminate tests that are implied byother tests performed previously on the path

{F1}

{F2}

Shost X?

Shost Y ?

F

T

F

F T

Dhost X?

T

T

Dhost Y ?

F

{F1}

{F2}

Shost Y ?

F

T

F

F T

Dhost X?

T

T

Shost X?

Dhost Y ?


Previous Techniques

Adaptive Traversal

Change order of testing to promote sharing

{p1, p2, p3}

{p1, p2, p3} {p1, p3}

x 6= ax = a

y = b y = a y 6= a && y 6= b

y = b

y = a

y 6= a && y 6= b

{p1} {p2, p3} φ {p1} {p3} φ

{p1, p2, p3}

{p1}

y = b y 6= a && y 6= b

φ

{p2, p3}

y = a

{p2, p3}

x = a x 6= a

{p3}


Objective

Promote Sharing of Tests

Adaptive automata traversal – developed in the context ofterm-matching

Restricted to equality tests – we need to supportinequalities, disequalities, and bit-masking operations

Several new techniques in the context of the applicationdomain

Flexibility to Support Diverse Applications

Ordered (firewalls) and unordered (intrusion detection)rulesets

Packet-filtering (network monitoring)


Organization of Talk



Part I - Packet Field Matching

AlgorithmTechniques –

Intrusion Detection SystemsFirewalls

Evaluation






Evaluation

Part II - Content Matching

Integrating String Matching






Evaluation

Part II - Content Matching

Integrating String Matching

Part III

Related Work

Summary


Techniques for Packet Classification

Naive technique

A test that occurs in multiple rules is tested once on behalfof each of the rules

Automata-based techniques

Automaton states used to “remember” tests

Avoids repetition of tests


Deterministic Packet Classification Automaton

F1 : (icmp type = ECHO)

F2 : (icmp type = ECHO REPLY ) ∧ (ttl = 1)

F3 : (ttl = 1)

icmp type = ECHO

ttl 6= 1

{F2, F3} φ

{F2, F3}

ttl = 1 ttl 6= 1

{F3}{F1, F3} {F1}

{F1, F3}

ttl = 1

φ

{F3}

icmp type 6= ECHO REPLY ∧icmp type 6= ECHO

{F1, F2, F3}

ttl 6= 1ttl = 1

icmp type = ECHO REPLY

All but one transitionslabeled with test

Remaining transitionlabeled “other” –conjunction ofnegations of all testson the rest of thetransitions





F3 : (ttl = 1)

icmp type = ECHO

ttl 6= 1

{F2, F3} φ

{F2, F3}

ttl = 1 ttl 6= 1

{F3}{F1, F3} {F1}

{F1, F3}

ttl = 1

φ

{F3}


{F1, F2, F3}

ttl 6= 1ttl = 1


Transitions aresimultaneouslydistinguishable –

All tests except “other ”are mutually exclusive

Applicable transitioncan be determinedusing a singleoperation – O(1)expected timecomplexity





F3 : (ttl = 1)

icmp type = ECHO

ttl 6= 1

{F2, F3} φ

{F2, F3}

ttl = 1 ttl 6= 1

{F3}{F1, F3} {F1}

{F1, F3}

ttl = 1

φ

{F3}


{F1, F2, F3}

ttl 6= 1ttl = 1


Each final state S correctlyidentifies the match setcorresponding to anypacket satisfying all thetests along a path from thestart state to S.


Non-deterministic Packet Classification Automaton



F3 : (ttl = 1)

icmp type = ECHO

{F2, F3} φ

{F2, F3}

{F3}

ttl = 1

φ

{F3}

icmp type 6= ECHO REPLY

{F1, F2, F3}

ttl 6= 1ttl = 1

{F1}icmp type = ECHO REPLY

ttl 6= 1

“other” – conjunctionof negations of asubset of tests on therest of the transitions

Nondeterminism issimulated usingbacktracking atruntime


Principal Design Criteria for PCA

Operate in real-time on high-speed networks withoutdropping packetsScale to support thousands of rules

typical in intrusion detection systems and firewalls

Computational Issues

Matching time – closely related to path lengths

Memory – size of automata


Problem FormulationTests

Involve a variable x and one or two constants (denoted by c).

Equality tests x = ctcp sport = 80

Equality tests with bitmasks x&c1 = ctcp flags & 0x03 = 0x03

Disequality tests x 6= ctcp sport 6= 80

Disequality tests with bitmasks x&c1 6= ctcp flags & 0x03 6= 0x03

Inequality tests x ≤ c or x ≥ ctcp dport ≤ 1024


Filters and Priorities

A filter F is a conjunction of tests.(dport = 22) ∧ (sport ≤ 1024) ∧ (flags&0xb = 0x3 )A set F of filters may be partially ordered by a priority relation.The priority of F is denoted as Pri(F ).




A filter F matches a packet p, if:

the packet satisfies F , i.e., F (p) is true

the packet does not satisfy any rule that has higher prioritythan F




Match Set of p consists of all filters that match p, with theexception that among equal priority filters, at most one isretained.


Example of Prioritized Matching



F3 : (ttl = 1)

p1: icmp echo packet with ttl of 1

p2: icmp reply packet with ttl of 1





F3 : (ttl = 1)



Multi-matching (intrusion detection systems) – setincomparable priorities

M(p1) = {F1, F3}M(p2) = {F2, F3}





F3 : (ttl = 1)



Ordered matching (firewalls) – assign monotonically decreasingprioritiesPri(F1) > Pri(F2) > Pri(F3)

M(p1) = {F1}M(p2) = {F2}





F3 : (ttl = 1)



Packet-filtering (network monitoring) – set equal prioritiesPri(F1) = Pri(F3) = Pri(F2)

p1 can match either F1 or F3

p2 can match either F2 or F3


Matching Automata Construction

Key New Idea

Decompose and reorder tests to increase sharing of testsamong rules

Example

F1: (x = 5), F2: (x & 0x03 6= 1)



Key New Idea


Example

F1: (x = 5), F2: (x & 0x03 6= 1)

φ

{F1, F2}

{F1, F2} {F2}

{F1} {F1, F2} {F2}

x = 5 x 6= 5

x & 0x03 = 1 x & 0x03 6= 1

x & 0x03 6= 1x & 0x03 = 1



Key New Idea


Example

F1: (x = 5), F2: (x & 0x03 6= 1)

{F1, F2}

{F2}

{F2} φ

x&0x03 = 1

x & 0xfc = 4 x & 0xfc 6= 4

x & 0x03 6= 1


Condition Factorization

Decomposing filters into combination of more primitivetests

Similar to factorization of integers

Based on the residue operation – analogous to integerdivision

Residue

We want to determine if there is a match for a filter C1

We have so far tested a condition C2

A residue captures the additional tests that need to beperformed at this point to verify C1


Residue Operation

Definition (Residue)

The residue C1/C2 is another condition C3 such that:1 C2 ∧ C3 ⇒ C1

2 C1 ∧ C2 ⇒ C3

Examples

C1: x ∈ [1, 20], C2: x ∈ [15, 25]

C1: x ∈ [1, 20], C2: x = 15

C1: x ∈ [1, 20], C2: x = 35

C1: x ∈ [1, 20], C2: y = 15

C3: x ≤ 20

C3: true

C3: false

C3: x ∈ [1, 20]


Residue Operation



2 C1 ∧ C2 ⇒ C3

Ideally

C3 would be the weakest condition such that (1) holds


Residue Operation



2 C1 ∧ C2 ⇒ C3

In Practice

We might not want minimal condition since —

Expensive to compute

Inefficient to use – contains many disjunctions


Residue Operation



2 C1 ∧ C2 ⇒ C3

Example of Approximation

C1: x ∈ [1, 20], C2: x 6= 15

C3: x ∈ [1, 14] ∨ x ∈ [16, 20]

C′3: x ∈ [1, 20]


Residue Operation



2 C1 ∧ C2 ⇒ C3

Need for (2)

C3 shouldn’t be too strong, or else we may miss matches for C1

C1: x ∈ [1, 20], C2: x ∈ [10, 30]

C3: x ∈ [10, 15]

C3 satisfies (1) but not (2)

Will miss match for x ∈ [1, 9] or x ∈ [16, 20]


Computing Residue on Tests

T1 T2 T1/T2 Conditions

T T trueT ¬T false

T x = c T [x ← c]

x = c x & c1 = c2 x & c1 = c & c1 c & c1 = c2

false c & c1 6= c2

x = c x & c1 6= c2 false c & c1 = c2

x = c x ∈ [c1, c2] false c 6∈ [c1, c2]

x 6= c x & c1 = c2 x & c1 6= c & c1 c & c1 = c2

true c & c1 6= c2

x 6= c x & c1 6= c2 true c & c1 = c2

x 6= c x ∈ [c1, c2] true (c < c1) ∨ (c > c2)

x ∈ [c1, c2] x ∈ [c3, c4] true c1 ≤ c3 ≤ c4 ≤ c2

x ∈ [−∞, c2] c1 ≤ c3 ≤ c2 ≤ c4

x ∈ [c1,∞] c3 ≤ c1 ≤ c4 ≤ c2

x ∈ [c1, c2] c3 ≤ c1 ≤ c2 ≤ c4

false (c2 < c3) ∨ (c4 < c1)

x ∈ [c1, c2] x & c3 = c4 false c4 > c2

x & c1 = c2 x & c3 = c4 x & (c1 & c3) = (c2 & c3) c2 & c3 = c1 & c4

false otherwisex & c1 = c2 x ∈ [c3, c4] false c2 > c4

x & c1 6= c2 x & c3 = c4 x & (c1 & c3) 6= (c2 & c3) c2 & c3 = c1 & c4

true otherwisex & c1 6= c2 x ∈ [c3, c4] true c2 > c4

T T ′ T


Build Algorithm

Recursive procedure

Takes an automaton state s as its first parameter

Builds the subautomaton that is rooted at sIt takes two other parameters

Cs, the candidate set of the state sMs, the match set of s

Candidate Set

Cs – filters that haven’t completed a match, but future matchescan’t be ruled out either.

Match Set

Ms – all filters for which a match can be announced at s.


Build Algorithm

1. procedure Build(s, Cs,Ms)2. if Cs is empty3. then match[s] = Ms

4. else5. (D, T ) = select(Cs)6. To = {

∧di∈D|di=true ¬Ti}

7. for each Ti ∈ (T ∪ {To}) do8. Ci = Cs/Ti

9. if ((Ti 6= To) ∧ ¬di) then Ci = Ci − C/To endif10. compute Msi and Csi from Ci and Ms

11. if a state si corresponding to (Csi ,Msi ) isn’t present12. create a new state si

13. Build(si , Csi ,Msi )14. endif15. create a transition from s to si on Ti

16. end17. endif


Improving Automata Size

Key Idea

Pick tests which avoid duplication of filters in next states

T = {x = 5, x = 6, (x 6= 5) ∧ (x 6= 6)}C = {x = 5, x = 6, x > 7}C′ = {x = 6, x > 4}

{C1, C2, C3}

{C1} {C2} {C3}

X = 5X = 6 X 6= 5 ∧X 6= 6

X = 5X = 6 X 6= 5 ∧X 6= 6

{C2} {C1, C2} {C2}


Improving Automata Size

Key Idea

Pick tests which avoid duplication of filters in next states

T = {x = 5, x = 6, (x 6= 5) ∧ (x 6= 6)}C = {x = 5, x = 6, x > 7}C′ = {x = 6, x > 4}

Definition (Discriminating Set)

A set T of conditions is said to be a discriminating set for a filterset F iff for every F ∈ F there exists at most one T ∈ T suchthat F belongs to the candidate set of F/T .

Concept of discriminating tests is similar to the concept of index in the context of term matching.


Ensuring Polynomial-Size Automata

Breadth of subautomaton rooted at s

B(|Cs|) =k∑

i=1

B(|Csi |),

P(n) – the desired polynomial on n that bounds the automatonsize.

P(|Cs|) ≥k∑

i=1

P(|Csi |) (1)

Pick tests that satisfy the bounds

Pick a test that comes closest to satisfying this constraintand make some outgoing transitions nondeterministic


Benign Nondeterminism

Two filters F1 and F2 are said to be independent of each ifthey do not have a common test

Build separate automaton for each independent set

Match packets against each automaton – non-determinismwithout incurring any performance penalties


Effect of Benign Nondeterminism on Automata Size

Leads to dramatic reduction in automata size especially forintrusion detection systems.

If F1 and F2 are independent, packet may match F1, F2,both, or neither.

Number of states of automaton for F1 is k1, for F2 is k2.

Number of states of automaton for F1 ∪ F2 is k1 ∗ k2.

Combined number of states of independent automata forF1 and F2 is k1 + k2.


Improving Matching Time

Utility

How much a test goes towards checking a filter

Based on notion of assigning costs to tests and filters

Compare cost of a filter with combined cost of a test andthe residue of a filter w.r.t the test

select strategy

Size reduction more important than matching time1 Pick discriminating test when available

Pick test with higher utility

2 Examine opportunities for benign-nondeterminism3 Pick tests that satisfy polynomial bound


Measuring Matching Time

Implementation-independent metric for matching time

Suppose we could guess the set of rules that match apacket

The match verification cost is lower bound for anyalgorithm that tries to identify the matching rules

We use the ratio of actual matching cost to the lowerbound for match verification as a metric for matching time


Experiments Setup for IDS

Snort – open source

Comprehensive default signatures

Signatures consist of packet field tests andcontent-matching operation

Snort Next Generation (Snort-NG) – matches packet fieldsin parallel

Snort version 2 (Snort v2) – tries to parallelize matching forsome fieldsUsed 1635 default rules that come with Snort – combinedrules with same packet field tests to get 305 rules

System: 1.70Ghz pentium 4 processor, 520MB, CentOS-4.2 (Linux kernel 2.6)


Automaton Size

0

5000

10000

15000

20000

0 50 100 150 200 250 300

No.

of s

tate

s

Number of Filtering Rules

Condition FactorizationSnort-NG


Effect of Optimizations on Size

0

5000

10000

15000

20000

25000

30000

35000

40000

0 50 100 150 200 250 300

No.

of s

tate

s


LR TreeLR DAG

Adaptive TreeAdaptive DAG

Adaptive DAG w/ benign non-det


Matching Time – Lower Bound

0

5

10

15

20

25

0 50 100 150 200 250 300Avg

. Pat

h L

engt

h (i

n te

rms

of te

sts)


Adaptive TraversalLower Bound


Matching Time

0

10

20

30

40

50

60

70

80

90

0 50 100 150 200 250 300

Mat

chin

g T

ime

(in

s)


Snort 2Snort-NG

Condition Factorization


Experiments Setup for Firewall

Department firewall rules

Firewall rules in the form of iptable rules for a Linuxmachine

Network divided into different subnets140 filtering rules

System: 1.70Ghz pentium 4 processor, 520MB, CentOS-4.2 (Linux kernel 2.6)


Automaton Size

0

500

1000

1500

2000

2500

3000

3500

4000

0 20 40 60 80 100 120 140

No.

of s

tate

s


Adaptive Traversal DAG


Matching Time – Lower Bound

0

2

4

6

8

10

12

14

0 20 40 60 80 100 120 140Avg

. Pat

h L

engt

h (i

n te

rms

of te

sts)


Lower BoundActual Path Length


Extending Our Techniques for Content Matching

Key Idea

Use boolean variables corresponding to strings being matched

Test boolean variables to check presence of correspondingstring in payload

Treat tests on these boolean variables just like tests onother packet fields

F1 : (tcp sport = 80) ∧ (content =“Command complete”)

F2 : (tcp sport = 80) ∧ (content =“Bad command”) ∧ (content =“Bad filename”)



Extending Our Techniques for Content Matching

Key Idea

Use boolean variables corresponding to strings being matched

Test boolean variables to check presence of correspondingstring in payload

Treat tests on these boolean variables just like tests onother packet fields


F2 : (tcp sport = 80) ∧ (content =“Bad command”) ∧ (content =“Bad filename”)


F ′1 : C1 ∧ (X1 = 1)

F ′2 : C2 ∧ (X2 = 1) ∧ (X3 = 1)

F ′3 : C3 ∧ (X1 = 1)


Interesting Questions

When to Perform String Matching?

Perform string matching before passing packet to packetclassification automata

Lazy evaluation – perform string matching only whenpacket classification automata can not proceed

How to Handle Regular Expressions?

Use combined packet-field and string matching as prefilterfor RE matching

How can we include parts of RE in string matching to getmaximize gains from prefiltering?


Related Work

Left-to-right traversal based techniquesPathFinder, DPF – share common prefixBPF+ – uses global data flow techniques to eliminateredundant tests

Can not reason about semantic redundancies in presence ofcomplex test

Adaptive traversalSekar et al, Adaptive Binary Matching [Gustafsson]

Do not handle inequalities, disequalities, bit-fieldsAutomata has exponential worst case space complexityLinears size guarded sequential automata – require runtimemanipulation of match sets

Dynamic reordering techniquesDPF, Al-Shaer et al – maintain statistics regarding traffic

Techniques for routers – work on fixed number of fieldsSrinivasan et al, Lakshman et al – multidimensionalsearching problemWoo et al, Gupta et al – decision tree based techniques


Conclusion and Future Work

Summary

Developed a new technique for fast packet classificationFlexible – support diverse applications in a uniformframeworkPromotes sharing of tests

Developed novel techniques for generating packetclassification automata that

Have polynomial sizeVirtually constant matching time

Demonstrated the gains from our technique for intrusiondetection systems and firewalls

Future Work

Complete the integration and then evaluate the combinedcontent matching operation and packet field matching


Thank You

Acknowledgement: Sreenath Vasudevan

Questions?


Computing Match and Candidate Sets

Ps – denotes the conjunction of tests on the path from the startstate to s

Maintain only the residuals of the original filters in Cs andMs with respect to Ps





Match Set

M1 = {M ∈ F/Ps|(M = true)}M2 = {M∈M1|¬∃M ′∈F/Ps Pri(M ′) > Pri(M)}Ms is obtained by considering filters with equal priorities inM2, and deleting all but one of them.





Candidate Set

C(F ,M) = {C∈F |¬∃M ′∈M with Pri(M ′) ≥ Pri(C)}Cs = C(F/Ps,Ms)


efﬁcient techniques for fast packet classiﬁcationseclab.cs.sunysb.edu/alok/alok_cisco.pdf ·...

Documents