towards vulnerability-based intrusion detection with event processing
DESCRIPTION
Towards Vulnerability-Based Intrusion Detection with Event Processing. Amer Farroukh , Mohammad Sadoghi , Hans-Arno Jacobsen University of Toronto. Limitation of Regular Expressions. Conficker worm infected more than 10 million hosts in 2008. Economic loss tallied up to $9.1 billion. - PowerPoint PPT PresentationTRANSCRIPT
Towards Vulnerability-Based Intrusion Detection with Event Processing
Amer Farroukh, Mohammad Sadoghi, Hans-Arno Jacobsen
University of Toronto
July 13, 2011DEBS'11
DEBS'11
Limitation of Regular Expressions
• Conficker worm infected more than 10 million hosts in 2008.
• Economic loss tallied up to $9.1 billion.
July 13, 2011
IDS
Attacker
RE-Signature------------------
bin/*sh
bin/sh
binbin/sh
bin//shUofT
Network bin/delete
DEBS'11
Signature-based IDSes
• Exploit-based (Snort/ Cisco /Proventia): Regular Ex
• Vulnerability-based: Leverage protocol semantics
• Complex signatures: Multiple PDUs (ex. Conficker)
July 13, 2011
Buffer overflow (all exploits)Filename ="login.htm" &&len(uri.assignment_sequence.variable["password"])>20
Buffer overflow (shellcode)content: "|74 07 eb|“ && distance: 1 && within: 1 && pcre: "/\xeb.[\x58-\x5b]\x31[\xc9\xd2 \xdb]/bin/sh”
Buffer overflow after binding to serverBIND PDU: ver=3.0 && UUID=“4b324fc8-1670-01d3-1278-5a47bf6ee188”ACK PDU: ver=3.0 && result[UUID] = Accept REQ PDU: ver=3.0 && opnum=0x1f && strlen(stub.PathName)> 256 && matchRE(stub.PathName, “/^\x05\x00\x00”)
DEBS'11
Outline
• Related Work & System Architecture• Matching Algorithms– Access Predicate Pruning (APP)– Early Elimination (EE)
• Multiple Protocol Data Units (MPDU) Support–Memory Conscious Network (MCN)
• Experimental Evaluations • Conclusions
July 13, 2011
Related WorkVulnerability-based signature matching• Evaluate signatures over a stream data packets
– High-speed matching [RAID’08]• Programmer has to hard code signatures into the parser.
– Candidate Selection (CS) [SIGCOMM‘10]• Only algorithm proposed in IDS to match many signatures• Re-compute candidate list for every field parsed
Event processing (Publish/Subscribe Matching)• Evaluate subscriptions (signatures) over a stream of events (packets)
– Propagation [SIGMOD’01]• Targets specific type of predicates
– Counting [ACM TODS‘94]• Predicate matching and signature matching are distinct.• Can support arbitrary matchers
– BE-Tree [SIGMOD’11] (EPTS Principle Award)• Two-phase space-cutting to iteratively refine and prune the search space
July 13, 2011 DEBS'11
DEBS'11
Event Processing vs. IDS
July 13, 2011
Metric Event Processing IDSWorkload Dynamic (subs
constantly enter and leave the system)
Static (DS torn down and rebuilt when a new signature is added)
Parsing Messages are parsed before they are passed to broker
Parsing is crucial to enhancing performance
Matching Probability Large number of subs are matched
Signatures are rarely matched
Memory Clean-up Partial matches may reside in the system for an extended time
Memory per connection must be minimal
Our Contribution
Multiple PDUComponent
(MCN)
Parser Generator
Signature Compiler
Protocol Specs
Vulnerability Signature set
Individual Matchers (e.g., String, RE Matchers)
Matching Algorithm
M1 M2 M3 M4 M5
APP EE
Traffic Capture(Libpcap)
TCP Reassembly(Libnids)
Protocol Identification
(Port or PIA_Bro)
Leverage Existing Systems
Packets
StubPAC
IDL File&
Signatures
Netshield Core Engine
Protocol Parser(Minimal)
System Architecture
July 13, 2011 DEBS'11
DEBS'11
Outline
• Related Work & System Architecture• Matching Algorithms– Access Predicate Pruning (APP)– Early Elimination (EE)
• Multiple Protocol Data Units (MPDU) Support–Memory Conscious Network (MCN)
• Experimental Evaluations • Conclusions
July 13, 2011
Predicate List
Access Predicate List
Ap1
P1
P2
Ap2
P3
P4
P5
Ap3
P6
P7
ApN
Pi
Pj
Pk
SN4
S33
S24
S13S1 S2 S3 SN
Pre-computation Phase
Partial Matches
Add to List
SiIndex
Counter
CN4
C31
C23
C11
Sj Create Counter Cj
1
Runtime Signature Matching
CheckCounters
String Matche
rAccess Predicat
e List
Predicate List
Number
MatcherAccess
Predicate List
Predicate List
Length Matche
rAccess Predicat
e List
Predicate List
Range Matche
rAccess Predicat
e List
Predicate List
RE Matche
rAccess Predicat
e List
Predicate List
Predicate Type
Pi
Runtime Predicate Matching
Access Predicate Pruning (APP)Access
Predicate
Signature
Predicate
SN is matched
Access Predicate List PartialMatches
Add to List
Sj Create Counter Cj1
Runtime Signature Matching
CN4
C31
C23
C11
CheckCounters
Signature Compilation
P1
P2
P9
S1 P1
P4
P5
P6
S2 P4
P7
P8
S3 Pi
Pj
Pk
SN
Sid Increases
S1
S2
S4
S5
S9 PredicateList
Dual ScanIncrement Counter
(If Matched)
Early Elimination (EE)
Ap1
P1
P2
Ap2
P3
P4
P5
Ap3
P6
P7
ApN
Pi
Pj
Pk
SN4
S33
S24
S13S1 S2 S3 SN
Pre-computation Phase SN is matched
DEBS'11
1000 5000 10000 200000
0.5
1
1.5
2
2.5
3
3.5
600 HTTP Attacks
CSCOAPPEE
Number of Signatures
Mat
chin
g Ti
me
(ms)
pe
r con
necti
on
1000 5000 10000 200000
0.10.20.30.40.50.60.70.80.9
1
600 HTTP Partial Attacks
CSCOAPPEE
Number of Signatures
APP and EE Evaluation
July 13, 2011
12
AP Selectivity
Clean Attacks Partial Attacks Partial Attacks with AP
0
20
40
60
80
100
120
140
160
180Netshield 794 HTTP Signature Set
CSCOAPPEE
Traffic Type
Mat
chin
g T
ime
(µs)
Per c
onne
ction
DEBS'11
Outline
• Related Work & System Architecture• Matching Algorithms– Access Predicate Pruning (APP)– Early Elimination (EE)
• Multiple Protocol Data Units (MPDU) Support–Memory Conscious Network (MCN)
• Experimental Evaluations • Conclusions
July 13, 2011
MPDU Signatures:-----------------------S4=S2&S3S5=S1->S2S6=S1&(S2&S3)S7=(S1||S2)&S3
S1 S2 S3
00&
00&
Sample run:---------------
S1 S3 S210&
01&11&
11&
S4
S7
S5S6
Output:---------------
S7
||
S5
S4
0->
00&
S7
S600&
HASH
Si
00&
JN1 JN2 JN3
JN4 JN5
Memory Conscious Network (MCN)
SignatureNodes
JoinNodes
---------------------------------------------------------------------------------
DEBS'11
0 30 60 1000
50
100
150
200
250
300
100 MPDU Signatures
SeqSeqGMCNMCNG
Percentage of Attacks
Mat
chin
g Ti
me
(µs)
per c
onne
ction
MCN Evaluation
Algorithm Sequential MCN
Signature Nodes
290 72
AND Nodes 80 58
NEXT Nodes 85 68
OR Nodes 30 20
Memory per connection (bytes)
31 24
July 13, 2011
DEBS'11
Conclusions and Future Work
• Vulnerability-based signature matching– Proposed two novel solutions APP and EE– Attack resilient and faster than CS– Access predicate selectivity (future work)
• MPDU support– One of the first efforts to match MPDU signatures– MCN is memory efficient and 29 times faster than sequential
scan– Balancing network depth and node sharing (future work)
July 13, 2011
DEBS'11
Thank You
July 13, 2011
DEBS'11
Challenges of Vulnerability Signatures
• Enable high speed parsing– Parse only relevant fields
• Support arbitrary matchers– RE, strings, length-checking, numbers, and ranges
• Reduce state maintenance– Avoid state explosion for MPDU matching
July 13, 2011
DEBS'11
• Time Complexity (Worst Case)– APP• For every predicate: O(Predicate List + AP List) • Final Scan: O(Partial Matches List)
– EE• For every predicate: O(Predicate List + Partial Matches AP
List) • Final Scan: O(Partial Matches List)
• Memory Footprint (APP & EE)– Determined by size of Partial Matches List
July 13, 2011
APP and EE Complexities