01/04/2006 ecs236 winter 2006 1

ecs236 Winter 2006:

Intrusion DetectionIntrusion Detection#1: IDS Architecture

Dr. S. Felix Wu

Computer Science Department

University of California, Davishttp://www.cs.ucdavis.edu/~wu/

sfelixwu@gmail.com

01/04/2006 ecs236 winter 2006 2

Intrusion PreventionIntrusion Prevention

Prevention: This should/must never be broken in!– “This” means a perfectly designed,

implemented, and managed/configured secure system!

01/04/2006 ecs236 winter 2006 3

Intrusion DetectionIntrusion Detection

Prevention: This should/must never be broken in!

Detection: “This” will need to face the reality check!– We had, have, will have so many “expected”

unexpected.– Industry never really serious about cyber

security – profit/market-driven

01/04/2006 ecs236 winter 2006 4

We accept it as a fact…We accept it as a fact…

01/04/2006 ecs236 winter 2006 5

And, we have to have…And, we have to have…

01/04/2006 ecs236 winter 2006 6

Intrusion DetectionIntrusion Detection

Prevention: This should/must never be broken in!

Detection: “This” will need to face the reality check!– We had, have, will have so many “expected”

unexpected.– We had, have, will have even more

“unexpected” unexpected!!

01/04/2006 ecs236 winter 2006 7

To: All Faculty, Staff and Students

On Tuesday, January 03, 2006, UC Davis implemented temporary measures to prevent the exploitation of a serious new computer vulnerability for which no patch is yet available. This vulnerability affects Windows 2000, Windows XP, Windows Server 2003, Windows 98 and ME systems and may be exploited when infected email file attachments or infected Web pages are viewed. Once a computer is infected, data may be permanently lost and/or a remote attacker could gain control of the computer. After extensive consultation with the campus leadership, the decision has been made to temporarily block wmf image attachments. These files can have a number of different extensions, but most commonly will have .wmf and .jpg extensions.

01/04/2006 ecs236 winter 2006 8

Max-Sequence # AttackMax-Sequence # Attack Block LSA updates for one hour by injecting

one bad LSA.– You can hit it once and come back in an hour.

Implementation Bug!– Two independently developed OSPF packages.

– MaxSeq# LSA Purging has not been implemented correctly!!

Announced in May, 1997.

01/04/2006 ecs236 winter 2006 9

What is Intrusion Detection?What is Intrusion Detection?

01/04/2006 ecs236 winter 2006 10

Intrusion DetectionIntrusion Detection Detecting intrusions such as

– Viruses, Worms, Spywares, Phishing, Spamming, Insider, Un-authorized activities, faults/failures, among many others

Detecting and Managing anything “unexpected”– Anomalies

Question: “Detecting what??”

01/04/2006 ecs236 winter 2006 11

Intrusion DetectionIntrusion Detection

IntrusionDetection

Model

Input eventsequence Results

01/04/2006 ecs236 winter 2006 12

Results??Results?? This email contains virus XYZ This email might be a spam with 80%

probability This email is somewhat trusted based on

your social network This email might be malicious This email might be malicious for reasons

ABC and DEF.

01/04/2006 ecs236 winter 2006 13

Intrusion DetectionIntrusion Detection

IntrusionDetection

Model

Input eventsequence Results

Pattern matching

01/04/2006 ecs236 winter 2006 14

IDS EventsIDS Events

TCPdump traces OS kernel and Host-level information BGP traces Application Logs Many others…

01/04/2006 ecs236 winter 2006 15

Anti-VirusAnti-Virus

VirusDetection

VirusDefinition

Input eventsequence Results

Pattern matching

01/04/2006 ecs236 winter 2006 16

Credit Card Fraud DetectionCredit Card Fraud Detection

FraudDetection

SpendingPatterns

Input eventsequence Results

Statistical Pattern Matching

01/04/2006 ecs236 winter 2006 17

SNORTSNORT

Rules

Input eventsequence Results

Pattern matching

01/04/2006 ecs236 winter 2006 18

Welcome to ecs236Welcome to ecs236

S. Felix Wu – sfelixwu@gmail.com, x4-7070

Office: 3057 Engineering II Office Hours:

– 2-3 p.m. on Tuesday and Friday– by appointment

01/04/2006 ecs236 winter 2006 19

Intrusion DetectionIntrusion Detection

Practical Engineering– Performance, Accuracy, Scalability,

CPU/Memory, Correlation, Deployment. Theoretical Foundation

– Detectability/Limitation, Dimensionality, Entropy, False Negative and Positive, Evaluation

01/04/2006 ecs236 winter 2006 20

In this quarter…In this quarter… The architecture of ID and IDS

– Stateful versus stateless– Signature, specification, anomaly

Analysis of ID Results– Explanation and Analysis– Event Correlation

IDS Evaluation or Attacking IDS– Attack Polymorphism and IDS Evasion

IDS Fundamental Principles

A balance between Engineering a High-Performance IDS system

Fundamentally understand our limitations

01/04/2006 ecs236 winter 2006 21

Starter: SNORTStarter: SNORT Understand the architecture and source code How to evaluate SNORT? What is the most critical performance

bottleneck of SNORT? Is SNORT stateful or stateless? Why?

– What are the pros and cons regarding SNORT versus Bro?

http://www.snort.org/

01/04/2006 ecs236 winter 2006 22

SyllabusSyllabus SNORT IDS engine Anomaly-based Approach Event Correlation and Analysis IDS Evaluation Advanced Research Topics

01/04/2006 ecs236 winter 2006 23

Course RequirementsCourse Requirements 30%: Starter 15%: Proposal 30%: Final Project 25%: Class Participation

– “develop interesting/creative research problems related to the lectures/reading assignments, and justify the reasons”

– And, you need to interact with the instructor!– 5 of them 5% each (1~2 pages)

01/04/2006 ecs236 winter 2006 24

Final ProjectsFinal Projects Polymorphic/Metamorphic Worm detection Integration of Network/Host IDSes Anomaly Detection in SNORT IDS Evaluation using TCPopera SNORT event correlation and explanation Stateful SNORT SNORT evasion

01/04/2006 ecs236 winter 2006 25

about Web siteabout Web site

http://www.cs.ucdavis.edu/~wu/ecs236/ all lectures, notes, announcements,

homework assignments, tools, papers will be there.

01/04/2006 ecs236 winter 2006 26

Let’s start it…Let’s start it…

SNORT 2.4.3– You might need to install the PCRE (Perl

Compatible Regular Expression) package. Get it compile and install

– Any platform you like…

01/04/2006 ecs236 winter 2006 27

SnortSnort Open Source, since 1998 Used by many major network security

products Signature-based (more than 3000+) Simple IP header protocol anomaly

detection Simple stateful pattern matching

01/04/2006 ecs236 winter 2006 28

The Spirit of SNORTThe Spirit of SNORT

They started with something very simple and extensible.

If we feel we need the XYZ feature (due to an attack like STICK), we will write a plug-in for XYZ!!

An evolving system– But, not sure how much in the future…

01/04/2006 ecs236 winter 2006 29

Signature-base NIDSSignature-base NIDSMartin Overton, “Anti-Malware Tools: Intrusion Detection Systems”,

European Institute for Computer Anti-Virus Research (EICAR), 2005

Signature found at W32.Netsky.p binary sample

Rules for Snort:

01/04/2006 ecs236 winter 2006 30

Signature-based RuleSignature-based Rulealert tcp $EXTERNAL_NET any -> $HOME_NET 139

flow:to_server,established

content:"|eb2f 5feb 4a5e 89fb 893e 89f2|"

msg:"EXPLOIT x86 linux samba overflow"

reference:bugtraq,1816

reference:cve,CVE-1999-0811

classtype:attempted-admin

01/04/2006 ecs236 winter 2006 31

SNORT RulesSNORT Rules

Alert Pass log

01/04/2006 ecs236 winter 2006 32

01/04/2006 ecs236 winter 2006 33

False Alarm Rate versus False Alarm Rate versus False PositiveFalse Positive

Rules

101 events100 good + 1 bad

Pattern matching

2 alerts1 good + 1 bad

False Alarm Rate = 50%False Positive = 1%

01/04/2006 ecs236 winter 2006 34

STICKSTICK

SNORTrules STICK

AttackPackets

StatelessSNORTAlerts

01/04/2006 ecs236 winter 2006 35

What Alerts do we want?What Alerts do we want?

This is an administrative/policy issue.– Do I want to know this?

Idea: How can we rank the information quantitatively (in a meaningful way)?– Maybe it is hard to rank “one particular alert”– But, it is much more useful to rank “a sequence

of alerts/events” Correlation & Anomaly Detection!!

01/04/2006 ecs236 winter 2006 36

01/04/2006 ecs236 winter 2006 37

01/04/2006 ecs236 winter 2006 38

PreprocessorPreprocessor

Stream4 Frag2 Telenet_negotiation HTTP normalization RPC_decode Portscan Back Orifice

01/04/2006 ecs236 winter 2006 39

Experimental Pre-Experimental Pre-

Arpspoof Asn1_decode Fnord (NOP detection) portscan2

01/04/2006 ecs236 winter 2006 40

01/04/2006 ecs236 winter 2006 41

01/04/2006 ecs236 winter 2006 42

RTNRTNRule Tree Node

01/04/2006 ecs236 winter 2006 43

RTNRTN//OTNOTN MatrixMatrixOptional Tree Node

alert tcp any any -> 192.168.1.0/24 111 (content:”|00 01 86 a5|”; msg:”mountd access”;)

01/04/2006 ecs236 winter 2006 44

01/04/2006 ecs236 winter 2006 45

SNORT RulesSNORT Rules

Dynamic Activation Alert Pass log

01/04/2006 ecs236 winter 2006 46

01/04/2006 ecs236 winter 2006 47

01/04/2006 ecs236 winter 2006 48

Fast Multiple PatternsFast Multiple Patterns

Wu-Manber (Bad-word shift)– Consume the least amount of memory

Aho-Corasick (FSM)– Fast, potential for parallelism and FPGA

Boyer-Moore (Bad-word shift)– For small rule sets

01/04/2006 ecs236 winter 2006 49

Example: P = {he, she, his, hers}

0

1

h

2

9

8

6

3

4

57

e

s

i h

s er

s

Initial State

Accepting State

StateTransition Function

h Sh

hh

hh

S

SS

S

S

S

i

h

r

h

•The Construction: linear time.•The search of all patterns in P: linear time

(Edges pointing back to State 0 are not shown).

01/04/2006 ecs236 winter 2006 50

DistanceDistance

content:"SITE"; nocase; content:"EXEC"; distance:0; nocase;

01/04/2006 ecs236 winter 2006 51

WithinWithin

content: "Content-type\: video/x-ms-asf"; content:"|0a|"; within:2;

01/04/2006 ecs236 winter 2006 52

Byte jumpByte jump

byte_jump:4,4, relative,align; byte_jump:4,4, relative,align; content: "|00 01 86 A5|"; within:4;

01/04/2006 ecs236 winter 2006 53

Byte testByte testbyte_test:1,>,7,1;

01/04/2006 ecs236 winter 2006 54

A few issues about A few issues about SNORT basicsSNORT basics

Performance– DoS attack against SNORT

Expressiveness– Statefulness in SNORT – is it enough?

01/04/2006 ecs236 winter 2006 55

HW#1HW#1

I will post HW#1 details on the class website this afternoon (01/11/2006)

01/04/2006 ecs236 winter 2006 56

BroBro

Packet stream

Filtered packet stream

Event stream

AlertsPolicy script

Event control

tcpdump filters

Event Engine

Network

Libpcap

Detection Engine

01/04/2006 ecs236 winter 2006 57

Be Stateful!! (Bro Be Stateful!! (Bro SNORT) SNORT) HTTP server attack

– Snort signature: simple pattern matching on MS ISS attack

– Bro rule: additional check to see if, e.g., host is running Apache ignore alarm

Error code checking– Snort signature: no checking of reply

– Bro rule: Looks at return code for HTTP/FTP/SMTP, signature match + error code = no alert

Multi-stage attacks– Easy in Bro to express “signature A but only if followed by signature B” or

“A unless followed by B”

– Easy to express “generate alarms if given host triggers N or more signatures” or “triggers against N or more local hosts”

01/04/2006 ecs236 winter 2006 58

Stateful is good?Stateful is good? How to design an IDS

– Performance– Statefully powerful??

01/04/2006 ecs236 winter 2006 59

01/04/2006 ecs236 winter 2006 60