Detection and Propagation Modeling of Internet Worms

Ph.D. research proposal by:Parbati Kumar Manna

Co-advised by:Dr. Sanjay Ranka and Dr. Shigang Chen

2

Overview

• Research opportunities in Internet worm

• Contributions towards my dissertation:Detection of text worm

Propagation modeling for Permutation-Scanning worm

Finding the optimal scanning strategy

• Current status and timeline

3

• Computer Security Vs. Network Security

• MalwareComputer Viruses

Internet Worms

Trojans

Rootkits

Introduction

4

Internet Worm

• Huge damage potentialInfects hundreds of thousands of computersCosts millions of dollars in damageMelissa, ILOVEYOU, Code Red, Nimda, Slammer, SoBig, MyDoom

• Mostly uses Buffer Overflow• Propagation is automatic• Characterized by its host-level

and network-level behavior

5

Recent Trends

• Worms becoming increasingly evasive and obfuscative

• Arrival of Script Kiddies

• Emergence of Zero-day worms

• Shift in hacker’s mindset

6

Defenses

• PreventionSecure code, Corruption detection, Address randomization, Non-executable stack

• DetectionPayload signature, NOP sled, CFG, Abstract Payload Execution, Emulation, Honeypot,PADS

• ContainmentAddress blacklisting, Content filtering, Rate Limiting, LaBrea tarpit, Failed connection

• Propagation ModelingSI model, SIR, RCS, two-fact model

7

Smart Worm

Evaluation Parameters

Traditional Worm

Worm of Future

Disruption of Service Maximum Minimal

Network Footprint

Significantly high Relatively low

Detectability High Low

8

Proposed Work

Worm characterizationand countermeasures

How to detectthe new worms?

What are the effects on the Internet?

How bad can thesituation get?

• Evaluate existing detection systems against advanced worms

• Devise detection strategy for ASCII worm

•Obtain propagation characteristics for Permutation-Scanning worm

•Enhance the potency for Permutation-Scanning worm

•Identify the desired goals of scanning

•Compare the existing scanning methods

•Evaluate if any of the existing propagation strategies are optimal

9

Problem I

Detection of ASCII Worm

10

Motivation

• Presumption of text being benign

• Prevalence of servers expecting text-only input

• Deployment of ASCII filter for bypassing text

• Exponential disassembly cost

• High processing overhead for IDS

11

Buffer Overflow

Overflowing a buffer using an ASCII string:

12

Creation of ASCII Worm

13

Proposed Solution

Malicious Benign

• Lack of opcodes• No negative

displacement • Long decrypter• Long sequence of

valid instructions

• Contains characters that correspond to invalid instructions

• Long sequence of contiguous valid instructions unlikely

No error during execution

14

Proposed Solution

Questions:• How long is “long”?• What is the probability of false

positive for that threshold?

• Find out the maximum length of valid instruction sequence

• If it is long enough, the stream contains a worm

15

• Toss a coin n times• What is the probability that the max

inter-head distance is ?

Probabilistic Analysis

Head Invalid Instruction

Tail Valid Instruction

τ

T H T T H T T T T T H T T T

V I V V I V V V V V I V V V

τ

16

Probabilistic Analysis

n = number of coin tossesp = probability of a headXi = R.V.s for inter-head distancesXmax = Max inter-head distance

C.D.F of Xmax = Prob [Xmax ≤ x] = [1 – p(1-p)x ]n

F.P. rate α = 1 - Prob [Xmax ≤ τ] = 1 - [1 – p(1-p)τ ]n

17

Threshold Calculation

n , p, α (false positive rate)

τ (max inter-head distance)

Known

Unknown

)1log(log))1(1log(

1

ppn

−−−−

=ατThreshold

18

Threshold Calculation

With increasing n, we must choose a larger τto keep the same rate of false positive α

19

Determine n

size)n instructio (average )charactersinput ofnumber (

ICn =

E[I] = E[Prefix chain length] +E[core instruction length]

Obtained from character frequency of input data

20

1.Privileged instructions2.Wrong Segment Prefix Selector3.Un-initialized memory access

Invalid Instructions

Determine p

Only 1. and 2. can be determined on a standalone basis

21

Implementation

Traffic Data

Internet

ASCII Filter

InstructionDisassembler

InstructionSequenceAnalyzer

ASCIIWormDetector

Server

BinaryWormDetector

binary

ASCII

22

Experimental Setup

• Benign data setupASCII stream captured from live CISE network using Ethereal

• Malicious data setupExisting framework used to generate ASCII worm by converting binary worms

• Promising experimental results for max valid instruction length

Benign: all max values all below threshold τMalicious: values significantly higher than τ

23

Contributions

• Analyzed the behavior characteristics & constraints of ASCII worms and devised a detection method

• Derived mathematical foundation for generic detection method used in other worm detection strategies

• Deterministic - no “parameter tuning”

24

Problem II

Propagation Modeling for

Permutation-Scanning Worm

25

Motivation

• Random scanning Wastes scanning power

• Simple Divide schemeNot fault tolerant

Unequal load

26

Permutation-Scanning

• Randomizes the real address space into a Permutation Ring

• Each freshly infected host starts scanning from a random location

• Retires upon hitting an already infected host

Real address space Permutation

ring

new host jumps

about to infect

activeactive

retiredGets

infected, jumps

27

Why Model?

• Simulation takes long time16 hrs / run for 400M hosts

• Simulation overhead could be prohibitively high

Impossible to scan full IPv6

• Simulation does not always provide mathematical insight

28

• Find # (active hosts) scanning– effectively (X)– ineffectively (Y)

• Among the scans from the effective hosts (X), calculate how many are hitting uninfected hosts.

• Find how many X and Y hosts hit a pre-infected host (and retire).

Solution Outline

X1 X2

Y

coveredarea

29

Vulnerable Host Classification

30

State Diagram

31

Interaction among Infected Hosts while scanning

32

Final Model for O-jump Permutation Worm

VttxtiVtf

Vttxtitf

ttxtiVtiVtf

ttxtiVttxtf

NVdtrf

eff

ineff

new

old

hit

)()()()(

))()(()()(

)()()()()(

)()()()()()(

α

αα

αα

−+−=

−−=

−+−−

=

−+−−

=

××=Y

X

X

α

(effective)

(ineffective)

Fraction (covered area)

33

Final Model for O-jump Permutation Worm

0)0()0()0(,)0()0()0()()()(

)()()()(

)()()()()(

)()()()()(

)()()()()()()()()(

======+=

+=

−=

−=

−==

syxaitdytdxtda

ftytfftxtds

ftytftfftxtdy

fttftfftxtd

tfftxtftfftxtdxtfftxtdi

hitoldhit

hitineffnewhit

hiteffnewhit

oldhiteffnewhit

newhit

αψ

αα

infected

Retired

Active

34

Model Vs. Simulation

N = 223 V = 213 ψ (hitlist size) = 100

35

Extending Model to k-jump Permutation-Scanning Worm

• Instead of retiring, jump another time and restart scanning

• Will retire only after hitting more than k old infections

• Higher infection speed and network footprint

36

State Diagram for k-jump Permutation-Scanning Worm

37

Propagation Model for k-jump Permutation Worm

Similar equations for dα(t), dy(t)

38

Propagation Results for k-jump permutation worm

N=223

V=213

ψ=100

39

Contributions

• Obtained propagation model for Permutation-Scanning worms

• Extended modeling for multiple-jump

• Obtained the effect of various worm/network parameters:

Bigger hitlist (ψ)Larger V (more vulnerable computers)Bigger N (IPv4 →IPv6)Increased k (more jumps allowed)

40

Problem III

Comparative Analysis of Different Scanning

Strategies

41

The Next Big One?

• Warhol worms

• Self-stopping wormsHigh infection speed

Very low network footprint

Modest fault tolerance

42

Motivation

• To find the optimal scanning strategy

• Achieve the most desirable goals of scanning

Infection speed

Stealth

Fault tolerance

43

The Three Proponents

• Random-Constant-Spread wormVery high fault tolerance

• Divide-and-Conquer wormVery low network footprint

• Permutation-Scanning wormHigh infection speed

44

Proposed Work

• Derive propagation curves for all the scanning strategies using same set of notations in order to compare them

• Show equivalence of RCS and Permutation-Scanning worm in terms of infection speed

• Explore the possibility of hybrid scanning strategies

45

Current Status And

Timeline

46

Current Status

• Detecting ASCII WormsConference paper titled “DAWN: A Novel Strategy for Detecting ASCII Worms in Networks” submitted to IEEE INFOCOM 2008 and is currently under review

• Modeling Permutation ScanningConference paper titled “Exact Modeling of Propagation for Permutation-Scanning Worms” is pending review for IEEE INFOCOM 2008

• Finding Optimal Scanning StrategyWork currently in progressObtained theoretical equivalence between RCS and Permutation-Scanning wormIn process of modeling Divide-and-Conquer worm

47

Questions

48

Thank you