Dr. XiaoFeng Wang

AGIS: Towards Automatic Generation of Infection Signatures

Zhuowei Li1,3, XiaoFeng Wang1, Zhenkai Liang4 and Mike Reiter2

1 Indiana University at Bloomington2 University of North Carolina at Chapel Hill3 Center for Software Excellence, Microsoft4 Carnegie Mellon University

Dr. XiaoFeng Wang

Exploit signatures vs. infection signatures

Exploit Signature

Infection Signature

Dr. XiaoFeng Wang

How to get infection signatures?

Manually analyze malware infections

Automated analysis Invariant extraction from replication code Checksum Invariance from network traffic

cannot handle even the simplest metamorphism

Dr. XiaoFeng Wang

Our solution: AGIS

Automated malware analysisRun malware in a sandboxed environment Identify mal-behaviors using generalized polices

Automated infection signature generationFrom the code necessary for infections’ missions “vanilla” infections and regular-expression signatures

Certain resilience to obfuscated infections

Dr. XiaoFeng Wang

Differences from prior work

Behavior-based malware detectionOnly analyze add-on based infectionsNo signature generation

Panorama Finer-grained analysis, but very slowNo signature generation

Dr. XiaoFeng Wang

How does AGIS work?

Dr. XiaoFeng Wang

Malicious behavior detection

Create an infection graph

Set detection policies

Detection and behavior extraction

Dr. XiaoFeng Wang

Infection graph and back tracking

downloader.exe

keylogger.exe

keylogger process

run registryhook.dll

key.log

1. dowload 1. dowload

2. modify

3. run

4. hook

5. save

Dr. XiaoFeng Wang

Detection policies

Specifications for malicious behaviors

Keylogger rule syscall for hooking keyboard, and callback function output syscalls (Writefiles, Sendto…)

Mass-mailing worm rule loop for searching directories to read file, and syscall SMTP servers

Dr. XiaoFeng Wang

Infection signature extraction

Dynamic analysis and static analysisGet instructions necessary for malicious behaviors

Build signatures from the instructions

Dr. XiaoFeng Wang

Analyses

Dynamic analysisFind API calls for malicious behavior (M-calls) Identify their call sites through stack walking

Static analysis Instructions prepares for M-calls’ parameters (chops)

Dr. XiaoFeng Wang

Obfuscated code

Metamorphism Junk-code injection: dealt by chopsCode transposition: dealt by CFG register assignment, instruction replacement: left for

scanner

PolymorphismModify code signature

Dr. XiaoFeng Wang

Get signatures

Vanilla malware Chop

Regular-expression signatureBlocks: consecutive instructions on a chopConjunction of blocks

Dr. XiaoFeng Wang

Implementation

Kernel driverHook SSDT

Static analyzerBuilt upon Proview PVDASM

Dr. XiaoFeng Wang

Evaluations

MalwareMydoom (D/L/Q/U)NetSky (B/X)Spyware. KidLogger Invisible KeyLoggerHome Keylogger

Evaluations of detection and signature generation

Dr. XiaoFeng Wang

Examples for detection

MyDoom Loop-read using NtReadFile Send messages through NtDeviceIOControlFile Violate the mass-mailing rule

Spyware.KidLogger Hook using NtUserSetWindowsHookEx Write through NtWriteFile Violate the keylogger rule

False positives Find none from 19 common applications (BiTorrent, browers, MS

office, google desktop…)

Dr. XiaoFeng Wang

Chop for Mydoom.D

Dr. XiaoFeng Wang

Chop for Spyware.KidLogger

Dr. XiaoFeng Wang

FP rate vs. sig length

False Positive Rate vs. Signature Length

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30Signature length (Bytes)

Fal

se p

osi

tive

rat

e

CreateProcessA(KidLogger) SetWindowsHookExA(KidLogger) RegSetValueExA(MyDoom)

ReadFile(MyDoom) WS2_32.dll: send (MyDoom)

Dr. XiaoFeng Wang

Other evaluations

FP of vanilla signatures Statically checked 1378 normal programs, no match

Obfuscation Obfuscate code with RPME: extracted right chop Encode using UPX: found encoding loop

Performance Detection: around 1 minute Signature generation: less than 1 minute

Dr. XiaoFeng Wang

Limitations

User-land infections only

Not for add-ons

Undecideabiblity of Static obfuscation analysis

Obfuscation of behaviors

Dr. XiaoFeng Wang

Conclusions and future work

Achievements1st infection signature generation approach for hostWork on today’s user-land infections

Future workEfficient dynamic analytic toolsBetter scanning techniques