an empirical approach to modeling uncertainty in … · an empirical approach to modeling...
TRANSCRIPT
![Page 1: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/1.jpg)
An empirical approach to modeling uncertainty in Intrusion Analysis
Xinming (Simon) Ou
Kansas State University
Joint work with S. Raj Rajagopalan at HP Labs and Sakthi Sakthivelmurugan
Departmental seminar, Computer Science at Virginia TechMarch 19, 2010
![Page 2: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/2.jpg)
system administrator
Network Monitoring
Tools
Abnormally high traffic
TrendMicro server communicating with
known BotNet controllers
memory dump
Seemingly malicious code
modules
Found open IRC sockets with other TrendMicro servers
netflow dump
These TrendMicro Servers are certainly compromised!
2
A day in the life of a real SA
Key challenge: uncertainty in data.
![Page 3: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/3.jpg)
An empirical approach
• In spite of the lack of theory or good tools, sysadmins are coping with attacks.
• Can we build a system that mimics what they do (for a start):– An empirical approach to Intrusion Analysis using
existing reality
• Our goal: – Provide some degree of automation in the process
3
![Page 4: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/4.jpg)
High-confidence Conclusions with Evidence
Targeting subsequent observations
Mapping observations to their semantics
IDS alerts, netflow dump, syslog, server log …Observations
Internal model
Reasoning Engine
4
![Page 5: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/5.jpg)
Capture Uncertainty Qualitatively
Confidence level
Uncertainty Modes
Low Possible pModerate Likely lHigh Certain c
• Arbitrarily precise quantitative measures are not meaningful in practice
• Roughly matches confidence levels practically used by practitioners
5
![Page 6: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/6.jpg)
High-confidence Conclusions with Evidence
Targeting subsequent observations
Mapping observations to their semantics
IDS alerts, netflow dump, syslog, server log …Observations
Internal model
Reasoning Engine
6
![Page 7: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/7.jpg)
Observation Correspondence
7
obs(anomalyHighTraffic) int(attackerNetActivity)
obs(netflowBlackListFilter(H, BlackListedIP))
obs(memoryDumpMaliciousCode(H))
obs(memoryDumpIRCSocket(H1,H2))
p
int(compromised(H))l
int(compromised(H))l
int(exchangeCtlMessage(H1,H2))l
Observations Internal conditionsmode
what you can see what you want to know
![Page 8: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/8.jpg)
High-confidence Conclusions with Evidence
Targeting subsequent observations
Mapping observations to their semantics
IDS alerts, netflow dump, syslog, server log …Observations
Internal model
Reasoning Engine
8
![Page 9: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/9.jpg)
Internal Model
9
Logical relation among internal conditions.
Condition 1 Condition 2Condition 1 infers Condition 2
int(compromised(H1)) int(probeOtherMachine(H1,H2))
int(sendExploit(H1,H2)) int(compromised(H2))
int(sendExploit(H1,H2))int(compromised(H2))
int(compromised(H1))int(probeOtherMachine(H1,H2))
direction of, inference mode
f,
f,
b,
b,
p
l
p
c
![Page 10: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/10.jpg)
High-confidence Conclusions with Evidence
Targeting subsequent observations
Mapping observations to their semantics
IDS alerts, netflow dump, syslog, server log …Observations
Internal model
Reasoning Engine
10
![Page 11: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/11.jpg)
Reasoning Methodology
11
• Simple reasoning– Observation correspondence and internal model
are inference rules
– Use inference rules on input observations to derive assertions with various levels of uncertainty
• Proof strengthening– Derive high-confidence proofs from assertions
derived from low-confidence observations
![Page 12: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/12.jpg)
Example 1Inference through Observation Correspondence
12
int(exchangeCtlMsg(172.16.9.20, 172.16.9.1), l )
obs(memoryDumpIRCSocket(172.16.9.20, 172.16.9.1))
obsMap
obs(memoryDumpIRCSocket(H1,H2)) int(exchangeCtlMessage(H1,H2))l
![Page 13: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/13.jpg)
Example 2 Inference through Internal Model
13
int(exchangeCtlMsg(172.16.9.20, 172.16.9.1), )
obs(memoryDumpIRCSocket(172.16.9.20, 172.16.9.1))
int(compromised(172.16.9.20), )l obsMap
Int rule
l
![Page 14: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/14.jpg)
Proof Strengthening
14
Observations:
f is likely true f is likely true
O1 O2
f is certainly true
proof strengthening
O3
![Page 15: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/15.jpg)
Proof Strengthening Example
15
int(exchangeCtlMsg(172.16.9.20, 172.16.9.1), l )
obs(memoryDumpIRCSocket(172.16.9.20, 172.16.9.1))
int(compromised(172.16.9.20), l )obsMap
intR
obs(memoryDumpMaliciousCode(’172.16.9.20’))
int(compromised(172.16.9.20), l ) obsMap
int(compromised(172.16.9.20), ) strengthenedPf
strengthen( l, l ) = c
c
![Page 16: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/16.jpg)
Evaluation
• Test if the empirically developed model can derive similar high-confidence trace when applied on different scenarios
• Keep the model unchanged and apply the tool to different data sets
16
![Page 17: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/17.jpg)
SnIPS (Snort Intrusion Analysis using Proof
Strengthening) Architecture
17
Reasoning Engine
Snort alerts
(convert to tuples)
Observation Correspondence
User query, e.g.which machines are “certainly” compromised?
High-confidenceanswers with
evidence
pre-processing
Internal ModelSnort Rule Repository
Done only once
![Page 18: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/18.jpg)
Snort rule class type
18
alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS(msg:"WEB-MISC guestbook.pl access”;uricontent:"/guestbook.pl”;classtype:attempted-recon; sid:1140;)
obsMap(obsRuleId_3615, obs(snort(’1:1140’, FromHost, ToHost)),int(probeOtherMachine(FromHost, ToHost)), ?).
Internal predicate mapped from “classtype”
![Page 19: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/19.jpg)
Snort rule documents
19
Impact: Information gathering and system integrity compromise. Possible unauthorized administrative access to the server. Possible execution of arbitrary code of the attackers choosingin some cases.
Ease of Attack: Exploit exists
obsMap(obsRuleId_3614, obs(snort(’1:1140’, FromHost, ToHost)), int(compromised(ToHost)), p)
Hints from natural-language description of Snort rules
obsMap(obsRuleId_3615, obs(snort(’1:1140’, FromHost, ToHost)),int(probeOtherMachine(FromHost, ToHost)), ). l ?
![Page 20: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/20.jpg)
Automatically deriving Observation Correspondence
• Snort has about 9000 rules.
• This is just a base-line and needs to be fine-tuned.
• Would make more sense for the rule writer to define the observation correspondence relation when writing a rule.
20
Internal Predicate % of rules
Mapped automatically 59%
Not mapped automatically 41%
![Page 21: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/21.jpg)
Data set description
• Treasure Hunt (UCSB 2002) – 4hrs – Collected during a graduate class experiment– Large variety of system monitoring data: tcpdump,
sys log, apache server log etc.
• Honeypot (Purdue, 2008) – 2hrs/day over 2 months
– Collected for e-mail spam analysis project– Single host running misconfigured Squid proxy
• KSU CIS department network 2009 – 3 days– 200 machines including servers and workstations.
21
![Page 22: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/22.jpg)
Some result from Treasure Hunt data set
22
| ?- show_trace(int(compromised(H), c)).
int(compromised(’192.168.10.90’),c) strengthenedPf
int(compromised(’192.168.10.90’), p) intRule_1
int(probeOtherMachine(’192.168.10.90’,’192.168.70.49’), p) obsRulePre_1
obs(snort(’122:1’,’192.168.10.90’,’192.168.70.49’,_h272))
int(compromised(’192.168.10.90’),l) intRule_3
int(sendExploit(’128.111.49.46’,’192.168.10.90’), l) obsRuleId_3749
obs(snort(’1:1807’,’128.111.49.46’,’192.168.10.90’,_h336))
An exploit was sent to
192.168.10.90
A probe was sent from
192.168.10.90
192.168.10.90 was certainly
compromised!
![Page 23: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/23.jpg)
Data Reduction
Data set Duration of Network
traffic
Snort alerts pre-processed alerts
High-confidence
proofs
Treasure Hunt 4 hours 4,849,937 278 18
Honeypot 2 hrs/day for 2 months
637,564 30 8
CIS Network 3 days 1,138,572 6634 17
23
![Page 24: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/24.jpg)
Future Work
• Continue the empirical study and improve the reasoning model
• Establish a theoretical foundation for the empirically-developed method– Modal Logic
– Bayes Theory
– Dempster-Shafer Theory
25
![Page 25: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/25.jpg)
Related work
• Y. Zhai et al. “Reasoning about complementary intrusion evidence,” ACSAC 2004
• F. Valeur et al., “A Comprehensive Approach to Intrusion Detection Alert Correlation,” 2004
• R. Goldman and S. Harp, "Model-based Intrusion Assessment in Common Lisp", 2009
• C. Thomas and N. Balakrishnan, “Modified Evidence Theory for Performance Enhancement of Intrusion Detection Systems”, 2008
26
![Page 26: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/26.jpg)
Summary
• Based on a true-life incident we empirically developed a logical model for handling uncertainty in intrusion analysis
• Experimental results show– Model simulates human thinking and was able to
extract high-confidence intrusion– Model empirically developed from one incident
was applicable to completely different data/scenarios
– Reduction in search space for analysis
27
![Page 27: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/27.jpg)
Thank you
Questions?
28
![Page 28: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/28.jpg)
Summarization
29
• Compact the information entering reasoning engine
• Group similar “internal condition” into a single “summarized internal condition”
![Page 29: An empirical approach to modeling uncertainty in … · An empirical approach to modeling uncertainty in Intrusion Analysis Xinming (Simon) Ou Kansas State University. Joint work](https://reader030.vdocuments.net/reader030/viewer/2022020304/5b9f39d109d3f25b318cbe22/html5/thumbnails/29.jpg)
Comparison of the three data sets
30
1
10
100
1000
10000
100000
1000000
10000000
atte
mpt
ed-a
dmin
atte
mpt
ed-d
os
atte
mpt
ed-r
econ
atte
mpt
ed-u
ser
bad-
unkn
own
defa
ult-
logi
n-at
tem
pt
mis
c-ac
tivity
mis
c-at
tack
not-
susp
icio
us
polic
y-vi
olat
ion
non-
stan
dard
-pro
toco
l
prot
ocol
-com
man
d-de
code
rpc-
port
map
-dec
ode
shel
lcod
e-de
tect
succ
essf
ul-a
dmin
succ
essf
ul-r
econ
-lim
ited
susp
icio
us-f
ilena
me-
dete
ct
syst
em-c
all-d
etec
t
troj
an-a
ctiv
ity
unkn
own
web
-app
licat
ion-
activ
ity
web
-app
licat
ion-
atta
ck
Department Treasure Hunt Honeypot