modern security analytics finding a needle in the hayblower · & memory anomalous malicious...
TRANSCRIPT
![Page 1: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/1.jpg)
Modern Security Analytics Finding a Needle in the Hayblower
Martin Rehak, Principal Engineer
July 1, 2014, AIMS 2014, Brno
![Page 2: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/2.jpg)
Cisco Confidential 2 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Stream Security Analytics
• What is malware and why do we need to fight it?
• Step-by-step walkthrough through a security incident
• Security domain considerations, evolutions and particularities
• Stream Analytics: understanding big data in flight
• Ensemble of anomaly detectors and collective classification
• False Positives Analysis
![Page 3: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/3.jpg)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
The Advanced Malware Attack Lifecycle PLAN EXPLOIT / ATTACK INFECT / SPREAD STEAL / DISRUPT
Attacker determines possible entry points,
formulates a plan of attack
Attacker exploits vulnerabilities and delivers
its weapon
Malware moves laterally through the internal network in search of
additional resources and data
Attacker takes action on its objectives and
exfiltrates data or disrupts systems
HACKER
![Page 4: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/4.jpg)
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
1. Command and Control
2. Reconnaissance
3.Propagation 4. Data Theft
Firewall
IPS
Web Sec
N-AV
Email Sec
Routers
Switches
Firewall
Threat Detection
Attack Kill Chain: Post Breach
![Page 5: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/5.jpg)
Cisco Confidential 5 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Technology Escalation in Advanced Cyber Threats
Improved Delivery Mechanism
Targeted execution, limiting sandboxing, detection and analysis options
Advanced Command & Control
Obfuscation of content, behavior and mission
Hiding between the random malware infections
![Page 6: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/6.jpg)
Cisco Confidential 6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Trivial) Incident Walkthrough
![Page 7: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/7.jpg)
Cisco Confidential 7 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 8: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/8.jpg)
Cisco Confidential 8 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
High-Level View of the System
![Page 9: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/9.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
Cloud Web Security The Security Perimeter in the Cloud
The Distributed Perimeter
Cloud Connected Network
Collective Security Intelligence
Telemetry Data Threat Research Advanced Analytics
Mobile Router Firewall
3M+ Cloud Web Security Users
6 GB Web Traffic Examined, Protected Every Hour
75M Unique Hits Every Hour
10M Blocks Enforced Every Hour
![Page 10: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/10.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
Web
Filt
erin
g
Web
Rep
utat
ion
Mal
war
e S
igna
ture
File
Rep
/ S
andb
ox
Thre
at A
naly
tics
Act
ive
Rep
ortin
g
Roaming Users Headquarters Branch Office
Cloud
Cisco Cloud Delivered Security Capability
File
Ret
rosp
ectio
n
![Page 11: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/11.jpg)
Cisco Confidential 11 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Three Worlds of Security Analytics
![Page 12: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/12.jpg)
Cisco Confidential 12 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Finds known malware or other attacks by matching the content of communication or files against patterns of known attacks.
§ Analyses known malicious behavior to infer general characteristics of attacks and uses the model to discover new attacks by their associations with known attacks.
§ Analyses the normal behavior of the network to build a predictive model and detects any patterns deviating from the normal behavior as potentially malicious.
Pattern Matching Threat Intelligence Anomaly Detection
![Page 13: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/13.jpg)
Cisco Confidential 13 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Lowest False Alerts
§ Very specific and verifiable convictions
§ Proven and traditional industry standard…
§ … that needs to be complemented by other techniques in order to cope with advanced attacks.
§ Medium false alerts
§ Convictions are not direct, but still understandable by humans
§ Allows the detection of malware by exploiting the operational and technical imperfections of the attackers…
§ … not committed by the advanced organizations.
§ Higher false alerts
§ Convictions are not specific and are difficult to explain
§ Allows the detection of broadest scope of malware, including the advanced attacks…
§ … provided that we can separate the malware from random statistical fluctuations
Pattern Matching Threat Intelligence Anomaly Detection
![Page 14: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/14.jpg)
Cisco Confidential 14 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Complexity: Proportional to the number of known and described unique samples/traits that need to be described
§ Complexity: Proportional to the number of known botnets, malicious infrastructures,
§ Complexity: proportional to the number of users and organizations protected
Pattern Matching Threat Intelligence Anomaly Detection
![Page 15: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/15.jpg)
Cisco Confidential 15 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Anomaly Detection – A Machine Learning Problem
Anomaly detection requires us to build a predictive model of
1. global internet traffic 2. each customer’s network, and 3. each host or user in each customer’s network
while respecting strict privacy and economic constraints.
![Page 16: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/16.jpg)
Cisco Confidential 16 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
“Traditional” Big Data Workflow
1. Receive the data
2. Perform an ETL (Extract-Transform-Load) process Check correctness, formatting, deduplication Extract Features Schedule for writing into DB/Storage
3. Store the transformed data in data store
4. Run the analytics process on the data to find value
5. Display the results
![Page 17: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/17.jpg)
Cisco Confidential 17 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cost Breakdown
1. Receive the data
2. Perform an ETL (Extract-Transform-Load) process Check correctness, formatting, deduplication Extract Features Schedule for writing into DB/Storage
3. Store the transformed data in data store 4. Run the analytics process on the data 5. Display the results
![Page 18: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/18.jpg)
Cisco Confidential 18 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Slow and Expensive
Low Sensitivity (Recall)
Context SizeSmall Context Large Context
Affordable Effectiveness
Traditional Approach
![Page 19: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/19.jpg)
Cisco Confidential 19 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Stream Analytics
1. Receive the data
2. Perform an ETL (Extract-Transform-Load) process Check correctness, formatting, deduplication Extract Features Schedule for writing into DB/Storage
3. Run fast and effective analytics process on the data
4. Store the transformed and filtered data in data store
5. Run (second) analytics process
6. Display the results
![Page 20: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/20.jpg)
Cisco Confidential 20 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Slow and Expensive
Low Sensitivity (Recall)
Context SizeSmall Context Large Context
Affordable Effectiveness
Simple Stream Analytics
![Page 21: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/21.jpg)
Cisco Confidential 21 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Slow and Expensive
Low Sensitivity (Recall)
Context SizeSmall Context Large Context
Affordable Effectiveness
?
![Page 22: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/22.jpg)
Cisco Confidential 22 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Inside View
![Page 23: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/23.jpg)
Cisco Confidential 23 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Model Decomposition – Hierarchical and Distributed
AnomalyDetection Trust Models Classification
BehaviourModels
Output
Trust Models
AnomalyDetectors
ActivityClassifiers
Long-TermBehaviour
Models
Normal Flows Trusted Flows Non-Security Anomalies Fluctuations
• M Rehak, M Pechoucek, M Grill, J Stiborek, K Bartos, P Celeda. Adaptive multiagent system for network traffic monitoring. Intelligent Systems, IEEE 24 (3)
![Page 24: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/24.jpg)
Cisco Confidential 24 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Identity
Web
NetFlow
3rd Party Feeds
DNS
…
Trust Models
Trust Score
Local Correlation
Correlation Across Multiple
Networks
Global & Local Correlation
Near real time processing
Anomaly Detection
Investigative Output
High Precision Output
5 billions requests per day ± 1% is anomalous
10 M events per day 1K-50K Incidents per day
± 0.5% is security-relevant
![Page 25: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/25.jpg)
Cisco Confidential 25 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Cognitive Threat Analytics Layered Detection Engine
Agent 1
Agent 2
Agent 3
Agent N
Trust Modeling
Layer
WPAD
Generated Domain
Data Exfiltration
Malware 1
Malware 2
Malware 2
….
….
Unsupervised Learning
Supervised Learning
Individual Detectors
Correlation & Memory
Anomalous Malicious Malware
Classification / Layer 2 Classification / Layer 1 Detection Filtering
Incidents Data
….
![Page 26: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/26.jpg)
Cisco Confidential 26 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Examples of AD output (HTTP, real and synthetic malware)
Typical anomaly detection algorithm does not quite work Problems: False negatives False positives AUC around 0.7-0.8 not an exception
![Page 27: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/27.jpg)
Cisco Confidential 27 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
More AD output samples
![Page 28: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/28.jpg)
Cisco Confidential 28 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Anomaly Detection Approaches
Statistics and Empirical Distribution, Information Theory: • M Rehak, M Pechoucek, K Bartos, M Grill, P Celeda. Network intrusion detection by means of
community of trusting agents - IAT 2007 - IEEE/WIC/ACM I.C. Intelligent Agent Technology
Principal Components Analysis: • T Pevný, M Rehák, M Grill. Identifying suspicious users in corporate networks. Proceedings of
workshop on information forensics and security, 2012, 1-6
Graph Theory: • Jusko, J. and Rehak, M. (2014), Identifying peer-to-peer communities in the network by
connection graph analysis. Int. J. Network Mgmt.. doi: 10.1002/nem.1862 • Jusko, J, Rehak, M. (2012) Revealing Cooperating Hosts by Connection Graph Analysis,
Securecomm 2012
![Page 29: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/29.jpg)
Cisco Confidential 29 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Anomaly Detectors
Individual AD models can detect malicious behavior as an outlier
• Making them better would compromise their generality
Base-Rate Fallacy as a fundamental limitation • Not enough attacks in the traffic
Precision and Recall are not sufficient for direct use Can we make the Precision better?
![Page 30: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/30.jpg)
Cisco Confidential 30 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
• Long-term behaviors
• Confined to subset of hosts/users
• No direct relationship to background
• Structured and explainable
Structured Unstructured
• Short-term artifacts
• Uniformly distributed over users
• Proportional to traffic volumes
• Frequently associated with novelty, rather than anomaly
![Page 31: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/31.jpg)
Cisco Confidential 31 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
High-Level False Positive Breakdown
Anomaly
Unstructured
Noise
Structured
Structured Anomaly Dual Behaviour Malicious
Behaviour
![Page 32: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/32.jpg)
Cisco Confidential 32 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Unstructured Anomalies - Noise • Unstructured, short-term anomalies
• Evenly distributed over hosts with the distribution proportional to the background traffic • Triggered by widespread, uniformly distributed behaviors (such as web browsing or SW updates). • Small proportion of high-volume behavior will be anomalous.
• Same anomaly instance generated by one or small number of hosts only, but noise categories and types can spawn over many or almost all hosts. • One specific favicon anomaly will be there for one user, but every user will have a favicon anomaly with some probability
• Examples: • Novelty: Users visiting unusual discussion forums or unusual documents (how to repair a dishwasher X) • Random failures: Redirects, dead links, incorrect retries • Fragments: Fragments of legitimate browsing behavior (ads from less known sites invoked by js) • Multimedia: Flash games & other multimedia fragments
• T Pevny, M Komon, M Rehak. Attacking the IDS learning processes. Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE
![Page 33: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/33.jpg)
Cisco Confidential 33 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 34: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/34.jpg)
Cisco Confidential 34 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Structured Anomaly
• Understandable outlier behavior with internal structure of each instance
• Long-term behavior
• Specific to one host or a small subset • The type of the anomaly will be restricted to a small subset of hosts • Triggered by a specific application (software update/software usage), context (wpad) or by
combination of both factors (Skype running in some context)
• Additional evidence allows its (ex-post) separation from malicious behavior
• Examples: • Software updates • Skype/P2P over HTTP(S) • Configuration scripts, remote log creation, regular calls of unusual APIs
![Page 35: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/35.jpg)
Cisco Confidential 35 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 36: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/36.jpg)
Cisco Confidential 36 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Dual Anomalies
• Major characteristics are identical with Structural Anomalies • Understandable behavior with internal structure of each instance • Long-term behavior • Specific to one host or a small subset
• Events, where the maliciousness can only be decided on the level of intent, as the identical events would appear in malicious and legitimate scenarios alike. • Features necessary for separation are not not be accessible (judging the HTTPS by the endpoint)
or the behaviors are inherently inseparable. • Direct effect is the same in case of attack or legitimate activity. Intent can be dramatically different.
• Example: • Large file download/upload • Long-lasting HTTPS connection to a single host outside the company network
![Page 37: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/37.jpg)
Cisco Confidential 37 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 38: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/38.jpg)
Cisco Confidential 38 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Malicious Behavior
• Major characteristics are close to Structural Anomalies or Dual Behaviors • Understandable behavior with internal structure of each instance • Long-term behavior • Specific to one host or a small subset
• Our experience, correlated evidence or the behavior itself allow us to conclusively confirm the behavior as malicious upon examination.
• Prioritization within the malware category is not trivial!
![Page 39: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/39.jpg)
Cisco Confidential 39 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 40: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/40.jpg)
Cisco Confidential 40 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Unstructured Noise Structured Anomaly Dual Behavior
§ Caused by inherent unpredictability & randomness of the modeled system, computational limitations imposed on anomaly detectors and predictive precision of the anomaly detection algorithms.
§ Can be reduced, but never fully eliminated.
§ The accessible features do not allow separation between the legitimate and malicious behavior of this class without a qualitatively different model.
§ Opens exciting new research areas: § Intent modeling § Strategic correlation § Plan recognition § …
§ Caused by the limited capability of anomaly detectors that are unable to model whole classes of behavior. Capability can be either a modeling problem (such as wrong timescale), or a feature selection problem.
§ Separable, but not separated…
![Page 41: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/41.jpg)
Cisco Confidential 41 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Assumptions for Ensemble Approach Model Diversity: We assume that the methods and features used by the same
model ensure that the fact that one model has singled out a legitimate object as a false positive is not correlated with other models doing so
Model Alignment: We assume that malicious behavior detection is correlated between the models that we combine
![Page 42: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/42.jpg)
Cisco Confidential 42 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
And should the assumptions hold…
Averaging the outputs of several AD’s should allow you to dramatically increase the precision of the system without the negative impact on recall But the assumptions hold only partially…
![Page 43: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/43.jpg)
Cisco Confidential 43 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Limitations
Question: Actually, as system designers, we ought to be able to design the detectors with enough diversity by careful detection of features and AD methods – right?
Short answer: Sort of. Assuming the attackers are nice/stupid enough, or the AD methods are almost flawless in terms of generality.
Better answer: • Diversity assumptions are relatively easy to satisfy and verify, as they rely on the background
traffic characteristics. They also tend to carry over between the contexts (networks) reasonably well.
• General Alignment is far more difficult to guarantee due to the diversity of malicious behaviors. In reality, we don’t average all the detectors, but select subsets of 4-10 detectors to optimize the recall.
![Page 44: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/44.jpg)
• M Rehák, E Staab, V Fusenig, M Pěchouček, M Grill, J Stiborek, K Bartoš, Thomas Engel. Runtime monitoring and dynamic reconfiguration for intrusion detection systems, RAID 2008 - Recent Advances in Intrusion Detection
• J Stiborek, M Grill, M Rehak, K Bartos, J Jusko. Game Theoretical Adaptation Model for Intrusion Detection System, PAAMS 2012
![Page 45: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/45.jpg)
Cisco Confidential 45 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
(Slides removed)
![Page 46: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/46.jpg)
Cisco Confidential 46 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
User-Level Perception of the System
• Precision alone defines the user experience & operations • Government – more recall sensitive in top security areas • Multinational corporation – precision determines the real usability • Smaller enterprises and companies – even higher precision requirements
• Recall is (almost) always argued on the “compared to the state without the system X deployed” basis
• Improving recall is our professional duty, but the business rationale and user perception is less clear than in case of precision • But there is a catch…precision and recall are directly related through the user bottleneck
![Page 47: Modern Security Analytics Finding a Needle in the Hayblower · & Memory Anomalous Malicious Malware ... workshop on information forensics and security, 2012, 1-6 Graph Theory:](https://reader033.vdocuments.net/reader033/viewer/2022042303/5ece768cafcfdc39ce5e0fa2/html5/thumbnails/47.jpg)