can the best defense be to attack? - carleton university · pdf filecan the best defense be to...

Can the Best Defense beto Attack?

MITACS Digital Security SeminarSeries at Carleton University

Presenter:Presenter:

Dr.Dr. Nur Zincir Nur Zincir-Heywood-HeywoodDalhousie University,Dalhousie University,

Faculty of Computer ScienceFaculty of Computer Science

MITACS Seminar, 2007 Zincir-Heywood 2

Arms RaceArms Race

Security engineers Security engineers vs vs black hat attackersblack hat attackers

Attackers to evade defense systemsAttackers to evade defense systems

Security research/technology to patch/newSecurity research/technology to patch/newsystemssystems

Even in research environmentsEven in research environments Defender systemsDefender systems

White hat attackers (vulnerability testing andWhite hat attackers (vulnerability testing andevasion)evasion)


Bad guysBad guys

IntelligentIntelligent

Malicious intentionsMalicious intentions

Automated toolsAutomated tools

BotnetsBotnets


Good guysGood guys

Confidentiality, integrity, authenticationConfidentiality, integrity, authentication cryptographycryptography

PoliciesPolicies

Access ControlAccess Control FirewallsFirewalls

Virus checkersVirus checkers

Worm detectorsWorm detectors

……


How do we know we areHow do we know we areattacked?attacked?

Alerts from security tools usedAlerts from security tools used Usually signature basedUsually signature based Poor in new attacksPoor in new attacks Low FP, butLow FP, but……

Monitoring trafficMonitoring traffic Expert(s) performExpert(s) perform

forensic analysis after the eventforensic analysis after the event Deep packet inspectionDeep packet inspection

Results in patches and new releasesResults in patches and new releases


First egg or chicken?First egg or chicken?

Do we need to be attacked first toDo we need to be attacked first tounderstand that there is a new attack?understand that there is a new attack?

What aboutWhat about Penetration and Vulnerability checkingPenetration and Vulnerability checking

Blind-spot analysis (evasion)Blind-spot analysis (evasion)


Mimicry AttacksMimicry Attacks

Assume a core attackAssume a core attack

Modify it to look different but actually doesModify it to look different but actually doesthe same/similar damagethe same/similar damage Hiding in normal behaviorHiding in normal behavior

Hiding in blind-spot of the detectorHiding in blind-spot of the detector

Hiding it in a less harmful attackHiding it in a less harmful attack


Two Sides of the Arms RaceTwo Sides of the Arms Race

DefenderDefender Signature based Signature based IDSsIDSs

Anomaly based Anomaly based IDSsIDSs

AttackerAttacker Mimicry attacksMimicry attacks

Evasion attacksEvasion attacks


DefenderDefender

Anomaly basedAnomaly based StideStide

Improved versionsImproved versions

Signature basedSignature based Snort (Snort (Mutz Mutz et al, et al, ‘‘03; 03; Vigna Vigna et al, et al, ‘‘04; Kayacik04; Kayacik

et al, et al, ‘‘05)05)

ISS ISS RealSecure RealSecure ((Mutz Mutz et al, et al, ‘‘03)03)

Symantec Net prowler (Symantec Net prowler (Vigna Vigna et al, et al, ‘‘04)04)


Anomaly Based Detectors:Anomaly Based Detectors:Black BoxBlack Box

Host basedHost based Three categories:Three categories:

Black box, gray box and white boxBlack box, gray box and white box

Black-box TechniquesBlack-box Techniques Extract info only from system callsExtract info only from system calls Fixed length - Fixed length - Stide Stide (Forrest et al, (Forrest et al, ‘‘96)96) Alternative data models (Alternative data models (Warrender Warrender et al, et al, ‘‘99)99) Variable length (Variable length (Wespi Wespi et al, et al, ‘‘00)00)


Anomaly Based DetectorsAnomaly Based Detectors::Gray-BoxGray-Box

Extract info from system calls + run-time processExtract info from system calls + run-time process’’execution stateexecution state

Utilize a Finite State Automata (FSA) toUtilize a Finite State Automata (FSA) tocharacterize normal behavior (characterize normal behavior (Sekar Sekar et al, et al, ‘‘01)01)

Instead of FSA utilize a Instead of FSA utilize a ““virtual pathvirtual path”” table table details not only where the system call is executeddetails not only where the system call is executed

from, but also describes the point where the executionfrom, but also describes the point where the executionis returned (is returned (Feng Feng et al, et al, ‘‘03)03)

Generate an execution graph to understand theGenerate an execution graph to understand themax subset of program control flow graph (max subset of program control flow graph (GaoGaoet al, et al, ‘‘04)04)


Anomaly Based DetectorsAnomaly Based Detectors::White-boxWhite-box

Extract info from the monitored programExtract info from the monitored program Static analysis of source code or binary imageStatic analysis of source code or binary image

System calls represented by a state machineSystem calls represented by a state machine extracted from control-flow graph (Wagner et al, extracted from control-flow graph (Wagner et al, ‘‘01; 01; GiffinGiffin

et al, et al, ‘‘02 and 02 and ‘‘04)04) System call System call inlining inlining and notify calls are introduced (Lam etand notify calls are introduced (Lam et

al, al, ‘‘04)04)

Static analysis to extract an automaton with callStatic analysis to extract an automaton with callstack information is introduced (stack information is introduced (Feng Feng et al, et al, ‘‘04)04)

multiple detection models are applied to system callmultiple detection models are applied to system callargumentsarguments overall aggregate score of these models is introduced overall aggregate score of these models is introduced

((Mutz Mutz et al, et al, ‘‘06)06)


Attacker:Attacker:Mimicry AttacksMimicry Attacks

Modify system call sequence of an exploit renderingModify system call sequence of an exploit renderingit undetectable to a specific IDS (Wagner et al, it undetectable to a specific IDS (Wagner et al, ‘‘02) -02) -wbwb, manual, manual Similar approach based on modifying the exploit code (TanSimilar approach based on modifying the exploit code (Tan

et al, et al, ‘‘02) - 02) - wbwb, manual, manual

Generate variations of signatures to test the qualityGenerate variations of signatures to test the qualityof detection against Snort, ISS of detection against Snort, ISS RealSecureRealSecure,,Symantec Net Prowler (Symantec Net Prowler (Mutz Mutz et al, et al, ‘‘03; Vigna et al,03; Vigna et al,‘‘04) - 04) - bb (!), automaticbb (!), automatic

Generate attack against gray-box detectors (Generate attack against gray-box detectors (Gao Gao etetal, al, ‘‘04) - 04) - gbgb, manual, manual

Evolve mimicry attack against Snort (Evolve mimicry attack against Snort (Kayacik Kayacik et al,et al,‘‘05) - 05) - bb, automaticbb, automatic


Mimicry Attacks:Mimicry Attacks:Kruegel Kruegel et al, et al, ‘‘0505

Automatic attack generationAutomatic attack generation White box testingWhite box testing Assumes vulnerable application is knownAssumes vulnerable application is known Assumes a core attack is knownAssumes a core attack is known Against a gray-box (Against a gray-box (Sekar Sekar et al, et al, ‘‘01) and a white-box detector01) and a white-box detector

((Feng Feng et al, et al, ‘‘04)04) Utilize the details of how the detector works in attackUtilize the details of how the detector works in attack

generationgeneration Statically analyze victim x86 binariesStatically analyze victim x86 binaries Approach employs symbolic executionApproach employs symbolic execution Objective is to identify code pointers that can be modified toObjective is to identify code pointers that can be modified to

point to the attacker codepoint to the attacker code Tested on 3 sample programs protected by the above Tested on 3 sample programs protected by the above IDSsIDSs


Symbolic executionSymbolic execution

((Kruegel Kruegel et al, et al, ‘‘05)05)


Deriving an AppropriateDeriving an AppropriateConfigurationConfiguration



Results for Real WorldResults for Real WorldApplicationsApplications



Execution Steps and TimeExecution Steps and Time



Design RequirementsDesign Requirements Developing a static analysis tool for each binary systemDeveloping a static analysis tool for each binary system White box testing approachWhite box testing approach Expensive (knowledge)Expensive (knowledge) Limited semantic coverageLimited semantic coverage Exhaustive search (constrained by the above item) usingExhaustive search (constrained by the above item) using

symbolic executionsymbolic execution Solving a linear constraint can be exponential in theSolving a linear constraint can be exponential in the

number of inequalitiesnumber of inequalities Assumes that each symbolic expression refer to differentAssumes that each symbolic expression refer to different

memory locationmemory location Not all symbolic expressions can be resolved (see theNot all symbolic expressions can be resolved (see the

above item)above item)


Mimicry Attacks:Mimicry Attacks:Giffin Giffin et al, et al, ‘‘0606

Automatic attack generationAutomatic attack generation White box testingWhite box testing Assumes vulnerable application is knownAssumes vulnerable application is known Assumes a core attack is knownAssumes a core attack is known Develop a model of OS Develop a model of OS wrt wrt security critical statesecurity critical state Manually construct the OS modelManually construct the OS model Manually construct the malicious OS stateManually construct the malicious OS state Apply model checking to prove that no reachable OSApply model checking to prove that no reachable OS

configuration corresponds to the effect of an attackconfiguration corresponds to the effect of an attack Test it againstTest it against Stide Stide IDS using IDS using wuwu--ftpdftpd, restore,, restore,

traceroutetraceroute, , passwd passwd applicationsapplications


OS ModelOS Model

Manually identify what OS state variablesManually identify what OS state variablesconstitute security relevant statesconstitute security relevant states

Initial assignment of values to OS stateInitial assignment of values to OS statevariables encode the OS statevariables encode the OS stateconfiguration before execution of aconfiguration before execution of aprocessprocess

For each system call a relation is providedFor each system call a relation is providedfor how it changes state based upon thefor how it changes state based upon theprevious state (pre- and post- conditions)previous state (pre- and post- conditions)


ArchitectureArchitecture

((Giffin Giffin et al, et al, ‘‘06)06)


Describing Describing Stide Stide Model forModel forEach ApplicationEach Application



Evaluation of the Evaluation of the Stide Stide ModelModelto Detect Attacksto Detect Attacks


Yes - indicates Detected, No - indicates Undetected


Model Checking RunningModel Checking RunningTimesTimes



Design RequirementsDesign Requirements

Developing OS model manuallyDeveloping OS model manually White box testing approachWhite box testing approach Expensive (knowledge)Expensive (knowledge) Limited semantic coverageLimited semantic coverage Exhaustive search (constrained by the aboveExhaustive search (constrained by the above

item) using model checkingitem) using model checking What if the OS model abstraction is wrong?What if the OS model abstraction is wrong?


Mimicry Attacks:Mimicry Attacks:Kayacik Kayacik et al, et al, ‘‘07a07a

Automatic attack generationAutomatic attack generation

Black box testingBlack box testing

Assumes vulnerable application is knownAssumes vulnerable application is known

Assumes a core attack is knownAssumes a core attack is known

Against Against StideStide, using, using traceroutetraceroute applicationapplication

Search space too large to deploy exhaustive methodsSearch space too large to deploy exhaustive methods Genetic Programming employedGenetic Programming employed


MethodologyMethodology

Motivations for using GPMotivations for using GP Goal based objectivesGoal based objectives

RepresentationRepresentation

Intron Intron CodeCode

Training data for Training data for traceroutetraceroute Previous work employ:Previous work employ:

Traceroute nisTraceroute nis..nsfnsf.net.net

Fitness function for GPFitness function for GP


Occurrence of System CallsOccurrence of System Calls

((Kayacik Kayacik et al, et al, ‘‘07a)07a)


Parameter TypesParameter Types



Fitness FunctionFitness Function



Training DataTraining Data



Stide Stide Anomaly Rates againstAnomaly Rates againstTraining DataTraining Data



Stide Stide Anomaly Rates againstAnomaly Rates againstExploitsExploits



Mismatch rates (%) ReportedMismatch rates (%) Reportedby by StideStide

((Kayacik Kayacik et al, et al, ‘‘07b)07b)


Contribution of PreambleContribution of Preamble

((Kayacik Kayacik et al, et al, ‘‘07b)07b)


Design RequirementsDesign Requirements

Attack = preamble + exploitAttack = preamble + exploit Anomaly rate should be calculated over bothAnomaly rate should be calculated over both

There is no attack with 0% anomaly even for theThere is no attack with 0% anomaly even for theprevious work when we analyze the whole attackprevious work when we analyze the whole attack

Can work with any IDS - bbCan work with any IDS - bb

Evolutionary computationEvolutionary computation Efficient sampling of large search spaceEfficient sampling of large search space

Longer training timesLonger training times


Van Van Oorschot Oorschot et al, et al, ‘‘0505

Hardware assisted circumvention of self-Hardware assisted circumvention of self-hashing software tamper resistancehashing software tamper resistance

Attack generation against self-hashingAttack generation against self-hashingtechnique on many modern processorstechnique on many modern processors(x86, (x86, UltraSparcUltraSparc, AMD64, ARM , AMD64, ARM ……))

White-boxWhite-box Assumes vulnerable application is knownAssumes vulnerable application is known

Manual generationManual generation


WhatWhat’’s next?s next?

White box White box vs vs black box testingblack box testing Preamble Preamble vs vs exploit generationexploit generation Dynamic Dynamic vs vs staticstatic Deterministic Deterministic vs vs stochasticstochastic Allergy attacksAllergy attacks Co-evolution of attackers & detectorsCo-evolution of attackers & detectors Theoretical modeling of the arms raceTheoretical modeling of the arms race Experimental results to explore the modelsExperimental results to explore the models


Why bother?Why bother?

To be able to predictTo be able to predict

To be a step a head if possibleTo be a step a head if possible

To understand attacker behaviorTo understand attacker behavior

To test defense systems before attackersTo test defense systems before attackers

To improve defense systemsTo improve defense systems Automatic signature generationAutomatic signature generation

Automatic attack training data generationAutomatic attack training data generation

To generate anti-To generate anti-botnet botnet teams :-)teams :-)


One final thoughtOne final thought

When asked When asked Vint Cerf Vint Cerf told that there are 2told that there are 2important events that started Internetimportant events that started Internet’’ssevolution:evolution: Launch of SputnikLaunch of Sputnik

Breakup of AT&TBreakup of AT&T

What about security:What about security: Bombing of 9/11Bombing of 9/11

????


ReferencesReferences Mutz Mutz D.,D., Vigna Vigna G., Kemmerer R., An Experience Developing an IDS Stimulator for the Black-Box TestingG., Kemmerer R., An Experience Developing an IDS Stimulator for the Black-Box Testing

of Network Intrusion Detection Systems, ACSAC, 2003.of Network Intrusion Detection Systems, ACSAC, 2003. VignaVigna, G., Robertson, W.,, G., Robertson, W., Balzarotti Balzarotti D., Testing Network Based Intrusion Detection Signatures UsingD., Testing Network Based Intrusion Detection Signatures Using

Mutant Exploits, ACM CCS, 2004.Mutant Exploits, ACM CCS, 2004. Kayacik Kayacik H. G.,H. G., Zincir Zincir-Heywood A. N., Heywood M. I., Evolving Successful Stack Overflow Attacks for-Heywood A. N., Heywood M. I., Evolving Successful Stack Overflow Attacks for

Vulnerability Testing, ACSAC, 2005.Vulnerability Testing, ACSAC, 2005. Forrest S.,Forrest S., Hofmeyr Hofmeyr S. A.,S. A., Somayaji Somayaji A.,A., Longstaff Longstaff T. A., A sense of self for Unix processes, IEEE SP,T. A., A sense of self for Unix processes, IEEE SP,

1996.1996. Warrender Warrender C., Forrest S., C., Forrest S., Pearlmutter Pearlmutter BA, Detecting intrusions using system calls: Alternative data models,BA, Detecting intrusions using system calls: Alternative data models,

IEEE SP, 1999.IEEE SP, 1999. WespiWespi, A.,, A., Dacier Dacier, M., and Debar, H., Intrusion Detection Using Variable-Length Audit Trail Patterns,, M., and Debar, H., Intrusion Detection Using Variable-Length Audit Trail Patterns,

RAID, 2000.RAID, 2000. Sekar Sekar R.,R., Bendre Bendre M.,M., Dhurjati Dhurjati D.,D., Bollineni Bollineni P., A Fast Automation-based Method for Detecting AnomalousP., A Fast Automation-based Method for Detecting Anomalous

Program Behavior, IEEE SP, 2001.Program Behavior, IEEE SP, 2001. Feng Feng H.,H., Kolesnikov Kolesnikov O.,O., Fogla Fogla P., Lee W., Gong W., Anomaly detection using call stack information, IEEEP., Lee W., Gong W., Anomaly detection using call stack information, IEEE

SP, 2003.SP, 2003. Gao Gao D., Reiter M., Song D., Gray box extraction of execution graphs for anomaly detection, ACM CCS,D., Reiter M., Song D., Gray box extraction of execution graphs for anomaly detection, ACM CCS,

2004.2004. Wagner D., Dean D., Intrusion detection via static analysis, IEEE SP, 2001.Wagner D., Dean D., Intrusion detection via static analysis, IEEE SP, 2001. J.J. Giffin Giffin, S., S. Jha Jha, and B. Miller. Detecting Manipulated Remote Call Streams,, and B. Miller. Detecting Manipulated Remote Call Streams, Usenix Usenix Security, 2002.Security, 2002.


ReferencesReferences J.J. Giffin Giffin, S., S. Jha Jha, and B.P. Miller. Efficient context sensitive intrusion detection, NDSS, 2004., and B.P. Miller. Efficient context sensitive intrusion detection, NDSS, 2004. H.H. Feng Feng, J., J. Giffin Giffin, Y. Huang, S., Y. Huang, S. Jha Jha, W. Lee, B. Miller. Formalizing sensitivity in static analysis for, W. Lee, B. Miller. Formalizing sensitivity in static analysis for

intrusion detection, IEEE SP, 2004.intrusion detection, IEEE SP, 2004. L. Lam and T.L. Lam and T. Chiueh Chiueh. Automatic Extraction of Accurate Application-Specific Sandboxing Policy, RAID,. Automatic Extraction of Accurate Application-Specific Sandboxing Policy, RAID,

2004.2004. Mutz Mutz D.,D., Valeur Valeur F.,F., Vigna Vigna G.,G., Kruegel Kruegel C., Anomalous System Call Detection, ACM Transactions onC., Anomalous System Call Detection, ACM Transactions on

Information system and Security, 2006.Information system and Security, 2006. Wagner D., Soto P., Mimicry attacks on host based intrusion detection systems, ACM CCS, 2002.Wagner D., Soto P., Mimicry attacks on host based intrusion detection systems, ACM CCS, 2002. Tan, K. M. C.,Tan, K. M. C., Killourhy Killourhy, K. S.,, K. S., Maxion Maxion, R. A., Undermining an Anomaly-based Intrusion Detection System using Common, R. A., Undermining an Anomaly-based Intrusion Detection System using Common

Exploits, RAID, 2002.Exploits, RAID, 2002. D.D. Gao Gao, M. Reiter, and D. Song. On Gray-Box Program Tracking for Anomaly Detection,, M. Reiter, and D. Song. On Gray-Box Program Tracking for Anomaly Detection, Usenix Usenix Security, 2004.Security, 2004. Kruegel Kruegel C.,C., Kirda Kirda E.,E., Mutz Mutz D., Robertson W.,D., Robertson W., Vigna Vigna G., Automating mimicry attacks using static binary analysis, USENIXG., Automating mimicry attacks using static binary analysis, USENIX

Security Symposium, 2005.Security Symposium, 2005. Giffin Giffin J. T., J. T., Jha Jha S., Miller BP, S., Miller BP, Autoated Autoated Discovery of Mimicry Attacks, RAID, 2006.Discovery of Mimicry Attacks, RAID, 2006. Kayacik Kayacik HG, HG, ZincirZincir-Heywood AN, Heywood MI, Automatically Evading IDS Using GP Authored Attacks, IEEE CISDA,-Heywood AN, Heywood MI, Automatically Evading IDS Using GP Authored Attacks, IEEE CISDA,

2007a.2007a. Kayacik Kayacik HG, HG, ZincirZincir-Heywood AN, On the Contribution of Preamble to Information Hiding in Mimicry Attacks, IEEE-Heywood AN, On the Contribution of Preamble to Information Hiding in Mimicry Attacks, IEEE

SSNDS, 2007b.SSNDS, 2007b. Van Van Oorschot Oorschot PC, PC, Somayaji Somayaji A., A., Wurster Wurster G., Hardware Assisted circumvention of self hashing software tamper resistance,G., Hardware Assisted circumvention of self hashing software tamper resistance,

IEEE Transactions on Dependable and Secure Computing, 2005.IEEE Transactions on Dependable and Secure Computing, 2005.


THANKS A LOT!THANKS A LOT!

ANY QUESTIONS?ANY QUESTIONS?

COMMENTS?COMMENTS?

can the best defense be to attack? - carleton university · pdf filecan the best defense be to...

Documents