anti-re techniques in drm code - def con techniques in drm code ... outline 1 introduction legal...

73
Introduction SEH Basics Anti-RE Techniques Decrypting the Content Anti-RE Techniques in DRM Code Jan Newger Seminar on Advanced Exploitation Techniques i4, RWTH Aachen, Germany DEF CON 16 / 2008 Jan Newger Anti-RE Techniques in DRM Code

Upload: vuonghanh

Post on 17-May-2018

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Anti-RE Techniques in DRM Code

Jan Newger

Seminar on Advanced Exploitation Techniquesi4, RWTH Aachen, Germany

DEF CON 16 / 2008

Jan Newger Anti-RE Techniques in DRM Code

Page 2: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Outline1 Introduction

Legal IssuesAbout DRMsApproaching the DRM

2 SEH BasicsOverviewException Dispatching

3 Anti-RE TechniquesOverviewTrampolinesDebug RegistersP-Code Machine

4 Decrypting the ContentThe AlgorithmDemo

Jan Newger Anti-RE Techniques in DRM Code

Page 3: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Outline1 Introduction

Legal IssuesAbout DRMsApproaching the DRM

2 SEH BasicsOverviewException Dispatching

3 Anti-RE TechniquesOverviewTrampolinesDebug RegistersP-Code Machine

4 Decrypting the ContentThe AlgorithmDemo

Jan Newger Anti-RE Techniques in DRM Code

Page 4: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Issues with this Talk

Legal IssuesLegal issues with publishing DRM researchProbably illegal in most countries, legal uncertainty

EFF to the Rescue!Electronic Frontier Foundation (EFF)[1]Non-profit organization dedicated to preserving freespeech rightsDiscussed solution with Jennifer Granick from EFF (thxJennifer!)Loophole in DMCA -> "Encryption Research"[2]But still too dangerous for me

Jan Newger Anti-RE Techniques in DRM Code

Page 5: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Issues with this Talk

Legal IssuesLegal issues with publishing DRM researchProbably illegal in most countries, legal uncertainty

EFF to the Rescue!Electronic Frontier Foundation (EFF)[1]Non-profit organization dedicated to preserving freespeech rightsDiscussed solution with Jennifer Granick from EFF (thxJennifer!)Loophole in DMCA -> "Encryption Research"[2]But still too dangerous for me

Jan Newger Anti-RE Techniques in DRM Code

Page 6: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Issues with this Talk (2)

ConsequenceStrip details about key setup and decryption algorithmDon’t reveal identity of the DRM

What it ISShow some not-so-common Anti-RE techniquesStrategies to defeat Anti-RE

What it is NOTHow to hack the DRM from *********No tutorial for writing decryption tools

Jan Newger Anti-RE Techniques in DRM Code

Page 7: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Issues with this Talk (2)

ConsequenceStrip details about key setup and decryption algorithmDon’t reveal identity of the DRM

What it ISShow some not-so-common Anti-RE techniquesStrategies to defeat Anti-RE

What it is NOTHow to hack the DRM from *********No tutorial for writing decryption tools

Jan Newger Anti-RE Techniques in DRM Code

Page 8: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Issues with this Talk (2)

ConsequenceStrip details about key setup and decryption algorithmDon’t reveal identity of the DRM

What it ISShow some not-so-common Anti-RE techniquesStrategies to defeat Anti-RE

What it is NOTHow to hack the DRM from *********No tutorial for writing decryption tools

Jan Newger Anti-RE Techniques in DRM Code

Page 9: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

What’s a DRM?

"Digital Rights Management"Restrict access to contentContent encryptedDecrypt online

Key often bound to user/hardwarePrevents copyingChange hardware -> new license

Media key, hardware key, player key, content key...Obviously: every DRM can be broken

Jan Newger Anti-RE Techniques in DRM Code

Page 10: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

What’s a DRM?

"Digital Rights Management"Restrict access to contentContent encryptedDecrypt online

Key often bound to user/hardwarePrevents copyingChange hardware -> new license

Media key, hardware key, player key, content key...Obviously: every DRM can be broken

Jan Newger Anti-RE Techniques in DRM Code

Page 11: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

What’s a DRM?

"Digital Rights Management"Restrict access to contentContent encryptedDecrypt online

Key often bound to user/hardwarePrevents copyingChange hardware -> new license

Media key, hardware key, player key, content key...

Obviously: every DRM can be broken

Jan Newger Anti-RE Techniques in DRM Code

Page 12: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

What’s a DRM?

"Digital Rights Management"Restrict access to contentContent encryptedDecrypt online

Key often bound to user/hardwarePrevents copyingChange hardware -> new license

Media key, hardware key, player key, content key...Obviously: every DRM can be broken

Jan Newger Anti-RE Techniques in DRM Code

Page 13: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Possible Strategies (1)

Ultimate GoalFind code for content decryption and the associated key setup

Obvious Approach1 BPs on file I/O APIs (CreateFile, ReadFile, MMF)2 Set BPM on filebuffer

either stops on copy operationor breaks on decryption

Obvious approach impossible!DRM System prevents this strategy by blocking the debugregisters!

Jan Newger Anti-RE Techniques in DRM Code

Page 14: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Possible Strategies (1)

Ultimate GoalFind code for content decryption and the associated key setup

Obvious Approach1 BPs on file I/O APIs (CreateFile, ReadFile, MMF)2 Set BPM on filebuffer

either stops on copy operationor breaks on decryption

Obvious approach impossible!DRM System prevents this strategy by blocking the debugregisters!

Jan Newger Anti-RE Techniques in DRM Code

Page 15: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Possible Strategies (2)

Code CoverageRuntime analysis to spot relevant code by recording executionof basic blocks / functions

Code Coverage Limitation

Here: Impossible to find DRM code itself using codecoverage!Gives some good starting points, though

Jan Newger Anti-RE Techniques in DRM Code

Page 16: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

Legal IssuesAbout DRMsApproaching the DRM

Basic Approach Summary

Our StrategyUse code coverage to spot some places to investigateUse obvious approach to find decryption code

Jan Newger Anti-RE Techniques in DRM Code

Page 17: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

Outline1 Introduction

Legal IssuesAbout DRMsApproaching the DRM

2 SEH BasicsOverviewException Dispatching

3 Anti-RE TechniquesOverviewTrampolinesDebug RegistersP-Code Machine

4 Decrypting the ContentThe AlgorithmDemo

Jan Newger Anti-RE Techniques in DRM Code

Page 18: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

Windows SEH

Structured Exception HandlingDispatch exception on a per-thread-basisLinked list of handlers starting at fs:[0]On exception OS walks list of faulting threadWhen called, a handler can:

Handle exception and ask OS to continue executionRefuse to handle exception

Jan Newger Anti-RE Techniques in DRM Code

Page 19: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler

EXCEPTION_DISPOSITION _except_handler(_EXCEPTION_RECORD* ExceptionRecord,void* EstablisherFrame,_CONTEXT* ContextRecord,void* DispatcherContext);

Jan Newger Anti-RE Techniques in DRM Code

Page 20: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler Invocation

Jan Newger Anti-RE Techniques in DRM Code

Page 21: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler Invocation

Jan Newger Anti-RE Techniques in DRM Code

Page 22: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler Invocation

Jan Newger Anti-RE Techniques in DRM Code

Page 23: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler Invocation

Jan Newger Anti-RE Techniques in DRM Code

Page 24: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

SEH Handler Invocation

Jan Newger Anti-RE Techniques in DRM Code

Page 25: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewException Dispatching

Remarks

SEH Handler InvocationSimplified view, because

No stack unwindingNo collided unwind, nested exceptionsHandler can decide not to return (C++, try...except)

But good enough for our analysis!

Jan Newger Anti-RE Techniques in DRM Code

Page 26: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Outline1 Introduction

Legal IssuesAbout DRMsApproaching the DRM

2 SEH BasicsOverviewException Dispatching

3 Anti-RE TechniquesOverviewTrampolinesDebug RegistersP-Code Machine

4 Decrypting the ContentThe AlgorithmDemo

Jan Newger Anti-RE Techniques in DRM Code

Page 27: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

The DRM Protection (1)

Control Flow ObfuscationUse fake exceptions to interrupt control flowHandlers change thread contextInter-/intra-modular calls through call tablesUse dynamically allocated trampolinesP-Code machine

Jan Newger Anti-RE Techniques in DRM Code

Page 28: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

The DRM Protection (2)

Anti-Debugging

Check PEB flagScan APIs for 0xCCUsage of debug registers (no BPM/BPX)Special files contain code uncompressed at runtimeUse fake exceptions to detect debugger

Jan Newger Anti-RE Techniques in DRM Code

Page 29: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampolines Overview

Trampoline DefinitionCopy code at runtime to randomized location (RDTSC),execute from there

Trampoline Execution1 Change control flow via fake exceptions (single step

exception)2 Exception handler modifies EIP based on debug register

values3 Execution resumes at next trampoline

Jan Newger Anti-RE Techniques in DRM Code

Page 30: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Trampoline Control FlowTrampoline A transfers control flow to trampoline BControl flow entirely depends on jumps and exceptionsNo call or ret instructions, no direct control flowbetween trampolinesTherefore, a call hierarchy emulation is implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 31: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Trampoline0TrampolineA copies trampoline0 and jumps to itTrampoline0 manages internal call hierarchyPut destination trampoline on stackCopies next trampoline to random location

Jan Newger Anti-RE Techniques in DRM Code

Page 32: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Trampoline1Copy trampoline0 to random locationInstall SEH frame and trigger singlestep exception

Trigger Exceptionpushfpop eaxor eax, 100hpush eaxpopf

Jan Newger Anti-RE Techniques in DRM Code

Page 33: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Trampoline1Copy trampoline0 to random locationInstall SEH frame and trigger singlestep exception

Trigger Exceptionpushfpop eaxor eax, 100hpush eaxpopf

Jan Newger Anti-RE Techniques in DRM Code

Page 34: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Exception HandlerChanges EIP based on debug register valuesClear TF bit, remove SEH frame, clean stack

Jan Newger Anti-RE Techniques in DRM Code

Page 35: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Trampoline2Copy destination trampolineJump to destination trampoline

Jan Newger Anti-RE Techniques in DRM Code

Page 36: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Trampoline Details

Call Stack EmulationThe ret instruction is emulated by a similar mechanism!

Special exception handler removes trampoline frominternal call stackModifies context, execution resumes

Jan Newger Anti-RE Techniques in DRM Code

Page 37: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

More Trampoline Details

Use of the Debug RegistersDR0 and DR6 are zeroed outDR1 contains pointer to a shared stack area to pass databetween trampolinesDR2 holds trampoline address, which is used to performreturn emulationDR3 holds the address of the starting trampoline(trampoline0)DR7 is used to turn hardware breakpoints on and veryfrequently

Jan Newger Anti-RE Techniques in DRM Code

Page 38: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Impact of Trampolines

Impact on REDebugging pretty annoying, trampoline addresses jitterControl flow depends on DRs, so no BPM/BPXNo call stack, i.e. back tracing difficultWe can´t execute until return, difficult to tell who called usNo direct call between subs, less X-RefsAbsence of ret instructions confuses disassembler

But: Once understood we get perfect call stack

Jan Newger Anti-RE Techniques in DRM Code

Page 39: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Impact of Trampolines

Impact on REDebugging pretty annoying, trampoline addresses jitterControl flow depends on DRs, so no BPM/BPXNo call stack, i.e. back tracing difficultWe can´t execute until return, difficult to tell who called usNo direct call between subs, less X-RefsAbsence of ret instructions confuses disassemblerBut: Once understood we get perfect call stack

Jan Newger Anti-RE Techniques in DRM Code

Page 40: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Ease Impact of Trampolines

IdeaFix trampoline addressesUse kernel mode driver

Driver1 Turn RDTSC into privileged instruction (TSD flag, CR4

register)2 Hook IDT3 Return zero upon exception if

Exception from user modeInstruction was RDTSC

else jump to original handler

Jan Newger Anti-RE Techniques in DRM Code

Page 41: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Ease Impact of Trampolines

IdeaFix trampoline addressesUse kernel mode driver

Driver1 Turn RDTSC into privileged instruction (TSD flag, CR4

register)2 Hook IDT3 Return zero upon exception if

Exception from user modeInstruction was RDTSC

else jump to original handler

Jan Newger Anti-RE Techniques in DRM Code

Page 42: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Reclaiming the Debug Registers (1)

Usage of DRs

DRM system uses DRs for storageUses SetThreadContext APIDebugger cannot use hardware breakpoints (crash or nobreak)

Strategy

But we need BPMs for our strategy!

Jan Newger Anti-RE Techniques in DRM Code

Page 43: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Reclaiming the Debug Registers (1)

Usage of DRs

DRM system uses DRs for storageUses SetThreadContext APIDebugger cannot use hardware breakpoints (crash or nobreak)

Strategy

But we need BPMs for our strategy!

Jan Newger Anti-RE Techniques in DRM Code

Page 44: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Reclaiming the Debug Registers (2)

Use API Hooking

Hook into Set/GetThreadContext APIRedirect modifications to internal storageDRM System cannot modify DRs anymore!Debugger can use DRs

Really?Hardware breakpoints still don’t work!Why?

Jan Newger Anti-RE Techniques in DRM Code

Page 45: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Reclaiming the Debug Registers (2)

Use API Hooking

Hook into Set/GetThreadContext APIRedirect modifications to internal storageDRM System cannot modify DRs anymore!Debugger can use DRs

Really?Hardware breakpoints still don’t work!Why?

Jan Newger Anti-RE Techniques in DRM Code

Page 46: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Context Emulation

ProblemModification of EIP depends on DRsTwo thread contexts: kernel mode vs. internal storage

Hook KiUserExceptionDispatcherIf fake exception, execute re-implementedKiUserExceptionDispatcher:

1 Pass fake context, DR values from internal storage2 On return copy modifications to real context3 Apply context via NtContinue

Jan Newger Anti-RE Techniques in DRM Code

Page 47: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Context Emulation

ProblemModification of EIP depends on DRsTwo thread contexts: kernel mode vs. internal storage

Hook KiUserExceptionDispatcherIf fake exception, execute re-implementedKiUserExceptionDispatcher:

1 Pass fake context, DR values from internal storage2 On return copy modifications to real context3 Apply context via NtContinue

Jan Newger Anti-RE Techniques in DRM Code

Page 48: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

KiUserExceptionDispatcher - Re-implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 49: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

KiUserExceptionDispatcher - Re-implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 50: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

KiUserExceptionDispatcher - Re-implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 51: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

KiUserExceptionDispatcher - Re-implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 52: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

KiUserExceptionDispatcher - Re-implemented

Jan Newger Anti-RE Techniques in DRM Code

Page 53: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Summary

Situation after CountermeasuresDRM System cannot modify DRs - API hookException handler gets expected values -KiUserExceptionDispatcher patchOur debugger can use hardware breakpoints!Implementation available as IDA plugin (IDA Stealth[3])

Jan Newger Anti-RE Techniques in DRM Code

Page 54: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

P-Code Machine Overview

Machine PropertiesStack based with result register256 fixed size opcodes (1 byte)Opcodes interleaved with data (ASN.1)Allocate memory in host machineHigh-level opcodes (load opcodes, call into other modules,music decoding)Low level opcodes, emulate virtual CPU

Jan Newger Anti-RE Techniques in DRM Code

Page 55: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Loading of Opcodes

Opcode Module FilesSpecial module which contains P-Code machineContain native code + opcodesDecompressed at runtimeNo PE, no IAT, no sections, etc.Relocation table + some fixed imports (MSVCRT)

Jan Newger Anti-RE Techniques in DRM Code

Page 56: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Obfuscation in the P-Code Machine

Executing OpcodesPer-module random poolRandomize opcode <-> opcode handlerDescramble opcodes with PRNG in machineGarbage data interleaved with opcodesData parsed via ASN.1

Jan Newger Anti-RE Techniques in DRM Code

Page 57: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Impact of the P-Code Machine

Static RE DifficultUnderstand machine itself firstDifferent meaning of opcodes per moduleASN.1 parsing

Debugging Difficult

Low signal to noise ratio (big "handler loop")Even lower due to opcode descrambling

Jan Newger Anti-RE Techniques in DRM Code

Page 58: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Impact of the P-Code Machine

Static RE DifficultUnderstand machine itself firstDifferent meaning of opcodes per moduleASN.1 parsing

Debugging Difficult

Low signal to noise ratio (big "handler loop")Even lower due to opcode descrambling

Jan Newger Anti-RE Techniques in DRM Code

Page 59: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

P-Code Machine in IDA

Jan Newger Anti-RE Techniques in DRM Code

Page 60: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Strategies to find Decryption Algorithm + Keysetup

1 Write custom Disassembler (Expensive Strategy )Many handlersLong and complex high level handlersRe-assemble randomization, descrambling, garbageinstructions, ASN.1

2 Trace until key written to memory (Brute Force Strategy )Single-step via debugger scriptSlow, but reaches code writing keyNot so clever

3 Use emulation (Cool Strategy )Use CPU emulation (PyEmu[4], x86 Emu for IDA[5], ...)Fast, very flexible

Jan Newger Anti-RE Techniques in DRM Code

Page 61: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Strategies to find Decryption Algorithm + Keysetup

1 Write custom Disassembler (Expensive Strategy )Many handlersLong and complex high level handlersRe-assemble randomization, descrambling, garbageinstructions, ASN.1

2 Trace until key written to memory (Brute Force Strategy )Single-step via debugger scriptSlow, but reaches code writing keyNot so clever

3 Use emulation (Cool Strategy )Use CPU emulation (PyEmu[4], x86 Emu for IDA[5], ...)Fast, very flexible

Jan Newger Anti-RE Techniques in DRM Code

Page 62: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Strategies to find Decryption Algorithm + Keysetup

1 Write custom Disassembler (Expensive Strategy )Many handlersLong and complex high level handlersRe-assemble randomization, descrambling, garbageinstructions, ASN.1

2 Trace until key written to memory (Brute Force Strategy )Single-step via debugger scriptSlow, but reaches code writing keyNot so clever

3 Use emulation (Cool Strategy )Use CPU emulation (PyEmu[4], x86 Emu for IDA[5], ...)Fast, very flexible

Jan Newger Anti-RE Techniques in DRM Code

Page 63: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Strategies to find Decryption Algorithm + Keysetup

4 Use BPMs / Attack machine memory (Lazy Strategy )Use what we haveExploit machine memory managementFilebuffer size 0x1800, DES keyschedule size 0x80Set BP, fire when keysetup memory allocatedSet BPM, fire when keysetup writtenBack-trace from there

Keen DisappointmentDecryption and keysetup in native code! High-level handlers!

Jan Newger Anti-RE Techniques in DRM Code

Page 64: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

OverviewTrampolinesDebug RegistersP-Code Machine

Strategies to find Decryption Algorithm + Keysetup

4 Use BPMs / Attack machine memory (Lazy Strategy )Use what we haveExploit machine memory managementFilebuffer size 0x1800, DES keyschedule size 0x80Set BP, fire when keysetup memory allocatedSet BPM, fire when keysetup writtenBack-trace from there

Keen DisappointmentDecryption and keysetup in native code! High-level handlers!

Jan Newger Anti-RE Techniques in DRM Code

Page 65: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Outline1 Introduction

Legal IssuesAbout DRMsApproaching the DRM

2 SEH BasicsOverviewException Dispatching

3 Anti-RE TechniquesOverviewTrampolinesDebug RegistersP-Code Machine

4 Decrypting the ContentThe AlgorithmDemo

Jan Newger Anti-RE Techniques in DRM Code

Page 66: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Keysetup Algorithm

Key DerivationHash some filesUse different hash algorithmsDifferent key for every music file

Jan Newger Anti-RE Techniques in DRM Code

Page 67: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Decryption Algorithm

Decrypt content with DES-CBC (Cipher Block Chaining)IV from DRM file

Jan Newger Anti-RE Techniques in DRM Code

Page 68: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Demo

Demoor

"Han shot first!"

Jan Newger Anti-RE Techniques in DRM Code

Page 69: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Conclusion

SummaryOverall: good protectionBPMs led us to success, P-Code machine almost useless!Implementation weaknesses

Room for Improvements1 Transform more native code to P-Code2 Make P-Code machine more complex (nesting,

polymorphic handlers, self-modifying machine, ...)3 Improve (very) weak debugger detection4 Use DRs, let control flow depend on BPM/BPX firing5 ...

Jan Newger Anti-RE Techniques in DRM Code

Page 70: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Conclusion

SummaryOverall: good protectionBPMs led us to success, P-Code machine almost useless!Implementation weaknesses

Room for Improvements1 Transform more native code to P-Code2 Make P-Code machine more complex (nesting,

polymorphic handlers, self-modifying machine, ...)3 Improve (very) weak debugger detection4 Use DRs, let control flow depend on BPM/BPX firing5 ...

Jan Newger Anti-RE Techniques in DRM Code

Page 71: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Thanks for your Attention!

Questions?Contact: [email protected]

Jan Newger Anti-RE Techniques in DRM Code

Page 72: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

Electronic Frontier Foundation.Electronic Frontier Foundation.http://www.eff.org/.

DMCA.DMCA Encryption Research Paragraph.http://www.law.cornell.edu/uscode/html/uscode17/usc_sec_17_00001201----000-.html.

Jan Newger.IDA Stealth.http://www.newgre.net/idastealth.

Cody Pierce.PyEmu: A Multi-Purpose Scriptable x86 Emulator.http://dvlabs.tippingpoint.com/team/cpierce.

Chris Eagle.

Jan Newger Anti-RE Techniques in DRM Code

Page 73: Anti-RE Techniques in DRM Code - DEF CON Techniques in DRM Code ... Outline 1 Introduction Legal Issues About DRMs ... Loophole in DMCA -> "Encryption Research"[2]

IntroductionSEH Basics

Anti-RE TechniquesDecrypting the Content

The AlgorithmDemo

The x86 Emulator plugin for IDAPro.http://www.idabook.com/x86emu/.

Jan Newger Anti-RE Techniques in DRM Code