- pemu: a pin highly compatible out-of-vm dynamic binary instrumentation...

36
Overview PEMU Design Evaluations Discussion Conclusion References PEMU:APIN Highly Compatible Out-of-VM Dynamic Binary Instrumentation Framework Junyuan Zeng, Yangchun Fu, Zhiqiang Lin Department of Computer Science The University of Texas at Dallas March 15 th , 2015

Upload: others

Post on 17-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

PEMU: A PIN Highly Compatible Out-of-VM DynamicBinary Instrumentation Framework

Junyuan Zeng, Yangchun Fu, Zhiqiang Lin

Department of Computer ScienceThe University of Texas at Dallas

March 15th, 2015

Page 2: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Dynamic Binary Instrumentation (DBI)

An extremely powerful technique for program analysis.Dynamically inserts extra analysis code into the runningbinary program to observe how it behaves.

Applications1 Performance profiling2 Architecture simulation3 Program shepherding4 Program optimization5 Dynamic taint analysis6 Reverse engineering7 Malware analysis8 ...

Page 3: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Dynamic Binary Instrumentation (DBI)

An extremely powerful technique for program analysis.Dynamically inserts extra analysis code into the runningbinary program to observe how it behaves.

Applications1 Performance profiling2 Architecture simulation3 Program shepherding4 Program optimization5 Dynamic taint analysis6 Reverse engineering7 Malware analysis8 ...

Page 4: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

In-VM DBI

test %eax,%eaxje 0x8052913…

print(pc)test %eax,%eax

print(pc)je 0x8052913

syscall

GuestRing 3

GuestRing 0

Orig

inal

Pin

In‐VM Instrumentation

Page 5: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

In-VM DBI vs. Out-of-VM DBI

test %eax,%eaxje 0x8052913…

print(pc)test %eax,%eax

print(pc)je 0x8052913

print(pc)mov 0x0(%ebp),%ebxmov %ebx,0x28(%ebp)

mov $0x18,%esimov %esi,0x34(%ebp)

test %ebx,%ebxprint(pc)

je 0xae1a8c06jmp 0xae1a8bf5

movl $0x80528ca,0x20(%ebp)mov $0xa9bbcc8c,%eax

jmp 0xb5f3cc15jmp 0xae1a8c0b

movl $0x8052913,0x20(%ebp)mov $0xa9bbcc8d,%eax

jmp 0xb5f3cc15…

syscall

GuestRing 3

GuestRing 0

HostRing 3

Orig

inal

Pin

syscall

HostRing 0

In‐VM InstrumentationOut‐of‐VM Instrumentation

Page 6: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

State-of-the-Art

Platforms Year Emulator, Simulator, Virtualizer

Kernel Level Instrumentation

User Level Instrumentation

w/ API for instrumentation

Out-of-V

M

Guest OS Agnostic

PINAPI Compatible

Open Source

EMBRA [WR96] 1996 X X X 7 X 7 7 7VMWARE [DBR98] 1998 X X X 7 X 7 7 7KERNINST [TM99] 1999 7 X X X 7 X 7 X

DYNINSTAPI [BH00] 2000 7 X X X 7 X 7 XDYNAMO [BDB00] 2000 7 X X 7 7 X 7 7

BOCHS [boc01] 2001 X X X 7 X 7 7 XSIMICS [MCE+02] 2002 X X X 7 X 7 7 7

VALGRIND [NS03, NS07] 2003 7 7 X X 7 X 7 XSTRATA [SKV+03] 2003 7 7 X X 7 X 7 X

DYNAMORIO [BDB00, BZA12] 2004 7 7 X X 7 X 7 XQEMU [Bel05] 2005 X X X 7 X 7 7 X

PIN [LCM+05] 2005 7 7 X X 7 X X 7

NIRVANA [BCdJ+06] 2006 7 7 X X 7 X 7 7HDTRANS [SSNB06] 2006 7 7 X X 7 X 7 XVIRTUALBOX [Wat08] 2007 X X X 7 X 7 7 X

PINOS [BL07] 2007 X X X X 7 7 X 7TEMU [YS] 2010 X X X X X X 7 X

DYNINST [MB11] 2010 7 X X X 7 X 7 XDRK [PFG12] 2013 7 X X X 7 X 7 X

DECAF [HPY+14] 2014 X X X X X 7 7 XPEMU 2015 X X X X X X X X

Page 7: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Limitations Among Prior Works

Process Level DBI (e.g., PIN, VALGRIND)Process-level DBI such as PIN and VALGRIND providesrich APIs to analyze user level binary code execution, butthe analysis code is executed inside the VM (i.e., in-VM)with the same privilege as the instrumented process.No kernel level instrumentationLimited type of OS (VALGRIND only for Linux)

VM Monitor Level DBI (e.g., QEMU)No general DBI APIs

Page 8: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

State-of-the-Art

Platforms Year Emulator, Simulator, Virtualizer

Kernel Level Instrumentation

User Level Instrumentation

w/ API for instrumentation

Out-of-V

M

Guest OS Agnostic

PINAPI Compatible

Open Source

EMBRA [WR96] 1996 X X X 7 X 7 7 7VMWARE [DBR98] 1998 X X X 7 X 7 7 7KERNINST [TM99] 1999 7 X X X 7 X 7 X

DYNINSTAPI [BH00] 2000 7 X X X 7 X 7 XDYNAMO [BDB00] 2000 7 X X 7 7 X 7 7

BOCHS [boc01] 2001 X X X 7 X 7 7 XSIMICS [MCE+02] 2002 X X X 7 X 7 7 7

VALGRIND [NS03, NS07] 2003 7 7 X X 7 X 7 XSTRATA [SKV+03] 2003 7 7 X X 7 X 7 X

DYNAMORIO [BDB00, BZA12] 2004 7 7 X X 7 X 7 XQEMU [Bel05] 2005 X X X 7 X 7 7 X

PIN [LCM+05] 2005 7 7 X X 7 X X 7

NIRVANA [BCdJ+06] 2006 7 7 X X 7 X 7 7HDTRANS [SSNB06] 2006 7 7 X X 7 X 7 XVIRTUALBOX [Wat08] 2007 X X X 7 X 7 7 X

PINOS [BL07] 2007 X X X X 7 7 X 7TEMU [YS] 2010 X X X X X X 7 X

DYNINST [MB11] 2010 7 X X X 7 X 7 XDRK [PFG12] 2013 7 X X X 7 X 7 X

DECAF [HPY+14] 2014 X X X X X 7 7 XPEMU 2015 X X X X X X X X

Page 9: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Introduction of PEMU

PIN + QEMU = PEMU

ObjectivesRich APIs

Provide rich and well-defined APIs for DBIPIN-API compatibility

Cross-OSOS agnostic for the introspection

Strong IsolationOut-of-VM Instrumentation to isolate analysis routings withtarget programs (QEMU)

VM IntrospectionSupport high level guest object introspection

Page 10: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

A sample PIN plugin1 static UINT64 icount;2 FILE *pFile;3 VOID docount(UINT32 c) { icount += c; }4 VOID Trace(TRACE trace, VOID *v) {5 for (BBL bbl = TRACE_BblHead(trace);6 BBL_Valid(bbl); bbl = BBL_Next(bbl)) {7 BBL_InsertCall(bbl, IPOINT_BEFORE,8 (AFUNPTR)docount, IARG_UINT32, BBL_NumIns(bbl),9 IARG_END);10 }11 }12 VOID Fini(INT32 code, VOID *v) {13 fprintf(pFile, "Count %lld\n", icount);14 fclose(pFile);15 }16 INT32 Usage(VOID) {17 return 0;18 }19 int main(int argc, char * argv[]) {20 if(PIN_Init(argc, argv)) return Usage();21 pFile = fopen("pemu_count", "w");22 TRACE_AddInstrumentFunction(Trace, 0);23 PIN_AddFiniFunction(Fini, 0);24 PIN_StartProgram();25 return 0;26 }

Page 11: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Challenges

1 How to implement PIN API based on QEMUQEMU does not have abstraction for TRACE, SEC,Function and IMG etc.QEMU basic block has limited size

⇒ Instrumentation Engine2 Semantic gap between guest OS and host OS

How to find the target process or threadPIN plugins may inspect guest OS state, such as getpid

⇒ Introspection Engine

Page 12: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Challenges

1 How to implement PIN API based on QEMUQEMU does not have abstraction for TRACE, SEC,Function and IMG etc.QEMU basic block has limited size

⇒ Instrumentation Engine2 Semantic gap between guest OS and host OS

How to find the target process or threadPIN plugins may inspect guest OS state, such as getpid

⇒ Introspection Engine

Page 13: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

PEMU Architecture

User Space

Guest OS Kernel

Instrumentation Engine

PEMU

IntrospectionEngineEngine

PEMU Tools

Engine

Plugin 1 Plugin 2 Plugin 3 …

Page 14: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Instrumentation Engine: PIN APIs

9.63%

17.35%

35.71%

2.31%

8.83%

2.91%

1.71%

7.32%

3.11%

11.13%

SYM

THREAD

CONTROL

IMG/SEC/RTN

EXCEPTION

CODECACHE

SYSCALL

INS/BBL/TRACE

REG

Other

Page 15: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Instrumentation Engine

1 Trace Instrumentation: occurs immediately before a codesequence is executed

Instruction Level, e.g., INS_InsertCall(ins,IPOINT_BEFORE,..)Basic Block Level, e.g., BBL_InsertCall(bbl,IPOINT_BEFORE,..)Trace Level, e.g., TRACE_InsertCall(trace,IPOINT_BEFORE,..)

⇒ Adding our own disassembler (TRACE Constructor)2 Ahead-of-time Instrumentation: caches the

instrumentation before the execution.IMG instrumentation, e.g.,IMG_AddInstrumentFunctionRTN instrumentation, eg.., RTN_InsertCall

⇒ Instrumenting code when image is loaded

Page 16: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Instrumentation Engine

1 Trace Instrumentation: occurs immediately before a codesequence is executed

Instruction Level, e.g., INS_InsertCall(ins,IPOINT_BEFORE,..)Basic Block Level, e.g., BBL_InsertCall(bbl,IPOINT_BEFORE,..)Trace Level, e.g., TRACE_InsertCall(trace,IPOINT_BEFORE,..)

⇒ Adding our own disassembler (TRACE Constructor)2 Ahead-of-time Instrumentation: caches the

instrumentation before the execution.IMG instrumentation, e.g.,IMG_AddInstrumentFunctionRTN instrumentation, eg.., RTN_InsertCall

⇒ Instrumenting code when image is loaded

Page 17: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Instrumentation EngineInstrumen

tatio

n En

gine

Code Injector

QEMU‐DBT

TRACEConstructor

push %ebp;mov %esp,%ebppush %ebx

MetaData

Instrumentation API

push %rbp;mov %rsp,%rbpcall Instr_funcpush %rbx

GuestCode

HostCode

Guest CPU

Host CPU

Page 18: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Instrumentation EngineInstrumen

tatio

n En

gine

Code Injector

QEMU‐DBT

TRACEConstructor

push %ebp;mov %esp,%ebppush %ebx

MetaData

Instrumentation API

push %rbp;mov %rsp,%rbpcall Instr_funcpush %rbx

GuestCode

HostCode

Type_i, Func_i, List_of_args …PCi

PCk

Hooking PointHash-table (HPHT)

PCj

Guest CPU

Host CPU

Type_k, Func_k, List_of_args …

TPC

Instances of TRACE, BBL and INS

Type_j, Func_j, List_of_args …

TRACETRACE_Address

TRACE_SizeTRACE_NumBbl

TRACE_BblHead*…

NULL

BBBBL_Address

BBL_SizeBBL_NumInsBBL_Next*

BBL_InsHead*…

INSINS_Address

INS_SizeINS_NumInsINS_Next*

…NULL

BBBBL_Address

BBL_SizeBBL_NumInsBBL_Next*

BBL_InsHead*…

INSINS_Address

INS_SizeINS_NumInsINS_Next*

INSINS_Address

INS_SizeINS_NumInsINS_Next*

PCi, PCj, PCk, …

Page 19: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Introspection Engine

Goals and SolutionsIdentification of the monitoring process/threads

⇒ Capture the data life time of PGD (CR3 in x86) toidentify the new process⇒ Kernel esp to identify new threads

Bridging the semantic gap for the out-of-VM plugins⇒ Using a system call redirection/forwarding approachHyperShell [YFL14]

Page 20: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Introspection Engine: Syscall Redirection

…  PEMU_open(..)  

…    

 Open(…)  

…  Syscall()  {                    set_up_syscall_arg()  

PEMU_syscall()  …  

}  …    

Plugin:

PEMU_glibc:

User

Kernel

❸ ❹

Page 21: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Compatibility Testing With PIN Plugins

Plugin Description Supportedcalltrace.so Call trace tracing X

extmix.so Instruction extension mix profile Xinscount2_vregs.so Counting executing instructions X

pinatrace.so Memory address tracing Xxed-cache.so Decode cache profile X

catmix.so Instruction category mix profile Xfence.so Runtime text modification guard X

jumpmix.so Jmp/branch/call profiling Xregmix.so Register usage mix profile X

xed-print.so XED usage testing Xcoco.so Code coverage analyzer X

icount.so Counting executing instructions Xldstmix.so Register/memory operand profiler X

topopcode.so Opcode mix profiler Xxed-use.so XED interface usage testing Xdcache.so Data cache simulation 7

ilenmix.so Instruction length mix profiler Xmalloctrace.so Tracing calls to malloc X

toprtn.so Hostest routines profiling Xedgcnt.so Control flow edge profiler X

inscount2_mt.so Counting executing instructions Xopcodemix.so Opcode mix profiler 7

trace.so Compressed instruction tracer X

Table 1. Compatibility Testing with Existing PIN Plugins.

beginning or end of an interval of the execution of a program.We leave the support of these APIs for future work.

5.2 Performance Evaluation

Next, we test the performance of PEMU. We perform two setsof experiments: one is to measure how slow PEMU is whencompared to a vanilla-QEMU, and the other is how slow whencompared to PIN. We directly use the instruction countingplugin described in Fig 2. This plugin increases the numberof instructions in a BB for an accumulated counter beforethe execution of each BB. We test this plugin with the SPEC2006 benchmark programs. Each of the benchmark programsis executed 100 times, and we use the corresponding averagenumber in our report.

Performance Comparison with vanilla-QEMU. In this ex-periment, we measure the overhead introduced by PEMU

instrumentation. We compare the execution when runningthe benchmarks with PEMU, directly with QEMU without anyinstrumentation.

We report the detailed experimental result in Table 2.Specifically, we show the total number of instructions exe-cuted in the 2nd column and also the execution time of QEMU

and PEMU is reported in the 3rd and 4th column (namely,TQemu and T_Pemu). We notice that on average there are17649.1 million instructions traced for these benchmarks,and the average slowdown over QEMU is about 4.33X, whichwe believe it is reasonable for practical use. This overheadincludes our TRACE Constructor, Code Injector, as well asruntime overhead of the analysis routine.

Performance Comparison with PIN. In the second experi-ment, we compare PEMU against PIN using the same pluginwith the same benchmark. The execution time of runningin PIN is presented in the 6th column, and the comparisonbetween PEMU and PIN is presented in the last column.

We notice that the average slowdown between PEMUand PIN is over 83.61X. The main reason is that PIN isrunning natively while PEMU (based on QEMU) needs extratranslation. The largest slowdown comes from 444.namdwhich is above 310X. However, we note that when runningthis program in vanilla-QEMU, it will have close to 100Xslowdown. We carefully examine the reason and find theroot cause due to the use of large amount of floating pointinstructions which needs time-consuming emulation insideQEMU.

It is also interesting to note that for 450.soplex, run-ning in QEMU is faster than that of PIN. The main reason isthis program contains more control flow instructions that willgo to the middle of a TRACE, thereby breaking the TRACE.In this case, QEMU (based on BBL disassembling) will justredisassemble the basic block that has not been disassembled,but PIN (based on TRACE disassembling) will redisassemblethe whole trace after a new trace is found. Meanwhile, therunning time of this program is relatively short. Thus, thetime is dominated by the disassembling time.

5.3 Memory Cost EvaluationSince PEMU uses an ahead-of-time instrumentation that willstore the hooking point to facilitate the instrumentation, wewould like to measure how much memory space this hookingpoint table consumes. Again, we evaluate this memory costwith our instruction counting plugin against the SPEC2006benchmark. The result is presented in Fig. 7. We notice thatthe average memory cost is about 9M.

More specifically, as shown in Fig. 7, the maximum mem-ory cost comes from 465.tonto (about 22M) because thisprogram contains the largest number of BB, resulting in thelargest hash table to store the hooking points. More interest-ingly, 464.h264ref is one of the most time consumingprograms but requires a relative small size of hash table. Thereason is that this program contains lots of loops and thuscertain instructions get executed repeatedly.

5.4 Case StudiesWe have demonstrated using PEMU to analyze Linux binaries.In fact, our system is cross-OS, which is one of our designgoals. To test this, we apply PEMU to analyze Windows bina-ries as we have evaluated with Linux binaries. In particular,we use a number of anti-PIN binaries during this test.

First, we test how PEMU would analyze the software pro-tected by tElock and safengine shielden, whichare two widely used tools to build anti-analysis software.We apply these protectors to the hostname binaries ina Win-XP SP3 machine, with anti-debugging and anti-instrumentation enabled, and produce two anti-analysis

Page 22: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Performance Evaluation: Speed

Program #Inst (M) TQemu (s) TPemu (s) TPemu / TQemu TPin (s) TQemu / TPin TPemu / TPin

401.bzip2 11500.27 24.55 81.15 3.31 11.17 2.20 7.26403.gcc 4940.36 18.35 169.21 9.22 13.56 1.35 12.48410.bwaves 29360.09 419.99 1336.57 3.18 7.44 56.45 179.65416.gamess 2121.15 23.19 84.99 3.66 3.35 6.92 25.37429.mcf 3562.67 23.91 70.58 2.95 3.55 6.74 19.88433.milc 39509.49 779.07 2570.44 3.30 9.28 83.95 276.99435.gromacs 4907.53 106.28 334.74 3.15 3.43 30.99 97.59436.cactusADM 9730.11 304.89 1019.89 3.35 4.42 68.98 230.74437.leslie3d 55857.54 900.01 3009.06 3.34 15.38 58.52 195.65444.namd 74037.63 1523.78 5037.00 3.31 16.22 93.94 310.54445.gobmk 314.88 2.43 4.17 1.72 1.71 1.42 2.44450.soplex 63.67 1.49 2.22 1.49 1.80 0.83 1.23453.povray 2987.41 36.16 193.17 5.34 3.52 10.27 54.88454.calculix 187.33 2.53 6.61 2.61 2.46 1.03 2.69456.hmmer 17862.2 46.43 260.56 5.61 6.95 6.68 37.49458.sjeng 15514.49 48.40 432.79 8.94 14.38 3.37 30.10462.libquantum 408.63 0.77 2.01 2.61 0.62 1.24 3.24464.h264ref 98144.32 392.21 2751.31 7.01 34.01 11.53 80.90465.tonto 3571.85 48.23 195.16 4.05 5.44 8.87 35.88470.lbm 7744.81 161.22 692.51 4.30 2.92 55.21 237.16471.omnetpp 2209.23 16.24 136.63 8.41 3.19 5.09 42.83473.astar 26645.10 102.95 734.52 7.13 13.67 7.53 53.73482.sphinx3 6198.21 77.95 322.26 4.13 4.94 15.78 65.23999.specrand 6198.21 1.42 2.46 1.73 0.92 1.54 2.67Avg. 17649.05 210.94 810.42 4.33 7.68 22.52 83.61

Table 2. Performance compared with vanilla-QEMU and PIN.

0

5

10

15

20

25

401.b

zip2

403.g

cc

410.b

wav

es

416.g

ames

s

429.m

cf

433.m

ilc

435.g

rom

acs

436.c

actu

sAD

M

437.l

esli

e3d

444.n

amd

445.g

obm

k

450.s

ople

x

453.p

ovra

y

454.c

alcu

lix

456hm

mer

458.s

jeng

462.l

ibquan

tum

464.h

264re

f

465.t

onto

470.l

bm

471.o

mnet

pp

473.a

star

482.s

phin

x3

999.s

pec

rand

Siz

e of

Has

hT

able

(M

)

Figure 7. Memory Cost Comparison with SPEC2006 Bench-marks

hostname binaries. We then use PIN and PEMU to ana-lyze the packed hostname.

More specifically, we developed a simple strace (asshown in Fig. 8) plugin to trace the syscall executed by thehostname binary. This plugin will print the syscall numberat syscall entry point, and the return value at syscall exit point.We compiled it into a PIN plugin and PEMU plugin with thesame source code. PIN failed on these two tests. Both packedprograms detected the presence of PIN and exited at earlystages. In contrast, hostname ran successfully on PEMU

and displayed the host name.In our other case study, we used eXait [13], a benchmark-

like tool to test anti-instrumentation techniques. eXait hasa plugin architecture, and each technique is implemented

1 FILE *trace;2 VOID SysBefore(ADDRINT ip, ADDRINT num) {3 fprintf(trace,"0x%lx: %ld\n",4 (unsigned long)ip, (long)num);5 }6 VOID SyscallEntry(THREADID threadIndex,7 CONTEXT *ctxt, SYSCALL_STANDARD std, VOID *v) {8 SysBefore(PIN_GetContextReg(ctxt, REG_INST_PTR),9 PIN_GetSyscallNumber(ctxt, std));10 }11 VOID Fini(INT32 code, VOID *v) {12 printf("program exit()\n");13 }14 INT32 Usage(VOID){15 return 0;16 }17 int main(int argc, char * argv[]){18 if(PIN_Init(argc, argv)) return Usage();19 trace = fopen("strace.out", "w");20 PIN_AddSyscallEntryFunction(SyscallEntry, 0);21 PIN_AddFiniFunction(Fini, 0);22 PIN_StartProgram();23 return 0;24 }

Figure 8. A cross-OS PEMU plugin to trace the syscall.

as a separated DLL. There are 21 plugins in total. Againwe run PIN with strace plugin to instrument eXait andthe loaded DLLs. We found that 17 anti-instrumentationtechniques detect the presence of PIN. But none of themdetect the presence of PEMU.

Through these case studies, we show there is a need for out-of-VM PIN alternatives. Also, even though future malwarewill be able to detect the presence of PEMU, we should be ableto add countermeasures against them, given that the sourcecode of PEMU is open.

Page 23: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Performance Evaluation: Memory Cost

Program #Inst (M) TQemu (s) TPemu (s) TPemu / TQemu TPin (s) TQemu / TPin TPemu / TPin

401.bzip2 11500.27 24.55 81.15 3.31 11.17 2.20 7.26403.gcc 4940.36 18.35 169.21 9.22 13.56 1.35 12.48410.bwaves 29360.09 419.99 1336.57 3.18 7.44 56.45 179.65416.gamess 2121.15 23.19 84.99 3.66 3.35 6.92 25.37429.mcf 3562.67 23.91 70.58 2.95 3.55 6.74 19.88433.milc 39509.49 779.07 2570.44 3.30 9.28 83.95 276.99435.gromacs 4907.53 106.28 334.74 3.15 3.43 30.99 97.59436.cactusADM 9730.11 304.89 1019.89 3.35 4.42 68.98 230.74437.leslie3d 55857.54 900.01 3009.06 3.34 15.38 58.52 195.65444.namd 74037.63 1523.78 5037.00 3.31 16.22 93.94 310.54445.gobmk 314.88 2.43 4.17 1.72 1.71 1.42 2.44450.soplex 63.67 1.49 2.22 1.49 1.80 0.83 1.23453.povray 2987.41 36.16 193.17 5.34 3.52 10.27 54.88454.calculix 187.33 2.53 6.61 2.61 2.46 1.03 2.69456.hmmer 17862.2 46.43 260.56 5.61 6.95 6.68 37.49458.sjeng 15514.49 48.40 432.79 8.94 14.38 3.37 30.10462.libquantum 408.63 0.77 2.01 2.61 0.62 1.24 3.24464.h264ref 98144.32 392.21 2751.31 7.01 34.01 11.53 80.90465.tonto 3571.85 48.23 195.16 4.05 5.44 8.87 35.88470.lbm 7744.81 161.22 692.51 4.30 2.92 55.21 237.16471.omnetpp 2209.23 16.24 136.63 8.41 3.19 5.09 42.83473.astar 26645.10 102.95 734.52 7.13 13.67 7.53 53.73482.sphinx3 6198.21 77.95 322.26 4.13 4.94 15.78 65.23999.specrand 6198.21 1.42 2.46 1.73 0.92 1.54 2.67Avg. 17649.05 210.94 810.42 4.33 7.68 22.52 83.61

Table 2. Performance compared with vanilla-QEMU and PIN.

0

5

10

15

20

25

40

1.b

zip

2

40

3.g

cc

41

0.b

wav

es

41

6.g

ames

s

42

9.m

cf

43

3.m

ilc

43

5.g

rom

acs

43

6.c

actu

sAD

M

43

7.l

esli

e3d

44

4.n

amd

44

5.g

ob

mk

45

0.s

op

lex

45

3.p

ov

ray

45

4.c

alcu

lix

45

6h

mm

er

45

8.s

jen

g

46

2.l

ibq

uan

tum

46

4.h

26

4re

f

46

5.t

on

to

47

0.l

bm

47

1.o

mn

etp

p

47

3.a

star

48

2.s

ph

inx

3

99

9.s

pec

ran

d

Siz

e o

f H

ash

Tab

le (

M)

Figure 7. Memory Cost Comparison with SPEC2006 Bench-marks

hostname binaries. We then use PIN and PEMU to ana-lyze the packed hostname.

More specifically, we developed a simple strace (asshown in Fig. 8) plugin to trace the syscall executed by thehostname binary. This plugin will print the syscall numberat syscall entry point, and the return value at syscall exit point.We compiled it into a PIN plugin and PEMU plugin with thesame source code. PIN failed on these two tests. Both packedprograms detected the presence of PIN and exited at earlystages. In contrast, hostname ran successfully on PEMU

and displayed the host name.In our other case study, we used eXait [13], a benchmark-

like tool to test anti-instrumentation techniques. eXait hasa plugin architecture, and each technique is implemented

1 FILE *trace;2 VOID SysBefore(ADDRINT ip, ADDRINT num) {3 fprintf(trace,"0x%lx: %ld\n",4 (unsigned long)ip, (long)num);5 }6 VOID SyscallEntry(THREADID threadIndex,7 CONTEXT *ctxt, SYSCALL_STANDARD std, VOID *v) {8 SysBefore(PIN_GetContextReg(ctxt, REG_INST_PTR),9 PIN_GetSyscallNumber(ctxt, std));10 }11 VOID Fini(INT32 code, VOID *v) {12 printf("program exit()\n");13 }14 INT32 Usage(VOID){15 return 0;16 }17 int main(int argc, char * argv[]){18 if(PIN_Init(argc, argv)) return Usage();19 trace = fopen("strace.out", "w");20 PIN_AddSyscallEntryFunction(SyscallEntry, 0);21 PIN_AddFiniFunction(Fini, 0);22 PIN_StartProgram();23 return 0;24 }

Figure 8. A cross-OS PEMU plugin to trace the syscall.

as a separated DLL. There are 21 plugins in total. Againwe run PIN with strace plugin to instrument eXait andthe loaded DLLs. We found that 17 anti-instrumentationtechniques detect the presence of PIN. But none of themdetect the presence of PEMU.

Through these case studies, we show there is a need for out-of-VM PIN alternatives. Also, even though future malwarewill be able to detect the presence of PEMU, we should be ableto add countermeasures against them, given that the sourcecode of PEMU is open.

Page 24: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Applications: anti-PIN malware analysis

BackgroundAnti-PIN malware: malware exits when it detects it is runinside PIN

Case StudiestElock, Safengine Shielden: two widely used tools tobuild anti-analysis software.eXait: a benchmark-like tool to test anti-instrumentationtechniques

Page 25: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Applications: anti-PIN malware analysis1 FILE *trace;2 VOID SysBefore(ADDRINT ip, ADDRINT num) {3 fprintf(trace,"0x%lx: %ld\n",4 (unsigned long)ip, (long)num);5 }6 VOID SyscallEntry(THREADID threadIndex,7 CONTEXT *ctxt, SYSCALL_STANDARD std, VOID *v) {8 SysBefore(PIN_GetContextReg(ctxt, REG_INST_PTR),9 PIN_GetSyscallNumber(ctxt, std));10 }11 VOID Fini(INT32 code, VOID *v) {12 printf("program exit()\n");13 }14 INT32 Usage(VOID){15 return 0;16 }17 int main(int argc, char * argv[]){18 if(PIN_Init(argc, argv)) return Usage();19 trace = fopen("strace.out", "w");20 PIN_AddSyscallEntryFunction(SyscallEntry, 0);21 PIN_AddFiniFunction(Fini, 0);22 PIN_StartProgram();23 return 0;24 }

Page 26: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Applications: anti-PIN malware analysis

Our ResulttElock, Safengine Shielden:

PIN failed to run the testing programs generated by tElockand Safengine Shielden, which detected the presence ofPIN and exited at early stages.The testing programs ran successfully on PEMU.

eXait:17 out of 21 anti-instrumentation techniques detect thepresence of PIN,None of them detect the presence of PEMU.

Page 27: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Applications: Virtual Machine Introspection

Building VMI tools

PEMU can be used to build many out-of-VM introspectiontools.A number of such tools have been built atop PEMU:

VMST [FL12]EXTERIOR [FL13]

Page 28: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Limitation and Future Work

Currently Incomplete support of the PIN-APIs⇒ Make PEMU open source

A weak semantic gap [JBZ+14]⇒ Not trust the guest OS kernel at all

No optimization with the generated instrumentation andanalysis routine yet

Page 29: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Limitation and Future Work

Currently Incomplete support of the PIN-APIs⇒ Make PEMU open source

A weak semantic gap [JBZ+14]⇒ Not trust the guest OS kernel at all

No optimization with the generated instrumentation andanalysis routine yet

Page 30: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Conclusion

PEMU

A new dynamic binary code instrumentation framework1 PIN-API compatibility2 Cross-OS (support both Windows and Linux)3 Out-of-VM (strong isolation with the analysis routine and

the target code)

Page 31: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Availability

The source code of PEMU is available at:https://github.com/utds3lab/pemu

Page 32: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

Thank you

Page 33: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

References I

Sanjay Bhansali, Wen-Ke Chen, Stuart de Jong, Andrew Edwards, Ron Murray, Milenko Drinic, DarekMihocka, and Joe Chau, Framework for instruction-level tracing and analysis of program executions,Proceedings of the 2Nd International Conference on Virtual Execution Environments (New York, NY, USA),VEE ’06, ACM, 2006, pp. 154–163.

Vasanth Bala, Evelyn Duesterwald, and Sanjeev Banerjia, Dynamo: A transparent dynamic optimizationsystem, Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design andImplementation (New York, NY, USA), PLDI ’00, ACM, 2000, pp. 1–12.

Fabrice Bellard, Qemu, a fast and portable dynamic translator, Proceedings of the annual conference onUSENIX Annual Technical Conference (Berkeley, CA, USA), ATEC ’05, USENIX Association, 2005.

Bryan Buck and Jeffrey K. Hollingsworth, An api for runtime code patching, Int. J. High Perform. Comput.Appl. 14 (2000), no. 4, 317–329.

Prashanth P. Bungale and Chi-Keung Luk, Pinos: A programmable framework for whole-system dynamicinstrumentation, Proceedings of the 3rd international conference on Virtual execution environments, 2007,pp. 137–147.

bochs: The open source ia-32 emulation project, 2001, http://bochs.sourceforge.net/.

Derek Bruening, Qin Zhao, and Saman Amarasinghe, Transparent dynamic instrumentation, Proceedings ofthe 8th ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments (New York, NY, USA), VEE’12, ACM, 2012, pp. 133–144.

Scott W. Devine, Edouard Bugnion, and Mendel Rosenblum,Virtualization System Including a Virtual Machine Monitor for a Computer with a Segmented Architecture,United States Patent 6,397,242 (1998).

Page 34: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

References II

Yangchun Fu and Zhiqiang Lin, Space traveling across vm: Automatically bridging the semantic-gap in virtualmachine introspection via online kernel data redirection, Proceedings of the 2012 IEEE Symposium onSecurity and Privacy (San Francisco, CA), May 2012.

, Exterior: Using a dual-vm based external shell for guest-os introspection, configuration, andrecovery, Proceedings of the Ninth Annual International Conference on Virtual Execution Environments(Houston, TX), March 2013.

Andrew Henderson, Aravind Prakash, Lok Kwong Yan, Xunchao Hu, Xujiewen Wang, Rundong Zhou, andHeng Yin, Make it work, make it right, make it fast: Building a platform-neutral whole-system dynamic binaryanalysis platform, Proceedings of the 2014 International Symposium on Software Testing and Analysis (NewYork, NY, USA), ISSTA 2014, ACM, 2014, pp. 248–258.

Bhushan Jain, Mirza Basim Baig, Dongli Zhang, Donald E. Porter, and Radu Sion, Sok: Introspections ontrust and the semantic gap, Proceedings of the 2014 IEEE Symposium on Security and Privacy(Washington, DC, USA), SP ’14, IEEE Computer Society, 2014, pp. 605–620.

Chi-Keung Luk, Robert Cohn, Robert Muth, Harish Patil, Artur Klauser, Geoff Lowney, Steven Wallace,Vijay Janapa Reddi, and Kim Hazelwood, Pin: building customized program analysis tools with dynamicinstrumentation, Proceedings of the 2005 ACM SIGPLAN conference on Programming language design andimplementation (New York, NY, USA), PLDI ’05, ACM, 2005, pp. 190–200.

Barton P. Miller and Andrew R. Bernat, Anywhere, any time binary instrumentation, September 2011.

Peter S. Magnusson, Magnus Christensson, Jesper Eskilson, Daniel Forsgren, Gustav Hållberg, JohanHögberg, Fredrik Larsson, Andreas Moestedt, and Bengt Werner, Simics: A full system simulation platform,Computer 35 (2002), no. 2, 50–58.

Page 35: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

References III

Nicholas Nethercote and Julian Seward, Valgrind: A program supervision framework, In Third Workshop onRuntime Verification (RV’03), 2003.

Nicholas Nethercote and Julian Seward, Valgrind: A framework for heavyweight dynamic binaryinstrumentation, Proceedings of the 2007 ACM SIGPLAN Conference on Programming Language Designand Implementation (New York, NY, USA), PLDI ’07, ACM, 2007, pp. 89–100.

Angela Demke Brown Peter Feiner and Ashvin Goel, Comprehensive kernel instrumentation via dynamicbinary translation, Proceedings of the seventeenth international conference on Architectural Support forProgramming Languages and Operating Systems, 2012.

K. Scott, N. Kumar, S. Velusamy, B. Childers, J. W. Davidson, and M. L. Soffa, Retargetable andreconfigurable software dynamic translation, Proceedings of the International Symposium on CodeGeneration and Optimization: Feedback-directed and Runtime Optimization (Washington, DC, USA), CGO’03, IEEE Computer Society, 2003, pp. 36–47.

Swaroop Sridhar, Jonathan S. Shapiro, Eric Northup, and Prashanth P. Bungale, Hdtrans: An open source,low-level dynamic instrumentation system, Proceedings of the 2Nd International Conference on VirtualExecution Environments (New York, NY, USA), VEE ’06, ACM, 2006, pp. 175–185.

Ariel Tamches and Barton P. Miller, Fine-grained dynamic instrumentation of commodity operating systemkernels, Proceedings of the Third Symposium on Operating Systems Design and Implementation (Berkeley,CA, USA), OSDI ’99, USENIX Association, 1999, pp. 117–130.

Jon Watson, Virtualbox: Bits and bytes masquerading as machines, Linux J. 2008 (2008), no. 166.

Page 36: - Pemu: A Pin Highly Compatible Out-of-VM Dynamic Binary Instrumentation …web.cse.ohio-state.edu/~lin.3021/file/vee-15.pdf · 2018-01-07 · Dynamic Binary Instrumentation (DBI)

Overview PEMU Design Evaluations Discussion Conclusion References

References IV

Emmett Witchel and Mendel Rosenblum, Embra: Fast and flexible machine simulation, Proceedings of the1996 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems(New York, NY, USA), SIGMETRICS ’96, ACM, 1996, pp. 68–79.

Junyuan Zeng Yangchun Fu and Zhiqiang Lin, Hypershell: A practical hypervisor layer guest os shell forautomated in-vm management, USENIX ATC’14 Proceedings of the 2014 USENIX conference on USENIXAnnual Technical Conference (USENIX Association Berkeley, CA, USA), USENIX Association, 2014,pp. 85–96.

Heng Yin and Dawn Song, Temu: Binary code analysis via whole-system layered annotative execution,Technical Report UCB/EECS-2010-3, EECS Department, University of California, Berkeley, Jan 2010.