![Page 1: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/1.jpg)
SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES
Alwyn Roshan [email protected]
1
Department of Computer Science & Engineering National Institute of Technology, Karnataka
![Page 2: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/2.jpg)
2
![Page 3: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/3.jpg)
OBJECTIVE To study the VM detection techniques in
popular Virtual machines. Develop strategy to counter the detection. Prevent analysis aware malwares from
detecting VM.
3
![Page 4: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/4.jpg)
PLAN OF ACTION Introduction VM detection techniques Detection techniques in VMware, VirtualBox
and VirtualPC. Related Work Prevent Analysis aware malwares from
detecting VM. VMDetectGuard – Tool to mask VM detection :
Windows Optimization of VMDetectGuard Results
4
![Page 5: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/5.jpg)
INTRODUCTION
5
![Page 6: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/6.jpg)
MALWAREMalware: It is a collective term for any
malicious software which enters the system without the authorization of the user of the system.
Anti-virus/anti-malware products do not guarantee complete protection.
6
![Page 7: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/7.jpg)
PRESENT SCENARIO Security researchers use malware
analysis tools to build defenses against the unknown malware forms.
They then build patches for the newly discovered vulnerabilities and exploits.
Virtualization has emerged as a very promising technology.
Malware analyst use Virtual Machine Environment (VME), debuggers and sandboxes in their analysis work. 7
![Page 8: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/8.jpg)
VIRTUALIZATION A software based representation of a
computer that executes programs in the same way as a real computer.
Examples, VMware, Virtual PC, VirtualBox. Advantages
Reduced capital and operational costs through more efficient use of hardware resources.
Simplifies maintenance . Improves scalability and deployment agility. Improves reliability.
8
![Page 9: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/9.jpg)
BENEFITS OF VIRTUALIZATION TO SECURITY RESEARCHERS
Researchers can intrepidly execute potential malware samples without having their systems affected.
If a malware destabilizes the OS, analyst just needs to load in a fresh image on a VM.
Reduces time and cost. Increases productivity.
9
![Page 10: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/10.jpg)
ANALYSIS AWARENESS FUNCTIONALITY Malware developers have added a new
functionality to malware. Detect the presence of analysis tools such as
VMs, debuggers and sandboxes. Hide their malicious behavior on detection. Analysis Aware / Split Personality malware.
10
![Page 11: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/11.jpg)
RELATED WORK Carpenter (Carpenter et al., 2007) proposes
two mitigation techniques. They aim at tricking the malware by
1. Changing the configuration settings of the .vmx file present on the host system and,
2. Altering the magic value to break the guest-host communication channel.
11
![Page 12: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/12.jpg)
DRAWBACKS OF THE FIRST APPROACH The configuration options break the
communication channel between guest and host not just for the program trying to detect the VM, but for all the programs.
Moreover the authors claim that these are undocumented features and that they are not aware of any side effects.
12
![Page 13: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/13.jpg)
RELATED WORK The work by Guizani (Guizani et al., 2009)
provides an effective solution for Server-Side Dynamic Code Analysis.
Small part of the solution deals with tricking the Split Personality malware that employ Memory Detection and VM Communication Channel Detection techniques.
13
![Page 14: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/14.jpg)
RELATED WORK Kalpa Vishnani et. al. 2011: Masks all the
detection techniques used in Vmware.
14
![Page 15: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/15.jpg)
RELATED WORK Other works concentrate
Detecting this category of malwares Running in host machine
Save the current state quickly restore to previous state
Virtual machines in the order of market share VMware, Virtual PC, and Virtual Box.
15
![Page 16: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/16.jpg)
VM DETECTION TECHNIQUES Hardware fingerprinting Registry Check Process and File Check Memory Check Timing Analysis Communication Channel Check Invalid Instruction Check
16
![Page 17: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/17.jpg)
HARDWARE FINGERPRINTING Involves looking for specific virtualized
hardware. VMs give an abstracted view of many
hardware components. Querying for such components reveals VM
presence. For Example: BIOS, Motherboard, SCSI
Controllers, USB Controllers, etc.
17
![Page 18: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/18.jpg)
HARDWARE FINGERPRINTING RESULTS
18
![Page 19: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/19.jpg)
REGISTRY CHECK The registry entries contain hundreds of
references to the string containing the name of the VM, Ex. “Vmware”, VirtualPC and VirtualBox.
Checking the registry values for certain keys clearly reveals the VM presence.
19
![Page 20: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/20.jpg)
REGISTRY CHECK For Example:
HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\Scsi\Scsi Port1\Scsi Bus 0\Target Id 0\Logical Unit Id 0\IdentifierVMware, VMware Virtual S1.0 HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4D36E968-E325-11CE-BFC1-08002BE10318}\0000\DriverDesc VMware SCSI Controller HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4D36E968-E325-11CE-BFC1-08002BE10318}\0000\ProviderName VMware, Inc.
20
![Page 21: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/21.jpg)
PROCESS AND FILE CHECK Check - VM specific processes and files
presence
Eg. VBoxService.exe : In VirtualBox for
synchronization with host
drivers like “vboxhook.dll” and “vpcbus” driver present in %SYSDIR%/drivers
21
![Page 22: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/22.jpg)
MEMORY CHECK This involves looking for values of critical
operating system data structures. These data structures are relocated on a virtual
machine so that they do not conflict with the host system's copies.
Store Interrupt Descriptor Table (SIDT), Store Local Descriptor Table( SLDT), Store Global Descriptor Table (SGDT), Store Task Register (STR), Store Machine Status Word (SMSW)
Redpill.exe, ScoopyNG.exe use this method.
22
![Page 23: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/23.jpg)
TIMING ANALYSIS Obvious yet rare attack. Involves looking at a local Time Stamp
Counter (TSC) value. By noting down the time difference VM
presence is detected.
23
![Page 24: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/24.jpg)
VM COMMUNICATION CHANNEL CHECK This check involves detecting the presence of
a host-guest communication channel. IN instruction and a magic number ‘VMXh’ VmDetect.exe uses this check. Not applicable to VirtualPC and VirtualBox. Runs in VMware without exception.
24
![Page 25: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/25.jpg)
INVALID OPCODE CHECK Specific to VirtualPC
Uses certain opcodes for guest host communication
In host system raise exception and no exception in VirtualPC.
25
![Page 26: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/26.jpg)
VMWARE DETECTION
hardware details motherboard serial number, graphics card and
network adapter captions
Windows Management Instrumentation (WMI) contains classes hardware, display, registry etc.
Check for VM specific strings
HARDWARE FINGERPRINTING
26
![Page 27: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/27.jpg)
REGISTRY CHECK Windows Registry stores
configuration settings low-level operating system components Applications running
Check for Strings like “VirtualPC”, “VBOX”, “VirtualBox” value that is specific to the corresponding virtual
machine being testing on.
27
![Page 28: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/28.jpg)
PROCESS AND FILE CHECK Check - VM specific processes and files
presence
Eg. VBoxService.exe : In VirtualBox for
synchronization with host
drivers like “vboxhook.dll” and “vpcbus” driver present in %SYSDIR%/drivers
28
![Page 29: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/29.jpg)
MEMORY CHECK involves looking at the values of specific
memory locations
STR (Store Task Register) stores the selector segment of the TR register
(Task Register) in the specified operand (memory or other general purpose register).
Value specific in Virtual Machine
29
![Page 30: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/30.jpg)
INVALID OPCODE CHECK Specific to VirtualPC
Uses certain opcodes for guest host communication
In host system raise exception.
30
![Page 31: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/31.jpg)
DETECTION OF VM RUNNING LINUX Techniques: (tested on Vmware)
Hardware Fingerprinting
Dmesg check - prints the message buffer of the kernel
/proc file system check - interface to internal data structures in the kernel.
Communication channel check31
![Page 32: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/32.jpg)
DMESG AND /PROC FILE SYSTEM CHECK Dmesg - prints the message buffer of
the kernel
Shows diagnostic message showing presence of hardware during boot contain strings like “VMware”,
/proc file system - an interface to internal data structures in the kernel Contains system dependent information
32
![Page 33: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/33.jpg)
COMMUNICATION CHANNEL CHECK IN instruction
Raises exception ““EXCEPTION PRIV INSTRUCTION” in host
Runs in VMware without exception initiates guest to host communication by calling the
“IN” instruction.
33
![Page 34: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/34.jpg)
VMWAREDETECT Is the proof of concept tool. It employs the various VM detection
techniques to detect the presence of VMware virtual machine. Memory Check VM Communication Channel Check Hardware Fingerprinting Registry Check Timing Analysis
34
![Page 35: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/35.jpg)
VMWAREDETECT
35
![Page 36: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/36.jpg)
VIRTUALMACHINEDETECT - VIRTUALPC Check using all the methods
In VirtualPC In Native Machine Hardware Fingerprinting BIOS American Megatrenda L900781
Graphics Card Virtual PC Integration Components S3 Trio32/64
NVDIA GeForce 310
Baseboard Manufacturer Microsoft co-orporation LENOVO
System Name VIRTUALXP User-think USB Controller USB Virtualisation Bus Driver Intel® 5 Series /3400 …
Registry Check SCSI: HARDWARE\\DEVICEMAP\\Scsi\\Scsi Port 0\\Scsi Bus 0\\Target Id 0\\Logical Unit Id 0
Virtual HD Hitachi HDS721050CLA362
Control class for usb : SYSTEM\\ControlSet001\\Control\\Class\\{36FC9E60-C465-11CF-8056-444553540000}\\0000
USB Virtualisation Bus Driver Intel® 5 Series /3400 …
Control class for graphics: SYSTEM\\ControlSet001\\Control\\Class\\{4D36E968-E325-11CE-BFC1-08002BE10318}\\0000
Virtual PC Integration Components S3 Trio32/64
NVDIA GeForce 310
Controlset for cd/dvd drive: SYSTEM\\CurrentControlSet\\Enum\\IDE
Disk Virtual_HD____1._1__ Registry not found
Invalid Opcode Did not raise exception Raised exception File Check Vpcubus Driver (Virtual USB Bus Driver) Present Not Present
Vpcgbus Driver (Virtual PC Guest Bus Driver) Present Not Present
Vpcuhub Driver (Virtual USB Hub Driver) Present Not Present
36
![Page 37: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/37.jpg)
VIRTUALMACHINEDETECT - VIRTUALBOX
Virtual Box running windows Host Windows Machine
Hardware Fingerprinting
BIOS 0 L900781
Graphics Card Virtual Box Graphics Adapter NVDIA GeForce 310
N/W adapter AMD PCNET Family PCI Ethernet Adapter WAN Miniport(SSTP) …
Processor Null CPU1
USB Controller Std Open HCD USB Host Controller Intel® 5 Series /3400 …
Registry Check
Dsdt: : HARDWARE\\ACPI\\DSDT VBOX__ Registry not present
Scsi P0 : HARDWARE\\DEVICEMAP\\Scsi\\Scsi Port 0\\Scsi Bus
0\\Target Id 0\\Logical Unit Id 0
VBOX HARDDISK Hitachi HDS721050CLA362
Scsi P1: HARDWARE\\DEVICEMAP\\Scsi\\Scsi Port 1\\Scsi Bus
0\\Target Id 0\\Logical Unit Id 0
VBOX CD-ROM Null
Vedio Bios Version: HARDWARE\\DESCRIPTION\\System\
VideoBiosVersion
Oracle VM VirtualBox Version 4.1.2 VGA Bios Version 70.18.3E.00.05
System Bios Version: HARDWARE\\DESCRIPTION\\System\
SystemBiosVersion
VBOX-1 LENOVO-133
Instruction Check
STR (store task register) 28 0 40 00
File Check
VBOXHook.exe Present Not Present
VBOXTray Present Not Present
VBOXService.exe Present Not Present
37
![Page 38: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/38.jpg)
VIRTUAL MACHINE DETECT
In VB
38
![Page 39: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/39.jpg)
REMOTE DETECTION Scenario
There is access to the terminal of a system
need not be administrator access
WMIC ( Windows management instrumentation command line) is used
39
![Page 40: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/40.jpg)
MASKING DETECTION OF VM Using PIN API provided by Pin tool.
Can get all the instructions, the arguments and return value
Steps followed for masking Get each call made by binary. Check if matches a predefined list of calls. E.g.
RegEnumValueA Str LoadLibraryA __emit 40
![Page 41: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/41.jpg)
MASKING DETECTION OF VM Provide false values if
VM specific values are read (matched from predefined list)
Eg. Registry read returns the value “VBOX” Pin Tool gets the return value and modifies it in
runtime. Registry read function returns modified value
41
![Page 42: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/42.jpg)
MASKING DETECTION OF VM Binary does not detect – manipulated value
received.
This currently supports 64 and 32 bit OS 64 and 32 bit applications
42
![Page 43: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/43.jpg)
MASKING DETECTION OF VMLoad Binary Detect if the binary
is 64 or 32 bit.Display the detection and give option to user to change it.
Detect the OS as 64/32 bit.
Detect the Underlying VM
Virtual PC VirtualBox
Register Check masking
Invalid Opcode Check Masking
File Check Masking
File Check Masking
Register Check Masking
Instruction Check Masking
Feedback
Save to db for further analysis
Execution of loaded binary completed
43
![Page 44: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/44.jpg)
OUR APPROACH
44
![Page 45: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/45.jpg)
OUR APPROACH STEP 1: Maintain a list of all the hardware as well as registry querying API calls. Also maintain a list of all the VM specific instructions such as SIDT, SLDT, SGDT, STR, IN.
45
![Page 46: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/46.jpg)
OUR APPROACH Following is a partial list of API calls to be monitored. Hardware Querying APIs
SetupDiEnumDeviceInfo SetupDiGetDeviceInstanceId SetupDiGetDeviceRegistryProperty
Registry Querying APIs RegEnumKey RegEnumValue RegOpenKey RegQueryInfoKeyValue RegQueryMultipleValues RegQueryValue 46
![Page 47: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/47.jpg)
OUR APPROACHStep 2:
Perform dynamic binary instrumentation of the sample under test in order to obtain its low level information as well as to intercept all the API calls made by it.
We hook into the sample under test by means of .dll injection.
This is achieved using the pin framework. 47
![Page 48: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/48.jpg)
OUR APPROACHStep3:
Check to see if the sample under test makes a call or executes any of the monitored API calls or instructions respectively. If a match is found, set the OUTPUT to “Split Personality Malware Detected”. Also, log the activity and provide fake values to the sample so as to make it feel that it is running on a host system.
48
![Page 49: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/49.jpg)
IMPLEMENTATION Designed, implemented and tested
VMDetectGuard. Implemented in the framework
provided by the Pin tool released by Intel Corporation.
Pin is a tool for the instrumentation of programs.
We made use of its framework to intercept the various API calls and low level instructions executed by the sample under test. 49
![Page 50: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/50.jpg)
COUNTERING HARDWARE FINGERPRINTING Hardware emulation. APIs that query for BIOS, Motherboard,
Processor, Network Adapter. Ex. VM returns a value “none” for motherboard
serial number. VMDetectGuard returns a more appropriate string such as “.16LV3BS.CN70166983G1XF” instead.
50
![Page 51: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/51.jpg)
COUNTERING REGISTRY CHECK VMDetectGuard monitors registry querying
APIs such as the following:RegEnumKey RegEnumValueRegOpenKeyRegQueryInfoKeyValueRegQueryMultipleValuesRegQueryValue
If the output contains the string "VMware", our tool replaces this string with a more appropriate value that would have been returned on a non virtual system. 51
![Page 52: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/52.jpg)
COUNTERING MEMORY CHECK
SIDT, SLDT, and SGDT and STR instructions are monitored.
The values of the target registers are then changed appropriately with the values that would have been obtained on a host OS.
52
![Page 53: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/53.jpg)
COUNTERING MEMORY CHECK
53
![Page 54: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/54.jpg)
COUNTERING VM COMMUNICATION CHANNEL CHECK Monitor execution of the IN instruction. We change the value of the magic number . This leads to generation of “EXCEPTION PRIV
INSTRUCTION” exception.
54
![Page 55: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/55.jpg)
COUNTERING TIMING ANALYSIS
Instructions such as CPUID and RDTSC (Read Time Stamp Counter) are monitored.
The tool maintains a log of each type of instruction executed.
If the threshold value for a particular type of instruction is exceeded, it logs this activity too.
Sample is tricked by deleting the CPUID instruction and modifying the values of ebx, ecx, and edx.
55
![Page 56: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/56.jpg)
VMDETECTGUARD VMDetectGuard is our solution tool to counter
Split Personality Malware. VMDetectGuard runs in two different modes.
VM Guard Mode Non VM Guard Mode
56
![Page 57: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/57.jpg)
VMDETECTGUARD Output Generated by VMDetectGuard
Result: Split Personality malware detected/not detected.
VM Specific Log Instruction TraceSystem Call TraceRegistry TraceOpcode Mix Instruction CountDiff Tool Feature
57
![Page 58: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/58.jpg)
VMDETECTGUARD
58
![Page 59: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/59.jpg)
RESULTS & ANALYSIS
59
![Page 60: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/60.jpg)
REDPILL Red Pill is a very well known VM detection
tool by Rutkowska J. Runs a single machine language instruction
SIDT and analyses its result.
60
![Page 61: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/61.jpg)
61
![Page 62: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/62.jpg)
SCOOPYNGScoopyNG is a very well known tool
for VM detection developed by Klein T. More reliable tool for VM detection in
comparison to Red Pill. It performs the following checks
SIDT checkSLDT checkSGDT checkSTR check IN check (VMware communication channel)
62
![Page 63: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/63.jpg)
63
![Page 64: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/64.jpg)
VMDETECT This is another well known proof of concept
VM detecting sample that makes use of the VMware communication channel to detect VMware Presence.
64
![Page 65: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/65.jpg)
65
![Page 66: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/66.jpg)
BACKDOOR.WIN32.SDBOT.FMN
Captured this malware from the internet.
Employs Memory check and Timing Analysis mechanisms .
In the absence of VMDetectGuard: “This application cannot run under a Virtual Machine.”
In the presence of VMDetectGuard, it behaved malicious.
66
![Page 67: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/67.jpg)
67
![Page 68: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/68.jpg)
68
![Page 69: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/69.jpg)
VMDETECTGUARDRunning VMDetect in VirtualPC
Running VMDetect under masking tool
69
![Page 70: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/70.jpg)
VMDETECTGUARDRunning DetectionChecks in VirtualBox
Running DetectionChecks under masking tool
70
![Page 71: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/71.jpg)
OPTIMIZATION
Before (sec) After (sec) % decrease in time taken
VirtualBox 167.310 112.411 32.08%
VirtualPC 294.786 205.953 30.13%
VMware 418.642 299.158 28.54%
Running Firefox binary under masking tool, in all the three virtual machines.
71
![Page 72: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/72.jpg)
RESULTS Tested VMDetectGuard
Malwares captured from internet Proof of concept tools
The results obtained after testing is given in table.
72
![Page 73: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/73.jpg)
RESULTSBinary Detection Technique Used Run without tool Run under tool
Virtual BoxVBDetect: calls others binaries for individual checks within.
Registry Check File and Process Check Instruction Check
Detected VirtualBox Did not detect VirtualBox
Rebhip File and Process Check Runs benignly Runs maliciously
VirtualPCVPCDetect: calls others binaries for individual checks within.
Registry Check File and Process Check Invalid Opcode Check
Detected VirtualPC Did not detect VirtualPC
Backdoor.Win32.SdBot.fmn File and Process Check Invalid Opcode Check
Displays a message, “This application cannot run under a Virtual Machine
Ran maliciously
VMDetect Invalid Opcode Check Detects VirtualPC Does not detect VirtualPC
Trojen.Karsh-252 Invalid Opcode Check Displays a message, “This application cannot run under a Virtual Machine
Ran Maliciously73
![Page 74: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/74.jpg)
CONCLUSIONSplit Personality malware is on a
gradual rise.Lack of academic research in this
field.There does not exist any full-fledged
tool to counter Split Personality Malware.
We have designed, implemented and tested VMwareDetect, a proof of concept tool that detects the presence of Vmware. 74
![Page 75: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/75.jpg)
CONCLUSION We also successfully designed and
implemented VMDetectGuard, a tool to counter Split Personality malware.
It detects as well as tricks the split personality binaries.
Leads to the effective analysis of malware in the virtualized environment.
Increases productivity.
75
![Page 76: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/76.jpg)
SCOPE FOR FUTURE WORK Further testing of more number of malware. Tool is currently built for Vmware, VPC and
VB. Providing solutions for other analysis tools
such as debugger, sandbox etc. The work currently aims at Native binaries
Can be extended to Managed binaries Extended to other operating systems.
76
![Page 77: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/77.jpg)
REFERENCES Rutkowska J. (2004). “Red Pill”. http://invisiblethings.org/papers/redpill.html (Nov
20, 2010) Quist D, Smith V. (2005). “Detecting the Presence of Virtual Machines Using the
Local Data Table”. http://www.offensivecomputing.net/files/active/0/vm.pdf, (Nov 14, 2010)
Klein, T. (2005) “Scoopy Doo” .
http://www.trapkit.de/research/vmm/scoopydoo/index.html (Nov 4, 2010) P. Ferrie. “Attacks on Virtual Machines”. In Proceedings of the Association of Anti-
Virus Asia Researcher Conference, 2007. Zhu D. and Chin E. (2007). “Detection of VM-Aware Malware.”
http://radlab.cs.berkeley.edu/w/uploads/3/3d/Detecting_VM_Aware_Malware.pdf (Dec 1, 2010)
Carpenter M., Liston T., Skoudis E. (2007). "Hiding Virtualization from Attackers and Malware". IEEE Security and Privacy, June 2007
Lau B, Svajcer V. (2008). “Measuring virtual machine detection in malware using DSD tracer”. In the Proceedings of Virus Bulletin, 2008
Balzarotti D., Cova M., Karlberger C., Kruegel C, Kirda E, Vigna G. (2010).“Effcient Detection of Split Personalities in Malware”. In the Proceedings of 17th Annual Network and Distributed System Security Symposium (NDSS 2010),San Diego, February 2010
77
![Page 78: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/78.jpg)
REFERENCES VMware Inc. (2011), “VMware KB: Changing a MAC address in a Windows
virtual machine”. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008473 (Jan 15, 2010)
Pin (2004). “Pin - A Dynamic Binary Instrumentation Tool”. http://www.pintool.org/ (Jan 10, 2010)
Liston T. and Skoudis E. (2006). “On the Cutting Edge: Thwarting Virtual Machine Detection”. http://handlers.sans.org/tliston/ThwartingVMDetection_Liston_Skoudis.pdf (Nov 1, 2010)
Tiga, 2007. “Sourpill”, http://www.woodmann.com/collaborative/tools/index.php/SourPill_VM_Detector (Nov 4, 2010)
VMDetect (2005). “VmDetect, Detect if your program is running inside a Virtual Machine”. http://www.codeproject.com/KB/system/VmDetect.aspx (Jan 4, 2010)
Guizani, W., Marion, J.-Y., Reynaud-Plantey, D., & Bp, C. S. (2009). “Server-Side Dynamic Code Analysis”. Analysis, 2009
Omella A. (2006). “Methods for Virtual Machine Detection”. http://www.s21sec.com (Nov 24, 2010)
OECD, “Malicious Software (Malware): A Security Threat to Internet economy”, (2007) http://www.oecd.org/dataoecd/53/34/40724457.pdf (Oct 20, 2010)
78
![Page 79: SPLIT PERSONALITY MALWARE DETECTION AND DEFEATING IN POPULAR VIRTUAL MACHINES](https://reader035.vdocuments.net/reader035/viewer/2022062323/5681680d550346895ddd95b1/html5/thumbnails/79.jpg)
Thank You!
79