in-band detection of virtual machines
DESCRIPTION
In-Band Detection of Virtual Machines. Estefan Ortiz & Cory Hayes Computer Science and Engineering Graduate Operating Systems December 16, 2011. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
In-Band Detection of Virtual Machines
Estefan Ortiz & Cory Hayes
Computer Science and Engineering
Graduate Operating Systems
December 16, 2011
1
2
Introduction
Malicious programs (malware) need to know if they are in a virtual environment so they can modify their behavior and avoid detection
Related work Red Pill Tests: Examine byte-level behavior of instructions
for physical and emulated CPUs. If any disagreements in output, create one or more “red pills” that can avoid detection
SubVirt: Virtual machine-based rootkit installed underneath host OS that runs OS as a guest to remain nearly undetectable
3
Our Approach
Similar to Red Pill and SubVirt, but client-server based
Idea: Instead of monitoring system call discrepancies, analyze network data sent to/from physical and virtual machines
Goal: Determine if there are sufficient differences in network traffic to detect if a client/server is being run on a virtual machine
4
Goal
Client <-> Native TCP/IP Packet Client <-> Virtual Machine TCP/IP Packet
Byte 0
Byte n
Byte k1
Byte k2
Difference Found
5
General Setup
6
Actual Setup
Host Server(Apache)
Wireshark
Port
Client
Port
Switch
Network output saved for analysis
Functions as the “MITM”
7
Experiment Setup
Using Wireshark, capture and compare the raw info of TCP/IP packets sent back and forth between a client and a physical/virtual server running Apache Bits 1-160: IP Remainder: TCP
Virtual machine OS matches the OS of the host (Ubuntu-Ubuntu, Vista-Vista)
Use a small set of Matlab commands to send regular and malformed packets
Dynex 5-port 10/100/1000 Gigabit Ethernet Switch
8
Sample Captured Wireshark Output
8th Packet sent between Client & VM running Apache
8th Packet sent between Client & Host running Apache
VM
Client
Host
9
Metrics
Bit Difference Comparison: Fractional Hamming distance between two packets
10
Metrics (cont.)
Round trip time: Time from SYN request sent by client to received ACK from server
11
Metrics (cont.)*
Pairwise Packet Length Comparison: Number of concurrent packet pairs that differ in length
12
Experiment #1
Client: Windows Vista (4GB RAM, 2.6GHz) Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
On isolated switch network (no other traffic)
13
Exp. #1: Frac. Hamming Distance
14
Exp. #1: Round-trip Timing
15
Example: Packet #9
These bits correspond to the header length & flags in the TCP header
16
Experiment #2
Client: Mac (4GB RAM, 2.4GHz, MacOSX 10.6.8) Server: Windows Vista 32-bit w/ Apache Web Server 2.2 Server: Host OS Windows Vista: VirtualBox w/ Windows
Vista running Apache
On isolated switch network (no other traffic)
17
Exp. #2: Frac. Hamming Distance
18
Exp. #2: Round-trip Timing
19
Example: Packet #4
Destination Address in IP header
Flags in TCP header
20
Experiment #3
Client: Windows Vista (4GB RAM, 2.6GHz) Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
Both client and server on CVRL subnet (at ~3:00 am)
21
Exp. #3: Frac. Hamming Distance
22
Exp. #3: Round-trip Timing
23
Example: Packet #3
Destination Address in IP header
24
Experiment #4
Internet
Sprint Mobile Hotspot
Host Server(Apache)
Port
ND/CVRL subnet
Client
PortFIREWALL
25
Experiment #4
Client: Windows Vista (4GB RAM, 2.6GHz) Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 Server: Host OS Ubuntu: VirtualBox w/ Ubuntu running
Apache
Could not monitor packet information; only ping tests Varied number of bytes sent using ping Performed 100 per fixed byte amount Calculated avg. & std. dev
Executed at ~3:30 am
26
Exp. #4: Ping Timing
27
Conclusion
Examined packet information from a high level (packet-length) down to specific bit difference comparisons
Packet length provided no insight
Timing tests didn’t provide conclusive evidence of a connection to a virtual machine
Fractional hamming dist. provided first level of insight
Further analysis of differences at the bit level provided clues where to look for VM traces
28
Future Direction
Experiments 1-3 were conducted under somewhat “ideal” scenarios
More realistic approach would be packet analysis on multi-hop connections with knowledge of which sections of the TCP/IP packets to monitor