Dependable and Secure Remote Managementin IaaS CloudsTomohisa Egawa (Kyushu Institute of Technology)Naoki Nishimura (Kyushu Institute of Technology)Kenichi Kourai (Kyushu Institute of Technology)

2Remote VM Management in IaaS

• In-band remote management is usually used• A server runs in a user VM• The user connects to the server with VNC or SSH• However, users cannot access their VMs• when they fail the configuration of the firewall or

network• when the systems in the VMs crashUser

IaaS

VNC Server

User VM

VNC Client VM VM

3Out-of-band Remote Management

• Users can access their VMs via a VNC server in the management VM• The VNC server directly accesses virtual devices

of a user VM• e.g. virtual keyboard, virtual video card

• More dependable method• Not rely on the network of the user VM• Users can check kernel messages when the system

crashes

VNC Client

User

User VM

Management VMVNC

Server

virtual devices

virtual drivers

IaaS

4The Management VM is Not Always Trustworthy• Administrators in clouds may not be trusted• Users cannot know where their VMs are running• Lazy administrators cause vulnerable

management VM to be penetrated by outside attackers

• Malicious administrators can act as inside attackersManagemen

t VM

Data Center 1

Legitimate Administrat

or

VMVM

Management VM

Data Center 2

Malicious Administrat

or

VMVM

VM VM

VNC Client

User

5Information Leakage to the Management VM

• Attackers in the management VM can steal sensitive information of user VMs• Keystrokes from VNC clients• e.g. Password, credit card number, etc.

• Screen updates from user VMs• e.g. Displayed passwords, software keyboard, etc.

VNC Client

User User VMManagement VMVNC

Server Password &

Screen Caputur

e

malware

device drivers

virtual devices

6

FBCrypt

• FBCrypt encrypts the inputs and outputs between a VNC client and a user VM• The VMM decrypts keyboard inputs• The VMM encrypts screen updates • The attackers in the management VM cannot

steal sensitive information

VMMencrypt / decrypt

User VMManagement VMVNC

Server

VNC Clientencrypt

/ decrypt

User

virtual devices

device drivers

intercept

7

Protecting the VMM inside IaaS

• Remote attestation of the VMM• To guarantee the integrity of the VMM at the

boot time• Runtime memory protection of the VMM against the management VM• The management VM cannot access the code

and data of the VMM

Management VM

Verifier

VMM

Hardware

TPM

Signedmeasurement

Hash

8

Protecting User VMs inside IaaS

• The memory and CPU states of user VMs can be protected by the VMM• They are encrypted when the management VM

accesses• Secure runtime environment [Li et al. '2010]• VMCrypt [Tadokoro et al. '2012]

• The management VM cannot access decrypted inputs or unencrypted screen updates in user VMs

Management VM

VMM encrypt

User VM

memory

Keystroke &

Screen

9

Encryption of Keyboard Inputs

• The VMM decrypts a keyboard input encrypted by a VNC client• A virtual keyboard device passes it to the VMM• The VMM stores a decrypted one into the

keyboard queue• In para-virtualized Linux of Xen, the queue is in a

user VM• The VMM also converts a keysym (ASCII code) into

a keycode

VMM

Management VMVNC

ServerVNC

Clientencrypt

User VM

queue

decrypt

convert

virtual keyboa

rd

User

10

Confidentiality and Integrity

• FBCrypt uses AES-CTR as a stream cipher• Inputs are encrypted to a different stream every

time• They cannot perform even replay attacks• The VMM checks the integrity of the inputs with the MAC• A VNC client sends the MAC with encrypted

inputs• Attackers cannot insert arbitrary inputs

VMM

Management VMVNC

ServerVNC

Clientencrypt

User VM

queuevirtual keyboa

rd

User

integrity

check

decrypt &

convert

11

Replication of VRAM

• The VMM replicates VRAM of a user VM• A virtual video card accesses the replicated

VRAM• A user VM can use the original one without

modification

• The VMM encrypts the pixel data in the replicated VRAM• A VNC client decrypts updated pixel data

VMMencryp

t

VNC Client

User

decrypt

Management VMVNC

Server

User VM

video driverVRAMVRAM

video card

12

Synchronization of VRAMs

• The VMM synchronizes the original and replicated VRAMs• It monitors updates to the original VRAM• Update events are sent from a user VM to a virtual

video card• It copies updated areas to the replicated VRAM

with encryption

VMMencryp

t

VNC Clientdecry

pt

User User VMManagement VMVNC

Server

VRAM

video driver

video card

monitor

VRAM

13

Key Management

• A VNC client securely shares a session key with the VMM• A VNC client generates a session key on a VNC

connection• The key is encrypted with the VMM's public key• Only the VMM can decrypt it with its private key• The management VM cannot decrypt it

VNC Client

User User VM

VMMprivate key

session key

public key

Verifier Management VM

VNC Server

encrypt decry

pt Attestation

14

Experiments

• We conducted several experiments for FBCrypt• We attempted to eavesdrop on inputs and

outputs of VNC• We examined the overhead and the response

time in remote managementCPU Intel Core2Quad

Q9550 2.83GHz

Memory 4GB (512MB for guest)

NIC Gigabit Ethernet

VMM Xen 4.1.1

Management VM

Linux 3.1.1

User VM (PV) Linux 2.6.38.8

CPU Intel Core2Quad Q9550 2.83GHz

Memory 4GB

NIC Gigabit Ethernet

OS Linux 2.6.38..8

VNC client

Tight VNC Java Viewer 2.0.95

Server Client

15

Attempts at Eavesdropping

• We embedded malware into the VNC server in the management VM• Key logger• Screen capture• Demo

VNC Client

UserUser VMManagement VM

VNC Server

Key logger

Screen capture

virtual devices

device drivers

16

Overheads in a Keyboard Input• We measured overheads when a keyboard input is sent to a user VM• Client side: 802μs• Encryption, hash calculation• Most comes from sending extra data for the MAC

• Server side: 15μs• Decryption, hash calculation

Client-side Server-side0

200

400

600

800

1000[μs]

802

15

VMM

Management VMVNC

Server

VNC Clientencry

pt

User VM

queue

integrity check

decrypt &

convert

Client side

Server side

17Response time of a Keyboard Input

• We measured the time after typing a character until it is displayed in the VNC client • The increase of the response time: 7 ms (6%)• Decryption of a keyboard input• Encryption of pixel data for the displayed character

original FBCrypt0

20

40

60

80

100

120

140[ms]

113 120

VMM

Management VMVNC

Server

VNC Clientencry

pt

User VM

queue

integrity check

decrypt &

convert

Keystroke!

’A’

18Overheads in a Full-screen Update

• We measured overheads when the full screen of 800x600 was updated• Server side: 37 ms• Synchronization and encryption of VRAM

• Client side: 47 ms• Decryption of pixel data

Client-side Server-side0

10

20

30

40

50[ms] 47

37

VMMencryp

t

VNC Clientdecry

pt

User VMManagement VMVNC

Server

VRAMVRAM

Client side

Server side

19

Response Time of a Full-screen Update

• We measured the time from a keyboard input to a full-screen update by terminating a screen saver• The increase of the response time: 46ms

(31.5%)• The server-side overhead was hidden• because of the long timer interval used in the VNC

server

original FBCrypt0

50

100

150

200

250[ms]

146

192

VMMencryp

t

VNC Clientdecry

pt

User VM

VRAM

Management VMVNC

Server

VRAM

20

Related Work

• Xoar [Colp et al. '2010]• It runs a VNC server in an isolated VM• The security is not improved against insider

attacks• vSphere Hypervisor [VMware Inc.]• It runs a VNC server in the VMM• No information leakage via the management VM

• Attackers can steal sensitive information by compromising the VNC server

• CloudVisor [Zhang et al. '2011]• The security monitor underneath the VMM

encrypts the memory of the user VMs• It does not consider the security in remote

management

21

Conclusion

• We proposed FBCrypt for dependable and secure remote management in IaaS clouds• FBCrypt prevents information leakage via the

management VM in out-of-band remote management

• It encrypts the input and outputs between a VNC client and a user VM using the VMM

• Future work• To support fully-virtualized guest OSes such as

Windows• To apply FBCrypt to other remote management

software such as SSH