from sandbox to playground: dynamic virtual environments in the grid

From Sandbox to Playground:

Dynamic Virtual Environments in the Grid

Kate [email protected]

Argonne National LaboratoryKarl Doering

University of California, Riverside Ian Foster

Argonne National Laboratory

Grid 2004 Kate Keahey

Realizing the Grid Vision

Quality of Service Protocol, agreement,

advance reservation The ability to enforce

what was agreed on

Quality of Life Being able to find the

right configuration on the Grid


Quality of Service

Some form of control over remote nodes Enforcement of multiple qualities

CPU, disk, memory, network traffic… More than per-process enforcement

Process group: a master process starts other processes

Dynamically modifiable to reflect changing policies and state in the Grid

Not just quality of service Quality of Protection, etc… QoX


Quality of Life

The right node configuration is hard to find Operating system and architectural differences

Different Linux distributions 64 bit vs 32 bit

Library signature and versioning The ability to customize a remote execution

environment Effortless configuration of remote nodes

Subject to policies Quality of Life for multiple groups of Grid users

Avoiding maintenance nightmare, etc.


We Need a Sandbox A configurable execution environment, container

Virtualizes Grid Node Configuration Sandbox = Dynamic Virtual Environment (DVE)

We need to be able to create and manage it Quota, termination, etc.

requirements available technology

solutions

How can DVEs be implemented? Relevance to our needs, quality of solution, etc.


DVE: Interfaces Implemented as Grid Services

OGSI, WSRF Factory

Creates and configures a DVE in implementation-specific way

e.g., dynamic account, deploys a VM Writes/configures access and management policy

E.g., modify the GT3 gridmapfile

DVE Service Interface providing DVE management

E.g., explicit or soft-state termination (implies policy updates) Access policy management

Allows for inspecting and modifying DVE properties E.g., hardware properties such as quota or software configuration


DVE Implementations: Requirements What is a “container”? General

Not require users to e.g., use a specific language Non-invasive

Proof-carrying code, etc. Strong protection environment

Otherwise users won’t trust sites and sites won’t trust users Isolate users from each other

Fine-grain enforcement Configurable architecture, software, environment

Configurable environment throughout the software stack Application software/libraries/licenses

Potentially: execution state Allow migration


DVEs and the Globus Toolkit

C

lien

t

(1) DN

(4) GSH

local DVEimplementationsetuid

(3)

gridmapfile

(5)

GRAM(6) Request+GSH

(2)

DVESservice PEP

DVEFactory Service PEP


DVE Implementations Unix accounts

Pros: efficient, ubiquitous Cons: very limited enforcement Enforcement properties can be improved if used in

conjunction with other technologies setrlimit, DSRT, chroot, chown, and others

Sandboxes VServer: protection, sharing and fine-grain

enforcement Pros: efficient, fine-grain enforcement, typically very

lightweight Cons: limited state enforcement, configuration flexibility Adjustments needed to fully leverage fine-grain

enforcement


DVE Implementations (cntd) Virtual Machines

VMware (not evaluated, but very promising: Xen) Pros:

Flexibility (run linux on linux, 32 on 64-bit, etc.) Enhanced security, audit forensics, etc. Great user state management Freezing/migration Customized environment A promising distribution/deployment tool

Cons: Potential for being less efficient (emulation) Potential for resource overhead Poor implementation of sharing, relatively little enforcement (but can

be combined with other technologies for enforcement) Maturity issues

The potential is excellent, but needs more work


The Need for Speed

0

0.2

0.4

0.6

0.8

1

1.2

110100jt 110105jt 110109jt

UNIX acctVserverVMware

Comparison using the Fusion EFIT application


Other efficiency concerns

Startup time

Resource usage overhead Memory use: VMware: 24MB + 1 MB per 32 MB

memory allocated Disk use: large for VMware

Table 1: DVE create/destroy times

Linux VServer VMware

Create 100 ms 360 ms 14-52 sec

Destroy 70 ms 200 ms 3-38 sec


Enforcement Capabilities

Unix account VServer VMware

CPU usage (sec) Via setrlimit() Not at present, but could be added

Not enforced

CPU usage (%) Not enforced Limited: no VServer can starve another

Not in VMware Workstation

Disk space usage Dynamically(per-user

quotas)

Dynamically (per context quotas)

Statically (virtual disks)

Memory usage No Not at present, but could be added

Statically

Network usage No Dynamically Dynamically


DVE Comparison Dynamic Accounts

Adduser versus pooled accounts A limited but one that is here to stay… at least for now

VServer Interesting: sharing and efficiency

VMware No sharing Least efficient Migration, flexibility, etc.

General criteria Efficiency: very acceptable, also see Xen Enforcement: uneven, needs more research Virtual Machines lead as far as configurability and user state representation Sharing

Potential for replication One VM per machine model?


Implementation Status

Prototype available (GT 3.2) Karl Doering:

http://www-unix.mcs.anl.gov/~keahey/DS/DynamicSessions.htm

GT4 Implementation adduser versus account pools Better policy handling

Virtual machines and other implementations Work in progress SC04 poster:

P05: “Quality of Life in the Grids: VMs Meet Bioinformatics Applications”, with T. Freeman and D. Galron




From Sandbox to Workspace

Virtual Workspaces VWs are represented by an ontology description

Virtual resource characteristics, software stack, etc. Potentially integrating community policy They can be copied, etc.

They can be implemented using different technologies

They can be customized by the user Deployed, managed and terminated in

implementation-specific way Entails some changes to the architecture


Virtual Workspaces in the Grids

Clie

nt

request

VW EPR

inspect and manage

deploy & suspend

use existing VW Create VW

VW Factory

VW Repository

VW Manager

create new VW

ResourceVW

start program


From Sandbox to Playground How will this affect interactions in the Grid?

Other than add many new capabilities A larger role for the virtual organization

Account screening process: resource owner -> virtual organization Should a VO be a legal entity? Needs new privileges if takes on more responsibility

Administration of VWs VW repository and other services, potentially VW certification

Sharing between VWs More policies

Changes to many Grid services May depend on the implementation we use Security, networking, potentially others

Top-down model for building a Grid Define a Grid in terms of requirements


Conclusions For Grids to scale we need a way to create and manage

remote environments in the dynamically and effortlessly Implementations will vary

Virtual is the new Real! VMs present a very compelling solution…

Efficiency, flexibility, migration, etc. …and introduce some new challenges

New services, different models of sharing, security, etc.

A growing role for Virtual Organizations Policy, Policy, Policy…

Policy of resource owners, VOs, users… Using WS-Agreement to negotiate virtual workspaces? Have we exchanged one problem for another?

www.mcs.anl.gov/~keahey

from sandbox to playground: dynamic virtual environments in the grid

Documents

dve managemente

right configuration

dve propertiese

limited state enforcement

finegrain enforcementpros

grid servicesogsi

softstate termination

trust sites