![Page 1: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/1.jpg)
Snowbird:Interactive Resource-Intensive
Applications Made Easy
H. Andrés Lagar-Cavilla*
Niraj Tolia† ‡, Eyal de Lara*,
M. Satyanarayanan‡ & Dave O’Hallaron‡
*University of Toronto, †HP Labs,‡Carnegie Mellon University
Middleware, November 2007
![Page 2: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/2.jpg)
Bimodal Applications
• Interactive– Cognitive phase
• Resource-Intensive– Crunch phase
• Digital Animation• Scientific Computing• Engineering Design• Bio/Pharma• Video Editing• ….
Maya
(digital animation)
![Page 3: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/3.jpg)
Dichotomy• Crunch (computation)
– Short completion time
– Remote computing resources
• Cognitive (interaction)– Crisp interactive performance
– User attention
![Page 4: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/4.jpg)
Execution Alternatives– Thick Clients (Desktop PCs)
Cognitive: excellent interactive performance
× Crunch: resource constrained
– Thin Clients (VNC, Remote Desktop) Crunch: use of remote resources (CPU, Data)
× Cognitive: latency and jitter impact interactions
– Custom Applications Pipeline: placement over different nodes
× Requires significant developer resources
![Page 5: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/5.jpg)
Execution Alternatives
Cognitive: excellent interactive performance
Crunch: use of remote resources (CPU, Data)
![Page 6: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/6.jpg)
Internet
ImprovedCompute Power
Compute Cluster
Snowbird: Agent Abstraction
Applications encapsulated within an agent– Agent: processes, libraries, IPC, OS, config data…– Migration: performance goals achieved by morphing
• a thin client for resource intensive crunch phases• a thick client for highly interactive cognitive phases
ImprovedData Access
User’s Desktop Data Repository
ImprovedInteractivity
Agent
![Page 7: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/7.jpg)
Bimodal Applications Made Easy
• Develop apps as monolithic blocks– Don’t worry about what executes where– Agent is migrated to satisfy each phase
• Seamless and transparent behavior
• Legacy support– Different OS’s (and versions/features)– Different languages (Fortran!)– No need for recompilation, relinking, etc…– Closed source apps just work
![Page 8: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/8.jpg)
Implementation
![Page 9: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/9.jpg)
Design Criteria• VM-based migration
– x86 interface most widely deployed– Transparently support OS, lang, etc…
• Internet scale– WAN migration– Long fat pipes: 50 Mbp/s…, 50-200 ms RTT
• Use of graphics HW acceleration a must– For the cognitive mode of bimodal apps
• Server-less design (P2P)– All hosts symmetric– Can execute anywhere
![Page 10: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/10.jpg)
Components
• VMM: suspend, resume, live migration – Xen 3.0.1
• Interaction-aware migration manager– Transparent
• Support for 3D acceleration in VMs– Vital for crisp interaction
• Virtual disk that maximizes locality– WAN area migration
![Page 11: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/11.jpg)
Agent Profiles: Migration Manager
• FSM that models an agent’s behavior• Provided by expert users, admins, or developers
– Default system-wide profile available
• Only deployment additional effort
CPUIntensive
cycles.org
NetIntensive
data.edu
InteractionIntensiveusr.home
Net > 4 Mbit/s
CPU > 95%
FPS < 20 &&
Input > 15
Internet
ImprovedCompute Power
Compute Cluster
ImprovedData Access
User’s Desktop Data Repository
ImprovedInteractivity
Agent
What’s this?
![Page 12: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/12.jpg)
Interaction-aware
• Novelty in our approach
• Measure the quality of the interactive response
• We use frames per seconds– More in sync with operations in bimodal apps– Stretch object, rotate, zoom, etc…– Pure latency not enough
• Ample space for future work
![Page 13: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/13.jpg)
Frames Per Second
• Non work-conserving (VNC): FPS = 2/latency
• Local: FPS = n/latency
• Work-conserving (X): FPS = n/(latency*k)
![Page 14: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/14.jpg)
The Rest
• VMGL: support for 3D acceleration in VMs– Coming live to this conference in 5 mins– Follow up: VEE 2007, >4K downloads
• WANDisk: virtual disk maximizes locality– Minimizes WAN communication– Simplifies state synchronization
• More details in paper
![Page 15: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/15.jpg)
Evaluation
![Page 16: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/16.jpg)
Benchmarks
Maya (closed source)ADF (closed source)
QuakeViz Kmenc15
• Broad set of domains
– Scientific Computing, Bio, Video Editing, Animation
• Closed and open source
• Straightforward installation
• Able to use generic profile on all four
• Partitioned mode for comparison
– ADF & Maya
![Page 17: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/17.jpg)
Methodology• Crunch + cognitive benchmarks• Performing “crunch” experiments is easy• Replaying long interactive traces is not
– Can’t expect a user to do it– Must be able to compare results
• VNC-Redux: record and replay interactive user sessions– Record input and screen state – This is matched during replay for accuracy
![Page 18: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/18.jpg)
ComputeServer
StorageServer User Desktop
LANEmulated
WAN
Experimental Setup
• Thick: No virtualization, on User Desktop– User Desktop: UP with graphics acceleration
• Thin: No virtualization, on Compute Server– Compute Server: 4-way SMP– 100 Mbit/s WAN, RTT: 33, 66, and 100 ms
• Partitioned– App-specific developer-brewed: Maya & ADF
• Snowbird– Agents are initially launched on User Desktop
– Xen+Migration Manager+VMGL+WANDisk
![Page 19: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/19.jpg)
Results: Crunch Phase
45
45
67
21
96
10
7
12
5
434
8
58
76
24
45
0
20
40
60
80
100
120
140
Maya QuakeViz ADF Kmenc15
Min
ute
s
ThinThickSnowbird 66Partitioned
Snowbird’s crunch performance– Much better than thick – Comparable to thin/partitioned
370…Ouch!
![Page 20: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/20.jpg)
• Better interactivity than thin clients• Is Snowbird any worse than a thick client?• > 20 FPS is ok, < 8 FPS is unusable
Results: Cognitive Phase
![Page 21: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/21.jpg)
Take Home Messages
• Bimodal applications – What they are and why they matter
• Thin clients are not almighty– There is no replacement for local interaction
• Best of both worlds: thick and thin clients– Necessary in an Internet world with remote computing
resources
• VM-based app migration is feasible– And with many advantages
• Future trends align well…
![Page 22: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/22.jpg)
Futurism
• More bandwidth: cheaper VM migration
• What about latency?– The earth is not shrinking– Speed of light is not increasing– More routers, overlays, firewalls– Toronto-London UK: ~109ms– Toronto-LA: ~84ms
• Insurmountable obstacle for thin clients
![Page 23: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/23.jpg)
Questions?
Thanks
H. Andrés [email protected]
Niraj Tolia, Eyal de Lara, Satya & Dave O’HallaronU of Toronto, HP Labs, Carnegie Mellon
![Page 24: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/24.jpg)
Backup
![Page 25: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/25.jpg)
VMGL: 3D Acceleration in VMs
• OpenGL virtualization
• Hardware specs closed, unavailable
• Focus instead on software standards– OpenGL -> cross-platform– Direct3D -> MS-only
• Intercept GL calls and forward them to the host. Proprietary driver renders there.
• More details: VEE 2007
![Page 26: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/26.jpg)
Open GL for X11 Apps
![Page 27: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/27.jpg)
VMGL for X11 Apps
![Page 28: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/28.jpg)
Admin VM
LocalChunk Store
WANDisk Manager
Kernel Module
WANDisk: Virtual Disk
foo.toronto.edu
Admin VM
LocalChunk Store
WANDisk Manager
Kernel Module
bar.cmu.edu
ChunkTable
ChunkMisses
Agent
Block DevOS
Application
baz.europe.org
Admin VM
LocalChunk Store
WANDisk Manager
Kernel Module
![Page 29: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/29.jpg)
Why Another Storage System?• Exploits Snowbird characteristics
• P2P model– No server interposition– Single-writer: simple metadata, no locks
• Minimizes WAN talk– Locality: persistent replicas– Differential transfers: rsync– On-demand fetching
![Page 30: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/30.jpg)
Experimental Setup
• Thick: No virtualization, on User Desktop– User Desktop: UP with graphics acceleration
• Thin: No virtualization, on Compute Server– Compute Server: 4-way SMP– 100 Mbit/s WAN, RTT: 33, 66, and 100 ms
• Partitioned– App-specific developer-brewed: Maya & ADF
• Snowbird– Agents are initially launched on User Desktop
![Page 31: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/31.jpg)
ADF Migration Time (secs)
Latency (ms) Detection Migration Pause
33 12.5 62 4.9
66 11.5 62 6.2
100 13.1 64.9 6.7
![Page 32: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/32.jpg)
Applicability of Snowbird
• Morphing time: our implementation• Speedup: application & resources• C: Crunch phase time locally
![Page 33: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/33.jpg)
Snowbird Limitations
• Parallelism up to SMP level– What about cluster-scale?
• SSE, 3DNow!– i.e. x86 is not that uniform
• Overlapping phases– Hysteresis, priority in migration manager
• Very short phases– Cost/benefit analysis
![Page 34: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/34.jpg)
Our Current Interests
• VM support for large parallel tasks– Relevant to commodity computing– Migration, the cloud, etc…
• How to measure interactive performance– Thin clients, desktop consolidation, VMs– Unknown effects for modern (3D-heavy) GUIs
![Page 35: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/35.jpg)
Really Backup Backup
![Page 36: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/36.jpg)
Keywords Of This Presentation
• Thin clients– Remote execution
• Interactive Performance– Thick clients
• Virtual machine migration– Application migration
• Bimodal applications – What they are and why they matter
![Page 37: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/37.jpg)
Frames Per Second
• Non work-conserving: same latency, less frames
• Local: FPS = frames/latency
• Work-conserving: same frames, more latency
![Page 38: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/38.jpg)
What We Need
• System support for bimodal applications
• Combine best of both worlds– Thick client, local execution
• Interaction
– Thin client, remote execution• Computation
• Make development easy
![Page 39: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/39.jpg)
Results: Crunch Phase
0
20
40
60
80
100
120
140
Maya QuakeViz ADF Kmenc15
Application
Tim
e (
Min
ute
s)
ThinThickSnowbird 33Snowbird 66Snowbird 100
Snowbird’s crunch performance– Much better than thick – Comparable to thin/partitioned
![Page 40: Snowbird: Interactive Resource-Intensive Applications Made Easy H. Andrés Lagar-Cavilla * Niraj Tolia † ‡, Eyal de Lara *, M. Satyanarayanan ‡ & Dave O’Hallaron](https://reader030.vdocuments.net/reader030/viewer/2022032723/56649d025503460f949d5b81/html5/thumbnails/40.jpg)
Talk Pointers
• Be more explicit demo: thin, frames
• Proxy FPS & input in migr manager