sponsored by the national science foundation university of massachusetts amherst november 2 nd, 2011...
TRANSCRIPT
Sponsored by the National Science Foundation
University of Massachusetts AmherstNovember 2nd, 2011
GENI DiCloud
Sponsored by the National Science Foundation 2
Project Summary
• DiCloud enables GENI experiments to incorporate cloud computing resources– Focuses on Amazon web services– EC2 (VMs), EBS (block storage), S3 (object storage)
• How are cloud resources different from other GENI resources?– They cost money!
• Requires active Amazon account and credit card
– Require authentication with Amazon
Sponsored by the National Science Foundation 3
Project Summary
• DiCloud tracks per GENI-user Amazon costs– Amazon charges can be complicated
• Pay per VM time used• Pay for data transfer in and out• Pay for IOs to block devices
– No current facilities for tracking per user costs
• Enables controlled access to cloud resources– Stand-alone: setup your own DiCloud instance to track
your own costs– GENI CF-mode: place “underneath” GENI CF, where
AM pays the bills, but gives GENI users access
Sponsored by the National Science Foundation 4
Elastic Compute Cloud (EC2)
• Rent VMs from Amazon “on-demand”– http://wwww.amazon.com/ec2
Sponsored by the National Science Foundation 5
Elastic Compute Cloud (EC2)
• Rent VMs from Amazon “on-demand”– Pay per hour based on VM performance– Pay for data transfer in/out
Sponsored by the National Science Foundation 6
Elastic Block Store (EBS)
• Rent block storage from Amazon– Pay per GB-month (currently $0.10)– Also pay per 1 million IO requests (currently $0.10)
Sponsored by the National Science Foundation 7
Simple Storage Service (S3)
• Rent object storage from Amazon– Pay per GB-month – Pay for data transfer in/out
Sponsored by the National Science Foundation 8
Web Portal
Sponsored by the National Science Foundation 9
Web Portal
Sponsored by the National Science Foundation 10
Web Portal
Sponsored by the National Science Foundation 11
Tutorial Step 5: Web Portal
Sponsored by the National Science Foundation 12
Tutorial Step 5: Web Portal• Now logout and log back in as user “geni”
– Then click on “Request EC2 resources”– Use ami-013d6c44 as the AMI id
Sponsored by the National Science Foundation 13
Tutorial Step 5: Web Portal
• Now go back to your user page– Click on status next to the VM you just created
Sponsored by the National Science Foundation 14
Tutorial Step 5: Web Portal
• Now lets create an S3 bucket– Click on “Request new S3 buckets”
Sponsored by the National Science Foundation 15
Tutorial Step 5: Web Portal
• Once back at the user page, put something in your bucket– Select the s3_test file in ~/Tutorials/DiCloud/s3_test
Sponsored by the National Science Foundation 16
Tutorial Step 5: Web Portal
• Now get that same thing back from your bucket
Sponsored by the National Science Foundation 17
Tutorial Step 5: Web Portal
• Now lets create an EBS volume and attach it to our VM– Click request new EBS volumes– Make it 1GB, in us-west-1, in us-west-1<availability num>
Sponsored by the National Science Foundation 18
Tutorial Step 5: Web Portal
• Attach the volume to our VM– Click on attach, enter instance id, and sdg for device
Sponsored by the National Science Foundation 19
Tutorial Step 5: Web Portal
• If we ssh into our VM we can see that the volume is now attached
Sponsored by the National Science Foundation 20
Demo Data Flow
• Dynamic end-to-end Nowcasting – Mapping Nowcast Workflows onto GENI
Archival Storage
Radar Nodes
“raw” live data
Upstream LDM feed
archived netcdf data
Nowcast Processing
aggregatedmulti-radar data
Post to Web
Nowcast images for display
Sponsored by the National Science Foundation 21
Multi-radar NetCDF Data
Nowcast Processing
1. DiCloud Archival Service (S3)2. LDM Data Feed (EC2)
“raw” live data
Generate “raw” live dataViSE/CASA radar nodes
http://stb.ece.uprm.edu/current.jsp
Use proxy to track usage-based spending on Amazon and enforce quotas and limits
http://geni.cs.umass.edu/vise/dicloud.php
1. Ingest mulit-radar data feeds2. Merge and grid multi-radar data2. Generate 1min, 5min, and 10min Nowcasts3. Send results over NLR to Umass4. Repeat
ViSE views steerable radars as shared, virtualized resources
http://geni.cs.umass.edu/vise
Nowcast images for display
Sponsored by the National Science Foundation 22
Demo Overview
• Dynamic end-to-end Nowcasting on GENI – Slice of sensing, networking, computing, and storage
Archival Storage
Radar Nodes
Upstream LDM feed
Nowcast Processing
Post to Web
Generate “raw” dataViSE/CASA radars
Archive radar dataAmazon S3
“raw” live data
Archived data availableto downstream nodes
archived netcdf data
aggregatedmulti-radar data
Nowcast images for display
GenerateNowcasts