![Page 1: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/1.jpg)
Data-Intensive Cloud Control for GENI
GEC 8 demoOrca control framework
July 20, 2010
Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel
![Page 2: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/2.jpg)
2
DiCloud Project Overview
Support researchers conducting data-intensive experiments
• Sensors network storage volumes processing
Extend Orca with
• Data-centric Slices: storage as a first-class resource with Amazon
Simple Storage Service (S3) and Elastic Block Storage (EBS)
• Cloud Computing: connect Amazon Elastic Compute Cloud (EC2)
resources to GENI
![Page 3: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/3.jpg)
3
Demo – Step 1 (Allocate resources)
ViSE Sensor NetworkS3 (storage)
EC2 servers
Researcher creates a slice with:• weather radar • storage space on S3• compute servers on EC2
OrcaDiCloud
![Page 4: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/4.jpg)
4
Demo – Step 2 (Storing data)
ViSE Sensor NetworkS3 (storage)
Weather radar data is stored in S3DiCloudS3 Proxy
![Page 5: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/5.jpg)
5
Demo – Step 3 (Processing data)
S3 (storage)EC2 servers
• Get radar data from S3• Process data on EC2 server• Put generated image in S3
DiCloudAWS
accounting
00101010110101110011001010001110000
![Page 6: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/6.jpg)
6
Demo – Step 4 (Visualizing data)
S3 (storage)
DiCloudS3 Proxy
• Get generated image in S3
![Page 7: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/7.jpg)
7
Technical details
Implemented on top of Amazon Web Service API
DiCloud monitors
• EC2 Server hourly usage
• Network usage (in and out traffic using CloudWatch statistics)
• S3 storage space and put/get operations using a dedicated proxy
• EBS disk usage (storage space and number of IOs using
CloudWatch statistics)
All accounting operations are logged in a database
Resources are automatically revoked when budget has expired
![Page 8: Data-Intensive Cloud Control for GENI GEC 8 demo Orca control framework July 20, 2010 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel](https://reader030.vdocuments.net/reader030/viewer/2022032516/56649c7d5503460f94931c38/html5/thumbnails/8.jpg)
8
Challenges
Network connectivity with AWS
Monitoring
• CloudWatch does not differentiate between free and paying network
traffic or disk IO
• S3 Proxy in the cloud would save network traffic and cost
What security model for storage
• How to share data with others?
• Storage leases can last years
What budget to allocate to make this useful?