tao zou slides — general clouds
TRANSCRIPT
![Page 1: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/1.jpg)
Introduction to Compute Cloud
Tao Zou
CS 5220 Applications of Parallel Computers
![Page 2: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/2.jpg)
About me
• 3Rd year PhD student in Computer Science
– Databases/Cloud Computing/Distributed Systems
• Have been burning thousands of dollars in
– Amazon Web Services
– Microsoft Azure
– Rackspace Cloud
• Who is paying for the cost?
(Amazon Pays)
(Microsoft Pays)
(My advisor Pays)
![Page 3: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/3.jpg)
Where is the Cloud?
![Page 4: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/4.jpg)
Where is the Cloud?
• Datacenter
![Page 5: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/5.jpg)
Where is the Cloud?
• Datacenter
![Page 6: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/6.jpg)
What is a Cloud?
• Service, rather than a product
• Cloud Models:
Infrastructure as a Service
• Provides raw computing resources.
• Amazon EC2
• Rackspace
Platform as a Service
• Provides programming languages and tools.
• Microsoft Azure
• Google AppEngine
Software as a Service
• Provides applications.
• Salesforce.com
• Google Docs
Do anything Program anything Use anything
hardware focus, higher flexibility application focus, less flexibility,
![Page 7: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/7.jpg)
Amazon Web Service: Elastic Compute Cloud(EC2)
![Page 8: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/8.jpg)
Amazon Elastic Compute Cloud(EC2)
……
![Page 9: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/9.jpg)
Amazon Elastic Compute Cloud(EC2)
……
![Page 10: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/10.jpg)
Amazon Elastic Compute Cloud(EC2)
……
![Page 11: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/11.jpg)
Amazon Elastic Compute Cloud(EC2)
……
Give me 10 instances
![Page 12: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/12.jpg)
Amazon Elastic Compute Cloud(EC2)
……
Give me 10 instances
![Page 13: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/13.jpg)
Amazon Elastic Compute Cloud(EC2)
……
Give me 10 instances
Run Computation takes X hours
Shutdown all my instances
![Page 14: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/14.jpg)
Amazon Elastic Compute Cloud(EC2)
……
Give me 10 instances
Run Computation takes X hours
Shutdown all my instances
![Page 15: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/15.jpg)
Amazon Elastic Compute Cloud(EC2)
……
Give me 10 instances
Run Computation takes X hours
Shutdown all my instances
![Page 16: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/16.jpg)
EC2 Pricing Model: Pay As You Go
• Pay only for what you use
– Machine hours ( 10 ⋅ X )
– Type of instances
• Large instances total cost (0.34 ⋅ 10 ⋅ X )
![Page 17: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/17.jpg)
EC2 Pricing Model: Instance Types • Large Instance
– 7.5 GB memory – 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each) – 850 GB instance storage – I/O Performance: High
• Cluster Compute Quadruple Extra Large Instance
– 23 GB of memory – 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem”
architecture) – 1690 GB of instance storage – 64-bit platform – I/O Performance: Very High (10 Gigabit Ethernet)
• EC2 Compute Units = CPU capacity of a 1.0-1.2 GHz 2007 Xeon
processor
![Page 18: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/18.jpg)
Why Cloud?
• Time-to-solution
– 1000 cores * 1 hour = 1 core * 1000 hours
• Scalability: Can easily get 1000 cores
– For $170 per hour
• Instance Availability
– No queuing time
![Page 19: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/19.jpg)
How to Use EC2
![Page 20: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/20.jpg)
AWS Management Console
• OR using command line
![Page 21: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/21.jpg)
EC2: Start Instance
• Load OS Image
– Similar to VMware, VirtualBox
– But to all instances in parallel
• Takes < 1 minutes
![Page 22: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/22.jpg)
EC2: Start Instance
• ec2-run-instances AMI [-n INSTANCE_COUNT] …..
![Page 23: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/23.jpg)
EC2: Login using SSH
• ssh -i KeyPair [email protected]
• mpirun -hostfile ~/hosts -n 100 fishSim.exec
![Page 24: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/24.jpg)
EC2: Save Image
• ec2-bundle-instance
• Takes ~10 minutes (except Cluster/GPU Compute Instance)
• Typical Procedure – Start one instance from a default image
– Install MPI, OpenMP,…
– Option 1: Upload program/input data
– Save Image
– Start 100 instances from the saved image
– Option 2: Upload program/input data; Sync 100 instances
![Page 25: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/25.jpg)
EC2: Terminate Instances
• ec2-terminate-instances INSTANCEID [INSTANCEID ...]
![Page 26: Tao Zou slides — general clouds](https://reader030.vdocuments.net/reader030/viewer/2022021007/62039236da24ad121e4af5b2/html5/thumbnails/26.jpg)
How to get it for free
• AWS in Education – http://aws.amazon.com/education/
• For teaching: – CS 5300 - The Architecture of Large- Scale Information
Systems
• For research: – Quarterly research grant applications
• Next deadline: February 10, 2012
• Renewable: maximum of two grant awards per academic year