aws amazon web services georges akpoly cs252. overview of aws amazon elastic compute cloud (ec2)...

AWS

Amazon Web Services

Georges Akpoly

CS252

Overview of AWS

• Amazon Elastic Compute Cloud (EC2)

• Amazon Simple Storage Service (S3)

• Amazon Simple Queue Service (SQS)

• Amazon Mechanical Turk (MTurk)

• Alexa Web Services

• Amazon E-Commerce Service (ECS)

• Amazon Flexible Payments Service (FPS)

Starting Point

One machine:• Apache, IIS• PHP / Perl / Java / …• SQL Server, MySQL, Oracle

Worst Case Populariy

A Difficult Path

Problem: Backup

• High availability

• Redundancy

• Very big data sets

Problem: File System

• Important files have to be accessed by many servers

• NFS / Samba not practical

Problem: Messaging

• Many servers on different locations have to communicate

• Creating a reliable and scalable message queue is really hard

Don’t reinvent the wheel!

Amazon Web Services

S3 - Simple Storage Service

• Redundant storage - as much as you like

• max. 5 GB per object

• Organized in “Buckets”

• Web Service API for uploads

• Downloads via– Web Service– HTTP / HTTPS– BitTorrent

S3 - Buckets

• Unique over all S3

• Contains many key-value-metadata tupel

• Cannot contain other buckets!

• Key can contain “/”

S3 with AWS::S3• Upload

• Download

S3 with AWS::S3

• How do you run a website from S3– #1 caveat: no index.html support– So it’s only useful to offload traffic and storage

from your main site

S3 with AWS::S3

• Set up a bucket name ‘vanlug’

• Store a key in it ‘tux.png’– Make it publicly accessible

• Set up a CName ‘static.vanlug.com’ and points it to ‘vanlug.s3.amazonaws.com’

• Load up http://static.vanlug.com/tux.png

• That’s it.

S3 with AWS::S3

• JungleDisk– This application lets you mount S3 like it is a

local drive– Cross platform, support linux, Mac, Windows– Encryption can be pre or post upload.

EC2 - Elastic Compute Cloud

• Based on XEN virtualization• On demand virtual servers - controlled with Web

Service API

• Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3• ACLs for hosts/ports access control

EC2 - Elastic Compute Cloud

• How much does it cost– Traffic between EC2 instances and S3 is free.

Even if they are not in the same datacenter– $0.10 per instance-hour consumed ( or part of

an hour consumed)– Same prices for traffic and storage as S3

EC2 - Elastic Compute Cloud

• What you can do with it– You are the root, do anything– Install software

• Apache• VPN• UT2004, Call of Duty• Jabber

– Load kernel modules

SQS - Simple Queue Service

• Distributed, highly scalable and reliable message system

• Unlimited number of queues and messages

• Locking / releasing built-in

• 256 KB message payload

• Basic permission model

SQS gemSetup

Usage

MTurk

• Amazon uses Artificial Artificial Intelligence • Submitters enter Human Intelligence tasks into

the queue and workers can solve them• Workers can be tested for capabilities before

they are trusted• Submitters can refuse results if they find them

unsatisfactory• There are feedback system in place so Workers

can know if the submitters will pay

MTurk

• Some examples– Podcast transcription services– Image recognition for mapping service– Amazon has used it themselves for internal

projects

Resources

• Developer Connectionhttp://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=59

• Forumshttp://developer.amazonwebservices.com/connect/forumindex.jspa

• Resource Centerhttp://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=59

• 265,000+ developers

Questions???

Thank you for your attention