cloud computing lec7
Post on 06-Dec-2021
4 Views
Preview:
TRANSCRIPT
Basics of Cloud Computing – Lecture 7
Cloud Computing – Summary,
Aneka, some more PaaS and
leftoversleftovers
Satish Srirama
Outline
• Summarize what we have learnt
• Some more PaaS
– Aneka
– Google App Engine– Google App Engine
– Force.com
• Amazon SQS
24.05.2011 Satish Srirama 2
What is Cloud Computing?
• Computing as a utility
– Consumers pay based on their usage
• Cloud Computing characteristics
– Illusion of infinite resources
– No up-front cost
– Fine-grained billing (e.g. hourly)
• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”
Satish Srirama24.05.2011 3/30
Cloud Computing – Services - Recap
• Software as a Service – SaaS– A way to access applications
hosted on the web through your web browser
• Platform as a Service – PaaS– A pay-as-you-go model for IT
resources accessed over the
SaaS
Facebook, Flikr, Myspace.com,
Google maps API, Gmail
Level of
Abstraction
resources accessed over the Internet
• Infrastructure as a Service –IaaS– Use of commodity computers,
distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data
– Virtualization
Satish Srirama
PaaS
Google App Engine,
Force.com, Hadoop, Azure, Amazon S3, etc
IaaSAmazon EC2, Joyent
Accelerators, Nirvanix Storage
Delivery Network, etc.
24.05.2011 4
Scientific Computing Cloud (SciCloud)
• Distributed Systems Group owned private cloud infrastructure
• Eucalyptus setup
• Goal of the project• Goal of the project
– To Establishing a private cloud at universities
– To efficiently use the already existing resources of universities
– To address computationally intensive scientific, mathematical, and academic problems
• Hope it was a pleasant experience !!!
24.05.2011 5Satish Srirama
Economics of Cloud Providers –
Failures - Recap
• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter
• Caveat: User software has to adapt to failures• Caveat: User software has to adapt to failures
• Solution: Replicate data and computation– MapReduce & Distributed File System
• MapReduce = functional programming meets distributed processing on steroids – Not a new idea… dates back to the 50’s (or even 30’s)
24.05.2011 6/30 Satish Srirama
MapReduce
• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together
• The execution framework handles everything else…• Not quite…usually, programmers also specify:• Not quite…usually, programmers also specify:
partition (k’, number of partitions) → partition for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic
24.05.2011 7Satish Srirama
Hadoop Processing Model
• Create or allocate a cluster
• Put data onto the file system (HDFS)– Data is split into blocks
– Replicated and stored in the cluster
• Run your job• Run your job– Copy Map code to the allocated nodes
• Move computation to data, not data to computation
– Gather output of Map, sort and partition on key
– Run Reduce tasks
• Results are stored in the HDFS
24.05.2011 Satish Srirama 8
MapReduce Examples
• Distributed Grep
• Count of URL Access Frequency
• Reverse Web-Link Graph
• Inverted Index • Inverted Index
• Distributed Sort
• MapReduce in Information Retrieval
24.05.2011 Satish Srirama 9
Synchronization in Hadoop
• Approach 1: turn synchronization into an ordering problem– Sort keys into correct order of computation
– Partition key space so that each reducer gets the appropriate set of partial results
– Hold state in reducer across multiple key-value pairs to perform computationcomputation
– Illustrated by the “pairs” approach
• Approach 2: construct data structures that “bring the pieces together”– Each reducer receives all the data it needs to complete the
computation
– Illustrated by the “stripes” approach
24.05.2011 10Satish Srirama
SOME MORE PAAS
24.05.2011 Satish Srirama 11
Aneka: The Cloud Application
Platform for Resource-Intensive Apps
• PaaS from Melbourne Clouds Lab
• Available as Manjrasoft product
• Lightweight Container hosting multiple servicesservices
• All programming models available from within the same container (unlike others)
• SDK containing APIs for multiple programming models and tools
24.05.2011 Satish Srirama 12
Aneka - continued
• Runtime Environment for managing application execution management
• Suitable for– Development of Enterprise Cloud Applications– Development of Enterprise Cloud Applications
– Cloud enabling legacy applications
• Portability for Customer Apps:– Enterprise ↔ Public Clouds
– .NET/Win ↔ Mono/Linux
24.05.2011 Satish Srirama 13
Aneka - architecture
ContainerPersistence
Task
Model
Thread
Model
Map Reduce
ModelOther
Models Security
Programming Models
Software Development Kit
ManagementStudio
Application
Foundation Services
APIsDesign Explorer
Management Kit
AdministrationPortal
SLA-NegotiationWeb Services
ManagementWeb Services
ContainerPersistence
Task
Model
Thread
Model
Map Reduce
ModelOther
Models Security
Programming Models
Software Development Kit
ManagementStudio
Application
Foundation Services
APIsDesign Explorer
Management Kit
AdministrationPortal
SLA-NegotiationWeb Services
ManagementWeb Services
24.05.2011 Satish Srirama 14
Private Cloud
LAN network
Amazon
Microsoft GoogleIBM
Data Center
Hardware Profile Services
Persistence
.NET @ Windows Mono @ Linux
Security
Membership
Services
Reservation
Services
License
Services
Storage
Services
Accounting
Services
Fabric Services
Dynamic Resource Provisioning Services
Infrastructure
Physical Machines/Virtual Machines
Private Cloud
LAN network
Private Cloud
LAN network
Amazon
Microsoft GoogleIBM
Data Center
Amazon
Microsoft GoogleIBM
Data Center
Hardware Profile Services
Persistence
.NET @ Windows Mono @ Linux
Security
Membership
Services
Reservation
Services
License
Services
Storage
Services
Accounting
Services
Fabric Services
Dynamic Resource Provisioning Services
Infrastructure
Physical Machines/Virtual Machines
Aneka and Hour Glass Model
(many in one, one in many)
multiple applicationsThread Task ... MapReduce
multiple applications
24.05.2011 Satish Srirama 15
TCP/IP
Internet ArchitectureAneka Architecture
multiple networksMulti-core Cluster Grid Cloud
2100 2100 2100 2100
2100 2100 2100 2100
Aneka
Aneka: components
public DumbTask: ITask
{
…
public void Execute()
{
……
}
}
for(int i=0; i<n; i++)
{
…
DumbTask task = new DumbTask();
app.SubmitExecution(task);
}Executor Executor
Aneka enterprise
Cloud
work units
24.05.2011 Satish Srirama 16
}
Executor
Scheduler
Executor
Client
Agent
work units
internet
internet
Client
Agent
Aneka User Agent
Aneka Worker ServiceAneka Manager
Programming / Deployment Model
Aneka – management studio
24.05.2011 Satish Srirama 17
Aneka Support for InterCloud /
Federation of Resource Pools
• XenServer Pool– Provisioning over private Cloud managed by Xen Server
• VMWare Pool
Executor
Executors
Schedulerprivate enterprise
Executor
24.05.2011 Satish Srirama 18
• VMWare Pool– Provisioning over private Cloud managed by VMWare
• Amazon EC2 Pool– Provisioning over public Cloud provider: Amazon EC2
Executors/SchedulersClient Libraries
Scheduler
Public Cloud
publicly available resources
(physical and virtual)
internet
Private Cloud
private enterprise
network
VPN
(virtual resources)
Aneka cloud at UT
• We will soon be establishing Aneka cloud at
UT
– Probably in summer
– Already have the basic setup ready– Already have the basic setup ready
• Check for updates
• If anybody is interested; contact me directly
– Can be taken as an internship topic
24.05.2011 Satish Srirama 19
Google AppEngine
24.05.2011 Satish Srirama 20
What is AppEngine?
• Google platform to build web application on
the Cloud
• Example of PaaS
• Does one thing well: running web apps• Does one thing well: running web apps
• Request → Processing → Response
2124.05.2011 Satish Srirama
No More LAMP
2224.05.2011 Satish Srirama
Why App Engine
2324.05.2011 Satish Srirama
Application Life Cycle
2424.05.2011 Satish Srirama
Development Life cycle
2524.05.2011 Satish Srirama
Physical Deployment Diagram
2624.05.2011 Satish Srirama
APIs
2724.05.2011 Satish Srirama
End User Administration
Access Control Authentication / SSO Services
Web Services API
Workflow Engine
A B D
C
Powerful Application Service
Multi-Device Multi-LanguageMulti-
CurrencyReporting &
AnalyticsSearchEngine
28
Force.com - cloud platform for business apps
24.05.2011 Satish Srirama
Infrastructure
Services
Network
Storage
Operating System
Database
Application
Services
Authentication
Security
Sharing
Integration
Operations
Services
Availability
Monitoring
Patch Mgmt
Upgrades
IT
Force.com Lets You Focus on
Innovation
You
InnovationApp Server
Web Server
Data Center
Disaster Recovery
Customization
Web Services
API
Multi-Language
Multi-Currency
Workflow
Analytics
Multi-Device
Messaging
Search
Backup
NOC
Sales force2924.05.2011 Satish Srirama
3024.05.2011 Satish Srirama
Create any table
Add any field
Build any relationship
Database as a Service
Build any relationship
Security & sharing
Automatic tuning
Automatic backups
Automatic upgrades
3124.05.2011 Satish Srirama
Integration as a Service
3224.05.2011 Satish Srirama
History Tracking
Workflow Rules
Logic As A Service
Declarative Logic
Formula-Based LogicFormula Fields
Procedural Logic
Formula Fields
Validation Rules
Apex Triggers
Apex Classes
Apex Web Services
3324.05.2011 Satish Srirama
User Interface as a Service
UI generated
automatically,
no technical
skills required
UI generated
by developer /
technologist
Profile /
Record Type /
Page Layout
mapping
controls UI
behavior
Full
Page Layout Visualforce
Limited
control over
look and feel,
but all UIs are
consistent
Full
control of UI
behavior
Point and click to
add Custom
Buttons and
Embedded
Mashups
3424.05.2011 Satish Srirama
LEFTOVERS
24.05.2011 Satish Srirama 35
Amazon Simple Queue Service (SQS)
• A reliable, highly scalable hosted distributed queue
for storing messages as they travel between
computers.
• Amazon's web-scale messaging infrastructure as a
web service
• Platform-agnostic, allowing any computer on the
Internet to add or read messages without any
installed software or special firewall configurations
24.05.2011 36Satish Srirama
Amazon SQS Highlights
• Reliable
– Runs within Amazon's high-availability data centers.
– Messages are stored redundantly across multiple servers and data centers.
• Simple: Only Six Methods• Simple: Only Six Methods
• Scalable
• Secure: Strong access controls
• $0.01 per 10,000 messages sent ($0.000001 per message sent)
• $0.20 per GB of data transferred
24.05.2011 37Satish Srirama
Next Week
• All the best with the exams
• On 25th and 26th May 2011, we will have the
project presentations
• Examination on 31st May• Examination on 31st May
– Duration – 2 hours
– 10:15 – 12:15
– Room probably the lecture hall
24.05.2011 Satish Srirama 38
References
• Aneka
http://www.manjrasoft.com/aneka_architectu
re.html
• Google App Engine • Google App Engine
http://code.google.com/appengine/
• Force.com http://www.salesforce.com/force/
• Amazon SQS http://aws.amazon.com/sqs/
24.05.2011 Satish Srirama 39
top related