Multifaceted Resource Management in Virtualized Providers
Íñigo Goiri
PhD DefenseJune 14th, 2011
Advisors:Jordi Guitart and Jordi Torres
2
Motivation
Book Store Internet
Companies offer their servicesover the Internet
3
Service Providers over the Internet
Book Store Internet
Number of usersincreases
Provider requires more resourcesLoad increases
4
Service Providers over the Internet
Book Store Internet
Number of usersdecreases
Provider requires fewer resources
5
Virtualization on Service Providers
Book Store Internet
Database
Web ServerApplication
ServerDatabaseApplication
ServerWeb Server
Database
Web Server
DDBBApplication
Server WebServerApplication
ServerDDBB DDBBWebServer WebServerEncapsulate tasks in
Virtual Machines
VM VM VM VM VM VM VM VM
Load is unbalanced
6
Providing Virtual Machines
Book Store InternetVirtualizedProvider
Desktop
WebServer
HPC
VM VM VM VM VM VM VM VM
VirtualizedProvider
Provider offers its idleInfrastructure as a Service
(IaaS)
We can also offerapplications encapsulated in VMs
7
Managing VirtualizedProvider’s Resources
Balanced loadVMs get enough resources
Energy consumption is high
8
Managing VirtualizedProvider’s Resources
Unbalanced loadSome VMs don’t get enough resources
Energy consumption is lower
9
Managing VirtualizedProvider’s Resources
ChallengeEfficiently Manage Virtualized Provider’s Resources
Maximize Provider’s Profit
Every VM gets enough resourcesEnergy consumption is lower
10
Contents
• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation
• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation
• Conclusion
11
Multifaceted Scheduling
• Cost-benefit model– Multiple facets to consider– Aggregate facets into costs– Consider impact of facets on other facets
• Scheduling policy– Maximize provider profit
• Evaluation
12
Multiple Facets to Consider
1. Service Level Agreement2. Virtualization Management3. Energy Consumption4. Infrastructure Cost
13
1. Service Level Agreement
• Contract between user and provider• User pays for resources– Pay as you go: SLARevenue(t(VM))
• If the provider does not provide the QoS– SLA Penalty: SLAPenalty(VM)
VirtualizedProvider
Revenue
Penalties
14
1. Service Level AgreementSupported Applications
• Batch– Deadlines
• Example– HPC jobs
• Service– Uptime– Performance
• Example– Web Servers
Task 1
Task 3
Time Time
Task 2
Resp
onse
Tim
e
15
1. Service Level AgreementResource Heterogeneity
• Xeon processor– High energy consumption– High performance
• Atom processor– Low energy consumption– Low performance
Task A
Task B
Task C
Task A
Task B
Task C
Max
Wh0 Max
Wh0
16
1. Service Level Agreement
• Estimate SLA Penalties– Actions may imply future violations– SLAPenalty’(Host, VM)
• Factors that might provoke violations– Runtime overhead– Slow host– High host utilization
• Other facets evaluate penalty estimations
17
Time
2. Virtualization Management
• Overhead to manage Virtual Machines– Start VMs– Migrate VMs between nodes
Time
18
Off
2. Virtualization Management
• Overhead to manage Virtual Machines– Start VMs– Migrate VMs between nodes
Time
Off
Mig
ratio
nM
igra
tion
19
2. Virtualization Management
• Overhead implies– Extra time to the VM– Extra load to the Host
• It can imply SLA penalties– Violate deadline– No enough resources to provide performance
• Estimate SLA penalty for every action(Time, Load) → SLAPenalty’(Host, VM) → €
20
3. Energy ConsumptionEnergy vs. SLA
• Low Consolidation– Consume a lot of energy– Fulfill SLA
• High Consolidation– Save energy– Violate SLA
21
3. Energy Consumption vs. SLABatch Type
• Low Consolidation
• High Consolidation
Off
Time
TimeTime
Max
Wh0
Max
Wh0 Max
Wh0
Max
Wh0
22
3. Energy Consumption vs. SLAService Type
• Low Consolidation
• High Consolidation
Off
Time
Time
Time
Max
Wh0
Max
Wh0 Max
Wh0
Max
Wh0
23
3. Energy Consumption• Energy cost
Wh → €• SLA penalties
Host Utilization→ SLAPenalty’(Host, VM) → €
24
4. Infrastructure Cost
• Provider owns infrastructure– Servers– Air conditioners– Racks…
• Capital Expense (CAPEX)• Cost is amortized over time– Provider has already paid for the hardware
€/Period
25
Multiple Facets to ConsiderCalculate profit of VM at Host
1. Service Level Agreement:SLARevenue(VM) +€
2. Virtualization Management: Time: SLAPenalty’(VM) -€Load: SLAPenalty’(VMs in Host) due to VM -€
3. Energy Consumption: Energy consumed by VM at Host -€
SLA Penalty’(VMs in Host) -€
4. Infrastructure Cost:Cost of Host running VM -€
Total Profit€
26
Scheduling Policy• Decide best VM placement• Maximize provider profit– Hill Climbing (Greedy)
• When to schedule?– System changes– Periodically
• Model Virtualized Provider as a matrix– VM x Host cells– Each is profit of placing VM in Host– Use cost-benefit model
27
OFF
Queue:
Scheduling Policy
1 2
3 4 5
6 7
A
B
C
28
Scheduling Policy
1.5€ 0.2€
-0.5€
1.2€
-0.1€0.2€
1.3€
1.2€0.7€
-1.5€-∞
-∞-∞ -∞
-∞
-0.5€
-0.3€
0.1€ 0.3€
-0.7€ 0.2€
1.5€ 0.2€
-0.5€
1.2€
-0.1€0.7€
0.2€
-0.2€0.7€
1.2€-∞
-∞-∞ -∞
-∞
-0.5€
-0.3€
0.1€ 0.3€
-0.7€ 0.2€
Calculate cost of allocatingEvery VM at every Host
1.5€ 0.2€
-0.5€
1.2€
-0.1€0.2€
1.3€
1.2€0.7€
-1.5€-∞
-∞-∞ -∞
-∞
-0.5€
-0.3€
0.1€ 0.3€
-0.7€ 0.2€
Recalculate cost of allocatingEvery VM at every Host
Schedule VM in maximumProfit placement
When cost is minimizedDispatch VMs
1 2 3 4 5 6 7
A
B
C
After multiple iterations…
29
Scheduling Policy
OFF
Queue:
1 2
3 4 5
6 7
A
B
C
30
EvaluationMultifaceted Scheduling
• One week heterogeneous workload– Batch: Grid5000– Service: SPECWeb2005
• SLA metrics– Batch: Deadline (Added +20% Base Runtime)– Service: Performance (Response Time)
• Provider with 65 nodes– Enough to satisfy workload peaks
31
1. Backfilling + Migration– Backfill VMs– Migrate to consolidate
2. Perfect SLA– Analytical (NP)– Perfect SLA fulfillment– Optimal energy consumption
3. Our proposal– Backfilling + Migration– Aggregate multiple facets– Uses cost-benefit model– Maximize profit
13
2
OFF
1
2
EvaluationScheduling Policies
3
OFF
4
4
32
Evaluation
Energy consumption SLA fulfillment
+ ← Consolidation → - + ← Consolidation → -
33
Evaluation
Energy Saving SLA Fulfillment Profit Increase0
20
40
60
80
100
120Backfilling + MigrationPerfect SLAOur proposal
%
34
Evaluation
• Power consumption over time
35
Multifaceted SchedulingLimitations
1. Service Level Agreement2. Virtualization Management3. Energy Consumption4. Infrastructure Cost– Fixed costs
36
Contents
• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation
• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation
• Conclusion
37
Multiprovider SchedulingOutsourcing
• Infrastructure cost– Capital Expenses (CAPEX)
Solution:CAPEX → OPEX
External Provider
38
Multiprovider SchedulingOutsourcing
• Reduce provider infrastructure (CAPEX)• Multifaceted Scheduling + Outsourcing– Add outsourcing cost (OPEX)– Slower VM creation– Limited VM management
External Provider
39
EvaluationOutsourcing
• Add outsourcing to “Multifaceted Scheduling”• Same environment– Reduce local resources: 65 → 20 → 0 nodes– 20 nodes is enough to provide the average
• External provider: EC2 US– 0.085 €/hour– 5 minutes to start a VM
40
EvaluationOutsourcing
Energy Saving SLA Fulfillment Profit Increase
-60
-40
-20
0
20
40
60
80
10065 Nodes65 Nodes + Outsourcing20 Nodes20 Nodes + Outsourcing0 Nodes + Outsourcing
%
41
Multiprovider SchedulingFederation
• How many local resources?– Optimal number of resources– Change capacity planning
• How to schedule?– New actuators– New trade-offs
• Characterize provider profitability– Cost-benefit model
42
Federated Provider Model
• Multidimensional problem• Evaluate provider profile– Provider capabilities– Expected workload– VM pricing
• Evaluate costs– CAPEX: Infrastructure– OPEX: Energy, Cooling,…
43
Multiprovider SchedulingCharacterize federation
Leverage federated provider model for:Phase 0. Capacity planning– Provider building and setup process– Decide optimal number of nodes
Phase 1. Scheduling– Online process– Decide actions to take
44
Phase 0. Capacity Planning• Load is variable over time• If infrastructure costs are fixed– Underprovision: Cannot support peaks– Overprovision: Underutilized resources
45
Phase 0. Capacity Planning Underprovision
• Solution: Outsourcing• Send peaks to other providers– Reduce provider infrastructure costs– Pay for using external resources
External Provider
46
Phase 0. Capacity Planning Overprovision
• Solution: Insourcing• Offer idle resources to other providers– Cheaper price– Enough resources to support peaks
Offer to other Providers
47
Phase 1. Scheduling
• Analyze provider profitability– Decide best actions to perform
• New actuators to consider– Outsourcing– Insourcing
• Old actuators change– Turn on/off nodes vs. Insourcing
48
Evaluation Characterization
• Provider profitability (darker is better)– Offering 80% of the idle resources– Amazon EC2 pricing
No Insourcing Insourcing
49
Evaluation Phase 0. Capacity Planning
• ISP workload over a week• Get optimal capacity– Leverage provider model– Revenue > Costs
Overprovision
Undeprovision
100 nodes
50
EvaluationPhase 1. Scheduling
51
EvaluationPhase 1. Scheduling
Multifaceted Scheduling
Multifaceted Scheduling + Outsourcing
Multifaceted Scheduling+ Insourcing
Multifaceted Scheduling +
Outsourcing + Insourcing
0
50
100
150
200
250Profit
%
52
Contents
• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation
• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation
• Conclusion
53
Conclusion
• Multifaceted Scheduling– Efficient Management of Virtualized Providers– Consider multiple facets– Handle effect of multiple facets– Every facet has a significant effect
• Multiprovider Scheduling– Add new actuators– New policies required– Tradeoff between Insourcing and Turn on/off
54
Future Work
• Support higher-level SLA metrics• Energy efficient datacenter– Reduce PUE, free-cooling…– Energy-aware scheduling: Green availability– Efficient datacenter placement
• Multiprovider– Apply to real-world scenarios– Enhance provider fault tolerance
55
Summary
• Developed from a multi-level perspective:– Enhancing Virtualization Fabrics– Managing a Virtualized Host– Multifaceted Scheduling– Multiprovider Scheduling
56
Publications• Enhancing Virtualization Fabrics
– [NCA09] VM Creation.
– [PDP09] VM Migration.
– [NOMS10] VM Checkpointing.
• Managing a Virtualized Host– [CPE09] SLA-driven resource management.
– [GECON10] SLA metric for Virtualized Environments.
• Multifaceted Scheduling– [Cluster10] [FGCS] Multifaceted Scheduling.
• Multiprovider Scheduling– [Grid10] Outsourcing into Multifaceted Scheduling.
– [Cloud10] Characterizing Federation.
57
Multifaceted Resource Management on Virtualized Providers
Íñigo Goiri
Advisors:Jordi Guitart and Jordi Torres
58
Publications
Enhancing Virtualization Fabrics[NCA09] Í. Goiri, F. Julià, J. Ejarque, M. De Palol, R. M. Badia, J. Guitart, and J.
Torres. Introducing Virtual Execution Environments for Application Lifecycle Management and SLA-Driven Resource Distribution within Service Providers. Proceedings of the 8th IEEE International Symposium on Network Computing and Applications.
[PDP09] Í. Goiri, F. Julià, and J. Guitart. Efficient Data Management Support for Virtualized Service Providers. Proceedings of the 17th Conference on Parallel, Distributed and Network-based Processing.
[NOMS10] Í. Goiri, F. Julià, J. Guitart, and J. Torres. Checkpoint-based Fault-tolerant Infrastructure for Virtualized Service Providers. Proceedings of the 12th IEEE/IFIP Network Operations and Management Symposium.
59
Publications
Managing a Virtualized Host [CPE09] J. Ejarque, M. de Palol, Í. Goiri, F. Julià, J. Guitart, R. Badia, and J.
Torres. Exploiting semantics and virtualization for SLA-driven resource allocation in service providers. Concurrency and Computation: Practice and Experience, 22(5): pages 541–572, 2010.
[GECON10] Í. Goiri, F. Julià, J. O. Fitó, M. Macías and J. Guitart. Resource-level QoS Metric for CPU-based Guarantees in Cloud Providers. In Proceedings of the 7th International Workshop on Economics of Grids, Clouds, Systems, and Services.
60
Publications
Multifaceted Scheduling[Cluster10] Í. Goiri, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres. Energy-
aware Scheduling in Virtualized Datacenters. Proceedings of the 12th IEEE International Conference on Cluster Computing.
[FGCS] Í. Goiri, J. Berral, J. O. Fitó, F. Julià, R. Nou, J. Guitart, R. Gavaldà and J. Torres. Energy-efficient and Multifaceted Resource Management for Profit-driven Virtualized Datacenters. Review process in Future Generation of Computer Systems.
61
Publications
Multiprovider Scheduling[Grid10] Í. Goiri, J. O. Fitó, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres.
Multifaceted Resource Management for Dealing with Heterogeneous Workloads in Virtualized Datacenters. Proceedings of the 11th ACM/IEEE International Conference on Grid Computing.
[Cloud10] Í. Goiri, J. Guitart, and J. Torres. Characterizing Cloud Federation for Enhancing Providers’ Profit. Proceedings of the 3rd International conference on Cloud Computing.
65
EvaluationOutsourcing
Profit Increase
-60
-40
-20
0
20
40
60
80
100 Backfilling+Migration
65 Nodes
65 Nodes + Outsourcing
33 Nodes
33 Nodes+Outsourcing
20 Nodes
20 Nodes + Outsourcing
0 Nodes + Outsourcing
%