multiple virtual machine live migration in federated cloud...
TRANSCRIPT
![Page 1: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/1.jpg)
Multiple Virtual Machine Live Migration
in Federated Cloud Systems
Walter Cerroni Dept. of Electrical, Electronic and Information Engineering
University of Bologna, Italy
![Page 2: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/2.jpg)
Motivations
• Success of cloud services and platforms
– significant savings in enterprise’s IT costs
– increasing number of mobile cloud users (e.g., social media)
• Huge growth of cloud computing investments
– public cloud market revenues in 2013: $ 58B
– expected to reach $ 191B by 2020 (source: Forrester, 2014)
• Incresing demand of computing, storage and
communication resources within Data Centers (DCs)
– R&D on DC infrastructure technologies
– advanced intra-DC and inter-DC networking solutions
2
![Page 3: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/3.jpg)
Federated Cloud Computing
• DC over-provisioning may be too costly
– expensive computing and communication equipment
– energy consumption
• Federated cloud systems
– mutual agreement among different cloud providers
– smart workload sharing across multiple DC resources
– increased flexibility and mobility of cloud services
• How to design the inter-DC interconnection network?
– efficiently planning the underlying communication infrastructure
– providing the required level of QoS
– considering the specific workload of cloud services
3
![Page 4: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/4.jpg)
Service Virtualization
• Service virtualization is widely used for DC administration
and maintenance
– decoupling service instances from underlying processing and
storage hardware
– key enabler for cloud federations
• Advantages of OS virtualization: Virtual Machines (VMs)
– platform independent
– quick deployment of new service instances
– easy service replication and migration flexibility and mobility
– effective load balancing and server consolidation
– easy backup and restore procedures
4
![Page 5: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/5.jpg)
Live Migration of Virtual Machines
• Live migration of VMs
– moving services from one host/DC to another with minimal
disruption to end-user service availability
– current state of VM’s kernel and running processes is maintained
• Generalized to multiple VMs live migration
– moving groups of correlated VMs and virtual networks
– many multi-tier applications run across multiple VMs
– emerging Network Function Virtualization (NFV) solutions to
transform vendor-dependent network equipment into software
apps running on standard HW
– Software Defined Networking (SDN) technologies can help
maintaining also the network state
5
![Page 6: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/6.jpg)
Live Migration of Virtual Machines
• Focus on memory migration
– storage migration through NAS syncronization (background traffic)
– network state migration through SDN
• Two approaches
– pre-copy: push most of the memory pages to destination host
before stopping VM at source host
– post-copy: pull most of the memory pages from source host
after resuming VM at destination host
• We assume the pre-copy approach (adoped by Xen, KVM, VirtualBox, etc.)
– iterative push phase: memory pages modified in a given round are sent
again in the next round, until total size of dirty
pages is below a given threshold or a maximum
number of iteration is reached
– stop-and-copy phase: VM is suspended at source host and the
remaining dirty pages are copied to destination
– resume phase: VM is resumed at destination with consistent
memory and network state
6
![Page 7: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/7.jpg)
Performance Metrics for VM Live Migration
7
copied memory pages
dirtied memory pages
iterative push phase
stop-and-copy phase
resume phase
• Downtime ( ): amount of time the VM is suspended
measures the end-user’s perceived quality
• Total Migration Time ( ): amount of time needed to copy the
whole memory
measures the impact of the migration process on both communication infrastructure
and computing resource utilization (busy during whole migration time)
time
![Page 8: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/8.jpg)
Simplified Model of VM Live Migration
• generic request to migrate a set of VMs
• all VMs in the given set have the same amount of memory
• all VMs show the same fixed page dirtying rate
• all VMs have the same memory page size
• the bit rate used to migrate VM is constant
• condition for pre-copy algorithm to be sustainable
8
dirty memory
size threshold max no. of
iterations
total migration time of VM
number of
iterations
![Page 9: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/9.jpg)
Sequential vs. Parallel VM Migration
Sequential
Migration of one VM at a time at full
network channel capacity
Parallel
Simultaneous migration of all VMs
equally sharing the channel bit rate
9
Smaller transfer bit rate but same dirtying rate leads to more iterations in parallel
migration than in sequential
![Page 10: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/10.jpg)
Sequential vs. Parallel VM Migration
10
Trade-off
![Page 11: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/11.jpg)
Federated Cloud Network Scenario
• Federated DCs are interconnected by a full mesh of
guaranteed-bandwidth network pipes
– pre-established MPLS LSPs between edge routers
– pre-established lightpaths on optical inter-DC network
• Workload of multiple VMs migrating from source DC can
be hosted by a subset of remote federated DCs
– not enough computing resource available in some DCs
– service-specific DC location constraints (e.g., due to latency)
– other constraints due to load balancing, energy savings, etc.
• Available remote DC resources assigned following the
anycast service model
– any DC in the available/suitable subset is equivalent for hosting
the group of VMs to be migrated
11
![Page 12: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/12.jpg)
Federated Cloud Network Scenario
12
MAN - WAN
VM set 1 VM set 2
![Page 13: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/13.jpg)
Inter-DC Network Model Hypotheses
• H.1: each request z needs to migrate the same number M of VMs
• H.2: each multi-VM migration consumes the same amount of
channel capacity b
• H.3: each network pipe provides the same total amount of
guaranteed capacity B
• H.4: each remote DC has the computing and storage capacity of
hosting up to k groups of M VMs
• H.5: each migration request is allowed to choose among m instances
of the requested computing/storage resources, which are randomly
distributed over the n remote DCs
– considering the general case when multiple instances of the same
resources can be available in the same DC
• H.6: network state, as seen by a given DC, is the number r of
ongoing multi-VM migrations originated by that DC
– r = 0, 1, 2, … , n*B/b
13
![Page 14: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/14.jpg)
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 0
14
z1
Cz11
Cz12
DC 1
DC 2
DC 3
source DC
![Page 15: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/15.jpg)
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 1
15
z1
Cz11
DC 1
DC 2
DC 3
Cz12
source DC
![Page 16: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/16.jpg)
z1
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 1
16
z2
Cz11
DC 1
DC 2
DC 3
Cz22
Cz21
source DC
![Page 17: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/17.jpg)
z1
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 2
17
z2
Cz11
DC 1
DC 2
DC 3
Cz22
Cz21
source DC
![Page 18: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/18.jpg)
z1 z2
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 2
18
z3
Cz11
DC 1
DC 2
DC 3
Cz22
Cz31
Cz32
Blocked!
source DC
![Page 19: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/19.jpg)
z1 z2
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 2
19
z4
Cz11
DC 1
DC 2
DC 3
Cz22
Cz41
Cz42
source DC
![Page 20: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/20.jpg)
z1 z2
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 3
20
z4
Cz11
DC 1
DC 2
DC 3
Cz22
Cz41
Cz42
source DC
![Page 21: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/21.jpg)
z1 z2 z4
Inter-DC Network Model
Example with n = 3 , k = 4, m = 2, b = B
Network state: r = 3
21
Cz11
DC 1
DC 2
DC 3
Cz22
Cz41
source DC
![Page 22: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/22.jpg)
• Multi-VM migration requests as a Poisson process
– request arrival rate
• Service time (when channel capacity b is busy) is the
total migration time
– service rate
– offered load
– loss system: results valid for any service time distribution with
same average
Markovian Model of Migration Request Blocking
22
![Page 23: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/23.jpg)
Approximate Sub-state Probabilities
• Given state r, many combinations of connections to DCs are possible
• Exact solution would require to compute all sub-states probabilities
• Approximate solution with reduced state space considering only
"forward" state evolution
• Recursive expression of sub-space probabilities
n = 3, B = 3b
Prob. that m suitable resources
are hosted by unreachable DCs:
Prob. request blocked in state 5:
23
![Page 24: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/24.jpg)
Markovian Model of Migration Request Blocking
24
Blocking probability:
![Page 25: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/25.jpg)
Numerical Results
• VM memory size distribution
– bimodal distribution: groups of large or small VMs
– with probability 75%
– with probability 25%
• Reference values for model parameters
• Model results plus simulations to validate model accuracy
25
![Page 26: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/26.jpg)
Sequential Migration Blocking Probability
• Good match with simulations reasonable accuracy
• Model allows to dimension the inter-DC network pipe capacity
26
![Page 27: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/27.jpg)
Parallel Migration Blocking Probability
27
• Parallel migration shows worse performance than sequential
migration, due to larger total migration time
![Page 28: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/28.jpg)
Seq. vs. Par. Migration Time and Downtime
28
• Model allows to quantify the trade-off between sequential and
parallel migration
B = 3 Gbps
![Page 29: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/29.jpg)
Impact of the Cloud Federation Size
29
• Blocking rate can be reduced by increasing the number of DCs
• Need to asses the resulting network infrastructure cost
![Page 30: Multiple Virtual Machine Live Migration in Federated Cloud ...elkhatib/crosscloud/2014/docs/CrossCloud… · Federated Cloud Computing • DC over-provisioning may be too costly –](https://reader036.vdocuments.net/reader036/viewer/2022070723/5f020db17e708231d402583a/html5/thumbnails/30.jpg)
Conclusion
• Analytical model for inter-DC network dimensioning in federated
cloud systems
• Network load generated by multiple VM live migration
– performance depends on migration schedule and resources
– sequential vs. parallel migration
– trade off network resource usage with end-user’s perceived quality
• Further study on-going
– release some simplifying assumptions
– different bandwidth allocation stategies
– consider real DC traffic profiles and VM memory profiles
– trade-off holds in general
– memory transfer synchronization may help limiting the downtime
30