t owards fit-aware scheduling policies for cloud computingvielmo/notes/navtalk-schedulin… ·...
TRANSCRIPT
![Page 1: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/1.jpg)
1/30
T owards FIT-awarescheduling policiesfor Cloud Computing
Vinicius Vielmo Cogo
Marcelo Pasin
T owards FIT-awarescheduling policiesfor Cloud Computing
![Page 2: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/2.jpg)
2/30
The Motivation:
Cloud Computing managers deal with large amounts of globally distributed resources.
![Page 3: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/3.jpg)
3/30
The Problem:
Scheduling policies are mainly focused on performance and economy.
![Page 4: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/4.jpg)
4/30
Main Goal:
Choose the best resource for a Fault and Intrusion Tolerant system.
![Page 5: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/5.jpg)
5/30
Background:
storagenetwork
- There are basically 3 types of resources in Cloud Computing (IaaS)
processing
![Page 6: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/6.jpg)
6/30
Background:
storage processingnetwork
- This work is focused on Processing resources
![Page 7: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/7.jpg)
7/30
Outline:
1 – CloudFIT Project
2 – Intrusion-Tolerance Mechanisms
3 – Resource Allocation Algorithm
4 – Requirements for Scheduling
5 – Future Work
6 – Open Questions
![Page 8: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/8.jpg)
8/30
1 – CloudFIT Project:
CloudFIT: Fault and Intrusion Tolerance for Clouds
- Goal: Create an infrastructure for FIT services in a cloud
environment
- Components:● Cloud Resource Manager (OpenNebula)
● Hosts with Virtual Machine Monitors (e.g.: Xen)
● Service that uses the bft-SMaRt library for State Machine Replication
● And others ...
![Page 9: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/9.jpg)
9/30
2 - Intrusion-Tolerance Mechanisms:
State Machine Replication ...
Proactive Recovery ...
Diversity ...
Replicated Service
432
Client
Client Requests
Service Answers1
Replicated Service
4321
Replicated Service
8765
Replacement 1
54321
Replicated Service
4321
Replicated Service
4321
...
![Page 10: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/10.jpg)
10/30
2 - Intrusion-Tolerance Mechanisms:
State Machine Replication ...… to increase integrity and availability
Proactive Recovery ...… to tolerate any number of intrusions over the
system's lifetime
Diversity ...… to increase the difficulty of attacks and
independence between faults
![Page 11: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/11.jpg)
11/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
![Page 12: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/12.jpg)
12/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
3
2
1
4
![Page 13: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/13.jpg)
13/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
3
2
1
4
![Page 14: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/14.jpg)
14/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
3
2
1
4
5
![Page 15: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/15.jpg)
15/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
3
2
1
4
5
![Page 16: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/16.jpg)
16/30
2 - Intrusion-Tolerance Mechanisms:
Clients
OpenNebula
PhysicalHost 01
PhysicalHost 02
PhysicalHost 03
PhysicalHost 04
PhysicalHost 05
PhysicalHost 06
PhysicalHost N
...
OtherClouds
3
2
4
5
![Page 17: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/17.jpg)
17/30
3 - Resource Allocation Algorithm:
- Choose one host to allocate a Virtual Machine
- Algorithms are normally based on two steps, or a combination of them:
● REQUIREMENTS (boolean expression)● RANK (numeric expression)
- OpenNebula uses a matchmaking algorithm:1. Filter out the hosts based on REQUIREMENTS2. Rank the remaining hosts based on RANK3. Choose the highest ranked host to allocate the VM
![Page 18: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/18.jpg)
18/30
3 - Resource Allocation Algorithm:
OpenNebula
InformationManager
ModuleModule Module Module
Clients
HOSTNAME HYPERVISOR RUNNING_VMS CPUSPEED TOTALMEMORY
host01 Xen 5 3.2 32
host02 Xen 2 3.2 16
host03 KVM 0 2.3 8
host04 VMWare 0 1.8 8
host05 VMWare 3 2.5 8
host06 KVM 6 2.7 16
host07 KVM 1 3.0 32
host08 VMWare 2 1.8 32
host09 VMWare 3 2.7 16
host10 Xen 4 2.3 4
... ... ... ... ...
hostN Xen 6 3.2 8
Requirement: Hypervisor=XenRank: Total of Memory
![Page 19: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/19.jpg)
19/30
4 - Requirements for Scheduling:
4.1 - Diversity Requirements
4.2 - Proactive Recovery Requirements
4.3 - State Machine Replication Requirements
![Page 20: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/20.jpg)
20/30
4.1 - The Various Diversities:
Diversity of …
… ApplicationN-Version programming
… Supporting SoftwareManifold Operating Systems, Server or Daemon implementation
… HardwareDistinct Architecture, CPU Model or Speed
… Administrative DomainDifferent Hostname, Cluster or Cloud Provider
… LocationVaried GPS coordinates, Geopolitical structures
![Page 21: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/21.jpg)
21/30
4.1 - Diversity of Administrative Domain:
- Replicas should not be placed in the same physical host, rack or cluster
- Replicas should not be placed in the same cloud provider
- Other measures:
* Cloud Providers: Private, Amazon, Rackspace, Salesforce, ElasticHosts, GoGrid, SliceHost, etc.
HOSTNAME != ”some-host” REQUIREMENT
RACK != ”some-rack-id” REQUIREMENT
CLUSTER != ”some-cluster” REQUIREMENT
CLOUD_PROVIDER != ”some-cloud-provider” REQUIREMENT
“- ROUND_TRIP_TIME to X.X.X.X” RANK
“- NETWORK_LATENCY to X.X.X.X” RANK
“- NUM_HOPS to X.X.X.X” RANK
*
![Page 22: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/22.jpg)
22/30
- Replicas should be placed in distinct architectures
- Replicas should use different processors
- Replicas should use different hardware components
* Trusted Platform Module: Atmel, Broadcom, Infineon, Intel, Sinosun, STMicroeletronics, Winbond, Toshiba, etc.
TPM != “some-tpm-implementation” REQUIREMENT
NETWORK_CARD != “some-network-card” REQUIREMENT
VIDEO_CARD != “some-video-card” REQUIREMENT
4.1 - Diversity of Physical Hardware:
ARCH != “some-arch” REQUIREMENT
MODELNAME != “some-processor-model” REQUIREMENT
“CPUSPEED” RANK
*
![Page 23: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/23.jpg)
23/30
4.1 - Diversity of Supporting Software:
- Replicas should be placed in different hypervisors
- Replicas must use different Wormholes
- Replicas should use different Operating System
- Replicas should use different Server implementation
HYPERVISOR != “some-hypervisor” REQUIREMENT
WORMHOLE != “some-wormhole” REQUIREMENT
Service Notion or Service Descripton Language PLUG-IN
Service Notion or Service Descripton Language PLUG-IN
![Page 24: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/24.jpg)
24/30
4.1 - Diversity of Application:
- Replicas should use different application versions/implementations
The service notion consists in deploy any service's replica instead of a specific image, on each replica replacement
Service Notion or Service Descripton Language PLUG-IN
Next Replica= Service
Next Replica= image2
Next Replica= image4
Next Replica= image4
Next Replica= image1
Without Service Notion With Service Notion
![Page 25: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/25.jpg)
25/30
“DISTANCE to +XX.XXXXXX -YY.YYYYYY” RANK
“- DISTANCE to +XX.XXXXXX -YY.YYYYYY” RANK
COUNTRY != “some-country” REQUIREMENT
4.1 - Diversity of Location:
- Recoveries should change replica location
- Service replicas should not be illegal in the location or should be allocated under a determined legislation (reputation or legal issues)
CITY != “some-city” REQUIREMENT
STATE != “some-state” REQUIREMENT
COUNTRY != “some-country” REQUIREMENT
CONTINENT != “some-continent” REQUIREMENT
GROUP != “some-group” REQUIREMENT
![Page 26: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/26.jpg)
26/30
4.2 – Proactive Recovery Requirements:
- Replicas must be added and removed in a controlled way
START_TIME=”2008-11-04 11:00:00”
END_TIME=”2008-11-04 12:00:00”
DURATION=”00:00:30”
PREEMPTIBLE=”yes”
PRIORITY=”10”
START_TIME="+00:00:30” Advanced Reservation
START_TIME=”2008-11-04 11:00:00” Advanced Reservation
START_TIME=”best_effort” Best-effort Provisioning
START_TIME=”now” Immediate Provisioning
- Haizea is a batch scheduler that provides 3 types of lease
- The scheduling can not block the resource allocation properties
- The important metrics are:
![Page 27: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/27.jpg)
27/30
4.3 – State Machine Replication Requirements:
Replicas could reuse the VM image from a local repository
ONE Server
ONE Node 01 ONE Node 02 ONE Node N
centralrepository
VM Image A VM Image B VM Image C
VM 01 VM 02
...VM 03 VM 04
cache cache cache
Image Cache on Physical Hosts PLUG-IN
![Page 28: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/28.jpg)
28/30
5 - Future Work:
- Delineating the influence of each metric in scheduling policies
- Specifying which metrics will be implemented
- Implementing and testing
- Organizing the metrics in two-level of scheduling: Cloud-of-clouds and Single Cloud
![Page 29: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/29.jpg)
29/30
6 - Open Questions:
- How independent are the faults in diversities cited here?
For diversity in Software of Support (Operating Systems) there is a new paper:
Miguel Garcia, Alysson Bessani, Ilir Gashi, Nuno Neves, and Rafael Obelheiro. OS Diversity for Intrusion Tolerance: Myth or Reality? In Proc. of the DSN’11: International Conference on Dependable Systems and Networks, Hong
Kong, China, July 2011.
- How guarantee that a Virtual Machine will be ready in an Advanced Reservation Lease for proactive recovery?
A cache for images, placed in hosts, helps but doesn't solve everything.
![Page 30: T owards FIT-aware scheduling policies for Cloud Computingvielmo/notes/navtalk-schedulin… · owards FIT-aware scheduling policies for Cloud Computing Vinicius Vielmo Cogo Marcelo](https://reader034.vdocuments.net/reader034/viewer/2022042418/5f33dc5614ad9d24dd621b35/html5/thumbnails/30.jpg)
30/30
http://cloudfit.di.fc.ul.pt/
http://opennebula.org
http://www.tclouds-project.eu/