supporting autonomic management of clouds: service-level-agreement, cloud monitoring and similarity...

Supporting Autonomic Management of

Clouds: Service-Level-Agreement, Cloud

Monitoring and Similarity Learning

byRafael Brundo [email protected]

Under the Supervision of:Prof. Rocco De Nicola and Prof. Francesco Tiezzi

Doctoral Thesis Defense - March 30th, 2015 - Lucca, Italy

Contents

1 Introduction

2 SLA for Clouds

3 Cloud Monitoring

4 Similarity Learning

5 Polus Framework

6 Conclusions

Rafael Brundo Uriarte 1/51

Introduction

Introduction Rafael Brundo Uriarte 2/51

Cloud Computing


Cloud Characteristics

I Services

I Heterogeneity

I Virtualization

I Large-Scale

I Complexity


Autonomic Computing


Knowledge for the Self-Management

I Policies

I Service Definition and Objectives

I Status of the Cloud and Services

I Specific Knowledge


Challenge and Scope


Research Questions

Research Question 1How to describe services and their objectives in the cloud

domain?

Research Question 2What is data, information, knowledge and wisdom in the

autonomic cloud domain?

Research Question 3How to collect and transform operational data into useful

knowledge without overloading the autonomic cloud?


Research Questions

Research Question 4How to produce a robust measure of similarity for services in

the domain and how can this knowledge be used?

Research Question 5How to integrate different sources of knowledge and feed the

autonomic managers?


SLA for Clouds

SLA for Clouds Rafael Brundo Uriarte 11/51

Service-Level-Agreement (SLA)

I Contract

I Service Description

I Quality-of-Service

I Formalism

I Guarantees!


SLA for Cloud Computing - SLAC

I Domain Specific

I Multi-Party

I Deployment Models

I Formalism

I Ease-of-Use


Yet Another SLA Definition Language?

Features WSOL WSLA SLAng WSA SLA* SLAC

General Deployment Models � � � � � �

Broker Support - - - - - �

Business

Pricing Schemes � � - � � �

Formal Semantics - - � - - �

Verification - - � - - �

“�” feature covered“�” feature partially covered

“-” no support


Main Concepts

I Predefined Metrics - Involved Parties and Unit

I Intervals for Metrics - Template and Variations

I Groups - Multiple Service, Community Cloud

I Constraint Solving Problem


Example


Business Aspects

I Business Actions

I Flat and Variable Models

I Pricing Schemes - Exchange, Auction, Tender,Bilateral, Fixed, Posted


Implementation

I Editor for SLAs (Ecplise-based using Xtext)

I SLA Evaluator (Z3 Solver)

I Integration with the Monitoring System


Cloud Monitoring

Cloud Monitoring Rafael Brundo Uriarte 19/51

DIKW in the Domain

I Data

I Information

I Knowledge

I Wisdom


Cloud Monitoring

The Role of the Monitoring System in Clouds:

I Collect data and Provide Information andKnowledge

I No Wisdom - Related to Decision-Making

I Sensor of MAPE-K Loop


Related Works

Property PCMONS Monalytics Lattice Wang

Cloud � - - -

Autonomic Integration - - - -

Scalability - � � �

Adaptability - � - �

Resilience - - - -

Timeliness - � - �

Extensibility � � � �

“�” feature covered“�” feature partially covered

“-” no support


Panoptes

I Multi-agent system

I Monitoring in different levels

I Monitoring Modules - What needs to bemonitored and how to process the data


Architecture


Architecture

Communication:

I Publish/Subscribe

I Private Message

Adaptativeness:

I Priority for Modules

I Change of Roles


Architecture: Autonomic Integration

I Urgency Mechanism

I Decentralised Architecture

I On-the-Fly Configuration

I Multiple Abstractions


Experiments

I Self-Protection System

I Urgency Mechanism

I Scalability


Similarity Learning

Similarity Learning Rafael Brundo Uriarte 28/51

Specific Knowledge

Generation of Knowledge for a Specific Purpose, i.e.not applicable in all clouds. For example, similarity.

But what is similarity?

I How much an object (service) resembles other


Applications of Similarity

Cluster Services:

I Group Similar Services

I Different Algorithms(K-Means, PAM, EM)

Applications in the Domain:

I Anomalous Behaviour Detections

I Service Scheduling

I Application Profiling

I SLA Risk Assessment


Domain Requirements

I Categorical Characteristics of Services

I On-line Prediction

I Large Number of Characteristics

I Fast Prediction


Random Forest

Clustering with Random Forest

I Originally Developed for Classification

I Calculate the Similarity

I Clustering Algorithm (PAM)


Similarity Using RF: Criteria


Problems

I Similarity Matrix (Big Memory Footprint)

I Re-cluster on Every New Observation

I Cannot be Used in the Domain


Solution: RF+PAM


Experiments

I Compared the performance of our algorithm toother 2 methodologies

I Compared the performance of RF+PAM withthe standard off-line similarity learning

I Use Case:

I Scheduler deploys together the mostdissimilar services

I Similarity based on their SLAs


Polus Framework

Polus Framework Rafael Brundo Uriarte 38/51

Polus Framework


Use Case


Conclusions

Conclusions Rafael Brundo Uriarte 43/51

Summary


Research Questions

Research Question 1How to describe services and their objectives in the cloud

domain?

SLAC

Research Question 2What is data, information, knowledge and wisdom in the

autonomic cloud domain?

DIKW Hierarchy

Research Question 3How to collect and transform operational data into useful

knowledge without overloading the autonomic cloud?

Panoptes


Research Questions

Research Question 4How to produce a robust measure of similarity for services in

the domain and how can this knowledge be used?RF+PAM

Research Question 5How to integrate different sources of knowledge and feed the

autonomic managers?Polus Framework


Limitations

I Intelligence of Autonomic Managers

I Wide Range of Specific Knowledge

I Off-line Training of RF+PAM


Contributions

I A theoretical and practical framework for thegeneration and provision of knowledge for theautonomic management of clouds (PolusFramework):

I SLAC - SLA Definition and EvaluationI Panoptes - MonitoringI RF+PAM - Similarity Learning


Publications

1. R. B. Uriarte, S. Tsaftaris and F. Tiezzi. Service Clustering forAutonomic Clouds Using Random Forest. In Proc. of the 15thIEEE/ACM CCGrid [In Press], 2015.

2. R.B. Uriarte, F. Tiezzi, R. De Nicola, SLAC: A FormalService-Level-Agreement Language for Cloud Computing. In IEEE/ACM7th International Conference on Utility and Cloud Computing (UCC),2014.

3. R.B. Uriarte, C.B. Westphall, Panoptes: A monitoring architecture andframework for supporting autonomic Clouds, In Proc. of the 16thIEEE/IFIP Network Operations and Management Symposium (NOMS),2014.

4. R.B. Uriarte, S.A. Chaves, C.B. Westphall, Towards an Architecture forMonitoring Private Clouds. In IEEE Communications Magazine, 49,pages 130-137, 2011.


Future Works

I Dynamic SLAs

I Negotiation of SLAs

I Cloud Case Study


Thank you!Questions?

Rafael Brundo [email protected]


SLAC - Expressivity

I Core Language

I Extensions - Business Aspects

I Formal Definition for Extensions


SLAC - Implementation

Compatibility only with OpenNebula

I Toy Implementation

I Easily adapted


SLAC - Cloud Metrics

DTMF Cloud Computing Service MetricsDescription

I Recent Document (Still a Draft)

I Creation of a Model for the Definition of Metrics

I The SLAC Metrics can be Adapted for thisModel


SLAC Violation

I Violation and Penalty are Separated Concepts

I “Violation” Concept Flexible

I Easy to Understand


Panoptes - Scalability

I Designed to be scalable

I Adapt itself

I Experiments suggest it is scalable

I More experiments for future works


Panoptes - Analysis of Apache Broklyn

I Not Focused on Monitoring

I Does Not Process the Data


Panoptes - Analysis with CSPARQL

I Data is not Decorated (e.g. RDF)

I Impact of Decorated Monitoring Data(Scalability)

I Very Interesting Option


supporting autonomic management of clouds: service-level-agreement, cloud monitoring and similarity...

Science

clouds rafael brundo

clouds sla

support sla

monitoring system sla

autonomic cloud domain

problem sla

use sla

example sla