pervasive computing self-organizing methods and models for...

2/17/11

1

Self-organizing Methods and Models for Software Development

Daniel J. Dubois Advisor: Prof. Elisabetta Di Nitto CoAdvisor: Prof. Raffaela Mirandola Tutor: Prof. Gianpaolo Cugola

Milan – February 17th, 2011

Motivation •  Context:

– Dynamic and distributed software architectures – Changing environment and requirements

Milan, 17/02/2011 Self-organizing Methods and Models for Software Development 2

•  Problem: –  Find efficient adaptation strategies to deal with the

complexity of such architectures

Pervasive Computing

Examples

Existing Solutions (1/2) •  Top-down adaptation approaches

– Adding one or more adaptation layers on top of the system elements

– Higher layers issue decisions for the lower layers


Advantages

•  Efficient for small and medium systems

•  Deterministic and optimal

Disadvantages

•  Scalability issues due to complexity

•  Difficult to deal with uncertainty

Existing Solutions (2/2) •  Bottom-up adaptation approaches

– Adaptation emerges through simple decisions made by elements at the lower levels

–  Inspired by natural phenomena – Also known as Self-organization

Self-organizing Methods and Models for Software Development 4

Advantages

•  Simple coordination

•  Scalable •  Fault tolerant

Disadvantages

•  Non-determinism •  Less guarantees •  Difficult to

engineer

Milan, 17/02/2011

2/17/11

2

Research Hypotheses


Assumptions: – Complex distributed software architectures – High number of elements at the same hierarchical level – High uncertainty and dynamism – No elements alone are essential

RH1: Usefulness of Self-organization

•  Self-organization is an efficient approach to address selected classes of problems under the aforementioned assumptions

RH2: Engineering Real Self-organizing Systems

•  The self-organization design guidelines we propose simplify the adoption of self-organizing approaches in real systems

Milan, 17/02/2011

Approach to Demonstrate Research Hypothesis 1

•  Study and analyze selected recurrent problems in distributed software architectures: – Load Balancing problems – Energy reduction problems – Publish/Subscribe middleware optimization

problems

Self-organizing Methods and Models for Software Development 6 Milan, 17/02/2011

self

organizing

algor.


Problem 1: Decentralized Heterogeneous Load-balancing algorithm [SASO08, Bionetics08]

Self-Aggregation Load Balancing

7

2

0

4

9

5

2

2

2

0

4

7

5

9

3

5

3

5

3

5

5

Milan, 17/02/2011

100% load

25% load

25% load

75% load

hibernated

75% load

Problem 2: Energy Optimization in a Self-organizing Data Center [SOAR10]

8 Self-organizing Methods and Models for Software Development

2/17/11

3

Problem 3: Traffic Reduction in a Content-based Publish-subscribe System [ICAC09]

Publish: soccer

Subscribe: sport

Publish: soccer

Subscribe: sport

soccer

Broker 2

Broker 1

Broker 3

soccer

socce

r

After Self-organization: 1 saved message


Broker 1

Broker 3

Broker 2 soccer

soccer

soccer

soccer

Milan, 17/02/2011

Approach to Demonstrate Research Hypothesis 2

•  Study a common software engineering approach for designing self-organizing software: –  Identify commonalities in self-organizing algorithms

in terms of principles – Reference architecture and transposition of the

principles into design patterns – Design guidelines and validation approaches


self

organizing

algor. software

engineering ?

integration

Milan, 17/02/2011

Self-organization Principles •  Derived from the observation of natural phenomena.

•  They may be composed together and translated into real deployable algorithms.

•  Most important evolutionary principles: –  Noise “a strategy of errors” –  Emergence “a team is better than the sum of its individuals” –  Evolution “natural selection”

•  Most important communication principles: –  Diffusion “gossiping is not necessarily a bad habit” –  Stigmergy “leave a note and I’ll read it”


Reference Architecture


2/17/11

4

Design Pattern for Diffusion (1/2)


Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09]

Identify principles/algorithms: −  Analyze the problem/requirements. −  Choose the principles/algorithms

that are most suitable for that class of problem.


Building Model: –  Compose and instantiate principles/

algorithms identified so far. –  Prove that the model satisfies its

requirements.



Example of Model Construction and Validation: The Load Balancing case

•  Abstract Model construction: – Graph representation – Transformation rules (may be non-

deterministic) – Expressed using TRIO language (first-order

temporal logic)

•  Considered validation alternatives: – Automatic Model Checking on the TRIO model – Monte Carlo Simulations


2/17/11

5

Implementation: –  Translate the model into a

deployable form. –  Use the reference architecture. –  Use self-organizing design

patterns based on the principles.



Advantages

•  Code reusability. •  Separation between self-organizing aspects

and application logic

Milan, 17/02/2011 Milan, 17/02/2011

Example of Use of Reference Architecture: Publish-subscribe Traffic Optimization Case

Self-organizing Methods and Models for Software Development

Identify and solve problems not captured by the model: –  Race conditions. –  Component synchronization. –  Frequency of iterations. –  Amount of communication. – …



Conclusions •  In this thesis we explore the problem of applying self-

organization techniques to solve problems in distributed software architectures.

•  Algorithmic contributions: –  Distributed Decentralized Load Balancing in Heterogeneous

Networks. –  Overlay Self-organization for Traffic Reduction in Public-

Subscribe Systems. –  Energy Optimization in a Self-organizing data center.

•  Methodological contributions: –  A method for modeling self-organizing software based on the

self-organizing principles. –  Examples of application of such method for implementing a

self-organization algorithm in a real software architecture.


2/17/11

6

Next Steps •  Future challenges:

–  Find optimal ways to self-calibrate self-organizing algorithms to reduce the needs of human-assisted preliminary simulations.

–  Specialize the principles and the design patterns. – Collect design patterns into a library and make them

available in existing CASE tools and in ready-to-use parameterizable implementations.

–  Extend the work with additional real case studies to support the validation of the patterns.

–  Investigate more on the validation techniques for self-organization.


Publications Related to this Work (1) •  International Conferences

1.  [Bionetics07] E. Di Nitto, D. J. Dubois, R. Mirandola “Self-aggregation algorithms for autonomic systems”. In Bionetics ‘07, Budapest, Hungary.

2.  [Autonomics07] D. Devescovi, E. Di Nitto, D. J. Dubois, R. Mirandola “Self-organization algorithms for autonomic systems in the SelfLet approach”. In Autonomics ‘07, Rome, Italy.

3.  [SASO08] E. Di Nitto, D. J. Dubois, R. Mirandola, F. Saffre, R. Tateson “Self-Aggregation techniques for Load Balancing in Distributed Systems”. In SASO ’08 (short paper), Venice, Italy.

4.  [Bionetics08] E. Di Nitto, D. J. Dubois, R. Mirandola “Applying self-aggregation to Load Balancing: Experimental results”. In Bionetics ’08, Hyogo, Japan.

5.  [ICAC09] E. Di Nitto, D. J. Dubois, R. Mirandola “Overlay self-organization for traffic reduction in multi-broker publish-subscribe systems”. In ICAC ‘09 (short paper), Barcelona, Spain.

6.  [ICSS10] D. J. Dubois, C. Nikolaou, M. Voskakis “A Model Transformation for Increasing Value in Service Networks through Intangible Value Exchanges”

7.  [Bionetics10] B. A. Caprarescu, N. M. Calcavecchia, E. Di Nitto, D. J. Dubois “SOS Cloud: Self-organizing Services in the Cloud” in Bionetics ‘10 (work in progress paper), Boston, USA.


Publications Related to this Work (2) •  Workshops and Doctoral Symposium

8.  [SEAMS09] E. Di Nitto, D. J. Dubois, R. Mirandola “On Exploiting Decentralized Bio-inspired Self-organization Algorithms to Develop Real Systems”. In SEAMS ‘09 (ICSE Workshop), Vancouver, Canada.

9.  [GIIS09] D. J. Dubois “Bio-inspired Self-organization Methods and Models for Software Development”, GIIS Doctoral Symposium, Salerno, Italy.

•  Book chapters 10. [SOAR10] D. Barbagallo, E. Di Nitto, D. J. Dubois, R. Mirandola “A Bio-

inspired Algorithm for Energy Optimization in a Self-organizing Data Center”. Self-organizing Architectures, Springer, 2010.

11. [ARTDECO10] N. M. Calcavecchia, E. Di Nitto, D. J. Dubois, C. Ghezzi, V. Mazza, M. Rossi “Complex Autonomic Systems for Networked Enterprises: Mechanisms, Solutions and Design Approaches”. Adaptive infRasTructures for DECentralized Organizations, 2010 (to appear).


QUESTIONS

24 Self-organizing Methods and Models for Software Development Milan, 17/02/2011

2/17/11

7

ADDITIONAL SLIDES







Design Pattern for Stigmergy (1/2)

2/17/11

8


Design Pattern for Stigmergy (2/2)


Design Pattern for Evolutionary Principles (1/2)


Design Pattern for Evolutionary Principles (2/2)

pervasive computing self-organizing methods and models for...

Documents