Utility Driven Elastic Services

Pablo Chacin, Leandro Navarropchacin@ac.upc.edu

Polytechnic University of CataloniaComputer Architecture Department

Computer Networks and Distributed Systems Group

Barcelona, Spain

DAIS Conference June 6, 2011

() June 6, 2011 1 / 21

Agenda

1 Motivation

2 eUDON

3 Experiments

4 Conclusions

() June 6, 2011 2 / 21

Motivation

Increased management

complexity

• Emergence of SOA as paradigmfor distributed systems

• Unpredictability of usagepatterns

• Need to adapt to unexpectedsituations: failures, flash crowds

• Adoption of large scalenon-dedicated infrastructures Source: Schroth et al. 2007

System developers cannot anticipate management needs at design or evendeployment time.

Handling unexpected situations may require changing algorithms, parameters,structure.

() June 6, 2011 3 / 21

Requirements

A solution to this management problem should have some desirable

properties:

Adaptiveness Support varying workloads and infrastructure changesApp. Independence Offer a generic infrastructure for multiple servicesComprehensiveness Support a broad range of QoS needsEfficiency Achieve good resource utilizationEndurance Degrade gracefully under overloadFlexibility Accommodate different resource management policiesManageability Ease of maintain and operateNon-intrusiveness Require a minimal infrastructure modificationsReliability Assign requests despite the uncertainlyResilience Handle continuous activation/deactivation & failuresRobustness Work with incomplete, stale or inconsistent informationScalability Scale to a very large the number of service instances

() June 6, 2011 4 / 21

Self-Adaptation

Self-adaptive systems

Self-adaptations has emerged as an alternative to direct engineering and operationof system management.

Characteristics

• Aware: of its own state and the environment

• Self-adjusting: capable of changing its behavior, parameters, etc, to copewith changes in its internal state or the environment

• Automatic: do not need intervention of humans to adapt.

() June 6, 2011 5 / 21

Problem Statement

Limitation of existing self-management approaches

• Scale of the system

• Platform/workload not fully under the control of the management component

• Lack of an accurate and up to date global view

• Handle delays and failures during adaptation actions

• Cope with multiple management policies

Objective

”Managing complexity is a key goal of self-adaptive software. If a program mustmatch the complexity of the environment in its own structure it will be verycomplex indeed! Somehow we need to be able to write software that is lesscomplex than the environment in which it is operating yet operate robustly.”Laddaga (2000)

() June 6, 2011 6 / 21

elastic Utility Driven Overlay Network

eUDON

A middleware for dynamically adapting services deployed on large-scaleinfrastructures of non-dedicated servers

Scope

• Membership Management

• Request Routing

• Load Balancing

• Admission Control

• (Limited) Service Placement

• Resource Discovery

Salient Features

• Does not require a performancemodel

• Do not require PerformanceIsolation

• Implemented by eachservice/service class

Limitations

• Service placement over a predefined set of instances

• Monitoring considered, but not currently implemented

() June 6, 2011 7 / 21

eUDON Model

() June 6, 2011 8 / 21

Overlay Construction and Request Routing

Selector Ranking RoutingRandom N/A Round RobinAge N/A Round Robin

Capacity GreedyTwo ChoicesProbabilistic

Routing Overlay

Selector Ranking RoutingRandom N/A Random WalkAge Utility GreedyGradient

Search Overlay

() June 6, 2011 9 / 21

Admission Control

0

0.2

0.4

0.6

0.8

1

RT0

Utility

Response Time

α = 0.3α = 0.5

() June 6, 2011 10 / 21

Admission Control

0.00.20.40.60.81.0

Util

izat

ion

Total utilization

0.00.20.40.60.81.0

Util

izat

ion

Total utilization

Background load

0

5

10

15

20

Cap

acity

1.0

Util

ity R

atio

() June 6, 2011 11 / 21

Promotion and Demotion

Heuristic

• Probabilistic adaptation strategy

• Based on current arrival rate

• Needs an estimated of global distribution of arrival rates

• Promote if above 50%• Demote if below 25%

• Single parameter controls how aggressively adapt

• Don’t require any coordination

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100

Arrival rate

k=-3 k=-5

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100

Pro

babi

lity

Arrival rate

k=3 k=5

() June 6, 2011 12 / 21

Experimental Model

Simulation Model

• Discrete event simulator

• Idealized network that mimics a large cluster

• Each service instance as a M/G/1/k ∗ PS queuing system

• Background load simulated as a random walk

Parameters

• Nodes: 128 . . . 2048

• Exchange set: 1,2,. . . 8

• Neighbor set: 16,32,48

• Update frequency: 1,2,3

• Background load variability

• . . .

Metrics

• Allocated Demand

• Target/Offered QoS Ratio

• Utilization

• Hops

Compared with

Theoretical maximum.

() June 6, 2011 13 / 21

Base Scenario

60

70

80

90

100

110

120

130

0 100 200

N´”

Nod

es

(a) Evolution of the routing overlay size overtime.

0.10.20.30.40.50.60.70.80.91.0

0 50 100 150 200

Util

izat

ion

1.0

0 50 100 150 200

Util

ity R

atio

Time (seconds)

(b) Utilization and QoS Ratio.

() June 6, 2011 14 / 21

Alternative Load Balancing Heuristics

0.70

0.75

0.80

0.85

0.90

0.95

1.00

Pc 2C RR RR-R

Allo

cate

d D

eman

d

(c) Allocated Demand.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

1 2 3 4 5 6 7 8 9 10

%R

eque

sts

Hops

Pc2CRR

RR_R

(d) Distribution of routing hops.

() June 6, 2011 15 / 21

Alternative Search Heuristics

0.45

0.50

0.55

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

1.00

UDON Gradient Random Walk

Allo

cate

d D

eman

d

(e) Allocated Demand

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

1 2 3 4 5 6 7 8 9 10

%R

eque

sts

Hops

UDONGradientRandom

(f) Routing Hops

() June 6, 2011 16 / 21

Peak Load Scenario

0 1000 2000 3000 4000 5000 6000 7000

0 50 100 150 200 250 300

Req

uest

s

0.020.040.060.080.0

100.0120.0

0 50 100 150 200 250 300

N´”

Nod

es

Time (seconds)

(g) Injected load and number of instances

0.10.20.30.40.50.60.70.80.91.0

0 50 100 150 200 250 300

Util

izat

ion

1.0

0 50 100 150 200 250 300

Util

ity R

atio

Time (seconds)

(h) Utilization and Utility Ratio.

() June 6, 2011 17 / 21

Failure Scenario

0.10.20.30.40.50.60.70.80.91.0

0 50 100 150 200

Util

izat

ion

1.0

0 50 100 150 200

Util

ity R

atio

Time (seconds)

(i) Aggregate utilization and utility ratio.

30.0

40.0

50.0

60.0

70.0

80.0

0 50 100 150 200

Nod

es

0.0

1.0

2.0

3.0

4.0

5.0

0 50 100 150 200

Hop

s

Time (seconds)

(j) Number of instances and Number ofHops.

() June 6, 2011 18 / 21

Conclusions

We addressed the problem of self-adaptation in large scale distributed services.

eUDON exhibits the intended properties.

• Simple yet powerful model

• Non-intrusive

• Easily extensible, adaptable.

• Unifies multiple cases (failures, peak load)

• Scalable to 1000’s of nodes,

• Efficient (95% utilization, 90% allocated demand

Amenable to be included as part of the standard stack of service providers.

We believe this work represents a significant contribution towards the developmentof future generation service oriented applications by providing a self-managementsolution specifically addressed to this increasingly important category of systems.

() June 6, 2011 19 / 21

Future Work

Extend the model to support service composition following the model proposed byAlrifai et al. (2008) decomposing the utility function into a series of utilityfunctions which can be evaluated independently for each basic service.

Implement the activation/deactivation mechanism using the same theoreticalapproach used to model the market entry decision problem.

Apply the framework to other problems. In particular, the many tasks problem,like parameter swap and Map Reduce.

() June 6, 2011 20 / 21

Thank you ... any questions?

Pablo Chacinpchacin@ac.upc.edu

http://personals.ac.upc.edu/pchacin

Polytechnic University of CataloniaComputer Architecture Department

Computer Networks and Distributed Systems Group

Barcelona, Spain

() June 6, 2011 21 / 21