Autoscaler Architecture

Lahiru Sandaruwan Apache Committer - Stratos(Incubating) , Software Engineer - WSO2 Inc

Agenda • Introduction to Autoscaling

• Apache Stratos Autoscaler Architecture

• Component Architecture

• Event Flow

• Autoscale Policy

• Introduction to Autoscaler Policy

• Autoscaling Strategies

• Deployment Policy -

• Introduction to Deployment Policy

• Capacity Planning with Deployment Policy

• Partition Selection Algorithms

• Rules Engine

• Reasons for a Rule Engine

• Rules for Apache Stratos Autoscaler

1

Introduction to Autoscaling • What is scalability

• Horizontal and vertical scaling

• What is high availability

• Proceedure

• Clustering

• Load balancing

• Autoscaling

• Automating the capacity planning

2

Introduction to Autoscaling Contd. • Flexible cloud solution

• User-defined policies, health status checks, and schedules.

• Use case, cost, performance, and infrastructure.

• SLA(Service Level Agreement) aware elastic cloud • QoS , SLA aware services

• Daecision factors to consumers

• Solves performance, availability, and economic costs issues

• Capacity planning • Automated control of cloud: cost vs. Qos, find appropriate cloud model.

• Cost Factor • Reduce economic cost and energy footprint

• Proceedure • Online observation and monitoring the cloud

• Trigger an event if a SLA violation happened

• Use control theory and mathematical operations

• Handling seasonal patterns. E.g. Year ends/ Weekends patterns

3

Autoscaler Component Architecture

4

Event Flow

5

Health Statistics as Events • CEP receives events

• Requests in flight from Load balancer

• Cartridge instance health statistics from Cartridge agent

• CPU consumption

• Memory consumption

• CEP summarize the Average, Gradient, and Second derivative events of,

• Requests in Flight

• CPU consumption

• Memory consumption

6

Autoscale Policy

• Deployable Xml model

• Keeps Load thresholds for threshold based rules evaluation.

• Deployed by Dev-ops or similar role at start or later

• Hot Deployable.

• Users Selects an Autoscale Policy on His Preference at Subscription Time.

7

Autoscale Policy contd. <autoscalePolicy id="economy-autoscale">

<loadThresholds>

<requestsInFlight>

<average value="40" />

<gradient value="1" />

<secondDerivative value="0" />

<scaleDownMarginOfGradient value="1.0" />

<scaleDownMarginOfSecondDerivative value="0.2" />

</requestsInFlight>

<memoryConsumption>

...

</memoryConsumption>

<loadAverage>

...

</loadAverage>

</loadThresholds>

</autoscalePolicy>

8

Deployment Policy

• Deployable xml model

• Keeps the Capacity Planning.

• Deployed by Dev-ops or similar role at start or later

• Hot Deployable.

• Users Selects an Deployment Policy on His Preference at Subscription Time.

9

Deployment Policy contd. <deploymentPolicy id =”economy-deployment”>

<partitionGroups>

<partitionGroup id=”group1” >

<partitionAlgo>OneAfterAnother</partitionAlgo>

<partition id="partition1" >

<max>3</max>

<min>1</min>

</partition>

<partition id="partition2" >

<max>5</max>

<min>1</min>

</partition>

<partition id="partition3">

<max>20</max>

<min>1</min>

</partition>

</partitionGroup>

...

</deploymentPolicy>

10

Rules Engine

• Why a Rules Engine • Ease of use: No byte code and easy to modify

• Readable

• Performances and sclability

• Uses Drools engine as the default rules engine

• Rules

• Minimum Rule

• Scale Up Rule

• Scale Down Rule

• Terminate All Rule

11

Autoscaling Rules: Sample in Drools

rule "Minimum Rule"

dialect "mvel"

when

$service : Service ()

$cluster : Cluster () from $service.getClusters()

$policy : AutoscalePolicy(id == $cluster.autoscalePolicyName ) from $manager.getPolicyList()

$partition : Partition () from $policy.getHAPolicy().getPartitions()

$clusterContext : ClusterContext() from $context.getClusterContext($cluster.getClusterId())

eval($clusterContext.getPartitionCount($partition.getId()) < $partition.getPartitionMembersMin() )

then

int memberCountToBeIncreased = 1;

if($evaluator.delegateSpawn($partition,$cluster.getClusterId(), memberCountToBeIncreased)){

$clusterContext.increaseMemberCountInPartition($partition.getId(), memberCountToBeIncreased);

}

end

12

Minimum Rule • This runs when a “cluster created” event is received

• Scan through all the partitions of the cluster and find minimums

• Call CC for spawning required minimum instances

• This will be also run periodically(with a higher time interval than scale up/down rules) to assure that the minimum count is preserved

13

Scale Up/Down Rule • These rules run periodically

• Evaluate load details(Received from CEP) against their thresholds(defined in Autoscale Policy).

• Decide whether to scale up, scale down, or do nothing

• Call CC for spawning instances in selected partitions

14

Autoscaling Strategies • Threshold based autoscaling

• Predictive or proactive auto-scaling techniques

• Kalman Filter

• Control Theory

• Time series analysis

15

Average of Requests In Flight at LB for a Specific Cluster

16

Average of CPU/ Memory Consumption for a Specific Cluster

17

Terminate All Rule • This runs when a “cluster removed” event is received

• Scan through all the partitions of the cluster and find member IDs to be terminated

• Call CC for terminating those instances

18

Fault Handling Scenarios Process VM Decision flow

Down Up • Cartridge agent publish event to CC • CC updates instance status in topology • Autoscaler decides to kill it

Down Down(It can be that agent is crashed)

• CEP identify that & publish event to Autoscaler • Autoscaler calls CC to terminate(if available) and remove the instance from

topology • Autoscaler will spawn another to cover that

Up Up(but network issue)

• CEP sends statistics on fault requests to Autoscaler • Autoscaler keep monitoring it and takes a decision to terminate the instance • Autoscaler will spawn another in the same partition to cover that

19

References 1. http://eurosys2010-dev.sigops-france.fr/workshops/FeBID2010/bouchenak.pdf

2. http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/WhatIsAutoScaling.html

3. http://www.sc.ehu.es/ccwbayes/isg/administrator/components/com_jresearch/files/publications/autoscaling.pdf

20

Website

http://stratos.incubator.apache.org

Mailing List

Subscribe: dev-subscribe@stratos.incubator.apache.org

Post (After subscription): dev@stratos.incubator.apache.org

Social Media

Google+: https://plus.google.com/103515557134069849802

Twitter: https://twitter.com/ApacheStratos

Facebook: https://www.facebook.com/apache.stratos

LinkedIn: http://www.linkedin.com/groups?home=&gid=5131436

Join Us

21

Thank you

22