configuration - brown university

Configuration

Yanzhi Xin, Xiping Peng

Motivation

Why configuration tuning:

● Optimal configuration significantly improve system performance

● Improve the performance without application code modification

MotivationConfiguration tuning challenges:

● Difficult to achieve optimal config, Misconfig happens often:

1. Increased system complexity, potentially several hundred configurable parameters

2. Parameters increase with every update

Common goals

● Achieving best performance

● Reduce the tuning complexity

● Tune efficiently and effectively

ConfAdvisor: Performance-centric Configuration Tuning Framework for Containers

● Tuning monolithic systems already a headache

● To make matters worse: Microservices architecture

1. More complex, more cascadings

2. Different application with various version in each container

3. Deployment environment

Observation

● Existing solutions:

1. Config management tools:

for servers, not for containers, not performance-centric

2. Performance sensitive config tuning system:

e.g. Ottertune, works well independently, but not with container

environment

3. Misconfiguration validation language:

huge learning curve, not performance-centric

Observation

Configuration file Example

ConfAdvisor

● Insights

1. Validating default config in image

ConfAdvisor● Insights continued

2. Throughout Container life cycle config tuning:

ConfAdvisor● Insights continued

3. Accepts both customized declarative and imperative rules:

Declarative: more general, easy to understand rules

Imperative: more complex, detailed, direct specific rules

ConfAdvisor design

ConfAdvisor implementation

L1/L2 Analysis

L3 AnalysisAdvisor Framework: exposing metrics

Plugins:

Analyze the metrics and loads specific rules;

Allow User Defined Functions

What-if Engine:

Consume the rules and generate final advices

Analytic plugin exampleImportant predicates: “What-if” and “advice”

Constraint:

Currently only support 6 softwares(Liberty,

Nginx, Node.js, MongoDB, Redis, and

Cassandra)

What-if rule engine“what-if” “advice”

ConfAdvisor evluationConfig & metrics

crawling(Runtime overhead) :

Config Advice with L1/L2

analysis:

Example: YCSB workload

on Cassandra container on

Kubernetes

Default config could not

run a workload due to a

JVM container related

parameter

Config Advice with

L3 analysis:

Automatic Database Management System Tuning Through Large-scale Machine Learning

Background

● Why Knobs Are Annoying

● What Makes The Situation Worse

As databases and applications grow in both size and complexity, optimizing a DBMS (tuning) to meet the needs of an application has surpassed the abilities of humans.

Given this, many organizations resort to hiring expensive experts to configure the system’s knobs for the expected workload.

Motivation

● Dependencies● Continuous Settings● Non-reusable Configurations● Tuning Complexity

OtterTune: Automatic Database Tuning Tool

Part 1: Controller

● The first part is the client-side controller that interacts with the target DBMS to be

tuned. It collects runtime information from the DBMS using a standard API (e.g.,

JDBC), installs new configurations, and collects performance measurements.

Part 2: Manager

● The second part is OtterTune’s tuning manager. It receives the information collected

from the controller and stores it in its repository with data from previous tuning

sessions. This repository does not contain any confidential information about the

DBMSs or their databases; it only contains knob configurations and performance

data.

Pipeline

Pipeline

(1) select the most impactful knobs,

(2) map previously unseen database workloads to known workloads,

(3) recommend knob settings.

Workload Characterization

1. Statistics Collection2. Pruning Redundant Metrics

Workload Characterization

Identifying Important Knobs

Feature Selection with Lasso

OtterTune uses a popular feature selection technique for linear

regression, called Lasso, to expose the knobs that have the

strongest correlation to the system’s overall performance.

Why LassoLasso works by adding an L1 penalty that is equal to a constant λ times the sum of absolute weights to the loss function.

● Shrinkage

Shrink small weights towards zero (discard irrelevant features)

● Selection

Automatically select more relevant features (those with non-zero weights)

Dependencies in Feature Selection

Many configuration knobs are non-independent and changing one may affect another. We can capture such dependencies between knobs by including polynomial features in the regression.

Automatic Tuning● Step 1: Workload Mapping

Workload repository

Input workload

Mapped WorkloadRelevant metrics

Automatic Tuning● Step 2: Configuration Recommendation

OtterTune uses Gaussian Process (GP) regression and gradient descent to recommend configurations

Results

The results show that OtterTune produces configurations that achieve up to 94% lower latency compared to their default settings or configurations generated by other tuning advisors. OtterTune generates configurations in under 60 min that are comparable to ones created by human experts.

Limitations

● Administrative privileges

● Restarting the DBMS is often necessary

Background

Ancestry relies on Kubernetes to quickly integrate and deploy applications across Ancestry’s website, which receives 50-million visitors a month, and generates more than a billion dollars in revenue.

To get optimum performance out of Ancestry’s cloud applications, Ancestry employed artificial intelligence for continuous optimization of the application runtime environment.

Solution

Ancestry used AI to cut the resources of one application by more than 50 percent, with zero drop in performance. In this instance, Ancestry has been able to get two times the performance out of Kubernetes for every dollar spent.

Continuous Optimization

Comparisons● ConfAdvisor

General Microservices config tuning framework, sort of hard coded

● Ancestry

Also targets on microservices architecture, specific to their own product

● OtterTune

Also based on machine learning and AI concepts, DBMS specific framework

Comparisons

Future directions:

● More AI/Machine learning involved● More Microservice oriented software config tuning tools● More Microservice tuning integration framework

Q & A