Guide to Anomaly Detection

A Practical

for DevOps

2categories

Anomaly Detection

log analysis metric analysis

identify suspicious event patterns in log files

log analysis

2categories

Anomaly Detection

log analysis metric analysis

identify misbehavingtime-series metrics

metric analysis

It reveals dangerous patterns that previously were undetected

Why is anomaly detection worth our time?

1The static nature of rule-based and threshold-based alerts encourages a) false positives during peak times b) false negatives during quieter times

2

It reveals dangerous patterns that previously were undetected

Why is anomaly detection worth our time?

12 The static nature of rule-based and threshold-based alerts

encourages a) false positives during peak times b) false negatives during quieter times

weapons of

mass detection

weapons of

mass detectionanomaly

Anomaly Detective by Prelert• Product: Anomaly Detective for Splunk • Pricing: $0-$225 / month (quote-based pricing > 10GB) • Setup: On premise (OS X, Windows, Linux & SunOS) • Installation: Easy (with Splunk Enterprise) • Main Datatype: Log lines

• Capable of consuming any stream of machine-data • Can identify rare or unusual messages. • A robust REST API, which can process almost any data feed • Offers an out-of-the-box app for Splunk Enterprise • Extends the Splunk search language with verbs tailored for anomaly

detection

Anomaly Detective by PrelertHighlights:

• Pricing: Quote-based • Setup: SaaS (+ on-premise data collectors) • Ease of Installation: Average (deploy Sumo Logic's full solution) • Main Datatype: Log lines

Sumo Logic

• LogReduce: a useful log crunching capability which consolidates thousands of log lines into just a few items by detecting recurring patterns.

• Sumo Logic scans your historical data to evaluate a baseline of normal data rates. Then it focuses on the last few minutes and looks for rates above or below the baseline.

• Anomaly detection will work even if the log lines are not exactly identical.

Sumo LogicHighlights:

• Pricing: $219/month for 200 instances & custom metrics • Setup: Dedicated AWS instance • Ease of Installation: Easy • Main Datatype: System Metrics

Grok

• Designed to monitor AWS (works with EC2, EBS, ELB, RDS). • Grok API for custom metrics (it’s fairly easy to process data from statsd). • Warns you in real time. • Customizable alerts for email or mobile notifications. • Grok uses their Android mobile app as their main UI. • Installation requires a dedicated Grok instance in your cloud environment.

GrokHighlights:

• Pricing: Open source • Setup: On-premise • Ease of Installation: Average (need python, redis and graphite) • Main Datatype: System Metrics

Skyline

• Etsy’s minimalist web UI lists anomalies & visualizes underlying graphs. • Horizon accepts time-series data via TCP & UDP inputs. • Stream Graphite metrics into Horizon. Horizon uploads data to a redis

instance where it is processed by Analyzer - a python daemon helping to find time-series which are behaving abnormally.

• Oculus, the other half of the Kale stack, is a search engine for graphs. Input one graph then locate other graphs that behave like it. Detect an anomaly using Skyline, then use Oculus to search for graphs that are suspiciously correlated to the offending graph.

SkylineHighlights:

But detecting anomalies !

is only half the battle...

BigPanda uses an algorithmic, data science approach to

simplify & automate incident management

BigPanda + Anomaly Detection

!

!

!

!

Anomaly Detection

incident management

http://bigpanda.io

Come take a look at what BigPanda is building!

Follow us online!