log analysis using open source scalable systems - uninett
TRANSCRIPT
Log Analysis using Open Source Scalable SystemsGurvinder SinghVerktøysamling Oct'2013
Motivation
● Distributed Systems
● In the moment of heat
● Centeralized interface to logs
● Easier access
10/2/13 SLIDE 2
http://www.themeparkreview.com/tatsumediaday/tatsumediaday57.jpg
Challenges
● Almost every component generates logs
● Different Formats and logging methods
● Different requirements for processing
● Dashboards
● Alerts
10/2/13 SLIDE 3
http://img72.imageshack.us/img72/3885/nephew2logs.jpg
Components
● Logstash
● Redis
● Elasticsearch
● Kibana
● Beaver
● lumberjack
● .....
10/2/13 SLIDE 4
Architecture
Architecture
Architecture
Architecture
Architecture
Demo
Possible Input/Processing/Output Options
10/2/13SLIDE
11
Thanks:-)
Questions?
10/2/13SLIDE
12