loganalysis using open source tools

12
Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013

Upload: gurvinder-singh

Post on 11-May-2015

324 views

Category:

Technology


7 download

DESCRIPTION

Loganalysis using Open source tools, elasticsearch, logstash, redis, kibana

TRANSCRIPT

Page 1: Loganalysis using Open source tools

Log Analysis using Open Source Scalable SystemsGurvinder SinghVerktøysamling Oct'2013

Page 2: Loganalysis using Open source tools

Motivation

● Distributed Systems

● In the moment of heat

● Centeralized interface to logs

● Easier access

10/2/13 SLIDE 2

http://www.themeparkreview.com/tatsumediaday/tatsumediaday57.jpg

Page 3: Loganalysis using Open source tools

Challenges

● Almost every component generates logs

● Different Formats and logging methods

● Different requirements for processing

● Dashboards

● Alerts

10/2/13 SLIDE 3

http://img72.imageshack.us/img72/3885/nephew2logs.jpg

Page 4: Loganalysis using Open source tools

Components

● Logstash

● Redis

● Elasticsearch

● Kibana

● Beaver

● lumberjack

● .....

10/2/13 SLIDE 4

Page 5: Loganalysis using Open source tools

Architecture

Page 6: Loganalysis using Open source tools

Architecture

Page 7: Loganalysis using Open source tools

Architecture

Page 8: Loganalysis using Open source tools

Architecture

Page 9: Loganalysis using Open source tools

Architecture

Page 10: Loganalysis using Open source tools

Demo

Page 11: Loganalysis using Open source tools

Possible Input/Processing/Output Options

10/2/13SLIDE

11

Page 12: Loganalysis using Open source tools

Thanks:-)

Questions?

10/2/13SLIDE

12