logstash + elasticsearch + kibana presentation on startit tech meetup

Download Logstash + Elasticsearch + Kibana Presentation on Startit Tech Meetup

Post on 26-Jan-2015

116 views

Category:

Technology

4 download

Embed Size (px)

DESCRIPTION

 

TRANSCRIPT

  • 1. Logstash + Elasticsearch + Kibana Centralized Log server (as Splunk replacement)Marko Ojleski DevOps Engineer

2. $plunk 3. Business as usual, untill 4. #Outage @03:00AM 5. Check logs.?!? 10 network devices 40 servers 100 logs 6. Massive RAGE 7. tail cat grep sed awk sort uniq 8. and looots of | 9. tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n 10. its just too much 11. 1. collect data 2. parse/filter 3. send dataLogstashwritten in JRuby Author: Jordan Sissel 12. inputparse/filteroutput 13. 1. collect data30+ inputs 14. 1. collect data filesyslogtcpudpzmqredislog4j Logstash input 15. Log shippersLogstash Beaver (Python) Lumberjack (Go) Woodchuck (Ruby) Nxlog (C) 16. Sample confinput { tcp { type => server1" host => "192.168.1.1" port => "5555" } 17. 2. parse/filter40+ filters 18. 2. parse/filter grokcsvgrepgeoipjson mutateLogstash filtersxml key/value 19. Grok filterREGEX pattern collection 20. Grok filter 21. Grok filter(?&$=` 24. `$=`;$_=%!;($_)=/(.)/;$==++$|;($.,$/,$,,$,$",$;,$^,$#,$~,$*,$:,@%)=( $!=~/(.)(.).(.)(.)(.)(.)..(.)(.)(.)..(.)......(.)/,$"),$=++;$.++;$.++; $_++;$_++;($_,$,$,)=($~.$"."$;$/$%[$?]$_$$,$:$%[$?]",$"&$~,$#,);$,++ ;$,++;$^|=$";`$_$$,$/$:$;$~$*$%[$?]$.$~$*${#}$%[$?]$;$$"$^$~$*.>&$=`Just another Perl hacker. 25. Grok filter120+ regex patterns USERNAME IP HOSTNAME SYSLOGTIMESTAMP LOGLEVEL etc 26. Grok filter2.10.146.54 - 2013-12-01T13:37:57Z - some really boring message 27. Grok filter2.10.146.54 - 2013-12-01T13:37:57Z - some really boring message %{IP:client} - %{TIMESTAMP_ISO8601:time} - %{GREEDYDATA:message} 28. Grok filterclient => 2.10.146.54 time => 2013-12-01T13:37:57Z message = > some really boring message 29. Grok filter input { tcp { type => server1" host => "192.168.1.1" port => "5555" }filter { if [type] == server1" { grok { match => { "message" => "%{IP:client} - %{TIMESTAMP_ISO8601:time} - %{GREEDYDATA:message} "} } } 30. 3. send data50+ outputs 31. 3. send data Logstash output statsdstdout tcpelasticredismongozmq 32. 1. RESTful api 2. JSON-oriented 3. Horizontal scale 4. HA 5. Full Text search 6. Based on LuceneElasticsearch Distributed RESTful search server 33. Logstash => elasticsearch input { tcp { type => server1" host => "192.168.1.1" port => "5555" }filter { if [type] == server1" { grok { match => { "message" => "%{IP:client} - %{TIMESTAMP_ISO8601:time} - %{GREEDYDATA:message} "} } } output { elasticsearch {} } 34. 1. Clean and simple UI 2. Fully customizable 3. Bootstrap based 4. Old version running on Ruby 5. Milestone 3 fully rewritten in HTML/Angular.jsKibana Awesome Elasticsearch Web Frontend to search/graph 35. Real Life Scenarios 36. Scenario 1 L2 switchCisco ASAL3 switchUDPUDPElasticsearchSyslog broker(lightweight shipper)UDPLogstash(main log server)Kibana 37. Scenario 2 Apache(lightweight shipper)IISTCPTCP(lightweight shipper)Jboss(lightweight shipper)ElasticsearchLogstash(main log server)TCPKibana

Recommended

View more >