Upload File

splunk app for streammany solutions, one goal. some history •splunk acquires cloudmeter, december...

  • Home
  • Documents
  • Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)
35
Many Solutions, One Goal. Splunk App for Stream David Shpritz, Aplura LLC. Baltimore Area User Group 3/21/2016

Upload: others

Post on 06-Jun-2020

10 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

SplunkAppforStreamDavidShpritz,ApluraLLC.BaltimoreAreaUserGroup

3/21/2016

Page 2: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Agenda

• WhatisSplunkAppforStream?• WhyuseSteam?• WheretouseStream?• DeployingStream• Questions

Page 3: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

WhatIsSplunkAppforStream?

Page 4: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Somehistory

• SplunkacquiresCloudmeter,December2013• RenamedSplunkAppforStream• ReleasedwithSplunk6.0(August,2014)• Nowatversion6.4.3(January,2016)

Page 5: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

PurposeofStream

• Rapiddeployment• Rapidconfiguration• Capturewiredata• Interpretwiredata• Summarize/filter/aggregate• Index• KindoflikeBro,butmoreSplunky,andGUI

Page 6: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Sowhatcanwecapture?

• Well,wearen’treallycapturingandindexingpackets• Forwarderscapturepackets,analyzetheprotocols• Whatprotocols(alot):• TCP/UDP• Applicationprotocols(HTTP,databases,email,filesharing,chat)• About30differentprotocolscurrently• http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/Whattypeofdatadoesthisappcollect

Page 7: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

WhytouseSplunkStream

Page 8: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Nologs

• Noownership• Novisibility• Noforwarders(asendpoints)• Nologgingoptions

Page 9: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Poorlogs

• Loggingishighoverhead• Logsmakenosense• Keyeventsarenotlogged

Page 10: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Cloud

• Manycloudservicesdon’tofferlogsonthings• Nochokepoints

Page 11: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

VS.BroIDS

• LowerCPUusage• LowerRAMusage• MoreOSsupport(Linux,Windows,OSX)But• Hightrafficrequiresnetworkpacketbrokers(Gigamon,Ixia,etc.)• Can’twriteyourworkinterpreters• NoSnortrules

Page 12: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Otherfeatures

• Filtering• Aggregation• EphemeralStreams(shortterm)• SSLdecrypt• Centralizedmanagement• IntegrationwithES• StartastreamafterNotableevent• Protocolanalysisdashboards

Page 13: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

DataEstimation

• “WhatifIturnthison?”• Tellsyouhowmuchdatayouwouldbeindexing

Page 14: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Granularcontrolofthedata

• Notjustwhichsystems,butalsowhatdata,whichfields

Page 15: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

GlobalFilters

• Filteroutnoisefromtheenterprise• Thingslikevulnerabilityscanners

Page 16: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

DistributedForwarderManagement• Setupgroupsforcapture• Usesregexforgroupsonthe“ForwarderID”• ForwarderIDisconfigurableviaXMLconfig file• Yes,it’sanotherSplunkdeployment/controlmechanism

Page 17: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

WheretouseSplunkStream

Page 18: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

DedicatedStreamForwarders

• SenddataoffofaswitchSpanorTap• ToolslikeGigamon,Ixia,Etc.• Youneedtheseforreallybigpipestospreadthelove

• Purposebuilt• HigherCPUandRAM• Betternetworkcards

• AlsoagoodoptionisyouwanttoperformSSLdecrypt• Notethatifyoudothisyouwillwanttochangesomeofyourkernelsettings(buffersizes)• Makesuretomonitoryourforwardersforthruput warnings!

Page 19: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Page 20: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

DeploytotheEndpoints

• Deploydirectlytothesystemsyouwanttomonitor• Goodforapplicationdebugging• NiceoptionforSplunkES• CanbedonefromDeploymentServer• Granularcontrolovergroups• Couldmeanalotof“handon”

Page 21: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Page 22: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

DeployingSplunkStream

Page 23: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Page 24: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Twoparts

• TheSplunkAppforStream• Dashboardsforanalyticsonprotocols• Administrativepanelsforconfiguration• StreamEstimate(reallycool,morelater)• GoesonSearchHead/Controller

• SplunkStreamAdd-on• Binaries• Index-timeoperations(linebreaking,timestamping)• GoesonIndexersandForwarders(UForHF)

Page 25: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

InstalltheSplunkAppforStream

• Canco-locatewithES• Canco-locatewithDMC• Insmaller(lessthan100forwarders)don’tusewiththeDS• Possibleexhaustedconnections(DSandStreampollseparately)

• InstallsjustlikeanyotherSplunkapp

Page 26: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

HarvesttheAddOn• Installstoafewplaces• $SPLUNK_HOME/etc/apps/Splunk_TA_stream• $SPLUNK_HOME/etc/apps/splunk_app_stream/install/Splunk_TA_stream• $SPLUNK_HOME/etc/deployment-apps/Splunk_TA_stream• Willcreatethelocalinputs.conf withtheappserverlocation

*SkipthisisyourSHisyourDS

Page 27: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Makesureyourforwarderscantalkback

• YourforwarderswillneedtobeabletotalktotheSHwithsplunk_app_stream installed• TheportisthesameastheGUIforyourSH

Page 28: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Configureyourforwarders

• Don’thavetoberootonLinux• Usetheincludedsetuid.sh script

• MustbelocaladminorlocalsystemonWindows• OnUFsyoushouldmonitoryourthruput limits

Page 29: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Inputs.conf

• Rememberthattheinputs.conf islayerable• JustlikeotherSplunkconfigs• Doesn’thavetobeintheSplunk_TA_stream• OntheDSyoucandeploytwoapps,onewiththeinputtopointbacktothesplunk_app_stream• ThenalsodeploytheSplunk_TA_stream

Page 30: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Configureyourstreams• Thedefaultsmaysendmorefieldsthanyouneed• Cantellforwarderswhichpartsofthedatayouwant• Youcanhavedifferentconfigs fordifferentgroups!

Page 31: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Configureyourforwardergroups

• Usesgoodol’regex• LetsyousayaheadoftimeifEphemeralStreamsshouldbeallowed

Page 32: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

GotchawithGroups• JustregexontheStreamforwarderID(notIP,hostname)• ThisisconfiguredinanXMLfile• Messy• The“defaultgroup”forwardergroupforallunmatchedhostswillgatherALLTHETHINGS

Page 33: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Waitfordatatoflowin

• That’sprettymuchit!• Docsmakeitlookalotharder

Page 34: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Questions?

Page 35: Splunk App for StreamMany Solutions, One Goal. Some history •Splunk acquires Cloudmeter, December 2013 •Renamed Splunk App for Stream •Released with Splunk 6.0 (August, 2014)

ManySolutions,OneGoal.

Credits• ThankstotheBaltimoreAreaSplunkUserGroup• CoverSlide:UpperSwallowFallsinOakland,MD,ChrisFlees,http://fineartamerica.com/profiles/chris-flees.html?tab=artwork&page=7

• Slide3:PotomacRiverinMaryland,TerryJ.Adams,http://www.fhwa.dot.gov/byways/byways/60807/photos

• Slide7:Timanus MillontheJonesFallsinBaltimore,“MonumentCity”,http://www.panoramio.com/photo/57148558

• Slide8:“MissingHomeworkLog”by“RedBeetleRB”.https://www.teacherspayteachers.com/Product/Missing-Homework-Log-4112• Slide9:Rotton log,NationalWildlifeFoundation,https://www.nwf.org/kids/family-fun/outdoor-activities/investigate-a-rotten-log.aspx

• Slide10:TheSimpsons,http://i.imgur.com/91sn32Q.jpg?fb

• Slide11:BroNetworkSecurityMonitor,https://www.bro.org/

• Slide17:IanAdamsPhotography,http://ianadamsphotography.com/news/galleries/bridges/• Slides19and21:SplunkConf 2015,“SplunkAppforStreamDeploymentsintheRealWorld:EnhanceOperationalIntelligenceAcrossApplication

Delivery,ITOps,SecurityandMore”,http://conf.splunk.com/session/2015/conf2015_SUdovicic_CChing_MDickey_Splunk_SplunkEntWhatsNew_StreamDeploymentsInTheReal.pdf

• Slide22:GunpowderFallsinBaltimoreCounty,MD,http://hdrcreme.com/photos/1818-gunpowder-falls• Slide23:SplunkDocs,http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/DeploymentArchitecture

• Slide34:YoughioghenyRiveratFriendsville,MDbyJoeDawson,https://www.flickr.com/photos/jmd41280/5066756138


Webinar: Neues zur Splunk App for Enterprise Security

ThreatConnect App For Splunk Enterprise · Overview . The ThreatConnect® App for Splunk Enterprise gives Splunk users the ability to leverage customizable threat intelligence integrated

IntSights for Splunk Integration Benefits · Splunk deployments and create orchestrated responses. GET STARTED: Download the INTSIGHTS APP FOR SPLUNK ENTERPRISE. Download the INTSIGHTS

Prezentacja z konferencji prasowejgsmonline.pl/app/articles/print/microsoft-przejmuje-nokie.pdf · Microsoft acquires Nokia's phone business NOKIA Microsoft acquires Nokia's Qualcomm,

NetFlow Analytics for Splunk · NetFlow Analytics for Splunk App User Manual 3 Introduction Overview NetFlow Analytics for Splunk App is designed to deliver next generation, real-time,

Splunk App for Stream - Einblicke in Ihren Netzwerkverkehr

Splunk App for AWS at Creative Artists Agencyfiles.meetup.com/4507922/Splunk App for AWS at... · First and foremost - replicating current AWS success in Azure to continue to provide

CYBERX IIoT & ICS HIGHLIGHTS THREAT MONITORING APP FOR SPLUNK · CYBERX IIoT & ICS . HIGHLIGHTS. THREAT MONITORING APP FOR SPLUNK. Deliver OT Network Visibility and . ... The Splunk

Skylight Powered Security for Splunk€¢ Splunk Security Essentials is the free Splunk app that makes Machine Learning enrichment from other data sources, such as the endpoint •

Vulnerability* Managementwith* the*SplunkAppfor* Enterprise*Security* · Splunk*App*for*Enterprise*Security* 14

WEB APPLICATION MONITORING AND ANALYTICS WITH SPLUNK · > Enter Splunk: Splunk Enterprise allowed us to build a custom searching app as well as a dashboard for monitoring service

Varonis App for Splunk · VARONIS APP FOR SPLUNK® USER GUIDE 1 1 OVERVIEW Refer to the following sections in this overview: • About the Varonis App for Splunk® • Prerequisites

How to Build a Simple App for Splunk

ThreatScape® App for Splunk Overview, Installation and ... · PDF fileThreatScape® App for Splunk Overview, Installation and Configuration December 23, 2015 !!

Splunk Cloud の製品概要 › ... › product-briefs › splunk-cloud.pdfSplunk App で機能を強化 Splunk Cloud は、Splunk Enterprise Security や Splunk IT Service Intelligence

Anomali Community App (for Splunk) Quick Start Guide · Chapter1:Introduction AnomaliThreatStreamCommunityAppforSplunkleveragesthreatintelligencefromAnomali's

ThreatStream Splunk App - Anomali · Download ThreatStream Splunk App from . 2. From the Splunk web interface, navigate to the Apps menu in the upper lefthand, and - select

Splunk App: Windows Security Operation Center

Splunk For IT Operational Intelligence · • Splunk Apps –Splunk App for VMware –Splunk Apps for Citrix & Hyper V –Splunk App for Microsoft Exchange –Splunk App for Active

SECUI MFI App for Splunk - ::: 시큐아이 · SECUI MFI App for Splunk 3 ... laws mentioned in this manual are used for reference only ... This explains all of the functions of the

ThreatStream Splunk App - · PDF fileUser Guide. v3.3.9 . ThreatStream Splunk User Guide . Published: 4/27/2016. Anomali, Inc. Twitter: @anomali . ThreatStream® Splunk App

Cloud & DevOps Program...Splunk App Administration for Enterprise Security 3.1 (Virtual) Searching & Reporting: Splunk 6 (Virtual) Advanced Searching & Reporting: Splunk 6 (Virtual)

NetFlow Analytics for Splunk · PDF fileMarch, 2016 . NetFlow Analytics for Splunk App User Manual NetFlow Logic Confidential 1 Contents Introduction

FIREEYE APP FOR SPLUNK ENTERPRISE 6.X: Configuration

Splunk のご紹介 - hitachi.co.jp · Splunkの歴史. 5 . 2006 . Splunk 1 . Splunk 2 . 2007 . Splunk 3 . 2009 . Splunk 4 . 2010 . Splunk 4.1 . 2011 Splunk 4.2 . App for Enterprise

FireEye App for Splunk Enterprise · 7 FireEye App for Splunk Enterprise Documentation Version 1.1 Installing the FireEye App for Splunk Enterprise Use the App Manager within Splunk

Splunk app for stream

Qualys(R) App for Splunk Enterprise with TA · splunk > Apps v Upload app Apps » Upload app Install successful App setup required Administrator Messages Settings v Activity Help

Splunk Stream Product Brief · The app passively collects streaming application ... analyze and process DNS communication in ... Splunk Stream Product Brief

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

NetFlow Analytics for Splunk · 2018-12-05 · NetFlow Analytics for Splunk App User Manual 1 Table of Contents ... Configuring Splunk Analytics for Hadoop (Hunk) ... optionally,

Splunk For IT Operational Intelligenceuploads.integrity360.com/1425903243-Splunk-for-IT-Operations.pdf · • Splunk Apps –Splunk App for VMware –Splunk Apps for Citrix & Hyper

DNS EDGE FOR SPLUNK - BlueCat · Cybersecurity analysts use the BlueCat DNS Edge for Splunk app to easily review, monitor and set alerts …

Introducing LucidWorks App for Splunk Enterprise webinar

Deploying*the*Splunk* App*for*Microso>* Exchange* · Splunk*App*for*Exchange* TALSMTPLReputaon* SplunkAddonforWindows SplunkAppforWindows Infrastructure* Splunk_TA_Windows* TALDomainControllerLNTx*