how advanced log management can trump siem: tales of woe and glory (166294760)

7/29/2019 How Advanced Log Management Can Trump SIEM: Tales of Woe and Glory (166294760)

http://slidepdf.com/reader/full/how-advanced-log-management-can-trump-siem-tales-of-woe-and-glory-166294760 1/22

How Advanced

Log Management

can trump SIEMTales of woe and glory

Vlad Grigorescu

cmu.edu

Don Becker Kees Leune

adelphi.edu



Presentation Outline1. Achieving and maintaining network

situational awareness

2. Log management vs. SIEM

3. From SIEM to Log Management

4. Adelphi's Lessons of Commercial Software

5. Carnegie Mellon's Approach



About usDr. Kees Leune (@leune) is AdelphiUniversity's Information Security Officer and a

GIAC Gold Adviser.

Don Becker is Adelphi University's InformationSecurity Engineer.

Vlad Grigorescu works for Carnegie MellonUniversity as Senior Information Security

Engineer.



Network Situational

AwarenessDefinitions:

An incident is a deviation from the norm that

causes harm, or that has the potential to causeharm.

An event is an observed action.

Security folks need to capture and analyzeevents to

a) establish a baseline

b) detect deviations from the norm



The problem isUseful events are not generated magically.

Once events are generated, they must becollected somewhere.

Once collected, you actually have to look themand try make sense of it.

Oh; you also need to get rid of them. Nothing isinfinite.



Log ManagementLog management is the art (forget aboutscience) of generating, collecting, storing,

retrieving, and disposing of logs.Log management is ridiculously hard.

Note: most people don't consider the partwhere events are generated as a step in the logmanagment process. I think it is one of themost crucial steps.



Fat logs vs. lean logsEvents are pointless without context.

Fat logs will contain the entire context in the

event to which it applies. Favorites of auditors.Example: Windows logs.

Lean logs contain a bare minimum information.The event reader will have to figure out thecontext on his own. Often made by engineers,and for consumption by technologists. Oftenlittle or no pattern. Example: firewall logs.



Log consumersEngineers who code logs often have one usecase in mind. E.g., consumption by automated

tools. Unfortunately, if not properly designed,such logs are often useless in other scenarios.

Example: try to automatically parse Windowsevents.



Log CollectionLog collection involves a few things, but mostcommonly, it

a) determines how logs are transmitted fromthe point they are generated to the point theyare received

b) when they are received, the logs have to bestored in some format that they can be postprocessed.



When logs become

interestingLogs by themselves are boring. They only start becominginteresting when you can use them to find somethingactionable.

In order to find something actionable, you need to know:

a) what you are looking for, and

b) how to ask for it, and

c) be able to interpret the results when you get them

In other words, logs become interesting when you canretrieve meaningful information from them.



When logs become

interestingLogs by themselves are boring. They only start becominginteresting when you can use them to find somethingactionable.

In order to find something actionable, you need to know:

a) what you are looking for, and

b) how to ask for it, and

c) be able to interpret the results when you get them

In other words, logs become interesting when you canretrieve meaningful information from them.

Hold thisthought!



SIEMI'm going to take a huge shortcut here, but

SIEM = Logs + Normalization + Correlation + Analysis + Alerting

I'm sure that hard-core SIEM affectionados willdisagree with me, but it is close enough.



Our SIEM experienceWe purchased a SIEM from NitroSecurity (R.I.P.) and werehoping to use it for pro-active security event detection, aswell as for forensic capabilities.

We implemented SIEM and we happily chugging away atsomewhere around 100 million events per day.

We got little to no actionable information from our SIEM,but the dashboards sure were pretty.

Forensic capabilities were good.It took the acquisition of our vendor, a steep price increase,and a sharp decline in technical support to realize this.



Our SIEM experienceWe purchased a SIEM from NitroSecurity (R.I.P.) and werehoping to use it for pro-active security event detection, aswell as for forensic capabilities.

We implemented SIEM and we happily chugging away atsomewhere around 100 million events per day.

We got little to no actionable information from our SIEM, but the dashboards sure were pretty.

Forensic capabilities were good.It took the acquisition of our vendor, a steep price increase,and a sharp decline in technical support to realize this.



Tales of Woe and Glory- we had heterogeneous data sources that were notsupported by the vendor, and

- it was difficult to write custom parsers, and

- the box was black magic, but

- it came with lots of pre-canned dashboards and reportsfocusing mostly on compliance reporting (not a big deal)

- hard to define custom reports

- not usable by ops groups because of caching

- unable to implement exceptions, or integrate with externaldata sources

- we did not really know what we were looking for

- noise of firewall logs drowned everything else out.



The real value of SIEM......comes from the ability to normalize andcorrelate events. We were never truly

successful at that.

When the vendor confronted us with a priceincrease, we decided to cut our losses and lookfor alternatives that better met our needs.



Log ManagementI made the key decision to no longer include myfirewall logs in my new <network situational

awareness and forensics thing>. Doing so cutcost drastically (both in terms of eps as in termsof log volume).

Firewalls are useful, but their logging usuallyleaves much to be desired.



Our solutionSplunk!

Reasonably priced, excellent tech support,good product, tickles many more geek sensesthat it checks auditors boxes.

Requires manual work to configure andcorrelate, but is so much more useful thandealing with pre-canned reports



CautionSplunk is great at answering questions, if youhave the data to help out. But, there are two

major impliciations here:

1. You know what questions to ask

2. You collected the information needed toanswer those question ahead of time

Without 1+2, there is no answer!



Daily routineMonitor odd stuff, dig deep to analyze.Example: off-hours authentication into sensitive

systems, general busyness statistics,authentication failures, alert conditions, etc.

IPS alerts also collected and monitored, butanalyzed in separate console.



Mandatory Screenshot



Analysis Workstation

how advanced log management can trump siem: tales of woe and glory (166294760)

Documents