your network security babelfish a.k.a. security event actionable log parser mike halsall &...

Your Network Security Babelfisha.k.a.

Security Event Actionable Log Parser

Mike Halsall&

Graeme Connell

©, Michael T. Halsall, 2006

2

Why We’re Here

• Scripting

• Creativity

• Automation

3

What We Had

• Pure Cisco architecture– Managed switches, 802.1q trunking capable– Small SNMP/Net::Telnet based scripts– Logging (via syslog) to a central location

4

Evolution of a System

• Purchased Bradford Networks’ Campus Manager– Forced us to trunk out to the edges to provide

Registration VLAN, Quarantine VLAN and Penalty Box VLAN

– Gave us a database centralizing users, their MAC addresses, their currently leased IP address and the port/switch they appear on, in real time

5

Campus Manager Benefits, Cont.

• Forced black hole DNS techniques

• Added multi-interfaced web server to handle each VLAN

• Forced Windows Update for students– Used GP to force Updates for Fac/Staff

6

Purchased Enterprise Level IPS

• Very fast

• Signature/Anomaly based detection of attacks

• In-line, allowing us to drop attack packets

• Custom signatures (not as nice as Snort)

• Exports alerts, in real time, to syslog

7

Privateye’s Conception (v. 1.0)

• Initially, a simple log parser (like swatch) for dealing with IPS logs– Hard-coded database capabilities– Threshold based– Ran external scripts

8

Privateye 1.0’s Config File

#<TRIGGERTYPE>:::<TRIGGER>:::<ACTIONTYPE>:::<ACTION>:::<THRESHOLD_VAL>:::<THRESHOLD_DUR>

ID ::: 0x40009a00 ::: SH ::: shun.sh ${7} ::: 0 ::: weekID ::: 0x49700500 ::: SH ::: penalty.sh ${4} ::: 5 ::: dayRE_NAME ::: /NULL/ ::: SH ::: shun.sh ${7} ::: 0 ::: weekRE_NAME ::: /Buffer Overflow/ ::: SH ::: shun.sh ${7} ::: 2 ::: weekRE_NAME ::: /Apache/ ::: MAIL :::emails/email1.txt::: 15 ::: hour

9

Problems With v. 1.0

• Not extensible

• Too specialized (IPS, CM)

• Restrictive triggers– Inside/Outside– CM/Non-CM users– Standard/Non-standard ports

10

Privateye 2.0

• Pushed functionality to config file

• Object oriented

• Complex trigger set– Boolean chaining– Regular expressions– Math

• Ability to remember user states

ANDOR

NOT

11

Privateye 2.0

Alert Data Flow

Input → AlertParser → UserHash → RuleList → Rule

Rule = Trigger + Threshold + Action

12


• Log file (syslog data)

#####2006-03-17 15:40:59 EST#####Medium#####irc#####Signature #####IRC: IRC Client Activity Detected#####140.233.203.41:1048 #####65.98.55.82:8080#####tcp#####Outbound#####0x40f00100

• TCP session• STDIN

13

• Uses Regular Expression to split input into relevent fields

Becomes• Severity: Medium• Input_IP: 140.233.203.41• Output_Port: 8080• CM_User: jstudent• Etc….


#####2006-03-17 15:40:59 EST#####Medium#####irc#####Signature #####IRC: IRC Client Activity Detected#####140.233.203.41:1048 #####65.98.55.82:8080#####tcp#####Outbound#####0x40f00100

14

• Severity: Medium• Input_IP: 140.233.203.41• Output_Port: 8080• CM_User: jstudent• Etc….

Becomes• User_ jstudent• User_everybody• User_inside


User_abc

User_def

User_Everyone

User_inside

User_outside

User_jstudent

User_xyz

User_jstudent

Rule1 dataRule2 dataRule3 data

…

15

• Chain of rules

• Checked serially

• Rules can execute RuleLists


Rule 1Rule 2Rule 3Rule 4Rule 5

Rule 1Rule 2Rule 3

Rule 1Rule 2

Rule 1Rule 2Rule 3Rule 4Rule 5

16

• All rules are threshold based• Each rule contains

– Trigger– Threshold data– Action


TRIGGER ssh_scan SUBSTR name "SSH"

ACTION src_shun SH scripts/shun.pl "${src_ip}"

RULE ssh_scan_rule ssh_scan src_shun 5 60

#####2006-03-20 13:41:41 EST#####Informational#####ssh#####Signature#####SSH: SSH Version 1 Server Detected#####216.180.225.242:43346#####140.233.190.2:22#####tcp#####Inbound#####0x45b00300

17

Recap

Input

AlertParser

UserHash

RuleList

Rule

18

External Actions• Shun at firewall

– Our IPS does not possess this capability for our model firewalls

• Flip a port into penalty VLAN– Fortunately, we could do this via an SNMP trap to CM,

maintaining central user status– We can also flip ports using same technique below

• Shut a port, if need be– Given an IP, find the port on campus network, even if not

known to CM (CDP walk)– Send a port ‘shutdown’ command to the switch

• Disconnect user from dial-in/VPN• TCP Reset

– Out of band– Sends TCP resets to internal host when triggered (pattern

match)

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet

Sniffer VLANWebserver

CurrentNetworkLayout

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


PrivateyeInput

Vectors

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


PrivateyeMitigation/

RemediationVectors

Firewall shun

VLAN switch

Port ShutVLAN switchcont.

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


Example

Single client infected by bot, sends bot command requestout to IRC server

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


Example

IPS Alert seen by Privateye through Logging Server

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


Example

Privateye contacts Campus Manager, which switchesclient into quarantined VLAN

VLAN switch

VLAN switchcont.

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


Example

Client redirected to web server by DNS black-holing, outside access to client disabled.

CampusManager

Hosts

Switch

FW

IPS

Privateye

Internal Network

LoggingServer

The Internet


Example

Client and web server are now in same VLAN

28

FYI How do they get here?Document root’s Index page is a PHP script

• Get IP of machine user is coming from• Look at local ARP table to get user’s MAC

address• Match MAC address to “Actions Done” DB

table Privateye updated when it told CM to place user in Penalty

• Redirect user to static “reason” page

30

Privateye 2.1

• User flags: Multi-alert correlation

• Rule Action thresholding

• User-state Backups

• Real time configuration

• Protected TCP sessions

31

Privateye Speed

• It all depends…– Database lookups– Trigger types / complexity

• Regular expression? Substring? Equality?

– Script speed

• Speed bottlenecks do occur in current version– Everything done serially– Second alert never starts until first finished

• Fully processed (including DB lookups, etc.)• All actions completed

32

The possibilities are endless…

Web Server Logs

LoggingServer

Look for SQLInjection

Scanning Logs

HoneyNet Logs

Look forPort Scans

Look forRecon

DarkNet Traffic Stats.

via Unencrypted TCP Connection

Look forMisbehavior Privateye

33

The possibilities are endless…

Web Server Logs

LoggingServer

Look for SQLInjection

Scanning Logs

HoneyNet Logs

Look forPort Scans

Look forRecon

DarkNet Traffic Stats.

via Unencrypted TCP Connection

Look forMisbehavior Privateye

Real TimeConfiguration

via Encrypted TCP Connection

34

What Privateye Has Handled

Version 2.0 was put into production in September, 2005

• 264 unique machines have visited the Penalty Box– 162 due to IRC Botnet activity

• Successfully stopped virus proliferation as soon as rule was added, on 3 separate occasions

• Mitigated thousands of attacks from outside world– Most popular:

• SSH Brute Force attacks• XMLRPC Exploits

35

IPS and CM Are Not Requirements

• If you have managed switches, you can find clients on network using creative scripting

• Snort can certainly be used

• ARP manipulation can take the place of VLAN switching (not as secure)

36

Scripting

• Be creative– Building blocks – small scripts beget big scripts

• PHP• Net-SNMP• Perl

– Net::Telnet::Cisco (cpan.org)

• Automate the mundane• Obsolete yourself• Automate carefully

37

Questions?

your network security babelfish a.k.a. security event actionable log parser mike halsall &...

Documents