security*operaons** with*splunk*app*for* enterprise*security* · siemtechnology*decision*...
TRANSCRIPT
Copyright © 2014 Splunk Inc.
David Casey, Vice President, IT Security OperaCons Manager Flagstar Bank
Security OperaCons with Splunk App for Enterprise Security
Disclaimer
2
During the course of this presentaCon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauCon you that such statements reflect our current expectaCons and
esCmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements,
please review our filings with the SEC. The forward-‐looking statements made in the this presentaCon are being made as of the Cme and date of its live presentaCon. If reviewed aSer its live presentaCon, this presentaCon may not contain current or accurate informaCon. We do not assume any obligaCon to update any forward-‐looking statements we may make. In addiCon, any informaCon about our roadmap outlines our general product direcCon and is subject to change at any Cme without noCce. It is for informaConal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligaCon either to develop the features or funcConality described or to
include any such feature or funcConality in a future release.
Personal Background
3
! CISSP/CISM/SnortCP/Dr.Evil ! Joined Flagstar Bank in early 2013 ! 15+ Yrs – IT Security ! 18 Yrs – U.S. Army Military Intelligence
Experience in the following sectors: ! DoD (Lockheed MarCn, NCI, SAIC) ! EducaCon ! Energy ! Finance
Specialize in building Security OperaCons programs from the ground up and major security ops overhauls due to compliance failures
Company Background
4
Flagstar Bank ! Full-‐service bank (Troy, Michigan) ! $9.4 billion in total assets ! 100 + Branches in Michigan ! 39 Home loan centers in 19 states ! NaConwide mortgage lender ! One of the naCon's top 10 largest savings banks
In the Beginning There was Darkness…
5
! 2009-‐2012 Flagstar expanded business operaCons very fast
! Infrastructure changes, mind sets, technology could not keep pace
! As Flagstar bank grew federal oversight shiSed
! New auditors were assessing security in the same manner as the Chase’s and Bank of America’s
! Flagstar had many, many audit findings
! People, processes, and technology had to change
! IT Security OperaCons grew significantly in order to meet compliance requirements effecCvely
! A SIEM was a criCcal component ! The ‘One Ring’ to rule them all!
SIEM Technology Decision When looking for a SIEM solu7on for Flagstar leveraged 12+ years of SIEM deployment experience as its guide. Lessons learned: • Difficult geong data in (ingesCng data) • Hard to get clear results from ad-‐hoc queries • Limited plaporm opCons • Costly to operate/maintain • Inflexible • SIEM sales hype. Product vendors only want to
sell you their product. No interest in truly helping you protect your organizaCon.
6
Splunk Experience… ü Easy to get all machine data
into the system
ü Simple plain language search
ü Uses commodity hardware
ü IntuiCve, easy to use
ü Flexible and easy to customize
ü They actually want you to be successful and take great strides to make it so!
Splunk Deployment
7
Current Design: ! 2 Search heads, 3 indexers, 300+ GB/day
! Data sources (current) – All servers via forwarders – Windows, 4 flavors of UNIX – All networking devices (switch, router, wireless, VPN, etc.) – Syslog systems – Firewall, IPS, DLP, AnC-‐Virus – Web proxy logs – DNS, DHCP – eMail
Applica7ons – Splunk for Windows apps (3) – Splunk for UNIX app – Various vendor security apps (<10) – DBConnect – More…
Splunk Deployment
8
Disaster Recovery (DR) Design: ! Overall Splunk ecosystem managed from HQ site
! 2 Search heads, 2 indexers
! DR site forwards all logs to HQ site
! HQ replicates last 72 hours of logs to DR
Future State: ! All data 100% replicated
! Heavy Forwarders Deployed to both HQ and DR sites
! SAN improvements >1000 IOPS sustained
Security OperaCons Monitoring Challenges
9
Some7mes security technology is simply not enough… it takes a human to help it all make sense ! The cyber security threat landscape is constantly morphing, ever changing, with threat actors intent on by-‐passing common security controls that rely on known payerns and detecCon techniques
! Humans are primarily a visual-‐based species ! Splunk can provide a visual that “speaks a thousand words” by taking the complex and making it simple to understand
! Take for instance the following case studies…
Case Study #1 – Are We Being Targeted?
10
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) App to monitor for advance threats, including exploits, malware infecCons, monitoring blacklists, and responding to spikes in threat trends
! One common threat gathering technique is finger prinCng/mapping out a targets public facing systems, its ports and services
! Being “scanned” is very common and generally considered background noise… just a part of doing business on the internet
! But when the scan is coming from a country that is frequently a hosCle cyber threat, and the scan is performed slowly, non-‐aggressively, it can oSen by-‐pass security controls that are designed to block more “aggressive” scans
Case Study #1 – Are We Being Targeted?
11
! Sample Splunk search:sourcetype="[hidden]" earliest=-‐1m inbound | geoip src | search src_country_code!=US | stats count AS count by src_country_name | sort -‐ count top limit=5
Case Study #1 – Are We Being Targeted?
12
! Upon closer inspecCon we were able to isolate the scans as originaCng from the City of Nanning, China
! We have no legiCmate customers in China…
! Answer? Block the network range
Case Study #1 – Con$nued
13
There are many hosCle actors all over the world. Some of the top actors are Russia, Ukraine, and China. Take Russia for example. Sure seems like there are a lot of outgoing connecCons to a Russian IP address. Could this be a compromised host?
Using Splunk we can watch closely outbound desCnaCons, by IP locaCon, and respond more quickly when we see an increase in potenCally risky IP traffic to known hosCle actor countries.
Case Study #1 – Con$nued
14
! Sample Splunk search (Russia Inbound): sourcetype=“[hidden]" src_ip!=“[internal networks excluded]" | iplocaCon src_ip | search Country="Russia" | where Country="Russia" | chart count by src_ip | sort -‐ count top limit=5
! Sample Splunk search (Russia Outbound): sourcetype=“[hidden]" src_ip!=[exclude DNS server IP, web proxies, etc.] " | iplocaCon dest_ip | search Country="Russia" | where Country="Russia" | stats count by src_ip,dest_ip | rename src_ip AS "Client" dest_ip AS "Russia IP Address" count AS "Count" | table Client,"Russia IP Address",Count | sort -‐ count by Count top limit=5
Case Study #2 – Firewall Control AyestaCon
15
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) app to help meet regulatory requirements (for IT Security Dept. only… at this Cme)
! One example was where federal auditors wanted to see where changes to the perimeter firewall were being monitored against approved firewall changes
! If a change occurred outside of the change control process it should be noted and invesCgated
! Splunk was used to help idenCfy all ‘write’ and ‘execute’ commands issued on the perimeter firewalls and graphically displayed for easy idenCficaCon
! This soluCon was accepted by the federal auditors ! YMMV…
Case Study #2 – Firewall Control AyestaCon
16
! Sample Splunk search: evenyype=“[hidden]_privileged_acCvity" "write" OR "111010" OR "101008" NOT ("Teardown" OR "connecCon" OR "exit" OR "ping" OR “[hidden]") | Cmechart span=15m count(host) by user | sort _Cme
Case Study #3 – Metrics Across Security Technologies
17
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) App to track security metrics
! Security metrics are commonly requested as *proof* that the $$$ invested in security technology is actually producing results
! Rather then running separate reports from each security technology to determine the “metrics”, using Splunk simplified the process greatly
! Remember that a picture tells a thousand words…
Case Study #3 – Metrics Across Security Technologies
18
! Sample Splunk search (IPS): index=[hidden] sourcetype=[hidden] rec_type_simple="IPS EVENT" | Cmechart span=1mon count
Case Study #4 – 24 x 7 Monitoring
19
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) app to provide 24x7 monitoring
! Instead of spending $$$ on an external Managed Security Services provider that provides aSer hours support, Splunk can be used to develop acConable dashboards monitored by the internal Network OperaCons Support Team (which works 24x7)
! PotenCal savings can go towards other criCcal security budget items
NOTE: This case study is currently being developed and tested within Flagstar. It has not yet reached a point where it is ready to replace an external MSS provider
Case Study #5
21
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) app to detect `Brute Force Login Ayempts’ and send automated alerts in real-‐Cme when detected
Case Study #6
22
! Flagstar’s IT Security OperaCons Team uses the Splunk Enterprise Security (ES) app to detect `Malware InfecCons’ and send automated alerts in real-‐Cme when detected
Case Study #5 & 6
23
! Sample Splunk search (Brute Force Ayempt Email Alert): EventCode=4625 sourcetype="WinEventLog:Security" earliest=-‐6m latest=now | bucket _Cme span=5m | stats count by _Cme, Account_Name, src_ip,dest | where count > 500
! Sample Splunk search (Malware Email Alert): index=[hidden] sourcetype=[hidden] NOT ("Actual acCon: Cleaned*" OR "Actual acCon: QuaranCned" OR "Actual acCon: Deleted") | rename "event_Cme" as "Detected" actual_acCon as "AcCon" dest_nt_host as "Host" dest_ip as "Host IP" user as "User" risk_type as "DetecCon Type" signature as "Malware Name" | table "Detected" "Host" "Host IP" "User" "DetecCon Type" "Malware Name" "AcCon" | sort by "Detected"
The Future of Splunk @ Flagstar
24
! We’re planning to bring addiConal data into Splunk over the next 12 months… – Database logs & custom applicaCon server logs – Wide range of banking applicaCons and regulatory data – Endpoint (client) systems – Third party hosted logs (various)
! Explore the value of the predicCve analysis capability ! Bring in Splunk Pro Services periodically to assist in maximizing Splunk's investment and to perform Splunk health checks
26
Security office hours: 11:00 AM – 2:00 PM @Room 103 Everyday Geek out, share ideas with Enterprise Security developers
Red Team / Blue Team -‐ Challenge your skills and learn new tricks Mon-‐Wed: 3:00 PM – 6:00 PM @Splunk Community Lounge Thurs: 11:00 AM – 2:00 PM
Learn, share and hack
Birds of a feather-‐ Collaborate and brainstorm with security ninjas Thurs: 12:00 PM – 1:00 PM @Meal Room