why cybersecurity needs big data

21
Why Cybersecurity Needs Big Data & Intro to Apache Metron James Sirota, Director Security Solutions March 2017 Michael Schiebel, Cybersecurity Strategy

Upload: hortonworks

Post on 21-Mar-2017

379 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Why Cybersecurity Needs Big Data

Why Cybersecurity Needs Big Data & Intro to Apache Metron

James Sirota, Director Security Solutions

March 2017

Michael Schiebel, Cybersecurity Strategy

Page 2: Why Cybersecurity Needs Big Data

2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Michael Schiebel, Cybersecurity Strategy

www.linkedin.com/in/michaelschiebel/

James SirotaDirector Security Solutions

www.linkedin.com/in/jsirota/

Anna YongCybersecurity Product

Marketing

www.linkedin.com/in/4everfusion/

Page 3: Why Cybersecurity Needs Big Data

3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Agenda

Why Cybersecurity Needs Big Data Intro to Apache Metron How big data experts can help IT security teams Case Study: Accelerating Investigation of a Phishing Attack

Page 4: Why Cybersecurity Needs Big Data

4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Why Cybersecurity Needs Big Data

Page 5: Why Cybersecurity Needs Big Data

5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Digital World Generates Big Data That Security Teams Need to Process

Page 6: Why Cybersecurity Needs Big Data

6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Existing Cyber Security Solutions Don’t Scale to the Challenge

82% of breaches happened in minutes

8 months: Average time an advanced security breach goes unnoticed

70%-80% of breaches are first detected by a 3rd party.

2016 Verizon Data Breach Investigations Report

Current security tools installed in the data center can’t handle volume of data & threats from everywhere

Page 7: Why Cybersecurity Needs Big Data

7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Cybersecurity JourneySingle View into Security Risk

Free data from security tools

Correlate and discover threats

Operational efficiency and governance

Predictive insights using machine learning

Single unified view of enterprise risk & security posture.

Innovate Renovate

Single Holistic View

HistoricalRecords

OPEXReduction

SecurityTool

Ingest

DigitalProtection

FraudPrevention

PublicData

Capture

A C T I V EA R C H I V E

DATAD I S C OV E RY

P R E D I C T I V EA N A LY T I C S

CyberSecurity

MachineData

RiskModeling

Page 8: Why Cybersecurity Needs Big Data

8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Apache Metron at Capital One

CapOne uses HDF to ingest log data into their cyber security data lake and uses Apache Metron to detect threats that cannot be detected by traditional cyber security tools

https://youtu.be/Nffx8SKn7l4?t=1h37m50s

Page 9: Why Cybersecurity Needs Big Data

9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Introduction to Apache Metron

Page 10: Why Cybersecurity Needs Big Data

10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Metron Journey

Jan 2016OpenSOC renamed Metron

Dec 2016Accepted

into Apache Incubation

Oct. 2015Hortonworks, Mantech, B23 press

release

Sept 2014First release of OpenSOC

Beta by CiscoApril 2014

OpenSOC in production

June 2014OpenSOC

Community Edition

July 2015Cisco stops supporting OpenSOC

March 2016

First Apache Release

Page 11: Why Cybersecurity Needs Big Data

11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Data Services and Integration Layer

Search andDashboarding

Portal

Security Data Vault

CommunityAnalytical

Models

Provisioning,Management

and Monitoring

ModulesReal-time ProcessingCyber Security Engine

TelemetryParsers Enrichment

ThreatIntel

AlertTriage

Indexersand

Writers

Cyber SecurityStream Processing Pipeline

Apache Metron: Incubating Project

Telemetry Ingest Buffer

TelemetryData Collectors

Real-timeEnrich / ThreatIntel Streams

PerformanceNetwork

IngestProbes

/ OtherMachine Generated Logs(AD, App / Web Server,

firewall, VPN, etc.)

Security Endpoint Devices (Fireye, Palo Alto,

BlueCoat, etc.)

Network Data(PCAP, Netflow, Bro, etc.)

IDS(Suricata, Snort, etc.)

Threat Intelligence Feeds(Soltra, OpenTaxi,third-party feeds)

TelemetryData Sources

Page 12: Why Cybersecurity Needs Big Data

12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

How Big Data Experts Can Help Security Teams

Page 13: Why Cybersecurity Needs Big Data

13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

A Day in the Life of An Analyst:

• Too many disparate tools • Too many alerts to process • Too much noise• How to connect the dots of

the relevant data points together?

Page 14: Why Cybersecurity Needs Big Data

14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Problem Posed (For Existing Tools)Security

InformationManagementSystem (SIEM)

• I am prohibitively expensive• I have vendor lock-in• I can’t deal with big data• I am not open• I am not extensible enough

LegacyPoint Tools

• I was built for 1995• I am super specialized• I don’t scale horizontally• I have a proprietary format• You need a PhD to operate me

BehavioralAnalytics

Tools (UEBA)

• I have a limited # of models• I am not trained on YOUR data• I am built by a small startup

Page 15: Why Cybersecurity Needs Big Data

15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Problem Posed (For Bad Guys)AdvancedPersistent

Threat

ScriptKiddie

• I am very unique in a way I do things• I live on your network for about 300 days• I know what I am after and I look for it, slowly• Your rules will not detect me, I am too smart• I impersonate a legitimate user, but I don’t act like one

Apache Metron can take everything that is known about me and check for it in real time

Apache Metron can model historical behavior of whoever I am impersonating and flag me as I try to deviate

• My techniques are predictable and known• My attack vectors are also known• I fumble around a lot• I set off a large number of alerts• You are not the only person I’ve attacked• I brag about what I did or will do

Repeatable Patterns Unique Patterns

Page 16: Why Cybersecurity Needs Big Data

16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Case Study: Accelerate Investigation of a Phishing Attack

Page 17: Why Cybersecurity Needs Big Data

17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

The “Threat Story” the Workflow Told….

Page 18: Why Cybersecurity Needs Big Data

18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

The Challenges faced by the SOC Analyst to Create this Story…

Challenge• The analyst had to jump from the SIEM to

more than  7 different tools that took up valuable time.

• It took more than 24 hours across 2 SOC shifts to investigate, determine scope, remediate and do further forensics/investigation.

• Half of my time was spending getting the  context needed for me to create the story

• The threat was detected too late. Instead of detecting the incident on 4/9, the threat should have been detected on 3/20 when the attacker spoofed Sonja’s email address

Need• Want a Centralized View of my data so I don’t

have to jump around and learn other tools Eliminate manual tasks to investigate a case

• Need to discover bad stuff quicker

• Need the System to create the context for me in real-time

• The current static rules in the SIEM didn’t detect the threat. Need smart analytics based on:

• User Sonja hasn’t used corp gmail in the last 3 months

• User Sonja can’t login from Ireland and Southern Cali at the same time

Page 19: Why Cybersecurity Needs Big Data

19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Old School vs. New School Security ControlsEmail

Security Rules

Firewall Rules IDS Rules Sandbox

Rules DLP RulesOld School ->(1-1)

New School ->(1-*) Email

Classifier Alerts TriageMalware

Family Classifier

Network Behavior Classifier

UEBA System

Page 20: Why Cybersecurity Needs Big Data

20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Apache Metron Resources

http://hortonworks.com/apache/metron/

https://metron.incubator.apache.org/

https://www.meetup.com/futureofdata-london/events/237165504/

Page 21: Why Cybersecurity Needs Big Data

21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Questions?